Semiotic tagging: Enriching the semantics of tags for improved image retrieval

Abstract

SemioTag is an approach towards tagging that utilizes the semiotic sign categories icon, index, and symbol as classification structures to be used by users during the annotation and search of images within social media-oriented repositories. We compared the influence of this approach on the tagging and querying behaviour of users, with respect to usability, efficiency, and user experience, between the standard Flickr tagging and querying method and the one used in SemioTag. Our results show that semiotic tagging is considered more tedious and takes about twice the time as standard tagging. However, subjects produced a larger number of tags with semiotic tagging. Finally, querying with semiotic tags is not considered more cumbersome than querying using standard tags. Subjects stated that semiotic-based search provides more reasonable results than search based on standard tagging because it provided more control on the query. Semiotic search turned out to be faster. Overall, the findings clearly indicate to further investigate in the direction of semiotic tagging. We anticipate application of semiotics for particular types of human-centered IR such as explorative search.