Semantic Media Retrieval is about the search of knowledge from multimedia data sources. In our research group we focus the attention to images and videos.

People describe their life experience using words, that is a representation that could have many differences, when we consider different persons, due to the users’ knowledge and experience and because of the context.
This represent a “semantic gap” between the conceptualizations of the world expressed using language, and the experience of the world, whose most direct representations are photos and media in general.
Due to this, current media understanding systems are still very much example-driven (e.g., find photos similar to a given one on the basis of a set of features).

Whatever they are everyday personal experiences (e.g., a wedding or a birthday party) or large social happenings (e.g., a concert or a sport event), events mark our lives and memories. Recent studies demonstrate that users find it easier to navigate and search through such huge multimedia galleries if the data are grouped into events.
Moreover, events are more and more associated to massive quantities of media: photos, videos, audio recordings, tweets, web pages, etc. For this reason it is fundamental to properly organize such big collection of data, in order to allow easy access to them also in the future.

The massive quantities of media available in everyday life, make the processing of these data, via innovative multimedia content-and-context-based search engines, a challenging task. In this framework, the interaction with the user together with the exploitation of a-priori knowledge are combined in order to bridge the gap between the current multimedia understanding algorithms and the richness and subjectivity of semantics in human interpretations of multimedia data.




Dupplaw, DavidPaul ; Matthews, Michael ; Johansson, Richard ; Boato, Giulia ; Costanzo, Andrea ; Fontani, Marco ; Minack, Enrico ; Demidova, Elena ; Blanco, Roi ; Griffiths, Thomas ; Lewis, Paul ; Hare, Jonathon ; Moschitti, Alessandro

Information extraction from multimedia web documents: an open-source platform and testbed Journal Article

International Journal of Multimedia Information Retrieval, pp. 1-15, 2014, ISSN: 2192-6611.

Links | BibTeX


Dao, Minh-Son ; Boato, Giulia ; De Natale, Francesco GB ; Nguyen, Truc-Vien

Jointly exploiting visual and non-visual information for event-related social media retrieval Inproceedings

Proceedings of the 3rd ACM conference on International conference on multimedia retrieval, pp. 159–166, ACM 2013.


Muratov, Oleg ; Boato, Giulia ; De Natale, Francesco GB

Salient object detection using scene layout estimation Inproceedings

Multimedia Signal Processing (MMSP), 2013 IEEE 15th International Workshop on, pp. 390–395, IEEE 2013.


Rosani, A; Boato, G; De Natale, FGB

Weighted symbolic analysis of human behavior for event detection Inproceedings

IS&T/SPIE Electronic Imaging, International Society for Optics and Photonics 2013.

Links | BibTeX


Broilo, Mattia ; Boato, Giulia ; De Natale, Francesco GB

Content-based synchronization for multiple photos galleries Inproceedings

Image Processing (ICIP), 2012 19th IEEE International Conference on, pp. 1945–1948, IEEE 2012.


Dao, Minh-Son ; Boato, Giulia ; DeNatale, Francesco GB

Discovering inherent event taxonomies from social media collections Inproceedings

Proceedings of the 2nd ACM International Conference on Multimedia Retrieval, pp. 48, ACM 2012.


Dao, Minh-Son ; Dang-Nguyen, Duc-Tien ; De Natale, Francesco G B

Robust event discovery from photo collections using Signature Image Bases (SIBs) Journal Article

ACM Multimedia Tools and Applications, pp. 1–29, 2012.


Dang-Nguyen, Duc-Tien ; Boato, Giulia ; Moschitti, Alessandro ; De Natale, Francesco GB

Supervised models for multimodal image retrieval based on visual, semantic and geographic information Inproceedings

Content-Based Multimedia Indexing (CBMI), 2012 10th International Workshop on, pp. 1–5, IEEE 2012.


Dao, Minh-Son ; Nguyen, Truc-Vien ; Boato, Giulia ; De Natale, Francesco GB

The Watershed-based Social Events Detection Method with Support from External Data Sources. Inproceedings

MediaEval, 2012.



Dellagiacoma, Michela ; Zontone, Pamela ; Boato, Giulia ; Albertazzi, Liliana

Emotion based classification of natural images Inproceedings

Proceedings of the 2011 international workshop on DETecting and Exploiting Cultural diversiTy on the social web, pp. 17–22, ACM 2011.


Muratov, Oleg ; Zontone, Pamela ; Boato, Giulia ; De Natale, Francesco GB

A segment-based image saliency detection Inproceedings

Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on, pp. 1217–1220, IEEE 2011.


Mattivi, Riccardo ; Boato, Giulia ; De Natale, Francesco GB

Event-based media organization and indexing Journal Article

Infocommunications Journal, 3 (3), pp. 9–18, 2011.


Dao, Minh-Son ; Dang-Nguyen, Duc-Tien ; De Natale, Francesco G B

Signature-image-based event analysis for personal photo albums Inproceedings

ACM International Conference on Multimedia, pp. 1481–1484, 2011.



Zontone, Pamela ; Boato, Giulia ; Hare, Jonathon ; Lewis, Paul ; Siersdorfer, Stefan ; Minack, Enrico

Image and collateral text in support of auto-annotation and sentiment analysis Inproceedings

Proceedings of the 2010 Workshop on Graph-based Methods for Natural Language Processing, pp. 88–92, Association for Computational Linguistics 2010.


Zontone, Pamela ; Carli, Marco ; Boato, Giulia ; De Natale, Francesco GB

Impact of contrast modification on human feeling: an objective and subjective assessment Inproceedings

Image Processing (ICIP), 2010 17th IEEE International Conference on, pp. 1757–1760, IEEE 2010.



Zontone, Pamela ; Boato, Giulia ; De Natale, FGB ; De Rosa, Alessia ; Barni, Mauro ; Piva, Alessandro ; Hare, Jonathan ; Dupplaw, David ; Lewis, Paul

Image diversity analysis: Context, opinion and bias Journal Article