This study aims to integrate knowledge and filtering techniques in order to improve Multimodal Information Retrieval systems. Traditional Information Retrieval (IR) Systems are primarily concerned with dealing with textual information. However, the amount of electronic information available today is not only textual, but rather multimodal.
By multimodality we mean any format including textual information, images, video or audio, and in most cases we usually find mixed information. There are specialized systems dealing with the extraction of textual information in different formats. Examples include the Content Based Image Recovery (CBIR) systems, systems which extract video features and systems that transcribe conversations to text. In most of these the information obtained is finally expressed in text, so that in the end traditional text processing techniques are often used . A multimodal system is a system that retrieves information from large collections in various formats. This can exploit the advantages of various specialized systems. This multimodality allows, for example, CBIR systems to improve using textual information that appears next to images. These systems are useful for different types of professionals who need to work with other formats than text. Within this area we can consider medical work, which generates large volumes of information on each clinical case, including text and images from the various tests.
This paper proposes the use of various techniques together to address the problem of Multimodal Information Retrieval. Systems of traditional text-based Information Retrieval are amply tested and analyzed, and techniques used in these systems have proven their effectiveness. However, in systems where the goal of the search is not a text or where the documental corpus is formed only by text, the technologies currently employed do not obtain the same performance as textual techniques.
That is why this study is focused on enhancing and improving text retrieval as part of a multimodal recovery system, applying methodologies and proven tools together. Among the techniques used and studied are the use of external knowledge to improve user’s queries, the filtering of textual collections to remove relevant data and the fusion of results obtained by the different retrieval systems within a multimodal system.
(Link TESEO)