Facilitating Image Search with a Scalable and Compact Semantic Mapping

Meng Wang, Weisheng Li, Dong Liu, Bingbing Ni, Jialie Shen, Shuicheng Yan

Research output: Contribution to journalArticleResearchpeer-review

Abstract

This paper introduces a novel approach to facilitating image search based on a compact semantic embedding. A novel method is developed to explicitly map concepts and image contents into a unified latent semantic space for the representation of semantic concept prototypes. Then, a linear embedding matrix is learned that maps images into the semantic space, such that each image is closer to its relevant concept prototype than other prototypes. In our approach, the semantic concepts equated with query keywords and the images mapped into the vicinity of the prototype are retrieved by our scheme. In addition, a computationally efficient method is introduced to incorporate new semantic concept prototypes into the semantic space by updating the embedding matrix. This novelty improves the scalability of the method and allows it to be applied to dynamic image repositories. Therefore, the proposed approach not only narrows semantic gap but also supports an efficient image search process. We have carried out extensive experiments on various cross-modality image search tasks over three widely-used benchmark image datasets. Results demonstrate the superior effectiveness, efficiency, and scalability of our proposed approach.

Original languageEnglish
Article number15292361
Pages (from-to)1561-1574
Number of pages14
JournalIEEE Transactions on Cybernetics
Volume45
Issue number8
Early online date17 Sep 2014
DOIs
Publication statusPublished - 1 Aug 2015
Externally publishedYes

Fingerprint

Semantics
Scalability
Experiments

Cite this

Wang, M., Li, W., Liu, D., Ni, B., Shen, J., & Yan, S. (2015). Facilitating Image Search with a Scalable and Compact Semantic Mapping. IEEE Transactions on Cybernetics, 45(8), 1561-1574. [15292361 ]. https://doi.org/10.1109/TCYB.2014.2356136
Wang, Meng ; Li, Weisheng ; Liu, Dong ; Ni, Bingbing ; Shen, Jialie ; Yan, Shuicheng. / Facilitating Image Search with a Scalable and Compact Semantic Mapping. In: IEEE Transactions on Cybernetics. 2015 ; Vol. 45, No. 8. pp. 1561-1574.
@article{cdf13dd7eca9422fafdce20080d54531,
title = "Facilitating Image Search with a Scalable and Compact Semantic Mapping",
abstract = "This paper introduces a novel approach to facilitating image search based on a compact semantic embedding. A novel method is developed to explicitly map concepts and image contents into a unified latent semantic space for the representation of semantic concept prototypes. Then, a linear embedding matrix is learned that maps images into the semantic space, such that each image is closer to its relevant concept prototype than other prototypes. In our approach, the semantic concepts equated with query keywords and the images mapped into the vicinity of the prototype are retrieved by our scheme. In addition, a computationally efficient method is introduced to incorporate new semantic concept prototypes into the semantic space by updating the embedding matrix. This novelty improves the scalability of the method and allows it to be applied to dynamic image repositories. Therefore, the proposed approach not only narrows semantic gap but also supports an efficient image search process. We have carried out extensive experiments on various cross-modality image search tasks over three widely-used benchmark image datasets. Results demonstrate the superior effectiveness, efficiency, and scalability of our proposed approach.",
keywords = "Compact semantic mapping (CSM), image search, semantic gap",
author = "Meng Wang and Weisheng Li and Dong Liu and Bingbing Ni and Jialie Shen and Shuicheng Yan",
year = "2015",
month = "8",
day = "1",
doi = "10.1109/TCYB.2014.2356136",
language = "English",
volume = "45",
pages = "1561--1574",
journal = "IEEE Transactions on Cybernetics",
issn = "1083-4419",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",
number = "8",

}

Wang, M, Li, W, Liu, D, Ni, B, Shen, J & Yan, S 2015, 'Facilitating Image Search with a Scalable and Compact Semantic Mapping', IEEE Transactions on Cybernetics, vol. 45, no. 8, 15292361 , pp. 1561-1574. https://doi.org/10.1109/TCYB.2014.2356136

Facilitating Image Search with a Scalable and Compact Semantic Mapping. / Wang, Meng; Li, Weisheng; Liu, Dong; Ni, Bingbing; Shen, Jialie; Yan, Shuicheng.

In: IEEE Transactions on Cybernetics, Vol. 45, No. 8, 15292361 , 01.08.2015, p. 1561-1574.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Facilitating Image Search with a Scalable and Compact Semantic Mapping

AU - Wang, Meng

AU - Li, Weisheng

AU - Liu, Dong

AU - Ni, Bingbing

AU - Shen, Jialie

AU - Yan, Shuicheng

PY - 2015/8/1

Y1 - 2015/8/1

N2 - This paper introduces a novel approach to facilitating image search based on a compact semantic embedding. A novel method is developed to explicitly map concepts and image contents into a unified latent semantic space for the representation of semantic concept prototypes. Then, a linear embedding matrix is learned that maps images into the semantic space, such that each image is closer to its relevant concept prototype than other prototypes. In our approach, the semantic concepts equated with query keywords and the images mapped into the vicinity of the prototype are retrieved by our scheme. In addition, a computationally efficient method is introduced to incorporate new semantic concept prototypes into the semantic space by updating the embedding matrix. This novelty improves the scalability of the method and allows it to be applied to dynamic image repositories. Therefore, the proposed approach not only narrows semantic gap but also supports an efficient image search process. We have carried out extensive experiments on various cross-modality image search tasks over three widely-used benchmark image datasets. Results demonstrate the superior effectiveness, efficiency, and scalability of our proposed approach.

AB - This paper introduces a novel approach to facilitating image search based on a compact semantic embedding. A novel method is developed to explicitly map concepts and image contents into a unified latent semantic space for the representation of semantic concept prototypes. Then, a linear embedding matrix is learned that maps images into the semantic space, such that each image is closer to its relevant concept prototype than other prototypes. In our approach, the semantic concepts equated with query keywords and the images mapped into the vicinity of the prototype are retrieved by our scheme. In addition, a computationally efficient method is introduced to incorporate new semantic concept prototypes into the semantic space by updating the embedding matrix. This novelty improves the scalability of the method and allows it to be applied to dynamic image repositories. Therefore, the proposed approach not only narrows semantic gap but also supports an efficient image search process. We have carried out extensive experiments on various cross-modality image search tasks over three widely-used benchmark image datasets. Results demonstrate the superior effectiveness, efficiency, and scalability of our proposed approach.

KW - Compact semantic mapping (CSM)

KW - image search

KW - semantic gap

UR - http://www.scopus.com/inward/record.url?scp=85028209417&partnerID=8YFLogxK

U2 - 10.1109/TCYB.2014.2356136

DO - 10.1109/TCYB.2014.2356136

M3 - Article

VL - 45

SP - 1561

EP - 1574

JO - IEEE Transactions on Cybernetics

JF - IEEE Transactions on Cybernetics

SN - 1083-4419

IS - 8

M1 - 15292361

ER -