Representative discovery of structure cues for weakly-supervised image segmentation

Luming Zhang, Yue Gao, Yingjie Xia, Ke Lu, Jialie Shen, Rongrong Ji

Research output: Contribution to journalArticleResearchpeer-review

Abstract

Weakly-supervised image segmentation is a challenging problem with multidisciplinary applications in multimedia content analysis and beyond. It aims to segment an image by leveraging its image-level semantics (i.e., tags). This paper presents a weakly-supervised image segmentation algorithm that learns the distribution of spatially structural superpixel sets from image-level labels. More specifically, we first extract graphlets from a given image, which are small-sized graphs consisting of superpixels and encapsulating their spatial structure. Then, an efficient manifold embedding algorithm is proposed to transfer labels from training images into graphlets. It is further observed that there are numerous redundant graphlets that are not discriminative to semantic categories, which are abandoned by a graphlet selection scheme as they make no contribution to the subsequent segmentation. Thereafter, we use a Gaussian mixture model (GMM) to learn the distribution of the selected post-embedding graphlets (i.e., vectors output from the graphlet embedding). Finally, we propose an image segmentation algorithm, termed representative graphlet cut, which leverages the learned GMM prior to measure the structure homogeneity of a test image. Experimental results show that the proposed approach outperforms state-of-the-art weakly-supervised image segmentation methods, on five popular segmentation data sets. Besides, our approach performs competitively to the fully-supervised segmentation models.

Original languageEnglish
Article number6677517
Pages (from-to)470-479
Number of pages10
JournalIEEE Transactions on Multimedia
Volume16
Issue number2
DOIs
Publication statusPublished - Feb 2014
Externally publishedYes

Fingerprint

Image segmentation
Labels
Semantics

Cite this

Zhang, Luming ; Gao, Yue ; Xia, Yingjie ; Lu, Ke ; Shen, Jialie ; Ji, Rongrong. / Representative discovery of structure cues for weakly-supervised image segmentation. In: IEEE Transactions on Multimedia. 2014 ; Vol. 16, No. 2. pp. 470-479.
@article{b571caabd5ce425f83e62c18ae1364ca,
title = "Representative discovery of structure cues for weakly-supervised image segmentation",
abstract = "Weakly-supervised image segmentation is a challenging problem with multidisciplinary applications in multimedia content analysis and beyond. It aims to segment an image by leveraging its image-level semantics (i.e., tags). This paper presents a weakly-supervised image segmentation algorithm that learns the distribution of spatially structural superpixel sets from image-level labels. More specifically, we first extract graphlets from a given image, which are small-sized graphs consisting of superpixels and encapsulating their spatial structure. Then, an efficient manifold embedding algorithm is proposed to transfer labels from training images into graphlets. It is further observed that there are numerous redundant graphlets that are not discriminative to semantic categories, which are abandoned by a graphlet selection scheme as they make no contribution to the subsequent segmentation. Thereafter, we use a Gaussian mixture model (GMM) to learn the distribution of the selected post-embedding graphlets (i.e., vectors output from the graphlet embedding). Finally, we propose an image segmentation algorithm, termed representative graphlet cut, which leverages the learned GMM prior to measure the structure homogeneity of a test image. Experimental results show that the proposed approach outperforms state-of-the-art weakly-supervised image segmentation methods, on five popular segmentation data sets. Besides, our approach performs competitively to the fully-supervised segmentation models.",
keywords = "active learning, graphlet, segmentation, Structure cues, weakly supervised",
author = "Luming Zhang and Yue Gao and Yingjie Xia and Ke Lu and Jialie Shen and Rongrong Ji",
year = "2014",
month = "2",
doi = "10.1109/TMM.2013.2293424",
language = "English",
volume = "16",
pages = "470--479",
journal = "IEEE Transactions on Multimedia",
issn = "1520-9210",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",
number = "2",

}

Representative discovery of structure cues for weakly-supervised image segmentation. / Zhang, Luming; Gao, Yue; Xia, Yingjie; Lu, Ke; Shen, Jialie; Ji, Rongrong.

In: IEEE Transactions on Multimedia, Vol. 16, No. 2, 6677517, 02.2014, p. 470-479.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Representative discovery of structure cues for weakly-supervised image segmentation

AU - Zhang, Luming

AU - Gao, Yue

AU - Xia, Yingjie

AU - Lu, Ke

AU - Shen, Jialie

AU - Ji, Rongrong

PY - 2014/2

Y1 - 2014/2

N2 - Weakly-supervised image segmentation is a challenging problem with multidisciplinary applications in multimedia content analysis and beyond. It aims to segment an image by leveraging its image-level semantics (i.e., tags). This paper presents a weakly-supervised image segmentation algorithm that learns the distribution of spatially structural superpixel sets from image-level labels. More specifically, we first extract graphlets from a given image, which are small-sized graphs consisting of superpixels and encapsulating their spatial structure. Then, an efficient manifold embedding algorithm is proposed to transfer labels from training images into graphlets. It is further observed that there are numerous redundant graphlets that are not discriminative to semantic categories, which are abandoned by a graphlet selection scheme as they make no contribution to the subsequent segmentation. Thereafter, we use a Gaussian mixture model (GMM) to learn the distribution of the selected post-embedding graphlets (i.e., vectors output from the graphlet embedding). Finally, we propose an image segmentation algorithm, termed representative graphlet cut, which leverages the learned GMM prior to measure the structure homogeneity of a test image. Experimental results show that the proposed approach outperforms state-of-the-art weakly-supervised image segmentation methods, on five popular segmentation data sets. Besides, our approach performs competitively to the fully-supervised segmentation models.

AB - Weakly-supervised image segmentation is a challenging problem with multidisciplinary applications in multimedia content analysis and beyond. It aims to segment an image by leveraging its image-level semantics (i.e., tags). This paper presents a weakly-supervised image segmentation algorithm that learns the distribution of spatially structural superpixel sets from image-level labels. More specifically, we first extract graphlets from a given image, which are small-sized graphs consisting of superpixels and encapsulating their spatial structure. Then, an efficient manifold embedding algorithm is proposed to transfer labels from training images into graphlets. It is further observed that there are numerous redundant graphlets that are not discriminative to semantic categories, which are abandoned by a graphlet selection scheme as they make no contribution to the subsequent segmentation. Thereafter, we use a Gaussian mixture model (GMM) to learn the distribution of the selected post-embedding graphlets (i.e., vectors output from the graphlet embedding). Finally, we propose an image segmentation algorithm, termed representative graphlet cut, which leverages the learned GMM prior to measure the structure homogeneity of a test image. Experimental results show that the proposed approach outperforms state-of-the-art weakly-supervised image segmentation methods, on five popular segmentation data sets. Besides, our approach performs competitively to the fully-supervised segmentation models.

KW - active learning

KW - graphlet

KW - segmentation

KW - Structure cues

KW - weakly supervised

UR - http://www.scopus.com/inward/record.url?scp=84893381680&partnerID=8YFLogxK

U2 - 10.1109/TMM.2013.2293424

DO - 10.1109/TMM.2013.2293424

M3 - Article

VL - 16

SP - 470

EP - 479

JO - IEEE Transactions on Multimedia

JF - IEEE Transactions on Multimedia

SN - 1520-9210

IS - 2

M1 - 6677517

ER -