Learning Full Pairwise Affinities
for Spectral Segmentation
This paper studies the problem of learning a full range of pairwise affinities gained by integrating local grouping cues for spectral segmentation. The overall quality of the spectral segmentation depends mainly on the pairwise pixel affinities. By employing a semi-supervised learning technique, optimal affinities are learnt from the test image without iteration. We first construct a multi-layer graph with pixels and regions, generated by the mean shift algorithm, as nodes. By applying the semi-supervised learning strategy to this graph, we can estimate the intra- and inter-layer affinities between all pairs of nodes together. These pairwise affinities are then used to simultaneously cluster all pixel and region nodes into visually coherent groups across all layers in a single multi-layer framework of Normalized Cuts. Our algorithm provides high-quality segmentations with object details by directly incorporating the full range connections in the spectral framework. Since the full affinity matrix is defined by the inverse of a sparse matrix, its eigen-decomposition is efficiently computed. The experimental results on Berkeley and MSRC image databases demonstrate the relevance and accuracy of our algorithm as compared to existing popular methods.
- Figure 1. Pixel affinity images. (a) Image with one pixel selected. (b)-(d) Similarities between that pixel and all other pixels in the image, where red represents greater. They are obtained by the conventional color+boundary affinity model used in MNCut, our proposed measure, and the groundtruth affinity function from human annotations, respectively.
- Figure 2. Introducing a spectral segmentation. (a) Original image. (b)-(d) Segmentation results by NCut, MNCut, and our algorithm with the boundaries drawn in red, respectively.
PaperCVPR 2010 paper. (pdf, 1.78MB), [Supplement]. (pdf, 9.71MB)
Tae Hoon Kim, Kyoung Mu Lee, Sang Uk Lee. Learning Full Pairwise Affinities for Spectral Segmentation, Proc. Computer Vision and Pattern Recognition (CVPR), 2010
(ORAL presentation, 4.5% acceptance rate)
Codecode. (zip, 1.30MB)
This research is supported in part by:
- IT R\&D program of MKE/IITA (2008-F-030-01).
- ITRC program of MKE/NIPA through 3DRC (NIPA-2009-C1090-0902-0018).