Looking Beyond the Image: Unsupervised Learning for Object Saliency and Detection

Parthipan Siva Chris Russell Tao Xiang Lourdes Agapito

We propose a principled probabilistic formulation of object saliency as a sampling problem. This novel formulation allows us to learn, from a large corpus of unlabelled images, which patches of an image are of the greatest interest and most likely to correspond to an object. We then sample the object saliency map to propose object locations. We show that using only a single object location proposal per image, we are able to correctly select an object in over 42% of the images in the Pascal VOC 2007 dataset, substantially outperforming existing approaches. Furthermore, we show that our object proposal can be used as a simple unsupervised approach to the weakly supervised annotation problem. Our simple unsupervised approach to annotating objects of interest in images achieves a higher annotation accuracy than most weakly supervised approaches.


Please cite following paper when using this data and/or code.

P. Siva, C. Russell, T. Xiang, and L. Agapito. Looking Beyond the Image: Unsupervised Learning for Object Saliency and Detection. CVPR 2013.

Saliency Map Examples

Pascal VOC 2007 TrainVal

Image Our CA SR FT

MSRA 1000


Comparison of Across and Within Image Saliency

Our Saliency (Across Image Saliency + Within Image Saliency)
Across Image Saliency
Within Image Saliency

Generic Object Proposals


Best bounding boxes taken from the top 10 proposed object locations by our coherent sampling method (Our), MSR, Alexe et al. NMS, and Rahtu et al. Blue is ground truth.

Weakly Supervised Annotation