ROC Analysis for my Spatio-temporal Saliency Model [1]:

ROC analysis in my CVPR paper is missing because I do not have human eye fixation data at that time.

In order to make my models (PQFT, PFT) comparable to other saliency models, I tested their ROC performance on the human eye fixation data I obtained recently.

All the experiments were run in Matlab 2008 on Linux platform.

The performance metrics are ROC curve [2] and inter-subject ROC curve [3]. Please refer to the relevant papers for the definitions of these two curves.

Data Set 1:

100 natural images with eye fixation data collected from 12 subjects (aged from 24 to 55).

The larger the ROC area is, the better the prediction power of a saliency model is. The perfect prediction of an ROC area is 1 and chance performance occurs at an area of 0.5.

The boundary line above is the upper bound of performance. The closer the curve is to the upper boundary line, the better the saliency model is consistent with human eye fixation data.

Data Set 2:

120 natural images with eye fixation data collected from 20 subjects used in [8][9].

Please note that the results of Discriminant Saliency and Informax model are from its original paper [8][9] for fairness.

Conclusion:

Our PQFT exhibits the best performance on ROC analysis because its ROC area is the largest and its inter-subject ROC curve is the closest to the upper boundary line.

Reference:

[1] C.L. Guo, Q. Ma and L.M. Zhang. "Spatio-temporal Saliency Detection Using Phase Spectrum of Quaternion Fourier Transform," CVPR 2008.
[2] B. W. Tatler, R. J. Baddeley, and I. D. Gilchrist. "Visual correlates of fixation selection: effects of scale and time," Vision Research, 45:643-659, 2005.
[3] J. Harel, C. Koch, and P. Perona, "Graph-based visual saliency," In Proc. NIPS, 2006.
[4] D. Walther and C. Koch, "Modeling attention to salient proto-objects," Neural Networks 19, 1395-1407, 2006.
[5] L. Itti, C. Koch, and E. Niebur. " A model of saliency-based visual attention for rapid scene analysis," IEEE Trans. on PAMI, 20(11), 1254-1259, 1998
[6] L. Itti and C. Koch, "A saliency-based search mechanism for overt and covert shifts of visual attention," Vision Research, 40:1489-1506, 2000.
[7] X. Hou and L. Zhang, "Saliency Detection: A Spectral Residual Approach," CVPR 2007.
[8] N. D. Bruce and J. K. Tsotsos, "Saliency based on information maximization," In Proc. NIPS, 2005.
[9] D. Gao, V. Mahadevan and N. Vasconcelos, "The discriminant centersurround hypothesis for bottom-up saliency," In Proc. NIPS, 2007.