I obtained a young researcher’s establishment grant from the CSIR to investigate what makes images interesting and to find algorithms that flag images of potential interest to users. At present, I am exploring the use of pairwise image comparisons to estimate image interest. These interest estimates can be improved for video by imposing temporal smoothness constraints. Further improvements are obtained by incorporating image content information using convolutional neural network features within a Gaussian process smoother. A particularly exciting byproduct of this is the generation of a saliency map highlighting content of interest to users.
Unfortunately, generating this overlay is extremely expensive. However, some savings can be made by using a Gaussian process approximation to this to speed up the generation process.
Michael Burke. 2017. Leveraging Gaussian process approximations for rapid image overlay production. In Proceedings of SAWACMMM’17, Mountain View, CA, USA, October 23, 2017, 6 pages. https://doi.org/10.1145/3132711.3132715
M. Burke, “User-driven mobile robot storyboarding: Learning image interest and saliency from pairwise image comparisons“, eprint arXiv:1706.05850, 2017.
M. Burke, “Image ranking in video sequences using pairwise image comparisons and temporal smoothing“, 2016 Pattern Recognition Association of South Africa and Robotics and Mechatronics International Conference (PRASA-RobMech), Stellenbosch, 2016, pp. 1-6.