Thema
Temporal Saliency Propagation for Efficient Saliency Computation
Typ Master, Forschungspraxis, Bachelor, Ing.prax.
Betreuer Mikhail Startsev
Tel.: +49 (0)89 289-28550
E-Mail: mikhail.startsev@tum.de
Sachgebiet Human and Computer Vision
Beschreibung Teaching a computer, which parts of a scene (static or changing through time) are important (ex. relevant or simply interesting), has been one of the important problems in computer vision, since this prediction can be used as a "preprocessing" step before virtually any learning procedure. For some of the applications, especially in video processing, saliency models [1] have been criticized for having high computational cost. Applying a full prediction model for each frame can be simply too slow.

When considering prediction or detection of salient regions in a continuous video, it makes sense that the shift of the average attention distribution map from one frame to another is minimal and could (in a perfect case) be mostly explained by objects simply being moved around the scene. This movement can to some extent be described as optic flow or any other form of motion description. Video codecs, for instance, implement a similar technique to efficiently store the frames [2].

In this case therefore, we could only use the saliency predictor on selected frames, morphing the resulting attention map for a few subsequent frames. The candidate would explore the extent to which such temporal propagation is useful and at which point the performance-quality trade-off is still sensible. Various motion description approaches can also be evaluated for the purposes of the project.

Image source: nickyguides.digital-digest.com/keyframes.htm

[1] en.wikipedia.org/wiki/Salience_(neuroscience)
[2] en.wikipedia.org/wiki/Key_frame
Voraussetzung Solid programming skills and/or an algorithmic background are desired. Familiarity with video codecs is a plus.
Bewerbung If you are interested in this topic, we welcome the applications via the email address above. Please set the email subject to “<Type of application> application for topic 'XYZ'”, ex. “Master’s thesis application for topic 'XYZ'”, while clearly specifying why are you interested in the topic in the text of the message. Also make sure to attach your most recent CV (if you have one) and grade report.