Machine Vision and Applications SequencesThe tracking sequences on this webpage were part of the experiments for:
Thermo-Visual Feature Fusion for Object Tracking Using Multiple Spatiogram Trackers
Ciarán Ó Conaire, Noel E. O'Connor, Alan Smeaton
submitted to Machine Vision and Applications
Abstract:In this paper, we propose a framework that can efficiently combine features for robust tracking based on fusing the outputs of multiple spatiogram trackers. This is achieved without the exponential increase in storage and processing that other multimodal tracking approaches suffer from. The framework allows the features to be split arbitrarily between the trackers, as well as providing the flexibility to add, remove or dynamically weight features. We derive a mean-shift type algorithm that allows efficient object tracking with very low computational overhead. We especially target the fusion of thermal infrared and visible spectrum features as the most useful features for automated surveillance applications. Results are shown on multimodal video sequences clearly illustrating the benefits of combining multiple features using our framework.
Exhaustive search trackingExplanation of Colours in tracking videos:
Single Modality Tracking shows the tracking results of two trackers
(i)Luminance-based spatiogram tracker (Green ellipse with blue inner ring) and
(ii)Thermal-infrared-based spatiogram tracker (Red ellipse with a cyan inner ring)
Combined Tracking shows the tracking results of our combined multi-modal tracker which used Y,U,V,thermal brightness and edge orientation (Cyan Ellipse with red inner ring)
Both sequences are aligned thermal infrared and visible spectrum video.
The person tracking sequence comes from the OTCBVS benchmark dataset.
In the following videos are shown some comparisons between different trackers. The columns on the right hand side of the table below indicate whether each tracker successfully tracked the target or not. Clicking on the images in the left column displays the tracking video. All trackers use 8 bins per component. The non-PETS sequences are multimodal, so include a thermal-infrared channel. This means that the histogram for those sequences, for example, is a 8x8x8x8 bin array.
[CYAN] = Template tracker
[RED] = Histogram tracker
[BLUE] = Bank of Histograms tracker
[YELLOW] = Spatiogram tracker
[GREEN] = Bank of Spatiograms (proposed tracker)
The above book-tracking sequence has infrared and visible spectrum data associated with it.
The car tracking sequence (from the PETS2001 dataset) contains visible spectrum only.