Machine Vision and Applications Sequences

The tracking sequences on this webpage were part of the experiments for:

Thermo-Visual Feature Fusion for Object Tracking Using Multiple Spatiogram Trackers
Ciarán Ó Conaire, Noel E. O'Connor, Alan Smeaton
submitted to Machine Vision and Applications
Abstract:In this paper, we propose a framework that can efficiently combine features for robust tracking based on fusing the outputs of multiple spatiogram trackers. This is achieved without the exponential increase in storage and processing that other multimodal tracking approaches suffer from. The framework allows the features to be split arbitrarily between the trackers, as well as providing the flexibility to add, remove or dynamically weight features. We derive a mean-shift type algorithm that allows efficient object tracking with very low computational overhead. We especially target the fusion of thermal infrared and visible spectrum features as the most useful features for automated surveillance applications. Results are shown on multimodal video sequences clearly illustrating the benefits of combining multiple features using our framework.



Exhaustive search tracking

Explanation of Colours in tracking videos:
Single Modality Tracking shows the tracking results of two trackers
(i)Luminance-based spatiogram tracker (Green ellipse with blue inner ring) and
(ii)Thermal-infrared-based spatiogram tracker (Red ellipse with a cyan inner ring)
Combined Tracking shows the tracking results of our combined multi-modal tracker which used Y,U,V,thermal brightness and edge orientation (Cyan Ellipse with red inner ring)

Tracking Type
Person Tracking

Bike Tracking
Single Modality Tracking [Viewed in Visible Spectrum]
[Viewed in Infrared]
[Viewed in Visible Spectrum]
[Viewed in Infrared]
Combined Tracking
(Multimodal spatiograms) 		[View in Visible Spectrum]
[View in Infrared] 		[View in Visible Spectrum]
[View in Infrared]

Both sequences are aligned thermal infrared and visible spectrum video.
The person tracking sequence comes from the OTCBVS benchmark dataset.


Tracker Comparison

In the following videos are shown some comparisons between different trackers. The columns on the right hand side of the table below indicate whether each tracker successfully tracked the target or not. Clicking on the images in the left column displays the tracking video. All trackers use 8 bins per component. The non-PETS sequences are multimodal, so include a thermal-infrared channel. This means that the histogram for those sequences, for example, is a 8x8x8x8 bin array.


Tracker colour-codes
[CYAN] = Template tracker
[RED] = Histogram tracker
[BLUE] = Bank of Histograms tracker
[YELLOW] = Spatiogram tracker
[GREEN] = Bank of Spatiograms (proposed tracker)
Tracking Video Description Track Track Track Track Track
Tracking a person in a white coat.
 OK---OK
Tracking a car from a moving vehicle.
All trackers manage to track the car but the template and histogram-based trackers shrink in size and lock onto parts of the car at small scales.
Source: PETS'2001 dataset 		OKOKOKOKOK
Tracking a girl in dark clothing.
Only the bank of spatiograms tracker provides good tracking. The bank of histograms stays locked on the target, but has the incorrect scale, locking onto the upper part of the person. 		--OK-OK
Tracking a football player.
Source: VS-PETS'2003 dataset 		--OKOKOK
Tracking a person with similar colours to the background.
All succeed in tracking, except the template tracking, though the bank of histograms has poor object scale. 		-OKOKOKOK
Tracking a man wearing a brown coat.
Source: OTCBVS dataset 		OK-OK-OK



Meanshift Tracking


Book Tracking sequence
Car Tracking sequence


The above book-tracking sequence has infrared and visible spectrum data associated with it.
The car tracking sequence (from the PETS2001 dataset) contains visible spectrum only.