Detector adaptation by maximising agreement between independent data sources

Ciarán Ó Conaire, Noel E. O'Connor and Alan Smeaton

The traditional methods for creating classifiers have two main disadvantages. Firstly, it is time consuming to acquire, or manually annotate, the training collection. Secondly, the data on which the classifier is trained may be over-generalised or too specific. This paper presents our investigations into overcoming both of these drawbacks simultaneously, by providing example applications where two data sources train each other. This both removes the need for supervised annotation or feedback, and allows rapid adaptation of the classifier to different data. Two applications are presented: one using thermal infrared and visual imagery to robustly learn changing skin models, and another using changes in saturation and luminance to learn shadow appearance parameters.

PDF Version


Matlab Source Code

Skin Detection Code
A non-parametric skin detector was developed based on this work.
Click to go to Source code page

If interested in obtaining code to perform adaptive bound selection, please contact: oconaire (at) gmail (dot) com.


Video Example

The video below contains the processed results from a number of sequences. Each frame is processed independently, using no temporal information. The bounds for HSV and for IR are adapted to each image uniquely in order to maximise the agreement between the colour and IR binary masks (bottom row of the video).
Click on image for a video of Skin Detection Results



Bound adaptation

As the graph (below) shows, the (upper and lower) bounds for the infrared brightness of skin adapt dynamically as the camera's contrast changes. The introduction of a hot kettle causes the camera to adjust its contrast to balance the saturated white pixels of the kettle (note the change in the infrared images from frame 1470 to 1550). Our algorithm adjusts its bounds to maximise agreement between the colour and thermal skin subspaces.

Frame 1470 Frame 1500 Frame 1520 Frame 1550


In the next example of adaptation, we show how the Hue and Value bounds adapt to a changing scene. In this short sequence, it begins with two people of similar skin tone. When a third person enters, the difference in their skin hue and brightness causes the bound parameters of the colour skin model to adapt, with the Hue and Value lower bound reducing to compensate. This adaptation is shown in the Hue and Value graphs below.

Frame 1220 Frame 1260 Frame 1280 Frame 1320



Shadow Detection

These testing images were acquired from the TERRASCOPE dataset.
Background Current Frame Saturation Brightness AND fusion Combined



Results on OTCBVS dataset



Probabilistic skin model creation

Using the image-based skin detection approach demonstrated above, these skin and background samples can be used to create a Bayesian classifier. A classifier was adaptively trained on previous frames and used to create a skin log-likelihood image for the current frame. Our results show a sharp improvement compared to a pre-trained skin/background-model that was trained using a large variety of hand-annotated skin and background pixels.
ROC curves
Our dynamically created models (red plot)
Pretrained models (blue plot)
32x32x32 histogram 64x64x64 histogram



Skin Log-likelihood images

Pre-trained Model Our Model