This page provides the reproduction angular errors of several illuminant estimation algorithms for different datasets.
The estimated illuminants of the scenes in different datasets for different illuminant estimation algorithms are already calculated and their recovery angular errors (the conventional metric for evaluation of the illuminant estimation algorithms) are provided by Gijsenij here:
The datasets used for benchmarking of illuminant estimation algorithms can also be found here:
Reproduction Angular Error
The Reproduction Angular Error is calculated for different benchmak data sets and different summary statistics (mean, median and quantiles). On this page you can see tables of one statistical moments for most of algorithms for three datasets.
| Recovery angular error | Reproduction Angular error | |||||||
|---|---|---|---|---|---|---|---|---|
| p | σ | Median | rank | p | σ | Median | rank | |
| Grey World | - | - | 7° | 11 | - | - | 7.49° | 11 |
| Max RGB | - | - | 6.48° | 10 | - | - | 7.44° | 10 |
| Shades of Gray | 7 | - | 3.74° | 9 | 7 | - | 3.94° | 8 |
| 1st Order Gray Edge | 7 | 4 | 3.18° | 7 | 14 | 4 | 3.59° | 6 |
| 2nd Order Gray Edge | 14 | 10 | 2.72° | 4 | 15 | 10 | 3.04° | 4 |
| Pixel-based Gamut | - | 4 | 2.27° | 2 | - | 4 | 2.83° | 3 |
| Edge-based Gamut | - | 2 | 2.28 | 3 | - | 2 | 2.70 | 2 |
| Union-based Gamut | - | 2 | 2.95° | 5 | - | 2 | 3.38° | 5 |
| Intersection-based Gamut | - | 4 | 2.10° | 1 | - | 3 | 2.48° | 1 |
| Spatial Correlations: Heavy-tailed based | - | - | 3.45° | 8 | - | - | 4.11° | 9 |
| Spatial Correlations: Weighted Gray-Edge | - | - | 3.09° | 6 | - | - | 3.62° | 7 |
| Recovery angular error | Reproduction Angular error | |||||||
|---|---|---|---|---|---|---|---|---|
| p | σ | Median | rank | p | σ | Median | rank | |
| Grey World | - | - | 6.97° | 11 | - | - | 7.62° | 11 |
| Max RGB | - | - | 5.30° | 6 | - | - | 5.52° | 5 |
| Shades of Gray | 8 | - | 5.28° | 5 | 14 | - | 5.61° | 6 |
| 1st Order Gray Edge | 2 | 1 | 4.64° | 3 | 2 | 1 | 5.61° | 6 |
| 2nd Order Gray Edge | 1 | 2 | 4.85° | 4 | 1 | 2 | 5.1° | 4 |
| Pixel-based Gamut | - | 2 | 5.67° | 9 | - | 2 | 5.9° | 8 |
| Edge-based Gamut | - | 1 | 5.62° | 8 | - | 1 | 5.9° | 9 |
| Intersection-based Gamut | - | 6 | 5.7° | 10 | - | 2 | 5.9° | 9 |
| Inverse Intensity Chroma Space | - | - | 5.6° | 7 | - | - | 6.1° | 10 |
| Using Natural Image Statistics | - | - | 3.92° | 2 | - | - | 4.3° | 2 |
| Exemplar-based colour constancy | - | - | 3.4° | 1 | - | - | 3.7° | 1 |
Colour-checker. dataset (by Shi)
| Recovery angular error | Reproduction Angular error | |||||||
|---|---|---|---|---|---|---|---|---|
| p | σ | 95% Quantile | rank | p | σ | 95% Quantile | rank | |
| Gray World | - | - | 11.25° | 7 | - | - | 12.41° | 6 |
| Max RGB | - | - | 19.01° | 18 | - | - | 20.05° | 18 |
| Shades of Gray | 2 | - | 10.56° | 5 | 2 | - | 11.97° | 5 |
| 1st Order Gray Edge | 1 | 1 | 11.33° | 8 | 1 | 1 | 14.56° | 11 |
| 2nd Order Gray Edge | 1 | 1 | 11.01° | 6 | 1 | 1 | 13.66° | 9 |
| Pixel-based Gamut | - | 5 | 13.60° | 14 | - | 5 | 15.44° | 14 |
| Edge-based Gamut | - | 5 | 16.1° | 16 | - | 5 | 19.93° | 17 |
| Intersection-based Gamut | - | 5 | 13.6° | 15 | - | 5 | 15.47° | 15 |
| Regression (SVR) | - | - | 17.25° | 15 | - | - | 18.89° | 16 |
| Bayesian | - | - | 12.6° | 13 | - | - | 15.39° | 13 |
| Spatial Correlations:Heavy tailed-based | - | - | 8.68° | 2 | - | - | 9.89° | 2 |
| Bottom-up | - | - | 9.53° | 3 | - | - | 11.57° | 4 |
| Top-down | - | - | 12.13° | 11 | - | - | 13.81° | 10 |
| Bottom-up+Top-down | - | - | 11.55° | 9 | - | - | 13.59° | 8 |
| Using Natural Image Statistics | - | - | 11.69° | 10 | - | - | 12.95° | 7 |
| CART-based selection | - | - | 12.49° | 12 | - | - | 14.63° | 12 |
| CART-based combination | - | - | 10.14° | 4 | - | - | 11.43° | 3 |
| Exemplar-based colour constancy | - | - | 6.95° | 1 | - | - | 8.23° | 1 |
Referencing
If you would like to use this data or the Reproduction Angular Error for your own research please provide the following reference:
Graham D. Finlayson and Roshanak Zakizadeh. "Reproduction Angular Error: An Improved Performance Metric for Illuminant Estimation." British Machine Vision Confernce BMVC. 2014.
Contact: rzakizadeh@gmail.com