You are here

3DOMcity Task 1: Results

The evaluation procedure consists of three parts:

1. PRECISION ASSESSMENT

Precision in image space is evaluated by computing the re-projection errors of targets centroids and corresponding statistics: min, mean, median, max, RMS(x), RMS(y), RMS. 

2. ACCURACY ASSESSMENT

Accuracy in object space is evaluated by computing the errors on 3 check scale bars.

3. RELATIVE ACCURACY ASSESSMENT

Since no independent nominal values are available for image external orientation parameters, experimental values are estimated by performing a self-calibrating BBA of the image network. A rigid similarity transformation (6DOF) is estimated to map the submitted network to the reference one (3DOM). The relative accuracy of the submitted EOP parameters is evaluated by computing their deviations from reference EOP and corresponsing statistics: min, mean, median, max, RMSE. 

 

In the following tables a short version of the evaluation results is reported.
The full overview is available here.

 

1. PRECISION ASSESSMENT (all values are in pixels)

Sub. Mean Median RMS(x) RMS(y) RMS
3DOM -0.0007 0.0010 0.1403 0.1474 0.2035
Wang-1 0.0001 -0.0003 0.1320 0.1378 0.1908
Wang-2 0.0001 -0.0003 0.1320 0.1378 0.1908
Wang-3 0.0001 -0.0003 0.1320 0.1378 0.1908

 

2. ACCURACY ASSESSMENT (CH: check scale bar; all values are in mm)

Sub. Error-CH1 Error-CH2 Error-CH3
3DOM -0.0340 -0.1046 0.0333
Wang-1 0.0915 0.1462 0.0533
Wang-2 0.0841 0.1482 0.0606
Wang-3 0.0761 0.1503 0.0685

 

3. RELATIVE ACCURACY ASSESSMENT (errors on X,Y,Z are in mm; errors on O,P,K are in degrees)

Sub. # Oriented Images RMSE(X) RMSE(Y) RMSE(Z) RMSE(O) RMSE(P) RMSE(K)
Wang-1 420 0.5645 2.4833 0.8064 0.1495 0.1444 0.2455
Wang-2 420 0.5665 2.4824 0.8334 0.1496 0.1435 0.2461
Wang-3 420 0.5687 2.5413 0.8237 0.1520 0.1440 0.2457