A multi-stripe triangulation-based laser scanner, NextEngine 3D Scanner, is used to acquire the reference point clouds of two buildings of interest. In order to ensure a consistent comparison between the submitted and reference point clouds, a decimation process is performed: the mean spatial resolutions of the two point sets are computed and the largest one is adopted as sub-sampling threshold for the denser cloud.
The evaluation procedure consists of two parts:
1. ACCURACY ASSESSMENT
Accuracy is estimated by measuring the signed distances from the submitted DIM result to the reference one. The evaluation is performed on well-distributed patches, selected such as to (i) exclude areas of missing or noisy data in the reference clouds, and (ii) include areas that differ in texture and geometric complexity. The following statistics are reported: mean, median, sigma and sigmaMAD.
2. COMPLETENESS ASSESSMENT
Completeness is estimated by measuring the absolute distances from the reference point cloud to the submitted DIM one. The whole population of reference points is included in the evaluation, and completeness is defined as the fraction of reference points that are within an allowable distance from their neighbors in the DIM point cloud. The maximum acceptable distance is set to 3 times the mean spatial resolution adopted during the initial decimation process.
In the following table a short version of the evaluation results is reported. Mean, median, sigma and sigmaMAD (accuracy assessment) are in mm; completeness is reported in percentage.
The full overview is available here.