The aggregation feature for bounding boxes enables generating an average high-confidence result from multiple judgments.
The aggregated box is available in the Aggregate Report on the Results page.
Aggregation consists of two steps.
- This step involves identifying the set of boxes from different judgments that potentially represent the same object.
- This is accomplished by clustering boxes based on the degree of overlap computed by the ratio of intersection over union (IoU).
- Each cluster has a maximum of one box from each contributor that worked on the image.
- The degree of overlap is configurable for each job.
- This step computes the union of intersections from the component boxes in each cluster.
- Any sections that do not overlap with at least one other box are discarded.
- An aggregate bounding box is generated from the sections that are part of at least two component boxes.
- Boxes that don’t have sufficient overlap with any cluster are ignored during aggregation.
Note: Aggregation cannot be applied to jobs with one judgement per unit.
The default setting for aggregation is “all”. Bounding boxes aggregation can be enabled by in the following ways.
- Results Options:
Select Box Aggregation (‘bagg_x’) in the Aggregation section for the annotation response. Degree of overlap can be specified as a number between 0 to 1, 0 represents no overlap, and 1 represents 100% overlap between boxes.
2. Job Design: This value can also be set within CML using the aggregation attribute on a CML element.
Applicable to cml:boundingbox
Returns aggregated bounding boxes from constituent contributor responses with degree of overlap greater than the specified value for x. x can be any floating point number between 0 & 1. Always prefix '0' for decimals; for example, bagg_0.4 will return all responses with overlap greater than 40%.
A confidence score is assigned to each box based on the measure of agreement.
- Agreement is computed as the ratio of intersection over union of the component boxes (Note: Unlike other aggregation methods, confidence for bounding boxes aggregation is not weighted by trust).
- In addition to each box, each row has a confidence score that is the average of confidence of all aggregate boxes in the row
- The confidence score ranges between 0 and 1.
- A value closer to 1 represents higher confidence.