Mapillary Vistas Object Detection Task

Organized by Mapillary - Current server time: Dec. 11, 2018, 5:30 p.m. UTC

Previous

validation
June 30, 2018, midnight UTC

Current

test
June 30, 2018, midnight UTC

End

Competition Ends
Never

Mapillary Vistas Object Detection Task

The goal of this task is to provide instance-specific segmentation results for a subset of 37 object classes on MVD. In such a way, results allow to count individual instances of classes like e.g. the number of cars or pedestrians in an image.

The detection task is part of the Joint COCO and Mapillary Recognition Challenge Workshop at ECCV 2018. Researchers are encouraged to participate in both, the COCO and Mapillary Detection Tasks (the tasks share [almost] identical data formats and the exact same evaluation metrics). The only difference between COCO and Mapillary submission formats is the value type for image_id per detection result, which has to be a string providing the file name without extension for Mapillary Vistas rather than an integer as in COCO. Please make a trial submission using validation data to ensure that your submission format is matching.

[{
"image_id": str,
"category_id": int,
"segmentation": RLE,
"score": float,
}]

The detection task uses all the publicly available Mapillary Vistas Research edition v1.1 images (18.000 train + 2.000 val) with 37 thing categories + 1 void class. There are no overlaps of annotations between different objects of same/different classes. The main performance metric used is Average Precision (AP) computed on the basis of instance-level segmentations per object category and averaged over a range of overlaps 0.5:0.05:0.95 with inclusive start and end. This CodaLab evaluation server provides a platform to measure performance on the val and test sets.

The main performance metric used is Average Precision (AP) computed on the basis of instance-level segmentations per object category and averaged over a range of overlaps 0.5:0.05:0.95 with inclusive start and end, see here for details.

validation

Start: June 30, 2018, midnight

Description: Development phase with validation data

test

Start: June 30, 2018, midnight

Description: Challenge phase with test data

Competition Ends

Never

You must be logged in to participate in competitions.

Sign In