COCO vs YOLO format vs Pascal VOC, which annotation format to use

COCO vs YOLO format at a glance

Here is the short version before the details.

If you train a YOLO family model, export YOLO. If you use a general detection or segmentation library, COCO is the safest default. Use Pascal VOC only when an existing pipeline already expects it.

How the COCO JSON format works

COCO stores a whole split in one JSON file with three main lists: images, annotations and categories. Each annotation points to an image by id and to a category by id. Boxes use the top left corner plus width and height, in absolute pixels.

{
  "images": [{"id": 1, "file_name": "shelf_001.jpg", "width": 1920, "height": 1080}],
  "categories": [{"id": 1, "name": "bottle"}, {"id": 2, "name": "can"}],
  "annotations": [
    {"id": 10, "image_id": 1, "category_id": 1, "bbox": [412, 220, 64, 180], "area": 11520, "iscrowd": 0}
  ]
}

The strengths are clear. One file holds everything, segmentation polygons and masks live next to boxes, keypoints have a defined structure, and metadata like image size travels with the labels. The weakness is that a single large file is harder to diff by hand and must be regenerated as a whole when anything changes.

COCO details that cause bugs

• Category ids do not have to start at 0 or be continuous. Some training code assumes they do and silently shifts classes.
• The iscrowd flag marks group regions. Many detectors ignore crowd regions during training, which is correct only if your labelers used the flag on purpose.
• The area field is used by some evaluators to bucket objects into small, medium and large. If it is missing or computed from the box instead of the mask, metrics shift.
• Image ids must be unique across the file. Merging two exports without remapping ids is a common source of orphaned labels.

The COCO annotation tool page explains how ImageLabeling handles ids, categories and polygon export.

How the YOLO txt format works

YOLO uses one text file per image, with the same base name as the image. Each line is one object: class index, then box center x, center y, width and height, all normalized by image width and height so they fall between 0 and 1. A data.yaml file lists the class names and the paths to the train, validation and test images.

# shelf_001.txt
0 0.231250 0.287037 0.033333 0.166667
1 0.512500 0.300000 0.030208 0.150000

path: dataset
train: images/train
val: images/val
test: images/test
names:
  0: bottle
  1: can

The format is compact, easy to read and fast to load. Changing one image means changing one small file. The cost is that coordinates are meaningless without the image size, and class meaning lives in a separate file that must stay in sync.

YOLO details that cause bugs

• Class indexes start at 0. A schema exported with indexes starting at 1 shifts every class by one, and training still runs.
• Coordinates are centers, not corners. Treating them as corners moves every box by half its size.
• Normalization uses the original image size. If images are resized after labeling with padding, the labels no longer line up.
• An image without objects should still exist in the image folder, with an empty label file or no file, so the model sees background.
• Rotated images with orientation metadata can be displayed upright in one tool and read sideways by a training loader.

The YOLO labeling tool exports txt files per image plus a ready data.yaml with your train, validation and test split.

How the Pascal VOC XML format works

Pascal VOC stores one XML file per image. Each object has a class name and a box defined by its top left and bottom right corners in pixels. The file also records image size and a few flags such as difficult and truncated.

<annotation>
  <filename>shelf_001.jpg</filename>
  <size><width>1920</width><height>1080</height><depth>3</depth></size>
  <object>
    <name>bottle</name>
    <truncated>0</truncated>
    <difficult>0</difficult>
    <bndbox><xmin>412</xmin><ymin>220</ymin><xmax>476</xmax><ymax>400</ymax></bndbox>
  </object>
</annotation>

VOC is readable and self contained, and class names as text avoid index mismatches. It is also verbose, has no standard place for polygons or keypoints, and fewer current training libraries read it directly.

Pascal VOC details that cause bugs

• Some older tools write 1 based pixel coordinates, others 0 based. A one pixel shift matters for tiny objects.
• Class names are free text, so "Bottle", "bottle" and "bottle " become three classes unless you normalize them.
• The difficult flag is often excluded from evaluation. Use it consistently or not at all.

Which annotation format should you use

Choose based on the training code you will actually run, then on the label types you need.

• Training a YOLO family detector or segmenter: YOLO txt with data.yaml.
• Training with a general detection or segmentation library, or evaluating against standard benchmarks: COCO JSON.
• Feeding an existing pipeline that already parses XML: Pascal VOC.
• Labeling keypoints or skeletons: COCO, or the YOLO pose variant if your trainer expects it.
• Needing masks: COCO polygons or RLE, or the YOLO segmentation variant.
• Sharing labels with a spreadsheet or a non ML stakeholder: CSV, with one row per object.

You do not need to pick only one. Keep labels in a tool that stores them in a neutral internal form, then export to whichever format each experiment needs. That way a switch from one model family to another is an export, not a conversion project.

Store labels once, export many times

Hand written converters are where most format bugs live. A converter that works for boxes may drop polygons, ignore empty images or reorder classes. ImageLabeling stores every label once and exports COCO JSON, YOLO txt plus data.yaml, Pascal VOC XML and CSV from the same frozen dataset version, so the class order and the split stay identical across formats. Existing labels in any of these formats can also be imported, which helps when you move a dataset from an older tool.

Mistakes when switching formats midway

Teams often start with one format and switch when they change model family. The risky moment is not the switch itself but the period when both formats exist. Someone fixes a label in the YOLO files, someone else trains from the old COCO file, and the two datasets drift apart without anyone noticing. Pick one source of truth for labels, treat every exported file as disposable, and regenerate exports from the source whenever labels change. Name export folders after the dataset version they came from so an old export is never mistaken for a new one.

How to check a format conversion before training

Never trust a conversion until you have drawn the converted labels back on the images. A five minute check saves a wasted training run.

• Pick twenty random images, including at least two with no objects and two with many objects.
• Render the converted boxes on top of each image and look. Shifted, mirrored or scaled boxes are obvious at a glance.
• Count objects per class before and after conversion. Totals should match exactly.
• Compare class names, not only indexes. Print the mapping from data.yaml or the categories list next to your schema.
• Check one image near each edge case: very small objects, objects touching the border, rotated photos.
• Confirm the train, validation and test splits did not leak images between them.

Coordinate math in one place

Converting a COCO box to YOLO: center x equals x_min plus half the width, divided by image width; center y equals y_min plus half the height, divided by image height; width and height are divided by image width and height. Converting VOC to COCO: width equals x_max minus x_min, height equals y_max minus y_min. Keep these formulas in one tested function, never in several scripts.

Where CSV fits next to COCO and YOLO

CSV is not a training format for most detectors, but it is useful around the training pipeline. One row per object, with image name, class, box coordinates and attributes, is easy to load into a spreadsheet, a notebook or a database. Data analysts use it to count objects per class, find images with unusual label density or check how attributes are distributed across splits.

CSV has limits. It flattens nested data, so polygons and keypoints become long strings that are awkward to parse, and there is no standard column order. If you share CSV exports, include a header row and state the coordinate convention (corners or center, pixels or normalized) in the column names, such as x_min_px or x_center_norm.

Segmentation and keypoints across formats

Boxes are where the three formats are most alike. Segmentation and keypoints are where they differ most.

• COCO stores polygons as flat lists of x and y pixel coordinates, and masks as run length encoding. Keypoints are triples of x, y and a visibility flag, with a skeleton defined per category.
• YOLO segmentation lines list the class index followed by normalized polygon points. YOLO pose lines add normalized keypoints after the box.
• Pascal VOC has no standard object level polygon or keypoint field. Segmentation in VOC style datasets lives in separate mask images.

If your project mixes boxes, masks and keypoints, COCO keeps everything in one structure, which is why many teams use it as the archive format even when they train with YOLO.

Formats and dataset versions

A format answers how labels are written. A dataset version answers which labels you trained on. Both matter for reproducibility. When you freeze a version, export it, and record the version name in your experiment log, you can always rebuild the exact training set later, even if labels have been corrected since. ImageLabeling lets you compare two versions and see labels added, changed or removed, which makes it clear why a model trained on version four behaves differently from version three.

For the labeling side of the pipeline, read how to label images for object detection. The bounding box annotation page covers the editor itself.