How do you prepare a large computer vision dataset when a single image may contain several vehicles, license plates are captured from different angles, characters are partially obscured by dirt or glare, and the vehicles come from different countries? In this case study, we describe a US-DATA project involving the annotation of more than 50,000 images for an automatic license plate recognition system.
Project Objective
The US-DATA team received a dataset of more than 50,000 vehicle images to prepare training data for a computer vision system.
The model had to do more than simply detect a vehicle or a license plate. The system needed to learn how to locate the registration plate in an image, read its characters, and take additional object attributes into account.
The dataset could not be treated as “one image — one vehicle.” Some images contained several vehicles at once, so every relevant object had to be detected and described correctly.
The geographic diversity of the data added another layer of complexity: the dataset included license plates from different countries, with varying formats, character layouts, country identifiers, and visual designs.
What We Annotated
Several related annotation operations were performed for each relevant object.
1. License Plate Localization
The annotator identified the location of the registration plate and marked it with a bounding box. This type of annotation is used to train object detection models: the algorithm must learn not only that a license plate is present in the frame, but also exactly where it is located.
2. License Plate Transcription
The text on the license plate was converted into a structured text value. In practice, the annotation also prepared data for subsequent OCR — optical character recognition in images.
For example, the system needs to distinguish between visually similar characters such as С/C, Р/P, У/Y, as well as Y/V, 0/O, and other combinations, especially when image quality is poor.
3. Country Identification
A country label was assigned to each license plate. If the relevant country code was present and clearly readable, that information was used.
In more complex cases, the country had to be identified from the structure and format of the license plate itself.
4. Vehicle Type Classification
Each object was also classified by vehicle type — for example, passenger car, truck, or motorcycle.
As a result, a single image could contain multiple objects, while each object could require several related types of annotation.
Why 50,000 Images Do Not Mean 50,000 Simple Tasks
The number of images alone says little about the actual complexity of an annotation project.
If one image contains three vehicles, the annotator has to detect and verify several objects. If a license plate is difficult to read, the number of decisions and quality checks increases further.
For computer vision projects, it is therefore more accurate to evaluate not only the number of files, but also object density, the number of attributes, and the complexity of each image.
Several categories of data proved especially challenging in this project.
Dirty and Damaged License Plates
Real-world road images differ significantly from clean examples found in open datasets.
The license plates included:
- dirt;
- snow;
- road salt and de-icing chemicals;
- wear and abrasion;
- physical damage;
- film and other coverings;
- partially missing characters.
In some cases, part of the registration number was almost visually absent.
For training a production recognition system, such images are particularly valuable because the model is likely to encounter the same conditions after deployment.
Glare, Overexposure, and Night Images
Lighting was another major challenge.
A reflective license plate can be clearly visible to a person in daylight, but when a flash or another intense light source is used, the plate may appear as little more than a white patch in the image.
Night-time images therefore required special attention.
It was important to define the boundary between a character that could still be read with confidence and an image where reconstructing the value would already require guesswork.
For a training dataset, this is a fundamental principle: an annotator should not invent information that is not objectively present in the source data.
Unusual Angles and Perspective Distortion
Vehicles were not always photographed head-on.
The dataset included images captured:
- at sharp angles;
- almost at ground level;
- from above;
- from the side;
- with strong perspective distortion;
- with the vehicle partially occluded;
- with the license plate at a significant distance.
As a result, a rectangular registration plate could appear as a heavily distorted quadrilateral, occupy only a small part of the frame, or be partially hidden by elements of the vehicle.
These examples are essential for the model because the production system must detect license plates regardless of camera position.
Mixed Cyrillic and Latin Characters
A separate class of errors is caused not by image quality, but by the characters themselves.
Russian license plates use a set of Cyrillic letters that are visually identical or very similar to Latin characters. At low resolution, the differences between individual characters can become minimal.
For example:
С / C
Р / P
У / Y
At the same time, an international dataset may contain license plates from several countries.
A person can often infer the correct value from context. For a model, however, the quality of the annotation directly affects whether it learns to distinguish these cases.
Multiple Vehicles in a Single Image
Another important scenario involved images containing several vehicles.
In such a frame, it is not enough to identify the “main” vehicle. The entire scene must be reviewed systematically, the required objects identified, and each license plate linked to the corresponding vehicle and its attributes.
This turns the task from simple recognition of an individual object into full annotation of a complex computer vision scene.
These images clearly demonstrate the difference between basic image classification and professional object-level annotation.
How to Organize Annotation for a Large Dataset
The main challenge in scaling is to preserve the same annotation logic on the first, ten-thousandth, and fifty-thousandth image.
For this reason, a unified set of rules is established before full-scale annotation begins.
The guidelines define:
- what should be treated as a license plate;
- how its boundaries should be marked;
- how partially occluded objects should be handled;
- what to do with unreadable characters;
- how ambiguous cases should be labeled;
- how the country should be identified;
- which vehicle types should be used;
- when an image or object should be sent for additional review.
Complex examples are reviewed separately and turned into reference cases for the entire team.
This allows the annotation guidelines to evolve with the project without creating multiple conflicting rules within the same dataset.
Quality Control
At large volumes, an individual annotator’s error can quickly become a systematic problem.
If one part of the team annotates the same type of license plate according to one rule and another part uses a different rule, the neural network receives contradictory training examples.
The project therefore used unified instructions and an additional review process.
Special attention was given to edge cases involving:
- poorly readable characters;
- non-standard license plates;
- unusual camera angles;
- low-quality images;
- ambiguous country identification;
- multiple objects in a single frame.
When necessary, disputed examples were reviewed separately and, once resolved, used as references for subsequent annotation.
What the Client Received
The project resulted in a structured dataset of more than 50,000 annotated images prepared for further training of a computer vision system.
The dataset combined several layers of information:
image → vehicle → license plate → plate text → country → vehicle type.
This approach makes it possible to use the data for more than one narrow task.
A single prepared dataset can be used to train and test several system components:
- a license plate detector;
- a character recognition algorithm;
- a vehicle type classifier;
- a license plate country classifier;
- integrated automatic license plate recognition systems.
Why Real-World Data Matters for License Plate Recognition
A model can easily show strong results on a set of photographs where vehicles are captured in daylight, from the front, and every license plate is clean and clearly readable.
A production environment looks very different.
The camera may be installed too high or too low. The vehicle may be moving. It may be raining or snowing. The plate may be dirty. Some characters may be in shadow. Several vehicles may be present in the frame. The camera may be operating at night.
That is why the goal of dataset preparation is not to create the most visually “perfect” sample possible.
On the contrary, a high-quality dataset should reflect the real-world variability that the system will encounter after deployment.
This is the kind of data that enables a model to perform beyond laboratory conditions.
Applications for This Type of Annotation
Vehicle and license plate annotation can be used in projects involving:
- automatic number plate / license plate recognition — ANPR/ALPR;
- intelligent transportation systems;
- entry and exit control;
- parking systems;
- toll roads;
- urban video analytics;
- vehicle monitoring;
- logistics;
- security systems;
- computer vision for transportation infrastructure.
From a technical perspective, the project combines several in-demand areas at once: image annotation, object detection, bounding boxes, classification, OCR annotation, and dataset preparation for computer vision.
US-DATA: Image Annotation for Complex Computer Vision Tasks
US-DATA provides image annotation and dataset preparation for training computer vision systems — from pilot datasets to projects involving tens or hundreds of thousands of files.
We can support a standalone annotation stage or help build the entire process: formalize requirements, prepare annotation guidelines, run a pilot, scale the team, and organize quality control.
Real-world data almost always contains edge cases. Professional annotation is therefore not only about processing large volumes quickly, but also about ensuring that the entire dataset remains logically consistent.
Services Used in Similar Projects
- Data Annotation for Machine Learning
- Image Annotation for Neural Networks