Auto-annotating camera and LiDAR data with foundation models
Labelling camera and LiDAR data is slow and expensive. How foundation-model pre-labels and camera-LiDAR fusion speed it up without lowering quality.
Annotation is often the most expensive part of a perception project. Every frame from a camera and every sweep from a LiDAR needs labels before a model can learn from it.
Pre-label, then review
The pipeline does not replace annotators. It gives them a first draft: masks from foundation models on the camera image, projected onto the point cloud through the camera-LiDAR calibration. Annotators correct the draft instead of drawing from scratch.
Keeping quality honest
A pre-label that is almost right can be worse than none, because people stop looking closely. Each batch is compared against a sample labelled by hand, and the pre-labels are switched off for any scene type where they fall below the agreed quality.
The result
On the project at Continental, pre-labelling cut labelling costs by 70%.
Keep reading
All posts
Zero-shot road segmentation with foundation models
What foundation models like SAM and CLIP got right on road scenes without labelled training data, where they failed, and how we measured it at Continental.
How to evaluate a RAG system without fooling yourself
A high retrieval score can still hide wrong answers. The reference sets, metrics and release gate I use to know whether a RAG change made answers better.
Running open-weight LLMs on your own GPUs
When health data cannot leave the building, the model has to come to the data. What self-hosting open-weight LLMs costs, and what it buys you.