Class imbalance is normal. Panic imbalance is optional.
If you only label what is easy to find, rare classes disappear. If you only chase rare classes, throughput collapses.
This guide is a practical sampling plan.
Name the business-critical classes
Not every class deserves the same effort.
List:
- must-not-miss classes
- nice-to-have classes
- experimental classes
Imbalance strategy starts with that list.
If your taxonomy is still fuzzy, fix guidelines first.
Use quotas instead of vibes
Define weekly targets such as:
- minimum rare-class items per week
- maximum easy-class items before rare quota is met
- a cap on duplicate near-identical scenes
Quotas turn imbalance into a schedule problem.
Hard negatives are part of the dataset
Rare classes fail in production when negatives are too easy.
Collect negatives that look like the class:
- similar color and shape
- partial matches
- confusing background objects
Hard negatives should be labeled, not skipped.
Reviewer time follows risk
Spend review hours where errors hurt most.
If a rare class is safety critical, tighten QA thresholds for that class.
Use ideas from data annotation QA checklist.
Sampling sources: diversify early
Rare classes often hide in narrow slices.
Plan sources:
- different lighting
- different cameras
- different locations or SKUs
- different seasons or layouts if relevant
If rare examples all look the same, the model overfits that look.
Metrics that do not lie to the team
Track weekly:
- raw count per class
- reviewed count per class
- disagreement rate per class
- time-to-label for rare items
If rare counts are zero for two weeks, the plan is broken.
Connect to training reality
Imbalance labeling should match training strategy.
If training uses class weights or focal loss, labeling still needs enough positives.
If training uses oversampling, labeling still needs real diversity.
For broader dataset design, read build image dataset for object detection.
Versioning when classes merge or split
Imbalance strategy changes when taxonomy changes.
Log:
- which classes merged
- how old labels map forward
- whether old items need relabel
See workflow automation and dataset versioning.
Active learning without starving rare classes
If you use model-guided selection, add a rule:
- always reserve budget for rare class discovery
Otherwise the model will steer you toward frequent errors only.
Common mistakes in 2026
Mistake: measuring only total images labeled
Rare classes can be at zero while charts look green.
Mistake: oversampling identical rare scenes
You create a different kind of imbalance.
Mistake: skipping negatives
Recall looks fine until deployment.
Mistake: no escalation path for ambiguous rare cases
Annotators guess, reviewers guess, metrics guess.
Final takeaway
Imbalance is a planning problem.
Quotas, negatives, and risk-based review beat heroic labeling sprints.
FAQ
What if rare items are truly hard to find?
Widen collection first. If you cannot find data, you cannot fix it with labeling tricks alone.
Should we balance the validation set?
Often yes for monitoring rare performance. Document how you built that eval slice.
How strict should quotas be?
Strict enough to protect rare classes. Loose enough to keep shipping weekly.