SAM 2 for Fast Image Annotation: The Complete Guide (with Python Script)

Manual polygonal segmentation is the most expensive operational bottleneck in computer vision. Tracing complex boundaries around vehicles, biological specimens, agricultural crops, or industrial components often takes 60 to 120 seconds per object.

With Meta AI's Segment Anything Model 2 (SAM 2), annotators can generate sub-pixel accurate segmentation masks in less than 2 seconds using simple point clicks or loose bounding box prompts.

This guide explores the mechanics of SAM 2 for dataset labeling and provides a production Python script to generate masks and convert them directly to COCO polygons and YOLO-seg formats.

---

1. Why SAM 2 Changes Dataset Annotation

Compared to previous foundation models and SAM 1:

• Higher Sub-Object Precision: Improved edge detection on thin, occluded, and transparent objects.
• Lower Latency: Optimized memory footprint and streaming architecture allows real-time inference directly in web applications.
• Multi-Prompt Support: Combine positive clicks (green), negative clicks (red), and bounding box priors simultaneously.

---

2. Python Implementation: Running SAM 2 Inference and Exporting Polygons

Below is a complete script demonstrating how to run SAM 2 image inference, predict binary masks from positive/negative prompt points, and convert those masks into COCO polygon coordinates using OpenCV:

#!/usr/bin/env python3
"""
sam2_annotation_pipeline.py
Runs SAM 2 point prompting and converts predicted binary masks into COCO/YOLO-seg polygons.
Requirements: pip install torch torchvision opencv-python numpy
"""

import cv2
import numpy as np
import torch
from sam2.build_sam import build_sam2
from sam2.sam2_image_predictor import SAM2ImagePredictor


def mask_to_polygons(mask: np.ndarray, epsilon: float = 1.5) -> list:
    """
    Converts a binary boolean mask into a simplified polygon coordinate list.
    """
    mask_uint8 = (mask * 255).astype(np.uint8)
    contours, _ = cv2.findContours(
        mask_uint8, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE
    )

    polygons = []
    for contour in contours:
        # Filter tiny noise contours
        if cv2.contourArea(contour) < 20:
            continue
        
        # Approximate contour to reduce vertex count while preserving shape
        approx = cv2.approxPolyDP(contour, epsilon, closed=True)
        if len(approx) >= 3:
            polygon = approx.flatten().tolist()
            polygons.append(polygon)

    return polygons


def polygon_to_yolo_seg(polygon: list, img_width: int, img_height: int) -> str:
    """
    Normalizes polygon coordinates [x1, y1, x2, y2, ...] to 0.0-1.0 for YOLOv8-seg.
    """
    normalized = []
    for i in range(0, len(polygon), 2):
        nx = polygon[i] / img_width
        ny = polygon[i + 1] / img_height
        normalized.extend([f"{nx:.6f}", f"{ny:.6f}"])
    return " ".join(normalized)


def main():
    # 1. Load SAM 2 Model
    checkpoint = "checkpoints/sam2_hiera_large.pt"
    model_cfg = "sam2_hiera_l.yaml"
    device = "cuda" if torch.cuda.is_available() else "cpu"

    print(f"Loading SAM 2 model on {device}...")
    sam2_model = build_sam2(model_cfg, checkpoint, device=device)
    predictor = SAM2ImagePredictor(sam2_model)

    # 2. Load Input Image
    image_path = "sample.jpg"
    image_bgr = cv2.imread(image_path)
    image_rgb = cv2.cvtColor(image_bgr, cv2.COLOR_BGR2RGB)
    h, w, _ = image_rgb.shape

    predictor.set_image(image_rgb)

    # 3. Provide Prompts (e.g. Foreground Click at [500, 350], Background Click at [620, 410])
    input_points = np.array([[500, 350], [620, 410]], dtype=np.float32)
    input_labels = np.array([1, 0], dtype=np.int32)  # 1 = Foreground, 0 = Background

    # 4. Predict Binary Mask
    masks, scores, logits = predictor.predict(
        point_coords=input_points,
        point_labels=input_labels,
        multimask_output=False
    )

    best_mask = masks[0]  # Shape: (H, W), boolean
    print(f"Mask prediction score: {scores[0]:.4f}")

    # 5. Extract Simplified Polygon Coordinates
    polygons = mask_to_polygons(best_mask, epsilon=2.0)
    print(f"Extracted {len(polygons)} polygon contours.")

    # 6. Format for YOLOv8/YOLOv11 Segmentation
    class_id = 0
    for poly in polygons:
        yolo_seg_str = polygon_to_yolo_seg(poly, w, h)
        yolo_line = f"{class_id} {yolo_seg_str}"
        print(f"YOLO-seg annotation line:\n{yolo_line[:80]}...")


if __name__ == "__main__":
    main()

---

3. Integrating SAM 2 into Annotation Workflows

While local Python scripts are suitable for offline batch pipelines, interactive labeling requires real-time point adjustments on a canvas.

LabelOp's Native SAM 2 Implementation

LabelOp integrates SAM 2 directly into the web labeling workspace:

1. Point Prompts: Click once inside any object; SAM 2 instantly draws a vector polygon around the contour.
2. Negative Points: Click to erase unwanted overlapping shadows or background segments.
3. Vertex Editing: Instantly switch to polygon mode to tweak individual control points if needed.
4. Universal Export: Export annotations directly as COCO segmentation polygons or YOLO-seg normalized masks.

---

4. Bounding Box vs. Polygon: When to Use SAM 2

• Use Bounding Boxes (Object Detection): When your downstream model is standard YOLOv8/v11 detection and speed is paramount. Read the full comparison in Polygon vs Bounding Box Annotation.
• Use SAM 2 Polygons (Instance Segmentation): For precision robotics, surface defect inspection, medical anatomy, or autonomous vehicle lane boundary mapping.

---

Summary

SAM 2 bridges the gap between bounding box speed and polygon precision. Try LabelOp Smart Annotation to start labeling with SAM 2 in your browser today.