Skip to main content
Blog
Tutorial
Oct 25, 20268 min

Search a Video by Describing the Scene: A Guide to LabelOp Moments

LabelOp Moments finds moments in a local MP4 when you describe what is on screen. Here is how it works, what to type, and where it falls short.

You know the scene is in there somewhere. A red car pulls into a parking lot. Someone holds up a whiteboard. A dog runs across a beach.

The video is forty minutes long, and scrubbing the timeline is slow, tiring, and easy to get wrong.

LabelOp Moments is a free in-browser app for exactly that problem. You choose a local video, describe the scene in your own words, and jump to the closest matching moments. The video stays on your device.

Moments is part of the LabelOp family, next to the LabelOp dataset labeling platform, LabelOp Tracker for video annotation, and LabelOp Retouch for photo editing.

Short answer

Moments looks at your video about once per second, turns each sampled frame into a numeric representation with an on-device model, and compares those representations with your description. It then shows up to five Closest moments, each with a timestamp you can jump to.

  • No account and no upload. The video and your queries stay in the browser.
  • A one-time model download (about 388 MB) after you agree to it, then cached.
  • Works offline once the model and app files are cached.
  • Best in current desktop Chrome or Edge with WebGPU and hardware acceleration turned on.

What you need

  • A video file: an MP4 containing H.264 video. Moments checks codec and browser compatibility before it starts any analysis.
  • A browser with WebGPU and WebCodecs: current desktop Chrome or Edge is the recommended setup. Compatibility guidance only appears if a real check fails.
  • About 388 MB of browser storage for the search model, plus an internet connection for that first download.

That is it. There is nothing to install and no sign-up.

Walkthrough

The first screen reads "Find any moment in a video." and lists three steps: Choose a video, Describe the scene, Jump to the moment.

1. Choose a video

Pick an MP4 from your computer, or drop it onto the page. Selecting a video does not start a download by itself.

2. Download the search model once

On first use, Moments shows a one-time download notice for the search model, EmbeddingGemma 2 Text Vision, which runs through the LiteRT-LM runtime. Choose Download model to start it.

A few details worth knowing:

  • Cancel download stops the transfer and keeps your selected video. Try again restarts setup.
  • The model file is checked against a pinned SHA-256 hash before it is stored. A mismatched download is never cached.
  • After that, the cached model is reused automatically. Your browser manages that cache and can evict it, so treat it as a convenience, not a permanent copy.

3. Let analysis start, then search early

Analysis samples about one frame per second and runs on your GPU through WebGPU. You do not have to wait for it to finish. You can search as soon as the first frames are ready, and the interface shows how far analysis has reached.

If you searched early, the results only cover the footage analyzed so far. Run the search again (or use Include newer footage) to cover more of the video. You can also pause and resume analysis.

4. Describe the scene

Type what you expect to see in the Describe the scene box. The placeholder gives the idea: "A rabbit standing in a field of flowers…" Press Enter to search; Shift+Enter adds a new line. Descriptions are limited to 240 characters.

5. Review the Closest moments

Moments returns up to five candidates, at least three seconds apart, so you do not get five copies of the same shot. For each result you can:

  • Click it to seek the player to that timestamp.
  • Preview it, which plays up to three seconds either side of the timestamp.
  • Save it as a favorite.
  • Open Details to see the raw similarity score and Refine this moment, which re-scans that candidate's window (up to six seconds) at two frames per second and returns the best local match.

Sort switches between Relevance and Time without running the model again. A compact timeline below the player marks where candidates and favorites sit.

6. Narrow the search with a time range

If you already know roughly where the scene is, enable Search a time range and enter a start and end as seconds, mm:ss, or hh:mm:ss. Only analyzed frames inside that range are ranked. The start is included and the end is excluded. If no frames have been analyzed in that interval yet, you get an empty result; widen the range or let analysis reach it.

7. Keep what you found

The Saved view has two tabs:

  • Favorites keeps a saved moment's timestamp and preview window.
  • History lets you rerun a previous query. On the same video it restores the exact range; on a different video it reuses the wording and searches the whole video.

Both store only compact metadata, such as file name, timestamps, query, and range, in your browser's local storage. They never store the video, a thumbnail, or a decoded frame. Favorites are capped at 50 and history at 20 recent searches.

Refreshing the page clears the analysis itself. Your favorites and history survive, but you need to choose the file again to search it.

How to write descriptions that work

Moments compares your text with individual sampled frames. That one fact explains most of what works and what does not.

Describe what a single frame would show. Good: "a person in a yellow jacket standing next to a forklift." Weak: "the moment the forklift starts moving."

Name visible objects, colors, and settings. "A white van parked on a snowy street at night" gives the model more to match than "the van scene."

Prefer static descriptions over events. A frame can show someone holding a cup. It cannot show that they just picked it up.

Keep it to one scene. If you want two different things, run two searches.

Rephrase instead of repeating. If the top five look wrong, try different words for the same scene, or narrow the time range.

Other languages can work, but test them. The project's own tests included Turkish queries alongside English ones. English did slightly better in that small set, so if a non-English query underperforms, try an English version.

What the results do and do not mean

The heading says Closest moments on purpose. Moments always returns the frames most similar to your description, even if the scene is not in the video at all.

In the project's own tests, every query for a scene that was absent from the video still returned candidates. The Details panel says it plainly: similarity is not confidence, and the scene may not be in the video.

So read results as "places worth checking," not as an answer. Preview them before you trust them.

Limits to know before you rely on it

Moments is deliberately narrow. From its own documentation:

  • One video at a time, MP4 with H.264 video.
  • Visual only. There is no audio analysis, so it cannot find spoken words or sounds.
  • Single frames, about one per second. Very short events can fall between samples. Direction of motion and the order of events are not understood.
  • No clip export. It finds and plays timestamps; it does not cut or export video.
  • No persistent index. Refreshing clears the analysis; the model download is what gets cached.
  • Preview windows are not event boundaries. They show a few seconds around a frame, not where an action starts and ends.

How well does it work?

The project publishes measured results with clear caveats, and they are worth reading as such.

On a test with one animated short film (about ten minutes) and 20 queries whose expected answers were written down before the model ran, the expected scene appeared in the top five for 19 of 20 queries on an NVIDIA RTX 5060 Ti, using real WebGPU. A follow-up on three other videos found the expected scene in the top five for 32 of 36 positive English and Turkish queries.

The same documentation is explicit that a handful of test videos does not establish accuracy on arbitrary footage, that single-frame embeddings can miss short events or motion, and that scores are not calibrated confidence. Treat those numbers as a sanity check, not a general accuracy claim. Your footage, GPU, and wording will change the outcome.

Privacy

The video file is read locally by the browser. Decoded frames, analysis, and your queries are never uploaded; there is no upload endpoint and no account. The app's offline cache stores the app shell and model runtime, never video data or queries.

The site does use privacy-friendly page analytics (Umami), so it is not accurate to say the page makes no network requests at all. The accurate statement is that your video and your queries never leave the device.

If privacy is your main reason for looking, the companion post on how on-device video search works explains where each piece of data lives.

Who it is for

  • Editors and researchers who need to find a shot in long footage without uploading it.
  • Computer vision teams looking for specific scenes to label. See how to find frames worth labeling in long videos.
  • Anyone with sensitive footage that should not go to a cloud service.

If you need to annotate the video itself, with boxes and tracks across frames, that is a different job; see LabelOp Tracker. For still images from that footage, LabelOp handles labeling, review, and export.

Try it

Open LabelOp Moments, choose an MP4, and describe one scene you know is in it. That is the fastest way to learn how your footage and your wording behave.

FAQ

Does LabelOp Moments upload my video?

No. The video is read and analyzed in your browser, and neither the video nor your queries are uploaded. The site uses privacy-friendly page analytics, but that does not include your video or searches.

Which video formats does Moments support?

MP4 files containing H.264 video. Moments checks codec and browser compatibility before it starts analysis.

Why is there a 388 MB download?

That is the search model, EmbeddingGemma 2 Text Vision. It downloads once after you agree, is verified, and then is cached by the browser so later sessions can reuse it, including offline.

Can Moments find something that was said in the video?

No. Moments analyzes visual frames only. There is no audio or speech analysis.

Why do I get results when the scene is not in the video?

Moments always shows the closest moments it found. Similarity is not confidence, so preview each candidate before relying on it.

Take the next step with your dataset.

Use local AI, managed cloud labeling, and team review in LabelOp. Export your work or train with your own Vast.ai account.

Start free