LabelMe video logo

MATLAB Toolbox Instructions for the LabelMe video Database
Written and maintained by Jenny Yuen and Antonio Torralba
(c) 2010 MIT, Computer Science and Artificial Intelligence Laboratory.

LabelMe video is a WEB-based video annotation tool that extends the image version for LabelMe. It allows researchers to annotate videos and share the annotations with the rest of the community.


Download (.zip) the toolbox and add it to the Matlab path.

Most of the function of the general LabelMe toolbox also work with LabelMe video. We describe here some of the function that are specific to video manipulation.


If you use this video dataset or the video-related functions on this toolbox, we would appreciate if you cite:

J. Yuen, B. C. Russell, C. Liu, and A. Torralba
LabelMe video: building a video database with human annotations
IEEE International Conference on Computer Vision (ICCV), 2009.


Download video database

The toolbox allows you using the dataset online without needing to download it first. Just execute the next lines to visualize the content of one of the folders (including all its children recursively) of the collection:

HOMEVIDEOS = '/desired/path/to/VLMFrames/';
HOMEVIDEOANNOTATIONS = '/desired/path/to/VLMAnnotations/';
folder = 'videos_iccv09/video_street_city_florida_usa';

where "/desired/path/to/VLMAnnotations" is the desired location where the annotations will be stored, and "/desired/path/to/VLMFrames/" is the desired path to store the videos (each video will be stored in a folder with all the frames as individual jpgs). This process will create the following directory structure under "/desired/path/to":



If you are looking for our ICCV 2009 dataset, download the following folder:

folder = 'videos_iccv09';

If you want to download the whole database, you can use call VLMinstall with two arguments only:


Toolbox description for video manipulation

A quick look into the dataset

Most of the function from LabelMe work also with video annotations. To build the index of videos, use the function LMdatabase:

HOMEVIDEOANNOTATIONS = '/desired/path/to/VLMAnnotations/';

The function LMplay visualizes one video and its annotations. To play the first video in the index D:


You can also extract the annotations that correspond to one frame:

annotationframe = LMgetvideoframe(D(1).annotation, frame);

The output is the annotation associated with the frame. This annotation has the same format than the annotation of an image in LabelMe. You can use all the same functions to plot the polygons, to extract segments than with the LabelMe Toolbox for images.


To perfom searches in video, you can also use the LMquery function available for LabelMe. This function allows searching for the content of any field.

[Dcar, j] = LMquery(D, '', 'car');

The new struct Dcar contains all the videos with cars and all other objects have been removed. The index array j points to the original index D. The struct D(j) contains all the images with cars without excluding other objects. For more details on the LMquery function, look in the LabelMe instructions page.


Let us know if you encounter bugs or have suggestions.

(c) 2009 MIT, Computer Science and Artificial Intelligence Laboratory.