Recent Crowd Analysis Projects:
M. Rodriguez J. Sivic I. Laptev JY. Audibert. Data-driven Crowd Analysis in Videos.
Proceedings of the IEEE International Conference on Computer Vision (2011), Oral.
Overview:
In this work we use a large collection of crowd videos to learn crowd motion patterns by performing long-term analysis in an off-line manner. The learned motion patterns can be used in a range of application domains such as crowd event detection or anomalous behavior recognition. In this particular work, we choose to use the motion patterns learned on the database to drive a tracking algorithm. The idea is that any given crowd video can be thought of as being a mixture of previously observed videos.
In this work we approached crowd analysis from a new direction. Instead of learning a set of collective motion patterns which are geared towards constraining the likely motions of individuals from a specific testing scene, we have demonstrated that there are several advantages to searching for similar behaviors amongst crowd motion patterns in other videos. Our experiments show that by leveraging a large database of previously observed crowd behavior patterns we are able to accurately track individuals in a crowd exhibiting both typical and rare behaviors.
Narrated Video Demo
Density-aware person detection and tracking in crowds
M. Rodriguez I. Laptev J. Sivic JY. Audibert. Density-aware person detection and tracking in crowds. Proceedings of the IEEE International Conference on Computer Vision (2011), Poster.
PDF | Abstract | BibTeX | Matlab Code and Dataset
Overview:
Detecting and tracking people in crowded scenes is a crucial component for a wide range of applications including surveillance, group behavior modeling and crowd disaster prevention. The reliable person detection and tracking in crowds, however, is a highly challenging task due to heavy occlusions, view variations and varying density of people as well as the ambiguous appearance of body parts, e.g. the head of one person could be similar to a shoulder of a near-by person. High-density crowds present particular challenges due to the difficulty of isolating individual people with standard low-level methods of background subtraction and motion segmentation typically applied in low-density surveillance scenes.
Rather than modeling individual interactions of people, this work exploits information at a more global level provided by the crowd density and scene geometry. We show that automatically obtained person density estimates can be used to improve
person localization and tracking performance in crowded scenes.
Tracking in Unstructured Crowded Scenes
Mikel Rodriguez, Saad Ali and Takeo Kanade, Tracking in Unstructured Crowded Scenes, International Conference on Computer Vision, September 2009.
A crowded scene can be divided into two categories: structured and unstructured. In a structured crowded scene, the crowd moves coherently in a common direction, and the direction of motion does not vary over time. That is, each spatial location of the scene supports only one dominant crowd behavior over the video. For instance, a video of a marathon race represents a structured crowded scene because all athletes run along the same path, thus generating a crowd behavior which has a fixed direction of motion/pattern at each location of the path.
Other examples of structured crowded scenes include processions, events involving queues of people, and traffic on a road (see Figure1). In an unstructured crowded scene, the motion of the crowd appears to be random, with different participants moving in different directions at different times. That is, in such scenes each spatial location supportsmore than one, or multi-modal, crowd behavior. For instance, a video of people walking on a zebra-crossing in opposite directions is an example of an unstructured crowded scene because, broadly speaking, at any point on the zebra crossing the probability of observing a person moving from left to right is as likely as observing a person walking from right to left. Other examples of such scenes include exhibitions, crowds in a sporting event, railway stations, airports, and motion of biological cells.
To overcome the problem of tracking in unstructured crowded scenes, we develop a tracking algorithm that uses Correlated Topic Model (CTM) to capture different overlapping and non-overlapping crowd behaviors in the scene. In our construction, words correspond to low level quantized motion features and topics correspond to crowd behaviors. We used CTM as it provides an elegant way to handle multi-modality of crowd behavior as each location can have a certain probability of belonging to certain crowd behavior (or topic). In addition, it models interactions among topics (or crowd behaviors) which is also desirable, as explained later, for the types of scene that we are handling. Note that, we will use terms ’crowd behavior’, ’behavior’ and ’topic’ interchangeably throughout the paper.
For illustration and understanding purposes we show crowd behaviors learned for a typical busy crosswalk scenario in the figure above. Our model is able to capture different behavior modalities at specific locations in the scene. This can be observed in the figure above, where we overlay the two most common crowd behaviors learned by our framework. By observing the colors (which represent directions of motion) in each of the crowd behaviors, it can be seen that one of the behaviors corresponds to pedestrians which walk towards the rightmost end of the crosswalk, whereas the other behavior corresponds to pedestrians walking in the opposing direction.
Also note that, in the above figure that the learned crowd behaviors are not spatially mutually exclusive. Therefore,multiple behaviors can occur at different spatial locations in the scene with certain probabilities. Each of these behaviors can then be incorporated as high level information which can aid tracking individuals in this class of scenes. The main contributions of our work are: 1) Extending the idea of using high-level knowledge for tracking in crowds by learning representations of unstructured and multi-modal crowd behavior; 2) Using CTM to solve an existing problem in a crowd tracking framework.
Crowd Tracking Matlab Application:
This standalone matlab tracking application implements the algorithm described in "Tracking in Unstructured Crowded Scenes", Rodriguez, Ali, Kanade, ICCV 2009. The application supports the following features:
o Learning crowd motion patterns
o Target tracking by manually selecting the initial target location in a frame
o Target tracking by specifying the initial target location via a comma separated text file (x,y)
o Outputs .mat files containing the tracker position at each frame
Crowd Tracking Application Demo:
Obtain the matlab tracking application here.
Resources:
Camera ready version of the paper
Matlab Tracking Crowds Application
Crowd patterns and tracks