Vision, Image Processing & Sound Lab

Overview

The data collection campaign focused on 4 hockey matches held in Trento (Italy) during the 26th Winter Universiade. We used 5 cameras, a full HD camera (1920x1080, 30 fps, focal length 4mm) for the ice rink, another one for a panoramic view of all the bleachers, and 3 high resolution cameras (1280x1024, 30 fps, focal length 12mm) focusing on different parts of the spectator crowd.
From each match we selected a pool of sequences in order to represent a wide, uniform and representative spectrum of situations, e.g. tens of instances of goals, shots on goal, saves, faults, timeouts (each sequence has more than one event).
Each sequence has been annotated frame by frame, spectator by spectator, by a first annotator, using the ViPER format. The annotator had to perform three different macro tasks: detection (localizing the body and the head), posture and action annotation, respectively. capturing fine grained actions such as hands on hips, clapping hands, watching the cellphone etc. (click here for more details), for a total of more than 100 millions of annotations.

After each match, we asked to a percentage of uniformly distributed spectators (30%) to fill a simple questionnaire with three questions:

• Which team did you support in this match?
• Did you know at the beginning of the match who was sitting next to you?
• Which has been the most exciting action in this game?

Downloads

• The whole dataset (Version 1.0)
• Dataset used in paper (Version 1.0)

Results

In this work we present a set of possible applications on S-HOCK. In particular we focus on two classical tasks, such as people detection and head pose estimation, and one more interesting application from the social point of view, such as spectator categorization.

People Detection

People detection is a standard and still open research topic in computer vision, with the HOG features and the Deformable Part Model (DPM) as workhorses, and plenty of alternative algorithms. Unfortunately, most of the methods in the literature are not directly usable in our scenario, mostly for two reasons: low resolution - a person has an average dimension of 70x110 pixels - and occlusions usually only the upper body is visible, rarely the entire body and sometimes only the face.
We provided 5 different baselines for people detection:

HOG+SVM

Detector based on HOG (cell size of 88 pixels) and a linear SVM classifier.

HASC+SVM

Detector based on Heterogeneous Auto-Similarities of Characteristics (HASC) descriptor and a linear SVM classifier.

ACF

Aggregate Channel Features detector which uses the Viola-Jones framework to compute integral images (and Haar wavelets) over the color channels, fusing them together.

DPM

Deformable Part Model which combines part's templates arranged in a deformable configuration fed into a latent SVM classifier.

CUBD

Calvin Upper Body Detector which is a combination of the DPM framework trained on near-frontal upperbodies - i.e. head and upper half of the torso of the person - and the Viola-Jones face detector.

On top of all these methods, we propose an extension based on the strong prior we have in our kind of crowd, i.e. the people are "constrained" by the environment to arrange in a grid - the seats on the bleachers. Assuming a regular grid (considering the camera perpendicular at the plane of the bleachers and ignoring distortion effects) and acounting for the fact that people are more likely to be located on the same rows and columns, we can just add to the detection confidence map the average of the map over the rows and the columns.

Head Pose Estimation

Spectator Categorization

The spectator categorization task consists in finding the supporters of each team, accounting for the fact that fan of the same team will have a similar behavior at some events (a goal etc.) which is strongly different from that of the other supporters.

Ground Truth	AS2007	MMS2010	Our
Accuracy	0.592	0.559	0.621

Citation

Contacts

• Davide Conigliaro: