PKU-NEC @ TRECVid 2011 SED: Sequence-Based Event Detection in Surveillance Video *

In this paper, we describe our system for surveillance event detection task in TRECVid2011. We focus on pair-wise events (e.g., PeopleMeet, PeopleSplitUp, Embrace) that need to explore the relationship between two active persons, and action-like events (e.g. ObjectPut and Pointing) that need to find the happenings of a person's action. Our team had participated in the TRECVidSED task in 2009 and 2010. This year the new improvements of our system are three-folds. First, we treat object detection and tracking as one problem, and integrate detection and tracking in one unified framework. That is mean "detection by tracking" and "tracking by detection". Also, we fuse multiple trackers to obtain a more accurate tracking result. Experimental results show that our system can achieve a much better precisionand recallthan our previous systems. Second, we propose sequence learningbased method for pair-wise events detection. Visual features are extracted as a cubic feature representation and the discrimination is based on multiple relational and sequence kernels. Experimental results show that our system can detect more correct events with less false alarms. Third, a Markov-model based classifier is employed for action-like event detection. We define some states and learn the transition relationamong these states to detect the event. Experimental results show our detectors are feasible and effective. Overall, we have submitted three versions of results, which are obtained by using different human detection, tracking and events detection modules. According to the results in the TRECVidSED formal evaluation, our experimental results are promising.