Visual Tracking via Online Part Matching

Tianzhu Zhang      Kui Jia      Si Liu      Changsheng Xu      Narendra Ahuja      Ming-Hsuan Yang

Abstract

Part-based representations have been shown to be effective in visual tracking for handling partial occlusion. While numerous methods have been developed, it remains a challenging problem to effectively exploit the confidence scores of individual parts for visual tracking. We address this problem by simultaneously matching parts in multiple frames with a locality-constrained low-rank sparse learning approach, which determines multi-frame part correspondences through optimization of partial permutation matrices. The proposed part matching tracker has a number of attractive properties. First, it exploits the spatial-temporal locality constraint property for robust part matching. Second, it is realized by using more information across multiple frames rather than a pair of images as used in existing methods. Third, the local parts from multiple frames are jointly modeled by considering their low-rank and sparse structure which can effectively describe appearance variations caused by occlusion or noise. Fourth, an online part matching model is proposed that exploits information from multiple frames for further improvement of robustness and efficiency in visual tracking. We evaluate the proposed tracking algorithm with comparisons to the state-of-the-art methods on challenging benchmark datasets for single and multiple object tracking. Experimental results show that the proposed algorithm based on online part matching performs favorably against existing methods.

Overview

Figure 1 - Tracking results of the proposed algorithm are denoted with red bounding boxes. The blue cross marks denote the positions of parts, and the blue lines represent their correspondences. It is clear that the part based matching is robust to partial occlusion.

Our Approach

Figure 2 - Proposed part matching across K frames. Given the K1 and K2 frames, after part sampling and matching across the K frames, we obtain the matching results in the K2 frames to help object tracking. The circles with different colors denote the part sampling with locality property. Note these circles have overlaps and these parts inside each frame are not matched independently. The 6 curves with different colors represent the part matching across the multiple frames. After part matching, the best matched parts in the K2 frames are denoted with the corresponding colors as in the K1 frames.

Experimental Results

Single Object Tracking

Datasets: we use a set of 25 challenging videos with ground truth object locations including the One Leave Shop Reenter1cor (olsr1), One Leave Shop Reenter2cor (olsr2), One Shop One Wait2cor (osow), human, ball, biker, bolt, car11, car4, coke11, david indoor (david), faceocc2, fernando, football, girl, jumping, panda, singer, skating, soccer, sunshade, surfer, surfing, sylv, trellis70 sequences. In addition, we also conduct the experiments on the recent visual tracking benchmark dataset.

Evaluated Tracking Methods: we compare the proposed tracking methods (PMT and OPMT) with 15 state-of-the-art algorithms including the real time compressive tracking (RTCT), incremental visual tracking (IVT), online multiple instance learning (MIL), online Adabost boosting (OAB), fragments-based (Frag), Struck, L1 tracking, tracking by detection (TLD), circulant structure tracking (CST), sparse collaborative model (SCM), distribution field tracking (DFT) [59], part-based visual tracking (PT), local-global tracker (LGT), multiple experts using entropy minimization (MEEM), and adaptive structural local sparse appearance tracking (ASLA) methods. For more information, please refer to the paper.

Results on 25 Challenging Image Sequences: Figure 3 shows the average success and precision plots based on OPE. Overall, the proposed OPMT performs favorably against the state-of-the-art methods on this set of image sequences. For example, the proposed OPMT performs well against the MEEM method (by 4.8%) in terms of success rate. In addition, the proposed OPMT algorithm performs well against the PT method (by 5.7%) in terms of precision. The proposed OPMT and PMT methods are able to track objects undergoing large appearance variations caused by occlusions, pose variations, illumination change and abrupt motions in cluttered backgrounds.

Figure 3 - Plots of OPE on the 25 image sequences where most objects are heavily occluded. For success plots the score is the AUC value while for precision plots the score is the precision value at the threshold of 20 pixels.

Results on the Benchmark Dataset: Figure 4 shows the success and precision plots of OPE of the top 10 performing tracking methods. Overall, the proposed OPMT algorithm performs well against the state-of-the-art tracking methods. We analyze the tracking performance based on challenging attributes of object tracking [20]. Due to space constraints, we present the success and precision plots of OPE for 11 attributes in Figure 6 and more results are available on the project web site mentioned above. For presentation clarity, we present the top 10 performing methods in each plot. Overall, the OPMT method performs well in all attributebased evaluation against the state-of-the-art methods.

Figure 4 - Precision and success plots of OPE on the visual tracking benchmark[20]. The performance score of each tracker is shown in the legend.

Figure 5 - Attribute based success plots and precision plots of OPE. 11 subsets are listed. The value at the end of the title is the number of sequences in that subset.

Related Publications

Video Tracking Results

We show tracking results of 25 challenging videos.