(with Allan Jepson)
The computation of optical flow relies on merging information available over an image patch to form an estimate of 2D image velocity at a point. This merging process raises a host of issues, which include the treatment of outliers in component velocity measurements and the modeling of multiple motions within a patch which arise from occlusion boundaries or transparency. We present a new approach which allows us to deal with these issues within a common framework. Our approach is based on the use of a probabilistic mixture model to explicitly represent multiple motions within a patch. We use a simple extension of the EM-algorithm to compute a maximum likelihood estimate for the various motion parameters. Preliminary experiments indicate that this approach is computationally efficient and can provide robust estimates of the optical flow values in the presence of outliers and multiple motions. The basic approach can also be applied to other problems in computational vision, such as the computation of 3D relative motion, which require the integration of several partial constraints to obtain a desired quantity.
The black box in the image contains an image region that spans a motion discontinuity.
The optical flow constraints in this region come from two different surfaces. The "X's" mark the best estimated motions.
The set of constraint lines corresponding to one of the motions are shown. The darkness of the line indicate how likely they are to have come from this population.
Motion constraint lines for the other motion.
Jepson A. and Black, M., Mixture models for optical flow computation, in Partitioning Data Sets, DIMACS Workshop, April 1993 Eds. Ingemar Cox, Pierre Hansen, and Bela Julesz, AMS Pub., Providence, RI., pp. 271-286; also, Tech. Report, Res. in Biol. and Comp. Vision, Dept. of Comp. Sci., Univ. of Toronto, RBCV-TR-93-44, 1993.
Jepson A. and Black, M., Mixture models for optical flow computation, IEEE Conf. on Computer Vision and Pattern Recognition, CVPR-93, New York, NY, June, 1993, pp. 760-761; also, University of Toronto, Technical Report RBCV-TR-93-44, April 1993.
Ju, S., Black, M. J., and Jepson, A. D., Skin and Bones: Multi-layer, locally affine, optical flow and regularization with transparency, IEEE Conf. on Computer Vision and Pattern Recognition, CVPR'96, San Francisco, CA, June 1996, pp. 307-314.