Motion capture, a brief description

Motion capture or mocap is the process of recording the movement of objects or people. It is used in military, entertainment, sports, and medical applications, and for validation of computer vision and robotics. In filmmaking and video game development, it refers to recording actions of human actors, and using that information to animate digital character models in 2D or 3D computer animation. When it includes face and fingers or captures subtle expressions, it is often referred to as performance capture. In many fields, motion capture is sometimes called motion tracking, but in filmmaking and games, motion tracking more usually refers to match moving.

In motion capture sessions, movements of one or more actors are sampled many times per second, early techniques used images from multiple cameras and calculate 3D positions, motion capture often records only the movements of the actor, not his or her visual appearance. This animation data is often mapped to a 3D model so that the model performs the same actions as the actor. This process may be contrasted to the older technique of rotoscope, such as the Ralph Bakshi 1978 The Lord of the Rings and 1981 American Pop animated films where the motion of an actor was filmed, then the film used as a guide for the frame-by-frame motion of a hand-drawn animated character.
Camera movements can also be motion captured so that a virtual camera in the scene will pan, tilt, or dolly around the stage driven by a camera operator while the actor is performing, and the motion capture system can capture the camera and props as well as the actor's performance. This allows the computer-generated characters, images and sets to have the same perspective as the video images from the camera. A computer processes the data and displays the movements of the actor, providing the desired camera positions in terms of objects in the set. Retroactively obtaining camera movement data from the captured footage is known as match moving or camera tracking.

The motion capture data is achieved through the use of an exoskeleton. Each joint is then connected to an angular encoder. The value of movement of each encoder (rotation etc. ..) is recorded by a computer that by knowing the relative position encoders (and therefore joints) can rebuild these movements on the screen using software. An offset is applied to each encoder, because it is very difficult to match exactly their position with that of the real relationship (and especially in the case of human movements).
This technique offers high precision and it has the advantage of not being influenced by external factors (such as quality or the number of cameras for optical mocap). But the catch is limited by mechanical constraints related to the implementation of the encoders and the exoskeleton .. It should be noted that the exoskeleton generally use wired connections to connect the encoders to the computer. For example, there is much more difficult to move with a fairly heavy exoskeleton and connected to a large number of simple son with small reflective spheres: the freedom of movement is rather limited. The accuracy of reproduction of the movement depends on the position encoders and modeling of the skeleton. It must match the size of the exoskeleton at each morphology. The big disadvantage comes from the coders themselves because if they are of great precision between them it can not move the object to capture in a so true. In effect, then use the methods of optical positioning to place the animation in a decor. Finally, each object to animate to need an exoskeleton over it is quite complicated to measure the interaction of several exoskeleton. Thereby bringing about a scene involving several people will be very difficult to implement.
Magnetic motion capture systems utilize sensors placed on the body to measure the low-frequency magnetic field generated by a transmitter source. The sensors and source are cabled to an electronic control unit that correlates their reported locations within the field. The electronic control units are networked with a host computer that uses a software driver to represent these positions and rotations in 3D space. Magnetic systems use 6 to 11 or more sensors per person to record body joint motion. The sensors report position and rotational information. DC Magnetic systems have issues with azimuth. If an actor is doing a push-up type posture, the system will get confused. Multiple actor magnetic setups also have problems with two or more actors in close proximity. Sensors from the different actors will interfere with each other, providing distorted results. DC Magnetic systems have very negative reactions to metal or magnetic fields in the environment. Often a stage can be built to lessen interference from metal in floors, but metal in walls, ceilings, structures, props and interference caused by electrical devices is another story. We've heard of companies resorting to setting up in a parking lot just to get away from metal. Setting up a DC magnetic system in an uncontrolled environment, such as at a trade show can be a harrowing experience.
Optical systems utilize data captured from image sensors to triangulate the 3D position of a subject between one or more cameras calibrated to provide overlapping projections. Data acquisition is traditionally implemented using special markers attached to an actor; however, more recent systems are able to generate accurate data by tracking surface features identified dynamically for each particular subject. Tracking a large number of performers or expanding the capture area is accomplished by the addition of more cameras. These systems produce data with 3 degrees of freedom for each marker, and rotational information must be inferred from the relative orientation of three or more markers; for instance shoulder, elbow and wrist markers providing the angle of the elbow. Newer hybrid systems are combining inertial sensors with optical sensors to reduce occlusion, increase the number of users and improve the ability to track without having to manually clean up data.
In recent films such as Avatar or Tron legacy, lip syncing techniques have been utilized to match realistic facial action to computer generated characters.Traditional marker based systems apply up to 350 markers to the actors face and track the marker movement with high resolution cameras. This has been used on movies such as The Polar Express and Beowulf to allow an actor such as Tom Hanks to drive the facial expressions of several different characters. Unfortunately this is relatively cumbersome and makes the actors expressions overly driven once the smoothing and filtering have taken place.

Markerless technologies use the features of the face such as nostrils, the corners of the lips and eyes, and wrinkles and then track them. This technology is much less cumbersome, and allows greater expression for the actor. These vision based approaches also have the ability to track pupil movement, eyelids, teeth occlusion by the lips and tongue, which are obvious problems in most computer animated features. Typical limitations of vision based approaches are resolution and frame rate, both of which are decreasing as issues as high speed, high resolution CMOS cameras become available from multiple sources.
The technology to capture the movements of actors seems to come down in price, the software to copy the exact movements of anyone are now in dance games and trial programs such as Face shift.
It could be possible that the reduction in price of producing a computer generated play maybe more economical then the finance of set design lighting and real effects in traditional films. As filming on a budget gets cheaper, the possibility of filming in a garage with special motion tracking for the whole actor could allow the idea of amateur film-makers to have the same production quality as Hollywood. The future of film making may use computer generated characters as a economical form of prosthetics, but there still a lot to be said about using real sets and locations. It will be interesting to see if the film industry will be favoring one method over the majority of films in the future...