Mouther

Mouther is a MEL-based system of my own devising for saving, blending, and keying skeletal poses. It was written with the aim of overcoming Doom3's lack of blend shapes - all faces are rigged with joints and painted weights, faces included. In order to animate facial expressions and lips to spoken dialogue in a timely fashion I needed some way to replicate blend shape-like behavior in Maya that could then be baked out to ordinary keyframes and fed to Doom3's importers.

In order to store the translate X Y & Z and rotate X Y & Z values necessary per joint, per shape, I would need multi-directional arrays. MEL in its infinite usefulness declines to provide any. Variables also wouldn't persist across file saves. Mouther works by creating new attributes on all the joints under its domain, one attribute for each saved shape. Those attributes are used to store translate X Y & Z and rotate X Y & Z for the appropriate shape in a 6-value array, relative to the bind pose which is backed up in its own attribute. Then, when a pose is set, the MEL script simply grabs the attributes off of each joint and applies the transformations.

[ Download mouther.mel ]

Instructions For Use

Mouther has to have a list of joints to work on before anything will work, which I refer to as the 'active' joints. The first order of business is setting the active list, by selecting all the joints you'll be conceivably moving when animating the mouth and pressing "Set Active." A confirmation window comes up because if you press this with nothing selected, it will empty the active list. If you miss a joint or grab one extra you can select it in the Mouther window list and press either Add or Remove.

Note: If you create a bunch of shapes on the active joints, then remove one from the list, the attributes remain saved on that joint, so you can always add it back to the list at a later time. However, if you add joints to the list after saving a bunch of shapes, the added joints won't have any shape-specific attributes saved and you'll get pink errors. Therefore, it's best to set your active joint list correctly first and then just leave the damn thing alone.

Now you're ready to start setting shapes. This is the important bit - the first pose you save has to be your base pose, and it must be named "X" (case sensitive). Note that this pose doesn't necessarily have to be the head's bind pose - if you bound the head with the mouth open you can shut it before saving this shape. X is the shape that Mouther always resets the face to when applying a new shape, because all transforms are stored relative to this shape. I reccomend making this pose the plain, expressionless, closed-mouth pose.

Move and rotate all active joints however you want, type a "X" into the upper right box, and press "Save New Shape."

Note: If you save a shape with a name that's already used, that shape will be overwritten without confirmation.

From this point on, you're free to start saving shapes. You needn't keep the joints selected - pressing "Save New Shape" after each pose is sufficient. You also don't need to reset the mouth to the X pose before posing each shape.

Note: All poses have to have single characters for names - this is for your own good and I'll explain it soon. You get 26 lowercase letters and 25 uppercase, and I haven't tested digits but they probably work too, so that's 61 shape names just for mouth phonemes, so don't complain.

Double-clicking on a pose name in the list will immediately set all active joints to that pose. The "Assume Shape" button, however, goes by what's typed into the box, but there's a good reason. One of the advantages of Mouther is that it can mix shapes on the fly. If you type a bunch of characters into the box and press "Assume Shape," Mouther will blend them all proportionate to the number of characters in the string. Typing "aam" into the box and pressing "Assume Shape" will move the active joints to 1/3 of a morph to "m" from "a."

Note: You can save new shapes at any time, so if you're halfway through a line of dialogue and need a funky mouth shape you hadn't anticipated, you can find an unkeyed frame, assume the bind pose, pose your new shape and save it, and continue animating.

Once you've got all your shapes set, you're ready to start animating. You have two options for animation. One is to advance along the timeline manually, assuming shapes and pressing "Key Active" to set keyframes for all the active joints. This is the more methodical approach. Mouther gives you a different option, the character string. Lipsync can often require keyframes nearly every frame, and the character string capitalizes on that. The string is simply a long list of frame shapes separated by commas and/or periods. They work the same as "Assume Shape" - one character or several defines the pose for a frame. A "-" in place of any letters will skip that frame and set no key on it, and the animation will blend from the previous to the next. This may not seem very easy at first but once you get the hang of the timing of your dialogue and what shapes are effective for what sounds, you'll be able to bang these strings out in very little time.

To test your animation, the controls are pretty simple - "Bake Keyframes" will march across the timeline and key all your shapes, starting at whatever number is typed into "Start Frame." The "Bulldoze" checkbox instructs Mouther to reset each frame to the base pose before keying it, thus wiping out any animation you might already have. If unchecked, Mouther will apply the relative transformations to wherever the joints are on that frame, which can lead to unpredictable facial explosions if not carefully controlled. Note also that if there is a keyframe set on a frame for which Mouther has a "-", it won't be touched, so you should hit "Select Active" and delete any old keyframes before doing a test.

Another problem with leaving "Bulldoze" off: if keys are already set and animation curves already exist, the new keys won't be set properly. Mouther simply advances currentTime as it sets its keys, so if there are keys set in the future, the face will already be morphing from any previously set mouth-shapes to the new one, and a relative morph from that shape won't produce the desired effect. I already know this sucks, and I'm planning on adding a second animation channel for overall facial expression, with a separate active joint list and a separate set of shapes for the entire face that would be blended correctly with lip sync when keys are baked.

"Load Audio" is a shortcut to putting sound in the animation - find the wav file and it will show up in the timeline starting at "Start Frame." If you want to move it to a different start time, change the number and load the same sound again.