It's a bit lengthy, but the stuff he mentions is invaluable, some of it may be something you already, but there may be something new... thanks for the insights Mr. Maestri
Animating lip sync can really frighten the beginning animator, and rightly so because it is one of the most difficult techniques for an animator to master. Live-action people have it easy; they just point the cameras at the actors and ask them to speak.
Facial animation is a lot more than just moving the mouth. When a character speaks dialogue, the shape and position of the mouth is perhaps 10–20% of the total effect. Much more important to the audience is the movement of the body and head, as well as the expression in the face and eyes. For the purposes of this article, however, we will start the process with the mouth.
The Eight Basic Mouth Positions
You must first understand how the mouth moves when it speaks. Dozens of different mouth shapes are made during the course of normal speech. Animators usually boil these down to a handful of standard shapes that are used repeatedly. Depending on the style of animation, some animators get away with as few as three or four shapes, and some may use dozens. For most situations, you can get away with approximately eight basic mouth positions. These eight positions usually provide adequate coverage and give you the ability to animate most dialogue effectively.
As Figure 1 shows, Position A is the closed mouth used for consonants made by the lips, specifically the M, B, and P sounds. Typically this can be made by pushing the open jaw pose into negative territory to close the mouth. In this position, the lips are usually their normal width. For added realism, you could mix in an additional shape, to get the lips slightly pursed, for sounds following an "ooo" sound, as in the word room.
Figure 1 Position A is the closed mouth used for consonants made by the lips, specifically the M, B, and P sounds.
Position B, shown in Figure 2, has the mouth open with the teeth closed. This position is a common shape and is used for consonants made within the mouth, specifically sounds made by C, D, G, K, N, R, S, TH, Y, and Z. All these sounds can also be made with the teeth slightly open, particularly in fast speech.
Figure 2 Position B has the mouth open with the teeth closed.
Position C, shown in Figure 3, is used for the wide-open vowels, such as A and I. It is essentially the same as the fundamental shape for an open jaw.
Figure 3 Position C is used for the wide-open vowels, such as A and I.
As Figure 4 shows, Position D is used primarily for the vowel E, but it can also be used on occasion for C, K, or N during fast speech.
Figure 4 Position D is used primarily for the vowel E, but it can also be used for C, K, or N during fast speech.
Position E has the mouth wide open in an elliptical shape, as shown in Figure 5. This is the position used for the vowel O, as in the word flow. It is created by mixing together an open jaw and the "oooh" sound. Sometimes, particularly when the sound is at the end of a word, you can overlap this shape with the one in Position F to close the mouth.
Figure 5 Position E has the mouth wide open in an elliptical shape
Position E is the position used for the vowel O, as in the word flow. Position F, shown in Figure 6, has the mouth smaller but more pursed. Position F is used for the "oooo" sound, as in food, and for the vowel U. It is one of the fundamental mouth shapes.
Figure 6 Position F is used for the "oooo" sound, as in food
Figure 7 shows Position G, which has the mouth wide open with the tongue against the teeth. This position is reserved for the letter L. It can also be used for D or TH sounds, particularly when preceded by A or I. It is essentially an open jaw with the tongue moved up against the top teeth. If the speech is particularly rapid, this shape may not be necessary, and you can substitute Position B.
Figure 7 Position G has the mouth wide open with the tongue against the teeth
Position G is reserved for the letter L. It can also be used for D or TH sounds Position H, shown in Figure 8, has the bottom lip tucked under the teeth to make the sound of the letters F or V. In highly pronounced speech, this shape is necessary, but the shape could also be replaced with Position B for more casual or rapid speech. This shape is one of the extra shapes modeled previously.
Figure 8 Position H has the bottom lip tucked under the teeth to make the sound of the letters F or V.
(apparently some of page 2 went missing, sorry)
Reading the Track
Now that you understand the basic mouth positions, it's time to break down the track. If you have animator's exposure sheet paper, use it. Otherwise, get a pad of lined paper on which to write your track, using one line per frame. (If you prefer, you can create a spreadsheet for this purpose and do it all digitally.) Load the dialogue into a sound-editing program.
Match your sound-editing program's timebase to the timebase that you're animating—30, 25, or 24 frames per second, for example. After your timebase is set, selecting a snippet of dialogue should enable you to listen to the snippet and read its exact length on the editor's data window. The visual readout of the dialogue gives you clues as to where the words start and stop. Work your way through the track and write down each phoneme as it occurs on your exposure sheet, frame by frame.
Some packages give you the capability to play back audio in sync with the animation. This feature is particularly helpful because you may be able to skip the step of reading the track and simply eyeball the sync. However, this will not be as accurate as a frame-by-frame read.
Once the track is read, you're ready to begin animating the dialogue. Dialogue is slightly different from lip sync because lip sync simply involves the lips. Dialogue, however, involves the whole character. When animating characters, be sure to get the character's entire body into the acting.
With the body so important to dialogue, one of the questions that you might have is whether to animate the mouth or the body first. Some animators simply do the mouth first, just to get the tedious task out of the way. It also is easier to get the mouth animated first on a still head rather than one that is moving. Other animators like to concentrate on the body first and then get the mouth. Both approaches work equally as well, and because you can always go back and tweak the body and the lips independently, the line between the two methods is pretty much a gray area.
Animating the Mouth
If the track was read properly, the phonemes and their location are pretty much known. In the track we just read, for example, we might know that there is a gasp at Frame 22 and the word Oh at Frame 42. One important trick that will work to your benefit is to always try to open the mouth quickly and close it slowly.
Vowels are those points in speech where the mouth opens. When animating a vowel, you need two positions. The first position is the accent pose, when the vowel is first uttered. The second position is the cushion pose, which happens toward the middle to the end of the vowel sound. The accent usually has the mouth open wider than the cushion. One good way to do this is to animate the jaw so that it closes slightly as the vowel progresses. For fast vowels of only two frames, this may not be much of an issue, but this rule applies to anything above four frames.
Consonants are those points where the mouth closes. With the possible exception of a long M, F, or V sound, most consonants are only a few frames in length; some can be less than one frame long. With this in mind, make sure that you leave each position on the screen long enough for the audience to read them. Consonants must be on the screen for at least two frames to be read. If the consonant is too short, steal time from a vowel or combine two consonants into one.
Eyes and Dialogue
Once the basic lip sync is accomplished, the eyes are next on the list. When animating eyes with dialogue, be sure you understand where the character needs to be looking. Ask yourself the question, "Who is the character talking to?" Try to keep the eyes focused on the subject at hand.
Head Motion and Dialogue
The head moves quite a bit when people talk. The head will bob, nod, and shake to emphasize certain words in a line of dialogue. When speaking loud sounds, the head usually raises to help open the throat. This is helpful when animating the loud sounds or accents in speech.
When animating an accent where the head raises up, it is always a good idea to anticipate the motion by lowering the head three or four frames before the accent and then popping up the head on the accented syllable. This is also known as a head bob and is usually accompanied by a blink. To get more action into the head bob, you can also get the body into the action. As the head moves down in anticipation of the accent, raise the shoulders a bit. As the head pops up, lower the shoulders.
Body Language and Dialogue
When talking, many people use their hands to clarify and emphasize the major points of their speech. Getting this part of the animation correct is a lesson in acting. If you want to see how not to animate, watch some really nervous or first-time actors. They usually are very self-conscious, stuff their hands in their pockets, wring them nervously, or hang their hands loose at their sides.
In real life, body language precedes the dialogue by anywhere from a few frames to as many as 20. Generally, a slow, dimwitted character has more time between his gestures and his dialogue than a sharp, quick character. Speedy Gonzales has considerably less of a lead time on his gestures than Forrest Gump. Someone giving a long, boring speech will be much slower than a fire-and-brimstone evangelist.
You should also make an effort to ensure that your gestures fit the dialogue smoothly. The first gesture every animator learns is the ubiquitous finger point for emphasis, followed soon after by the fist pounding into the palm. These gestures certainly have their place, but within a much larger palette. Simply watching people in their natural habitat is always your best reference.