Have you ever run the automated FaceFX analysis, only to get less than optimal lip sync? Here are a few simple tricks to make sure you’re getting optimal alignment every time.
1. Check that your text transcription exactly matches the recorded audio. I realize that sounds simplistic, but sometimes voice actors will take a certain level of "artistic freedom" while recording. Deviations from the script can wreak havoc on your audio analysis if you aren't aware of it. If you see poor lip sync, take a couple of seconds to double check that the text transcription exactly matches what was said.
2. Phonetic spellings sometimes help with unusual words. For example, FaceFX does better automatically analyzing the word "FaceFX" when it's spelled in the text file as "Face Eff Ex". This removes ambiguities and lets the audio analyzer do a better job of phoneme alignment. Try this if you have words that you wouldn’t find in a standard English dictionary.
3. Use the original recorded audio to process in FaceFX. Any effects, like pitch shifts, echo, or radio static, will make it harder for the FaceFX analysis to process the audio. This one takes a little planning ahead, but if you save a copy of the unprocessed audio cut to the same length as your final processed clip, you can process that original audio while generating your animations, and then replace the audio with your processed clip once it has been analyzed. This can make a huge difference for characters with artificially deep or high voices.
4. The text file should be encoded in either ASCII or UTF-8. While FaceFX tries to warn on bad text encodings, sometimes they'll slip through and cause a lot of problems with the analysis. I use Notepad++ when I need to check and convert text file encodings. Saving UTF-8 with a byte order marker will eliminate any possibility that poor analysis was caused by the text file encoding.
This is the first in a series of FaceFX tips that I'll be posting each Tuesday. Some of these tips will be information found in our documentation that I think might be a helpful reminder. Some will be very specific and may only apply to a few users. Have a question or comment about today's tip? Let me know on Twitter: @facialanimation