FaceFX + MetaHuman?

We've recently started looking at beginning to integrate MetaHuman assets into future character pipelines. Part of my tasking is to see if we can get a MetaHuman face through FaceFX.

Right at the beginning, just a simple (MetaHuman -> Maya -> FBX -> FaceFX) won't work as Ogre has a 256 joint limitation, and the MetaHuman face has 700+ bones. My FBX import gets a lot of the face and jaw bones, but nothing much in the mouth area.

So my first question would be, are there any plans for potentially officially supporting a MetaHuman workflow? There are some potential benefits here in that the facial control rig for MH has all of its controls standardized into a 0-1 or -1 to 0 to 1 value space. One being that if those were the curves that drive animation using FaceFX, face animations would largely be transferrable between characters.

Anyway, I am pursuing some ideas about how I can hack it through with blendshape representations and control rig bones, but if there's going to be some type of official solution, either in software or techniques, I would love to know. The MetaHuman system looks really nice, and I think it'll be worth pursuing even if it's only what I can hack together.

Thanks!

Permalink

MetaHumans is a technology package that includes art assets and plugins/code to drive the assets. FaceFX lacks the proprietary code to drive the art assets, so we can’t support metahumans in FaceFX.

That said, you can use FaceFX to create animation curves that drive Metahuman targets in UE4. The process is:

1. Create a root bone (named "root") in Maya and export an FBX with it. Do not export an animation.

2. Create an actor in FaceFX, and create FxAttribute nodes with the following naming format:

CTRL_expressions_eyeLookRightR
CTRL_expressions_browDownL
CTRL_expressions_jawOpen

3. Animate the FxAttribute how you want

4. Export an FBX from FaceFX, specifying your FBX from step 1 as the input file.

5. Drag your FBX from step 4 onto UE4, and select "Face_Archetype_Skeleton" as the Skeleton under the Mesh section at the top

6. Double-click on the newly created Animation Sequence and watch the metahuman targets animate.

To play one of the above animation sequences in sequencer, you need to first delete the Face_ControlBoard_CtrlRig object from sequencer after adding the Actor to Sequencer. Then you can add the animation sequence to the Face object and it will play. To "bake" the animation sequence, right-click the Face object and select "Bake Animation Sequence". This will create large animation files, but this is due to how metahumans is implemented in UE4 and Epic is aware of the issue.

There is currently no way to use the FaceFX Runtime or the FaceFX plugin in UE4 to drive metahumans content directly.