Creating an interactive 3d avatar

Dear All,
Need help on a new project I am working on.
At a high level, our requirements are..

1. Create a 3d avatar which will be projected on a big screen.

2. The avatar should support motion capture - for eg. look at the person standing in front on the screen and look in the direction which the person moves. We are planning to use Kinect for this purpose.

3. We will have a speech recognition system which will capture what the person says and will respond with some answers.

4. The answer text will be streamed to the avatar using a web service.

5. The avatar should speak out the answer text with reasonable lip sync and animation.

We are in the process of evaluation various tools/technologies which can be used to accomplish these requirements.

Is FaceFx the right tool? Any help/suggestion will be much appreciated.

Permalink
Sounds like an interesting project, and I think FaceFX would be a good tool to have.

Have you seen our TTS Demo (look at the blog section on the left)? Be sure to check out the pandora bot version. Like your project, it has an avatar that responds to queries. You are adding voice recognition to replace the text input, and a mocap system that will use Kinnect. You will also need to license a TTS solution.

Keep in mind that FaceFX Studio Professional does not let you analyze audio programatically, so you will need to contact us at info (at) oc3ent.com to inquire about licensing the FaceFX Analysis Engine for the project.