Ends Today: Save up to 95% on the Time Travel Mega Bundle!

This plugin uses voice recognition engine to generate lip-sync animation from SoundWave assets or PCM audio data. Animation is saved as curves in data assets and can be played in runtime together with audio. This approach allows to achieve well-looking lip-sync animation easily without subtitles.


Video #1

Video #2: Lip-sync Curves to Anim Seqiences

Video #3: Generate Pose Asset with visemes (CC3)

Video #4: MetaHuman Setup (tutorial)

Video #5: Pose asset for MetaHuman from default visemes


New tutorial for MetaHuman.

New tutorial for CC3/CC4.


Unlike text-to-lipsync solution, this is true voice-to-lipsync plugin. You don't need subtitles to get lips animatied, and resulted animation is much closer to speech then in case of subtitles-based solution.


Lip-sync can be generated in runtime, but not in real-time.


Supported language: English. Partially supported: Russian, Chinese and Italian (tutorial).

Technical Details

Features:

  • subtitles aren't needed;
  • can process procedural SoundWave objects created in runtime (Windows only);
  • can generate Anim Sequence assets with lip-sync in curves;
  • lip-sync for animation curves or morph targets;
  • asynchronous audio recognition and building lip-sync;
  • (beta!) create lip-sync in runtime on PC and Android using remote server. 

Code Modules:

  • YnnkVoiceLipsync (Runtime)
  • YnnkVoiceLipsyncUncooked (UncookedOnly) 

Number of Blueprints: 0

Number of C++ Classes: 8

Network Replicated: No

Supported Development Platforms: Windows x64

Supported Target Build Platforms: Windows x64 (Android - without runtime features)

Documentation: [Doc]

Example Project: [5.0-5,1] [4.27] [MetaHuman UE5.1 with Enhancer Plugin]

Executable Demo: [ZIP]

Enhancer Plugin: [5.1 + 5.2]

YNK
Yuri N Kalinin
All Assets by Author
86.28 
Platforms Windows 64-bit, HoloLens 2, Android
UE Versions 4.26 - 4.27, 5.0 - 5.2
Tags CHARACTER ANIMATION, LIP-SYNC
Release date 23.06.2021

Similar products

Image
Sign In
Image
Sign Up
Image
Recovery