commit a3f91c2 ·
The trade-off triangle: consistency, performance, lip-sync
Why AI character video keeps giving you two out of three, and how to get the third.
Every AI character video is judged on three things at once:
- Consistency. Is it the same face, the same age, the same glasses, in every shot?
- Performance. Does he act? Pauses, eyebrows, the half-smile before the punchline.
- Lip-sync. Do the lips match the words, closely enough that nobody notices?
There is a fourth one hiding inside the second: body language in sync with the voice. A shrug that lands a beat late kills a joke as surely as a bad cut.
What kept happening
With the Haj, the tools kept handing back two out of three.
- When the character stayed consistent, the performance went flat: a good-looking statue reading lines.
- When the performance came alive, it was often too alive: big, theatrical gestures that don't belong to a dry, sarcastic old man.
- When the performance was tuned down, the lip movement drifted from the audio.
- When the lips were fixed, the body language fell out of sync with the voice.
Why it happens
Each of these models is optimising for something slightly different. Avatar systems protect the face and the mouth. Expressive video models protect motion. Nobody's default is "subtle, deadpan, in sync".
What finally worked
Not a better model. A better input. Once the performance itself came from a real person on camera, with the exact timing, the exact shrug and the exact mouth shapes, the model only had to transfer it onto the Haj instead of inventing it. Consistency came from one strong reference image; performance and lip-sync came from the driving video.
That shift is the subject of the next note: the driving video matters more than the prompt.