Skip to content
Lab

commit a3f91c2 ·

The trade-off triangle: consistency, performance, lip-sync

Why AI character video keeps giving you two out of three, and how to get the third.


Every AI character video is judged on three things at once:

  1. Consistency. Is it the same face, the same age, the same glasses, in every shot?
  2. Performance. Does he act? Pauses, eyebrows, the half-smile before the punchline.
  3. Lip-sync. Do the lips match the words, closely enough that nobody notices?

There is a fourth one hiding inside the second: body language in sync with the voice. A shrug that lands a beat late kills a joke as surely as a bad cut.

What kept happening

With the Haj, the tools kept handing back two out of three.

  • When the character stayed consistent, the performance went flat: a good-looking statue reading lines.
  • When the performance came alive, it was often too alive: big, theatrical gestures that don't belong to a dry, sarcastic old man.
  • When the performance was tuned down, the lip movement drifted from the audio.
  • When the lips were fixed, the body language fell out of sync with the voice.

Why it happens

Each of these models is optimising for something slightly different. Avatar systems protect the face and the mouth. Expressive video models protect motion. Nobody's default is "subtle, deadpan, in sync".

What finally worked

Not a better model. A better input. Once the performance itself came from a real person on camera, with the exact timing, the exact shrug and the exact mouth shapes, the model only had to transfer it onto the Haj instead of inventing it. Consistency came from one strong reference image; performance and lip-sync came from the driving video.

That shift is the subject of the next note: the driving video matters more than the prompt.

Next note7be0d48Garbage in, garbage out: the driving video matters more than the prompt

HAJFLIX://contact

Talk to the Haj

Pick the way you like. WhatsApp is the fastest; he answers between naps.

«عندك شغل؟ الحاج فاضي… لحدي ما يجي الزبون الجاي.»