AudioRig

Exact versions required:

• CUDA Toolkit 12.9
• TensorRT 10.13.3.9

Other versions (newer or older) are not accepted — AudioRig checks the exact version and will tell you if something doesn't match.

2.1 Install CUDA 12.9

2.2 Install TensorRT 10.13.3.9

How AudioRig finds these tools: on startup it checks, in order, the environment variables CUDA_PATH / TENSORRT_ROOT_DIR, then the standard locations shown above. If anything is missing or the version is wrong, the add-on shows a clear message telling you exactly what to fix.

6.1 Input

A note on rig quality and expressive values — read before your first bake. A2F is a state-of-the-art model: its output is highly expressive and closely mirrors real human articulation, including lip compression on consonants like M, B and P. On a rig whose shape keys were sculpted without proper constraints or corrective shapes, that expressiveness can expose rigging weaknesses — most notoriously on mouthClose, where baked values above roughly 0.5 are excessive on many imperfect rigs and make the upper and lower lips intersect each other.

The honest fix is at the source: build your ARKit shape keys — mouthClose above all, but ideally all of them — with solid constraints and corrective techniques (sculpt-checked extremes, enough lip-loop topology, symmetric validation, testing against deliberate maximum poses). A well-built shape key survives A2F's full expressive range beautifully. As an in-add-on remedy while you improve the rig, Override Per-Blendshape Gain (section 6.4) lets you scale an individual channel down — capping a runaway mouthClose so its peaks land around 0.5 instead of clipping through the other lip.

6.2 Models

Shows the status of the AI engines ("Engine ready" / "Engine not built"). One model always runs; a second one is optional:

• Use Audio2Emotion (off by default) — adds an emotional layer on top of the lip sync, read from the tone of the voice. The performance reads much more alive on longer takes. Costs a little extra generation time. Leave it off while validating basic lip sync.
• Advanced (collapsed) — only for special cases:
  • A2F / A2E Model Override — point to a model stored outside the add-on. Empty means "use the bundled model", which is what you want almost always.
  • Use Config File — drive all parameters from an external JSON file instead of the panel (useful to pin an exact setup to a shot or share it with a teammate). While enabled, the Parameters section is greyed out.

6.3 Inference

How the animation is computed:

• FPS — frames per second of the generated animation. Match your scene frame rate or picture and sound will drift apart. 30 is enough for most dialogue; 60 resolves fast consonants more crisply.
• Identity — the built-in performance style: Claire (most neutral), James (moderate), Mark (most pronounced articulation). This changes the character of the motion, never your mesh. After changing it, press Load Model Defaults again (see 6.4).
• Solver — where part of the computation runs. GPU is recommended; CPU is a slower fallback for when the GPU is saturated by other work.
• Constant Noise (on by default) — keep it on: identical audio then gives identical output and the face doesn't shimmer between frames. Turn it off only for subtle variation, at the cost of flicker you may need to smooth out.
• CUDA Device — which GPU to use. 0 is correct on single-GPU machines.

Extra Channels (all off by default; enable only if your rig uses them):

• Output Tongue Poses — additionally outputs 16 tongue poses. Only useful if your rig has shape keys with those names.
• Output Head Rotation — additionally outputs the head rotation inferred from the delivery, stored as custom properties you can drive a head bone with.
• Output Emotions — additionally outputs 10 emotion values per frame as custom properties, so you can drive your own expression layers or correctives. Requires Use Audio2Emotion for meaningful values.

6.4 Parameters

Every model ships with values tuned by its authors — you don't need to touch anything here for a good result. The recommended workflow is:

• Override Face Parameters — smoothing (calms jitter; too much softens consonants), strength (how far the face travels from neutral), face mask blending, skin strength, eyelid/lip offsets, and procedural eye life: blink amount, gaze amplitude, and the small darting eye movements (saccades) that keep eyes from looking dead.
• Override Tongue Parameters — tongue strength and rest-position offsets. Reach for these if the tongue pokes through teeth or the floor of the mouth on your particular head.
• Override Emotion Parameters — how strongly emotion colours the performance, how sharply the winning emotion stands out, how many emotions may be active at once, how fast emotion may change between frames, and an optional preferred emotion pinned at a chosen strength (for when the delivery is flat but the scene needs a mood). Requires Audio2Emotion.
• Override Per-Blendshape Gain — a per-channel multiplier and offset applied to each of the 52 channels just before baking. This is the surgical fix for rig-specific problems: one shape that overshoots, an asymmetric smile, a jaw that opens too far, or a mouthClose that regularly climbs past ~0.5 and makes the lips intersect. Press Init ARKit 52 to fill the list with neutral values (multiplier 1, offset 0), then touch only the entries that misbehave.

6.5 Bake

Where the animation lands:

• Action — base name for the created animation. One Action is created per animated mesh, named <base>_<object name> (e.g. AudioRig_Head, AudioRig_Eyes). Change the base name to keep several takes side by side instead of overwriting the previous one.
• Start Frame — the scene frame where the first generated frame lands. Set it where your audio sits on the timeline.
• Threshold — discards keyframes that barely move. 0 keeps every frame (most faithful, heaviest curves). A small value like 0.001–0.01 thins the curves, making them far easier to edit by hand and lighter to scrub. First and last frames are always kept.

6.6 Run

• Build / Rebuild TensorRT Engines — the one-time preparation from section 4. Red means action needed; green means ready.
• Generate + Bake — the main event. AudioRig reads your audio, computes the facial performance, and bakes it as keyframes onto your character. Blender stays responsive while it works, and you can cancel with Esc. Re-running is safe: curves for the same channels are replaced, not layered.

If the button is greyed out, the panel lists exactly what's missing (engines not built, no audio picked, or no target mesh picked).

6.7 Animation Data — Restore, Export, Import

At the bottom of every animation tab (A2F, UniTalker, SAiD, Wav2ArKit, UnLip — never on TTS or GNM) sits the Animation Data panel with three tools, each with its own help toggle:

• Restore Previous Animation — every time a Generate + Bake replaces existing curves, the curves (and static values) it is about to overwrite are saved automatically to a temporary file. This button puts that previous state back: your pre-generation animation returns exactly as it was. Handy when a new take came out worse than the last one. It stays greyed out until a generation has actually replaced something in this blend file, and it always reflects the most recent generation.
• Export Animation — saves the currently baked animation to a portable .json file, keyed by ARKit channel names (jawOpen, mouthSmileLeft, …) and sampled frame by frame. Because it stores channel names instead of scene references, the same file can be re-applied to this character, to a different one, or read by external tools.
• Import Animation — bakes a previously exported .json onto the mesh picked in this tab (respecting Include Children / Scan Whole Scene), as dense keyframes. The scene's frame rate and range are set from the file's metadata so playback stays in sync. The action is named after this tab's Action field.

When would you use these? Restore: a new take came out worse and you want the previous performance back without re-generating. Export/Import: keep takes as lightweight files outside the blend, move an animation to a different character (any mesh with proper ARKit shape keys), or archive performances without saving heavy .blend files.

The GNM tab creates a full human head from scratch — skin, eyeballs, teeth and tongue included — with a rig you can pose, and hands it to any of the three animation tabs when you're done. It's the fastest way to get an animatable character when you don't have one.

How to think about it: a head starts as a neutral "average face". You then shape who they are (Identity), what expression they make (Expression), how they're posed (Pose), and you can sculpt specific regions (Customize). Everything updates live in the viewport — there is no "apply" button.

8.1 Getting Started

• Import Head — creates the head in your scene, with its control rig (yellow handles: neck, head and an eye-aim control) and clean materials per part (skin, teeth, tongue, eyes). Already have one? The button becomes Rebuild Head, which resets everything to the neutral face.
• Reset (circular arrow) — returns all parameters to the neutral face.
• Randomize Face — one click = a brand-new person. The fastest way to explore.

8.2 Randomization Settings

• Seed — a number that identifies "this exact face". Same seed + same choices = same face, every time. The dice button rolls a new one.
• Auto-Randomize Seed (on by default) — every randomize click rolls a fresh seed, so each press looks different. Turn it off to reproduce results.
• Randomize Intensity — how strongly each randomize deviates from the current face. Lower it for subtle refinement passes instead of full re-rolls.
• Use Semantic Sampler (on by default) — generate realistic, label-guided faces. Off = plain random noise (a fallback that always works, less realistic).

8.3 Presets

Save faces you like and reuse them later (or share them):

• New saves the current face as a named preset (with a thumbnail).
• Load / Overwrite / Delete manage the selected preset in the list.
• Refresh rescans the preset folder.
• Export Bundle / Import Bundle pack all presets into a single .zip for backup or sharing, and restore them on another machine.

8.4 Mode Switch: 1. Generate / 2. Customize

• Generate — build the face with guided controls and sliders (sections 8.5–8.7).
• Customize — sculpt one facial region at a time (section 8.8).

Both work together: changes made in Generate apply on top of whatever you sculpted in Customize.

8.5 Identity (who they are)

• Single mode — pick a Gender and an Ethnicity (or leave them on Any for a surprise), then Resample Identity for a new variation. Changing the dropdowns applies immediately.
• Blend mode — mix continuously: Female ↔ Male weight, and two ethnicities with a blend weight. The face updates live while you drag.
• Advanced (253 components) — the full manual control deck. Sliders are ordered by effect: early ones move broad shapes, late ones add subtle detail. To keep the list manageable, each region shows the most impactful sliders first, with a Load more (x/N) button for the rest and Show less to collapse them again.

8.6 Expression (what they're doing)

• Single mode — pick an expression label (Happy, Surprise, Disgust, Wink, Pucker, Tongue Center…) and Resample Expression for a new take on it.
• Blend mode — pick two expressions (A and B) and crossfade with the Blend Weight slider. Resample Blend Endpoints rolls fresh variations of A and B.
• Mirror Eyes — copy the left eye's expression onto the right, or vice-versa. Perfect symmetry in one click.
• Advanced (383 components) — the full expression deck, paginated exactly like Identity's advanced section.

8.7 Pose

• Use Rig (on by default) — pose the head by grabbing the yellow circle handles directly in the viewport: neck, head, and the eye-aim control that both eyes follow. The sliders below are greyed out in this mode.
• Eye Aim (on by default) — both eyes look at the eye-aim control; drag it to direct the gaze. Turn it off to rotate each eyeball manually with the sliders.
• With Use Rig off, the Neck / Head / Left Eye / Right Eye Rotation and Translation sliders pose the head numerically.

8.8 Region Sculpt (Customize mode)

Shape one region at a time without disturbing the rest of the face:

• Region — Eyes, Brows, Nose, Mouth, Jaw, Rest (everything else), or All to re-roll the whole head region by region in one click.
• Gender / Ethnicity — what the randomizer samples from for that region.
• Falloff — how softly the region's edge blends into the rest of the face. 0 is a hard edge; higher is softer. You can drag it after a randomize and watch the boundary re-blend live.
• Include Tongue (off by default) — also randomize the tongue when working on the Mouth region.
• Auto Freeze (on by default) — lock in each region right after randomizing, so the next region doesn't undo it. Turn it off to preview first, then commit manually with Freeze Masked.
• Randomize Region — re-roll just the selected region.
• Freeze Masked — commit the current preview as the new baseline.
• Discard Sculpt — drop all sculpted work and return to the pure parametric face.

Sculpted regions are real mesh edits, so the Identity/Expression sliders describe a slightly different face than what's on screen — the panel shows a notice when that's the case. Slider changes still apply on top of your sculpt; Reset or loading a preset discards it.

8.9 Sending the Head to the Animation Tabs

• Set as A2F Target — makes the GNM head the target of the A2F tab in one click. Remember: the head is born without ARKit shape keys, so the coverage counter will read 0/52 until you add them — the quickest way is running Faceit's landmark-based shape key generation on it (see section 5).

Full walkthrough: generate a head with GNM, rig it with Faceit, and animate it from audio.

Credit where credit is due. The GNM tab was inspired by the free GNM Importer add-on by Nathan Dickson. We loved the idea and built our own take that simplifies everything — installation and assets included — into the single-click tab described above. If you want more advanced control over Google's GNM model, we genuinely recommend downloading that free add-on.

SAiD takes a different approach: instead of predicting the whole face at once, it uses a diffusion model — the same family of techniques behind modern image generators — focused entirely on the 32 mouth-area channels: jaw, lips, cheeks and nose sneer. The result is exceptionally natural mouth movement, and because the mouth is what sells a line of dialogue, it can make an otherwise static character feel remarkably alive. It also runs on the CPU, with no GPU requirement.

Two things to know before using it:

• It only drives the 32 mouth-area channels. Eyes and brows are not part of this model and are left untouched — pair it with your own blinks, or use the A2F or UniTalker tabs when you need the upper face too.
• It's the most demanding of the two CPU engines: it refines its result over many small steps (that's the Inference Steps parameter), so a take takes a few seconds rather than under a second. Still very reasonable — just the slowest of the three engines.
• Keep SAiD takes around a minute or less. The model processes a take as one continuous piece, and on ~6 GB GPUs a DirectML run of much longer than that runs out of video memory. AudioRig handles this for you: takes that would not fit in your GPU's free VRAM are computed on the CPU automatically — same result, just slower — and takes that fit use the GPU (roughly 10× faster). Prefer the CPU anyway? The Use CPU checkbox in the Run section (section 10.4) skips the GPU entirely, immune to VRAM limits at any audio length.

10.1 Input

• Audio, Target, Include Children, Scan Whole Scene — identical to the UniTalker tab (section 9.1).

The coverage counter here reads "N/32 SAiD channels present (of 52 ARKit)": how many of the 32 mouth-area shape keys exist on your character. 32/32 is perfect; a head with only the basic mouth shapes still works.

10.2 Sampler

• Inference Steps (default 50) — how many refinement passes the diffusion model makes. More steps = slightly cleaner motion but longer waits. 50 is the sweet spot; 100 doubles the time for a marginal gain.
• Guidance Scale (default 2.0) — how strongly the audio steers the animation. Low values follow the voice loosely (softer, sometimes mushy); high values exaggerate the mouth motion. The paper's best value is 2.0.
• Seed — the model starts from random noise, so the same audio can produce different (all plausible) performances. A fixed seed makes the result reproducible: same audio + same seed + same settings = exactly the same animation. The dice button rolls a new variation.

10.3 Inference

• FPS — frames per second of the generated animation. Match your scene frame rate. This tab defaults to 60, the rate where SAiD's mouth motion reads most naturally.

10.4 Bake

• Action, Start Frame, Threshold — identical in behaviour to the A2F Bake section (6.5).

10.5 Run

• Use CPU (off by default) — skips the GPU entirely: the whole generation runs on your processor. Slower (the GPU path is roughly 10× faster when it fits), but immune to VRAM limits: on ~6 GB GPUs even 40-second takes can exhaust video memory with DirectML, so if you generate long dialogue regularly — or just want the predictable, always-works path — turn this on. With it off (default), SAiD runs GPU-first and falls back to the CPU automatically when the take does not fit your free VRAM.
• Generate + Bake — computes the mouth performance on your CPU and bakes it as keyframes. A typical dialogue take takes a few seconds at the default 50 steps. Same seed gives the same result; re-running is safe.

Below the Run section sits the shared Animation Data panel — Restore Previous Animation, Export Animation, Import Animation — described in section 6.7. It appears on every animation tab and never on TTS or GNM.