> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs-dev.ltx.io/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs-dev.ltx.io/_mcp/server.

# Video-to-Audio (Foley)

> Generate synchronized, speech- and music-free Foley for a muted input video, or text-to-audio Foley, with the LTX-2.3 Foley LoRA.

The Foley LoRA generates synchronized, music- and speech-free sound effects. Its primary use is **video-to-audio**: feed a muted clip and a prompt describing the on-screen action, and the model freezes the video and synthesizes a matching foley track, muxed onto the original frames. It also works for **text-to-audio** foley (prompt only, no video). It's the video-conditioned counterpart to [Text-to-Audio](/open-source-model/usage-guides/text-to-audio).

> **Note**
>
> **LTX-2.5 support is in development.** This LoRA is validated on **LTX-2.3**; it has not been validated on LTX-2.5 yet.

## Prerequisites

* For video-to-audio: a **muted** input clip (strip the audio first — the workflow replaces it).
* Base model, Gemma text encoder, and the Foley LoRA (see Model files).
* Custom nodes: **ComfyUI-LTXVideo** (audio + V2A nodes) and **ComfyUI-VideoHelperSuite** (`VHS_LoadVideo`).

## Model files

| File                                                         | Description        | Placement                       |
| ------------------------------------------------------------ | ------------------ | ------------------------------- |
| `ltx-2.3-22b-lora-foley-v2a-1.0.safetensors`                 | Foley V2A LoRA     | `ComfyUI/models/loras/`         |
| `ltx-2.3-22b-dev.safetensors`                                | Base model         | `ComfyUI/models/checkpoints/`   |
| `gemma-3-12b-it-qat-q4_0-unquantized` (readout-proj variant) | Gemma text encoder | `ComfyUI/models/text_encoders/` |

Download the LoRA from [Lightricks/LTX-2.3-22b-LoRA-Foley-V2A](https://huggingface.co/Lightricks/LTX-2.3-22b-LoRA-Foley-V2A) on HuggingFace.

## Two ways to run it

* **Text-to-audio foley** — add the Foley LoRA to the standard [Text-to-Audio workflow](https://github.com/Lightricks/ComfyUI-LTXVideo/blob/master/example_workflows/2.3/LTX-2.3_T2A_Single_Stage_Distilled.json) and prompt the sound directly, no video.
* **Video-to-audio foley (primary)** — use the dedicated [Foley V2A workflow](https://huggingface.co/Lightricks/LTX-2.3-22b-LoRA-Foley-V2A/blob/main/ltx-2.3-foley-v2a.json), which freezes the input video and generates only the audio.

## Step by step (video-to-audio)

1. Load the [Foley V2A workflow](https://huggingface.co/Lightricks/LTX-2.3-22b-LoRA-Foley-V2A/blob/main/ltx-2.3-foley-v2a.json) in ComfyUI.
2. In **VHS\_LoadVideo**, pick your muted clip. The workflow rounds the loaded frame count down to a valid `1 + 8n` value and trims to match; `frame_load_cap` defaults to 169 (\~7s). Trained and demoed at **960×544, 89 frames @ 24fps** (\~3.7s) — process longer clips in windows.
3. Write the **positive prompt** in concrete foley language (name the sound sources and their sync), ending with `No speech is present. No music is present.` for clean SFX-only output.
4. Keep the **negative prompt** (suppresses music and speech).
5. If output is near-silent, change the **seed** and re-run — quiet or delicate scenes (rain, typing, footsteps) are seed-sensitive; take the loudest of a few seeds.
6. Queue. The output is the original frames plus generated foley.

## How it works

The input video is VAE-encoded and concatenated with an empty audio latent, then `LTXVSetAudioVideoMaskByTime` freezes the video (`mask_video=false`) and opens the audio for generation (`mask_audio=true`). The LoRA targets the audio attention/FFN blocks and video→audio cross-attention, so the model keeps the input video and synthesizes only the audio under `MultimodalGuider`. The audio is separated, VAE-decoded, and muxed onto the original video frames.

## Customization — recommended settings

| Setting                | Node                       | Value                                   | Notes                                             |
| ---------------------- | -------------------------- | --------------------------------------- | ------------------------------------------------- |
| Foley LoRA strength    | `LoraLoaderModelOnly`      | 0.8–1.0                                 | Demos used full strength; lower for subtler foley |
| Guidance scale (audio) | `GuiderParameters` (audio) | 6.0                                     | Prefer 6–8; \~4 tends to collapse to near-silence |
| Steps                  | `LTXVScheduler`            | 30                                      |                                                   |
| STG                    | `MultimodalGuider`         | scale 1.0, blocks `[29]`, mode `stg_av` |                                                   |
| Resolution & frames    | —                          | 960×544, 89 @ 24fps                     | Trained bucket                                    |
| Seed                   | `RandomNoise`              | 42                                      | Multi-seed and pick the loudest on quiet scenes   |

**Prompting** — describe the visible action and materials concretely. Example:

```text
A sledgehammer smashes a glass bottle: a heavy whoosh, a sharp glass shatter and tinkling shards. No speech is present. No music is present.
```

Negative prompt:

```text
music, melody, song, singing, vocals, score, soundtrack, beat, rhythm bed, instrumental backing, speech, dialogue, talking, narration, tinny, thin, harsh, clipped, distorted, low bitrate
```

## Tips & troubleshooting

* **Near-silent output** — raise guidance to 6+, try a new seed (multi-seed, pick the loudest), and confirm you're on the released `…-1.0` weights.
* **Music or speech bleed** — keep the negative prompt and reinforce `No music is present` / `No speech is present` in the positive prompt.
* **Weak ambience** (rain, fire, typing) — seed-sensitive; multi-seed, and lean on "loud / close-mic / prominent" wording.
* **Unsynced SFX** — prompt the action you can actually see; mismatched prompts produce plausible but unsynced audio.
* **Original audio still present, or static output** — mute the input before loading; if the audio comes out static, the freeze masks are inverted (confirm `mask_video=false, mask_audio=true`).

## Technical notes

* LoRA (rank 32) on the audio and video→audio attention paths, trained with the video fixed and the audio generated. The released `…-1.0` file is the validation-selected checkpoint (later training steps over-suppress loudness).
* The V2A workflow is ComfyUI API-format (the `/prompt` format) — load it via the Load button (auto-detected) or POST it to `/prompt`.

> **Note**
>
> A Gemini prompt-enhancer variant of the V2A workflow can auto-write the foley prompt from the video. It requires a Google Gemini API key and additional third-party nodes and is not published yet — the base workflow above is the primary path.