LatentSync (ByteDance)
App : A product you use directly
High-quality diffusion-based lip sync, running locally.
The most convincing local lip-sync tool, demanding in VRAM.
Security reflects where data goes, requested permissions and incidents on record at the verification date. It is not a guarantee of zero risk: check the deployment mode and the connections you enable.
What it does, concretely
Making a filmed person's lip movements match a different audio track is a complex task that often produces blurry or out-of-sync results. LatentSync, developed by ByteDance, addresses this problem using an audio-conditioned latent diffusion model without any intermediate motion representation.
In practice, the tool analyzes audio through OpenAI's Whisper model to extract embeddings, which are then integrated into a U-Net network via cross-attention layers. The result is a video where the subject's lips are reconstructed frame by frame to match the provided audio. Version 1.6, released in June 2025, works at 512x512 resolution to reduce blurriness.
This tool is aimed at content creators, researchers, and developers who are comfortable installing a Python environment and have a graphics card with sufficient VRAM: at least 8 GB with version 1.5, and 18 GB with version 1.6.
However, LatentSync is not suitable for people without command-line experience or those looking for a ready-to-use solution. Installation requires setting up a technical environment and downloading models from HuggingFace.
What it offers
- ·Audio-driven lip synchronization using a latent diffusion model based on Stable Diffusion.
- ·Audio feature extraction via OpenAI's Whisper model for accurate speech analysis.
- ·Built-in Gradio interface to run inference without using the command line.
- ·Support for Chinese and other language videos, improved since version 1.5.
- ·Better temporal consistency thanks to the addition of a temporal layer in version 1.5.
- ·512x512 pixel inference resolution with version 1.6 to reduce blurriness.
- ·Complete data processing pipeline provided, including scene detection and quality filtering.
- ·Training code and checkpoints available open source for users wishing to fine-tune the model.
In practice
- Price
- Free
- Commercial use
- Commercial use: yes
- Available on
- linux, windows
- Licence
- Apache-2.0
- Runs locally
- Yes, 16 GB VRAM recommended
- FR interface
- No
Security and data
- Rating
- Security: one caveat, see the card
- Your data
- Data stays local
- Maintenance
- Inactive for over six months
Links and repository checked on 25 October 2026 · added on 1 December 2024
How to install LatentSync (ByteDance)
source setup_env.sh