DaVinci MagiHuman

Overview

daVinci-MagiHuman is an open-source audio-video generative foundation model from SII-GAIR and Sand.ai. It uses a single-stream Transformer that takes text tokens, an optional reference image latent, and noisy video and audio tokens as input, and jointly denoises the video and audio within a unified token sequence. The architecture is described as a unified 15B-parameter, 40-layer Transformer that jointly processes text, video, and audio via self-attention only, with no cross-attention or multi-stream complexity. Key design choices include a sandwich architecture in which the first and last 4 layers use modality-specific projections while the middle 32 layers share parameters across modalities, timestep-free denoising, per-head gating, and unified conditioning. The model supports Chinese (Mandarin & Cantonese), English, Japanese, Korean, German, and French, and is reported to generate a 5-second 256p video in 2 seconds and a 5-second 1080p video in 38 seconds on a single H100 GPU. Efficient inference techniques include latent-space super-resolution, a turbo VAE decoder, full-graph compilation via MagiCompiler, and DMD-2 distillation enabling generation with only 8 denoising steps. The project releases the complete model stack: base model, distilled model, super-resolution model, and inference code, under the Apache License 2.0.