A mosaic of faces from Mega-MMDF

Mega-MMDF: Benchmarking Multimodal Deepfake Detection across Diverse Forgery Pipelines with DeepfakeBench-MM

* Equal contribution · † Corresponding author

100,000Real audiovisual clips
1,200,000Forged audiovisual clips
22Forgery pipelines
30Forgery methods
Dataset ↗ Application form ↗ Benchmark code ↗ arXiv ↗

The dataset

Scale and diverse
forgery pipelines.

Mega-MMDF combines audio and visual forgery methods, audio-driven face reenactment, and joint audio-video generation.

10Audio forgery
methods
12Visual forgery
methods
6Audio-driven face
reenactment methods
2Joint audio-video
generation methods
Dataset comparison by forgery methods, scale, and method recency
Dataset scale and forgery-method coverage. The axes count methods producing each modality, including joint generation and audio-driven reenactment.

Dataset statistics

Clip counts across real pipeline P0 and forged pipelines P1–P22
Pipelines
Demographic composition of the training, validation and test splits
Demographics
Clip-duration histogram and cumulative distribution
Duration (0–20 s)
Stored video-resolution distribution
Resolution
Pipeline definitions22 forgery pipelines + real reference
IDAudio forgeryVisual forgeryAudio-driven FRType

VC: voice conversion; TTS: text-to-speech; PF: partial audio forgery; FS: face swapping; EFS: entire face synthesis; FE: face editing; FR: face reenactment; T2AV: text-to-audio-video generation. A dash denotes no operation at that stage.

Pipeline showcase

How the forgeries
are composed.

Explore 22 forgery pipelines and one authentic reference. Each slide pairs a construction diagram with video frames and an audio spectrogram.

01 / 23P0 · Authentic reference

Swipe across each image to view the full diagram or sample.

Pipeline construction
P0: authentic audio and video, without forgery
Example output Video frames · Audio spectrogram
Video frames and audio spectrogram from an authentic sample

Authentic reference

Retain the original audio and video as an authentic reference.

Auto-advance · 10 seconds per slide

The benchmark

DeepfakeBench-MM.

A unified, modular codebase for training, evaluating, and extending multimodal deepfake detectors.

DeepfakeBench-MM modular framework: data processing, detectors, training and evaluation components
Shared workflows with extensible data, detector, training, and evaluation components.

8 supported datasets

FakeAVCeleb · LAV-DF · PolyGlotFake · IDForge · Mega-MMDF · TalkingHeadBench · DigiFakeAV · MMDF

10 evaluated detectors

Baseline · AVFF · AVH · FauForensics · FGMDF · FRADE · MCL · SpeechForensics · Qwen2.5-Omni · VideoLLaMA2.1

Use DeepfakeBench-MM →

Benchmark results

All eight test datasets.
The complete comparison.

Seven detectors trained separately on five datasets, alongside three methods evaluated without benchmark-specific training.

Clip-level AUC ↑38 evaluations · 8 test datasets
Complete evaluation results in clip-level AUC. Training sources appear in six row groups; columns cover all eight test datasets and their mean.

Results reproduce Table III of the paper. Average is the unweighted mean across all eight test datasets. Training-free denotes no benchmark-specific training.

Access & use

Data access and code

Mega-MMDF uses an access request on Hugging Face and a signed application form.

Request Mega-MMDF

Applications are currently closed. When they open, complete both steps.

  1. Submit a Hugging Face request

    Request access on the dataset page using your personal Hugging Face account.

    Open the dataset page ↗

  2. Email the signed application form

    Complete and sign the public form. Email it to kangranzhao@link.cuhk.edu.cn with the subject Mega-MMDF access / YOUR_USERNAME.

    Download the application form (PDF) ↗

Review begins after we receive both the Hugging Face request and the signed application form.

Use DeepfakeBench-MM

DeepfakeBench-MM is the benchmark for data preparation, detector training, and evaluation.

Benchmark code ↗

PrepareConfigure paths and preprocess data.

TrainChoose a detector and training configuration.

EvaluateRun tests and compare benchmark results.

Dataset repository on GitHub ↗

Citation & contact

Build on Mega-MMDF.

If Mega-MMDF or DeepfakeBench-MM supports your research, please cite the accompanying work.

BibTeX · publication details to be updated

Secure Computing Lab of Big Data (SCLBD)

See the Mega-MMDF dataset license for the terms of dataset use.

This project is built by SCLBD at the School of Artificial Intelligence (SAI) of The Chinese University of Hong Kong, Shenzhen, directed by Professor Baoyuan Wu. The lab focuses on research in trustworthy AI, including adversarial machine learning, agent security and safety, deepfake and AIGC detection, and data governance.

If you have any suggestions, comments, or wish to contribute code or propose methods, we warmly welcome your input.

Contact

Kangran Zhao kangranzhao@link.cuhk.edu.cn Baoyuan Wu wubaoyuan@cuhk.edu.cn