What Is Deepfake Detection and Why Scores Lie
Deepfake detection is a computational forensic assessment of whether a video was manipulated or generated by AI, and the result is a probability for triage, not proof. The strongest model in the 2020 public challenge reached 82.56% accuracy on evaluation clips that resembled its training world, then only 65.18% on previously unseen examples.
That gap is what people miss when they ask what is deepfake detection and expect a yes or no. A system can learn visual and temporal artifacts from synthetic media and still go quiet on a new generator, a different camera, or a file that has already passed through a social app. For a newsroom, a court, or a fraud desk, the useful record is which signals were checked, how uncertain the result is, and whether provenance agrees.
What Deepfake Detection Actually Measures
Deepfake detection uses computational forensic methods to estimate whether a video was manipulated or generated by artificial intelligence. The public stress test that made the limits obvious was the 2020 Facebook Deepfake Detection Challenge. Meta's results write-up describes a dataset of approximately 100,000 videos, 2,114 participants, and about 35,000 submitted models. The strongest model hit 82.56% accuracy on public evaluation data close to its training setting. On a separate black-box set of previously unseen examples, that same winner reached only 65.18%.
The model had learned artifacts that were common in its training world. It had not acquired a permanent test for every future generator, face, camera, or social upload. A smoke alarm can be dependable in the room where it was tested and quieter once the walls move. Generation and detection stay adversarially linked. When generators change, detectors fitted to older artifacts lose their edge.
A score is a sorting tool
Newsrooms, investigators, platforms, and enterprises should read the output as probabilistic authenticity assessment, not an infallible yes-or-no judgment. A strong lab percentage is evidence that helps someone decide what to check next. It is not a finding that the person on screen said those words, and it is not a finding that the event happened.
A detector that only works on clips like its homework will look brilliant in a paper and ordinary on the file in your inbox.
The checks underneath a score cover spatial traces, motion over time, sound, and the file's own history. One of those clues is a weak witness on its own. Compression and an unfamiliar generator are where confident numbers fall apart. If a story, a case, or a payment depends on the clip, keep the original file before anyone re-uploads it, crops it, or saves a screen recording over it.
How Detection Grew Beyond a Single Clue
Early detectors often searched for one bodily tell, such as blinking, in small collections of short, low-resolution fake clips. A missed blink could seem decisive in a brief lab test, yet reveal little about a side-view recording or a social video compressed by a phone. A single clue is like one page in a book: it can offer evidence, but not the whole story.
Later benchmarks represented more of the variation found in real footage. A digital-forensics review of DeeperForensics-1.0 describes a collection of 60,000 videos and about 17.6 million frames of face-swap footage at 1920 by 1080. Boundary errors, unnatural facial motion, inconsistent lighting, and breaks in time may appear only in some frames. Evaluation across varied faces, devices, codecs, manipulation methods, and recording conditions gives a broader test than relying on a single tell.
Why the headline rise is not a universal rate
The review reports detection performance rising from about 65% in 2017 to nearly 90% in 2023 across surveyed methods. Those figures reflect different datasets, manipulation types, and test conditions, so they cannot predict how a detector will handle the next clip. A high benchmark score is not proof of reliability on a recompressed video or a generator the model has not encountered. The useful result shows which signals were checked, how uncertain the estimate is, and whether the video's provenance supports it.
Before trusting a tool or study, ask three questions:
- Breadth: Do the training and test sets cover varied faces, codecs, and manipulation types?
- An outside test: How does the model perform on footage it did not train on?
- Updates: Is the model refreshed as generation methods change, or does it reflect only older artifacts?
The field has moved from treating a blink as a key clue to examining evidence across many frames and varied clips. More data and broader tests can improve detection, but neither turns a score into a verdict. Use the result to decide what to check next, not to settle whether a video is authentic.
The Four Forensic Signals Detectors Use
Treat detection as multimodal forensic classification, not a test of one face photo. A production system samples frames, listens to the track, watches identity and motion over time, and reads the container. Then it says which of those channels moved.
| Signal | What it inspects | Typical clues | Where it fails |
|---|---|---|---|
| Frame-level analysis | Sampled frames for generator traces | Blending edges, odd skin texture, bad lighting, diffusion artifacts, frequency patterns | Compression or a new generator erases the traces |
| Audio forensics | Soundtrack spectrum and speech timing | Synthetic spectral patterns, voice that does not match the face | Real audio dubbed on fake video, or a silent clip |
| Temporal consistency | Motion and identity across frames | Flicker, irregular facial motion, lip-audio desync, frame warping | Very short, heavily edited, or odd-angle clips |
| Metadata inspection | Container, codec, timestamps, encode history | Re-encoding, contradictory or missing creation data | Platforms strip metadata; it never proves the event |
Why one channel is a weak witness
Frame-level analysis looks for blending boundaries, abnormal skin texture, inconsistent illumination, generator-specific frequency patterns, diffusion artifacts, and unnatural textures. Audio forensics looks for synthetic spectral patterns and speech that does not belong with the mouth. A clean human voice can sit on a fake face. A silent clip has nothing for the audio model to score.
Temporal consistency tracks irregular facial motion, unstable identity, flicker, lip-audio desynchronization, and frame-to-frame warping. A very short or heavily edited clip may not contain enough motion to judge. Metadata inspection reads the container, codec, timestamps, and signs of re-encoding. Stripped metadata cannot name the creator, and it cannot prove the event. Agreement between texture and motion is a stronger lead than either one alone, which is why consistency and texture need to be read together.
What a production check should report
DeepfakeBench supplies a reproducible frame with 15 detection methods and 9 datasets, plus shared preprocessing and evaluation protocols. The point of that shared setup is a familiar trap. A model can look excellent only on the manipulation family and the compression it trained on. A detector that memorized a codec, a camera, or a dataset background can fail when the clip is re-encoded or made by an unseen model.
A practical detector aggregates frame scores over time, calibrates confidence, and reports results separately by manipulation type, source identity, resolution, and compression. Accuracy alone is not a deployment decision. Ask which signals fired, and how hard, not only what the headline percentage was.
Where Accuracy Collapses and How to Read the Result
Compression and re-encoding are the main reason a real fake gets missed. Re-encoding can wipe the high-frequency traces convolutional detectors use, and it can add new artifacts that resemble synthesis fingerprints. Low resolution, heavy compression, crops, screen recordings, and clips that were already edited deserve extra doubt. The original file, when you still have it, is a different object from the copy a platform served.
| Test setting | Reported result | What it means |
|---|---|---|
| DFDC public evaluation, best model | 82.56% accuracy | Strong when clips resemble training |
| DFDC black-box unseen set, same winner | 65.18% accuracy | Drops on new examples |
| XceptionNet across four FaceForensics++ types | about 81.39% accuracy | A blended average hides weak types |
| XceptionNet on NeuralTextures only | about 52.04% accuracy | Near chance on a different generator |
| Surveyed methods, 2017 to 2023 | about 65% to nearly 90% | Gains depend on the test, not a universal rate |
| 2025 in-the-wild evaluation | F1 above 84%, AUC under 70% | Headline metrics miss ordinary online video |
A new generator is a different problem
In FaceForensics++ evaluations covered by a forensic methods paper, XceptionNet averages about 81.39% accuracy across DeepFakes, Face2Face, FaceSwap, and NeuralTextures, then falls to roughly 52.04% on NeuralTextures alone. The blended average hides a near-chance result on one generation mechanism. Standardized cross-dataset protocols exist to expose that gap instead of stopping at in-domain scores.
A 2025 evaluation found that, despite F1 scores above 84%, no tested detector exceeded an AUC of 70% on in-the-wild material. Headline metrics can look strong and still miss ordinary online video. Detectors trained only on recent synthetic examples do not necessarily travel better than systems trained on older, more diverse data. Public audits of general models show the same kind of miss, including Grok's performance in a World Cup audit, where image and video checks on live event footage were far from a clean sweep.
Three ways the call goes wrong
False positive: The tool labels a real clip synthetic. That can wreck a source or a defendant if someone treats the label as guilt.
False negative: A fake gets through after blur, recompression, or an unseen generator. The clip looked ordinary to the model because the traces it expected were gone.
Abstention: The clip is too short, too crushed, or outside the model's experience. Declining to call it is a valid output. Forcing a binary label is not.
The detection score represents probability, not certainty, and no detection is 100% perfect. Confidence calibration is how you stop a raw percentage from pretending to be certainty.
The note to file should say which signals were examined, how reliable that result is for this media type, and whether independent evidence agrees.
Four operating moves keep the write-up honest. Sample the original file when it exists. Inspect codec and metadata changes. Score many frames rather than one thumbnail. Keep the uncertainty in the note instead of issuing a verdict. Pair the model output with provenance, chain-of-custody records, source checks, and human forensic review.
How Newsrooms and Investigators Use Results
A newsroom screens a user tip or a wire clip before air. The model score decides whether the piece waits, not whether it runs. The desk still finds the earliest credible source and looks for a second camera or a witness. Publishing on a single percentage is how a recompressed meme becomes a segment.
A legal or investigative team hashes the file and logs who touched it. The model output is one exhibit beside the chain of custody. It is not the finding that the depicted event occurred. A qualified reviewer still has to say what the file can and cannot support.
An enterprise fraud desk uses a suspicious CEO video, or a video-call clip, to pause a payment. The score buys time. Confirmation goes out through a known channel, a callback on a number the company already trusts, not the number in the message.
What agreement still cannot prove
Recent evaluation work reports that detectors stay vulnerable to ordinary blur and compression. Benchmarks often lean on synthetic data and do not systematically represent politically significant real incidents. Many systems are thinly tested against adversarial edits, and they can struggle when audio and video are out of sync or the camera angle is odd.
Adding frame artifacts, audio anomalies, temporal inconsistencies, and metadata inspection can widen coverage. Agreement among those signals still does not establish who created the file, when it was created, or whether the event happened. Provenance and chain of custody can outweigh a single model score once a file has been through social platforms.
The same caution covers political misinformation, travel and listing scams, dating clips, and identity fraud. The file in circulation is usually a recompress of something else.
- Retain the original before you upload a copy anywhere.
- Hash it and log access so later copies can be tied to that file.
- Compare copies across platforms and note what each re-encode changed.
- Find the earliest credible source, not the loudest repost.
- Seek corroborating footage or witnesses that do not depend on the same file.
- Report uncertainty in the write-up, including what the model did not see.
Preserve the Original, Then Score It as Triage
If the clip can change a story, a case, or a payment, preserve the original before you upload any copy. A second save can be the step that strips the traces a later review needed. Evidence preservation comes first. Scoring comes second.
AI Video Detector analyzes an uploaded video frame by frame and returns a confidence score estimating whether AI created or altered it. The check draws on frame content, audio, temporal consistency, and metadata. Basic detection does not require an account. You can analyze a video without signing in, and the free analysis is privacy-first with no image storage. That matters when a clip is unpublished or may become evidence.
How to read the band
A high score still needs the provenance steps. It is a reason to slow down, not a badge you can print without a source check. Newsrooms can verify a user-submitted tip in seconds, screen wire footage, and add an authenticity badge only after that extra check, not instead of it.
A low or middle score means pause. Do not publish, and do not pay, on a number that is sitting in the middle or leaning real while the file is a screen recording. Record the score and the signals behind it. Corroborate with the earliest source, another recording, or a known channel before the decision is final.
Keep the file. Write down what moved the score. Treat the result as triage evidence beside provenance, not as the last word.
Upload the original clip to AI Video Detector before you air it, file it, or release a payment. Run the free check without signing in, write down the confidence score and which signals moved it, and hold the decision until provenance agrees. If the file is sensitive or the result is unclear, email ivan@writehuman.ai and keep the untouched original in your own custody.
