Threat-Modeling AI Output Provenance: C2PA, SynthID, and the Adversaries That Break Them

On May 19, 2026, OpenAI announced it was doing something it had resisted for years: embedding Google’s SynthID watermarks into every ChatGPT image output, layering them on top of C2PA Content Credentials already attached to DALL-E and Sora generations. The same day, the company previewed a public verification portal at openai.com/verify. The announcement, covered in detail by The Verge, contained a line that should stop every security engineer mid-scroll: “No detection method is foolproof, so we take a cautious approach in cases when detection fails.” Translation: the two largest AI labs on the planet, deploying the two most widely adopted provenance standards simultaneously, still cannot guarantee detection. That is the threat model.

Key Takeaways:

• C2PA Content Credentials provide a cryptographically verifiable chain of custody, but a single screenshot strips them entirely.
• SynthID pixel watermarks survive recompression, cropping, and screenshots in many cases, but adversarial attacks like MarkSweep can degrade them below detection thresholds.
• The regenerator adversary, who passes AI output through their own model, defeats every current provenance defense simultaneously.
• A production defender stack requires three layers: a signing pipeline with hardware-attested keys, an append-only provenance database, and a multi-signal verification chain at consumption time.
• The Integrity Clash, showed at CVPR 2026, shows that C2PA and watermarking can produce cryptographically valid but contradictory results when not cross-audited.

Four Adversary Profiles: Who You Are Defending Against

Threat modeling starts with the adversary. For AI output provenance, four distinct profiles emerge, each with different resources, methods, and targets. As we explored in our earlier threat-modeling analysis of AI output provenance, mapping these profiles against specific defenses reveals gaps that a single-layer approach cannot close.

Profile 1: The Regenerator

This adversary has access to their own generative model, possibly the same architecture as the original. Given a piece of AI-generated content, they feed it back through their model. The output preserves semantic content but carries none of the original’s provenance signals. C2PA metadata is gone because the regenerator creates a new file from scratch. SynthID watermarks do not survive because the regenerator’s model never embedded them. This is the hardest adversary to defend against. A 2024 paper from ETH Zurich researchers, “Image Watermarks are Removable Using Controllable Regeneration from Clean Noise,” showed that regeneration from a clean noise starting point effectively nullifies current watermarking techniques while preserving visual quality. The regenerator requires moderate resources, a capable generative model, but no cryptographic compromise.

Profile 2: The Screenshotter

The lowest-resource adversary. They take a screenshot of an AI-generated image or video frame. C2PA credentials die instantly because the screenshot creates a new file with no metadata chain. SynthID watermarks may survive, though detection confidence degrades. OpenAI’s May 2026 announcement acknowledged this explicitly, stating that “watermarking can be more durable through transformations like screenshots, while metadata can provide more information than watermark alone.” The screenshotter requires no technical skill and no special software. Anyone who can press a key combination can strip metadata-based provenance.

Profile 3: The Paraphraser

This adversary targets text. They take AI-generated text and run it through a different LLM with instructions to paraphrase. The statistical signature of SynthID-Text dissolves because the token distribution shifts. A 2025 paper, “Adversarial Paraphrasing: A Universal Attack for Humanizing AI-Generated Text,” showed that adversarial paraphrasing guided by an AI text detector reduces detection rates dramatically. Across a diverse set of detectors, including neural network-based, watermark-based, and zero-shot approaches, the attack achieved an average true-positive-at-1%-false-positive reduction of 87.88% under the guidance of OpenAI-RoBERTa-Large. Another paper, “Cross-Lingual Summarization as Black-Box Watermark Removal Attack,” showed that translating text to a pivot language, summarizing it, and optionally back-translating systematically destroys token-level statistical biases while preserving semantic fidelity. For SynthID-Text specifically, cross-lingual summarization drove detection to near-chance levels.

Profile 4: The Spicer

They splice AI-generated segments with authentic content: a real photo with an AI-generated person inserted, real audio with synthetic phrases overlaid. This defeats both C2PA and watermarking because the composite file contains genuine provenance for the authentic parts and no provenance for the AI-generated parts. Detection requires pixel-level or sample-level analysis that neither scheme provides natively. The spicer is especially dangerous in news contexts, where a mostly-authentic image with a single fabricated element can change the meaning of a story.

The Defense Matrix: Which Defense Holds Against Which Adversary

Each provenance defense catches different adversaries and fails against others. The matrix below maps the four adversary profiles against three primary defense layers, drawing on published research and vendor documentation as of August 2026.

The regenerator is the only adversary against which every defense fails simultaneously. The Paraphraser is close behind for text. The Screenshotter is the most common adversary in practice and the one that C2PA-only deployments are most exposed to. The Spicer is the hardest to detect at scale because composite content looks mostly authentic to both human reviewers and automated systems.

For images and video, the practical defense is layering C2PA with watermarking. C2PA provides strong provenance when metadata survives, which happens mostly in professional contexts, direct downloads, and platforms that deliberately preserve it. SynthID detection catches AI-generated content that has been stripped of metadata but still carries the embedded signal. The overlap is not redundancy; it is defense in depth. For a deeper dive into how these layers perform under real-world image transformations, see our analysis of AI image detection methods, which covers recompression, cropping, screenshots, and production verification workflows.

STRIDE and DREAD: Formalizing the Threat Surface

Applying the STRIDE framework to AI output provenance surfaces which threats each defense covers and which it leaves exposed.

Spoofing: An adversary presents AI-generated content as human-created. C2PA counters this when metadata is intact and signatures verify. Watermarking counters it when the embedded pattern is detectable. Both fail against the regenerator and the spicer. Classifier-based detection adds a general layer, but a 2026 zero-shot benchmark evaluating 23 pretrained detector variants across 12 datasets containing 2.6 million samples from 291 generators reported a mean accuracy of 75.0% for the best detector and 37.5% for the worst, as documented in the benchmark study.

Tampering: An adversary modifies content after creation and claims it is original. C2PA’s signature chain detects tampering if metadata survives. If the adversary strips metadata first, tampering is invisible. Watermarking does not detect tampering because the watermark only proves origin, not integrity. A signing pipeline with hash chaining addresses this, but only within the metadata track.

Repudiation: An adversary denies having generated content. C2PA provides non-repudiation through cryptographic signatures tied to a specific identity or device. The Sony A9 III camera signs each image with a hardware key. The photographer cannot plausibly deny taking that photo. Watermarking provides weaker non-repudiation because the watermark proves content came from a model, not which user or device triggered generation.

Information Disclosure: Provenance metadata can leak sensitive information. A C2PA manifest may include GPS coordinates, camera serial numbers, edit history, and software versions. An adversary who intercepts a signed manifest learns details about the creator’s workflow and location. Watermarking embeds no explicit metadata, only a statistical signal, so the disclosure surface is smaller.

Denial of Service: An adversary floods a verification system with fake content to overwhelm detection capacity. This is a platform-level threat. Rate limiting, caching, and async verification queues mitigate it. Neither C2PA nor watermarking addresses DoS directly.

Elevation of Privilege: An adversary compromises signing infrastructure to forge provenance. If an attacker gains access to a camera’s signing key or a cloud-based signing service, they can issue valid C2PA credentials for any content. This is the most severe threat to the entire provenance model. Key management and hardware security modules are the only mitigations. A real-world example: Nikon’s C2PA implementation suffered a signing vulnerability discovered after the Z6 III received C2PA support via firmware in August 2025. Nikon had to revoke all issued certificates, invalidating every credential those cameras had produced, and the service had not been restored as of early 2026.

DREAD scoring assigns the highest overall risk to the regenerator, paraphraser, and spicer, all of which combine high damage potential with high reproducibility and low detectability. Key compromise scores highest on damage but lowest on exploitability, making it a classic low-probability, high-impact threat that security teams lose sleep over.

The Defender Stack: Signing Pipeline, Provenance Database, Verification Chain

A production provenance system in 2026 requires three layers. This is the architecture that the EU AI Act’s Code of Practice on Transparency of AI-Generated Content effectively mandates, and it is the architecture that OpenAI and Google jointly deployed in May 2026.

Layer 1: The Signing Pipeline

Provenance is created at the point of content generation. For camera-captured content, signing happens in firmware using hardware-attested keys. Sony, Nikon, Leica, and Canon all ship cameras with C2PA signing at the firmware level. The BBC worked with Sony’s PXW-Z300 camera in 2025, one of the first to embed digital signatures into video files at source, as documented in BBC R&D’s technical report. For AI-generated content, the model provider’s inference pipeline signs outputs before they leave the generation environment. OpenAI signs DALL-E 3 and Sora outputs. Adobe Firefly signs every output by default. Google embeds SynthID across Imagen, Veo, Lyria, and Gemini Text.

The architectural question is where signing keys live. Hardware-level signing (camera firmware, HSM-backed cloud services) provides the strongest trust anchor. Software-level signing (app-layer C2PA embedding) is more flexible but vulnerable to key extraction. For platforms that generate content at scale, the signing pipeline should use a dedicated key management service with audit logging and automatic key rotation.

The C2PA specification version 2.2, standardized under ISO/IEC 22144, uses standard PKI infrastructure, not blockchain. The reference implementation is c2pa-rs, a Rust SDK with 384 stars and 178 forks on GitHub, actively maintained. Bindings exist for JavaScript, Python, Java, C, and iOS. The Content Authenticity Initiative provides a public web verification tool and open-source libraries for manifest extraction and validation. Certificate costs run approximately $289 per year from DigiCert, and there is no equivalent of Let’s Encrypt providing free trusted certificates for C2PA signing, creating a cost barrier for smaller organizations and individual creators.

Layer 2: The Provenance Database

A provenance database stores signed manifests and makes them queryable at verification time. This is separate from the content itself. When a user uploads a C2PA-signed image to a social platform, the platform extracts the manifest, verifies the signature, and stores the verification result alongside the content ID. When another user encounters that image, the platform can retrieve the stored provenance without re-verifying.

The database must be append-only and tamper-evident. Any modification to a stored manifest invalidates the trust model. Immutable storage with cryptographic hash chaining is the appropriate architecture. The BBC R&D team, working with the IBC Accelerator program, developed an open-source “stamping” tool that performs digital signatures using the company’s certificate and inserts C2PA metadata into content at the time of publishing, alongside a complementary open-source plugin to decode and verify these credentials.

Layer 3: The Verification Chain

At consumption time, the verification chain checks every piece of content against multiple signals. First, extract and verify C2PA credentials using libc2pa or the JavaScript SDK. Second, run watermark detection via Google’s SynthID API, which is now integrated into Google Search, Chrome, and Pixel devices. Third, apply classifier-based detection as a fallback for content that carries no provenance signals.

The verification chain must produce a graded result, not a binary pass/fail. A photo from a C2PA-signed camera with intact metadata and no watermark receives high confidence. A photo with stripped metadata but a detectable SynthID watermark receives medium confidence. A photo with no metadata and no watermark receives low confidence, and the platform applies additional scrutiny through behavioral signals, account reputation, and manual review.

For discrepancies, the system must flag them. An image that carries a C2PA claim of human capture but also triggers SynthID detection is either misattributed or adversarially manipulated. This is the Integrity Clash.

What This Looks Like in Real Products

Three product contexts illustrate how the defender stack works in practice and where it breaks.

News Platform

A major wire service requires C2PA credentials on all submitted images. The signing pipeline is hardware-level: photojournalists use Sony A9 III or Nikon Z9 cameras that sign every image at capture. The provenance database stores manifests alongside article metadata. The verification chain runs at ingestion time, rejecting any image whose C2PA signature does not verify or whose camera serial number is not registered to an accredited journalist. For images from non-C2PA sources, the platform applies SynthID detection and classifier-based analysis, flagging anything above a confidence threshold for human review. This works because the news platform controls its ingestion pipeline end to end. The BBC’s collaboration with Sony on the PXW-Z300 camera showed exactly this workflow for video.

Social Media Platform

A social platform with user-generated content faces a harder problem. It cannot require C2PA credentials on upload because most users do not have C2PA-signed cameras. The platform extracts and verifies C2PA manifests when present, runs SynthID detection on all images, applies classifier-based detection as a general fallback, and produces a confidence score per piece of content that feeds into the trust and safety pipeline. TikTok has labeled over 3 billion pieces of content as AI-generated through a combination of Content Credentials, invisible watermarking, creator education, and labeling tools, according to the C2PA announcement when TikTok joined the Steering Committee in July 2026.

Education Platform

An online learning platform needs to verify that student submissions are original work. The signing pipeline is irrelevant because students are not generating content through controlled infrastructure. The platform relies entirely on classifier-based detection and behavioral analysis. This is the weakest provenance model of the three and the one where adversarial attacks are most effective. A student who paraphrases an AI-generated essay through a second LLM defeats detection at near-chance levels. The education case illustrates a fundamental limitation: provenance technology works best when you control the creation pipeline and degrades rapidly when you do not.

The Integrity Clash: When Provenance and Watermarking Disagree

The most important academic finding for developers building provenance systems in 2026 comes from a paper accepted at the CVPR 2026 Workshop on Authenticity and Provenance in the Age of AI. “Authenticated Contradictions from Desynchronized Provenance and Watermarking” formalizes what the authors call the Integrity Clash: a condition in which a digital asset carries a cryptographically valid C2PA manifest asserting human authorship while its pixels simultaneously carry a watermark identifying it as AI-generated, with both signals passing their respective verification checks in isolation.

The authors constructed metadata washing workflows that produce these authenticated fakes through standard editing pipelines, requiring no cryptographic compromise, only the semantic omission of a single assertion field permitted by the current C2PA specification. They then proposed a cross-layer audit protocol that jointly evaluates provenance metadata and watermark detection status, achieving 100% classification accuracy across 3,500 test images spanning four conflict-matrix states and three realistic perturbation conditions.

The practical implication is clear: any verification chain that checks C2PA and watermarking independently, without cross-referencing their results, is vulnerable to authenticated contradictions. The fix is technically straightforward, the paper argues, but requires that verification systems treat the two signals as cross-auditable rather than independent.

A related attack vector comes from MarkSweep, a no-box watermark removal attack published in February 2026. MarkSweep amplifies watermark noise in high-frequency regions via edge-aware Gaussian perturbations and injects it into clean images for training a denoising network. It reduces the bit accuracy of HiDDeN and Stable Signature watermarking schemes to below 67% while preserving perceptual quality. The attack requires no access to the watermarking algorithm, making it a practical threat against any deployed watermarking system.

The Limits of the Model

Threat-modeling AI output provenance reveals uncomfortable truths. The regenerator and spicer defeat every current defense. C2PA is strong against casual forgery but trivial to strip. Watermarking persists through recompression but is vulnerable to adversarial removal and paraphrasing. Classifier-based detection is brittle against novel generation methods.

The EU AI Act Article 50 transparency obligations became enforceable on August 2, 2026, as confirmed by the European Commission’s formal adequacy opinion on the Code of Practice, published July 9, 2026. The regulation requires machine-readable marking of AI-generated content, with penalties of up to €15 million or 3% of global annual turnover. The Code mandates that providers implement at least two distinct layers simultaneously: C2PA cryptographically signed metadata and imperceptible watermarking. A third layer, fingerprinting or hash logging, is available for cases where both preceding layers have been defeated.

The regulation does not require that marking survive every adversary. Compliance and security are related but not identical goals. A platform that meets Article 50’s requirements by deploying C2PA and SynthID detection is compliant. It is not immune to adversarial attack. The distinction matters because regulators will ask about compliance, but users will ask about trust, and trust is what breaks when an adversary wins.

For security engineers building provenance systems, the practical takeaway is to design for the adversary you can stop, not the one you cannot. The screenshotter and metadata stripper are stoppable with C2PA plus watermarking. The key compromise threat is stoppable with hardware security modules and audit logging. The regenerator and spicer are not stoppable with current technology, and any system that claims otherwise is overpromising.

The production standard for 2026 should be layered evidence: validate provenance, check embedded marks, run recently tested classifiers, inspect forensic traces, cross-audit C2PA and watermarking results for Integrity Clashes, and retain an inconclusive category. No single layer is sufficient. No negative result can prove that content came from a camera. The system should explain which signals survived, which were absent, and which transformations may have destroyed them. As we covered in our analysis of C2PA and SynthID detection limits, the practical engineering task is deploying all layers as defense in depth while being honest about what each can and cannot prove.

For a broader view of how these technologies sit within the 2026 regulatory and strategic landscape, see our guide to AI watermarking and provenance and our analysis of Meta’s 2024 watermarking threat model, which identified many of the same attack vectors that remain unresolved two years later.

Related Reading

More in-depth coverage from this blog on closely related topics:

• How to tell if an image is AI generated
• Anthropic openclaw claude usage allowed
• Latest Mistral AI Summit Paris Highlights
• Tailscale Peer Relay Feature Guide
• Best GPU for Local Large Language Models

Sources and References

Sources cited while researching and writing this article:

• OpenAI says it’s getting serious about AI detection and labeling
• “Image Watermarks are Removable Using Controllable Regeneration from Clean Noise,”
• “Adversarial Paraphrasing: A Universal Attack for Humanizing AI-Generated Text,”
• “Cross-Lingual Summarization as Black-Box Watermark Removal Attack,”
• benchmark study
• Content Credentials: The new camera that verifies video at the point of capture
• c2pa-rs
• public web verification tool
• C2PA announcement
• “Authenticated Contradictions from Desynchronized Provenance and Watermarking”
• MarkSweep