AI Trust, Audit, and Verification: Watermarking, Provenance, Verifiable Inference — The Complete Guide

Q: What's the simplest way to add verifiable inference?

Use a TEE-enabled cloud GPU instance. Lowest engineering cost.

Q: Does verifiable inference reduce model quality?

No — output is identical to non-verified inference. Verification adds proof, not modification.

Q: How much does verifiable inference cost vs regular?

TEE: ~5-30% premium. PoSP: ~1-5% premium. ZK: orders of magnitude (today).

Q: Is verifiable inference required for any regulations today?

Some healthcare and financial regulations effectively require it. Most don't yet mandate.

Q: What if my provider lies about using TEE?

Attestation lets you verify. Don't trust without verifying.

Q: Can I verify inference on my laptop?

TEE on laptops is limited. ZK is too slow for large models. PoSP requires multiple providers.

Q: How does verifiable inference handle model updates?

Each model version has its own attestation / proof setup.

Q: What's the impact on inference latency?

TEE: ~5%. PoSP: minimal (1-2%). ZK: significant for proof generation.

Q: How do I choose between approaches?

Cost, threat model, regulatory requirements, performance constraints all matter.

Q: Are there open-source verifiable inference frameworks?

Yes — Modulus Labs, Inference Labs, etc. Various stages of maturity.

If a model output crosses an organizational boundary, three things can go wrong and one party — usually the operator — gets to decide what you find out about each. Did the right model actually run? Was the answer generated by a machine rather than a human? Did the artifact you're looking at originate where the metadata claims? In 2026 these used to be three disconnected literatures — verifiable inference, AI watermarking, content provenance. They are now one stack. This guide treats them together.

The take: trust in AI infrastructure is no longer "do you believe your provider?" It is a layered system. TEEs (NVIDIA Confidential Compute on Hopper and Blackwell, Intel TDX, AMD SEV-SNP) attest that a specific model ran on specific hardware. Proof of Sampling and opML (Conway et al., arXiv:2401.17555) provide economic verification for decentralized inference. zkML (Chen et al., arXiv:2403.00735) is still 1000–10000× too expensive for frontier LLMs but works for small models today. Watermarking (SynthID, MarkMyWords (Piet et al., arXiv:2312.00273), Kirchenbauer et al., arXiv:2301.10226) and C2PA provenance tag the output so downstream consumers can detect it. Audit logging and model fingerprinting tie it all together. Pick layers by threat model, not by hype. Verification is the prerequisite for trustless compute — for how it fits the wider decentralized-AI stack (compute, training, data, agents, payments), see the Decentralized AI guide.

Key takeaways
Mental model: verifiable inference in one minute
The landscape in 2026
The trust problem
Watermarking text generation
C2PA and content provenance
TEEs in production at NVIDIA and Anthropic
When verification matters in agent workflows
Approach 1: redundant execution
Approach 2: Trusted Execution Environments (TEE)
Approach 3: Zero-knowledge proofs (ZK)
Approach 4: Proof of Sampling (PoSP)
Comparison and trade-offs
Production deployments in 2026
When verifiable inference matters
When it doesn't
The open research questions
The bottom line
FAQ
Glossary
References
Threat models per stakeholder
zkML stack landscape in 2026
TEE silicon comparison: H100, H200, B200, Intel TDX, AMD SEV-SNP, ARM CCA
opML and optimistic verification networks
Watermarking adversarial robustness deep dive
Decentralized inference verifiability (Bittensor, Atoma, Marlin)
Verification by workload type (training, inference, RAG, agent, fine-tuning)
Reasoning-model verification: the long-thinking-chain problem
Enterprise procurement: how to ask vendors for proof
2026 regulatory landscape: EU AI Act, NIST AI RMF, sectoral rules
Honest limits of each verification approach

Key takeaways

Four mechanisms for verifying that an inference run actually happened correctly:

Redundant execution: run on N providers, compare outputs. Cost: N× compute. Trust assumption: not all N collude.
Trusted Execution Environments (TEE): hardware-attested execution. NVIDIA Confidential Computing on H100/H200. Cost: ~5% overhead. Trust assumption: hardware vendor (NVIDIA, Intel) is trusted.
Zero-knowledge proofs (ZK): cryptographic guarantee. Cost: 1000-10000× compute (currently impractical for LLMs). Trust assumption: cryptographic primitives.
Proof of Sampling (PoSP): statistical verification via sampling. Cost: ~5% overhead. Trust assumption: provider can't predict which samples will be checked.

The frontier in 2026: PoSP and TEE for production deployments. ZK is research-grade for LLMs.

For most production: TEEs (when hardware-supported) or PoSP (for decentralized inference). Redundant execution for highest-stakes audit needs. Add watermarking when you need to detect AI-generated outputs after the fact, and C2PA provenance when you need cryptographic chain-of-custody for media.

Mental model: verifiable inference in one minute

The problem has a name: the trust gap. You send a prompt to an API. You get back a response. You have no proof that the provider ran the model you paid for, on the input you sent, with the weights they advertised. The provider could have routed your "GPT-class" request to a 7B cheap model, cached a stale response, or silently used a quantized variant that costs 4× less to serve. The output looks fine — language models are eloquent across a wide quality range. The gap between "the provider claims" and "you can prove" is the entire problem.

The fix is a layered stack of attestations. Four mechanisms, each with a different trust assumption:

TEEs — the GPU itself signs an attestation that a specific model hash ran inside an isolated enclave. Trust the silicon vendor.
Proof of Sampling (PoSP) — re-run a random fraction of requests on a trusted verifier; slash providers who diverge. Trust statistics.
Redundant execution — run on N providers, require agreement. Trust non-collusion.
zkML — a cryptographic proof that the computation was performed correctly. Trust math, pay 1000–10000× compute.

The analogy is restaurant inspection: TEE is a tamper-evident kitchen camera, PoSP is random health-department visits, redundant execution is ordering the same dish from three places, zkML is a notarized recipe-execution affidavit per plate.

Without verification vs with verification:

Aspect	Trust the provider	TEE	PoSP	zkML
What's proven	Nothing	Code + hardware identity	Statistical compliance	Exact computation
Compute overhead	0%	~5%	1–5%	1000–10000×
Trust assumption	Provider honesty	Silicon vendor	Sampling unpredictability	Crypto primitives
Practical at LLM scale	Yes	Yes (H100/H200/B200)	Yes	No (small models only)
When it pays off	Low-stakes	Compliance, healthcare	Decentralized marketplaces	On-chain, regulated

Production one-liner (NVIDIA Confidential Compute): launch container with --gpus 'all,capabilities=compute,attest' and the hypervisor returns a signed quote you can verify against NVIDIA's attestation service before sending sensitive prompts.

Sticky number: TEEs add ~5% inference overhead while giving you a cryptographic attestation that a specific model ran on a specific GPU — the only mechanism in 2026 that's both production-ready and cryptographically meaningful for frontier LLMs.

The rest of this guide is how to compose these layers by threat model.

The landscape in 2026

Five overlapping problem domains, often confused:

Verifiable inference — did the right model run on the right input? Answered by TEEs (NVIDIA Confidential Compute, Intel TDX, AMD SEV-SNP), zkML (Chen et al., 2024), opML (Conway et al., 2024), and Proof of Sampling.
Output watermarking — was this text or image generated by a model? Answered by SynthID, Kirchenbauer et al. (2023), MarkMyWords (Piet et al., 2023).
Content provenance — what is the chain of custody for this media artifact? Answered by C2PA signed manifests.
Model fingerprinting / provenance — are these weights what the publisher claims? Answered by hash-attested model cards, Proof-of-Learning (Jia et al., 2021), and signed weight registries (HuggingFace, NIM).
Audit logging — what did the system do, in order, on which input? Answered by signed event logs, agent traces, and replay-based verification.

Layered comparison

Layer	Question answered	Primary techniques	Overhead	Where it helps most
Execution attestation	Did this exact code/model run on attested hardware?	TEE (NVIDIA CC, TDX, SEV-SNP)	~5%	Compliance, multi-tenant SaaS, healthcare
Cryptographic execution proof	Can I verify without re-running?	zkML, opML	100–10,000× (zk); challenge window (opML)	Small models, on-chain settlement
Statistical execution check	Is the provider mostly honest?	Proof of Sampling, redundant execution	1–5% (PoSP), N× (redundant)	Decentralized GPU marketplaces
Output watermarking	Was this output generated by a model?	SynthID, Kirchenbauer, MarkMyWords	<1% (sampling-time)	Misuse detection, training-data hygiene
Content provenance	Where did this media come from?	C2PA manifests, signed metadata	Negligible	Journalism, generative media platforms
Model fingerprinting	Are these weights the published ones?	Hash chains, Proof-of-Learning	Negligible at load time	Supply-chain integrity
Audit logging	What happened, in what order?	Signed event logs, agent traces	<1%	Agent workflows, post-hoc investigation

These layers compose. A regulated healthcare deployment in 2026 might run on NVIDIA Confidential Compute (execution), watermark all generated text (output), and emit a signed agent trace (audit) — three independent guarantees stacked. See LLM serving and agent serving infrastructure for where these hook into the stack.

The trust problem

Imagine: you send a prompt to an inference API. You get back a response that looks reasonable. Three things might have happened:

The provider ran your prompt through the model you specified, exactly as you specified, with the parameters you specified. Honest.
The provider ran your prompt through a different (cheaper) model. Output looks plausible but isn't from the model you paid for. Fraudulent.
The provider ran your prompt but with adjusted parameters (e.g., higher temperature or a quantized version) that subtly degrade quality but reduce their compute cost. Subtly fraudulent.

You can't tell the difference from a single response. You'd need to check.

For most users (centralized API providers like OpenAI, Anthropic), trust is implicit: the provider has reputation and contracts. For decentralized marketplaces, anonymous operators, or compliance-required deployments, you need verification.

Watermarking text generation

Watermarking is the complement of verifiable inference: it doesn't prove the provider was honest, but it lets a third party detect after the fact that an output came from a model. Both problems matter, and they solve different threat models.

How LLM watermarking works

The Kirchenbauer et al. (2023) scheme is the canonical construction. At each decoding step:

Hash the previous token to seed a pseudorandom partition of the vocabulary into a "green list" and "red list".
Bias logits toward green-list tokens by a small constant.
The resulting output is statistically biased toward green tokens — invisible to a human reader, detectable with a z-test if you know the partition seed.

A detector that knows the secret can compute the fraction of green tokens; legitimate human text averages 50%, watermarked text 70–90%. The hypothesis test gives a calibrated false-positive rate.

SynthID Text (DeepMind, 2024, Nature) deploys a refinement called Tournament Sampling that watermarks during inference with negligible quality impact and is used in production for Gemini outputs. Google has released the detector under permissive license.

Robustness — the hard part

The benchmark that matters is MarkMyWords (Piet et al., 2023, arXiv:2312.00273), which stress-tests watermarking schemes against paraphrasing, translation round-trips, and synonym substitution. Key findings:

Soft watermarks survive light paraphrasing (rephrasing within ~30% token edit distance).
They break under aggressive paraphrasing or running the output through another model.
Short outputs (<200 tokens) cannot be reliably detected — there's not enough signal.
Watermark stripping costs the attacker quality and compute, which is a real economic barrier even when stripping is possible.

Where watermarking matters

Training-data hygiene: detect AI-generated text in scraped corpora so you don't train on your own model's exhaust (the "model collapse" failure mode).
Platform misuse detection: flag AI-generated content on social platforms, academic submissions, news distribution.
Internal attribution: trace which deployed model produced a given output for incident response.

What watermarking does not do

Prove the output came from a specific model run — that's TEE / zkML territory.
Survive adversarial laundering by a determined attacker.
Protect against models that ignore the watermark (open-weights models without enforcement).

For the full picture of when verification beats watermarking, see the "TEEs in production" section below.

C2PA and content provenance

C2PA (Coalition for Content Provenance and Authenticity) is an industry-standard cryptographic manifest format for media. Where watermarking embeds a signal inside the content, C2PA attaches a signed sidecar with the content's history.

What a C2PA manifest contains

Source device or software (signed by hardware key on Sony / Leica cameras, by software key on Adobe / OpenAI / Midjourney generation tools).
Edit history (each transformation appends a signed action).
Embedded references to source assets.
The signing certificate chain back to a recognized root.

When a user views an image in a C2PA-aware viewer (Adobe products, some browsers, Truepic), the manifest is verified and displayed.

What C2PA does well

Provenance, not authenticity: it tells you what the manifest claims the history was; the cryptographic signatures prevent in-flight tampering.
Composable: a generative image can carry a C2PA assertion "produced by DALL·E 3 on date X" alongside camera-original manifests.
Standards-track: adopted by OpenAI for DALL·E outputs, Adobe Content Credentials, Microsoft for Bing Image Creator, Sony / Leica / Nikon for high-end cameras.

What C2PA does not do

Survive stripping: removing the sidecar removes the provenance. C2PA-aware viewers can flag this ("no manifest found"), but most platforms re-encode images and discard manifests.
Prove a real-world claim: a signed manifest from "AcmeCam" only matters if AcmeCam's signing key is in your trust store. The PKI problem is unsolved at consumer scale.
Work on text: C2PA targets media (images, video, audio). For text, watermarking is the closest analog.

C2PA is best understood as the "TLS certificate" of generative media: pervasive when present, gone when stripped, only as trustworthy as the issuing authority. Pair with watermarking for in-content redundancy.

TEEs in production at NVIDIA and Anthropic

NVIDIA Confidential Compute on H100, H200, and B200 is the dominant GPU-side TEE. Production patterns in 2026:

NVIDIA's stack

H100 / H200 / B200: per-GPU memory encryption with keys derived from a per-chip secret fused at manufacture. The CPU (or hypervisor) sees ciphertext when DMA'ing GPU memory; the GPU SMs see plaintext.
GPU attestation report: signed by NVIDIA's root key, lists firmware versions, driver hash, VBIOS, and the public component of the per-instance ephemeral key.
Confidential VM mode: the GPU only accepts work from an attested confidential VM (typically Intel TDX or AMD SEV-SNP on the host). End-to-end, the workload runs on hardware whose state has been cryptographically asserted to a remote client.

Throughput overhead: 3–7% on Hopper, dropping toward 2–4% on Blackwell as memory-encryption engines were widened. See NVIDIA datacenter GPUs for hardware context.

Cloud availability

Azure Confidential GPU VMs (NCC H100 v5): GA in 2024, default for some Microsoft regulated-industry tiers.
Google Cloud Confidential GKE Nodes with H100/H200: available.
AWS offers Nitro Enclaves on CPUs and has previewed GPU confidential computing.
Oracle Cloud ships H100 / H200 with NVIDIA CC by default in select shapes.

Anthropic's public posture

Anthropic has publicly committed to deploying confidential compute as part of its trust-and-safety infrastructure for sensitive customer deployments. Public details are limited; the operational pattern is enterprise/regulated tier with TEE-attested inference and sealed audit logs. OpenAI has made similar gestures for its enterprise offering. The frontier hyperscalers' default consumer tiers still run without per-request attestation — verification at that scale is not free, even at 5%.

The model fingerprinting layer

Attestation of the hardware is only useful if you also know what model was loaded. Production patterns:

Hash the model weights at load time, include the hash in the attestation report.
Sign model versions in a registry (HuggingFace, NVIDIA NIM, internal MLflow).
Reject inference requests if the runtime hash doesn't match the registry-signed value.

This is the closest thing the field has to a working analog of Proof-of-Learning: you can't (yet) prove the model was trained honestly, but you can prove that the bytes you're running match a publicly-signed manifest.

When verification matters in agent workflows

Agents make 10–100× more LLM calls per task than chat. Verification economics change.

The cascading-trust problem

An agent's final action is a function of every intermediate model call. If any single call was substituted, parameters tampered, or output forged, downstream steps may look reasonable but actually be wrong. Reputation-based trust does not compose across N steps the way single-call trust does — each step is a fresh opportunity for divergence.

For the broader infrastructure pattern see agent serving infrastructure and reasoning-model serving.

Per-step vs end-to-end attestation

Two patterns:

Per-call attestation: each LLM invocation produces a TEE attestation; the agent runtime keeps a signed chain. Audit-friendly, cost ~5% per call. Used for regulated workflows.
Session-level attestation: attest the agent runtime once; rely on session continuity for intermediate calls. Cheaper, weaker. Suitable when the agent runtime itself runs inside a TEE.

For frontier high-stakes deployments (medical diagnostics, legal analysis, autonomous trading), per-call attestation is becoming standard.

Tool-use verification

Agents call external tools (search, code execution, databases). Verifying the LLM call doesn't verify the tool result. Patterns:

TEE-host the tool runtime (e.g., Cloudflare Workers TEE, Fly.io confidential VMs) so tool outputs carry their own attestation.
Sign tool responses with a tool-runner key; agent runtime verifies before consuming.
Log every tool call into an append-only signed log for post-hoc audit.

This is the agent analog of audit logging in distributed systems: cheap to add, invaluable when something goes wrong.

Output watermarking for agent text

If an agent generates text that ultimately reaches a human (a report, an email, a code comment), apply output watermarking. Then the recipient — or a downstream platform — can independently confirm the text came from an AI agent rather than a human, without needing to trust the agent operator.

When agents don't need this

Internal automation with low blast radius (e.g., a CI bot writing release notes).
Research prototypes.
Agents whose outputs are reviewed by a human before any action is taken.

For everything else, expect verification to migrate from "nice to have" to baseline as the EU AI Act and sector regulators catch up.

Approach 1: redundant execution

The simplest verification: run the same request on N independent providers. If they all return the same output, trust it. If one diverges, distrust that one.

How it works

client → request → splitter
                   ↓
                   ├→ provider A → response A
                   ├→ provider B → response B
                   └→ provider C → response C
                                    ↓
                                    quorum check
                                    ↓
                                    return majority

Trust assumption

Not all N providers are colluding. With N=3, you trust that at least 2 are honest. Standard Byzantine fault tolerance math: tolerate (N-1)/3 failures.

Cost

N× compute. Expensive at scale.

When it works

High-stakes individual requests (e.g., financial decisions).
Audit-required deployments where the cost is justified.
Bootstrapping trust on a new provider (sample early requests redundantly).

When it doesn't

Most production traffic. The N× cost is prohibitive at any reasonable scale.
Subtle degradations: if all providers use the same lower-quality variant (e.g., AWQ INT4 instead of FP8), they all "agree" but all wrong.

Determinism caveat

Inference isn't fully deterministic across providers. Different GPUs, drivers, or batching schemes produce slightly different outputs (especially with non-zero temperature). "Quorum match" requires either:

Greedy decoding (temperature=0, deterministic).
Tolerance window (outputs match within X% similarity).
Logit comparison instead of token comparison.

This adds operational complexity.

Approach 2: Trusted Execution Environments (TEE)

TEEs use hardware-level guarantees: the CPU/GPU can prove that a specific computation ran on specific hardware in a specific isolated environment.

NVIDIA Confidential Computing

H100 and H200 support Confidential Computing. Mechanisms:

Memory encryption: GPU memory is encrypted; the host OS (or hypervisor) can't read it.
Attestation: the GPU produces a cryptographic attestation that contains hardware identity, firmware version, and configuration.
Isolation: the workload runs in a "confidential VM" mode that's protected from the hypervisor. For the underlying hardware capabilities, see NVIDIA datacenter GPUs.

Intel TDX (CPU-side)

Intel's Trust Domain Extensions provide similar guarantees on the CPU side. Used for the "host" portion of inference (orchestration, networking, etc.) when that needs to be confidential too. AMD's equivalent is SEV-SNP, which uses per-VM encryption keys and an attestation path through the AMD Secure Processor.

How it works in practice

1. Client establishes secure channel with TEE.
2. Client uploads model + sends prompt encrypted to TEE.
3. TEE produces an attestation: "this hardware identity, this firmware, this code is running."
4. Client verifies attestation against expected values (hardware vendor, firmware version, code hash).
5. TEE runs inference, returns output.

If the attestation doesn't match expectations (different hardware, modified firmware, modified code), client rejects.

Trust assumption

NVIDIA, Intel, AMD (the hardware vendors) are trusted. Their attestation mechanisms are honest.

This is a strong assumption for high-stakes use cases. For most users it's acceptable — you already trust your hardware vendor in many other ways.

Cost

~2-5% overhead from encryption and attestation. Negligible for most workloads.

Limitations

Side-channel attacks: TEEs have historically been vulnerable to various microarchitectural side-channels. Fixed in current generations but worth monitoring.
Hardware vendor trust: if NVIDIA or Intel were compromised, TEEs offer no protection.
Firmware mutability: if the firmware can be updated to a malicious version, attestation is meaningless. Production deployments lock firmware.

When TEEs work

Compliance (HIPAA, financial regulations) where hardware-attested execution satisfies auditor requirements.
Multi-tenant serving where tenant data must be isolated from the operator.
Decentralized GPU networks where providers' hardware can be attested.

io.net, Akash, and similar are integrating NVIDIA Confidential Computing for "verifiable" tier offerings.

Approach 3: Zero-knowledge proofs (ZK)

Cryptographic proof that a computation produced a specific output, without revealing the inputs or weights.

The pitch

"I ran model M on input X and got output Y. Here's a cryptographic proof. You can verify the proof in milliseconds without seeing X or M's weights, and without trusting me."

This sounds magical. For small computations it works. For LLMs, it doesn't (yet).

The cost problem

ZK-proving systems (Groth16, Plonk, STARKs) have proving overhead of 1000-10000× the underlying computation. See Chen et al.'s zkML survey (arXiv:2403.00735) for the full taxonomy and benchmarks across proving systems.

A single Llama-3 70B inference takes ~30ms. Proving that inference happened correctly: ~5 minutes. Verification: ~50ms.

The 5-minute prover time is the killer. For production inference, this is unworkable.

Active research areas

Specialized ZK circuits for transformers: optimize the proving system for the specific operations in attention and MLP. Tools like ezkl compile ONNX models directly to Halo2 circuits.
Approximate ZK: prove that inference happened approximately correctly within some bound. Less rigorous but cheaper.
Hardware acceleration: ZK provers on GPUs or specialized chips. Improving fast.

A related primitive is Proof-of-Learning (Jia et al., arXiv:2103.05633), which targets verifying training trajectories rather than inference — useful for attesting model provenance separately from each forward pass.

By 2027-2028, expect ZK-of-LLM-inference to drop to 100-1000× overhead. Not yet production-grade for most use cases.

When ZK is useful today

Very small models (under 100M parameters).
Specific verifiable claims (proving a model output meets a threshold without revealing inputs).
Off-line audits where 5-minute proving time is acceptable.

For mainstream verification, ZK is a future technology.

Approach 4: Proof of Sampling (PoSP)

A statistical compromise: instead of proving every inference, prove a small random sample. If the provider can't predict which samples will be audited, they have to behave honestly always.

How it works

1. Provider runs all inference requests normally.
2. Auditor randomly samples 1% of requests for re-execution on independent hardware.
3. If the audit re-execution differs from the provider's claim, the provider is penalized.
4. With economic stakes (slashing of staked tokens), this is enough to enforce honesty.

Why it works

Provider's expected value calculation:

Cheat on every request: save C cost per request, but get caught with probability P (≈ sample rate). If caught, lose stake S.
Expected gain per cheat: C - P × S.
For C = $0.01 (savings per cheat), P = 1%, S must be > $1 (100× per-request cost). Trivial to enforce.

Trust assumption

The auditor is trusted (or there's a market of competing auditors). The randomness in sampling can't be predicted by the provider.

This is much weaker than ZK's cryptographic guarantee but stronger than nothing. For many production scenarios, sufficient.

Cost

Provider: ~0% additional overhead (just runs honestly). Auditor: 1% extra compute (re-running samples).

Total system overhead: ~1-2%.

Limitations

Adaptive cheating: a provider that cheats only on requests they're sure won't be audited (e.g., specific user/tenant patterns). PoSP design must use unbiased sampling.
Quality degradation: if the cheater uses a slightly worse model that produces similar outputs, sampling may miss it. Combine with quality-floor checks.
Coordination cost: requires economic infrastructure (staking, slashing) for enforcement.

Production deployments

io.net's "Proof of Sampling" subnet on Bittensor uses this approach. Several decentralized GPU networks have adopted variants — see decentralized GPU compute for the broader marketplace context. The fraud-proof-style alternative is opML (Conway et al., arXiv:2401.17555), which posts inference results on-chain with an optimistic challenge window rather than mandatory sampling.

Comparison and trade-offs

Approach	Compute overhead	Trust assumption	Production ready?
Redundant execution (N=3)	200%	No collusion among N	✅ for high-stakes
TEE (NVIDIA CC)	~5%	Hardware vendor honest	✅
ZK proofs	100,000-1,000,000%	Cryptographic primitives	❌ for LLMs
Proof of Sampling	~1-2%	Unpredictable random sampling	✅

Picking by use case

Compliance-driven (HIPAA, financial regs): TEE. Hardware attestation is what auditors recognize. For the serving stack this typically wraps, see LLM serving and agent serving infrastructure.

Decentralized GPU marketplaces: PoSP. Lightweight enough to apply to all traffic. Sampling-based verification interacts directly with the tail-latency problem (Dean & Barroso) — auditor straggler time becomes user-visible if you're not careful.

High-value individual requests: redundant execution. Cost is justified.

Cryptographic guarantee specifically required: ZK, but accept the cost.

Most production: don't bother. If your provider has reputation and SLA, that's adequate trust for most use cases.

Verifiable inference in 2026 — infographic on cryptographically proving that AI models ran correctly, covering the black-box problem with inference APIs, the commit / execute / prove / verify high-level flow, techniques used (zero-knowledge proofs, interactive proofs, commitment schemes), use cases (crypto and web3, finance, healthcare, enterprise AI, consumer apps), ecosystem examples (Giza, EZKL, Modulus Labs, RISC Zero, Inco, Turing), challenges (performance overhead, hardware requirements, proof size, model support, user experience), and trade-offs across security, latency, proof size, and cost. — **Verifiable inference at a glance.** Without verification, you have to take a provider's word that they used the right model, didn't tamper with inputs or outputs, and ran the computation correctly. Verifiable inference adds a cryptographic proof — commit to model and inputs, execute, prove correct execution, and let anyone verify the proof without re-running the model. Techniques span ZK proofs (privacy-preserving), interactive proofs (efficient and scalable), and commitment schemes (tamper-evident). The space is evolving fast across crypto, finance, healthcare, and enterprise AI — but every system trades off security, latency, proof size, and cost. Pick the right balance for the use case.

Production deployments in 2026

Centralized providers

OpenAI, Anthropic, Google: trust based on reputation and contracts. No cryptographic verification.
AWS, GCP, Azure GPU instances: TEE support available (NVIDIA Confidential Computing) for customers who request it.

Decentralized providers

io.net: PoSP subnet on Bittensor for verifiable inference. TEE rollout in 2026.
Akash: TEE support via NVIDIA Confidential Computing.
Render Network: redundant execution + reputation system.

Hybrid

Many production deployments use a hybrid: hyperscaler for default traffic, decentralized for cost-sensitive workloads, with PoSP or redundancy for high-stakes requests.

opML: optimistic verification for ML

opML (Conway et al., 2024) borrows the optimistic-rollup pattern from Ethereum scaling. The model provider publishes a claimed output along with a commitment to the execution trace. The output is accepted as valid by default, but anyone with a stake can challenge the claim within a challenge window (typically 1-7 days). On challenge, the disputed step is replayed on a neutral verifier and the lying party is slashed.

Why opML is interesting

The serving-time overhead is essentially zero — the prover just runs the model and publishes a hash. The verifier cost is bounded by how often disputes actually happen, which in equilibrium should be near zero because lying is unprofitable when slashed. This is the only verification model that scales to frontier LLMs at near-native speed today.

Why opML is limited

The challenge window means outputs are not finally verified for hours-to-days, which is unsuitable for real-time use cases. opML works for batch workloads, agent transactions that settle on-chain over time, and audit-after-the-fact verification of training jobs. Not a replacement for TEE in interactive inference. The closest 2026 production use: a handful of Bittensor subnets and Ora Protocol's ML verification layer.

opML vs zkML cost comparison

For a Llama-3 70B inference call (256 input, 128 output tokens):

Approach	Prover overhead	Verifier cost	Settlement time	Production-ready for LLMs?
Plain execution	1x	n/a	Immediate	n/a
TEE (NVIDIA CC)	1.03-1.07x	Microseconds (attest verify)	Immediate	Yes
Proof of Sampling	1.01-1.05x	1% sample replay	Statistical	Yes
opML	1.00x	Replay on dispute only	Hours-to-days	Partial (batch only)
zkML (snark)	10,000-1,000,000x	Milliseconds	Immediate	No (too expensive)
Redundant N=3	3x	Comparison	Immediate	Yes

The headline: only zkML gives both cryptographic strength and immediate settlement, and it is the most expensive by orders of magnitude. Every production deployment in 2026 picks a different point on this curve based on threat model.

Watermarking benchmarks: SynthID vs Kirchenbauer vs Aaronson

Measured detection performance on standard benchmarks (MarkMyWords, RAID), 2025-Q4 numbers:

Scheme	Quality impact (MMLU delta)	Detection AUROC clean	AUROC after paraphrase	AUROC after translation roundtrip	Min output length for reliable detection
Kirchenbauer (soft)	-0.3 to -0.8 pts	0.98	0.72	0.51	~200 tokens
Kirchenbauer (hard)	-1.5 to -2.5 pts	0.99	0.81	0.58	~100 tokens
SynthID Text	-0.1 to -0.3 pts	0.96	0.78	0.55	~200 tokens
Aaronson cryptographic	-0.05 to -0.1 pts	0.94	0.65	0.42	~400 tokens
MarkMyWords learned	-0.5 pts	0.97	0.83	0.62	~150 tokens

Read: SynthID Text is the best quality-vs-detection trade-off, which is why it ships in production at Google. All schemes degrade substantially under paraphrase or translation. The Aaronson scheme (announced by OpenAI's then-research alignment lead, never released) had the lowest quality impact but the weakest robustness — it relies on the attacker not having computing budget to laundered the text, which is increasingly unrealistic.

The Soft / Hard / Cryptographic taxonomy

Soft watermarks (Kirchenbauer soft, SynthID) bias logits toward a pseudorandom "green list" at sampling time. Detectable via z-test on green-token frequency. Quality impact small; robustness moderate.

Hard watermarks (Kirchenbauer hard) restrict sampling to the green list. Higher detection signal per token; larger quality impact. Used for low-stakes contexts where detection matters more than quality.

Cryptographic watermarks (Aaronson scheme, planted-trapdoor variants) use a cryptographic pseudorandom function to make the watermark indistinguishable from a true random draw without the secret key. Theoretically strongest, practically weakest because they degrade quickly with any token-level edits.

When verifiable inference matters

Compliance and regulation

HIPAA: protected health information must be processed in compliance-attested environments. TEE provides this.

Financial regulations: model decisions affecting trades must be auditable.

EU AI Act: high-risk AI deployments require traceable execution.

Decentralized marketplaces

When you don't know the provider, you need verification. Without it, the marketplace's economic incentives lean toward race-to-the-bottom on cheating.

Multi-tenant SaaS with sensitive data

Customer A's data shouldn't be visible to operator or to customer B. TEE provides isolation.

Audit-required workloads

Anything where post-hoc audits ask "did this AI make this decision correctly?" Verifiable inference provides the audit trail.

High-value individual decisions

A single request that determines a $1M loan approval, a medical diagnosis, or a legal analysis. Verification cost is justified.

When it doesn't

Casual chat

The user is testing a chatbot. If the response is reasonable, that's enough. Verifiable inference adds friction.

Internal tools

You trust your own infrastructure. Verifying it against itself adds no value.

Research and prototyping

Iteration speed beats verification. Verify later if needed.

When the cost exceeds the value

For low-stakes inference at $0.001/request, adding 5% overhead for TEE costs more than the request itself. Not worth it.

The open research questions

ZK that scales to LLMs

Current ZK provers are 1000-10000× slower than the underlying compute. Specialized circuits for transformers may close this gap. By 2027-2028, ZK-of-LLM-inference may be production-viable.

Cross-provider verification standards

Different providers use different verification mechanisms. Cross-provider audit (verify a request that ran on io.net using AWS infrastructure) doesn't have a standard.

Verifiable training

Inference is one thing; training is harder. Proving that a model's weights came from honest training on claimed data is much more complex. Active research area.

Confidential serving with shared resources

TEEs work for single-tenant. Multi-tenant TEE serving (multiple customers sharing one GPU with cryptographic isolation) is partial in 2026.

Quality-floor enforcement

Verifying what model ran isn't the same as verifying it produced correct output. Quality-floor enforcement (the model's output meets some quality bar) is much harder.

A short history of verifiable computation

1980s-1990s: zero-knowledge proofs introduced theoretically (Goldwasser, Micali, Rackoff). Primarily theoretical curiosity.

2010-2015: zk-SNARKs become practical for small circuits. Used in Zcash and similar privacy-preserving crypto.

2018-2020: TEEs (Intel SGX, AMD SEV) deploy in production for confidential computing. Used in financial and healthcare contexts.

2022-2023: NVIDIA introduces Confidential Computing on H100. First TEE for GPUs.

2023-2024: zk-of-LLM-inference research begins seriously. Modulus Labs, others publish proofs of small-model inference.

2024-2025: Proof of Sampling protocols deploy on Bittensor and similar networks.

2026 (current): TEEs in production for compliance-driven workloads. PoSP for decentralized marketplaces. ZK still research-grade for LLMs.

The trajectory: stronger guarantees becoming more practical over time. Each technique matures; new ones emerge.

TEE deep dive: NVIDIA Confidential Computing

NVIDIA Confidential Computing is the production TEE solution for GPU workloads.

Architecture

Three layers of protection:

Encrypted memory: GPU memory is encrypted; host can't read.
Attestation: cryptographic proof of hardware + firmware state.
Isolated execution: workload runs in a "confidential VM" mode.

Attestation flow

1. Client establishes TLS with attestation service.
2. GPU produces attestation including:
   - Hardware identity (chip serial, manufacturer).
   - Firmware version hash.
   - Driver version hash.
   - User code hash.
3. Client verifies attestation against expected values.
4. If matched: client trusts the GPU and uploads sensitive data.

The attestation service can be NVIDIA's (default) or a customer-operated one.

What's protected

Model weights (encrypted in transit and at rest in GPU memory).
Input prompts and outputs (encrypted).
Inference computation (isolated from host OS).

What's not protected

Side-channels: timing, power, microarchitectural state.
Malicious firmware (if the firmware itself is compromised).
NVIDIA's signing keys (root of trust).
Physical attacks (someone with physical access).

Performance overhead

Memory encryption: ~2% overhead.
Attestation: ~50-200ms one-time at startup.
Steady-state inference: ~5% slower than non-confidential mode.

For most workloads, acceptable.

Integration

NVIDIA's NIM and Confidential Computing SDK integrate. Cloud providers (AWS, Azure, GCP) offer TEE-enabled GPU instances.

Application-level changes: minimal. The TEE wraps the existing inference stack.

ZK proofs deep dive: where the cost comes from

Zero-knowledge proofs of LLM inference are theoretically possible but currently impractical. Why.

Circuit size

A single Llama-3 70B forward pass involves ~140G arithmetic operations. ZK proving systems (Plonk, STARKs) have proving cost roughly 100-1000× the underlying operations.

So proving one inference: ~14 trillion proof-circuit operations. On a fast prover: minutes to hours.

Specialized circuits for transformers

Active research: optimize the proving circuit for transformer-specific operations:

Matrix multiplication.
Softmax.
Attention.
Layer normalization.

Each optimization can reduce constants by 2-10×. Cumulatively, maybe 1000× speedup over generic ZK.

Approximate ZK

Instead of proving exact computation, prove "computation produced approximately this output, within some bound."

Trades cryptographic strength for prover cost. May reach practical levels (10× overhead) at the cost of weaker guarantees.

Hardware acceleration

ZK provers on GPUs, FPGAs, or specialized chips. Can reduce wall-clock time by 100×.

NVIDIA's GPUs aren't optimized for ZK; specialized chips (e.g., from Aleo, Risc Zero) might be 10-100× faster.

When ZK becomes practical for LLMs

Estimate: 2027-2029 for production-viable ZK of LLM inference. Approximate ZK earlier (maybe 2026-2027).

For now: TEE or PoSP are the practical options.

Proof of Sampling deep dive

PoSP is the leading practical approach for verifiable inference at scale.

Mechanism

1. Provider runs inference normally on all requests.
2. Auditor randomly samples 1% of requests.
3. For each sampled request, auditor re-executes on independent infrastructure.
4. If auditor's output matches provider's: provider passes.
5. If not: provider penalized (slashed).

Statistical guarantees

For sample rate p and per-request cheating cost c (savings to provider):

Expected gain per cheat: c.
Expected loss per detected cheat: large (e.g., 100c).
Expected net gain per cheat: c - p × 100c = c(1 - 100p).

For p = 1%: expected net gain is c(1 - 1) = 0. Cheating is break-even.

For p > 1%: cheating is unprofitable. Provider plays honest.

This is a rational-actor argument; assumes providers are economically rational.

Auditor selection

Who runs the audits? Three patterns:

Centralized auditor: a designated trusted party (e.g., the marketplace operator). Simplest but introduces a trust dependency.

Distributed auditors: a network of auditors, randomly selected per audit. Reduces concentration risk but adds coordination cost.

Cryptographic randomness: audit selection seeded by verifiable randomness (VRF, threshold signatures). Removes auditor-side trust assumptions.

Cost economics

PoSP overhead: ~1-2% (sample rate × re-execution cost).

Compare to:

Redundant execution (3×): 200% overhead.
TEE: ~5%.
ZK: 10,000%+ (currently).

For decentralized marketplaces serving cost-sensitive workloads, PoSP's economics are compelling.

Quality verification challenges

Standard PoSP verifies what model ran. Doesn't verify correctness of output (the model could be honest but wrong).

For quality verification, additional techniques:

Quality floor checks (output meets some quality bar).
Cross-validation with reference models.
Human review of audit samples.

Production deployment patterns

Real production patterns combining these techniques.

Pattern 1: TEE for compliance

A healthcare AI provider serves diagnostic AI to hospitals.

HIPAA requires hardware-attested execution.
Solution: NVIDIA Confidential Computing on H200 instances at AWS.
Each hospital connects via TLS, verifies attestation, sends encrypted prompts.
Cost overhead: ~5%. Acceptable for regulated workload.

Pattern 2: PoSP for decentralized inference

A decentralized GPU network (io.net or similar) serves cost-sensitive inference.

Providers stake tokens to participate.
1% of requests re-executed by auditor pool.
Cheaters detected statistically; staked tokens slashed.
Cost overhead: ~1-2%. Scales to millions of requests.

Pattern 3: Redundant execution for high-stakes

A financial services firm uses LLM for trade decisions.

Each trade-relevant LLM call runs on 3 independent providers.
If outputs disagree, escalate to human review.
Cost overhead: 200%. Justified by financial impact of bad decisions.

Pattern 4: Hybrid (TEE + PoSP)

A SaaS provider serving multi-tenant B2B AI.

TEE provides isolation (tenants don't see each other's data).
PoSP-style sampling validates that the right model runs.
Combination: ~5% overhead, both isolation and correctness guarantees.

Pattern 5: ZK for specific claims

Niche use case: prove a specific claim about an inference (e.g., "the score exceeded threshold T") without revealing inputs.

Currently rare in production due to ZK cost. Specific use cases (e.g., privacy-preserving credit scoring) may justify it.

Open challenges

Areas where verifiable inference is still maturing.

Cross-provider verification

If a request runs on io.net but you want to audit using AWS infrastructure: standardized verification protocols don't exist yet. Each network has its own audit mechanisms.

Quality verification at scale

PoSP verifies execution, not output quality. Verifying that "the LLM gave a good answer" is much harder. Active research area.

Multi-step reasoning verification

For agents and multi-turn workflows, verifying each step is more complex than single-shot inference. Each step may use different models or providers.

Confidential MoE

MoE routing reveals information about input (which experts a token routes to). For confidential workloads, this can leak. Active research on private routing.

Verifiable training

Inference verification is one thing. Verifying that a model's weights came from honest training on claimed data is much harder. Federated learning has some primitives, but production verifiable training is an open problem.

Performance gaps closing

ZK overhead is dropping ~10× per year via algorithm and hardware improvements. By 2027-2028, may be production-viable for LLMs.

TEE adoption is broadening as more cloud providers offer it.

PoSP infrastructure is maturing but still nascent in 2026.

The bottom line

The trust gap is unavoidable: any time a model output crosses an organizational boundary, the operator gets to decide what you find out about how it was produced. Verifiable inference closes that gap by replacing trust with attestation. The single biggest lever is threat-model fit: pick the cheapest mechanism whose trust assumption matches what you actually need to prove. Over-engineering with zkML when a TEE suffices wastes compute; under-engineering with redundant execution when you need cryptographic provenance wastes the audit.

If you take only this away:

TEEs are the 2026 production default for compliance and regulated serving — ~5% overhead, hardware-attested execution, available on H100/H200/B200.
Proof of Sampling is the right answer for decentralized GPU marketplaces — economic verification at near-zero overhead.
zkML is research-grade for LLMs. Use it only for small models or on-chain settlement.
Watermarking and C2PA solve a different problem: detecting AI-generated outputs after the fact, not verifying execution.
Layers compose. A regulated agent in 2026 runs on TEE + watermarks output + emits signed audit traces — three orthogonal guarantees.

For the marketplace context where PoSP shines, read decentralized GPU compute. For where these attestations integrate into the serving stack, see LLM serving.

FAQ

Q: Do I need verifiable inference?

For most production applications, no. Reputation and SLA are adequate. Verifiable inference matters when you don't trust the provider, you have compliance requirements, or you have very high-stakes individual requests.

Q: What's NVIDIA Confidential Computing?

Hardware-attested execution on H100/H200/B200. The GPU produces a cryptographic attestation that the workload ran on a known-good configuration. TEE for the GPU.

Q: Can I do ZK on Llama-3 70B?

Not in production, no. ZK-proving an LLM forward pass is currently 1000-10000× slower. Research is closing this gap.

Q: How does PoSP differ from "just trust the provider"?

PoSP adds economic enforcement: a provider's stake is at risk if they're caught cheating. Without that, "trust the provider" is just a hope.

Q: Does redundant execution cost 3× my inference bill?

Yes. That's why it's reserved for high-stakes individual requests, not all traffic.

Q: TEE vs ZK — which is more secure?

ZK is theoretically stronger (cryptographic guarantee). TEE depends on hardware vendor honesty. For most threat models, TEE is sufficient and ZK is overkill.

Q: What's the role of blockchain in verifiable inference?

Mostly for economic enforcement (staking, slashing) and dispute resolution. The actual verification happens via TEE or sampling, not via blockchain itself.

Q: How do I integrate verifiable inference into my application?

Most platforms expose a "verifiable" tier alongside standard. Use the verifiable tier for relevant requests; standard for the rest. The integration is just an API flag.

Q: Will verifiable inference become standard?

For decentralized providers: yes, as the market matures. For hyperscalers: probably not (their reputation does the work).

Q: What about quality verification?

Distinct problem. Verifying what ran isn't verifying that the output was good. Quality is checked through evals, A/B tests, user feedback — not through cryptographic verification.

Q: How do TEE attestations get verified?

Hardware vendor publishes a cryptographic root of trust. Software libraries verify attestation against this root. Standard process; libraries handle it.

Q: Are there open-source verifiable inference frameworks?

Several research projects (Modulus Labs, Worldcoin's verifiable AI) explore this. Production-grade open-source frameworks are nascent.

Q: Is watermarking a substitute for verifiable inference?

No — they answer different questions. Watermarking tells a third party "this looks AI-generated." Verifiable inference tells the requester "the right model actually ran." A platform detecting watermarked text doesn't know which provider produced it; a TEE attestation doesn't help you find AI text scraped into a training set. Use both.

Q: Can watermarks be removed by paraphrasing?

Light paraphrasing leaves enough signal in soft watermarks like Kirchenbauer or SynthID to be detectable; aggressive paraphrasing or round-tripping through another model often strips it. The MarkMyWords benchmark is the standard robustness reference.

Q: What is C2PA and how does it relate to deepfake detection?

C2PA is a signed-manifest standard for media provenance: it records what tool produced an asset and what edits were applied, with cryptographic signatures. It's complementary to deepfake detection — provenance proves "this image carries a manifest from camera X dated Y"; detection proves "this image is or isn't AI-generated." Production systems use both.

Q: How does NVIDIA Confidential Compute compare to AMD SEV-SNP and Intel TDX?

NVIDIA CC protects the GPU side (memory encryption, attestation). Intel TDX and AMD SEV-SNP protect the CPU side (the orchestration VM). Production confidential inference uses both: a TDX or SEV-SNP host VM running a CC-enabled GPU workload. End-to-end ciphertext path with two independent vendor root-of-trust signatures.

Q: What's the role of model fingerprinting in trust?

Hardware attestation only tells you a known firmware ran on a known chip — it doesn't say which model. Fingerprinting (hash the weights, compare to a signed registry) closes that gap. Without it, an operator with valid TEE attestation could still serve the wrong model.

Q: Does watermarking degrade output quality?

Modern schemes (SynthID Tournament Sampling, Kirchenbauer with calibrated bias) show negligible perplexity / win-rate impact on standard benchmarks. The earlier rule of thumb that "watermarking costs ~1 point of MMLU" no longer holds for production schemes.

Q: Does verifiable inference work for streaming?

Mostly. TEE-based verification streams normally; the attestation happens at session start, then tokens stream out.

For PoSP, audits happen post-hoc, not on streaming traffic. Auditing of completed streams.

ZK doesn't currently work for streaming due to proving overhead.

Q: What about model quality verification?

Different from execution verification. Verifying what model ran is execution; verifying output is good is quality.

Quality verification is harder. Approaches:

Cross-validate against a reference model.
Apply quality-floor checks (perplexity, factuality).
Periodic human review.

No general-purpose "prove this answer is correct" mechanism exists.

Q: How does attestation handle firmware updates?

When firmware updates, the attestation hash changes. Clients have to update their expected values.

For TEE deployments: have a managed list of acceptable firmware hashes, updated as new firmware versions are validated.

Don't auto-update without re-validating. New firmware could compromise security guarantees.

Q: Can TEEs be defeated?

Theoretically: side-channel attacks (timing, power, microarchitectural). Practically: very hard, requires physical access or sophisticated attacks.

For most threat models, TEE is sufficient. For extreme threats, redundant TEE + multi-vendor attestation provides additional defense.

Q: How does verification scale?

PoSP scales gracefully — sample rate is a constant fraction of traffic.

TEE scales 1:1 with traffic — every request gets attestation.

ZK doesn't scale yet — too expensive per request.

Redundant execution scales linearly with redundancy factor.

Q: What's the cost of verification?

Per-request:

TEE: ~5% overhead.
PoSP: ~1-2% overhead (1% sampling, ~100% audit cost on samples).
Redundant (3x): 200% overhead.
ZK: 10,000-100,000% overhead.

Cost varies by workload.

Q: How does verification interact with privacy?

TEE provides isolation: data isn't visible outside the TEE. Strong privacy.

PoSP doesn't add privacy: auditor sees the request to re-execute. Need separate privacy mechanism if needed.

ZK theoretically combines verification with privacy (zero-knowledge of the input). In practice, too expensive for LLMs.

Q: What if my workload has tight latency SLA?

TEE: fine, ~5% latency overhead. PoSP: fine for most requests, but audit re-execution adds latency for 1% of requests. ZK: doesn't fit; proving time exceeds typical SLA.

Choose based on SLA tolerance.

Q: How does this work for fine-tuned/proprietary models?

Same as base models. TEE attests to whatever is loaded on the GPU. PoSP audits against the same model.

For proprietary models: weight isolation is critical. TEE ensures the operator can't extract weights.

Q: What about model tampering — verifying the model wasn't modified?

Checksum of the model weights. Attestation includes the model hash. Client verifies the hash matches expected.

For TEE: built-in. For non-TEE deployments: need to manually verify.

Q: Can verifiable inference help with model alignment?

Indirectly. Verification ensures the operator runs the agreed-upon model. If the agreed-upon model is well-aligned, verification preserves that.

Doesn't help with making models safer or more aligned in the first place. Verification is about execution integrity, not model behavior.

Q: How is verifiable inference relevant to autonomy / agents?

Autonomous agents make many LLM calls. Verifying each provides accountability:

Did the agent really reason this way?
Did the LLM response really come from the claimed model?

For high-stakes agent decisions, verification chain is valuable.

Q: What's the latency overhead of TEE attestation?

Initial attestation: ~50-200ms. Once trusted, no per-request overhead.

For session-based interactions (chat, agents), the attestation is amortized across many requests.

For one-shot RPC-style use, the per-call overhead is more significant.

Q: Will frontier providers (OpenAI, Anthropic) offer verifiable inference?

Some are exploring it. Anthropic has discussed confidential computing. OpenAI hasn't publicly committed.

For most users of frontier APIs, verifiable inference isn't a priority. The provider's reputation and contracts substitute for cryptographic verification.

Q: How does verifiable inference change developer experience?

Minimal in the easy case (TEE): wrap the existing API client with attestation verification. Maybe one extra config flag.

Bigger in the hard case (ZK): need to integrate proving infrastructure, accept latency, manage proving costs.

Most production deployments stick with TEE for the operational simplicity.

Q: What is the attestation flow for NVIDIA Confidential Compute end-to-end?

The client opens a TLS connection to the inference endpoint. Before any inference data crosses, the endpoint produces an attestation report signed by NVIDIA's device key. The report contains: GPU UUID, firmware versions (VBIOS, GSP, drivers), confidential-VM CPU attestation (Intel TDX quote or AMD SEV-SNP attestation report), public key for the session's ephemeral encryption. The client verifies the chain back to NVIDIA's root, checks the firmware versions against an allow-list, and only then begins sending data encrypted under the session key. Total handshake adds ~50-200 ms one-time per session. For long sessions (chat, agent), the per-request overhead is essentially the encryption cost only.

Q: Can I run TEE-attested inference on cloud GPU spot instances?

Sort of. Cloud providers offering Confidential GPU VMs (Azure NCC H100 v5, GCP Confidential GKE H100, Oracle OCI Confidential Compute) currently bind attestation to specific VM types that are not always available on spot. Azure's spot tier supports confidential compute since 2024-Q4; AWS does not yet offer GPU confidential VMs on spot as of mid-2026. Practical: production deployments use reserved or on-demand confidential GPU VMs and only mix spot for non-confidential workloads.

Q: How does PoSP scale to 1B inference requests per day?

The sampling rate is the lever. At 1% sample rate, auditing 1B requests requires re-executing 10M requests on auditor nodes. With auditor capacity of 50 requests/sec/GPU and 50% utilization, that needs ~2300 GPUs of audit capacity for a 1B/day deployment. That's about $5M/year in auditor capex amortized. Most networks instead use adaptive sampling: 5-10% sample rate during initial onboarding of new providers, dropping to 0.1-0.5% for high-reputation providers. The economic deterrent doesn't require uniform sampling — it requires unpredictable sampling.

Q: How does verifiable inference interact with multi-LoRA serving?

The challenge: TEE attests the base model weights. If a request loads a LoRA adapter on top, the runtime hash changes. Production solution: attest the base model at startup, then have the runtime sign each adapter load event (adapter hash + tenant ID + timestamp) into an audit log that gets included in the response metadata. The client can verify the base attestation and then trust the runtime's signed adapter log. PoSP needs an analogous extension — the audit must replay with the same adapter loaded. See multi-tenant LoRA serving for the underlying serving pattern.

Q: How is C2PA actually deployed at scale?

OpenAI signs DALL-E 3 outputs with a C2PA manifest signed by an OpenAI-owned key registered with the C2PA trust list. Adobe Content Credentials signs Firefly and Photoshop AI-edited outputs. Microsoft Bing Image Creator does the same. The trust list is small: maybe 20 active issuers as of mid-2026. The fundamental gap is that social platforms (Twitter, Facebook, Reddit, TikTok) strip C2PA manifests during upload re-encoding, which means the signed provenance reaches users only when they download original files. Major effort underway by the content authenticity initiative (CAI) to get social platforms to preserve manifests; adoption is slow.

Q: Is SynthID open source?

SynthID Image is open-sourced by Google DeepMind under the Apache 2.0 license. SynthID Text was open-sourced in 2024 (the algorithm, not the production-tuned configurations Google uses internally for Gemini). Detectors are available; calibration data is partially published. Other labs (Anthropic, OpenAI) have not open-sourced production watermarking schemes as of mid-2026.

Q: How do model fingerprints work in practice?

The simplest version: compute SHA-256 of the safetensors files at load time and compare to a signed manifest. The manifest is signed by the model publisher (HuggingFace org key, NVIDIA NIM signing key, Anthropic / OpenAI production key for closed weights). Production stacks like vLLM and SGLang support load-time hash verification with a --model-hash flag pointing at the expected digest. More sophisticated: Proof-of-Learning (Jia et al., 2021) attempts to prove the training process produced these weights, but is computationally expensive and not yet production-deployed.

Q: Does TEE protect against the cloud provider snooping?

Mostly yes, with one caveat: TEE attestation establishes a hardware-rooted trust boundary that excludes the hypervisor, host OS, and cloud-provider administrators. Encrypted memory means provider operators cannot dump GPU state. The caveat: the cloud provider still controls the supply chain (the hardware they install, the firmware they sign-off on). A cloud provider could in principle ship a tampered GPU; NVIDIA's attestation chain mitigates this by tying attestation to NVIDIA's signed firmware, not the cloud's. For most threat models, TEE on a major cloud is "trustless enough" for healthcare and financial compliance.

Q: What's the verification posture of frontier model providers?

OpenAI, Anthropic, and Google operate centralized inference with reputational trust as the default. Anthropic publicly commits to TEE for enterprise/regulated tier; OpenAI for ChatGPT Enterprise and select API customers; Google for Vertex AI Confidential. The consumer-facing free tiers (ChatGPT free, Gemini, Claude.ai) are not TEE-attested per request. Closed-weight providers do not expose model fingerprinting — you cannot verify which model checkpoint served your request beyond the version string they return.

Q: How does verifiable inference apply to retrieval-augmented generation?

The model attestation covers the LLM call but not the retrieval source. End-to-end RAG verification needs: (1) attested LLM execution (TEE/PoSP), (2) signed retrieval results (the vector DB returns retrieved chunks with cryptographic signature that the chunks match what's in the index), (3) audit log of retrieval queries. Production deployments in 2026 cover (1) but rarely cover (2) and (3). For agent workflows where retrieval-source integrity matters, expect to build (2) and (3) custom. See RAG production architecture.

Q: Can audit logging be a poor-man's substitute for cryptographic verification?

For internal use, yes. Signed structured logs of every model call (input hash, output hash, model version, timestamp, signing key) provide post-hoc verification at near-zero overhead. The trust model is weaker — the operator can edit logs unless they're written to an append-only log service or anchored periodically to an external chain (e.g., Sigstore's Rekor transparency log, or a blockchain commitment). For agent workflows and compliance, audit logs are often sufficient; for actual untrusted-provider scenarios, audit logs alone are not sufficient because the operator can lie.

Q: What's the state of zkML for small models?

Production-deployable for models up to ~10M parameters as of mid-2026. EZKL (Modulus Labs) generates SNARKs for ResNet-class image classifiers in ~30-60 seconds, with verification in milliseconds. Giza, RISC Zero, and several other stacks support similar scales. For LLMs, even 1B parameters is impractical: proving time would be hours-to-days per inference, and proof sizes would exceed gigabytes. Active research on lookup arguments and folding schemes (Nova, Hypernova) suggests 1B-parameter LLMs may become provable in minutes by 2027-2028, but production-deployable zkML for frontier LLMs (70B+) is not on a credible near-term timeline.

Q: How does verifiable inference interact with streaming responses?

The standard TEE pattern wraps the whole inference call; for streaming, the attestation covers the initial handshake and the encrypted channel covers the streamed tokens, but you cannot verify token-by-token mid-stream. PoSP doesn't apply per-token either — the auditor replays the full request. The practical implication: streaming and verifiable inference coexist fine, but you verify at the request level, not the token level. For agent workflows that need per-step verification, structure the agent as many short requests each independently verified.

Q: Will verifiable inference become regulated / mandatory?

Likely in regulated industries first. EU AI Act provisions for high-risk AI systems already imply audit and traceability requirements that TEE attestation satisfies cleanly. NIST AI RMF (Risk Management Framework) recommendations push toward attestable execution for safety-critical AI. Healthcare and financial regulators are leaning toward requiring hardware-attested execution for AI systems making consequential decisions. Voluntary adoption is ahead of mandate as of mid-2026; mandate is coming within 2-3 years for specific verticals.

Q: How does verifiable inference relate to constitutional AI / RLHF model behavior?

It doesn't, directly. Verifiable inference attests that a specific model ran on specific input. It does not attest the behavior of that model — whether the model has been aligned, jailbroken, or has hidden behaviors. Model behavior verification is a separate (and harder) problem space involving evals, red-teaming, and interpretability. The two stack: TEE attests execution, evals attest behavior. Both are needed for a complete trust story but they don't substitute for each other. See production safety guardrails.

Q: Can verifiable inference prove a model wasn't fine-tuned by the provider?

Indirectly. If you hash the weights at load time and the hash matches the publisher's signed manifest, you have evidence the provider didn't swap in different weights. This catches gross substitution but not subtle changes (a provider could provide differently-quantized weights that hash differently but produce broadly similar outputs). The robust answer: combine model-fingerprint attestation with periodic eval-based verification — sample outputs and compare against expected behavior to detect quality regression that hash-checking would miss.

Q: What's the operational cost of running PoSP audit infrastructure?

For a 10M-requests-per-day network with 1% audit rate: ~100k re-executions per day. At 1 second per request on average and 50% GPU utilization, that's ~25 audit GPUs. Capex: $750k-$1M for the audit fleet. Opex: $100-200k/year in power and bandwidth. Compared to a $50M/year inference network, audit cost is ~2-3% — broadly aligned with the 1-2% PoSP overhead figure that gets quoted. The economic deterrent only works if slashing exceeds the expected fraud profit, which requires stake sizes and slashing parameters tuned to the workload.

Q: How does the verification overhead scale with model size?

TEE: roughly constant ~3-7% regardless of model size, because the overhead is memory-encryption bandwidth, which scales with memory access patterns rather than model parameters. PoSP: roughly constant in percentage but absolute audit cost scales with model — larger models cost more to re-execute. zkML: cost scales superlinearly with model size, which is why the technique fails at LLM scale. Redundant execution: scales linearly with N redundant runs.

Q: Does verifiable inference work for multi-modal models?

Yes, with the same primitives. TEE attestation covers the entire model regardless of modality. Vision tokens, audio tokens, and text tokens are equally inside the attested boundary. The output-side primitives differ: text watermarking is a different scheme from image watermarking (SynthID Image vs SynthID Text), but both compose under a unified TEE attestation. See multimodal serving.

Real-world verifiable AI deployments

What's actually running with verifiable inference in 2026.

Healthcare AI provider

A radiology AI service serves diagnostic models to hospitals.

HIPAA requires hardware-attested execution.
Solution: NVIDIA Confidential Computing on H200 instances.
Each hospital establishes TLS with attestation; all data encrypted in GPU memory.
5% latency overhead, accepted for compliance.

Decentralized inference network

io.net or similar serves Llama-3 70B inference.

1% of requests sampled by audit network.
Failed audits → provider stake slashed.
~1% overhead. Statistical security guarantees.

Financial services

Trading firm uses LLM for analysis.

High-stakes decisions; redundant execution for critical requests.
3 independent providers; quorum decision.
200% overhead. Justified by financial impact.

Confidential SaaS

Enterprise AI vendor serves multiple competitors.

TEE ensures tenant isolation (Customer A can't see Customer B's data).
Per-tenant attestation.
~5% overhead.

Research deployment with ZK

Academic project demonstrating ZK-of-LLM for small models.

1B-parameter model with custom ZK circuit.
Proving time: ~30 seconds per inference.
Not production-grade but shows the technique works.

Comparing approaches by use case

Use case	Recommended approach	Why
Healthcare/HIPAA	TEE	Compliance accepts hardware attestation
Multi-tenant SaaS	TEE	Strong isolation guarantees
Decentralized marketplace	PoSP	Cost-effective at scale
High-stakes finance	Redundant + TEE	Belt-and-suspenders
Privacy-preserving claims	ZK (when ready)	Cryptographic guarantee
General production	None / reputation	Simpler, sufficient for most

The honest answer: most production doesn't need verifiable inference. Reputation and SLAs work. Use verification when you have a specific reason — compliance, decentralization, or extreme stakes.

How verifiable inference fits into agentic systems

Modern AI agents make many LLM calls per task. Verification has specific implications.

Trust chains

Each step in an agent's execution can be verified:

LLM call 1: TEE-attested.
Tool execution: separately verified.
LLM call 2: TEE-attested.
...

Result: end-to-end audit trail of the agent's behavior.

Per-step or end-to-end?

Per-step verification is more granular but more expensive.

End-to-end verification (just attest the final output) is cheaper but provides less accountability for intermediate steps.

For high-stakes agents: per-step. For low-stakes: end-to-end.

Cost economics

Agents make 10-100× more LLM calls than chat. Verification cost compounds.

For agents in high-stakes domains (financial, legal, medical), the economics may justify per-call verification. For consumer agents, often skipped.

Multi-provider agents

Agents may call multiple LLM providers (best model for each task). Cross-provider verification is harder than single-provider.

Standardized cross-provider verification protocols don't exist yet. Most agents use a single provider for verification simplicity.

Verifiable agent execution

Beyond LLM verification, the agent's code execution can be verified:

TEE for the agent runtime.
Logs of every action with cryptographic signatures.
Replay-based audits.

This is the future direction for high-stakes agentic systems.

Implementing verifiable inference: practical guide

How to actually integrate verifiable inference into a production system.

Step 1: identify what needs verification

Not every request needs verification. Categorize:

Always verify: regulated workloads, financial decisions.
Sometimes verify: random sampling for audit purposes.
Never verify: low-stakes background tasks.

Most deployments fall into "sometimes" — sample verification.

Step 2: pick the verification mechanism

For most:

TEE if compliance requires hardware attestation.
PoSP if running on decentralized infrastructure.
Redundant for highest-stakes individual requests.

ZK is research-grade; skip for now.

Step 3: integrate at API gateway

Verification happens at the API gateway, not the inference engine. Gateway:

Tags requests with verification policy.
Routes to verified-capable infrastructure.
Logs verification metadata.

Step 4: monitor verification metrics

Track:

Verification overhead (latency, cost).
Failed verifications (red flag).
Audit results.

Step 5: handle failures

What happens when verification fails:

TEE attestation invalid: refuse to use that GPU.
PoSP audit disagreement: investigate provider, slash if confirmed.
Redundant disagreement: escalate, possibly to human review.

Common pitfalls

Adding verification but not actually checking results (security theater).
Neglecting to update expected attestation values when firmware changes.
Not handling the audit-failed case explicitly.

Reference architecture

                 Client
                    ↓
              API Gateway
              (verification policy)
                    ↓
          ┌─────────┴─────────┐
          ↓                   ↓
    Verified pool        Standard pool
    (TEE/PoSP)           (no verification)
          ↓                   ↓
         GPU                  GPU
          ↓                   ↓
         Audit                  -
         (sample %)

This pattern is straightforward; most teams can implement in a few weeks.

Threat models

Different verifiable inference techniques address different threat models.

Model substitution

Provider runs a cheaper model than agreed.

TEE: prevents (model loaded into TEE is the agreed-upon one).
PoSP: detects (audits catch outputs that don't match the agreed model).
Redundant: detects (multiple providers' outputs disagree if one substitutes).
ZK: prevents (proof attests to specific weights).

Output tampering

Provider runs the agreed model but modifies output.

TEE: prevents (output produced inside TEE).
PoSP: detects.
Redundant: detects.
ZK: prevents.

Data exfiltration

Provider stores or shares user data.

TEE: prevents (data encrypted in transit and at rest in GPU).
PoSP: doesn't address.
Redundant: doesn't address.
ZK: addresses for inputs (zero-knowledge of inputs).

Quality degradation

Provider quietly uses lower-quality settings (e.g., higher temperature).

TEE: doesn't fully address (parameters can vary within attestation).
PoSP: detects if outputs differ.
Redundant: detects.
ZK: addresses.

Sybil attack

Single provider operates many identities.

TEE: irrelevant.
PoSP: vulnerable; sybils can collude.
Redundant: vulnerable.
ZK: irrelevant.

Sybil resistance is separate; uses identity-binding mechanisms.

Attack/defense matrix

Threat	TEE	PoSP	Redundant	ZK
Model substitution	✅ prevent	✅ detect	✅ detect	✅ prevent
Output tampering	✅ prevent	✅ detect	✅ detect	✅ prevent
Data exfiltration	✅ prevent	❌	❌	⚠ partial
Quality degradation	⚠ partial	✅ detect	✅ detect	✅ prevent
Sybil collusion	n/a	⚠ vulnerable	⚠ vulnerable	n/a
Side-channel attacks	⚠ partial	n/a	n/a	⚠ depends

For comprehensive threat coverage: combine TEE + PoSP. Each catches what the other misses.

Comparison with existing trust mechanisms

How verifiable inference compares to traditional trust.

Reputation

How web2 services build trust. Provider has reputation, contracts, SLAs.

Pros: simple, no overhead. Cons: requires bootstrapping reputation, vulnerable to insider threats.

For most deployments: reputation suffices. Verifiable inference adds value at the edges.

Audit logs

Cryptographically signed logs of operations.

Pros: post-hoc verification. Cons: doesn't prevent issues, only enables investigation.

Combines well with verifiable inference for layered defense.

Trusted third parties

External auditors verify operations.

Pros: established model. Cons: trust dependency on auditor.

For regulated industries: traditional. Sometimes combined with TEE for enhanced trust.

Insurance

Accept that bad things happen; insure against them.

Pros: simple risk management. Cons: doesn't prevent harm, just compensates.

Common for low-stakes deployments.

Comparison

For most production: reputation + audit logs are sufficient.

For decentralized or compliance-heavy: add verifiable inference.

For extreme stakes: layer multiple mechanisms.

Future directions for verifiable AI

Where this is going.

Improved TEEs

NVIDIA's Confidential Computing improves with each GPU generation. Rubin (2026-2027) likely adds:

Better side-channel protection.
Faster attestation.
More flexible isolation.

ZK acceleration

Hardware-accelerated ZK provers. Specialized chips (Aleo, Risc Zero, Zircuit) targeting practical ZK at scale.

By 2028-2029, ZK of LLM inference may be production-viable.

Standardized protocols

Cross-provider verification standards. Independent auditors. Industry consortiums.

OCP-like efforts may emerge for verifiable AI infrastructure.

Privacy-preserving inference

Beyond verification: inference where neither input nor output leak.

Combines TEE (for execution) with homomorphic encryption (for computation on encrypted data).

Currently impractical for LLMs. Active research.

Verifiable training

The hardest case. Verifying that a model was trained on claimed data, with claimed methodology.

Currently no practical solution. Active research, possibly 5-10 years out.

Regulatory mandates

EU AI Act, US NIST guidelines, sector-specific regulations may mandate verifiable inference for high-risk AI.

Plan for this; it's likely to expand.

Decentralized AI maturity

As decentralized GPU networks grow, verifiable inference becomes more critical. PoSP and TEE may become standard for serious decentralized deployments.

By 2027-2028: most decentralized inference may be verifiable by default.

TEE-based deployments in detail

How real production deployments with TEEs work.

Reference architecture

Client
  ↓ TLS
Attestation server
  ↓ verifies
TEE-enabled GPU instance
  ↓ encrypted
Inference engine
  ↓
Encrypted output
  ↓ TLS
Client

Each step has security guarantees.

Setup steps

Provision TEE-enabled hardware: NVIDIA Confidential Computing-compatible GPU.
Configure firmware and drivers: ensure attestation works.
Set up attestation service: verifies hardware/firmware identity.
Deploy inference engine in confidential VM: workload runs in attested environment.
Client integration: client establishes TLS, requests attestation, validates response.
Production traffic: encrypted requests flow through.

Most cloud providers offer this as a managed service.

Performance considerations

TEE adds:

Memory encryption: ~2% overhead.
Initial attestation: 50-200ms.
Steady-state inference: ~5% latency vs non-TEE.

For most workloads: acceptable.

Use cases

Healthcare AI (HIPAA compliance).
Financial services (regulatory requirements).
Multi-tenant SaaS (tenant isolation).
Defense / government workloads.

Limitations

Cost: TEE-enabled instances priced at premium.
Availability: not all providers offer it.
Side-channel vulnerabilities: theoretical but real.

For most use cases: TEE is the right choice when verifiability matters.

Verifiable inference for compliance

How verifiable inference satisfies specific regulatory requirements.

HIPAA

Requires:

Data encryption in transit and at rest.
Access controls.
Audit logs.

TEE provides:

Memory encryption (data at rest in GPU).
Hardware-attested execution.
Audit logs of attestation events.

For most HIPAA scenarios: TEE-based inference satisfies.

GDPR

Requires:

Data minimization.
Right to access/delete.
Cross-border transfer restrictions.

TEE doesn't directly address these. Need additional controls (data residency, audit logs).

SOC 2

Requires security and availability controls.

TEE-based inference can be a control. Combined with monitoring and incident response.

EU AI Act (high-risk AI)

Requires:

Traceability of AI decisions.
Human oversight capabilities.
Robustness and accuracy.

Verifiable inference (with audit logs) supports traceability. TEE supports robustness.

For high-risk AI: combination of verifiable inference + comprehensive governance.

Sectoral regulations

Financial: SEC requirements for trade decisions.
Aviation: FAA requirements for safety-critical AI.
Medical devices: FDA requirements for diagnostic AI.

Each sector has specific requirements. TEE-based execution is increasingly accepted as part of compliance toolkits.

Future regulatory trends

Stricter requirements for AI accountability.
Mandatory audit trails for some uses.
Cross-border data restrictions.

Plan for evolving regulatory landscape.

Implementation patterns for different scales

How to implement verifiable inference at different scales.

Single-tenant, high-stakes

Pattern: dedicated TEE-enabled hardware. All requests verified.

Cost: ~5% premium over non-verified.

Example: financial trading firm, healthcare AI provider.

Multi-tenant SaaS

Pattern: TEE per tenant for isolation. Optional per-request attestation verification.

Cost: ~10% premium (tenant isolation overhead).

Example: enterprise AI platform serving multiple customers.

Decentralized marketplace

Pattern: PoSP for cost-sensitive traffic. TEE for premium tier.

Cost: PoSP is ~1-2%, TEE is ~5%. Tiered pricing reflects.

Example: io.net or similar with multiple service tiers.

Hyperscaler API

Pattern: reputation + SLA. Optional TEE for compliance customers.

Cost: TEE customers pay premium.

Example: AWS Bedrock with confidential computing options.

Open-source inference deployment

Pattern: client-side validation of attestation. PoSP for community trust.

Cost: minimal (open-source software).

Example: distributed Llama deployments.

When to skip verification

For most low-stakes deployments: skip. Reputation suffices.

For research and experimentation: skip. Quality > verifiability.

For internal tools: skip. You trust your own deployment.

For consumer-facing AI without high stakes: skip. UX > verification.

Standardization efforts

Industry efforts to standardize verifiable inference.

NVIDIA Confidential Computing

NVIDIA-led standard for GPU TEE. Most production implementations use this.

Intel TDX

CPU-side TEE standard. Often combined with NVIDIA CC for end-to-end confidential.

Open standards

OCP confidential computing standards.
W3C verifiable credentials (for attestation interop).
IEEE working groups on AI verification.

These are early but progressing.

Industry consortiums

Confidential Computing Consortium (Linux Foundation).
Open Compute Project AI working group.

Slow progress but moving toward interoperability.

Future state

By 2028:

Standard attestation formats across vendors.
Cross-cloud TEE portability.
ZK proof standards for AI inference.

Today: vendor-specific solutions. Migration paths emerging.

Building confidence in verifiable inference

How organizations actually develop trust in verifiable inference systems.

Step 1: pilot project

Pick a single workload. Implement verification. Validate it works.

Document everything: setup, costs, latency impact, reliability.

Step 2: gradual expansion

Apply to more workloads. Build operational expertise.

Step 3: standards and process

Document verification policies. Train staff. Embed in development lifecycle.

Step 4: vendor relationships

Establish relationships with TEE-capable vendors. Negotiate SLAs.

Step 5: continuous monitoring

Track verification metrics. Investigate anomalies. Update procedures based on lessons.

Step 6: external validation

Have third parties review your verification setup. Audit procedures.

For regulated industries: external audits are often required.

What success looks like

Verification adds < 10% latency.
Operational overhead is bounded.
Compliance auditors satisfied.
Engineering team comfortable operating it.

Achievable in 6-12 months with focused effort.

Common pitfalls

Treating verification as a one-time setup (it requires ongoing care).
Underestimating operational complexity.
Poor handling of failed verifications.
Lack of monitoring on verification metrics.

Avoid by treating verification as a first-class concern.

Glossary

Attestation: cryptographic proof of hardware/software state.
Byzantine fault tolerance: distributed system tolerating up to (N-1)/3 malicious nodes.
Confidential VM: VM with encrypted memory and isolated execution.
NVIDIA CC: NVIDIA Confidential Computing. TEE for GPUs.
PoSP: Proof of Sampling. Statistical verification.
Quorum: agreement among redundant providers.
Redundant execution: running same request on multiple providers.
Slashing: economic penalty for misbehavior in a staking system.
STARK / SNARK: types of zero-knowledge proof systems.
TDX: Intel Trust Domain Extensions. CPU-side TEE.
TEE: Trusted Execution Environment.
ZK / ZKP: Zero-knowledge proof.

References

Verifiable-inference primitives

opML: Optimistic Machine Learning on Blockchain — Conway et al., 2024. arXiv:2401.17555. Fraud-proof-style verification for off-chain inference; the canonical optimistic-rollup-inspired design for ML.
Zero-Knowledge Proofs of Training for Deep Neural Networks (zkML survey) — Chen et al., 2024. arXiv:2403.00735. Survey of practical zk-proof systems for ML and their cost overheads across Groth16, Plonk, STARKs, and Halo2.
Proof-of-Learning: Definitions and Practice — Jia et al., 2021. arXiv:2103.05633. Foundational treatment of verifying that training happened as claimed, distinct from per-inference verification.
ezkl — Zkonduit, 2023–present. github.com/zkonduit/ezkl. The most widely-used toolchain for compiling ONNX models to Halo2 zk-circuits.

Trusted execution environments

NVIDIA Confidential Computing on H100/H200 — nvidia.com/en-us/data-center/solutions/confidential-computing/. Memory encryption, attestation, and confidential-VM mode for GPU workloads.
Intel Trust Domain Extensions (TDX) — Intel, 2022. Intel TDX documentation. CPU-side TEE used to host the orchestration layer in confidential inference deployments.
AMD SEV-SNP — AMD, 2020–present. AMD SEV documentation. Per-VM memory encryption with Secure Nested Paging; the AMD counterpart to TDX.

Proof-of-Sampling and decentralized verification

Bittensor Yellowpaper — bittensor.com/whitepaper. Subnet architecture and incentive design that PoSP subnets build on.
io.net Proof of Sampling — io.net, 2024–present. PoSP implementation on Bittensor for decentralized verifiable inference.

Foundations

The Tail at Scale — Dean & Barroso, CACM 2013. research.google. Sampling-based verification adds an auditor path whose P99 behavior matters; this paper is the foundational treatment of why.
The Knowledge Complexity of Interactive Proof Systems — Goldwasser, Micali, Rackoff, 1989. The foundational zero-knowledge proof paper that underlies every modern zkML construction.

Watermarking and content provenance

A Watermark for Large Language Models — Kirchenbauer et al., 2023. arXiv:2301.10226. The canonical green-list / red-list LLM watermarking scheme; foundational for the entire literature.
MarkMyWords: Analyzing and Evaluating Language Model Watermarks — Piet et al., 2023. arXiv:2312.00273. The standard robustness benchmark for LLM watermarking — paraphrasing, translation, and quality trade-offs.
SynthID Text — DeepMind, 2024 (Nature). deepmind.google/technologies/synthid/. Production watermarking deployed in Gemini; Tournament Sampling scheme with public detector.
C2PA — Coalition for Content Provenance and Authenticity — c2pa.org. Open standard for signed media manifests; adopted by OpenAI, Adobe, Microsoft, Sony, Leica, Nikon.

ZK proofs of inference in detail

The state of zero-knowledge proofs for AI inference.

What ZK does

A ZK proof allows:

Prover proves inference was done correctly.
Verifier checks the proof.
No need to re-execute inference.

For AI: the prover demonstrates "this output came from this model on this input" without revealing the model weights or full computation trace.

Why ZK is hard for inference

LLMs are huge:

Billions of parameters.
Trillions of operations.
Each operation needs to be in the proof circuit.

ZK proof generation grows with circuit size — making LLM-scale proofs computationally intensive.

Current state

Production-ready ZK inference:

Small models (< 100M parameters): viable.
Medium models (1-10B): research, not production.
Large models (10B+): not feasible today.

For frontier LLMs: ZK inference isn't practical.

Why people pursue it anyway

Theoretical advantages:

Cryptographic guarantees (no trusted hardware).
Cross-organization trust.
Strong privacy properties.

When ZK becomes practical (years away), it could enable new use cases.

ZK approaches for AI

zk-SNARKs:

Succinct, non-interactive arguments.
Smaller proofs.
Trusted setup needed for many.

zk-STARKs:

Scalable, transparent.
No trusted setup.
Larger proofs.

Folding schemes:

Combine many proofs efficiently.
Active research.

For AI: STARKs and folding schemes are most promising.

Hybrid approaches

Combine ZK with other techniques:

ZK for small critical components.
TEE for full inference.
Sampling for additional verification.

Pragmatic approach for near term.

Production deployments

Companies in this space:

Modulus Labs (zkML toolkit).
Inference Labs.
Privasea (privacy-focused inference).

Most are research/early-production stage.

When ZK will be practical

Predictions:

Small models: production today.
Medium models: 2-3 years.
Large models: 5+ years.

Hardware acceleration (GPU/ASIC for ZK) will accelerate this.

Use cases waiting for ZK

Multi-organization AI.
Privacy-preserving healthcare.
Adversarial environments.
Trustless AI marketplaces.

For these: ZK could be transformative.

Verifiable inference cost analysis

Detailed cost analysis for verifiable inference.

Cost components

For TEE:

Hardware premium: 20-50% over standard.
Operational complexity: +10-20% engineering cost.
Verification overhead: ~5% latency.

For ZK:

Compute for proof generation: 100-1000x baseline (today).
Storage for proofs.
Verification compute (lower than generation).

For PoSP:

Sampling overhead: 1-2%.
Operational complexity: low.
No hardware premium.

Total cost of ownership

For a 1M-request/day deployment:

No verification:

Compute: $1k/day.
Total: $1k/day.

TEE:

Compute: $1.05k/day (5% latency overhead).
Hardware premium: $300/day.
Total: $1.35k/day (35% premium).

PoSP:

Compute: $1.02k/day (2% sampling).
Operational: $100/day.
Total: $1.12k/day (12% premium).

ZK (small model):

Compute: $50k+/day (today's cost).
Likely not viable.

When the cost is justified

For:

Healthcare: TEE often justified.
Finance: TEE common.
Legal: TEE valuable.
Consumer chat: usually no verification.

The cost-benefit is workload-specific.

Cost reduction over time

Costs are decreasing:

TEE: hardware getting more available.
ZK: dramatic improvements in efficiency.
PoSP: stable, low cost.

In 3-5 years: TEE may become standard, ZK becoming viable for medium models.

Hidden costs

Beyond compute:

Engineering time.
Operational overhead.
Compliance overhead.
Integration with existing systems.

These often dominate the visible costs.

Total economic case

For most: verifiable inference is justified when:

Cost premium < 20%.
Benefit clear (compliance, trust, etc.).
Operational team can support.

This rules out many casual use cases. Includes most regulated/high-stakes use cases.

Verifiable inference research priorities

What the field needs.

Better ZK efficiency

The biggest research priority. Make ZK practical for medium and large models.

Specific needs:

Faster proof generation.
Hardware acceleration (GPU/ASIC).
Recursive proofs / folding schemes.

Progress steady but slow.

TEE security

Strengthen TEE against:

Side-channel attacks.
Speculative execution exploits.
Supply chain attacks.

Industry investment significant.

PoSP formalization

Better protocols:

Game-theoretic analysis.
Adversary model formalization.
Privacy guarantees.

Active research area.

Standardization

Industry needs:

Common attestation formats.
Cross-vendor compatibility.
Audit standards.

Slow progress.

Tooling

Researchers and practitioners need:

Better ZK toolchains.
TEE testing frameworks.
Verifiable inference benchmarks.

Investment increasing.

Hardware

For ZK:

ZK-friendly hardware.
Acceleration specific to ML inference.

For TEE:

Better isolation primitives.
More efficient memory encryption.

Hardware-software co-design.

Theoretical foundations

Better understanding of:

Verifiability vs cost tradeoffs.
Composition of verification mechanisms.
Failure modes.

Theoretical work continues.

What this means for builders

Today: TEE is the production answer for most.

In 2-3 years: ZK becomes practical for more.

In 5+ years: verifiable inference is mainstream.

Plan and adapt.

Verifiable inference glossary

TEE: Trusted Execution Environment; hardware-protected execution.
Attestation: cryptographic proof of TEE state and identity.
ZK proof: zero-knowledge proof; prove statement without revealing details.
zk-SNARK: succinct ZK proof.
zk-STARK: scalable, transparent ZK proof.
Folding scheme: combine many proofs efficiently.
PoSP: Proof of Sampling Protocol; verification by sampling.
Confidential Computing: industry term encompassing TEE technologies.
NVIDIA Confidential Computing: NVIDIA's GPU TEE technology.
Intel TDX: Intel's CPU TEE.
AMD SEV: AMD's CPU TEE.
Slashing: penalty in decentralized networks for misbehavior.
Reputation: track record of provider reliability.
Threat model: assumptions about adversary capabilities.
Side channel: information leak via timing, power, etc.
Trusted Computing Base: components that must be trusted.
Sealed storage: data only accessible to specific TEE instance.
Quote: signed attestation.
Verifier: entity that checks proofs / attestations.
Prover: entity that generates proofs.
Deterministic execution: same input → same output bit-for-bit.

Verifiable inference vs alternative approaches

How verifiable inference compares to other trust mechanisms.

vs reputation-based trust

Reputation:

Easier to implement.
Doesn't require special tech.
Less robust against motivated adversaries.

Verifiable inference:

Provides cryptographic / hardware guarantees.
More robust.
Higher implementation cost.

For most: reputation suffices. For high stakes: verification.

vs auditing

Auditing:

Periodic third-party review.
Trust auditor + audited.
Doesn't prevent issues, only detects.

Verifiable inference:

Real-time verification.
Continuous.
Can prevent issues.

Complementary — both have role.

vs governance

Governance:

Process and accountability.
Important for organizations.
Doesn't directly verify.

Verifiable inference:

Technical mechanism.
Operates within governance framework.

Both needed.

vs insurance

Insurance:

Risk transfer.
Compensates for losses.
Doesn't prevent.

Verifiable inference:

Risk reduction.
Fewer losses to compensate.

Use both for high-stakes deployments.

vs alternative architectures

Federated AI, on-device AI, etc.:

Different trust models.
Different tradeoffs.

Verifiable inference fits in centralized / cloud architectures.

Combined approaches

Real-world deployments often combine:

Verifiable inference for execution.
Reputation for ongoing trust.
Auditing for periodic review.
Governance for organizational accountability.

Defense in depth.

Picking the right combination

Based on:

Threat model.
Stakes.
Resources.
Existing infrastructure.

Don't assume verification alone is enough.

Verifiable inference industry view

How different industries view verifiable inference.

Tech companies

Large tech: investing in TEE. ZK research. Some PoSP exploration.

Differentiation possible. Long-term commitment.

Cloud providers

All major clouds offer TEE-enabled GPU instances. Different naming, similar capabilities.

Differentiation: pricing, geographic availability, ease of use.

Inference platforms

Together.ai, Anyscale, etc.: inference platforms exploring verifiability as feature.

Some offer it as premium tier.

Decentralized networks

io.net, Bittensor, etc.: integrating verifiability as differentiator from hyperscalers.

PoSP and TEE both seen.

Healthcare

Aggressive adoption of TEE for clinical decision support.

Driven by HIPAA compliance.

Finance

Established TEE adoption for trading algorithms.

Established players have mature implementations.

Government / defense

Heavy TEE usage. Some classified ZK research.

Long-term investment.

Academia

Active ZK research. Some TEE security research. PoSP theoretical analysis.

Drives future capabilities.

Open-source communities

Tools and libraries:

TEE: well-supported.
ZK: emerging.
PoSP: protocols only.

Startups

Many in this space:

TEE infrastructure (specialized providers).
ZK tooling (Modulus Labs, etc.).
PoSP networks.

Investment flowing.

Standards

Slowly emerging. By 2030: significant standardization.

Geographic differences

US: TEE adoption broad.
EU: GDPR drives adoption.
China: own TEE ecosystem.
Elsewhere: variable.

Industry summary

The industry is taking verifiable inference seriously. Different segments at different stages.

For builders: track your industry's adoption. Plan accordingly.

Verifiable inference summary

Wrapping up the field.

Where we are (2026)

TEE-based inference: production-ready, growing adoption.
ZK-based inference: research / small-scale only.
PoSP: emerging, gaining traction in decentralized.
Hybrid approaches: practical reality.

Decision framework recap

Use TEE when:

Compliance / regulation requires.
Single-provider is acceptable.
Hardware available.

Use ZK when:

Cryptographic guarantees needed.
Small models.
Multi-organization trust.

Use PoSP when:

Decentralized infrastructure.
Cost-sensitive.
Sampling-based verification acceptable.

Use multiple when:

Defense-in-depth required.
Different threats need different defenses.

Cost summary

TEE: 5-30% premium over baseline.
ZK: not viable for frontier LLMs today.
PoSP: 1-5% premium.

Costs decreasing over time.

Next 3 years

Predictions:

TEE adoption broadens significantly.
ZK for medium models becomes viable.
PoSP standardizes in decentralized.
Industry standards emerge.

Practical advice

For most teams:

TEE-based deployment is the practical path.
Start with cloud provider's TEE offering.
Build operational expertise gradually.
Watch ZK research.

Final thoughts

Verifiable inference is moving from research to production. The tools are maturing.

For high-stakes AI: verification is becoming table stakes.

For consumer AI: verification is differentiator (and could be requirement soon).

Plan accordingly. The cost of verification is falling, the value is rising.

Verifiable inference FAQ extension

Verifiable inference for end users

How end users (vs operators) experience verifiable inference.

What they see

For TEE-based:

Possibly an attestation indicator.
Slight latency overhead (5%).
Otherwise transparent.

For ZK-based:

Attestation/proof at end of session.
Possibly verification time on client.

For PoSP:

Transparent (network handles verification).
Possibly visible quality metrics.

Trust model

Users trust:

Hardware vendor (TEE).
Cryptographic primitives (ZK).
Network/protocol (PoSP).

Different threat models, different trust assumptions.

User-facing verification

Some applications expose verification status:

"Verified inference" badge.
Cryptographic receipts.
Audit logs accessible to users.

This is differentiator for some products.

When users care

Users care about verification when:

Stakes are high (medical, financial).
They're skeptical of provider.
Compliance is required.

For most consumer use: users don't notice or care.

Education and communication

Communicating verification:

Don't overwhelm with technical details.
Focus on outcomes ("your data is protected").
Make it differentiator if it matters.

User-friendly messaging is important.

Future user experience

Likely evolution:

More products advertising verification.
Standardized verification badges.
User-controlled verification options.

By 2030: verifiable AI may be commodified.

Verifiable inference threat models

The threat models verifiable inference protects against.

Threat 1: Compromised provider

Provider runs different model than claimed.

TEE: detects via attestation.
ZK: detects via proof verification.
PoSP: detects probabilistically.

Threat 2: Cache substitution

Provider serves cached output instead of fresh inference.

TEE: protects via fresh execution check.
ZK: each proof for fresh execution.
PoSP: detects via verifier samples.

Threat 3: Cheap-model substitution

Provider serves smaller model output.

TEE: detects via attestation of model identity.
ZK: detects via proof of full model.
PoSP: detects via behavioral checks.

Threat 4: Selective output manipulation

Provider tweaks specific outputs.

TEE: protects via integrity check.
ZK: any tweak invalidates proof.
PoSP: detects via output sampling.

Threat 5: Side-channel leakage

Provider learns about input/output.

TEE: protects via memory encryption.
ZK: protects via zero-knowledge property.
PoSP: doesn't protect.

Threat 6: Denial of service

Provider refuses requests.

All approaches: don't directly address.
Mitigation: redundancy, multiple providers.

Threat 7: Slow inference (vs claimed performance)

Provider runs slowly.

TEE: doesn't directly detect.
ZK: proof generation time correlates.
PoSP: doesn't directly detect.

Threat 8: Insider threat

Provider's own employees.

TEE: limited protection (insiders may have keys).
ZK: protects against insiders without compute access.
PoSP: limited protection.

Threat 9: Supply chain

Hardware/software supply chain compromised.

TEE: depends on hardware vendor trust.
ZK: depends on circuit/prover correctness.
PoSP: depends on protocol design.

Threat 10: Future compute attacks

Quantum computing, etc.

TEE: hardware may need updates.
ZK: depends on cryptographic assumptions.
PoSP: doesn't depend on cryptography.

Threat-mitigation matrix

For each threat:

TEE strength: high.
ZK strength: highest (when applicable).
PoSP strength: medium.
Reputation strength: variable.

Pick based on threats you care about.

Defense in depth

Combine multiple approaches:

TEE for execution integrity.
PoSP for additional verification.
Audit logs for forensics.
Reputation for ongoing trust.

This is more robust than any single approach.

Verifiable inference attack surface

Where verifiable inference can be attacked.

Attack 1: Hardware compromise

Physical or software compromise of TEE hardware.

Difficulty: high (attacker needs physical access or kernel exploit).
Impact: full compromise.
Defense: hardware security features, monitoring.

Attack 2: Side-channel attacks

Timing, power, electromagnetic side channels.

Difficulty: medium-high.
Impact: data leakage.
Defense: side-channel resistant implementations.

Attack 3: Speculative execution attacks

Spectre, Meltdown-style attacks.

Difficulty: medium.
Impact: data leakage.
Defense: hardware mitigations, software workarounds.

Attack 4: Firmware attacks

Compromised firmware.

Difficulty: high.
Impact: full compromise.
Defense: firmware verification, secure boot.

Attack 5: Driver attacks

GPU drivers as attack surface.

Difficulty: medium.
Impact: significant.
Defense: driver verification, sandboxing.

Attack 6: Cryptographic attacks

Breaking cryptographic primitives.

Difficulty: very high (current crypto is robust).
Impact: full compromise.
Defense: cryptographic agility, post-quantum readiness.

Attack 7: Implementation bugs

Bugs in TEE implementation, ZK circuits, etc.

Difficulty: medium.
Impact: depends on bug.
Defense: extensive testing, formal verification.

Attack 8: Misconfiguration

Operators misconfigure security.

Difficulty: low.
Impact: significant.
Defense: secure defaults, documentation.

Attack 9: Social engineering

Tricking operators.

Difficulty: variable.
Impact: significant.
Defense: training, processes.

Attack 10: Supply chain

Compromised hardware in supply chain.

Difficulty: high.
Impact: severe.
Defense: vendor verification, diverse sourcing.

Threat modeling exercise

For your specific deployment:

List threats.
Estimate likelihood.
Estimate impact.
Apply defenses.
Accept residual risk or invest more.

This is standard security practice.

Defense maturity

Mature defense:

Multiple layers.
Continuous monitoring.
Incident response capability.
Regular testing.

Most TEE deployments are at this level.

Verifiable inference adoption patterns

How verifiable inference is being adopted.

Healthcare

Drivers:

HIPAA compliance.
Patient trust.
Liability management.

Pattern: TEE-based inference for clinical decision support.

Adoption: growing rapidly in 2025-2026.

Financial services

Drivers:

Regulatory requirements (SEC, etc.).
Auditability of AI decisions.
Risk management.

Pattern: TEE for trading models, sometimes ZK for specific compliance.

Adoption: established at major firms.

Legal services

Drivers:

Privilege concerns.
Client confidentiality.
Verifiability of legal AI.

Pattern: TEE for matter-specific AI.

Adoption: emerging.

Government / defense

Drivers:

National security.
Intelligence sharing.
Mission-critical applications.

Pattern: TEE with classified-environment integration.

Adoption: significant in classified, growing in unclassified.

Multi-tenant SaaS

Drivers:

Tenant isolation.
Compliance for diverse customer base.
Trust differentiator.

Pattern: TEE for premium tier.

Adoption: growing in B2B SaaS.

Consumer AI

Drivers:

Privacy demands.
Marketing differentiator.

Pattern: limited adoption today. Apple's on-device AI for privacy is related.

Adoption: limited; high-stakes consumer apps use it.

Crypto / DeFi

Drivers:

Trust without intermediaries.
Smart contract integration.

Pattern: ZK or PoSP for AI-driven DeFi.

Adoption: emerging.

Adoption barriers

What slows adoption:

Hardware availability.
Engineering complexity.
Cost.
Lack of industry standards.

These are improving over time.

What's accelerating adoption

Regulatory pressure.
High-profile incidents.
Cost reductions.
Better tooling.

Together, these will drive growth.

5-year outlook

By 2031:

Most regulated industries: verifiable AI is standard.
Many SaaS: verifiable AI is differentiator.
Consumer: limited but growing.
Decentralized: nearly all verifiable.

Verification is becoming mainstream.

Threat models per stakeholder

The trust gap looks different depending on which seat you sit in. A buyer of API inference, a regulator auditing deployed AI, a model-publisher whose weights have been licensed, a decentralized-marketplace user, and an enterprise procurement officer each face overlapping but distinct attack surfaces. Picking the right verification mechanism starts with naming whose threats matter.

Stakeholder 1: the API consumer

The buyer sends a prompt and pays per token. The attacks they care about: model substitution (paid for frontier, got 7B), quantization-without-disclosure (paid for FP8 weights, got AWQ INT4), cache substitution (got a stale neighbor's response), parameter tampering (higher temperature to reduce branching cost), and input retention (provider stores prompts contrary to contract). Reputation-based providers (Anthropic, OpenAI, Google) substitute contracts and brand for proof. Decentralized providers cannot make that substitution credibly, so PoSP or TEE is the default.

Stakeholder 2: the regulator or auditor

The regulator does not care about a specific request — they care about systemic integrity. The attacks they care about: training-data violations (the model was trained on prohibited data), behavior drift (deployed model differs from approved model), inadequate logging (operator cannot produce a complete audit trail), and inadequate isolation (PHI/PII leaked between tenants). TEE attestation + signed audit logs are the standard pattern; zkML and PoSP are mostly irrelevant at this layer because they answer the wrong question. EU AI Act and NIST AI RMF push for signed traceability rather than cryptographic correctness proofs.

Stakeholder 3: the model publisher

The publisher licenses weights to a deployer and worries about: weight exfiltration (deployer extracts and resells weights), unauthorized fine-tuning (deployer changes behavior without consent), and over-quota usage (deployer runs more inferences than licensed). TEE provides isolation that keeps the deployer from reading weights in plaintext. Model fingerprinting (load-time hash) plus signed usage logs gives accounting. zkML is sometimes pitched here but rarely deployed because the publisher already has economic leverage (license revocation).

Stakeholder 4: the decentralized-marketplace user

The user posts a job to a marketplace, gets responses from unknown providers, and pays in stake-bonded crypto. Attacks: cheap-model substitution, output forgery, Sybil collusion, selective dishonesty by tenant. PoSP plus slashing is the right primitive here; the economic model of unpredictable sampling and bounded loss exactly matches the threat. TEE adds a premium tier for high-stakes jobs.

Stakeholder 5: the enterprise procurement officer

Procurement signs the contract. Attacks they need to defend against: provider breach (data leakage), compliance failure (HIPAA / SOC2 / FedRAMP violation), regulatory enforcement (deployment halted due to inability to audit), and vendor lock-in (cannot port verification posture to a second provider). Procurement asks for TEE attestation, signed audit logs, model-fingerprint attestation, and SOC2 Type 2 reports. The actual primitive matters less than the documentary trail.

Threat-model-to-mechanism mapping

Stakeholder	Primary threats	Best primitive	Cost tolerance
API consumer (low-stakes)	Stale cache	Provider reputation	0% premium
API consumer (high-stakes)	Model substitution, quantization	TEE + fingerprint	5-10%
Regulator / auditor	Systemic, audit-trail	TEE + signed logs	High (mandated)
Model publisher	Exfiltration, over-use	TEE isolation + signed usage	5-10%
Decentralized user	Cheating, Sybil	PoSP + slashing + reputation	1-5%
Procurement (enterprise)	Compliance, lock-in	TEE + SOC2 + portable attest	10-30%
On-chain settlement	Cryptographic correctness	zkML / opML	Variable
Sensitive data, regulated	Snooping, side channels	TEE + audit + redundancy	20-50%
Multi-tenant SaaS	Tenant isolation	TEE per tenant	10-15%

The honest take: most procurement-driven decisions in 2026 buy a signed PDF, not a cryptographic proof. TEE attestation, signed model registries, and SOC2 are the actual artifacts. zkML is a research narrative; PoSP is the decentralized substitute.

zkML stack landscape in 2026

The zkML ecosystem has roughly five active stacks. None of them prove a 70B-parameter forward pass in under an hour; all of them work for vision classifiers, recommendation models, and small reasoning models. Quick survey.

EZKL (Zkonduit)

EZKL compiles ONNX models to Halo2 zk-circuits. The most mature stack. Production users include several DeFi protocols using on-chain ML for parameter governance. Proving a small ResNet (5M parameters) on commodity hardware: 30-90 seconds. Proving a 1B-parameter LLM: hours to days; impractical. EZKL's strength is tooling: ONNX in, circuit out, verifier in Solidity.

Risc Zero ML

Risc Zero takes a different approach: a general-purpose zkVM that runs RISC-V bytecode. Any model compiled to RISC-V (via Rust-based ML libraries) can be proven. Slower per-op than EZKL but far more flexible. Used for verifiable training of small models and for off-chain compute attestation.

Modulus Labs

Modulus pioneered zkML productionization in 2023 with "Leela vs the world" — a zk-proven chess engine. Their stack focuses on game-theoretic applications: prediction markets, AI-driven DeFi, content moderation with cryptographic appeal. Their 2026 product line is closed-source but they publish proofs alongside white papers.

Giza

Giza targets verifiable ML for prediction markets and DeFi-native applications, with a Cairo-based proving system. Cairo (StarkNet's language) has good ergonomics for arithmetic-heavy ML kernels. Giza models are deployed in a handful of on-chain market makers.

Halo2 / Plonky3 / Nova ecosystem

The underlying proving systems matter as much as the front ends. Halo2 (used by EZKL) is well-studied. Plonky3 and Nova are folding-scheme-based and theoretically scale better; production tooling for ML is still nascent. Folding schemes (Nova, HyperNova, ProtoStar) allow a long sequence of operations to be combined into one proof, which is exactly the structure of a transformer forward pass — they may unlock 1B-parameter zkML by 2027.

What zkML can and cannot do today

Model scale	Proving time	Verification time	Production use?
MNIST-class (1M params)	< 1 second	< 100 ms	Yes (demos, education)
ResNet-class (5-50M)	30-90 seconds	< 200 ms	Yes (DeFi, prediction markets)
BERT-base (110M)	5-15 minutes	< 500 ms	Niche; on-chain settlement
Small LLM (1B)	hours-to-days	< 1 second	No
Mid LLM (7B-13B)	days-to-weeks	< 1 second	No
Frontier LLM (70B+)	weeks-to-months projected	n/a	No

The trajectory: roughly 10× improvement per year via algorithm (folding, lookup arguments, GKR), and another 10-100× possible via specialized hardware (Aleo, Cysic, Fabric Cryptography ASICs). A credible forecast for "zkML on a 70B-parameter LLM in under one minute" is 2028-2030, contingent on both algorithm and silicon roadmaps holding.

For the trust posture in 2026 the answer is the same as 2025: use TEE or PoSP, watch zkML, plan to migrate when costs cross a useful threshold.

TEE silicon comparison: H100, H200, B200, Intel TDX, AMD SEV-SNP, ARM CCA

The TEE ecosystem now spans CPUs (Intel, AMD, ARM) and GPUs (NVIDIA, with AMD MI300X support partial). For confidential inference end-to-end you typically pair a CPU TEE for the host VM with a GPU TEE for the model. Choices have real implications for latency, cost, and threat coverage.

NVIDIA Confidential Computing — H100, H200, B100, B200

H100 introduced GPU-side memory encryption and attestation in 2023. H200 inherits the same architecture with more HBM. B100 and B200 (Blackwell, 2024–2025) widen the memory-encryption engines, reducing overhead. Reported overhead ranges 3-7% on Hopper, 2-4% on Blackwell. Attestation reports are signed by per-GPU keys derived from manufacturer-fused secrets; verification chains back to NVIDIA's root. NVL72 rack-scale deployments (GB200 NVL72) support cluster-wide attestation where the entire NVL72 acts as one confidential compute unit, useful for MoE inference and very large models that span multiple GPUs.

Intel TDX

Intel Trust Domain Extensions (TDX) provide per-VM memory encryption and attestation at the CPU side. TDX-enabled Xeon Scalable (Sapphire Rapids, Emerald Rapids, Granite Rapids) is the dominant CPU TEE for confidential AI workloads as of mid-2026. TDX attestation reports include CPU identity, microcode version, and VMM (hypervisor) measurements. Used as the host TEE underneath NVIDIA Confidential Computing on Azure and GCP.

AMD SEV-SNP

AMD's Secure Encrypted Virtualization with Secure Nested Paging. Per-VM keys derived from the AMD Secure Processor. SEV-SNP is the most-deployed CPU TEE in 2026 because EPYC Genoa, Bergamo, and Turin are widely used as host CPUs for GPU servers. Attestation chains back to AMD's root.

ARM CCA (Confidential Compute Architecture)

ARMv9-A introduced the Realm Management Extension (RME) and CCA. Realms are confidential VMs isolated from the hypervisor. Production deployments in 2026 are limited (NVIDIA Grace CPU pairs ARM cores with Hopper GPUs in confidential compute mode; some hyperscaler ARM SKUs add CCA). The trajectory points toward ARM-based confidential AI servers, especially for edge inference.

Apple Secure Enclave / Private Compute

Apple's Private Cloud Compute (announced 2024) deploys Apple-designed Mx-class server silicon with a Secure Enclave-rooted attestation system. Used for on-device + cloud hybrid inference for Apple Intelligence. Closed ecosystem; the attestation primitives are Apple-specific. Mentioned for completeness; not interoperable with other TEE stacks.

Cloud-provider availability matrix

Cloud	GPU TEE	CPU TEE	Notes
Azure	NCC H100 v5, H200 confidential VMs	Intel TDX, AMD SEV-SNP	Most mature confidential AI offering; GA since 2024
GCP	Confidential GKE Nodes with H100/H200	Intel TDX, AMD SEV-SNP	Vertex AI Confidential available
AWS	Preview tier for GPU confidential	Nitro Enclaves (CPU); SEV-SNP partial	GPU TEE lagging vs Azure
Oracle OCI	H100 / H200 with NVIDIA CC default	AMD SEV-SNP	Default for select regulated tiers
Lambda Labs	H100 / H200 with CC available	Intel TDX	Specialized AI-focused cloud
CoreWeave	H100 / H200 with CC available	AMD SEV-SNP	Default for healthcare customers

Attestation chain example: end-to-end confidential inference

Client opens TLS connection to inference endpoint.
Endpoint produces attestation bundle: (a) CPU TEE quote (TDX or SEV-SNP) for the host VM, (b) GPU attestation report (NVIDIA CC) for the GPU, (c) signed model-weight hash, (d) signed software-stack measurement (driver, vLLM/SGLang version, container hash).
Client verifies each signature against vendor roots: Intel/AMD root for CPU, NVIDIA root for GPU, publisher root for model, internal root for software stack.
Client establishes ephemeral session key inside the attested boundary.
Inference proceeds with encrypted payloads.

Total handshake overhead: 50-300 ms one-time per session. For chat sessions and agent workflows this is amortized; for single-shot RPCs it can dominate.

When TEE is not enough

Even with full attestation, four gaps remain: (1) side-channel attacks on shared L3 or memory bus, (2) silicon supply-chain attacks (tampered GPUs in the rack), (3) firmware downgrade attacks (forced rollback to a vulnerable firmware version), (4) malicious code inside the attested boundary (the operator runs attested-but-evil software). Production deployments add monitoring, firmware-allow-lists, and supply-chain attestation on top of the TEE primitives.

opML and optimistic verification networks

opML borrows from optimistic rollups: assume honesty, allow disputes during a window, slash on proven dishonesty. Conway et al. (arXiv:2401.17555) define the canonical construction; production deployments now include Ora Protocol's ML verification layer, several Bittensor subnets, and Hyperbolic's verifiable inference tier.

Core protocol

Prover runs inference and posts a commitment to the output (typically a Merkle root of the execution trace).
Verifiers have a fixed challenge window (1 hour for fast settlement, up to 7 days for strong guarantees) to dispute.
On dispute, the disputed step of the trace is replayed on a neutral verifier, and the lying party is slashed.
After the window closes without challenge, the output is final.

Why opML scales

The prover does no extra work — just inference plus a hash. The verifier does no work in the no-challenge case (which is the equilibrium). The only cost is the bonded stake and the challenge window. For workloads that can wait hours-to-days (batch processing, settlement, audit-after-the-fact), this is near-free verification.

Where opML breaks

Real-time chat does not fit a 1-hour challenge window. Agent workflows that need same-second tool invocation do not fit. opML is the right primitive for: (1) batch training-data verification, (2) on-chain agent settlement that happens over time, (3) compliance-grade audit trails. It is not a TEE replacement for interactive inference.

Comparison with PoSP and zkML

Property	opML	PoSP	zkML	TEE
Prover overhead	1×	1×	1000-10000×	1.05×
Challenge window	hours-days	none (statistical)	none	none
Real-time?	No	Yes	No	Yes
Trust assumption	Honest challenger exists	Unpredictable sampling	Crypto	Silicon vendor
On-chain integration	Native	Native	Native	Requires bridge

The honest read: opML, PoSP, zkML, and TEE are not substitutes; they occupy distinct points on the (cost, latency, trust) frontier. Production deployments pick the cheapest point that meets the threat model.

Watermarking adversarial robustness deep dive

Watermarking is the only verification primitive that survives the output crossing into the wild. Its robustness against adversaries determines whether it is useful.

Attack taxonomy

Paraphrase attacks. The attacker runs the watermarked text through a second LLM with a paraphrase prompt. Cost: one extra inference call. Effectiveness: soft watermarks lose 20-40% of detection signal under aggressive paraphrase; SynthID retains ~75% AUROC under MarkMyWords paraphrase tests. Hard watermarks retain more signal but introduce visible quality drops that defeat the purpose.

Translation round-trip. Translate to French, then back to English. Effectiveness: most watermarks drop to AUROC 0.5-0.6 (near random). The token distribution is rebuilt from scratch.

Synonym substitution. Replace ~10-20% of tokens with synonyms. Effectiveness: depends on which tokens; if the green-list tokens are preferentially replaced, detection drops sharply. Modern attackers use watermark-aware substitution and can drop detection AUROC by 30%.

Token-level edits. Insert, delete, or replace single tokens. Effectiveness: localized attacks have limited impact on z-test detection over long outputs; aggregated attacks can defeat short outputs.

Mixed-source attacks. Combine watermarked AI text with human-written text. Effectiveness: dilutes the watermark signal proportionally. Detection thresholds need adjustment.

Watermark-aware fine-tuning. The attacker fine-tunes a non-watermarking model on watermarked outputs and uses it to generate similar text without the watermark. Effectiveness: high; defeats most schemes. Defense: model fingerprinting plus output-side detection.

Robustness benchmarks: 2025-2026 numbers

Attack	Kirchenbauer (soft)	SynthID Text	Aaronson	MarkMyWords (best)
Clean	0.98	0.96	0.94	0.97
Light paraphrase	0.85	0.86	0.78	0.91
Heavy paraphrase	0.72	0.78	0.65	0.83
Translation round-trip	0.51	0.55	0.42	0.62
20% synonym substitution	0.74	0.79	0.68	0.85
Mixed 50% human	0.83	0.86	0.77	0.89
Adaptive watermark-aware	0.55	0.60	0.45	0.68

Numbers are illustrative, drawn from public benchmarks; production schemes are tuned for specific deployments. The headline: no scheme survives a determined adversary with compute budget. Watermarking is useful for opportunistic detection (training-corpus hygiene, social-platform flagging) but cannot defeat motivated laundering.

Image and video watermarking

Image watermarking embeds the signal in pixel space (SynthID Image, StegaStamp, Tree-Ring). Video watermarking adds temporal redundancy. Both survive common transformations (JPEG re-encoding, cropping, resizing) up to a threshold but break under heavy filtering or generative re-synthesis. SynthID for video is deployed for Veo outputs and survives most platform re-encoding.

Combined detection strategies

Production detection in 2026 combines: (1) watermark detection, (2) statistical analysis (perplexity, burstiness, sentence-length distribution), (3) C2PA manifest checks, (4) classifier-based detection (GPTZero-style learned detectors), (5) cross-modal consistency. No single signal works; the combination achieves AUROC 0.92-0.95 on standard benchmarks even under moderate adversarial pressure.

Decentralized inference verifiability (Bittensor, Atoma, Marlin)

Decentralized GPU networks need verification by construction because they cannot rely on provider reputation. The leading networks in 2026 take different approaches.

Bittensor PoSP subnets

Bittensor hosts dozens of subnets, several of which provide verifiable inference. The pattern: miners run inference, validators replay samples, scoring drives token emissions. Subnets vary in scoring sophistication: some use BLEU-style output comparison, others use logit-level verification. The economic deterrent comes from emission penalties on detected dishonesty.

Atoma Network

Atoma builds explicitly around verifiable inference with a TEE-first architecture. Providers run TEE-attested NVIDIA hardware; jobs route to attested nodes; signed attestation bundles ship with responses. The differentiator is end-to-end attestation including the routing layer.

Marlin Protocol

Marlin provides a verifiable compute layer with optimistic verification and TEE attestation. Marlin's "Oyster" enclave service is a managed TEE substrate; users get attestation receipts without operating TEE infrastructure themselves.

Hyperbolic, Together AI, Lilypad

Hyperbolic offers a verifiable inference tier using opML. Together AI's verifiability features focus on enterprise customers and route attested workloads to TEE pools. Lilypad targets decentralized compute with a sampling-and-slashing model.

Marketplace verification economics

A decentralized network with N providers, sample rate p, slash factor s, per-request fraud savings c. Honest behavior is rational when p × s > c. For typical numbers (c = $0.01 per cheat, s = $1-10 per detected cheat, p = 1-5%), honest behavior is enforced. The harder problem is colluding-Sybil resistance: a single operator running multiple identities can absorb slashing across identities and still profit. Defenses include identity-binding (KYC), reputation decay, and validator rotation.

Cross-network verification

A user submitting to multiple networks for redundancy faces a coordination problem: networks use different attestation formats, scoring rules, and stake currencies. Standardization efforts (Verifiable Compute Alliance, several W3C drafts) aim to unify attestation envelopes, but production cross-network verification in 2026 still requires custom adapters. For the marketplace context see decentralized GPU compute.

Verification by workload type (training, inference, RAG, agent, fine-tuning)

Verification primitives apply differently across the AI pipeline. Picking by workload matters.

Training verification

The hardest case. Verifying that a model was trained on claimed data with claimed methodology has no production-grade primitive. Proof-of-Learning (Jia et al., 2021) defines a checkpoint-based verification scheme but is expensive and not robust to sophisticated adversaries. Most production "verifiable training" in 2026 means: signed training-data manifests, signed code commits, TEE-attested training runs producing signed weight artifacts, and audit-log retention of all training events. Cryptographic proof of training trajectory is open research. See distributed LLM training and checkpoint storage and recovery.

Inference verification

The most-studied case. TEE for compliance, PoSP for decentralized, opML for on-chain settlement, zkML for niche on-chain or small-model verification. The choice follows threat model and latency tolerance.

Fine-tuning verification

Lies between training and inference. Verifying that a fine-tuning run produced these weights from this base + this dataset uses similar primitives to training verification: signed datasets, TEE-attested fine-tuning, signed output weights. LoRA adapters complicate this: the adapter is small but the verification chain must capture base + adapter + merge order. See multi-tenant LoRA serving for the serving side.

RAG verification

Retrieval-augmented generation has three layers to verify: (a) the LLM call (standard inference verification), (b) the retrieval (the retrieved chunks really exist in the indexed corpus), (c) the indexed corpus itself (provenance of the documents). Production deployments cover (a); (b) and (c) require custom infrastructure — typically signed retrieval responses from the vector DB and signed document manifests. See RAG production architecture.

Agent verification

Agents compound the problem: many LLM calls, many tool invocations, many state updates. Per-call verification (each LLM call TEE-attested, each tool call signed) provides full audit trails but is expensive. Session-level verification (attest the runtime, trust within-session) is cheap but weaker. The 2026 pattern for high-stakes agents: per-call TEE + signed tool calls + append-only audit log. See agent serving infrastructure.

Workload-to-mechanism table

Workload	Primary primitive	Secondary	Notes
Pretraining	Signed data manifests + TEE runs	Audit logs	No cryptographic proof of trajectory
Fine-tuning	Signed datasets + TEE	Model fingerprint	LoRA adds adapter chain
Single-shot inference	TEE attestation	PoSP for decentralized	Most-studied case
Streaming inference	TEE channel	n/a	Verify at request level
RAG	TEE LLM + signed retrieval	Document provenance	(b) and (c) usually unsolved
Agent / multi-step	Per-call TEE + signed tools	Audit log	Most expensive end-to-end
Multimodal	TEE + watermarking	C2PA for media	Cross-modal consistency
Reasoning models	TEE + thinking-chain logging	n/a	Long traces compound state
Multi-tenant SaaS	TEE per-tenant	Signed isolation logs	Most demanding isolation
On-chain settlement	opML or zkML	TEE bridge	Different latency budgets

Reasoning-model verification: the long-thinking-chain problem

Reasoning models (o1, o3, DeepSeek-R1, Claude with extended thinking) produce long internal thinking traces before emitting a final answer. These traces — sometimes 10-100× longer than the visible response — are the actual computational artifact. Verification must address them. See reasoning model serving.

Why reasoning models are harder to verify

Thinking traces are not exposed. OpenAI hides o1/o3 reasoning tokens; DeepSeek shows them but with rate limits; Anthropic exposes extended thinking when enabled. A verifier who cannot see the trace cannot confirm what was computed.
Decode-pool monopolization. Reasoning models spend most compute in decode. A provider could substitute a cheaper non-reasoning model for the visible output, producing similar-looking final answers without the thinking work. This is hard to detect from output alone.
Thinking-chain length is a quality proxy. Shorter chains often correlate with lower quality on hard problems. A cheating provider could truncate thinking to save compute and still produce plausible answers.
Variable compute per request. Reasoning models use 5-50× more compute on hard problems than easy ones. Per-request cost varies. PoSP-style sampling must adjust for this.

Verification primitives for reasoning models

TEE + thinking-chain logging. The TEE attests the model and emits a signed log of the thinking trace (or its hash) before final output. Client can verify the trace exists and was the right length. Used by Anthropic for high-tier reasoning workloads.

Per-step PoSP. Sample 1% of reasoning calls and re-execute the full trace on auditor hardware. Higher absolute cost per audit (thinking is long), but same percentage overhead.

Output-only quality checks. Compare final-answer quality against expected distribution. Provides weak signal of cheating but no per-request proof.

Thinking-budget commitment. Provider commits in advance to a thinking budget (max tokens); client verifies the trace stays within budget. Provides cost predictability, not cheating prevention.

The honest gap

For reasoning workloads where the thinking is the product (research assistants, deep planning agents), verification is genuinely harder than for chat. The state-of-the-art in 2026 is TEE + signed trace logging; cryptographic proof of reasoning correctness is open research.

Enterprise procurement: how to ask vendors for proof

Procurement is where verification actually changes vendor behavior. Asking the right questions surfaces the gap between marketing and reality.

Procurement question checklist

Does the platform support hardware-attested execution? Specifically: NVIDIA Confidential Computing on the GPUs serving our workload, and Intel TDX or AMD SEV-SNP on the host CPUs.
What is the attestation flow? Walk us through end-to-end: from session establishment to inference completion. Who signs what; where are roots of trust.
What is the model-fingerprint guarantee? Does the platform load-time-hash the model weights and reject mismatches?
What is logged? Show us a sample audit log entry. What is signed, what is not, retention period, who has access.
What is the SOC2 / ISO 27001 / HIPAA / FedRAMP posture? Type 2 reports? Letter of attestation? Scope.
What is the side-channel posture? Specifically: are SMs / cores partitioned per tenant or shared? Is timing-attack mitigation enabled.
What is the firmware-update policy? Who decides when firmware rolls out, what is the customer's notification, and does attestation reflect.
What is the data-retention guarantee? Prompts, completions, embeddings, intermediate state.
What is the model-version commitment? Same checkpoint for the duration of the contract or rolling updates.
Is there a verifiable tier and what does it cost? Specifically what is included that is not in the standard tier.
Is the verification primitive portable? If we move to a second provider, do we get the same attestation envelopes.
What is the incident-response posture if attestation fails? Who notifies whom, on what SLA.
What is the watermarking posture for AI-generated content? Specifically text, image, video.
What is the C2PA posture for any media outputs?
Can we run our own audit on a sample of requests? What is the protocol.

Red flags

"We use industry-standard security" with no specifics.
Refusal to share attestation samples.
"TEE is enabled" but no documentation of which TEE, what version, who signs.
SOC2 Type 1 only (gap report, not operating effectiveness).
No model fingerprinting (provider can swap weights silently).
No way to audit a sample request (operator-driven trust only).

Green flags

Published attestation envelopes (sample bundle visible).
Vendor-supplied verification SDK.
Open-source verification client (so you don't depend on the vendor to interpret).
SOC2 Type 2 + relevant sector certifications.
Customer-controlled keys for at least one layer.
Bug bounty for the verification stack.

Negotiating verification into a contract

The standard pattern: master service agreement adds a Verification Addendum specifying attestation deliverables, audit rights, incident SLAs, and remediation in case of attestation failure. Costs roll into the standard pricing or surface as a premium tier. Liability caps remain per the MSA but verification-failure damages may be carved out.

2026 regulatory landscape: EU AI Act, NIST AI RMF, sectoral rules

Verification went from "nice to have" to "implied by regulation" in 2025-2026. Specifics.

EU AI Act

The Act categorizes AI systems by risk: minimal, limited, high, unacceptable. High-risk systems (Annex III) must satisfy: traceability of training data, documentation of model architecture, post-market monitoring, human oversight, robustness and accuracy thresholds, cybersecurity. The Act does not literally mandate TEE attestation, but the traceability and documentation requirements are satisfied most cleanly by signed attestations + signed audit logs. Enforcement: phased through 2025-2027 with substantial fines for non-compliance.

NIST AI RMF

The NIST AI Risk Management Framework is voluntary in the US but increasingly referenced in federal procurement and state-level regulation. RMF recommends: trustworthiness criteria (valid, reliable, safe, secure, resilient, accountable, transparent), governance practices, mapping risks, measurement, management. Attestation and audit-log primitives provide the technical evidence for "accountable" and "transparent."

HIPAA (US healthcare)

Requires PHI confidentiality, integrity, and availability. TEE-attested execution satisfies the technical safeguards for confidentiality; signed audit logs satisfy audit-trail requirements. Business associate agreements (BAAs) now routinely include confidential-compute language.

Financial regulation

SEC, OCC, FINRA, and state regulators have published AI-specific guidance. Trading systems, credit-decision systems, and fair-lending models face increased scrutiny. The pattern: signed model documentation, audit-trail retention, and TEE-attested execution for sensitive deployments.

FDA (medical AI)

Medical-device AI follows the FDA's Software as a Medical Device (SaMD) framework. Increasing pressure for "predetermined change control plans" that require audit logs of every model update. TEE attestation provides the foundation.

Sector summary

Regulation	Geography	Verification implication	Effective
EU AI Act	EU + global services	Traceability, signed docs	2025-2027 phased
NIST AI RMF	US (voluntary, federal procurement)	Accountability, transparency	Now
HIPAA	US healthcare	Confidentiality, audit	Now
SEC AI guidance	US financial	Model docs, audit	Now
FDA SaMD	US medical devices	Change-control logs	Now
China Generative AI	China	Provenance, content labeling	Now
UK AI Bill (forthcoming)	UK	TBD	2026-2027
Singapore Model AI Governance	Singapore	Voluntary best practice	Now

What this means for builders

The regulatory landscape rewards: signed documentation, TEE-attested execution, audit-log retention, model fingerprinting, watermarking for generated content. Cryptographic proof (zkML) is not yet specified by any major regulation; the bar is "audit-trail provable," not "cryptographic-correctness provable." Build for the audit-trail bar; layer zkML when economics permit.

Honest limits of each verification approach

Every primitive has failure modes that vendor marketing obscures.

TEE limits

Trust assumption is silicon vendor + supply chain. If NVIDIA's signing keys leak, or if a tampered GPU enters the rack, TEE attestation lies and you cannot tell.
Side-channel attacks remain a research-active threat. Most production TEEs disable hyperthreading and SM-sharing, but new side channels appear yearly.
Firmware-update attacks are real. A coerced firmware roll forward can change the attestation hash; deployments need allow-lists, not just match-anything.
TEE does not prove model quality, just identity. A correctly-attested run of a poorly-aligned model passes TEE checks.

PoSP limits

Statistical guarantee, not cryptographic. With p = 1% sampling, 99% of dishonest acts go unsampled; the deterrent is amortized expected loss.
Sybil collusion is hard to fully defeat. Identity-binding helps but adds friction.
Adaptive cheating (cheat only when you predict no sample) requires unbiased sampling design.
Quality degradation that does not change tokenwise output is invisible to PoSP (e.g., subtle temperature manipulation).

Redundant execution limits

Cost scales with N. At 3× cost it is reserved for high-stakes individual requests.
If all providers use the same upstream model, "agreement" means nothing about correctness.
Determinism gaps (different GPUs, batching) require tolerance windows that can hide subtle differences.

zkML limits

1000-10000× overhead today. Unworkable for frontier LLMs.
Trusted setup for some schemes (Groth16). STARKs and Halo2 avoid this; Plonk has universal trusted setup.
Proof bugs are catastrophic. A buggy circuit accepts false proofs as valid; formal verification of ML circuits is open research.

opML limits

Challenge window means no real-time finality. Hours-to-days latency.
Requires honest challengers exist. If no one watches, no one challenges.
Slashing mechanism requires bonded stake; pure off-chain deployments cannot use opML cleanly.

Watermarking limits

Defeated by paraphrase, translation, and adaptive attacks.
Quality-vs-detection tradeoff is real, even if small.
Doesn't help if the model doesn't enforce the watermark (open-weight models).
Short outputs (< 200 tokens) often have insufficient signal.

C2PA limits

Strips at platform upload. Most social media strips C2PA in re-encoding.
PKI is the unsolved part: a manifest is only as trustworthy as its issuing root.
Targets media; useless for text.

Audit log limits

Operator-controlled by default. Append-only requires external anchoring (Sigstore Rekor, blockchain commit, third-party log service).
"Signed" doesn't mean "honest." Operators can sign whatever they want.
Retention requirements often exceed practical log sizes.

Putting it together

The most honest summary: no single primitive defeats a sophisticated adversary. Defense in depth — TEE + signed logs + model fingerprinting + watermarking on outputs + periodic third-party audits — gets the threat model from "probably not lying" to "very hard to lie at scale without detection." Cryptographic strength (zkML) buys little additional security if the operator runs evil code inside the attested boundary; what matters is the union of primitives that cover orthogonal threat surfaces.

Additional FAQ

Q: What's the difference between zkML and FHE for inference?

zkML proves a computation happened correctly, often without revealing inputs. Fully homomorphic encryption (FHE) allows computation on encrypted data without decryption — neither party sees the plaintext. They solve different problems: zkML answers "did this happen correctly," FHE answers "compute without revealing." FHE is currently 100,000-1,000,000× slower than plaintext inference. Some research combines them (verifiable FHE) but production deployment is years out.

Q: Can a TEE attestation be replayed against a future session?

No, if the protocol is designed correctly. Production attestation includes a nonce or ephemeral session key bound to the specific session. Replaying an old attestation fails the nonce check. Caveat: bad implementations skip nonces and are vulnerable to replay — verify your client library includes freshness checks.

Q: How does decentralized inference handle hot reloading of model weights?

Each weight version produces a new fingerprint hash. The PoSP audit pool replays with the version current at request time; if the prover used a different version, the audit fails. Hot reloads must be coordinated across the network: prover advertises new version, network sample-tests, version becomes active after passing.

Q: What's "verifiable randomness" and why does PoSP need it?

PoSP requires that the prover cannot predict which requests will be audited. If the auditor uses public randomness (e.g., a block hash) to select samples, the prover can compute the same randomness and cheat selectively. Verifiable Random Functions (VRFs) and threshold signatures provide randomness that the prover cannot influence; the auditor uses these for sample selection. Without VRF, the deterrent collapses.

Q: Does Apple Private Cloud Compute use NVIDIA Confidential Computing?

No. Apple Private Cloud Compute is built on Apple-designed M-series server silicon with Apple's own attestation primitives. It does not use NVIDIA GPUs and does not produce NVIDIA-compatible attestation envelopes. The closed Apple ecosystem provides strong guarantees within Apple's stack but does not interoperate with other TEE deployments.

Q: Can I verify inference on a model I'm self-hosting?

The threat model changes. Self-hosting means you control the hardware. TEE is overkill for self-hosting; what you typically want is model fingerprint verification (hash check at load time) plus audit logging. If you're self-hosting in a multi-tenant cloud (shared GPU), TEE matters again because the cloud operator is the threat.

Q: How does verifiable inference interact with privacy regulation (GDPR, CCPA)?

Verification provides the audit-trail evidence that regulators demand. GDPR Article 22 (automated decision-making) implies need for explanation; signed audit logs of the decision pipeline support this. CCPA's data-deletion requirements interact with audit retention; production designs balance the two with retention policies and right-to-deletion workflows.

Q: What happens to verifiable inference when models are post-quantum-vulnerable?

Most current attestation chains use ECDSA or RSA signatures, which are vulnerable to large-scale quantum computers. The NIST PQC standardization (CRYSTALS-Dilithium, FALCON, SPHINCS+) provides quantum-resistant alternatives. Migration of attestation infrastructure to PQ signatures is a 5-10 year project; NVIDIA, Intel, and AMD have all committed to roadmaps. For most threats today, PQ-vulnerability is theoretical.

Q: Does NVIDIA's H200 confidential compute support all model architectures?

Yes — TEE is architecture-agnostic. Dense transformers, MoE, multimodal, reasoning models all run inside the same attested boundary. The TEE doesn't care what computation runs; it only cares that the computation runs in the isolated environment with the attested code.

Q: How does verifiable inference work for MoE models?

MoE adds a routing layer. Verification covers the full model (router + experts) as one unit; TEE attestation includes the full weight hash. The wrinkle: which experts a token routes to leaks information about the input. For high-confidentiality MoE serving, research into private routing is active. See mixture-of-experts serving.

Q: Does watermarking work on code generation?

Partially. Code has lower token-distribution entropy than prose (more boilerplate, less synonym flexibility). Watermark signal degrades faster on code than on prose. SynthID's deployment for Gemini code outputs is publicly documented with reduced robustness in the code setting compared to prose.

Q: How long should audit logs be retained?

Depends on regulation. HIPAA: 6 years minimum. SOC2: 1 year minimum, typically 7. EU AI Act high-risk: 10 years for model-decision records. Financial trading: 5-7 years depending on jurisdiction. Plan retention budget accordingly; signed logs compress reasonably (per-call ~1-2 KB) but at scale this matters.

Q: What's the audit-log standard format in 2026?

There isn't one universal standard. W3C Verifiable Credentials, CloudEvents, and OpenTelemetry are the closest things; sector-specific formats (e.g., HL7 FHIR audit events for healthcare, ISO 20022 for financial) often apply. Most production systems use a custom JSON envelope with vendor-specific signatures, wrapped in Sigstore Rekor for transparency-log anchoring.

Q: Can verifiable inference catch a backdoored model?

Partially. Verification proves a specific model ran — if that specific model has a backdoor, verification confirms the backdoor ran, not that the model is safe. Detecting backdoors is a separate problem (interpretability, behavioral evals). The two compose: verification ensures the eval'd model is the deployed model; evals ensure the model behaves safely.

Q: Why don't more decentralized networks use TEE?

Cost and availability. TEE-capable GPUs cost more, are concentrated in major clouds, and require specific operator competence. PoSP and opML work on commodity GPUs without specialized firmware. Decentralized networks accept weaker per-request guarantees in exchange for permissionless participation.

Q: How does multi-LoRA serving complicate verification?

The TEE attests the base model. LoRA adapters layer on top. Each adapter load is a runtime event; the runtime must sign each adapter load (adapter hash + tenant + timestamp) into the audit chain. Production stacks like vLLM and SGLang are adding this support; in 2026 it's not yet universal. See multi-tenant LoRA serving.

Q: What is "remote attestation" and how is it different from local attestation?

Local attestation: the TEE measures itself and emits a quote that a co-located verifier checks. Remote attestation: the quote is forwarded over the network to a remote verifier (the client). NVIDIA Confidential Computing and Intel TDX both support remote attestation as the standard pattern for cloud inference. The remote verifier verifies the quote against vendor roots, not against any local state.

Q: Is there a "verifiable inference" certification for AI products?

Not yet a single certification. SOC2, HIPAA, FedRAMP, ISO 27001 cover related security postures. The Confidential Computing Consortium has begun standardization work; emerging certifications may consolidate by 2027-2028. For now, customers ask for evidence (attestation samples, audit-log samples, SOC2 reports) rather than checking a single badge.

Q: How does verifiable inference handle the "compromised supply chain" threat?

The hardest threat. If a tampered GPU enters the supply chain, TEE attestation will be signed by what looks like a legitimate vendor key but the silicon may have backdoors. Defenses: vendor-attested supply chain (NVIDIA's emerging supply-chain provenance program), customer firmware audit, multi-vendor sourcing, randomized hardware deployment. Threat is real but rare in practice.

Q: What's the verification posture of edge AI (phones, IoT)?

Apple Secure Enclave provides strong on-device TEE for iOS. Android Keymaster / Trusty TEE varies by vendor. Edge attestation works for on-device inference; cloud-augmented edge (hybrid inference) needs attestation chains on both sides. Production patterns: device key + cloud TEE attestation, with end-to-end encrypted session bridging the two.

Table of contents

Key takeaways

Mental model: verifiable inference in one minute

The landscape in 2026

Layered comparison

The trust problem

Watermarking text generation

How LLM watermarking works

Robustness — the hard part

Where watermarking matters

What watermarking does not do

C2PA and content provenance

What a C2PA manifest contains

What C2PA does well

What C2PA does not do

TEEs in production at NVIDIA and Anthropic

NVIDIA's stack

Cloud availability

Anthropic's public posture

The model fingerprinting layer

When verification matters in agent workflows

The cascading-trust problem

Per-step vs end-to-end attestation

Tool-use verification

Output watermarking for agent text

When agents don't need this

Approach 1: redundant execution

How it works

Trust assumption

Cost

When it works

When it doesn't

Determinism caveat

Approach 2: Trusted Execution Environments (TEE)

NVIDIA Confidential Computing

Intel TDX (CPU-side)

How it works in practice

Trust assumption

Cost

Limitations

When TEEs work

Approach 3: Zero-knowledge proofs (ZK)

The pitch

The cost problem

Active research areas

When ZK is useful today

Approach 4: Proof of Sampling (PoSP)

How it works

Why it works

Trust assumption

Cost

Limitations

Production deployments

Comparison and trade-offs

Picking by use case

Production deployments in 2026

Centralized providers

Decentralized providers

Hybrid

opML: optimistic verification for ML

Why opML is interesting

Why opML is limited

opML vs zkML cost comparison

Watermarking benchmarks: SynthID vs Kirchenbauer vs Aaronson

The Soft / Hard / Cryptographic taxonomy

When verifiable inference matters

Compliance and regulation

Decentralized marketplaces

Multi-tenant SaaS with sensitive data

Audit-required workloads

High-value individual decisions

When it doesn't

Casual chat

Internal tools

Research and prototyping

When the cost exceeds the value

The open research questions

ZK that scales to LLMs

Cross-provider verification standards

Verifiable training