Fully Homomorphic Encryption Is Finally Fast Enough for Real AI Workloads

In July 2026, Seoul-based startup DESILO announced something that would have sounded impossible a few years ago: a framework called THOR that runs an entire BERT language model, end to end, on data that never leaves its encrypted state. The model completed its benchmark inference workload on a single GPU while keeping accuracy within about one percentage point of the unencrypted baseline. That single result collapses the core objection that has kept fully homomorphic encryption (FHE) in the lab for more than a decade: it was too slow to be useful. The company behind it, and the Google-affiliated benchmark that validated it, are now betting that private AI is about to move from research paper to production infrastructure.

Google’s role in this shift is less about shipping a single product and more about building rails. The company co-leads a global FHE benchmarking suite, hosts FHE-based confidential AI tools on Google Cloud Marketplace, and supplies confidential-computing hardware that encrypts Apple users’ AI queries inside Google data centers. The through-line is the same: close the gap where encrypted data gets decrypted to be processed, and private AI becomes something enterprises can actually deploy.

The Cleartext Gap: Why “Encrypted” AI Was Never Actually Encrypted

Data is well protected in two states: at rest (sitting in a database) and in transit (moving over TLS). The problem is the third state. When a processor actually works on data, that data sits in memory as plaintext, readable by anyone with privileged access to the host. DataKrypto, the company behind FHEnom for AI platform, calls this the “cleartext gap,” and it is the exact vulnerability that has kept regulated industries from moving sensitive AI workloads to the cloud.

This is not a hypothetical exposure. In a June 2026 report from the Confidential Computing Summit, security researchers pointed to a technique called TDXRay that showed how information could be observed across virtual-machine boundaries by watching the processor cache, a layer that sits below every isolation boundary. Even a hardware-enforced trusted execution environment (TEE) only protects the workload itself; it does not eliminate what the shared cache sees. That is the precise reason for layering mathematical encryption on top of hardware isolation rather than treating either as sufficient alone.

The stakes are concrete. Healthcare models trained on patient records, financial models processing transaction data, and public-sector systems handling citizen information all stall at the same wall: the moment a model needs data in plaintext to compute, the confidentiality guarantee collapses. FHE attacks that wall directly by keeping data encrypted during computation itself.

How Homomorphic Encryption Actually Works

The intuition is simpler than the math. Normal encryption is like a locked box you cannot open without a key. Homomorphic encryption is a locked box you can still rearrange the contents of without ever opening it. You can add two encrypted numbers, multiply them, or run a neural network over them, and the result, when finally decrypted, matches what you would have gotten by computing on plaintext. The cloud never sees the data, only ciphertext and the encrypted result.

The scheme that makes this practical today is CKKS, which operates on approximate real numbers rather than exact integers. That approximation is what lets CKKS handle the floating-point arithmetic that neural networks depend on, at the cost of introducing a small amount of noise into every operation. The ElGamal scheme, by contrast, supports exact operations over integers but is limited to a narrower set of computations. A 2025 paper on confidential control systems shows both being applied to a gain-tuning problem, with numerical examples under 128-bit security showing performance comparable to conventional methods.

Two dominant costs define FHE’s difficulty. The first is the Number Theoretic Transform (NTT), the polynomial multiplication step that dominates encryption and evaluation time. The second is bootstrapping, the periodic “refresh” operation that reduces accumulated noise so computation can continue. Both are memory-bound and wide-precision, which is why general-purpose GPUs, optimized for low-precision AI math, struggle with them. A February 2026 paper from Boston University and collaborators proposes a specialized functional unit called FHECore that maps both NTT and base conversion onto wide-precision modulo-multiply-accumulate hardware, reporting up to 2.12x speedup and 50% reduction in bootstrapping latency for a 2.4% area overhead.

Here is what a minimal CKKS operation looks like in practice, using the TenSEAL library that Microsoft-affiliated researchers benchmarked, showing an encrypted convolutional network evaluated in under a second on MNIST:

Note: The following code is an illustrative example and has not been verified against official documentation. Please refer to the official docs for production-ready code.

import tenseal as ts

# Create CKKS context (approximate arithmetic, real numbers)
context = ts.context(
 ts.SCHEME_TYPE.CKKS,
 poly_modulus_degree=8192,
 coeff_mod_bit_sizes=[60, 40, 40, 60]
)
context.global_scale = 2**40
context.generate_galois_keys()

# Encrypt two vectors; cloud only ever sees ciphertext
a = ts.ckks_vector(context, [1.0, 2.0, 3.0])
b = ts.ckks_vector(context, [4.0, 5.0, 6.0])

# Homomorphic addition and multiplication happen on encrypted data
encrypted_sum = a + b
encrypted_product = a * b

# Only key holder can decrypt result
print(encrypted_sum.decrypt()) # approximately [5.0, 7.0, 9.0]
print(encrypted_product.decrypt()) # approximately [4.0, 10.0, 18.0]

# Note: production use requires noise management via bootstrapping,
# rescaling after each multiplication, and careful scale selection to
# avoid precision loss over deep circuits.

The output is approximate, not exact, because CKKS trades a small precision loss for the ability to run real-number math. That trade is acceptable for neural networks, where weights and activations are already floating-point, but it is a real constraint for exact integer workloads like financial ledgers.

Google Cloud’s Confidential AI Bet: FHEnom and Marketplace

The clearest signal that Google is treating private AI as a commercial priority came in March 2026, when DataKrypto’s FHEnom for AI launched on Google Cloud Marketplace. DataKrypto had just completed the Google Cloud ISV Startup Springboard program, and the launch press release framed the product as closing the cleartext gap across the entire AI lifecycle, from ingestion and training through inference.

The architecture is worth noting because it does not rely on FHE alone. FHEnom combines fully homomorphic encryption with trusted execution environments in a layered design, running computation on ciphertext while the TEE provides hardware-backed memory encryption and attestation. The company describes the architecture as TEE-agnostic, working across Intel TDX and AMD SEV hardware, and positions it as supporting HIPAA- and GDPR-ready workflows. The pitch is “continuous encryption,” where models, prompts, and GPU memory stay encrypted throughout, rather than encryption bolted on as a perimeter defense.

That framing is the vendor’s own, and it deserves standard skepticism. DataKrypto’s claims about eliminating performance barriers and protecting “every model and data point” come from its press release, not from independent benchmarks. Ritika Suri, Managing Director of AI and Data Partnerships at Google Cloud, is quoted in the same release describing the goal as enabling “continuous encryption for AI workloads at scale,” which tells you Google Cloud is actively promoting the category even though performance and security guarantees have not yet been independently audited. This is the same pattern as our earlier analysis of searchable encryption, where an honest-state estimate put FHE between 1,000x and 1,000,000x slower than plaintext depending on operation. The gap between that reality and “no performance barriers” is precisely what buyers should probe.

The FHE Benchmarking Suite: Turning Research Into a Reproducible Standard

The most consequential Google connection is not a product but a standard. Shruthi Gorantala of Google is co-lead of the global FHE benchmarking suite, an effort that selected DESILO’s THOR as the first reference implementation for encrypted language-model inference. Gorantala’s framing, quoted in the July 2026 announcement, is blunt about why this matters: “Homomorphic encryption has advanced quickly, but because measurement conditions differ from one result to the next, systematic, objective comparison of real progress has remained out of reach. Much like MLPerf in machine learning, FHE Benchmarking Suite is a reference point shared across the entire community.”

The comparison to MLPerf is a key insight. For years, FHE researchers reported results under wildly different hardware and parameter conditions, making it impossible to tell whether a claimed speedup was real progress or just a favorable benchmark. The suite defines shared, reproducible workloads and performance measures. Adding Transformer inference to the benchmark means encrypted inference is no longer a one-off demo but something anyone can reproduce, compare, and build on. That is what turns a research field into an ecosystem.

THOR’s numbers give a concrete sense of where the field stands. The framework runs BERT on encrypted data on a single GPU, keeping accuracy within about one percentage point of the unencrypted model. Through continued optimization, inference time dropped from roughly 10 minutes to about 2 minutes. Its matrix multiplication, the core of the computation, is accelerated by up to 9.7x over prior published results. These are still far from real-time interactive speeds, but they are a different order of magnitude from the “hours per query” that characterized FHE a few years ago.

The Performance Reality: What “Practical” Actually Costs

The honest conversation about FHE has to start with numbers. The overhead is real and it is large. The TenSEAL benchmark showed an encrypted CNN running on MNIST in under a second with under half a megabyte of communication, which is genuinely impressive for a small model but does not translate to a 70-billion-parameter LLM. The honest-state survey cited in our prior coverage put FHE between 1,000x and 1,000,000x slower than plaintext depending on operation, and that remains the right mental model for general workloads even as specialized hardware narrows the gap.

Research is pushing the ceiling up on multiple fronts. The Independent Vector Evaluation (IVE) paper from June 2026 targets a specific bottleneck: private embedding lookup, which turns a simple table access into an expensive homomorphic computation. The prior state of the art built a one-hot vector from an encrypted index, requiring O(p log p) operations. IVE instead evaluates a linearly independent vector from successive powers of a single encrypted value, cutting the cost to O(p) and improving amortized lookup time by up to 78.4x. On the Enron-Spam dataset, replacing one-hot generation with IVE reduced the share of vector generation in encrypted FastText inference time from 99.6% to 66.3%.

The pattern across all of this work is the same: wins are large but narrow. Each optimization attacks one primitive, one embedding layer, one transform. The field is making FHE practical for specific, bounded workloads, not for arbitrary computation. That is a crucial distinction for anyone evaluating whether to adopt it.

FHE vs. TEEs vs. Federated Learning: Choosing the Right Tool

FHE is not the only way to make AI private, and it is often not the best one. The three main approaches solve different versions of the problem, and conflating them leads to expensive mistakes.

Trusted execution environments protect data in use through hardware isolation. The data is decrypted inside the enclave, but the enclave’s memory is encrypted and hardware provides attestation proving what code ran. TEEs are fast, often near-native speed, but they trust the hardware vendor and have repeatedly been shown vulnerable to side-channel attacks like TDXRay. They are the right choice when you need speed and can accept hardware-root-of-trust risk.

Federated learning keeps raw data on each participant’s device and shares only model updates. It is the approach Google itself pioneered for apps like keyboard prediction, and it protects data by never centralizing it. But it does not protect the model from the aggregator, and gradient updates can still leak information about training data. It solves a different problem: collaborative training without data sharing, not inference on encrypted data.

Homomorphic encryption is the only approach where the cloud literally cannot see the data, even in principle, because it never holds the key. The trade is performance. The practical answer for most production systems in 2026 is layering: TEEs for heavy compute, FHE for the most sensitive operations where mathematical guarantees matter more than speed, and federated learning where data cannot leave the device at all. DataKrypto’s FHEnom explicitly combines FHE with TEEs for exactly this reason.

Limitations and What Practitioners Report

The gap between vendor claims and independent reality is the most important thing to understand before committing budget. DataKrypto’s press release promises “real-time AI inference on encrypted data without introducing performance barriers,” but independent benchmarks tell a more measured story. Even the best-published FHE result, THOR’s roughly two-minute BERT inference, is orders of magnitude slower than plaintext inference on the same hardware. “Practical” in the FHE world still means “practical for batch and offline workloads,” not “practical for interactive chat.”

There are also fundamental limits that no optimization removes. CKKS introduces approximation error that accumulates with circuit depth, which is why bootstrapping exists and why deep networks are harder than shallow ones. The ciphertext is also dramatically larger than plaintext, inflating memory and bandwidth costs. And the security model, while strong, is not absolute: FHE protects data from the compute provider, but it does not protect against a malicious client who submits crafted inputs, nor does it guarantee anything about the model’s own behavior on encrypted data.

The attestation gap is a separate concern flagged at the Confidential Computing Summit. As Microsoft researcher Antoine Delignat-Lavaud put it, attestation “doesn’t tell you where it is running,” leaving data residency and sovereignty questions unresolved. A workload can be provably running on genuine confidential hardware while still being processed in a jurisdiction the data owner never approved. For regulated industries, that is a material gap that FHE’s mathematical guarantees help close but do not eliminate, since encrypted computation still physically executes somewhere.

Simpler approaches often win for narrow tasks. If you only need equality search on encrypted fields, a deterministic blind index adds a few milliseconds to lookup, as we documented in our comparison of encrypted search methods, versus FHE’s thousandfold-to-millionfold overhead. The right question is not “should we use FHE” but “which specific computation genuinely requires data to stay encrypted end to end, and is that computation bounded enough to run under FHE at all.”

What to Watch in 2026

Several threads will determine whether private AI becomes the default rather than a specialty. The FHE benchmarking suite is the most important one to watch, because a shared, reproducible standard is what lets buyers distinguish real progress from marketing. If Transformer inference benchmarks gain traction the way MLPerf did, expect a wave of comparable, independently verifiable claims in the coming year.

The hardware story is equally significant. FHECore and Niobium’s The Fog, an infrastructure-as-a-service platform purpose-built for FHE that opened its developer partner program in 2026, both signal that the industry is moving past “run it on a general GPU” toward purpose-built acceleration. That is the same trajectory machine learning itself followed, and it is usually the moment a technology’s cost curve starts bending down.

Google’s position is strategic rather than product-led. By co-leading the benchmark, hosting FHE tooling on Cloud Marketplace, and supplying confidential-compute hardware for third parties like Apple, Google is positioning itself as a neutral infrastructure layer for a category it expects to grow regardless of which startup wins. The company’s own Gemini models do not yet run under FHE for consumer use, and there is no indication that is imminent. What Google is building is a market: standards, marketplace, and compute substrate that private AI will run on if it scales.

The honest read is that private AI is crossing from research to early production, not to ubiquity. The cleartext gap is real, the techniques to close it are maturing fast, and Google is assembling the rails. But the performance gap remains large enough that FHE will serve bounded, high-sensitivity workloads, layered with TEEs and federated learning, rather than replacing plaintext inference wholesale. That is still a meaningful change: for the first time, “private AI” is a deployment decision rather than a contradiction in terms.

Related Reading

More in-depth coverage from this blog on closely related topics:

• How to Move Cursor on Computer
• How to Analyze Tech Company Reports
• How to Speed Up GPT-5.6 Sol for Production
• GLM-5.3 Update: Key Features
• Trezor Wallet Security and the ShipMonk Data

Sources and References

Sources cited while researching and writing this article:

• Agentic AI security steals the spotlight at Confidential Computing Summit
• [2510.26179v1] Confidential FRIT via Homomorphic Encryption
• A GPU Microarchitecture Optimized for Fully Homomorphic Encryption
• Microsoft-affiliated researchers benchmarked
• DataKrypto’s FHEnom for AI (TM) Now Available on Google Cloud …
• Independent Vector Evaluation (IVE) paper from June 2026