Encrypted Machine Learning


Bridge2AI Seminar

Slides: hal.cse.msu.edu/talks
VishnuBoddeti

Michigan State University


Proliferation of AI in Our Lives

Tesla FSD
Chat Bots
Rapid AI
Walker from UBTECH

State of Affairs

(report from the academic-world)

Attacks on Face Recognition Systems

Template inversion attack enables Presentation attack



[SM23] Comprehensive vulnerability evaluation of face recognition systems to template inversion attacks via 3D face reconstruction

Presentation attack via digital replay and printed photograph

Presentation attack via printed photograph

Attacks on Large Language Models

Attacks on Text Embeddings

Attacks on Language Models

Attacks on User Prompts

State of Affairs

(report from the real-world)
"Australia's biggest medical imaging lab is training AI on its scan data. Patients have no idea"

Healthcare Data Breaches of 500+ Records (2009-2024)

Only going to get worse with AI Chatbots
Real world AI systems are very effective, but


suffer from privacy and security vulnerabilities.

Privacy Requirements in Healthcare

Strict regulations on data privacy in transit, rest, and use.

Privacy Enhancing Technologies

Differential Privacy
Output Privacy, Accuracy-Privacy Tradeoff

Homomorphic Encryption
Input Privacy, Fire and Forget Privacy

Today's Agenda



Encryption for AI

What are we trying to protect in healthcare AI?





  • $x$: Medical Imaging, EHR, Genomics, Voice, Identity

Data Privacy

    • Protect patient privacy (e.g., identity, diagnosis).
    • Prevent unauthorized access to PHI.

Model Privacy

    • Protect proprietary diagnostic models.
    • Prevent model inversion attacks.
    • Prevent reconstruction of training data (e.g., patient records).
    • Maintain competitive advantage.

The blind spot of traditional encryption

Privacy of user data is not guaranteed.

Encryption Schemes

What we have.

Is there an encryption scheme that satisfies our security desiderata?

Fully Homomorphic Encryption

FHE can help AI models achieve trustworthiness



FHE enables AI models to process encrypted data without decryption.

What is Fully Homomorphic Encryption?

Run programs on encrypted data without ever decrypting it.
FHE can—in theory—handle universal computation.















Apple: Secure Caller ID and Secure Photo Search
Microsoft: Secure Password Search in Edge Browser

Foundational Concepts

CKKS FHE Scheme

Making Neural Networks FHE-Friendly

FHE Foundational Concepts

Evolution of FHE schemes

Learning with Errors (LWE)

LWE problem prevents attackers from breaking FHE schemes' security

The Hardness Foundation of LWE-Based FHE

Private Key

$\mathbf{s} = \begin{bmatrix}10 \\ 82 \\ 50 \\ 51\end{bmatrix}$
Breaking FHE $\Leftrightarrow$ Solving LWE problem: recovering private key from public key.
System Problem Complexity Solution
$b=As$ in $\mathbb{R}$ System of Linear Equations P Gaussian Elimination
$b=As+e$ in $\mathbb{R}$ Least Squares Problem P Least Squares Estimator
$b=As+e \mod q$ in $\mathbb{Z}_q$ Learning with Errors Problem NP-hard No known efficient algorithm (not even quantum)
$b(X)=A(X)s(X)+e(X) \mod q$ in $\mathbb{Z}_q[X]/(X^N+1)$ Ring Learning with Errors Problem NP-hard No known efficient algorithm (not even quantum)

Pipeline of Homomorphic Evaluation of Encrypted Data



Using CKKS scheme as an example
Encoding/Decoding
Key generation
Encryption/Decryption
Evaluation

Data Encoding and decoding

Ex: CKKS encoding operates in the cyclotomic ring $\mathbb{Z}_q[X]/(X^N+1)$

Key generation

    Example of CKKS Keys
  • Private key: $sk = s(X)$

  • Public key: $pk=(a(X),b(x))$

  • Evaluation keys: $evK$
    • Multiplication keys for ciphertext size reduction.
    • Rotation keys to enable rotate and conjugate.

  • Main parameters: N and log(q)

Encryption and Decryption

Ex: CKKS ciphertext is composed of two polynomials.
  • Encryption
    • $\left[\!\left[ m \right]\!\right]=(c_0(X),c_1(X))$
  • Decryption
    • $\tilde{m}(X) = \text{round} (c_0(X) + c_1(X) \cdot s(X) \mod q) $
  • Multiplicative Depth
    • Ciphertexts are associated with a specific level.

FHE Schemes support basic HE capabilities



Bootstrapping enables unlimited HE operations over encrypted data


Bootstrapping is slowest of HE operations. Avoid if possible.

Data Packing and SIMD operations

Choice of packing scheme significantly affects latency.

Functions natively supported by FHE



Vector operations
Polynomial evaluation
Matrix operations

Vector operations Under Encryption


Polynomial evaluation Under Encryption

    • Polynomial evaluation $$P(X) = a_0 + a_1 X + a_2 X^2 + \cdots + a_n X^n$$
    • Viewed as an inner product $P(X) = <\mathbb{a}, \mathbb{X}>$
      • $\mathbb{a} = (a_0, a_1, \cdots, a_n)$
      • $\mathbb{X} = (1, X^1, \cdots, X^n)$
    • Encrypted polynomial evaluation
      • Pt-Ct multiplication $ct = \mathbb{a} \times \left[\!\left[ \mathbb{X} \right]\!\right]$
      • Ct-Ct multiplication $ct = \left[\!\left[ \mathbb{a} \right]\!\right] \times \left[\!\left[ \mathbb{X} \right]\!\right]$
      $$\left[\!\left[ P(X) \right]\!\right] = \sum_{i=0}^{n} a_i \cdot \text{rot}(ct, i)$$

Matrix operations Under Encryption

Matrix-vector multiplication
Complexity
  • Additions: $2$
  • Multiplications: $n$ rows
  • Rotations: $3$
Latency
Medium latency that depends on the matrix dimension.
Applications
Neural Networks

Matrix operations Under Encryption

Matrix-Matrix multiplication
Complexity
  • Additions: $2$
  • Multiplications: $n \times m$
  • Rotations: $0$
Latency
High latency that depends on the dimensions of the matrices.
Application
Transformers

Story So Far...

Right security parameters
Adequate packing scheme


Minimize #Multiplications and #Rotations

Apply bootstrapping to avoid exhausting ciphertexts
Primitive mathematical operations are feasible under encryption.

Applying FHE to AI

What are the challenges for applying FHE to AI?


    • FHE libraries operate at a low-level and are not user friendly.
    • FHE is computationally expensive (10,000x slower than standard computation).
    • Standard non-linear layers are not natively supported (e.g., ReLU, MaxPool).
    • Needs both cryptography and AI expertise.

User Friendly Software for Encrypted AI

From PyTorch to FHE Inference

Neural Architecture
Trained Weights
Orion 
import orion
net = ResNet50()
orion.fit(net, trainloader)
orion.compile()
net.he()
ctOut = net(ctIn)

Orion: A Fully Homomorphic Encryption Framework for Deep Learning, ASPLOS 2025

Adapting AI Models for FHE Computation

how to Adapt CNNs for FHE

Polynomial approximation for non-linear activations

Polynomial Approximation for non-linear activations

  • High-degree approximation
    • Slow: more multiplications, more bootstrappings
    • Accurate: high-degree polynomials
    • Training/fine-tuning: not necessary

Co-design CNNs and FHE systems

    • Security Requirement

Encryption Parameters
  • Cyclotomic polynomial degree: $N$
  • Level: $L$
  • Modulus: $Q_l=\prod_{i=0}^{l} q_l, 0 \leq q_l \leq L$
  • Bootstrapping Depth: $K$
  • Hamming Weight: $h$
    • Latency



space of Homomorphic neural Architectures

How to effectively trade-off between accuracy and latency?

Limitations of handcrafted architectures


Joint Search for Layerwise EvoReLU and Bootstrapping Operations



Joint Search Problem Multiobjective Search
    • Flexible Architecture
    • On demand Bootstrapping

Experimental Setup

Dataset: CIFAR10
  • 50,000 training images
  • 10,000 test images
  • 32x32 resolution, 10 classes


Hardware & Software
  • Amazon AWS, r5.24xlarge
  • 96 CPUs, 768 GB RAM
  • Microsoft SEAL, 3.6

Latency and Accuracy Trade-offs under FHE


Practical Application: End-to-End Encrypted Face Recognition

CryptoFace: End-to-End Encrypted Face Recognition

Mixture of Shallow Patch CNNs

Encrypted Face Recognition Evaluation

Hardware & Software
  • Amazon AWS, r5.24xlarge
  • 96 CPUs, 768 GB RAM
  • Orion (w/ Lattigo)

Encrypted Face Recognition Evaluation

Approach Resolution Backbone 5 Datasets Latency(s) Memory(GB)
Network Params Boot Average Accuracy1
MPCNN 64x64 ResNet32 0.53M 31 85.60 1,277 286
64x64 ResNet44 0.73M 43 89.64 1,640 286
AutoFHE 64x64 ResNet32 0.53M 8 82.69 667 286
CryptoFace 64x64 CryptoFaceNet4 0.94M 2 89.42 220 269
CryptoFace 96x96 CryptoFaceNet9 2.12M 2 90.99 232 276
CryptoFace 128x128 CryptoFaceNet16 3.78M 2 91.46 241 277
  1. Average Accuracy: the average one-to-one verification accuracy across five face datasets, ie LFW, AgeDB, CALFW, CPLFW, CFP-FP↩
7.5x speedup (27 mins → 3.6 mins), while preserving accuracy (89.64 vs 89.42)
Near-constant latency across different resolutions

Practical Application: End-to-End Encrypted LLM

End-to-End Encrypted LLMs


Closed-source implementations only.

Practical Application: End-to-End Encrypted SecureRAG

SecureRAG: End-to-End Secure Retrieval-Augmented Generation

Amina Bassit and Vishnu Boddeti, SecureRAG: End-to-End Secure Retrieval-Augmented Generation, NeurIPS GenAI4Health Workshop 2025

Story So Far...


Co-designing AI and FHE architectures is critical for efficiency.

Missing Piece in the Puzzle

Hardware Accelerators for FHE

Source: Duality Technologies

Concluding Remarks

Key Takeaways
  • Secure Healthcare AI is achievable with Fully Homomorphic Encryption (FHE).
  • Efficient and specialized architectures are crucial for practical encrypted inference.
  • Real-world applications, like secure facial recognition, secure LLMs and genomic analysis, demonstrate feasibility today.

Next Steps
We appreciate your interest. Let's advance secure healthcare AI together.