Encrypted Biometric Systems


JP Morgan Chase

Michigan State University


Biometric Systems are vulnerable to many attacks


State of Affairs

(report from the academic-world)

Attacks on Face Recognition Systems

Attacks on Face Recognition Systems: Template Inversion

Template inversion attack on High resolution image



From Template inversion attack to Replay and Presentation attack

Template inversion attack enables Presentation attack



[SM23] Comprehensive vulnerability evaluation of face recognition systems to template inversion attacks via 3D face reconstruction

Presentation attack via digital replay and printed photograph

Presentation attack via printed photograph

Biometric Template Protection

    • Goal: Protecting templates in a biometric system.
    • Conceptual idea of BTP:
      • Template -> Transform -> Protected Template

Encrypted Biometric Systems






Key Driver
Privacy and Security Concerns

Standard Encryption: Data is Encrypted Only During Communication

Privacy of user data is not guaranteed.

Encryption Schemes

What we have.

Is there an encryption scheme that satisfies our security desiderata?

Fully Homomorphic Encryption

What is Fully Homomorphic Encryption?

Run programs on encrypted data without ever decrypting it.
FHE can—in theory—handle universal computation.















Apple: Secure Caller ID and Secure Photo Search
Microsoft: Secure Password Search in Edge Browser

Encrypted Biometric Template Protection

Encrypted Template Protection

Encrypted Biometric Search Protocol

Three-Party Solution

Key Management

3-Party System: Key Management

3-Party System: Enrollment

3-Party System: Authentication

Biometric Matching Accuracy

Encrypted Biometric Template Search

Biometric Search Performance

Scaling Biometric Search on a single GPU (A100)

FHE-based search takes $3sec$ for a 512-dim 10 Million vector gallery.
Solution with further 3x-4x speedup is in the pipeline.

End-to-End Encrypted Biometric Systems

Going Beyond Template Protection

End-to-End Encrypted Face Recognition

Effectively prevents score or decision-based attacks.

Experiments on Encrypted Face Datasets

Hardware & Software
  • Amazon AWS, r5.24xlarge
  • 96 CPUs, 768 GB RAM
  • Microsoft SEAL, 3.6
Approach Backbone Dataset Latency(s) Memory(GB)
Network Params Boot LFW AgeDB CALFW CPLFW CFP-FP Avg
MPCNN ResNet32 529K 31 97.02 83.02 87.00 78.90 82.07 85.60 7,367 286
ResNet44 724K 43 98.27 87.45 90.85 83.72 87.90 89.64 9,845 286
AutoFHE1 ResNet32 531K 8 93.53 80.88 85.40 75.67 77.96 82.69 4,001 286
CryptoFace PCNNs 3.78M 1 98.78 92.90 93.73 83.95 87.94 91.46 1,446 277
  1. Architecture searched on CIFAR10.↩

Going Beyond Biometrics

Other Encrypted AI Applications

Data and Function Privacy

What are we trying to protect in AI?





  • $x$: images, audio, video, text

Data Privacy

    • Protect user privacy.
    • Prevent unauthorized access.

Function Privacy

    • Protect intellectual property.
    • Prevent attacks against model.
    • Prevent leakage of training data.
    • Comply with industry security standards.

Our Solution: Secure Data and AI Model

Encrypted GenAI

Attacks on Large Language Models

Attacks on Text Embeddings

Attacks on Language Models

Attacks on User Prompts

Our Solution: Encrypted LLM

SecureRAG: Secure Retrieval Augmented Generation

Thank You

Questions?