Homomorphic Encryption for Biometric Security: Challenges, Progress and Opportunities

Vishnu Naresh Boddeti

Michigan State University

7th March, 2023

VishnuBoddeti

Vulnerabilities in Biometrics

Biometric systems suffer from vulnerabilities.

Mitigating Security Vulnerabilities

Biometrics + Encryption

Encrypted Biometrics

Data encryption is an attractive option

protects user's data from unauthorized access
protects service provider's models from unauthorized access
facilitates free and open sharing of private data
mitigate legal and ethical issues

Traditional solutions need data decryption for computation.
Security only during data transmission.

Homomorphic Encryption: The Holy Grail?

Cryptographic scheme needs to allow computations directly on the encrypted data.

Solution: Homomorphic Encryption
Attractive Property: Conjectured to be post-quantum secure for appropriate choice of encryption parameters.

Limitations: only supports additions and multiplications.

Existing Applications of FHE for Biometric Security

Template protection using Homomorphic Encryption:

Encrypt database of features.
Encrypt query feature.
Match score computed directly in encrypted domain.

Prior Work: Template Protection with Homomorphic Encryption

Boddeti, "Secure Face Matching Using Fully Homomorphic Encryption," BTAS 2018

Bassit et.al, "Multiplication-Free Biometric Recognition for Faster Processing under Encryption," IJCB 2022

Engelsma, Jain, Boddeti, "HERS: Homomorphically Encrypted Representation Search," TBIOM 2022

Bauspieß et.al, "Improved Homomorphically Encrypted Biometric Identification Using Coefficient Packing," IWBF 2022

CITeR Project

Biometric Template Fusion: Aug 2021-Aug 2022 Biometric Score and Decision Fusion: 2023-Present

Fusion of Biometric Information

"A comprehensive overview of biometric fusion."Information Fusion, 2019"

CITeR Project Focus: Template Fusion

"Deep learning approach for multimodal biometric recognition system based on fusion of iris, face, and finger vein traits." Sensors, 2020

Privacy Attacks from Features

Attacks on face features

"Assessing Privacy Risks from Feature Vector Reconstruction Attacks," arXiv:2202.05760

Face reconstruction from template

"On the reconstruction of face images from deep face templates," TPAMI, 2018

Finger vein reconstruction from binary templates

"Inverse Biometrics: Reconstructing Grayscale Finger Vein Images from Binary Features," IJCB, 2020

HEFT

Homomorphically Encrypted Fusion of Biometric Templates

HEFT: Overview

HEFT: Concatenation

Homomorphic Concatenation

HEFT: Linear Projection

Linear Projection

Naive

Hybrid

SIMD

Linear Projection Comparison

Computational Complexity

Space Complexity

Hybrid

Pros: Low memory and runtime overhead
Cons: Scales linearly with number of samples

SIMD

Pros: Scales well with number of samples
Cons: High memory and runtime overhead

HEFT: Feature Normalization

$\ell_2$-Normalization of Vector

$\hat{\mathbf{u}} = \frac{\mathbf{u}}{\|\mathbf{u}\|_2} \quad \rightarrow \quad$ division^$\dagger$

where

$\|\mathbf{u}\|_2 = \sqrt{\sum_{i=1}^d u_i^2} \quad \rightarrow \quad$ square-root^$\dagger$

^$\dagger$: problematic operations for FHE

Inverse Square Root: Polynomial Approximation

$$\frac{1}{\sqrt{x}} = \sum_{i=1}^6 a_i x^i$$

FHE-Aware Learning

Account for the limitations of FHE to improve performance

FHE is limited to specific operations on encrypted data.

Normalization is not directly computable - need to approximate.

Approximation is a source of error and hence a loss of matching performance

We incorporate approximate normalization into our training of the projection matrix to recover performance

Loss Function

Main Idea: FHE-Aware Learning

$$Loss = \lambda \underbrace{\frac{\sum_M d(\mathbf{c}_i, \mathbf{c}_j)}{|M|}}_{ \color{orange}{Pull} } + (1-\lambda)\underbrace{\frac{\sum_{V}[m + d(\mathbf{c}_i, \mathbf{c}_j) - d(\mathbf{c}_i, \mathbf{c}_k)]_{+}}{|V|}}_{ \color{orange}{Push} }$$
where $$d(\mathbf{c}_i, \mathbf{c}_j) = 1-P\underbrace{f(\mathbf{c}_i)}_{ \color{cyan}{approximation} } \cdot P\underbrace{f(\mathbf{c}_j)}_{ \color{cyan}{approximation} }$$ $f(\cdot)$ approximates the inverse norm of a vector.

Numerical Evaluation

Experimental Setup

Cross-Posed Labelled Faces in the Wild

Down

Dog

Bird

Backward

Google Speech Commands

Synthetic fusion dataset by randomly pairing classes.
10,760 samples over 188 classes.

Fusion Improves Performance, Reduces Dimensionality

Fusion improves performance:
- Face by 11.07%
- Voice by 9.58%

Dimensionality Reduction: $512D \rightarrow 32D$ (16$\times$ compression)

Comparison of Normalization Methods

Computational Complexity

Projection is costliest operation

Projection is costliest operation

What Next for Biometric Encryption?

Opportunities for Biometric Encryption

Going beyond template fusion

Ongoing CITeR Project (Jan 2023-Present)

End-to-End encrypted biometric recognition

End-to-end image classification on CIFAR-10

Preliminary Work:

Ao and Boddeti, "AutoFHE: Automated Adaption of CNNs for Efficient Evaluation over FHE," Cryptology ePrint Archive 2023

Opportunities for Biometric Encryption

Secure federated learning

Federated learning for face recognition

Aggarwal et al. "FedFace: Collaborative Learning of Face Recognition Model," IJCB 2021
Meng et al. "Improving Federated Learning Face Recognition via Privacy-Agnostic Clusters," ICLR 2022

Preliminary Work

Yonetani, Boddeti, Kitani, Sato "Privacy-Preserving Visual Learning Using Doubly Permuted Homomorphic Encryption," ICCV 2017

Distributed Encrypted Biometric Authentication

Multi-Key Homomorphic Encryption

Problems with existing FHE solutions for biometrics:

Cannot perform distributed authentication.
Even centralized authentication has practical limitations.

Either, need shared public key, impractical for some use cases.
Or, needs key-switching to work with different private keys.

Solution: Multi-Key FHE

Critical for practical deployments.

A Note on Security vs Privacy

Security and privacy are very often conflated with each other.

Different but related concepts.
Homomorphic encryption: controls access to private information.
Differential Privacy: allows analysis + controls information.

Postulates:

There is no privacy without security.
Homomorphic encryption is an ideal tool for enhancing privacy but it is not a privacy technique in and of itself.

Ideal solution: Differential privacy + Homomorphic Encryption

Summary: CITeR Projects on Biometric Encryption

Biometric System Threat Model

Biometric Template Fusion over Homomorphically Encrypted Templates

Many avenues for leveraging homomorphic encryption to enhance biometric security and privacy.

VishnuBoddeti