Fairness, Privacy and Efficiency in AI

Vishnu Boddeti

September 02, 2022


Progress In Machine Learning

Speech Processing
Image Analysis
Natural Language Processing
Physical Sciences

Key Driver
Data, Compute, Algorithms


(report from the real-world)
"Facial recognition is accurate, if you're a white guy"

  • Boulamwini and Gebru, "Gender Shades:Intersectional Accuracy Disparities in Commercial Gender Classification," FAT 2018
"The Secretive Company That Might End Privacy as We Know It"

"Modern AI models consume a massive amount of energy, and these energy requirements are growing at a breathtaking rate."

Real world machine learning systems are effective but,

are biased,

violate user’s privacy and are

computationally expensive.

Research Agenda

Build machine learning systems that are fair, trustworthy and efficient.

Fairness in AI

Research Questions

    • What are the exact fundamental limits and trade-offs between utility and fairness?

    • How can we design machine learning systems to achieve these fundamental limits and trade-offs?

Trade-Offs in Algorithmic Fairness

  • "Characterizing the Fundamental Trade-offs in Learning Invariant Representations," Arxiv 2021
  • "Adversarial Representation Learning With Closed-Form Solvers," ECML-PKDD 2021
  • "On the Global Optima of Kernelized Adversarial Representation Learning," ICCV 2019
  • "Mitigating Information Leakage in Image Representations: A Maximum Entropy Approach," CVPR 2019

Notions of Algorithmic Fairness

  • There are many notions of fairness (21 definitions):
    • Demographic Parity
    • Equalized Odds
    • Eqality of Opportunity

  • Estimate the fundamental limits and trade-offs for all definitions of fairness.

Fairness Under Practically Relevant Scenarios

  • Application Domain: Face Recognition
  • Summary: Next generation of machine learning systems have to be designed with fairness constraints.

Privacy in AI

Why Privacy in AI?

...consent should be given for all purposes...
Privacy Preserving Computer Vision

Mechanism: machine learning on encrypted data

Research Directions

    • Adapting homomorphic encryption for machine learning constraints.

    • Adapting machine learning for homomorphic encryption constraints.

Privacy-Preserving Biometrics with Homomorphic Encryption

BTAS 2018

Privacy-Preserving Learning on Homomorphically Encrypted Data

"Privacy-Preserving Visual Learning Using Doubly Permuted Homomorphic Encryption" ICCV 2017

Computationally Efficient Deep Learning

DNN Design Involves...

Deep Neural Network


Many More $\Huge \dots$

Early DNN Architecture Development

  • Primarily driven by skilled practitioners and elaborated design.
    • a.k.a "Graduate Student Design"
#--- { "data": { "datasets" : [{ "borderColor": "#0f0", "borderDash": ["5","10"], "backgroundColor": "#333333", "fill": false }] }, "options": { "scales": { "yAxes": [{ "ticks": { "min": 43, "max": 83 } }] } } } ---# #--- { "data": { "datasets" : [{ "borderColor": "#0f0" }, { "borderColor": "crimson" }, { "borderColor": "cyan" }] }, "options": { "scales": { "yAxes": [{ "ticks": { "min": 43, "max": 83 } }] } } } ---# #--- { "data": { "datasets" : [{ "borderColor": "#0f0" }, { "borderColor": "crimson" }, { "borderColor": "cyan" }] }, "options": { "scales": { "yAxes": [{ "ticks": { "min": 43, "max": 83 } }] } } } ---#
Not scalable to the increasing demand for AI solutions.

Limited Hardware Resources in Real World

VR/AR Headsets
A backpack full of computers!

We need more efficient algorithms that consumes less computation.
Democratizing Deep Learning

Mechanism: automated design of deep learning models through NAS

Research Directions

    • Automatically design neural networks while optimizing multiple objectives.

    • Automatically design bespoke neural networks for non-standard tasks.

Deployment Aware NAS

  • Autonomous driving needs to balance performance and latency.
  • Mobile applications need to balance performance and power consumption.

Neural architecture search is a multi-objective optimization problem

Automating DNN Design

Search Process

  • NSGA-NET: Neural Architecture Search using Multi-Objective Genetic Algorithm, GECCO 2019
  • Multi-Objective Evolutionary Design of Deep Convolutional Neural Networks for Image Classification, IEEE TEVC 2020

Versatility: NAS Beyond Standard Datasets

  • Summary: Next generation of machine learning systems have to be designed with efficiency constraints.
  • NSGANetV2:Evolutionary Multi-Objective Surrogate-Assisted Neural Architecture Search, ECCV 2020
  • Neural Architecture Transfer, IEEE TPAMI 2021

Summary of Research Agenda

Build machine learning systems that are fair, trustworthy and efficient.

Open Source Code: https://github.com/human-analysis
Papers: http://hal.cse.msu.edu/papers