Multi-Objective Neural Architecture Search


3M Seminar

Vishnu Boddeti

October 23, 2020

VishnuBoddeti

Computer Vision Today

DNN Design Involves...


Deep Neural Network

Connectivity

Operations
Many More $\Huge \dots$

Early DNN Architecture Development

  • Primarily driven by skilled practitioners and elaborated design.
    • a.k.a "Graduate Student Design"
#--- { "data": { "datasets" : [{ "borderColor": "#0f0", "borderDash": ["5","10"], "backgroundColor": "#333333", "fill": false }] }, "options": { "scales": { "yAxes": [{ "ticks": { "min": 43, "max": 83 } }] } } } ---# #--- { "data": { "datasets" : [{ "borderColor": "#0f0" }, { "borderColor": "crimson" }, { "borderColor": "cyan" }] }, "options": { "scales": { "yAxes": [{ "ticks": { "min": 43, "max": 83 } }] } } } ---# #--- { "data": { "datasets" : [{ "borderColor": "#0f0" }, { "borderColor": "crimson" }, { "borderColor": "cyan" }] }, "options": { "scales": { "yAxes": [{ "ticks": { "min": 43, "max": 83 } }] } } } ---#
Not scalable to the increasing demand for AI solutions.

Automating DNN Design



NAS Outperforms Human Design

NAS Has Many Applications


Image Classification


Video Recognition
Object Detection

Instance Segmentation
Semantic Segmentation

Medical Image Classification

Trends in Neural Architecture Search


#--- { "data" : { "datasets" : [{ "backgroundColor": "darkseagreen" }] }, "options": { "cutoutPercentage": 0, "responsive": true, "legend": { "labels": { "fontColor": "white" }}, "scales": { "xAxes": [{ "ticks": { "fontColor": "#ffffff" } }], "yAxes": [{ "ticks": { "fontColor": "#ffffff" } }] } } } ---#

,Academia, Industry My first dataset, 25, 75



  • An exponential increase in interest of NAS.
  • Primarily drive by industry
  • Commercial products/service

NAS Overview

NAS Components

  • Search Space
  • Search Methods
  • Performance Predictor

Evolutionary Search

  • Evolutionary Operations
    • Mutation
    • Crossover

Limited Hardware Resources in Real World

VR/AR Headsets
A backpack full of computers!

We need more efficient algorithms that consumes less computation.

Deployment Aware NAS

  • Autonomous driving needs to balance performance and latency.
  • Mobile applications need to balance performance and power consumption.

Neural architecture search is a multi-objective optimization problem

NAS as a Multi-Objective Problem

Our solution: Population Based Multi-Objective Optimization

Search Process



Search Efficiency



Multi-Objective Search

Multi-Objective Search

Evaluating NN Architecture Is Expensive

Our Approach: fine-tune supernet

Efficient Search Through Surrogate Modeling



#--- { "data" : { "datasets" : [{ "backgroundColor": ["cornflowerblue","cornflowerblue","cornflowerblue","cornflowerblue","cornflowerblue"] }] } } ---# #--- { "data" : { "datasets" : [{ "backgroundColor": ["cornflowerblue","cornflowerblue","cornflowerblue","cornflowerblue","cornflowerblue","orange"] }] } } ---#
  • AmoebaNet takes one week on 450 GPU cards.
  • Our approach takes one day on 8 GPU cards.
    • $77\times$ more efficient

  • Google AAAI'19 Google CVPR'18 MIT ICLR'20 Google CVPR'19 Google ECCV'18
  • Ours ECCV'20

Comparison to Existing Methods

  • New state-of-the-art 80.5% ImageNet Top-1 accuracy under mobile setting.

Generalization

Image Corruptions

Robustness to Image Corruptions

Versatility: NAS Beyond Standard Datasets

    • Superordinate Classes
    • Sufficient training data (5,000 images per class)
    • Superordinate Classes
    • Insufficient training data (500 images per class)
  • Existing NAS methods:
    • Train on large scale datasets.
    • Adapt weights only to new datasets.
  • Our approach: an integration of Transfer Learning and NAS.
    • Adapt both, weights and architectures to new datasets.

Comparison to Google AutoML

Oxford Flowers
    • #Class = 102
    • Train Size = 20 imgs/class
Oxford-IIIT Pets
    • #Class = 37
    • Train Size = 100 imgs/class
FGVC Aircraft
    • #Class = 100
    • Train Size = 66 imgs/class
    • Our approach is consistently more efficient ($3\times$-$9\times$) than Google AutoML


  • Zhichao Lu et al "NSGANetV2:Evolutionary Multi-Objective Surrogate-Assisted Neural Architecture Search", ECCV 2020
  • Zhichao Lu et al "Neural Architecture Transfer", Arxiv 2020

Versatility: NAS Beyond Standard Datasets

  • Zhichao Lu et al "NSGANetV2:Evolutionary Multi-Objective Surrogate-Assisted Neural Architecture Search", ECCV 2020
  • Zhichao Lu et al "Neural Architecture Transfer", Arxiv 2020

Versatility: NAS for Medical Imaging

Versatality: NAS for Instance Segmentation

Network #MAdds() $AP_{bbox}$ $AP_{mask}$
MobileNetV3 219M 44.02% 43.60%
FBNetV2 238M 44.83% 43.86%
NAT-M1 225M 45.23% 44.27%

Versatality: NAS for Semantic Segmentation

Network #MAdds() #Params() mIoU() Acc()
ResNet18+C1 1.8B 11.7M 33.82% 76.05%
MobileNetV2+C1 0.3B 3.5M 34.84% 75.75%
MUXNet-m+C1 0.2B 3.4M 32.42% 75.00%
ResNet18+PPM 1.8B 11.7M 38.00% 78.64%
MobileNetV2+PPM 0.3B 3.5M 35.76% 77.77%
MUXNet-m+PPM 0.2B 3.5M 35.80% 76.33%

Versatality: NAS for Object Detection

Network #MAdds() #Params() $mAP$()
VGG16+SSD 35B 26.3M 74.3%
MobileNet+SSD 1.6B 9.5M 67.6
MobileNetV2+SSDLite 0.7B 3.4 67.4%
MUXNet-m+SSDLite 0.5B 3.2M 68.6%
MUXNet-l+SSD 1.4B 9.9M 71.1%
PASCAL VOC

Summary

  • NAS improves productivity
  • NAS outperforms human design
  • NAS is versatile: has many applications
  • Neural Architecture Transfer democratises utility of NAS

Open Source Code: https://github.com/human-analysis
Papers: http://hal.cse.msu.edu/papers
VishnuBoddeti