Optane PMem as an Enabler for Large DNN Models with Homomorphic Encryption

June 18, 2021

Guillermo Lloret-Talavera (BSC), Marc Jorda (BSC), Harald Servat (Intel),
Fabian Boemer (Intel), Chetan Chauhan (Intel), Shigeki Tomishima (Intel),
Nilesh N. Shah (Intel) and Antonio J. Peña (BSC)

Introduction

Homomorphic Encryption (HE)

• HE is a type of encryption that enables computation of the ciphertext, without use of the secret key

 

• HE can be used to process confidential data in an untrusted environment, for example for Privacy Preserving Machine Learning (PPML)

 

• 100x-10,000x memory and runtime overhead

 

• Intel Optane can be used by cloud service providers to accommodate the memory requirements

Intel Optane Persistent Memory

• Non-volatile byte-addressable memory DIMMS

 

• Compatible with 2nd Generation Intel Xeon Scalable

 

• Allows for large capacity (128 GB, 256 GB and 512 GB)

 

• Power consumption per byte ~10 times lower than DRAM

 

• Read/Write bandwidth ~3/~9 times lower than DRAM

 

• Read/Write latency up to ~6/~30 times higher than DRAM

Objectives

• Analyze the memory requirements for two of the most popular deep neural networks (MobileNetV2 & ResNet50)

 

• Compare the performance of DRAM only systems versus hybrid systems (DRAM + Optane)

 

• Analyze the behavior and access pattern of the different neural network operations

 

• Evaluate whether this particular application is suitable for Intel Optane's Memory Mode.

Experimental Setup

Intel nGraph Compiler

• Open-source graph compiler and runtime for artificial neural networks.

 

• Compatible with popular frameworks like Tensorflow and ONNX

 

• Support for different hardware platforms (CPU, Nvidia/AMD GPU...)

 

• Intel HE-Transformer
• Backend to Intel nGraph Compiler that support Homomorphic Encryption on CPU
• Relies on Microsoft Simple Encrypted Arithmetic Library (SEAL)
• Currently support the CKKS encryption scheme
• Approximate arithmetic for real numbers
• Encrypted addition and multiplication

Deep Neural Network Models

MobileNetV2

ResNet50

• Mobile neural network for visual recognition (classification, object detection and semantic segmentation)

 

• About 9 times less computations than comparable deep neural networks

 

• Two adjustable parameters that directly affect the size of the model (only smallest settings were tested using HE before)
• Width multiplier (0.35 - 1.4)
• Input resolution (96x96 px - 224x224 px)

• Deep neural network used as a backbone for many computer vision tasks

 

• 50 layers deep

 

• Input images resolution: 224x224 px

 

• Never performed inference before with this model using HE

We have used the pre-trained models of both networks.

Our experiments only take into account inference.

MobileNet V2

Extrae & Paraver

Intel VTune Platform Profiler

Intel VTune Platform Profiler results

(Width multiplier: 0.75 | Resolution: 96x96 px)

DRAM traffic

Optane traffic

Bandwidth:

• Total: 42.6 GB/s

Bandwidth:

• Read:  42.6 GB/s
• Write: 14.4 GB/s

Intel VTune Platform Profiler results

(Width multiplier: 0.75 | Resolution: 96x96 px)

• CPU request data to Optane in chunks of 64 bytes

 

• Each  Optane DIMM controller integrates a media prefetch buffer of 256 bytes

 

• If the data is in the buffer the response time is similar to DRAM

 

• In a sequential access pattern 0.75 hit ratio is expected (MISS - HIT - HIT - HIT)

ResNet 50

Analysis

(Batch size 2048)

Conclusions

Thank you!

guillermo.lloret@bsc.es