Marten van Dijk (Computer Security)
Remote Verifiable Computation without Digital Secrets & Differential Privacy for Stochastic Gradient Descent
Abstract: In this 2 in 1 presentation, we introduce the Computer Security group (CSY).
Our recent work on remote verifiable computation starts with the observation that state-of-the-art secure processor technology in industry has not yet been able to implement efficient resource sharing and at the same time eliminate or protect against side channels as a result of shared caches and other buffers. As of yet, implemented hardware isolation cannot provide confidential computing. On the other hand verifiable computation only requires uninterrupted and untampered execution flow and this is possible with current technology. We address the problem that the output of a verifiable computation still needs to be digitally signed (for proving freshness and authenticity) and this requires a secret digital key – however, we just argued that we cannot rely on any confidential computing. We show how signatures can be generated by means of secure analogue processing without any secret digital footprint – the adversary can be strong in that it can see all digital state present in the processor.
The second part of the presentation discusses how Stochastic Gradient Descent (SGD) can be made Differentially Private (DP). SGD is used for training machine learning models. Can we achieve a high accuracy while guaranteeing strong privacy based on DP? We show how the worst-case adversary in DP proofs can be made slightly stronger to allow proofs for DP guarantees of a much wider class of algorithms. In particular, our algorithmic framework accepts sampling based on shuffling and works with batch clipping, which can be used to implement other optimization algorithms beyond SGD. It is still open whether the algorithmic framework can be used to improve accuracy. We are able to show significant improved memory utilization for mini-batch SGD with batch clipping.
This is based on joint works with W. Burleson, D. Gurevin, C. Jin (@CSY CWI), O. Khan, K. Mahmood, P. H. Nguyen, U. Ruhrmair, and D. P. Sahoo and L. M. Nguyen, N. Nguyen, T. Nguyen, and P. H. Nguyen.
