Variance Reduction

Variance Reduction (VR) reduced noise while estimating gradient in SGD. Here’s a brief slide.

We implemented SVRG, SAGA on MPI, see the vr folder.

Asynchronous

See svrg_async.py , yet another MPI implementation mentioned in [Reddi et. al.].