Contents

• Welcome!
  • Abstraction
  • Topics
• Tips AutoGrad on Python
  • PyTorch
  • JAX
• Consensus Optimization
  • Consensus Matrix
  • Consensus Algorithms
  • Application
• MPI4Py 101
  • Beginning
  • Collective Communication
  • One-sided communication
  • Non-blocking communication
  • Probing node
• Variance Reduction
  • Asynchronous