8 Advanced parallelization - Deep Learning with JAX

Using easy-to-revise parallelism with xmap() · Compiling and automatically partitioning functions with pjit() · Using tensor sharding to achieve parallelization with XLA · Running code in multi-host configurations

Fully Sharded Data Parallel: faster AI training with fewer GPUs

Intro to JAX for Machine Learning, by Khang Pham

Why You Should (or Shouldn't) be Using Google's JAX in 2023

Lecture 2: Development Infrastructure & Tooling - The Full Stack

Top 11 Machine Learning Software - Learn before you regret

Introducing Neuropod, Uber ATG's Open Source Deep Learning

Deep Learning with JAX

High-Performance LLM Training at 1000 GPU Scale With Alpa & Ray

A Brief Overview of Parallelism Strategies in Deep Learning

Breaking Up with NumPy: Why JAX is Your New Favorite Tool