Posts

A deep dive into distributed training and efficient finetuning - DeepSpeed ZeRO, FSDP, practical guidelines and gotchas with multi-GPU and multi-node training

Diving into “An Observation on Generalization”, a talk by Ilya Sutskever at the Large Language Models and Transformers Workshop, Simons Institute, 2023.

Who doesn’t like a good quote?

How do you reason about moral rules? Isn’t it all relative? What can we learn from Adam Smith and Newton’s third law of motion?