Father Drew™ on Twittaer: A list of 50 individual video links about how LLM's actually work. Start here: the b
A list of 50 individual video links about how LLM's actually work.
Start here: the big picture
3Blue1Brown, "Large Language Models explained briefly": https://www.youtube.com/wa...
Andrej Karpathy, "Intro to Large Language Models": https://www.youtube.com/wa...
Andrej Karpathy, "Deep Dive into LLMs like ChatGPT": https://www.youtube.com/wa...
Andrej Karpathy, "How I use LLMs": https://www.youtube.com/wa...
Andrej Karpathy, "State of GPT" (Microsoft Build): https://www.youtube.com/wa...
How transformers work
6. 3Blue1Brown, "But what is a GPT? Visual intro to transformers": https://www.youtube.com/wa...
7. 3Blue1Brown, "Attention in transformers, visually explained": https://www.youtube.com/wa...
8. 3Blue1Brown, "How might LLMs store facts": https://www.youtube.com/wa...
9. StatQuest, "Transformer Neural Networks, ChatGPT's foundation, Clearly Explained": https://www.youtube.com/wa...
10. StatQuest, "Attention for Neural Networks, Clearly Explained": https://www.youtube.com/wa...
11. Yannic Kilcher, "Attention Is All You Need" (paper walkthrough): https://www.youtube.com/wa...
Neural network foundations
12. 3Blue1Brown, "But what is a neural network?": https://www.youtube.com/wa...
13. 3Blue1Brown, "Gradient descent, how neural networks learn": https://www.youtube.com/wa...
14. 3Blue1Brown, "Backpropagation, intuitively": https://www.youtube.com/wa...
15. 3Blue1Brown, "Backpropagation calculus": https://www.youtube.com/wa...
16. 3Blue1Brown, full Neural Networks playlist: https://www.youtube.com/pl...
Build one yourself (code-along)
17. Karpathy, "The spelled-out intro to neural networks and backpropagation" (micrograd): https://www.youtube.com/wa...
18. Karpathy, "The spelled-out intro to language modeling" (makemore): https://www.youtube.com/wa...
19. Karpathy, "Let's build GPT: from scratch, in code, spelled out": https://www.youtube.com/wa...
20. Karpathy, "Let's build the GPT Tokenizer": https://www.youtube.com/wa...
21. Karpathy, "Let's reproduce GPT-2 (124M)": https://www.youtube.com/wa...
22. Karpathy, Neural Networks: Zero to Hero playlist: https://www.youtube.com/pl...
23. Umar Jamil, "Coding a Transformer from scratch on PyTorch": https://www.youtube.com/wa...
24. Umar Jamil, "LLaMA explained": https://www.youtube.com/wa...
University lectures and full courses
25. Stanford CS229, "Building Large Language Models": https://www.youtube.com/wa...
26. Stanford CS336, Language Modeling from Scratch: https://stanford-cs336.git...
27. Stanford CS224N, NLP with Deep Learning: https://web.stanford.edu/c...
28. MIT 6.S191, Introduction to Deep Learning: http://introtodeeplearning...
29. fast.ai, Practical Deep Learning: https://course.fast.ai
30. Hugging Face LLM Course: https://huggingface.co/lea...
31. DeepLearning.AI short courses: https://www.deeplearning.a...
32. Karpathy, "Software Is Changing (Again)" (YC talk): https://www.youtube.com/wa...
Interactive companions (not videos, but they pair well with them)
33. LLM Visualization, a 3D walkthrough of a GPT: https://bbycroft.net/llm
34. Transformer Explainer: https://poloclub.github.io...
35. Karpathy's nn-zero-to-hero code and notes: https://github.com/karpath...
Channels worth following
36. 3Blue1Brown: https://www.youtube.com/@3...
37. Andrej Karpathy: https://www.youtube.com/@A...
38. StatQuest: https://www.youtube.com/@s...
39. Welch Labs: https://www.youtube.com/@W...
40. Umar Jamil: https://www.youtube.com/@u...
41. Yannic Kilcher: https://www.youtube.com/@Y...
42. Serrano.Academy (Luis Serrano): https://www.youtube.com/@S...
43. AI Coffee Break with Letitia: https://www.youtube.com/@a...
44. Computerphile: https://www.youtube.com/@C...
45. Anthropic (interpretability research talks): https://www.youtube.com/@a...
46. Stanford Online: https://www.youtube.com/@s...
47. Google DeepMind: https://www.youtube.com/@G...
48. Sebastian Raschka: https://www.youtube.com/@S...
49. freeCodeCamp: https://www.youtube.com/@f...
50. Two Minute Papers: https://www.youtube.com/@T...
A good order to watch in: #1 , then #6 through #8 for intuition, then #2 and #3 for how training and chat models fit together. After that, go to #17 through #21 if you want to understand it by building it.
Start here: the big picture
3Blue1Brown, "Large Language Models explained briefly": https://www.youtube.com/wa...
Andrej Karpathy, "Intro to Large Language Models": https://www.youtube.com/wa...
Andrej Karpathy, "Deep Dive into LLMs like ChatGPT": https://www.youtube.com/wa...
Andrej Karpathy, "How I use LLMs": https://www.youtube.com/wa...
Andrej Karpathy, "State of GPT" (Microsoft Build): https://www.youtube.com/wa...
How transformers work
6. 3Blue1Brown, "But what is a GPT? Visual intro to transformers": https://www.youtube.com/wa...
7. 3Blue1Brown, "Attention in transformers, visually explained": https://www.youtube.com/wa...
8. 3Blue1Brown, "How might LLMs store facts": https://www.youtube.com/wa...
9. StatQuest, "Transformer Neural Networks, ChatGPT's foundation, Clearly Explained": https://www.youtube.com/wa...
10. StatQuest, "Attention for Neural Networks, Clearly Explained": https://www.youtube.com/wa...
11. Yannic Kilcher, "Attention Is All You Need" (paper walkthrough): https://www.youtube.com/wa...
Neural network foundations
12. 3Blue1Brown, "But what is a neural network?": https://www.youtube.com/wa...
13. 3Blue1Brown, "Gradient descent, how neural networks learn": https://www.youtube.com/wa...
14. 3Blue1Brown, "Backpropagation, intuitively": https://www.youtube.com/wa...
15. 3Blue1Brown, "Backpropagation calculus": https://www.youtube.com/wa...
16. 3Blue1Brown, full Neural Networks playlist: https://www.youtube.com/pl...
Build one yourself (code-along)
17. Karpathy, "The spelled-out intro to neural networks and backpropagation" (micrograd): https://www.youtube.com/wa...
18. Karpathy, "The spelled-out intro to language modeling" (makemore): https://www.youtube.com/wa...
19. Karpathy, "Let's build GPT: from scratch, in code, spelled out": https://www.youtube.com/wa...
20. Karpathy, "Let's build the GPT Tokenizer": https://www.youtube.com/wa...
21. Karpathy, "Let's reproduce GPT-2 (124M)": https://www.youtube.com/wa...
22. Karpathy, Neural Networks: Zero to Hero playlist: https://www.youtube.com/pl...
23. Umar Jamil, "Coding a Transformer from scratch on PyTorch": https://www.youtube.com/wa...
24. Umar Jamil, "LLaMA explained": https://www.youtube.com/wa...
University lectures and full courses
25. Stanford CS229, "Building Large Language Models": https://www.youtube.com/wa...
26. Stanford CS336, Language Modeling from Scratch: https://stanford-cs336.git...
27. Stanford CS224N, NLP with Deep Learning: https://web.stanford.edu/c...
28. MIT 6.S191, Introduction to Deep Learning: http://introtodeeplearning...
29. fast.ai, Practical Deep Learning: https://course.fast.ai
30. Hugging Face LLM Course: https://huggingface.co/lea...
31. DeepLearning.AI short courses: https://www.deeplearning.a...
32. Karpathy, "Software Is Changing (Again)" (YC talk): https://www.youtube.com/wa...
Interactive companions (not videos, but they pair well with them)
33. LLM Visualization, a 3D walkthrough of a GPT: https://bbycroft.net/llm
34. Transformer Explainer: https://poloclub.github.io...
35. Karpathy's nn-zero-to-hero code and notes: https://github.com/karpath...
Channels worth following
36. 3Blue1Brown: https://www.youtube.com/@3...
37. Andrej Karpathy: https://www.youtube.com/@A...
38. StatQuest: https://www.youtube.com/@s...
39. Welch Labs: https://www.youtube.com/@W...
40. Umar Jamil: https://www.youtube.com/@u...
41. Yannic Kilcher: https://www.youtube.com/@Y...
42. Serrano.Academy (Luis Serrano): https://www.youtube.com/@S...
43. AI Coffee Break with Letitia: https://www.youtube.com/@a...
44. Computerphile: https://www.youtube.com/@C...
45. Anthropic (interpretability research talks): https://www.youtube.com/@a...
46. Stanford Online: https://www.youtube.com/@s...
47. Google DeepMind: https://www.youtube.com/@G...
48. Sebastian Raschka: https://www.youtube.com/@S...
49. freeCodeCamp: https://www.youtube.com/@f...
50. Two Minute Papers: https://www.youtube.com/@T...
A good order to watch in: #1 , then #6 through #8 for intuition, then #2 and #3 for how training and chat models fit together. After that, go to #17 through #21 if you want to understand it by building it.
3 hours ago