How might LLMs store facts | Deep Learning Chapter 7

Unpacking the multilayer perceptrons in a transformer, and how they may store facts An equally valuable form of support is to share the videos. AI Alignment forum post from the Deepmind researchers referenced at the video's start: Anthropic posts about superposition referenced near the end: Some...

Intermediate - 23 min - AI & Data - 3Blue1Brown

The full course: Learn Deep Learning free

What you'll need

• GPU-ready laptop: Train small models locally.
• Laptop stand: Screen at eye level for long study sessions.
• Noise-cancelling headphones: Focus through long video lessons.

Books

1. Deep Learning with Python by Francois Chollet - View on Amazon
2. Deep Learning by Ian Goodfellow - View on Amazon

Next lessons

• Let's build GPT: from scratch, in code, spelled out. (116 min, Andrej Karpathy, Beginner)
• Transformers, the tech behind LLMs | Deep Learning Chapter 5 (27 min, 3Blue1Brown, Intermediate)
• But what is a neural network? | Deep learning chapter 1 (19 min, 3Blue1Brown, Beginner)
• Gradient descent, how neural networks learn | Deep Learning Chapter 2 (21 min, 3Blue1Brown, Intermediate)
• Let's reproduce GPT-2 (124M) (241 min, Andrej Karpathy, Intermediate)
• The spelled-out intro to neural networks and backpropagation: building micrograd (146 min, Andrej Karpathy, Beginner)
• Attention in transformers, step-by-step | Deep Learning Chapter 6 (26 min, 3Blue1Brown, Beginner)
• Let's build the GPT Tokenizer (134 min, Andrej Karpathy, Intermediate)