Create a Large Language Model from Scratch with Python – Tutorial

Learn how to build your own large language model, from scratch. This course goes into the data handling, math, and transformers behind large language models. You will use Python. ✏️ Course developed by @elliotarledge ⭐️ Contents ⭐️

Beginner - 344 min - AI & Data - freeCodeCamp.org

The full course: Learn Deep Learning free

What you'll need

• GPU-ready laptop: Train small models locally.
• Laptop stand: Screen at eye level for long study sessions.
• Noise-cancelling headphones: Focus through long video lessons.

Books

1. Deep Learning with Python by Francois Chollet - View on Amazon
2. Deep Learning by Ian Goodfellow - View on Amazon

Next lessons

• Let's build GPT: from scratch, in code, spelled out. (116 min, Andrej Karpathy, Beginner)
• Transformers, the tech behind LLMs | Deep Learning Chapter 5 (27 min, 3Blue1Brown, Intermediate)
• But what is a neural network? | Deep learning chapter 1 (19 min, 3Blue1Brown, Beginner)
• Gradient descent, how neural networks learn | Deep Learning Chapter 2 (21 min, 3Blue1Brown, Intermediate)
• Let's reproduce GPT-2 (124M) (241 min, Andrej Karpathy, Intermediate)
• The spelled-out intro to neural networks and backpropagation: building micrograd (146 min, Andrej Karpathy, Beginner)
• Attention in transformers, step-by-step | Deep Learning Chapter 6 (26 min, 3Blue1Brown, Beginner)
• How might LLMs store facts | Deep Learning Chapter 7 (23 min, 3Blue1Brown, Intermediate)