Build a Large Language Model (From Scratch)

This is a very good book. I recommend to do the code exercises along reading. The author provides all the code, and it's easy to follow in notebooks to really see what is happening. You can modify the code easily and learn a lot. Imho this is very good investment for anyone who wants to learn how LLM work

This review may be pre mature because I’ve only made it through the first two chapters but so far absolutely amazing. The language is perfect. So many concepts that I’ve struggled with for a while are laid out so clearly. I look forward to doing all the exercises and finishing this book But I would just like to thank the author personally because this is a game changer for my understanding of General ML and AI concepts I struggled with in the past.

I've bought tons of ML, DE, programming, cloud architecture books, etc...This book is absolutely fantastic! Especially combined by the current YouTube series published by the author (March 2025).Sebastian's Packt books are also excellent but I must say this book stands on its own. This book is extremely well written and clear, builds each component in the Transformer Architecture piece by piece, it makes me feel like I can actually build an LLM on my own.At a minimum this book will help you understand the Transformer Architecture (Attention Mechanism, Feed Forward, Layer Norm, etc...) rather than importing models from HugginFace and not really know what's going on in the background.If you are like me and are not satisfied with just building RAGs/LLM applications without understanding the model architecture, this book is for you!I'll keep buying from this author as long as the quality of his content is as good as this.

As an Undergraduate in Intelligent Systems Engineering, this book is amazing. definitely had some good points not covered in classes!

I appreciated the book for its thoroughness and attention to detail. However, I believe it would benefit from being printed in color, as many images on the O'Reilly website are more vibrant and clearer when viewed in color. Additionally, enhancing the resolution of some images would improve the overall experience. For these reasons, I would rate the book 4 out of 5. With these adjustments, I think it could easily earn a perfect score of 5 out of 5.

This book was perfect for me. I'm a computer performance specialist, but haven't yet gotten serious about ML and language models. I've read occasional overview articles, so have an idea what things like "vectors" and "matrix multiplication" are, but I didn't see the full picture. I had bought some other machine learning books before that tried to cover everything about everything and never got even half-way through reading them. This book covers not only the practical examples (and source code) with all the steps for training your own toy language models (Python/pytorch code), but also it explained how all the training layers work together in unison. On the training architecture topic, this book did a better job in a handful of pages than all the deep papers I had read in the past, so I should probably have started from this book, not the other way around.Also, the book does a good job incrementally building the knowledge by adding a new layer after another as you progress through the book. Highly recommended!

This book shows step by step all ingredients which are put together in order to build a GPT-2 model from scratch. All functions are explained explicitely in python, before the equivalent functions of pytorch are used. I really liked to follow the book to the end.There is also a discussion forum about the book on github, where readers can ask questions, which are promptly answered by the author.That said, there remain many questions about WHY the method works, and why some steps are made. E.g. why use multihead attention: to my understanding this completely scrambles the embedding vectors, and it is like a miracle that the method works so well. But there were page limits for the book, and and going deeper into this kind of questions would pprobably have doubled the size of the book.

The best way to learn something is to build it for yourself, and that is exactly what this book does for LLMs. You can get explanations of how LLMs work from a lot of sites on the Internet. What this book does uniquely (as far as I know) is combine that information with a guide for you to implement it for yourself. If you finish the book and work through the code examples and exercises, you will have a solid and up-to-date understanding of how LLMs work under the hood.