AI

A few weeks ago I have implemented Self Attention mechanism from scratch using PyTorch and this post is a sequel to the first one. If not already, do read that article from here Building Self-Attention from scratch. In this post we will explore various steps involved in building Multi-Head Attention, how it’s different from Self-Attention and why it’s needed. What is a Multi-Head Attention In simple words, Multi-Head Attention is an extension of Self-Attention but the main idea here is to use the Self-Attention multiple times in parallel on a same input sequence to understand the hidden intricate relationships. ...

From last couple of months I have been learning the core building blocks of LLM’s using a well known book called Build Large Language Model from Scratch. The book is very well written and highly recommended for someone who is interested in learning the internal architecture of LLM’s, but please note you will need to put some serious time & effort to get hold of it. This writing is inspired by the book hence full credit to the author Sebastian Raschka. ...