The Interactive Transformer (note: still WIP)
Intro
The Interactive Transformer
Intro

Welcome to the Interactive Transformer

This is an interactive 3D Visualization of a Large Language Model (LLM), of the likes that powers GPT-3 & ChatGPT.

We show a very small model of the same design, to help you understand how these models work.

As well as being interactive, we provide a walkthrough of the model showing the step-by-step process of how it works, with every single add, multiply & math operation described.

Beginner Mode
tok embedpos embed;;~;;;layer normmulti-head, causalself-attention;;;layer normfeedforward;;;transformer ilayer normlinearsoftmaxLLM
Table of Contents
Intro
Introduction
Preliminaries
Components
Embedding
Layer Norm
Self Attention
Projection
MLP
Transformer
Softmax
Output