AI researcher Sebastian Raschka has released the LLM Architecture Gallery, an interactive reference tool that visually compares 93 text-only language model backbones. The site provides technical fact sheets, implementation links, and a side-by-side architecture diff tool for comparing model designs.
Visualizing Architectural Trade-Offs
The gallery covers architectures ranging from historical baselines like GPT-2 XL (1.5B parameters) to large-scale open models such as DeepSeek V3 and Kimi K3, which reach into the trillions of parameters. Each model card breaks down engineering details including context window limits, decoder type, layer configuration, and KV cache usage per token (calculated in bf16 precision).
Rather than sifting through disparate technical reports, users can inspect how attention mechanisms differ across model generations — including variations such as multi-head attention, learned absolute positional embeddings, and custom layer mixes.
Side-by-Side Model Diffs
The platform includes a diff tool that lets users select two models for direct side-by-side comparison. Key data points tracked across the catalog include:
- Decoder and attention mechanisms: breakdowns of attention types, layer mixes, and decoder classifications.
- Memory overhead: KV cache size per token in bf16 precision.
- Licensing and parameters: scales from sub-billion models to multi-trillion-parameter backbones, alongside open-source or custom license terms.
- Direct repository links: to model config files, technical reports, and code implementations.
Scope Limits
Raschka notes the gallery focuses strictly on text decoders and language-model backbones. For multimodal releases, cards describe only the text backbone rather than any auxiliary image or audio components.
Why It Matters
As language model design branches into specialized decoders, custom attention mechanisms, and varied memory footprints, a standardized reference like this can reduce the time researchers spend digging through individual papers and repositories to compare architectural trade-offs.