KellerJordan/modded-nanogpt

Star KellerJordan / modded-nanogpt NanoGPT (124M) in 2 minutes

Python5.7K stars868 forks

What it does

Modded-NanoGPT is an optimized implementation of NanoGPT that dramatically reduces training time for language models using advanced techniques. It achieves a target cross-entropy loss of 3.28 on the FineWeb dataset in under 90 seconds with 8 NVIDIA H100 GPUs.

Star history

Not enough history yet — 1 day(s) recorded. The daily snapshot builds this up.

Tracking

Last trending: 2026-02-21

Creator kit

Hook

Discover how Modded-NanoGPT is revolutionizing language model training speed with cutting-edge techniques!

Content angles

Create a tutorial on how to set up and use Modded-NanoGPT for fast language model training.
Analyze the performance improvements and techniques used in Modded-NanoGPT compared to traditional models.
Interview contributors to discuss the collaborative efforts behind Modded-NanoGPT and its impact on AI research.

Who should care

AI researchers, machine learning engineers, and developers interested in optimizing model training.