# nanoGPT

## Docs

- [Benchmarking](https://mintlify.wiki/karpathy/nanoGPT/advanced/benchmarking.md): Use bench.py to profile and benchmark nanoGPT model performance
- [Performance optimization](https://mintlify.wiki/karpathy/nanoGPT/advanced/performance.md): Optimize nanoGPT training speed with PyTorch 2.0 compile, Flash Attention, and precision settings
- [Troubleshooting](https://mintlify.wiki/karpathy/nanoGPT/advanced/troubleshooting.md): Common issues and solutions when training nanoGPT models
- [Causal self-attention](https://mintlify.wiki/karpathy/nanoGPT/api/attention.md): Multi-head causal self-attention implementation with Flash Attention support
- [GPTConfig](https://mintlify.wiki/karpathy/nanoGPT/api/gpt-config.md): Configuration dataclass for GPT model architecture and hyperparameters
- [Model](https://mintlify.wiki/karpathy/nanoGPT/api/model.md): GPT model class and configuration
- [Sampling](https://mintlify.wiki/karpathy/nanoGPT/api/sampling.md): Text generation and sampling utilities
- [Training](https://mintlify.wiki/karpathy/nanoGPT/api/training.md): Training loop and utilities for GPT models
- [Transformer blocks](https://mintlify.wiki/karpathy/nanoGPT/api/transformer-blocks.md): Core transformer components including Block, MLP, and LayerNorm classes
- [Model parameters](https://mintlify.wiki/karpathy/nanoGPT/configuration/model-params.md): Complete reference for GPT model architecture configuration parameters
- [Configuration overview](https://mintlify.wiki/karpathy/nanoGPT/configuration/overview.md): Learn how to configure nanoGPT training using the configurator system
- [Training parameters](https://mintlify.wiki/karpathy/nanoGPT/configuration/training-params.md): Complete reference for all training configuration parameters in nanoGPT
- [Generation parameters](https://mintlify.wiki/karpathy/nanoGPT/inference/generation-parameters.md): Control text generation quality and behavior with sampling parameters
- [Sampling](https://mintlify.wiki/karpathy/nanoGPT/inference/sampling.md): Generate text from trained nanoGPT models using the sampling script
- [Installation](https://mintlify.wiki/karpathy/nanoGPT/installation.md): Set up your environment and install nanoGPT dependencies
- [Introduction](https://mintlify.wiki/karpathy/nanoGPT/introduction.md): Learn about nanoGPT, a simple and fast repository for training and finetuning GPT language models
- [Quickstart](https://mintlify.wiki/karpathy/nanoGPT/quickstart.md): Train your first GPT model on Shakespeare in minutes
- [Character-level training](https://mintlify.wiki/karpathy/nanoGPT/training/character-level.md): Train a character-level GPT on Shakespeare in minutes
- [Distributed training](https://mintlify.wiki/karpathy/nanoGPT/training/distributed-training.md): Scale training across multiple GPUs and nodes with PyTorch DDP
- [Finetuning](https://mintlify.wiki/karpathy/nanoGPT/training/finetuning.md): Finetune pretrained GPT-2 models on custom datasets with minimal compute
- [Reproducing GPT-2](https://mintlify.wiki/karpathy/nanoGPT/training/reproducing-gpt2.md): Train GPT-2 (124M) on OpenWebText with distributed data parallel