Project note
Micro-LLM Lab
CompleteA small training project to understand, at a practical level, what it takes to train a language model from scratch.
What it is
I trained small GPT-style models from scratch and compared validation loss with generated samples. The goal was not to build a model for use; it was to see the work and tradeoffs firsthand.
Built with
PyTorch, a Hugging Face GPT-2 implementation, TinyStories, a 4,096-token vocabulary, and Apple MPS acceleration.
Where it stands
Complete as a learning project. It gave me a practical view of data preparation, model size, training, and evaluation. The next question is how LoRA fine-tuning can adapt an existing model without starting from scratch.