https://snuderek.github.io/categories/vintage-slm/ Recent content in Vintage Slm on Derek Hommel Hugo en-us Wed, 15 Jul 2026 17:22:00 +0900 https://snuderek.github.io/posts/vintage-slm-1/ Wed, 15 Jul 2026 17:22:00 +0900 https://snuderek.github.io/posts/vintage-slm-1/ <h2 id="why-this-project">Why This Project</h2> <p>This is the first in a series about my attempt to train a small &lsquo;vintage&rsquo; language model. I wanted to tackle this because in my day job, I was mostly working on agentic systems built on proprietary model backbones, so most of the work was harness construction, tool-use development, and agent evaluation. I did experiment with basic fine-tuning for style transfer using OpenAI&rsquo;s now-defunct fine-tuning API for GPT 3.5-turbo, and also conducted limited open-weight model LoRA fine-tuning using HuggingFace Deep Learning Containers on GCP, but I haven&rsquo;t had the opportunity to train a model from scratch.</p>