[{"data":1,"prerenderedAt":195},["ShallowReactive",2],{"lang-switch-post-\u002Fen\u002Fplaylists\u002Fhow-to-build-and-finetune-slm\u002Fsmall-vs-giant-language-models":3,"post-en-how-to-build-and-finetune-slm-small-vs-giant-language-models":4},"\u002Fplaylists\u002Fhow-to-build-and-finetune-slm\u002Fsmall-vs-giant-language-models",{"id":5,"title":6,"body":7,"cover":177,"date":178,"description":179,"extension":180,"meta":181,"navigation":182,"order":183,"path":184,"playlist":185,"seo":186,"status":187,"stem":188,"tags":189,"__hash__":194},"posts\u002Fen\u002Fplaylists\u002Fhow-to-build-and-finetune-slm\u002Fsmall-vs-giant-language-models.md","SLM vs LLM: the Delivery Bike and the Moving Truck",{"type":8,"value":9,"toc":169},"minimark",[10,25,39,44,47,54,60,66,72,75,82,86,97,105,109,154,158,161,166],[11,12,13,14,24],"p",{},"Picture this: you need to deliver a pizza a few blocks away. Do you call a moving truck for that? Of course not, you call a delivery bike. The truck can haul a fridge, a couch, an entire household, but it's expensive, slow to turn corners, and you pay the full fare even if your \"move\" is one pizza box. That's basically the argument in Chapter 1 of ",[15,16,20],"a",{"href":17,"rel":18},"https:\u002F\u002Fleanpub.com\u002Fhowtobuildandfine-tuneasmalllanguagemodel",[19],"nofollow",[21,22,23],"em",{},"How to Build and Fine-Tune a Small Language Model",", just swap \"move\" for \"language model\".",[11,26,27,28,33,34,38],{},"An ",[29,30,32],"glossary-term",{"definition":31},"Large Language Model, a giant language model like GPT-4\u002F5, Claude or Gemini, with hundreds of billions of parameters trained to know a bit of everything","LLM"," is the moving truck. It writes poetry, solves advanced math, explains quantum physics, knows a bit of everything. But it also costs a lot to run at scale, sends your data to someone else's servers, and you can't pop the hood and swap a part when it keeps failing at the same kind of question. A ",[29,35,37],{"definition":36},"Small Language Model, a small model, typically between 125 million and 7 billion parameters, specialized in one task instead of being a generalist","SLM"," is the delivery bike. It can't do everything, but for a fast, specialized, cheap delivery, it beats the truck every time.",[40,41,43],"h2",{"id":42},"why-small-beats-giant-sometimes","Why \"small\" beats \"giant\" sometimes",[11,45,46],{},"The book lists five practical reasons, and I'd rather retell each one with a real example than stay theoretical:",[11,48,49,53],{},[50,51,52],"strong",{},"Cost."," Running a giant-LLM API at high volume costs anywhere from hundreds to thousands of dollars a month. Training or fine-tuning the giant model from scratch? Millions. The book gives a concrete example I liked: a support center processing 10,000 tickets a day would spend 500 to 1,000 dollars a month on an API. A local model running on your own machine costs about 50 dollars a month, and that's just the electricity bill.",[11,55,56,59],{},[50,57,58],{},"Privacy."," Patient records, legal case files, unpublished research data, none of that should just travel to a third party's server. If a hospital sends patient notes to an outside API, that's a HIPAA violation right there (the US healthcare privacy law). A local model solves that instantly: the data never leaves the building.",[11,61,62,65],{},[50,63,64],{},"Control."," If the giant LLM keeps getting your field's terminology wrong, you have no way to fix it, you just complain and wait for the next version. Your own model, you adjust.",[11,67,68,71],{},[50,69,70],{},"Availability and speed."," APIs go down, change pricing, or slow to a crawl during peak hours. A local model doesn't depend on the internet or anyone else's business decisions but your own.",[11,73,74],{},"The book gives two \"when it's worth it\" examples I liked a lot: a geologist mining mineral data out of thousands of research papers will do better with a model trained on geology vocabulary (\"plagioclase feldspar\", \"phenocryst\") than with a generalist. And a farm using a small vision model to spot plant disease on-site, with no internet dependency, wins on both speed and not paying an API fee per photo.",[11,76,77,78,81],{},"But the rule isn't \"SLM always wins\". If you need broad general knowledge, complex multi-step reasoning, or real-time up-to-date information, the moving truck still wins. The book sums it up in a line I fully agree with: ",[50,79,80],{},"giant LLMs are generalists, SLMs are specialists."," If you can clearly define your task and have relevant data to train on, the small model tends to serve you better. If you can't, the generalist giant covers the gap.",[40,83,85],{"id":84},"what-the-book-will-have-you-build","What the book will have you build",[11,87,88,89,92,93,96],{},"Most of Chapter 1 is really the roadmap for the whole book, and I'll summarize it quickly because it helps place where the two posts I'm writing fit in. The book uses two reference architectures: ",[50,90,91],{},"GPT-2"," (2019, from OpenAI, 125 million to 1.5 billion parameters, the classic starting point) and ",[50,94,95],{},"MiniMind"," (more recent, 26 to 218 million parameters, optimized to run on smaller hardware with a three-stage training pipeline: pretraining, supervised fine-tuning, and direct preference optimization). It's 12 chapters total, organized into four phases: foundations (chapters 1-3, where you understand the architecture and do your first 30-minute fine-tune), deep-dive training from scratch (chapters 4-7), the complete MiniMind case study (chapters 8-10), and production (chapters 11-12, real deployment).",[11,98,99,100,104],{},"I'm only covering ",[15,101,103],{"href":102},"\u002Fen\u002Fplaylists\u002Fhow-to-build-and-finetune-slm\u002Fbuilding-gpt-from-scratch-dom-casmurro","Chapter 2"," for now, which is \"Phase 1\" in the book's own words: build a tiny GPT from scratch to understand the engine before fine-tuning anything pre-built.",[40,106,108],{"id":107},"wrapping-up","Wrapping up",[110,111,112,126],"table",{},[113,114,115],"thead",{},[116,117,118,123],"tr",{},[119,120,122],"th",{"align":121},"left","What I already knew",[119,124,125],{"align":121},"What this chapter settled",[127,128,129,138,146],"tbody",{},[116,130,131,135],{},[132,133,134],"td",{"align":121},"Bigger models are more powerful",[132,136,137],{"align":121},"More powerful isn't the same as \"better for your specific task\"",[116,139,140,143],{},[132,141,142],{"align":121},"Training a model is expensive",[132,144,145],{"align":121},"Only if it's the wrong model for the size of the problem: 50 dollars a month local vs 500 to 1,000 dollars a month API is a math anyone can do",[116,147,148,151],{},[132,149,150],{"align":121},"\"Fine-tuning\" is vague jargon",[132,152,153],{"align":121},"It's literally taking a model that already knows \"language in general\" and adjusting it just enough to know your specific task",[40,155,157],{"id":156},"practical-application","Practical application",[11,159,160],{},"The chapter's most concrete number is that support-center example, 10,000 tickets a day. I didn't invent anything here, these are the two values the book itself cites: a giant-LLM API sits between 500 and 1,000 dollars a month, a local model sits at 50 dollars a month (the electricity bill, since you already own or rent the hardware once). I put them side by side so you can see the scale of the difference at a glance, using the low end of the API range to keep it fair to the smaller number.",[162,163],"slm-cost-bar-chart",{":bars":164,"y-label":165},"[{\"label\":\"API (giant LLM)\",\"cost\":500,\"color\":\"#cc3300\"},{\"label\":\"Local model (SLM)\",\"cost\":50,\"color\":\"#0033cc\"}]","US$\u002Fmonth",[11,167,168],{},"Ten times cheaper, every month, for the same ticket volume. That's the kind of math that separates \"it'd be nice to have my own model\" from \"it makes financial sense to have my own model\". Next post, we build the engine.",{"title":170,"searchDepth":171,"depth":171,"links":172},"",2,[173,174,175,176],{"id":42,"depth":171,"text":43},{"id":84,"depth":171,"text":85},{"id":107,"depth":171,"text":108},{"id":156,"depth":171,"text":157},null,"2026-08-23","Chapter 1 of J. Paul Liu's book argues the giant model isn't always the right call. I tell it through the most Brazilian metaphor I know (a delivery bike vs a moving truck) and check the real cost numbers the book uses.","md",{},true,1,"\u002Fen\u002Fplaylists\u002Fhow-to-build-and-finetune-slm\u002Fsmall-vs-giant-language-models","how-to-build-and-finetune-slm",{"title":6,"description":179},"published","en\u002Fplaylists\u002Fhow-to-build-and-finetune-slm\u002Fsmall-vs-giant-language-models",[190,191,192,193],"slm","llm","cost","hardware","w629qGi_LbJOFTHqaUX4z2WSPvv4Q6GSV-hJ4XvaC6g",1787605213280]