The Honest Guide To Fine-Tuning Local AI In 2026
Video Overview & Insights
🎁 Get my FREE local AI projects: https://zenvanriel.com/open-source
Learn real AI fine tuning via my FREE starter projects: https://zenvanriel.com/open-source
What model are you going to train?
⚡ Become a high-paid AI Engineer: https://aiengineer.community/join
Fine-tuning an open-source LLM on consumer hardware is no longer a research-lab skill - it's the next career floor for AI engineers in 2026.
I'm building my first own PC. I'll probably go with the 5070 Ti over the 9070 XT only because I'm interested in running local AI. How good is local AI actually, especially with a 16GB VRAM GPU? Looking forward to use LLMs, TTS, photo and video generation, music, and adjust everything to my own liking. Afraid that I'm investing so much and that the results will be disappointing compared to the big services ChatGPT, 11Labs, Suno, Kling etc
In this honest guide, I fine-tuned a 27B Qwen 3.5 model on every YouTube transcript from this channel using my own home lab over a single weekend. No cloud GPUs, no six-figure training run - just a realistic pipeline you can actually replicate. You'll see the raw before-and-after outputs, learn exactly when fine-tuning beats RAG and prompting, and walk through the real 5-step pipeline (data collection, dataset engineering, LoRA training, evaluation, GGUF export) that almost nobody teaches properly.
If you've ever wondered how to make a local LLM sound like you, follow your own writing style, and bake in knowledge that no system prompt can guarantee - this is the series to watch.
The writing on your blog is terrible. Don't make me read things written by AI, if I want to know what the AI has to say I can just ask it.
What You'll Learn:
- The real difference between fine-tuning, RAG, and prompt engineering (and when each one actually wins)
Good video explaining fine tuning. Regarding the hardware part, the most cost effective way is to use GPU compute providers which gives you access to top enterprise GPUs like the Nvidia A100 with 80 gigs of VRAM or even H/B100 GPUs for as low as 75c/hour/gpu. The other day, I made a LoRA finetuning attempt on a Qwen 3.5 2b model using a A100 and in 2h it was done. Total bill = 1.5 euro xD
- How a LoRA adapter works and why you only train 0.5 to 1.5 percent of the parameters
- The simple flowchart for deciding if your use case really needs fine-tuning
there are so many tutorials out there, but since you got "honest" in your title, i knew i had to click it, as all the others aren't honest.
- The 5-step fine-tuning pipeline: data collection, dataset engineering, LoRA training, evaluation, GGUF export
- How to turn raw YouTube transcripts (or logs, docs, examples) into prompt/response training pairs
It is way simpler. "Train Gemma 4 by Google with Unsloth" explains everything right away. Without pretending to give the answer
- Hardware requirements: VRAM needs for 8B, 14B, and 27B models on a single GPU
- NVIDIA vs AMD ROCm vs Apple Silicon MLX: which GPU is actually worth buying for fine-tuning
Hi, what’s your hardware? It would be great if you included it in your video descriptions - that way people can see how fast local models run on specific hardware.
- Why LM Studio and Ollama need a GGUF export and how to ship your model to them
Timestamps:
Wow, such a high quality video. Thanks!
0:00 The honest guide to fine-tuning
0:34 Demo: why base Qwen sounds generic
thanks for your explanation🎉
3:31 What fine-tuning actually is (LoRA adapters explained)
4:36 Fine-tune vs RAG vs Prompt: when to use what
Google colab should be cheapest solution to access hardware I guess? I am trying to fine tune a small model for android and onsloth notebooks on google colab make it very easy and cheap.
8:55 The simple flowchart for choosing fine-tuning
10:00 The 5-step fine-tuning pipeline
Excellent video, far too few views! There's not much information on fine tuning on YT.
13:52 LoRA training explained: why 1 percent of parameters is enough
14:20 Hardware requirements: the VRAM reality
I think you missed the reason why the first response was so out there: it probably interpreted "you are zen" as you telling the AI that it was Zen Buddhism, and in some kind of spiritual enlightenment, thus the metaphorical language :P
17:00 NVIDIA vs AMD vs Apple Silicon: what to avoid
19:00 Evaluation and GGUF export
You always come with the top quality vidz! Great work as always
#LocalAI #FineTuning #AIEngineer #LLM #LoRA #Qwen #RTX5090 #OpenSource #MachineLearning #AITutorial
Connect with me:
Hi and greetings from Germany.
I like your work, watched a lot of your Videos.
I use Visual Studio for Desktop development. But somehow all AI-Assistance is almost focused on VSCode.
What about the others, are them excluded?
I am curious how the integration of the File System into the Workflow would go ...
Assuming a Format-C-Exception or an ext4-rebramd of the HDD, or something very upsetting but nerve breaking.
Have You ever used Visual Studio? And if a may ask about your preferences? Dark, Light or the System-Theme?
I try to finetune my Coffee, need more Sugar, I think … and have a nice day. Be the Zen of the Zen's.
https://www.linkedin.com/in/zen-van-riel
https://www.skool.com/ai-engineer
Great video but I disagree on the apple part, MLX currently supports over 20+ ports of models including Qwen, Meta, and NVIDIA, etc. With mlx-lm-lora, you can fine-tune a 4B model on a 32k context window using under 32GB of RAM. To achieve that elsewhere, you’d need an enterprise GPU costing $5,000+, whereas a $1,500 Mac Mini handles it. The efficiency of unified memory is a game-changer for local fine-tuning as specially when you are also able to link multiple Macs together.
Sponsorships & Business Inquiries: business@aiengineer.community
you move your mouse like a computer move mouse 💬
More User Perspectives
do you think it is doable to finetu
e the model on cloud 5090?
since i only have make mini to run the model
Wow. This is so freaking cool.
@LukeAvedonI genuinely appreciate the value you give in your videos man, would you be interested in a video editor so that you can focus on other stuff while growing your channel?
@ShuaCTRFrom my experience it's high work, low reward
I remember after doing everything right last year, LoRa, mixing and shuffling my data, I still get catastrophic forgetting
I'll stick to RAG for now until it gets better, I only think of fine tuning for audio models on maybe African dialects
im out as soon as that custom model output 😂
@jav65Hi dude, I see you use qwen3.6 27b in lmstudio. Could you please share the details about how much VRAM it actually eats and what quantization/context length do you use? Have you tried coding with one? Are you satisfied?
Thanks in advance)
like find out it from the 1 second of the vide that its AI?
@smilejjjj87"Zen" has other meanings too 😂
@one-anachronismI think it is faster since it does not need to think. more direct. like your question. the thinking is for more complex steps that might need to be done. but great video.
@LearnAsUGrow501what do you think about DPO and ORPO after SFT which you are presenting only SFT, I'm using qwen 3 8b and want to make a philosophy model, but I want to reward thinking processes, I'm working on it already, but I would love to use 3.5 9b because it scored higher in my own testing than gemma 4 26b, but it is multi modal without vision tower making text only harder to train. The problem I had first run is it just q and a's correct answers rather than think and deliberate and debate, long contexts, it just responds in 5 sentences max, my fault tho it was q and a dataset nothing else, I gathered free books on philosophy first and secondary sources and converted them with big llm to include q and a's. Now I'm augmenting the dataset to include thinking and socratic dialogue.
@notcyfhrPerfect! Can’t wait for the actual “tutorial”
@pfabiszewskiNice. I've noticed that some models have their own mind at times, and they start ignoring the md file...lol
@lancemarchettiBrilliant video, thank you. Easy to follow and genuinely useful 👍
@jamesguy6473Do you think fine-tuning could be done to teach the model business rules of your project? Like I have a data classifier algorithm written with traditional code which misses some edge cases because it's not inherently intelligent. Do you think Qwen could be fine tuned on thousands of these kind of examples to "learn" classification rules?
@eyemazedThis video dropped with perfect timing now im just about to start researching how to do this
@FedorLE