Everyone says they want to “detect fake news.” That phrase is hiding two completely different jobs.
Job one: someone makes a claim. Is it true? That’s a fact-checking problem.
Job two: a million people are tweeting about that claim. How do they feel, and what are they actually mad about? That’s a mood problem.
For a grad-school project on the 2020 US election, I built both. And the fun part is: the AI models were the easy bit. The hard, messy, interesting work was everywhere else.
Two jobs, two models
I didn’t try to make one giant model do everything. That almost never works. Instead I picked the right tool for each job.
text comes in
│
┌──────────────┴──────────────┐
▼ ▼
Is this claim fake? How do people feel?
Llama 3 (8B) RoBERTa
trained on labeled claims trained on tweets
│ │
└──────────────┬──────────────┘
▼
What are they arguing about?
(topic grouping)For the fake-news job, I needed a model that understands the meaning of a short, slippery political statement. I used Llama 3, a big language model, and trained it on a public set of claims that experts had already labeled true or false.
For the mood job, I had 1.8 million tweets and zero patience for a slow, expensive model running on all of them. So I used RoBERTa, a smaller, faster model that’s great at reading tone. The right tool isn’t always the biggest one. At 1.8 million tweets, “cheap and fast” wins.
The cheap trick that made the big model affordable
Training an 8-billion-parameter model sounds like it needs a server farm. It doesn’t, if you’re a little clever.
I used a trick called QLoRA. The short version: instead of retraining the whole giant model (expensive, slow, needs lots of memory), you freeze it and train a tiny add-on layer on top. The big model stays put; you only teach the small new piece. It fits on a single regular GPU, and the thing you ship at the end is a few megabytes instead of gigabytes.
76% accuracy might not sound flashy, but these claims are short and politically loaded with no extra context to lean on. They’re genuinely hard. If I’d reported 95%, I’d have assumed I’d cheated somewhere by accident.
What actually broke (and what it taught me)
The model learned to be lazy. The labeled claims weren’t evenly split, so my first model figured out it could look smart just by guessing the most common answer. Accuracy looked fine; the model was useless. The fix was to stop trusting one number and start checking how it did on each type of claim. Lesson that stuck: a single accuracy score can lie to your face.
My two models disagreed on basic cleanup. Early on, the tweet cleaner and the claim cleaner were slightly different, so “the same sentence” got chopped up two different ways depending on which model saw it. I merged them into one shared cleanup step. Boring fix, big difference.
My “topics” were mostly punctuation. I also grouped the tweets by topic to see what people were arguing about. The first run proudly told me the hottest topics were… hashtags and exclamation marks. After cleaning those out, real themes finally showed up.
So what’s the actual takeaway?
The flashy way to describe this is “I fine-tuned an LLM to detect fake news.”
The honest way is “I figured out there were two different problems, used a cheap fast model for the big firehose and a smart one only where it mattered, and refused to believe a single accuracy number without looking underneath it.”
The model is the footnote. Knowing which problem you’re solving, and how you’ll know you solved it, is the actual job.
That’s the habit I took from this one: when someone hands you a big fuzzy goal like “detect fake news,” the first move isn’t to grab a model. It’s to ask what you’re really being asked to do.
Full project: Fake News and Sentiment Analysis.
