There’s a pattern I’m seeing in almost every enterprise AI project. Someone discovers how capable frontier models are, and suddenly every problem gets routed to an LLM.
It works…until the bill arrives.
Recently, we worked with a customer that needed an AI search term classification system capable of classifying millions of search terms every week. You could solve that by sending every search term to a frontier model. It would also be one of the most expensive ways to solve it.
The better architecture turned out to be surprisingly familiar.
Why Using an LLM for Every Prediction Is a Costly Mistake
Too many AI teams ask:
“Which LLM should we use?”
That’s rarely the right question.
The better question is:
“Where does an LLM actually create leverage?”
Those aren’t the same thing.
Good AI engineering isn’t about using the smartest model everywhere. It’s about putting the smartest model in the right place.
It’s about knowing where you don’t need one.
The Challenge: AI Search Term Classification at Scale
Our customer needed to classify millions of search terms every week.
Think queries like:
- sherpa dog bed
- sherpa throw blanket
Every search term needed to land in the correct category.
The straightforward approach looked like this:
Search Term
↓
LLM
↓
Category
Simple.
Powerful.
Expensive.
Instead, we built an AI search term classification architecture that combines embeddings, traditional machine learning, and frontier models.
Why Embeddings Changed the Game
The first step wasn’t classification.
It was generating embeddings.
If embeddings are new to you, think of them as turning language into coordinates in a mathematical space.
Suddenly, phrases like:
- sherpa dog bed
- orthopedic dog bed
- pet blanket
naturally live close together.
Meanwhile:
- sherpa throw blanket
- fleece blanket
- living room throw
cluster somewhere else.
No engineer had to invent a “pet product” feature.
No one had to manually define what makes a blanket different from a dog bed.
The embedding model had already learned a useful semantic representation of those relationships. No human can point to one dimension and say, “That’s the pet feature.” The meaning is distributed across hundreds or thousands of dimensions, and the classifier learns the boundaries from the data.
That alone eliminates an enormous amount of engineering work.
Why We Used an SVM Instead of an LLM
Once we had embeddings, we trained a Support Vector Machine (SVM).
Yes…
An SVM.
A machine learning algorithm that’s been around since the 1990s.
Why?
Because it was the right tool.
The relationships didn’t separate cleanly with a linear decision boundary. We needed something that could model more complex boundaries through the embedding space.
Once trained, the SVM could classify millions of search terms quickly, accurately, and for a tiny fraction of the cost. The frontier model was reserved for the work only it could do, while embedding models generated vector representations, and the LLM labeled representative examples and refreshed the training data as distributions changed.
How Generative AI Makes Traditional Machine Learning Better
This is where things get interesting.
Five years ago, building this pipeline would have been painful.
Not because of the classifier.
Because of everything around it.
Machine learning teams spent weeks doing two things:
- Engineering features.
- Labeling thousands of examples.
Generative AI dramatically simplified both bottlenecks. Embeddings replaced much of the manual feature engineering, while an LLM generated the initial training labels and refreshed them as the data evolved.
We had the frontier model label roughly a thousand representative search terms.
That became our training set.
As new search behavior emerged, the LLM periodically labeled fresh examples, and the classifier could be retrained with minimal human effort.
The frontier model didn’t disappear. It simply moved to where it creates the most leverage.
Instead of making millions of predictions…
…it enabled a cheaper model to make millions of predictions.
That’s a very different architecture.
The Future of Enterprise AI Is Better Architecture, Not Bigger Models
One misconception I hear all the time is that LLMs are replacing traditional machine learning.
I’m seeing the opposite.
Generative AI is making traditional machine learning dramatically easier to build.
- Feature engineering? Mostly automated.
- Label generation? Mostly automated.
- Model retraining? Increasingly automated.
The result isn’t fewer classical ML systems.
It’s more of them.
The best production AI systems aren’t built around LLMs.
They’re built around good engineering decisions.
Use frontier models where they create leverage.
Use traditional machine learning where the problem is repetitive, high-volume, and predictable.
The whole system is still powered by generative AI. We simply moved the LLM to the part of the pipeline where it creates the most leverage.
The result behaves like a modern AI system—but runs much closer to traditional machine learning economics.
That’s how you build systems that scale—not just technically, but financially.
Frequently Asked Questions
What is AI search term classification?
AI search term classification is the process of automatically assigning search queries to predefined categories using machine learning or AI. It’s commonly used in e-commerce, advertising, search, and recommendation systems to improve relevance and downstream decision-making.
Why not use an LLM to classify every search term?
While LLMs can accurately classify search terms, they’re often too expensive for high-volume prediction tasks. A hybrid architecture using embeddings, traditional machine learning, and LLMs for edge cases typically delivers similar accuracy at a much lower cost.
What are embeddings in machine learning?
Embeddings are numerical representations of text that capture semantic meaning. Similar words and phrases are positioned close together in vector space, allowing machine learning models to recognize relationships and patterns without manually engineered features.
Why use an SVM instead of an LLM?
Support Vector Machines (SVMs) remain highly effective for many classification problems, especially when working with high-quality embeddings. Once trained, an SVM can process millions of predictions quickly and inexpensively, making it one excellent choice for large-scale inference workloads where reasoning isn’t required.
How do embeddings reduce feature engineering?
Before modern embedding models, machine learning teams often spent weeks designing handcrafted features for every classification problem. Embeddings automatically capture semantic relationships within the data, eliminating much of that manual work and allowing models to learn from richer representations with far less engineering effort.
How can enterprise AI teams reduce LLM inference costs?
One of the most effective ways to reduce LLM costs is to reserve frontier models for tasks that require reasoning, generation, or handling ambiguous edge cases. For repetitive, high-volume predictions, combining embeddings with traditional machine learning significantly reduces inference costs while maintaining strong performance.
When should you use traditional machine learning instead of generative AI?
Traditional machine learning is often the better choice when solving repetitive, well-defined prediction problems at scale. Generative AI excels at reasoning, language generation, and creating training data, while classical machine learning is typically faster, more cost-effective, and easier to operate for routine classification tasks.
What is a hybrid AI architecture?
A hybrid AI architecture combines generative AI with traditional machine learning, allowing each technology to handle the tasks it’s best suited for. For example, an embedding model generates vector representations of text, an LLM labels representative examples or handles ambiguous edge cases, and a lightweight classifier performs the majority of high-volume predictions. This approach improves scalability, reduces costs, and delivers more efficient production AI systems.
Is traditional machine learning still relevant in the age of LLMs?
Absolutely. Generative AI hasn’t replaced traditional machine learning—it has made it more powerful. Embeddings and LLM-generated training data remove much of the manual effort involved in feature engineering and labeling, allowing classical models to solve high-volume prediction problems more efficiently than ever. In many production AI systems, the best results come from combining LLMs with proven machine learning techniques rather than replacing them entirely.
Building an AI search term classification system or looking for ways to reduce LLM inference costs?
We help enterprise teams identify where traditional ML, agentic workflows, and frontier models each create the most value—so you get better performance without unnecessary cost. Contact us to review your AI architecture.
Related Insights
If you found this article useful, explore these related insights on building practical, production-ready AI systems:
