I have been a data analyst for the past ten years. My dream was always to give my stakeholders enough independence with data to answer everyday questions, so I could focus on in-depth analysis. We, as analysts, used to try and do that with dashboards, emails, alerts, and now we move on to AI data assistants. If you've tried building an AI data assistant recently, you probably noticed the gap between the demo and reality.
In a demo, you connect an LLM to your database, ask it to "show me revenue by region," and it writes a perfect query. But when you deploy that same assistant to production, things change. It starts making up column names, misinterpreting business metrics, and delivering answers that look very confident but are completely wrong.
The problem usually isn't the LLM itself. Models like GPT-5.6 Sol, Opus 5, and Gemini 3.6 Flash are incredibly smart. The real problem is context.
For AI to actually analyze enterprise data, it needs the same tribal knowledge that a human data analyst learns over time and is saved in her head. But engineering teams usually fall into one of two traps: Context Starvation or Context Overload.
Trap 1: Context Starvation
Context starvation happens when we treat an LLM like an analyst who has worked at the company for five years, rather than a brilliant but newly hired analyst.
Imagine a user asking the AI: "How many active users did we have last month?"
To an LLM, "active user" is just a regular English phrase. But inside your company, an active user might mean someone who logged in, spent at least five minutes on the platform, and did a specific action. You also probably need to exclude internal test accounts.
If the AI only sees the raw database schema, it takes the path of least resistance. It will likely do a simple COUNT(*) on a users table where status = 'active'. It will proudly present the number, and a business stakeholder might make a bad decision based on it. When LLMs lack specific definitions about your business data, they guess. And in data analysis, guessing is just another word for hallucinating.
Trap 2: Context Overload
Recognizing the starvation problem, teams often go to the other extreme. They think that if the AI needs to know how metrics are defined, they should just give it everything.
They stuff the prompt with the entire database schema, 50 pages of Confluence docs, dbt models, and historical SQL queries. With today's massive context windows, this is technically possible. But practically, it is a bad idea for three reasons:
- The "Lost in the Middle" Syndrome: Research shows that as the context window grows, an LLM's ability to find and use information drops. If the definition of "active user" is buried on line 4,200 of a 10,000-line prompt, the AI will probably ignore it.
- Attention Dilution: When you give an LLM irrelevant tables alongside relevant ones, it struggles to focus. If someone asks about marketing ROI and the AI is also reading complex HR schemas, the chance of a bad join goes up significantly.
- Latency and Cost: Sending 100,000 tokens of static context for every single question leads to slow response times and high API bills. You are basically paying a premium to confuse your own model.
The Middle Path: Dynamic Context
The right foundation for AI data analysis is not providing zero context, nor is it providing all the context. It is providing the exact right context at the exact right time.
To move past fragile RAG setups, when you build your data agents your architecture should shift to intelligent and dynamic context retrieval.
When a user asks a question, an intermediate layer should interpret the intent and assemble a payload containing only the necessary ingredients:
- Structural Context: Only the schemas and relationships relevant to the specific tables needed.
- Semantic Context: The specific business definitions (e.g., the exact formula for "active users" or "gross margin") relevant to the prompt.
- Behavioral Context: A few highly relevant historical queries or edge-case rules ("Always filter out test_accounts when querying revenue").
By injecting this assembled context into the prompt, the LLM gets a focused, distraction free environment.
Building the Right Foundation
Treat context as a priority. Treating context as an afterthought is why so many AI data projects fail to move past the proof-of-concept phase.
An LLM is ultimately a reasoning engine, but reasoning is only as good as the information it is based on. If you starve the AI, it guesses. If you overload the AI, it loses focus. By maintaining a live understanding of your business and feeding the AI only what it needs, you get an analytical tool that you can actually trust. That is the layer modus is built to be.
This is what modus builds.
A context warehouse for data agents - automatically mined, intent-composed, securely delivered.