"AI in your dashboard" is not AI on your data
An assistant that summarises the chart you were already looking at answers the question you had already answered. The expensive question is a different one, and it needs raw data — with the business definitions written down.
Every data tool grew an assistant this year. You open the dashboard, click the shiny icon, and it explains that sales dropped 12% on Tuesday. You were looking at the chart. You already knew.
That is not useless — it saves you the read. But it answers the question that already had an answer. The expensive question is the one that fits no dashboard, because it crosses sources nobody joined.
The three layers, and where each AI stops
| Layer | What the AI does there | The ceiling |
|---|---|---|
| On the chart | Describes and summarises what is already on screen | Sees nothing outside the dashboard's slice |
| On the table | Writes queries against an already modelled database | Gets the business definition wrong if it is not written down |
| On the business | Joins sources, resolves identity, answers "who" | Depends on the data living in one place |
Most of what is sold as data AI lives in the first layer. The third is the one that changes decisions.
Text-to-SQL is not the problem
There is a statistic going around that models now hit around 95% accuracy turning questions into SQL. It is true and misleading at the same time: that measurement runs on well documented benchmark databases, with column names that describe what they hold.
In your database, the model will find four date columns, two amount columns, and a column called status with fourteen possible values, two of which mean a sale. It will write syntactically perfect, semantically wrong SQL. And it will present the result with exactly the same confidence.
The risk of AI on business data is not that it fails. It is that it gets the syntax right and the definition wrong — because then the number looks fine and nobody checks.
What makes the difference: the definition, written once
The piece separating an assistant that misses from one that lands is not the model. It is the business definitions, written somewhere the AI reads — and written once, not repeated in every question.
- What counts as a sale. A typical checkout has approved, completed, disputed, refunded and cancelled states. If "sale" is not defined, every answer uses a different slice.
- Is a renewal a new sale? On subscriptions, counting renewals as acquisition inflates everything that depends on cost per sale.
- Which currency. A single transaction in another currency, summed without conversion, once inflated an entire month's revenue in a case we measured.
- Which date matters. Order date, approval date and payout date give three different answers to the same question.
- What a lead is. An identified visitor, a CRM contact and a list subscriber are three things, and everyone calls all three a lead.
None of these is an artificial intelligence question. They are modelling questions, and they were already hurting the human team before AI arrived — except an analyst asks when unsure, and the model does not.
What identity resolution changes
There is a class of question that only becomes answerable when events are tied to people rather than sessions. The "who" questions:
- Who bought today and had never appeared in the CRM?
- How many sales came from people who clicked an ad more than thirty days ago?
- Which campaign brings the lead that replies, not just the lead that arrives?
- How much of this month's revenue came from existing customers?
In a stack where every tool keeps its own slice, none of these has an answer — not for the AI, not for you. That is why the conversation about AI on business data always ends up being a conversation about where the data lives.
The honest test
Before signing anything with AI in the name, ask a question that crosses two sources. Something like: of yesterday's sales, how many came from someone who visited the site last week through a paid ad?
If the tool asks you to pick a dashboard, it is in the first layer. If it answers with a number, ask for the query behind the number — and check which definition of sale it used.
In CrazyLeads the AI talks to raw data through the MCP connector, scoped to the active project, and the business definitions live in the model itself — not in the prompt of whoever asked.