Conversational BI demos are always impressive and often stall within a month of launch — not because the language model is bad, but because it's answering from raw tables instead of a governed semantic layer that encodes what 'revenue' or 'active customers' actually means. Power BI, Tableau and Looker all now root their natural language features in a semantic layer for exactly this reason; the AI didn't create the modelling debt, it just made it visible at speed in front of executives.
The demo is always impressive. Someone types "show me revenue by region last quarter" into a chat box, and a chart appears in three seconds. Executives in the room visibly relax — finally, analytics without the analyst bottleneck.
Then it gets deployed, and within a month someone asks about "active customers" and gets a number that does not match the one in the monthly review. Trust collapses faster than it built, and the feature quietly stops being used.
This failure is predictable, and it has almost nothing to do with the quality of the language model.
The actual problem
When a human asks for "revenue", they mean a specific thing that their organisation has agreed on. Gross or net of returns. Including or excluding GST. Recognised on invoice or on delivery. Consolidated or entity-level.
A language model pointed at raw database tables has none of that context. It sees a column called revenue_amt and another called net_rev, and it makes a reasonable guess. Reasonable guesses are fine for exploration. They are catastrophic when the answer goes into a board pack, because the wrongness is invisible — the output is fluent, formatted and confidently presented.
This is why the semantic layer became the centre of gravity in 2026 analytics architecture. It is the artefact that encodes "revenue means this" in a form both humans and machines can resolve against.
How the three major platforms handle it
Looker made this bet earliest and most completely. Conversational Analytics uses Gemini to interpret the question, but it resolves that question against the LookML semantic model as its source of truth. Because metrics like revenue and churn rate are defined in LookML, the system uses those definitions rather than inferring from schema.
The technical architecture combines semantic parsing for text-to-SQL, a knowledge graph supplying metadata context, and a reasoning agent that reviews the query and conversation history to pull the right fields and relationships. For heavier work — forecasting, anomaly detection — a code interpreter generates and runs Python rather than forcing everything through SQL.
Because it sits above LookML rather than above the warehouse, the same governance applies whether the underlying data lives in BigQuery, AlloyDB, Redshift, Snowflake or Databricks.
Power BI grounds Copilot in Fabric semantic models. Microsoft's 2026 extension, Fabric IQ, pushes this further by connecting Power BI semantic models into Microsoft 365 Copilot Chat — respecting existing user permissions and using the same models, so answers stay consistent with what users see in their reports. Microsoft's guidance is explicit that semantic models should be configured as AI-ready, with defined relationships, business-friendly field naming, and AI instructions set within the model.
That last requirement is the part organisations skip. A semantic model with fields named dim_cust_v2_final will produce worse AI answers than one with fields named Customer Segment, and no amount of model capability compensates.
Tableau ships Tableau Agent, with Tableau Semantics providing the governed layer inside Data 360. Its 2026 direction has been toward making that governed semantics portable — MCP servers now expose it into Slack, Microsoft Teams, Google Workspace, and third-party assistants including Claude and ChatGPT through marketplace connectors. Tableau Agent in Pulse was upgraded during 2026 for sharper intent recognition and multi-part question handling.
The strategic logic across all three is identical: let people ask questions wherever they work, but make sure every answer resolves through one governed definition.
Why rollouts stall
Devoteam's 2026 analysis of Looker conversational deployments concluded that most failed rollouts come down to LookML quality rather than the technology — and recommended focusing there first. That finding generalises across vendors.
The specific failure patterns:
Incomplete modelling. Half the metrics people ask about are not defined anywhere, so the AI improvises for those and is correct for the rest. Users cannot tell which is which, so they distrust all of it.
Cryptic naming. Warehouse-native names that made sense to the engineer who wrote them provide no signal to a language model trying to map business vocabulary onto schema.
No feedback loop. Nobody reviews what questions users actually asked, which ones returned nothing useful, or which returned something plausible but wrong. Without that loop, the model of user intent never improves.
Skipping the permissions question. If the AI layer does not inherit row-level security correctly, a regional manager can ask a question and receive national data. This is a governance incident, not a bug.
What good looks like
Before enabling conversational analytics, a workable readiness checklist:
- The twenty metrics executives actually use are defined, once, in the semantic layer
- Field names read as business vocabulary, not database shorthand
- Row-level security is tested against the AI path specifically, not just the dashboard path
- There is a named owner for metric definitions and a change process
- You have decided which questions the system should refuse rather than guess at
That last point deserves emphasis. A conversational analytics system that says "I don't have a governed definition for that" is more valuable than one that always answers. Confident wrongness is the failure mode that destroys trust; admitted ignorance is merely inconvenient.
The broader warning
Gartner's 2026 predictions flag that by 2030 half of AI agent deployment failures will stem from insufficient governance platform runtime enforcement and multisystem interoperability gaps, with near-term ungoverned LLM decisions causing financial and reputational loss. The recommended approach is experimentation in controlled, low-risk environments with a required evaluation stage built into analytic workflows.
Applied to conversational BI, that translates simply: pilot it on a domain where you have modelled well, measure answer accuracy against known-correct values, and expand only where the semantic layer is genuinely ready.
The uncomfortable summary
Conversational BI is not a feature you buy. It is a capability you earn by doing modelling work that was always worth doing and that most organisations deferred because nothing forced them to.
The AI did not create that debt. It just made it visible, at speed, in front of executives.
The AI did not create that debt. It just made it visible, at speed, in front of executives.
Referenced in this piece: Google Cloud Looker — semantic modelling (LookML).
Want this level of rigor applied to your own analytics stack?
This guide comes from running BA/BI systems audits for real Indian enterprises — where the actual fix is decided by which stage of your analytics function is broken, not by which tool has the best demo. A Systems Audit tells you exactly where to start.
Book a Systems Audit arrow_forward