AI Analytics Reference Architecture for Data Leaders

McKinsey's 2025 State of AI says 88 percent of respondents report regular AI use in at least one business function, but only about one-third say their organizations have begun scaling AI across the enterprise. The technical reason is often architectural: many teams add an AI interface before they decide where definitions, context, permissions, and evaluation actually live.

What This Architecture Is For

This is not a component glossary. It is a stack-level blueprint for answering one question:

If a business user asks a natural-language question about company data, which layers determine whether the answer is correct, governed, and reusable?

The architecture below is designed for data leaders who want:

• self-serve answers without shadow BI
• reusable metric logic
• governed access paths
• portability across multiple agent interfaces
• an evaluation loop that survives beyond the pilot

If you only need a component definition, read what is a context layer or what is MCP. This post is about how those pieces fit together.

The Six Layers

1. Source and Storage Layer

This is the physical data foundation:

• warehouses and lakehouses
• operational systems
• transformation outputs
• raw access policies

At this layer, data is stored, transformed, and protected. But business users do not think in table names or physical joins. That translation happens higher in the stack.

The important architectural point is that governance should begin here, not here alone. Access policies, lineage primitives, and authoritative tables come from this layer, but they do not automatically become understandable to AI systems.

2. Semantic Layer

The semantic layer standardizes business logic:

• metric definitions
• dimensions
• joins
• aggregation rules

This is the layer represented by tools and patterns such as the dbt Semantic Layer, LookML, Snowflake Semantic Views, and Databricks metric views.

It is the first place where business definitions become executable and reusable, which is why it is foundational to trustworthy AI analytics.

3. Context Layer

The context layer extends the semantic layer into a broader machine-readable operating context.

In practice, that includes:

• semantic definitions
• lineage
• metadata
• business glossary terms
• documentation
• verified examples
• defaults and exclusions
• source-system context

This layer exists because AI systems need more than a formula. They need to know which entities matter, which synonyms map to which terms, which sources are canonical, and how to explain the result in a way a reviewer can check.

That is why context layer vs semantic layer is not a category battle. The semantic layer is a necessary subset. The context layer is the broader operational surface.

4. Policy and Control Layer

The policy layer carries security and governance into AI delivery:

• row and column-level access
• audit logging
• human review rules
• retention and deletion controls
• interface-specific permissions

NIST AI 600-1 reinforces that governance and monitoring belong inside deployment design. Snowflake Cortex Analyst explicitly ties question answering to RBAC and semantic access controls, and BigQuery conversational analytics documents dedicated IAM roles and permissions for agent operations.

If this layer is missing, the architecture might still answer questions. It just cannot be trusted.

5. Agent and Interface Layer

This is where users actually interact with the system:

• Slack bots
• embedded chat
• email digests
• BI copilots
• custom APIs
• general-purpose assistants

The key architectural decision here is whether those interfaces consume shared context or each build their own interpretation of the data.

BigQuery's Conversational Analytics API shows one production pattern: the system receives business context plus tools such as SQL and Python. The MCP specification shows another important pattern: a standardized protocol for exposing resources, prompts, and tools across AI applications.

This is the layer where portability matters. If context is only usable inside one tool, the architecture is brittle.

6. Evaluation and Improvement Layer

Every production AI analytics stack needs a loop for:

• testing real questions
• validating correctness
• capturing failures
• improving context
• tracking regressions

Snowflake Cortex Analyst includes evaluation of semantic views, while Snowflake's verified-query optimization flow uses verified examples to expand coverage. BigQuery authored context recommends structured context, example queries, glossary terms, and relationship definitions for exactly the same reason.

Without this layer, the stack looks complete but cannot improve safely.

How the Layers Work Together

The architecture works like this:

1. data is stored and governed in source systems
2. metric logic is standardized in a semantic layer
3. context broadens those definitions with metadata, documentation, and lineage
4. policies constrain what each interface can expose
5. agents and applications consume that shared context
6. evaluation feeds corrections back into the context and semantic layers

The stack fails when those responsibilities collapse into one layer. For example:

• if you keep definitions inside the chat app, you get shadow BI
• if you rely only on warehouse schemas, you lose business meaning
• if you rely only on semantic definitions, you still miss documentation, examples, and interface behavior
• if you skip evaluation, drift goes undetected

Common Anti-Patterns

The UI-First Stack

The team starts with a chatbot or copilot and tries to add governance later. This creates an attractive interface with no stable definition layer underneath it.

The Semantic-Layer-Only Stack

The team centralizes metric logic and assumes the job is done. That helps, but it leaves gaps around documentation, examples, lineage exposure, and portable agent access.

The Single-Agent Stack

The first agent works, so the team hardcodes context into it. The second interface then has to repeat the work. By the third interface, no two experiences behave the same way.

The Evaluation-Free Stack

The team goes from demo to rollout without a controlled question set or owner-approved baselines. Failures are discovered only after executives start relying on answers.

Design Principles for Data Leaders

If you are reviewing or redesigning your stack, use these principles:

1. Keep business definitions separate from interfaces.
2. Make context portable across multiple agent surfaces.
3. Treat access and auditability as architectural requirements.
4. Give the system verified examples and glossary context, not only schemas.
5. Build a correction loop before expanding scope.

These principles sound basic, but they are what separate a pilot from a reusable operating model.

How Kaelio Fits

Kaelio auto-builds a governed context layer from your data stack. Its built-in data agent (and any MCP-compatible agent) can then deliver trusted, sourced answers to every team.

In the reference architecture above, Kaelio sits between the semantic layer and the agent layer. It ingests definitions, lineage, BI logic, and documentation from across the stack, organizes them into one governed layer, and exposes that layer to multiple AI interfaces.

That means the same approved context can support:

• Kaelio's built-in data agent
• custom applications
• Slack and email workflows
• any MCP-compatible assistant

The value is not only better answers. It is architectural reuse.