The question.
Natural-language-to-SQL demos can generate convincing answers while silently changing metric definitions, crossing access boundaries, or querying the wrong time window.
The design places a semantic metric layer between language generation and warehouse execution. Every answer retains the query, filters, freshness, and relevant caveats.
The proposed approach.
- Resolve questions against approved measures and dimensions
- Validate SQL against schema and access constraints before execution
- Display the executed query and supporting result table
- Allow users to refine assumptions and save reproducible analyses
Target architecture.
Business questionMetric resolverQuery plannerSQL validationWarehouseChart and evidence
The difficult parts.
- Ambiguous definitions of revenue and active users
- Join fan-out and missing denominators
- Freshness and row-level access consistency
How to evaluate it.
- Metric and SQL correctness on expert-authored question sets
- Answer consistency under paraphrased questions
- Unsupported-claim rate and query cost
Planned deliverables.
- A metric catalog and analytical question suite
- An auditable question-to-chart reference flow
- A governed answer and query interface
Design principles.
An attractive chart is not evidence of a correct query.
Make the metric definition visible alongside the answer.