Context Stability
A Context Test Harness for Power BI Models
A test harness validates that key questions return stable results.
TL;DR
- • Define representative queries and expected outcomes.
- • Use them to detect context regressions.
The problem (layman)
- • Context issues are hard to detect until users complain.
- • Model changes can silently break answers.
Why it matters
- • Tests provide early warning for instability.
- • They enable safe iteration on the model.
Symptoms
- • Unexpected changes in KPI values after model updates.
- • AI answers that drift without data changes.
Root causes
- • No regression tests for filter context.
- • Model updates lack validation checks.
What good looks like
- • A small set of representative queries with expected ranges.
- • Regular test runs before deployment.
How to fix (steps)
- • Define 10–20 representative questions.
- • Capture expected outcomes or ranges.
- • Run tests after every model change.
Pitfalls
- • Testing too many edge cases and ignoring core KPIs.
- • Using stale expected values.
Checklist
- • Test harness defined and documented.
- • Tests run on every release.
- • Failures reviewed and resolved.