A Governed Knowledge Platform for Revenue Growth Management¶

Prototype delivered Delivery Manager / AI Engineer May – September 2026 Client identity and internals withheld. Last verified 2026-09-09.

A global CPG manufacturer could not answer its own questions about its own numbers. I led the design and build of a knowledge platform that could — then made the case for what it would take to trust it at enterprise scale.

The situation¶

Revenue growth management at a large consumer goods company runs on a dense layer of institutional knowledge: KPI definitions, BI measure formulas, data lineage, and business rules that determine what a number means in one market versus another. That knowledge existed — spread across thousands of pages of documentation, BI model definitions, wikis and spreadsheets — but it was not answerable. A business user asking "how is this KPI actually calculated, and does it mean the same thing in every market?" had no reliable way to find out.

The cost was not abstract. Definitional drift between markets undermines confidence in reporting, and every unanswered question routes to a small number of overloaded experts.

My role¶

I owned the technical design and did a large share of the build. I contributed roughly 60% of the platform's commits (431 of ~721) between May and September 2026, spanning every layer — ingestion notebooks, agent provisioning, the evaluation harness, the chat interface, security fixes, and the architecture decision records. A small core team took the platform from ideation to a working proof of concept in roughly three months.

I also wrote and delivered the scale-up case to client leadership. The client owned the domain knowledge and the draft taxonomy; the platform team owned the lakehouse environment it ran on.

Constraints¶

No enterprise semantic contract existed. The taxonomy was a draft, unowned, and not stable enough to build hard dependencies on.
Source quality varied wildly across wikis, SharePoint, BI model definitions and direct uploads — 7 formats, and only some worth extracting in full.
Prototype-grade security posture. Credential handling and hosting assumptions were fit for a demo, not for an enterprise review.
Very small capacity against an open-ended problem, on a timeline set by a commercial decision point rather than by the work.

Key decisions¶

1. Contract validation with quarantine, not best-effort ingestion.

Alternatives considered: ingest everything and let retrieval quality absorb the noise; or validate softly and log warnings.
Why this approach: in a knowledge system, silent partial failure is worse than a hard stop, because it produces confident answers from an incomplete corpus. Rows violating their contract went to a quarantine log instead of degrading the corpus.
Trade-off accepted: a smaller corpus. 542 BI measures passed contract validation out of a source library of 1,102. Half the library sat outside the answerable set until its contract issues were fixed — visibly, which was the point.

2. Route by intent before reaching for a bigger model.

Alternatives considered: a single hybrid-retrieval path for everything, with model capability compensating for retrieval imprecision.
Why this approach: a large share of real questions have exact answers. Deterministic lookup returns them faster, cheaper and more accurately than any retrieval path. Four routes were built: deterministic lookup, hybrid retrieval, hierarchical summarisation for cross-document questions, and natural-language-to-SQL for analytics.
Trade-off accepted: more moving parts, and a new failure mode — a classification error becomes a routing error, which is harder to diagnose than a bad retrieval.

3. Tier-routed ingestion rather than full extraction everywhere.

Alternatives considered: extract full content from every source.
Why this approach: documents were classified along two business dimensions and routed to full-content extraction, metadata pointer, or table extraction, depending on what each source was actually good for.
Trade-off accepted: pointer-tier sources are discoverable but not deeply answerable. That was the correct trade for sources whose value was locating an authority, not quoting one.

4. Build the evaluation harness before scaling features.

Alternatives considered: ship more retrieval capability first and evaluate once the surface stabilised.
Why this approach: it changed the conversation with the business from impressions of a demo to measured quality on their own questions.
Trade-off accepted: build capacity spent on evaluation instead of features, against a hard timeline.

5. A stable response contract behind the prototype surface.

Alternatives considered: build the production interface immediately, or keep iterating on the prototype UI.
Why this approach: the interface that made rapid iteration possible was not the interface that would scale. Preserving grounded answers, citations and conversation behaviour behind a reusable API meant the presentation layer could be replaced without rebuilding the knowledge core.
Trade-off accepted: the prototype UI was disposable by design, and known to be so while people were still demoing it.

System and approach¶

A knowledge platform spanning the full lifecycle, from raw document to governed answer.

Ingestion. A medallion pipeline over a lakehouse. Documents landed in a governed volume, were classified, and were tier-routed as above — 7 formats across 459 source documents.

Enrichment and validation. A fail-loud contract validation layer between raw and curated tiers, with quarantine.

Retrieval. Intent classification with the four distinct paths. 990 curated knowledge articles produced 3,042 enriched chunks, with 96.8% index synchronisation.

Answering. A supervisor agent routing each query across those paths, with tool-calling over 11 governed catalog functions and 4 metric views, returning grounded answers with citations and context continuity.

Evaluation. A competency-question harness with LLM-as-judge scoring — 34 questions covering all four retrieval routes, tracking answer quality, route distribution, context leakage and latency.

Operations. Runtime telemetry on route selection, latency, index health and errors, with distributed tracing across agent calls.

Delivery¶

The prototype worked end to end. That was not the difficult part, and I did not present it as though it were.

Building a retrieval-augmented chat experience has become commodity work. What an enterprise cannot buy off a shelf is an honest account of the distance between a prototype that impresses in a demo and a platform a regulated business can depend on. I delivered that account to client leadership, framed around three decisions that were the actual critical path:

Semantic ownership has to become a control system, not a document. The draft taxonomy accelerated learning but was never an enterprise semantic contract. Scaling required named business owners for each concept, competency questions promoted into a golden question-and-answer set, route-level quality metrics, and regression gates that block a release when semantic quality degrades. Until a business owner can be pointed to for each definition, the platform is a very good search tool rather than a source of truth.

Architecture and security review are design inputs, not final checkpoints. Credential shortcuts, hosting assumptions and data provenance all had to be resolved before industrialisation. Treating enterprise review as a gate you reach at the end is the most common way a promising prototype dies.

The working experience has to be decoupled from the prototype surface. A stable response contract, so the knowledge core outlives the UI.

I structured the investment case around evidence gates rather than a fixed scope: discover, align, enable, harden, scale — each with an explicit go/no-go criterion. The argument to leadership was deliberately modest, and stronger for it: the next phase buys evidence, alignment and options, not another demo.

Evidence of value¶

Measured on the platform as built, between May and September 2026:

A working end-to-end knowledge platform, from multi-source ingestion to grounded, cited answers, built by a small core team in roughly three months.
3,042 enriched knowledge chunks served at 96.8% index synchronisation, backed by contract validation with quarantine rather than silent failure.
A measurable quality baseline — 34 competency questions across 4 routes, LLM-as-judge scored — established before scale-up rather than retrofitted.
36.7s average answer latency. Recorded here because it is the number that most constrains the next phase, not because it flatters the build.
A board-ready scale-up case built on evidence gates, and a target architecture separating governed meaning from channels and agents.

Status, stated plainly: this was a prototype and a proof of concept, built deliberately as the evidence base for a much larger engagement. It demonstrated capability and produced a measured quality baseline. It was not adopted at production scale, and at the point this record was written the scale-up phase was a live proposal with the client's decision still outstanding.

No business-outcome metric is claimed. The platform did not run long enough in front of enough users to produce one, and a number invented for a case study would be worth less than saying so.

What changed in my thinking¶

I would build the golden question set with the business before writing pipeline code. The competency-question harness was the right instrument built at nearly the right time — but 34 questions authored largely from the platform side is a thin baseline. Had those questions come from the business first, they would have been the specification, not the test.

I under-weighted latency. 36.7 seconds is defensible for a research question and disqualifying inside an operating workflow. I treated it as an optimisation problem for later; it is closer to a design constraint that should have shaped routing from the start.

Taxonomy breadth was the wrong early investment. Coverage felt like progress. Depth on a small set of owned, high-traffic definitions would have produced a more convincing system and surfaced the ownership problem months earlier.

Naming the gap yourself is the most valuable thing you can deliver. The prototype was not the deliverable that mattered most. Telling the client precisely what it could not yet do, and what it would cost to close that — before someone found it in a security review — was.

Technologies¶

Databricks · Unity Catalog · Delta Lake · Vector Search · MLflow · Python · DSPy · Streamlit · Power BI / DAX · Docling · medallion architecture · agentic RAG · supervisor-router multi-agent · hierarchical summarisation · LLM-as-judge evaluation

Artefacts¶

The platform repository is client-confidential and cannot be published. The patterns derived from it are:

Disclosure boundary: client identity, product names, internal system paths and ticket identifiers are withheld. This record describes architecture, method and measured outcomes only. Generic technology names are retained — they describe the craft, not the client.