From Building a Data Warehouse to Building a Governance System Around It

Two months into building a data platform with an AI coding agent, an AI partner in a separate chat session offered a diagnosis I did not expect: “The agent has crossed a line from ‘building a warehouse’ into ‘building a governance system around the warehouse.’” It was meant as amused commentary. I took it as the title of this post.

I have been building a data platform for financial planning and analysis. The project started in Power BI — Power Query for the transformations, Power Pivot for the dimensional model, CUBE functions at the reporting layer. I wrote about that architecture in an earlier post. But as the project grew, I hit the limits of the locked-down corporate environment: no admin rights, PowerShell in Constrained Language Mode, no ability to install packages. When I finally found a way to access pip, I migrated the whole pipeline to Python, DuckDB, Parquet, and SQLite3. A proper stack. The kind of thing I had been wanting to build from the start.

The AI agent was central to the build. I was using Claude — Opus models, mostly — through the Claude Code interface. For the first few weeks, everything worked the way I described in my earlier post about the virtuous cycle: I supplied the architecture, the agent supplied the syntax, and every day the balance shifted a little more toward my own fluency in the new tools. The cycle was accelerating. Then it stopped.

What happened was not dramatic. It was gradual, which made it harder to diagnose. At some point — I could not pin down exactly when — the agent started losing the thread. It would forget decisions we had made three turns ago. It would propose schema changes that contradicted the dimensional model we had just agreed on. It would solve a problem, then re-introduce the same bug two exchanges later. The conversations became circular. I found myself re-explaining the project architecture from scratch in prompts that grew longer and longer, which only seemed to make things worse.

I had seen this pattern before, on a smaller scale, when I was building a flashcard application. I never really recovered from it. My conclusion at the time was that I needed to step up my own learning so I could keep a better handle on the agent — understand the code well enough to catch its mistakes and steer it back on course. That was not wrong, but it slowed development dramatically. Instead of the virtuous cycle accelerating, I was spending most of my time studying rather than building. The data platform was too complex for that approach to scale. Starting over meant losing hours of accumulated context about table relationships, transformation logic, and design decisions. And yet staying in the same conversation meant watching the agent’s performance degrade in real time.

I did what I always do when something does not make sense: I went looking for an explanation.

The first book I picked up was Mastering Claude AI by Ryan Dickey. It was not helpful. The advice was the kind of thing you figure out in your first week of using ChatGPT — be specific in your prompts, provide examples, iterate. Prompt engineering, the book called it. I had been doing all of that already. The problem was not that my prompts were bad. The problem was that good prompts were no longer enough.

The second book was Agentic Coding with Claude Code by Eden Marco, and it changed my understanding of the problem entirely. Marco did not start with prompt engineering. He started with context engineering. The distinction is not semantic. Prompt engineering is about crafting a single instruction well. Context engineering is about managing the entire information environment in which the agent operates — what it knows, what it remembers, what it can access, and critically, what it forgets.

The light bulb went off. My problems were not prompt problems. They were context problems. And the context was failing in ways I could now name.

An article by Drew Breunig, “How Long Contexts Fail,” gave me the taxonomy. Breunig identifies four failure modes, each of which I recognized from my own experience.

Context poisoning is when a hallucination or error makes it into the context and stays there. The agent generates an incorrect assumption early in the conversation — say, that a particular column is a foreign key when it is actually a derived field — and that error gets referenced in subsequent exchanges, compounding into a chain of decisions built on a false premise. The deeper the error sits in the conversation history, the harder it is for the agent to course-correct, because correcting it means unwinding everything that followed.

Context distraction is when the accumulated history overwhelms the agent’s training. As the conversation grows, the model starts leaning on what it said before rather than reasoning from first principles. The Gemini team observed this with their Pokémon-playing agent: past 100,000 tokens, the agent began repeating earlier strategies instead of developing new ones. I was seeing the same thing. My agent would propose the same flawed migration script it had already tried, because the conversation history made that approach feel familiar.

Context confusion is when superfluous information in the context leads the model to make poor choices. Every tool definition, every schema description, every instruction that is loaded into the prompt has to be attended to by the model. More is not better. A Databricks study found that model correctness began degrading around 32,000 tokens for some models — well before the context window was full. The problem is not capacity. It is noise.

Context clash is when different parts of the context contradict each other. This happens naturally in multi-turn conversations where the user changes direction, or where the agent proposes something and the user corrects it but the original proposal remains in the history. The Microsoft and Salesforce teams demonstrated that when information arrives in stages rather than all at once, model performance drops by an average of 39 percent. The agent makes assumptions early, and when later information contradicts those assumptions, it does not fully recover. As the researchers put it: “When LLMs take a wrong turn in a conversation, they get lost and do not recover.”

All four of these were happening to me simultaneously, and I did not have the vocabulary to distinguish them until I read the article. Once I could name the failure modes, I could start addressing them.

Eden Marco’s book pointed me to the solution that I should have been using all along: CLAUDE.md files. These are configuration files that sit in your project directory and provide the agent with persistent context — the project’s architecture, coding conventions, key decisions, things not to forget. They survive across conversations. They do not get compressed when the context window fills up. They are, in effect, the project’s constitution: the foundational rules that every session starts from, regardless of how long the conversation runs or how many times the context gets summarized.

I had been aware of CLAUDE.md files when I used Claude from the command line. But when I switched to the Claude desktop application and Claude Code, I assumed the setup carried over. It did not. For two months, I was running every conversation without a project constitution. Every session started from zero. Every agent had to rediscover the project’s architecture by reading files, making inferences, and asking me questions I had already answered a dozen times before. No wonder the performance felt like it was getting worse — it was not that the models were regressing. It was that each new conversation was genuinely uninformed.

But the CLAUDE.md was only the beginning. Once I started thinking in terms of context governance, other pieces fell into place.

I had been reading Amy Edmondson’s The Right Kind of Wrong, a book about organizational failure. Her central argument is that intelligent organizations do not try to eliminate failure — they try to eliminate predictable failure. You want to make new mistakes, not repeat old ones. That idea mapped directly onto my experience with the agent. The same bugs kept coming back not because the agent was incapable of avoiding them, but because it had no memory of having made them before. Each conversation was a clean slate, including a clean slate of past errors.

So I started building an error database. Not a complex system — just a structured log of mistakes the agent had made, with the context that produced them and the fix that resolved them. A CLAUDE.md section could reference it. A checklist could enforce it.

The checklist idea came from Atul Gawande’s The Checklist Manifesto. Gawande’s insight is that in complex environments, failure is not usually caused by ignorance — it is caused by inattention. Surgeons know how to avoid infections. They just sometimes forget to wash their hands. The checklist is not a training tool. It is a governance tool. It forces a pause at critical junctures to verify that the known-important steps have been taken.

Applied to agentic coding, the checklist became something the agent would consult before making changes to the schema, before running a migration, before refactoring a module. One item on the checklist: check the error database for known failure patterns. Another: confirm that the proposed change is consistent with the dimensional model described in the CLAUDE.md. A third: if the context has been compressed, re-read the key architecture files before proceeding.

This was the moment the AI partner flagged. I had spent weeks building a data platform — tables, transformations, reporting layer. And now I was spending my time building governance infrastructure around the platform: configuration files, error databases, checklists, context management protocols. The warehouse was done, or close enough. What I was building now was the system that kept the warehouse from falling apart every time a new conversation started.

It took me a while to see the irony, but it is hard to miss once you do. I am an FP&A data engineer. My day job is building data infrastructure for a corporate organization. And here I was, at night and on weekends, building data infrastructure for an AI agent — and running into exactly the same problems that organizations run into when they scale their data systems. Governance is not glamorous. Nobody sets out to build a governance layer. You set out to build the thing, and then you discover that the thing does not hold together without rules, memory, and accountability.

Every enterprise data team learns this the hard way. You start with a database. Then you need a data dictionary so people know what the columns mean. Then you need lineage tracking so you can tell where a number came from. Then you need access controls, audit logs, quality checks, documentation standards. The warehouse is 20 percent of the work. The governance is the other 80.

The AI agent recapitulates this journey in miniature. When the project is small, context management is trivial — everything fits in the conversation, the agent remembers what you discussed, performance is excellent. This is the honeymoon phase, and it feels like magic. Then the project grows. The context window fills up and gets compressed. Decisions made early in the project lose their salience. Errors creep in and persist. The agent starts working against its own prior output. And suddenly you are not writing code anymore. You are writing policy.

I do not think this is a temporary problem that will be solved by larger context windows. Breunig’s taxonomy makes clear that longer contexts create their own failure modes. A million tokens is not better than a hundred thousand if half of those tokens are contradictory, outdated, or irrelevant. The solution is not more context. The solution is better-governed context: curated, structured, and designed to give the agent exactly what it needs without the noise that degrades its performance.

The governance system I ended up with is simple. A CLAUDE.md file that describes the project architecture, the dimensional model, the naming conventions, and the key design decisions. An error log that records failure patterns with enough context to prevent recurrence. A checklist that the agent consults at decision points. And a discipline — mine, not the agent’s — of starting new conversations when the context gets too long, rather than trying to push through with a degraded window.

None of this is technically sophisticated. A CLAUDE.md file is just a markdown document. An error log is just a structured text file. A checklist is just a list. The sophistication is in recognizing that these simple artifacts solve the right problem. The agent does not need more intelligence. It needs more memory — persistent, structured, curated memory that survives the conversation boundary.

Eden Marco understood this. Drew Breunig understood this. Amy Edmondson and Atul Gawande understood it in entirely different domains. The through-line is the same: in complex systems, the constraint is not capability. It is coordination. And coordination requires governance — rules, structures, and checklists that ensure the capable parts of the system work together rather than at cross-purposes.

In my earlier post, I argued that the most important skill in the age of AI is critical thinking — the ability to evaluate, question, and judge. I still believe that. But I would now add a companion skill: context engineering. Not prompt engineering — the craft of writing a single good instruction. Context engineering — the craft of designing the information environment in which an AI agent operates over time.

This is not a skill that most people are thinking about yet. The popular discourse is still about prompts. “How do I ask ChatGPT the right question?” That is the wrong question. The right question is: “How do I set up the environment so that any reasonable question produces a good answer?” The difference is the same as the difference between writing a good SQL query and designing a good database. One is a single act of craft. The other is architecture.

I suspect context engineering will become one of the defining skills of the next few years, not because it is hard — the individual pieces are simple — but because it requires a way of thinking that most people have not yet developed. You have to think about what the agent knows, what it has forgotten, what is contradicting what, and what is distracting it from the task at hand. You have to think, in other words, like a manager — not of people, but of information. And as the Breunig taxonomy shows, the failure modes of badly managed information are just as real and just as costly as the failure modes of badly managed teams.

• Drew Breunig, “How Long Contexts Fail” (dbreunig.com, June 2025) — the taxonomy of context failure modes
• Eden Marco, Agentic Coding with Claude Code — the book that introduced me to context engineering
• Atul Gawande, The Checklist Manifesto (2009) — on governance through simple checklists in complex systems
• Amy Edmondson, The Right Kind of Wrong (2023) — on intelligent failure and organizational learning
• Gemini 2.5 Technical Report, DeepMind — on context poisoning in agentic systems
• Berkeley Function-Calling Leaderboard — on tool-calling performance degradation with context size