A pairing session

Last week I spent an hour pair programming with a director of engineering on his system. He was using Claude Code and we were in a Google Meet together, discussing the changes.

We were changing feature-flagging, a feature that allows his company to slowly roll out new features and changes, ensuring that things don’t break as they release software. During the session we discovered that a misconfigured flag would unintentionally be released to all users. This was a big bug that needed to be fixed; however, there was a bigger problem we needed to address - the tests for this feature would pass regardless of the bug.

How did this happen? The code wasn’t careless, but the tests weren’t validating the right things. To a naive observer or an agent, the presence of tests could have been enough, but the tests hadn’t captured an important decision: what should happen when a flag is misconfigured? Pairing gave us a chance to make that decision explicit and check that the tests enforced it.

Using agents

Coding agents will happily write tests. Over time the quality of test code generated by LLMs has improved, but a common mistake is creating structural tests - they assert the system is structured a certain way, not that it behaves a certain way. This can be helpful for sticking to system standards, spotting coding errors etc, but in this case it was the reason for a significant behaviour gap.

The key question to be asking in this scenario - is the right approach? - is a judgment call. Agentic coding systems aren’t necessarily going to answer that the same way as you. The correct judgement exists in your organization, and it needs to be shared.

Why pairing?

When an agent does the typing, most of what the human is doing is judgment. In the session I described, we spent a little time discussing what, and most of the time describing how. Here are some examples of our asks:

• Write tests before touching the code (clear acceptance criteria)

• Ensure the tests fail before implementing a fix (high quality tests)

• Use mutations to validate the quality of the tests (testing the right thing)

• Code review the change against our standards. (a second set of “eyes”)

• Update coding standards (ensure the next person doesn’t repeat mistakes)

None of that is new. What’s new is that an agent will follow best practices that you ask for, or write down. You need to be clear about what good looks like.

Human code reviews are critical but don’t serve the same function. A reviewer sees the finished change, not the questions that shaped it. Pairing puts those questions in the room while the work is happening, and influences both the outcome, and future sessions.

If you’re paying for AI tools

If you’re using AI tools in your organization, ensure “what good looks like” has been defined. A weekly pairing session with an experienced, senior member of your team helps to solve this: it’s one of the most effective ways I know to disseminate best practices within a team.

This approach has worked for me for years, and I now do this with my clients: 90 minutes pairing on their own work, and a short note afterwards summarizing key points. If you’d like to try one, hit me up at ken@talltree.io.

These are familiar culprits, now with bigger impact:

Code drifts from declared architecture — codebases have guidelines, rules, and principles that are described in documentation. These can become obsolete if they’re not kept up to date. With the increased velocity of AI, this happens much more quickly.

Context drifts from code — in order to guide agent behaviour, users define context files like CLAUDE.md, cursorrules, etc. Agents also store internal memory files with key information. Agents can and do read all of this, so if they’re not up to date, you get undesirable behaviour.

To slow the decay I built a couple of skills.

architecture-drift-review

I run this after major feature work, before a release, and as a weekly job. It audits the codebase against its described architecture and verifies claims against code.

Three rules make it work:

• Deterministic evidence first — “Seventeen modules open the database directly; the doc says one” is a finding. “Coupling seems high” is not.

• No finding without an exit — every finding leaves with a fix, a new automated guard, or an explicit dated acceptance. A finding that ends as “should be looked at” will regress — that’s a failure of the review.

• The doc is a byproduct — keeping ARCHITECTURE.md accurate happens in passing. If a review produces only an updated document, it described the system instead of making it more robust.

If the project has no architecture doc at all, the skill inverts: it scans the system and creates a baseline document, so the next run has something to audit.

agent-context-audit

I run this every time I’m getting unexpected behaviour from agents - the “haven’t we already discussed this?” moment. It’s tool-agnostic so Claude memories, CLAUDE.md, AGENTS.md, Cursor rules, Copilot instructions, decision-record statuses, skill files, permission and MCP configs are all checked.

If you run more than one tool, it also diffs their instruction files against each other, because CLAUDE.md and .cursorrules can drift too.

Two principles worth sharing:

The false-OK — when you verify a claim with a grep and get nothing back, that can be read as OK. But what if the file was moved? Empty is not necessarily OK, it can also mean unverified.

The graduation pass — a number of findings from my audits have graduated into deterministic checks that I run locally. By doing that, they evolve from “instructions the agent must remember” to “checks that can’t be ignored.” Bonus: they’re cheap, run without a network connection, and support both agents and human teams.

Give them a try

Both skills are MIT-licensed and easy to install:

npx skills add ken-talltree-io/agent-skills@architecture-drift-review
npx skills add ken-talltree-io/agent-skills@agent-context-audit

The repo is here: github.com/ken-talltree-io/agent-skills. The skills are extracted from real production use, so they’re likely biased to my work. Issues and PRs are welcome.

AI alone is not a capability upgrade

AI can significantly increase local throughput; however, if the system and organization can’t absorb that throughput, the result is pressure rather than progress. Review queues grow, decisions stall, and coordination costs rise.

This is where many “AI readiness” efforts quietly fail. The tools work as expected, but the organization lacks the structures needed to operate effectively once implementation is no longer the constraint.

When using AI code-generation, teams need to ask a more fundamental question: how do we address these new constraints?

Decision infrastructure matters

High-performing teams respond by making decisions explicit. Ownership is clear. Trade-offs are written down. Decisions are made earlier, when change is cheap, rather than revisited later under pressure.

What matters here isn’t how quickly decisions are made, but how often they need to be revisited. Teams that struggle under AI-assisted development are rarely blocked by lack of options, they’re blocked by ambiguity. The goal isn’t faster decisions, it’s fewer reversals.

Architectural simplicity is a competitive advantage

AI-generated code makes it easy to add complexity unintentionally and unnecessarily. Strong teams counter this by being conservative about architecture. They limit abstractions, strengthen boundaries, and protect core systems from frequent churn.

These actions aren’t about resisting change. They’re about maintaining systems that can be understood, reasoned about, and safely modified as throughput increases. When architecture supports comprehension, teams retain their ability to adapt, even as code volume grows.

Review and comprehension are the work

In AI-augmented environments, review is no longer a secondary activity, it is the work. As more code moves through the system, understanding intent becomes more valuable than generating output. Effective teams optimize for this by keeping changes small, communicating intent clearly, and relying on shared conventions to reduce cognitive load.

When review breaks down, AI amplifies the failure. Superficial approvals, eroding confidence, and fragile systems are not tooling problems, they’re signals that the organization can’t yet absorb the speed it has created.

Speed without comprehension leads to fragility. This is true at all levels of the organization.

Seeing the system

AI is often framed as a shortcut to better software delivery. In practice, it’s a stress test of the overall system.

It reveals whether teams can make high-quality decisions, understand changes as they move through the system, and coordinate effectively under increased throughput. When those capabilities are weak, AI doesn’t compensate, it exposes the limits.

The opportunity in the age of AI isn’t to generate more code, it’s to build organizations that think clearly when moving quickly.

From Implementation to Decision-Making

AI produces plausible prototypes quickly. Ideally, it produces several. This shifts the limiting factor from how quickly code can be written to how effectively decisions can be made. Teams must now choose between approaches, trade off short-term speed against long-term maintainability, and decide what risks they are willing to accept.

When ownership is unclear or decisions remain implicit, progress slows, not because engineers can’t act, but because alignment becomes harder to achieve. As throughput increases, the cost of weak or delayed decisions rises quickly, and teams feel the impact sooner.

From Writing Code to Understanding Code

As AI increases output, engineers spend more time reading than writing.

Generated code expands overall volume and surface area, and context becomes harder to maintain. Understanding why something exists, not just what it does, takes longer. This creates pressure in review - approvals can become shallow, while confidence in changes and tools can erode.

Over time, the cost of misunderstanding catches up with and exceeds the cost of implementation, making recovery from mistakes becomes more expensive.

From Execution to Coordination

Faster individuals do not produce faster teams. As local throughput increases, coordination costs begin to dominate delivery speed. Dependencies matter more, and interfaces are exercised more frequently. Organizational boundaries become visible in ways they weren’t before.

At scale, delivery is governed less by individual capability and more by how effectively work moves across those boundaries. With less slack in the system, misalignment propagates faster and coordination failures surface sooner.

What This Means for Leaders

When these become the dominant constraints, improving outcomes requires a shift in focus. The limiting factors are no longer tools or implementation capacity, but how decisions are made, how changes are understood as they move through the system, and how work is coordinated under increased throughput.

In my next post, I’ll look at how high-performing teams redesign themselves around this shift, and what changes once decision-making, comprehension, and coordination become the primary work.

What’s been less consistent is an increase in actual delivery speed.

The gap between local productivity and system-level outcomes is becoming a common source of confusion for engineering leaders. It shouldn’t be surprising. Most software organizations were never constrained by typing speed in the first place.

Speed Wasn’t the Bottleneck

Before AI entered the picture, constraints in software delivery were well documented:

• Ambiguous or shifting requirements

• Architectural coupling

• Review and coordination overhead

• Decision latency across teams

AI accelerates the act of producing code, but it doesn’t resolve any of these constraints. In practice, it brings them to the forefront.

Teams now push higher volumes of change through systems that were never designed to absorb that rate of modification. Review queues lengthen. Context switching increases. Architectural boundaries are stressed more frequently.

The result isn’t faster delivery.

It’s more pressure on already constrained systems.

Local Throughput vs. System Flow

Improving the speed of one component doesn’t improve the throughput of the whole system unless that component was the constraint.

AI dramatically improves local throughput at the individual developer level. Software delivery, however, is a coordinated activity. As individual speed increases, constraints shift elsewhere, most often to decision-making, review, and coordination.

From the inside, this can feel like “we’re moving faster but getting less done.”

From the outside, it looks like more activity without corresponding outcomes.

The Wrong Conclusion

Many teams interpret this mismatch as a tooling issue: better prompts, better models, or better integrations will solve the problem. Others assume it’s growing pains.

In reality, what AI is exposing is not a gap in capability but a structural constraint.

AI doesn’t remove bottlenecks.

It reveals them.

Understanding where those constraints move and how to design teams to operate effectively under them is now one of the central challenges of modern engineering leadership.

In the next post, I’ll look at where the bottleneck actually moves in AI-augmented teams and why decision-making, not implementation, becomes the limiting factor.