“I do think code review / validation is going
to start becoming the bottleneck of building products”
For the most part, the processes and systems that underpin the way we develop software have not changed for the last two decades. Amidst vast differences in the way we architect software (e.g. the move to the cloud) and the infrastructure we use to build software (e.g. React), the so-called “outer development loop” of software engineering still looks eerily similar to what it was when Github first emerged - e.g. git based version control, human-reviewed pull requests, unit & integration tests running on merge, staging environments & canary rollouts.
In some respect, this is to be expected - these systems and processes have more to do with the way humans collaborate to write software than they do with the nature of the software itself.
But, over the last few years, we have seen a new stakeholder emerge in the process of writing software - the AI coding agent. And so it is no surprise that many of these human-oriented processes are beginning to feel immense stress in the wake of a new type of AI “software engineer” that thinks, and acts, very differently from what we’re used to.
A good, and obvious example of this is validating code. AI coding models and agents make it very easy to produce a lot of code. But, does that code work? Many engineering teams I talk to nowadays not only feel that they now spend more time reviewing code than writing code, but also that the code review process has also become dramatically more complicated. Common problems include:
- How do you more rapidly triage PRs to understand what is worth paying attention to vs not?
- How do you know which PRs were particularly agent-driven, and might therefore require a closer look?
- How do you more rapidly analyze a PR? Is there a way to more easily understand the semantic changes in a PR & the lines of code that map to each change?
- How do you know which lines of code were written by an AI vs. not in a PR? Is there a way to see the reasoning or thinking that led to such a line, in that case?
- How do you accelerate the code review workflow so you’re not now spending half your day looking at PRs?
- Should you be using AI to review PRs? In what situations?
In a nutshell: we need to rethink the entire code review process in a world of AI coding agents. However, code review is just one piece of the way we develop software, and it increasingly feels like these sorts of fundamental questions can be asked about every step of the software development lifecycle.
We’ll need to be better at testing software. Previously “niche” testing techniques like load testing, fuzz testing, formal verification, and similar are likely going to become far more commonplace. And we’ll need to get smarter about where, and when, to run these tests.
We’ll need to be better at specification. Most tickets assigned in tools like Jira lack detail, and rely on the vast amounts of implicit context that the typical human software engineer on a team has. This won’t work for AI agents, and will prevent us from reliably evaluating agents relative to what they were supposed to do. Test-driven development will make a comeback, but in a different form and based more on extremely tightly specified requirements documents.
We’ll need to further explore the ways AI can be used to leverage humans in the software development lifecycle. AI code reviews are a good start, but we’ll also want AI systems to do versions of design reviews, product reviews, QA testing, and much more to lighten the load for human SWEs and PMs.
We’ll need to rethink the core infrastructure primitives the outer development loop is built upon. Git isn’t really designed to handle thousands of agents concurrently modifying the codebase, and improvements in areas like conflict resolution, concurrent edits, and treating each edit as a commit will be essential. We also need to more natively bake metadata about AI-changes vs. human changes into version control systems, such that we can treat AI-generated changes differently throughout the entire SDLC. For example, a company may want different policies for testing & deploying AI written PRs vs. human ones.
And this is just the beginning…
We’ve been interested in this theme of the downstream impact of agentic code development for the past few years, but had never found a team attempting to holistically rethink the outer development loop in a world of coding agents, as opposed to just building point solutions for software testing or verification.
That was, until we met Oliver and Ben, two founders who had such a profound vision on what the software development lifecycle of the future would look like. So today, we’re announcing our lead investment into Mesa’s seed round.
Mesa launches today with an AI code review platform that helps eliminate AI-driven tech debt before it happens. Oliver and Ben’s starting point is a set of agents that learn your codebase, business logic, and team preferences to deliver precise, contextual code review. In other words, Mesa is more like a staff engineer helping you catch anti-patterns and architectural drift, rather than the pedantic software intern telling you about a “syntax error on line 35”.
Despite being <6 months old, Mesa is already being used by a number of companies in lieu of more established AI code review tools. This is because the platform allows for more personalized, context-rich code review agents and more precise policy on when to run each review agent, ultimately leading to a superior signal-to-noise ratio.
This is just the start, and you can shortly expect Mesa to launch a number of other capabilities, including a more substantial rethinking of the UX for doing human code reviews, and a suite of capabilities for test-driven development for agents.
Beyond functionality, what you’ll immediately notice about Mesa is the beautiful branding and design, evocative of a future where software engineers help architect & guide fleets of agents to build the future. This design-first lens is essential to successfully re-inventing how humans and AI agents will collaborate to develop software in the future, and a key part of why we were excited to partner with Oliver & Ben.
Try Mesa today for free at https://mesa.dev/
