From Inbox to Pull Request

A year ago, AI in customer support felt all about efficiency. But our whole thing at Ambrook is that we’re kind of obsessed with our customers. We don’t want to be more “efficient” with them. We want to make our product better for them, build with them, and have every part of their experience (including support) be delightful.

But that doesn’t mean we’ve shied away from AI. On the contrary, it’s enabled us to scale our four-person customer success team to 10x our customer base while shipping product changes and bug fixes directly to address customer needs.

Last November, we turned on an AI support agent. Well beyond providing instant answers to standard questions, we can now debug issues, build dashboards, and change the product itself at a pace and depth we’d never been able to before.

Setting up our team with the literal tools (Claude Cowork, coding agents, MCP server connections to all our other software) was just the first step. The real process has been rethinking the job of a customer success team in this new era, and balancing the risk of slop: When AI enables you to do a million things, it doesn’t actually mean you should do all of them, or that it’s okay to drop the bar for quality.

Defining the right problems to work on, and choosing where AI actually adds leverage, turned out to be the most important step. Here’s what we’ve learned so far. We’d love to compare notes!

So you want a better support agent?

At the beginning of Momentum Month—a company-wide experiment to rapidly upskill our team on AI—our team went through a stack-rank of every activity that takes more than five minutes per week. Answering questions in the support inbox was the biggest single use of our team’s time by far—an obvious top of the list. We had done some prior automation work by turning on Fin, Intercom’s customer service agent trained primarily on the contents of our Help Center but it plateaued at resolving around 30% of questions with the basic configuration.

The quality of human support is one of the things our customers cite most often in reviews, and we didn’t want to put the trust and connection we’ve built with them at risk. But our small, U.S.-based support team was facing an exponentially growing customer base, and at the height of our busy season, we knew we couldn’t keep up with the volume on our own. So we thought carefully about where Fin could be trusted, where a human had to stay in the loop, and how escalation should work. We came to a few guiding principles.

The value of AI is speed and conversations any time, day or night.

Our users might be in the field or on a job site when they reach out, or grabbing the one free night they have each week to work on their books. They need answers fast—even at 1am. What we didn’t expect was customers embracing Fin to fill another role we’ve long held as a support team—helping customers learn bookkeeping. Fin was teaching our customers in real time how to understand accounting and feel more confident in their numbers. One user put it well:

“Fin is helping me to clear up messy transactions and transfers! I appreciate that AI is available when I have time to work on this well into the night!! I always feel bad to bother the team but Fin is available 24/7!”

The value of a human is presence in moments of uncertainty, panic, or reassurance.

When a customer hits a bug or something sensitive around money movement, it matters enormously to have a person say, “I’m here, and I’m going to work with you to make this right.” That’s not a job for a bot, no matter how fast or accurate the response might be. This has informed the escalation rules we’ve built and are constantly evolving. We autoclassify the sentiment and content of a conversation from a user’s first reply. If it’s negative, a potential bug, or a handful of other sensitive topics we’ve identified, it goes straight to a person.

Confusion is killer, and you don’t have time to write all those support articles.

As we worked toward our first milestone of 50% AI-resolution, we realized there were two clear issues with the content our agent was trained on, capping our resolution at 30%: (1) contradictions in our knowledge base and (2) keeping content updated as the product grew.

Leverage AI to continuously identify and remove contradictions in your knowledge base.

An AI support agent is only as good as the information it’s trained on. We have a pretty exhaustive library of hundreds of support articles with screenshots, videos, and webinars. But that was also part of the problem. Before AI, it was almost impossible to re-review every article when a product change was made or a button was renamed.

We realized pretty quickly that contradictory sources were actually one of the biggest reasons for incorrect agent responses—not a lack of content. AI enabled us to go back and review our knowledge base for contradictions as a whole, and also to build a Claude skill that checks all existing articles for any content that needs to be updated whenever there is a product change or launch, and automatically suggests updates for us to review and approve. These improvements alone improved our resolution rate to XX%.

Create a self-healing content update loop.

The next step was finding a solution for continuous content updates to address questions that don’t have a support article. We used Claude Cowork to build a content automation workflow made up of several Claude Skills and an orchestrator. Each week, we run it on the prior week’s conversations that Fin did not fully resolve. The workflow identifies whether the cause is a content gap; if it is, Claude proposes updates to existing articles or drafts new ones, and stages them for review. The hardest part has been making the suggestions good enough that reviewing them is quick, so the editing doesn’t become its own time sink and the whole workflow gets dropped when things get busy.

The SQL queries that power that dashboard were also written with Claude—some of our team hadn’t worked with SQL much but quickly learned that was not a barrier.

Ship it with Claude Code.

What to do when the source of support tickets is not a lack of product knowledge, but a functional gap in the product? Feature requests and bugs can’t be solved with even the best Help Center.

Closing the loop in real time is a morale multiplier—for customers, support, and engineers.

It is hard to overstate what it does to a team to feel friction in a conversation and remove it the same week. Customers experience something close to magic. Support people get to use their product knowledge to change the product, not just describe it. AI has collapsed the cost of that tier of work. The tickets that used to languish indefinitely in the backlog can now be revisited and shipped in hours to days with a good spec and a coding agent.

Build the infrastructure to push code authorship to the people closest to the customer.

For smaller bugs and features, our support team can now scope the ticket, write the spec, put up the PR, and test the change themselves. Engineering only steps in for a final code review. A few weeks ago, Katie was on a call with a customer who couldn’t export her items list to Excel. The ticket had been sitting in Linear for 11 months because there was a workaround and not enough customers asking. Katie ran our Claude triage skills over the ticket, layered in the missing context, handed it to a coding agent, got a quick eng review, and shipped it in two days.

This didn’t just happen overnight. The accelerator has been twofold and very deliberate:

• Teaching our customer-facing team how to use coding tools: During Momentum Month, we broke down the basics of Github, code reviews, and deploy cycles for operators. Basically, everything except writing the code itself became fair game for upskilling our entire team.

• The engineering team building the infrastructure for us to be successful (and not break too many things): The eng team made a conscious effort to not just make the changes for us. Instead, they invested in standing up the infrastructure—including piloting a coding agent, currently Niteshift, that wraps Claude Code to write code in a sandbox environment, completely isolated from any customer data so there’s no risk of breaking something.

Spend 90% of your time making tickets a coding agent can finish, and 10% finishing them yourself.

Claude Code turned us into junior engineers with a disproportionate amount of product and customer knowledge. The new skill is knowing where our judgment is the unlock—and where we should instead invest in better specs, sharper triage, and tighter handoffs so the agent can do the work without us. The 10% we build ourselves isn’t the point; it’s what keeps us fluent enough to write the other 90% well.

Beware of drowning your engineering team in slop.

One of the hardest parts of starting to ship code as a non-engineer, for us and our engineering team, is that we can’t really review our own work. Here’s what has helped:

• Testing in other ways before asking for a review: Even though we can’t read our code, we can test whether it works in a staging environment before asking engineering to spend time reviewing. We also have a code review agent that gives a first line of feedback before we ask for a human review.

• Have a patient eng team that has bought into the vision and is okay with iterating on the process: It’s inevitable that there will be mistakes along the way. It’s been absolutely critical that engineering sees the vision for how empowering and teaching our CX team will ultimately help build a better product across the board.

• Being okay spending time on things that will never get shipped: This is probably the biggest one, especially early on. When we started, Katie had five open coding sessions for every one that merged (affectionately, her “Niteshift zombies”). You need a good two-way feedback loop here: the eng team giving constructive but honest feedback when something is off-base, and your team accepting that, especially while you’re building the processes and your intuition, you’ll spend a lot of time on things that never make it in.

There’s a lot more ahead—automated bug fixes, feature drafts straight from customer feedback, hundreds of small changes a week. But the shift is already here: The people closest to the customer are building the product alongside them.

We’re all agent managers now.

The value of a human is in turning support into product improvement.

The inbox has always been our richest source of insight—the most direct, consistent signal we get from customers. But when we’re heads-down answering questions, it’s really hard to also make the time to pull out the meta observations and package them into something the team can act on. That’s the real work. An AI agent says, “Here’s the answer.” Our team should be thinking, “The product confused you, and that’s why you had to ask. Let me go fix it so no one else has to.”

We now frame every teammate as an AI agent builder and/or an agent manager.

• Agent builder: You construct the agent. Write the skill, wire the tools, encode the judgment.

• Agent manager: You direct the agent. Hand it work, set the brief, check the output.

In some ways, this means customer success is collapsing into product work. That is part of why our team’s titles can move fluidly between “customer success” and “product operations.” What has changed is that customer success can ship the smaller, customer-facing fixes ourselves, freeing engineering to focus on the deeper work that scaling actually demands.

A year in, the shape of customer success at Ambrook looks quite different. We answer fewer of the same questions twice, close content gaps the same week they surface, and ship the small, customer-facing fixes ourselves instead of waiting on the engineering backlog. The time we get back goes into deciding which problems are actually worth solving, which is the part of the job AI cannot do for us. Being pro-AI has not meant being anti-human. It has meant pushing human judgment to where it matters most, and automating what we aren’t learning from anymore.

If you are a customer success leader thinking about agentic AI in your own context, especially in a high-trust, relationship-driven industry, we would love to compare notes. What are you automating? What are you deliberately leaving for humans?

Drop us a line. We’d love to hear from you.