Scaling Human Judgment with LLM Planning

We built a hybrid employee-agent system for modifying customer data that marries the scale and speed of LLMs with the quality of human judgment. When someone on the Ambrook team needs to do bulk changes inside a customer’s books they use this system:

1. First, they instruct a specialized AI agent that has broad access to read-only tools, and have it draft a plan for how to change the customer’s books;
2. Then, they review the plan in a comprehensible UI that makes the machine-written plan human-legible;
3. Then, they press a button and execute the plan deterministically.

It works well. Through this system, our team has quickly performed tens of thousands of mutations on production data while keeping the decision making squarely in their capable hands. This approach scales human decision making rather than replacing it, and we think the pattern is generalizable well beyond accounting.

The problem

When shadowing my coworker Haley to better understand her work, I was surprised by how much of it comprised repeated patterns of clicking around Ambrook. Haley is one of the (fabulous) Account Managers on our services team, and she was helping our customer Josh, owner of American Disposal, a Texas-based septic installation business, with yet another (not so-fabulous) loan statement. She would create an account to track the loan, create a transaction for each payment on the loan statement for the same amount, and finally itemize each transaction using the loan statement PDF to figure out the percentage of each payment on the loan that should be tagged as principal or interest.

Each step was a straightforward set of clicks in the app. But a single loan can mean dozens of payments, a customer might have many loans, and even a few seconds per click can multiply out into minutes or hours. Plus, Josh wasn’t the only customer Haley had to help out on that day.

In the weeks before, Haley and I had spent lots of time together as I tried to understand and accelerate her work as an Account Manager (AM) for Ambrook’s customers. Much of the day-to-day work of an AM consists of helping customers by clicking around the product: setting up their account to reflect their operation, and helping them with some basic bookkeeping. I would see a repeated pattern of clicks and build automations across the pieces of our system (tagging, reconciliation, invoices), trying to find a simplified way of achieving the task in bulk. Working hand-in-hand, the AMs and I had reworked Ambrook’s authorization model to prevent switching tabs and windows, built many small product shortcuts, and automated common sequences. Each change helped, and together they made AMs meaningfully faster.

But even though each optimization seemed like progress, it increasingly felt like I was missing the forest for the trees. The problem with task-oriented solutions was how heterogeneous an Account Manager’s job can be:

• One customer might need help itemizing loan statements by interest and principal amounts.
• Another might need help selectively changing the categorization of a set of transactions.
• A third might need check memos transcribed from a PDF of a bank statement, and annotated on the relevant check.

In each instance, the actual task the AM performs for the customer is fairly mechanical, something that could be automated, but the set of tasks is so diverse that making any individual one much faster was a drop in the bucket. We could build a bulk tool for loan itemization, then another for check memos, then another for recategorization. Each would be a 10x improvement for that task, but wouldn’t move the needle on the topline goal: freeing up AM time. What we needed was a single tool that could be flexible enough to 10x a wide range of tasks.

We had toyed around with using LLMs for this, connecting them to demo MCP servers and other tools, and found that the AI was incorrect too frequently to be allowed to change data directly in accounting workflows, where correctness is paramount.

After lots of observation of, and collaboration with the team, I prototyped a generic solution that has since graduated into a system we now call PlanBrook and AmDo. Over the last few months it has changed how AMs work and what we can take on for customers. It also points toward a way of using AI that scales human judgment rather than trying to replace it.

The solution

The core insight came from thinking about how software engineers use coding agents. Most of us use them liberally to debug, summarize, propose changes, and even to do a first pass of code review, but we still recognize that AI is too often wrong to ship code without any human review. Code review before submission is the fence that separates a low consequence playground (where we’re comfortable with AI’s unpredictability) and the high consequence factory (where we aren’t). Account Managers needed a version of “code review” where they could see what was being proposed by the LLM, and carefully consider whether it was the right thing to execute.

Thus, we built this system with two halves, “PlanBrook” and “AmDo”, separated by a human in the loop.

PlanBrook is a planning agent based on Claude Code, one that has broad access to a single customer’s data in a read-only mode. This means it can search the codebase, Linear, other Ambrook systems, and when granted permission can parse documents that our customers submit to us - loan statements, photographed invoices, third party system exports, etc. Critically, PlanBrook does not have any access to modify any customer data. Our AMs have free flowing conversations with PlanBrook where they can pin down and discuss the goals with the Agent. They provide it with instructions, meeting recordings, files, and context about the customer’s data setup. It then uses its tools (including OCR, python scripts, MCP calls, SQL queries, and bash) to construct a “plan document” (a giant chunk of JSON that describes changes to be made on Ambrook). The AM and agent can go back and forth, repeatedly refining the plan.

AmDo is PlanBrook’s trustworthy sidekick. It is a deterministic piece of code that lives as an employee-only page in our app. It takes in PlanBrook’s plan file, and gives the human agent a comprehensive review interface that lets them quickly understand the plan and examine its details. Though the plan file is itself machine readable and hard for a human to parse, the UI makes a complex plan easy to understand, even if it is large, or includes many nested or looped steps (more on that later). Beyond the review interface, AmDo’s other job is executing the plan that the user selects or approves, using the user’s credentials.

This technical separation comes with a clear division of responsibility - PlanBrook can suggest how to write to Ambrook, but the AM is the actor approving and executing the change. This makes the implementation easy: AmDo executes in the browser, on the reviewer’s computer, as straightforward API calls to our server using existing endpoints, with standard authorization (and thus, access controls). There is no new privileged surface area to secure. A plan can only do what the AM reviewing it could have done by hand, and nothing runs until they approve it, no matter if the planning agent goes astray.

I like to think of these two systems as Don Quixote (PlanBrook) and Sancho Panza (AmDo). PlanBrook is an inventive, flexible, and courageous explorer, unconstrained by consequence. AmDo is a trustworthy rate-limiter: informed, careful, and prudent.

A data model for anything

The core challenge of this project was building the data model (and corresponding review interface) to represent the variety of plans we wanted to support, and the user interface to allow an Account Manager to understand what would happen at each step.

I. Intentional omissions

Some of the most consequential choices in technical designs are what you decide you should not build. We decided that in order for plans to be fully comprehensible they needed to be repeatable and deterministic, and so we skipped conditionality: the ability for a plan to specify a condition and two branches to take based on the evaluation of that condition.

We first thought we would need conditionality to support the common “upsert” scenario, where the AM asks PlanBrook “to create or update the transaction”. We ended up finding we could work around this need by ensuring that PlanBrook has access to an internal Ambrook MCP that gives live read-access to the customer’s data. This allowed the planning agent to act conditionally during planning to create plans that are based on the state of the data, rather than encoding that uncertainty into the plan. In other words, the branching still happens, but at planning time, where it’s cheap to be wrong and easy to check. This has the nice side effect that if the data has changed in between when the plan was constructed and when the plan runs, it alerts the AM to the change, rather than silently ignoring it.

This keeps our plans fully deterministic, and highly comprehensible to the AM reviewing them. For the same reason, we opted not to build in lookups or error recovery - nondeterminism by other names. Making sure the human reviewer can have confidence in what will happen when they press “Run Plan” is invaluable.

Within that constraint, we still had to solve a variety of smaller problems.

II. Operation set

The plan schema needed a registry of operations that could constitute individual steps. To achieve this we turned to our GraphQL API, adding some lightweight scaffolding to expose most of the mutations to a schema in PlanBrook. This scaffolding handles things like:

• Hiding methods that should never be used;
• Renaming methods to use more commonplace terms;
• Adding hints about how each API should be called in brief documentation; and
• Winnowing the input arguments to the set that are most commonly needed.

Each of these optimizations was found by looking at the stumbling blocks models ran into when interacting with our GraphQL schema, which is enormous because of the breadth of our product. The end result is that adding a new GQL mutation to this set just takes a line of configuration, and can be tweaked when agents get confused.

III. Loops and data

PlanBrook and AmDo are only needed when AMs need to do large volumes of operations. If a change just requires a few clicks, it’s still faster for them to do it in our app. To express repetition, we turned to loops and data sets called tables: plans can include named tables that have columns and rows, and then have blocks in the plan that say “for each row in this table”. As an example, PlanBrook might use OCR to extract out the check numbers and memo lines from a PDF of a checking account statement, and then include in the plan both the data table that was extracted (column of Memo and Check Number), and then run an operation in a loop to set the memo for each check.

IV. Dependency and reference

If an AM specifies that first an invoice should be created, then it should be itemized, PlanBrook needs to describe (in a single plan document) how that second step relies on knowing the existence of the previous step (in order to know the ID that should be sent in the itemization request).

When this is only two steps it’s complicated enough, but we also have concrete use cases where dependencies are 1:N (i.e. create a single invoice and match it to multiple payments) and N:1 (i.e. create multiple transactions, then use a single batch tagging operation to update them together).

Building this graph structure required careful tinkering with not only what was theoretically supportable, but what kinds of JSON agents could reliably produce. We found that data models that were too abstract took many more tokens for the agent to construct, and caused PlanBrook to hallucinate more frequently.

What a plan looks like

Here’s a trimmed plan that pulls these pieces together: a data table, a loop over it, and a step that references the output of an earlier one.

{
 "version": "1",
 "displayMetadata": {
   "name": "Tag feed purchases",
   "description": "Apply Feed category and Cow-Calf enterprise to 45 transactions"
 },
 "rowSchema": {
   "key": "transaction",
   "label": "Transactions",
   "columns": [
     { "key": "transactionId", "label": "Transaction", "type": "scalar", "value": "ID" },
     { "key": "categoryTags", "label": "Category", "type": "tag", "value": "category" },
     // ...
   ],
   "actions": [
     {
       "key": "tagTxn",
       "label": "Tag transaction",
       "graphqlOperationName": "ExampleBatchTag",
       "paramBindings": {
         "input.orgId": { "source": "literal", "value": "org_T5JnOjEY6WJjMQIz" },
         "input.byIds.ids[0].transactionId": { "source": "column", "columnKey": "transactionId" },
         // ...
       },
       "resultBindings": []
     }
   ]
 },
 "rows": [
   {
     "id": "row_1",
     "cells": {
       "transactionId": "ct_MTp0eG5fTWdNS1dIUU4y",
       "categoryTags": [
         { "tagId": "bk_cat_feed_supplies", "display": "Feed", "perMilleAllocation": 1000 }
       ],
       // ...
     }
   },
   // ...
 ]
}

The impact

AmDo has been launched for a few months:

• Each week it performs 4k+ operations (each operation would have required someone to navigate to a page and click button(s) in the app).
• It has 100% weekly active adoption within our AM team and has also been adopted by our sales team for creating demo accounts and by engineers building sample data sets, among other uses.
• PlanBrook produces valid plans 98% of the time, as a result of agent-friendly tooling that helps PlanBrook vet and assemble plans.

The numbers don’t capture the quality of life improvement, and our hard-working AMs‘ feedback and enthusiasm for the tool has made me very happy.. My favorite Slack message: “Consider it AmDONE!”

The thing our AMs love about PlanBrook isn’t just the automation, it’s the way it allows them to “spend less time squinting” at their screen. With agents, AMs can trust that Claude will figure out the rote pieces of updating customer books. They can spend more time thinking about the customer’s operation, and less time just trying to double check that the buttons they pressed were the right ones.

AmDo has also become a primitive that we compose into other workflows - human and agentic. The most-repeated tasks our AM team spent time on (including the original example, of itemizing loan statements by principal and interest) have now been turned into standalone Claude Skills which make it even easier for PlanBrook to build plans through common instructions and tooling.

This process hasn’t been all roses. AMs still sometimes go back and forth with PlanBrook 2+ times in order to get to the plan that they want. Sometimes that’s due to a lack of specificity in the instructions, but more often than not, AMs have to correct misunderstandings the agent makes about the scope of what it was tasked with, or even basic accounting concepts that the AM is fluent in but the agent isn’t yet aware of.

We’re using context engineering to solve these misunderstandings, but this problem isn’t one we expect to be able to solve in 100% of cases. Even though our AMs are now fluent in this tool, LLMs can still misunderstand their directions, or lack an understanding of the specific customer’s setup and goals. There’s a lesson in that.

Generalizable lessons

I. Design scaffolds weaknesses

One lesson from this work is that human-in-the-loop systems should design each phase to amplify the strengths of the actor, and scaffold/minimize their weaknesses.

In the planning phase, the actor is an LLM, and they can be tireless, diligent, and have almost any data thrown at them and make sense of it. However, they’re prone to hallucination, bad at math, susceptible to prompt injection, and missing or limited context routinely holds them back. The environment we built for this agent thus has access to tools that help it learn, get detailed context, do math, and have firewalls to ensure it can’t access the outside internet.

In the review phase, the actor is one of our AMs, and they are brilliant experts in both their customer’s operation, books, and Ambrook. However, they have limited time and attentional capacity to review plans, so we built the review interface so that it can help them quickly sample the data, validate they’re comfortable with the changes, and build confidence in the results when done. .

In the execution phase, the actor is plain deterministic code, so its weakness is rigidity - we build scaffolding to support that through pre-flight checks that PlanBrook can run so that by the time a plan has been authored by PlanBrook and reviewed by an AM, we’re confident the code can handle it.

By building software that helps us scale the strengths (and accommodate the weaknesses) of each part of the system, we’re getting more out of AI and helping our team do more.

II. Augment, don’t replace

My experience with AI (including PlanBrook and AmDo) has made me bullish on strategies for AI that aim for “scaled judgment” rather than “full automation” in complex knowledge work. This is a humanizing way to think about AI at work, because it appropriately recognizes the components of intellectual labor that aren’t captured by keystrokes and clicks.

The AMs that I’ve gotten to work with on our team (shoutout Aryn, Haley, Shannon, Taylor, and many others) are brilliant and kind people with deep experience. Many have worked in bookkeeping, many have run their own farm or ranch or small business. The reason our customers like working with them isn’t because they can click around a website for them; it’s because they bring understanding and empathy with them when they jump in to help. Working with the Ambrook services team provides some customers with accountability, others with encouragement to try something new, and others with partnership in thinking about their business. The value is more than the tasks to be done.

I’m proud that PlanBrook and AmDo allow Ambrook’s experts to spend more time with their customers. This experience has convinced me that some of the best use cases for AI will be the ones that help our team get more done, at a higher quality, rather than ones that try to distill their expertise into a button or a text box. Full automation is only possible when your objectives are unitary and easily described. Few problems in our complex world have that shape.

What’s next

AmDo and PlanBrook represent one promising avenue among many for using AI at Ambrook. We’re, just at the beginning of exploring how it can expand and be made maximally useful, and most of the interesting problems are still open:

AmDo and PlanBrook are just one promising avenue of AI use that Ambrook is constantly exploring and deepening. As far as we’ve come, there is so much to explore, optimize, and refine:

• Getting the plan right the first time. AMs still go back and forth 2+ times in some cases. How much of the accounting knowledge about the customer’s configuration can we encode as context? What tools could automatically encode learnings from revisions?
• Repeatable plans. Skills let us turn a plan that worked once into one that works every time for the same kind of task. Which tasks deserve that, and how do we keep those skills correct as the product changes?
• Realtime Responses. One of the few complaints we have from AMs is that PlanBrook is slow - they often have to kick off planning sessions and then pivot to other work. How can we make PlanBrook 10x faster so it can not interrupt the AM flow? Open harnesses? Context Engineering? Agent swarms? Jev for validation?
• Review at scale. A plan with 2,000 rows can’t reliably be read line by line. What could we do to create a higher-signal interface for reviewers?
• General availability. Today only our team uses these tools. What would it take for a customer to safely review and run a plan on their own books?

If these sound like your kind of problems, Ambrook is hiring, and is (IMO) a phenomenal place to build responsibly with the cutting edge of AI: ambrook.com/careersambrook.com/careers.

***

This article was also published on grady.devgrady.dev.