What Building With AI Really Costs: The Whole Bill

Contents10 sections

In short:

• In about five months, one developer and a team of AI agents merged 2,488 pull requests across nine repos, about 500 a month, for about $303 a month in tools: roughly 60 cents each.
• Every subscription buys an allowance, and what it delivers inside that allowance is the real price. No code review service is yet priced for AI-speed work.
• Moving CI onto a server we already owned cut the monthly GitHub bill from about $108 to $24.74, in exchange for slower jobs.

Using AI to build software takes more than a license to a big LLM service (a large language model such as Claude, GPT or Grok). Production-grade software still needs development, testing and deployment, and everything that comes with them: pipelines, runners, hosting and monitoring.

Quality also means checking the work with more than the tool that built it. Code written by one AI is reviewed by others, and tested for security, for mutations (small planted bugs the tests have to catch), and at the functional, unit, integration and system levels. So you end up with a whole set of tools, each with its own license, allowance and bill.

Our team of AIs works across nine GitHub repositories (repos), from an enterprise SaaS platform, software sold as a hosted service, to SaaS tooling and applications. It has run more than 31,000 CI runs to check its work. CI, continuous integration, is the set of automatic build and test checks that runs on every change before it merges. If you're adopting AI to build real software, this is the bill you'll actually get. It covers the whole toolchain, and the model is one line of it.

Eight meters, one ledger

Our toolchain runs on eight tools, seven paid and one free. Each has its own price, its own allowance, and its own rule for what happens when the allowance runs out:

Table 1. The eight tools on the ledger, with a month's spend: $302.74 against $282 in list prices, and the whole gap is GitHub's.

The only extra spend was GitHub's — $20.74 in hosted minutes on top of its $4 plan. The $20 xAI credit was bought once, sits outside the monthly total, and was never drawn on.

Most allowances belong to the account rather than to any one repository. Nine repos draw from the same 3,000 minutes and the same 1,500 Copilot credits. None of them can see what the others have used. No vendor shows you the other vendors either.

So our usage and cost advisor, MU/TH/UR (pronounced Mother, after the ship's computer in the Alien films), reads every meter for every repo and keeps one ledger. For each tool she records the license cost, usage against the allowance, extra usage and its budget, a projection to month end, and which repo spent it. If she can't read something, it shows as NOT READ, never as $0. MU/TH/UR is our own internal tool, and the method matters more than the tool: the same ledger can start as a spreadsheet.

Spend you can only see one bill at a time is spend nobody manages. One ledger across every tool and every repo is the first step.

What the reviewers delivered

The code reviewers are where price and value came apart most clearly. A code review service is an AI tool that reads a pull request (a proposed change to the code) and comments on its problems. Here is what each one did when it was asked to review one:

Table 2. Review requests each reviewer answered, across all nine repos from its first use, and what each answered review cost.

Copilot, the cheapest paid reviewer, refused most of what it was asked. That was 762 of 822 requests. CodeRabbit answered a little over half. Gemini, on free access, answered every request — 874 inline comments and no refusals. CodeRabbit's allowance is 8 reviews an hour per developer. On our account, fair use cut it to 1 an hour once a developer passed 70 reviews in a week. One developer driving a team of AI agents passes that in a couple of days.

These weren't the only reviewers we tried. We also ran trials of Qodo, which reviewed 260 pull requests, and Greptile, which reviewed 10. The tools in the table are the ones we settled on, because they work for us.

The trials taught us something no vendor's pricing page will. None of the external reviewers is yet priced for the true velocity of AI-aided development, or for AI built into everyday development practice. A team of AI agents opens pull requests faster than any human team; every allowance, hourly limit and monthly quota still assumes a human pace.

Until the vendors catch up, you need an easy way to track real metrics, real data and real dollars. That means the usage costs as well as the subscription, because what a subscription includes is never enough at the speed AI works.

What a subscription really buys

The reviewer numbers raise a bigger question about every tool on the ledger: what does the subscription actually buy? CodeRabbit answers it most plainly, and the answer holds across all eight tools.

For CodeRabbit, the subscription buys the allowance and nothing more. Once it runs out, requests are refused until it refills, unless extra usage is turned on at 25 cents a reviewed file. Inside the allowance the value is real. It came to about 250 answered reviews and about 1,170 inline comments a month, roughly 4 cents a comment. At that price it's still worth it. The problem is that it runs out, and 43 percent of what we asked for was turned away.

The same pattern holds across all the tools:

• A subscription buys an allowance, and the outcome depends on what fits inside it. A tool's value is what it delivers inside that allowance, which is why cost per answered review says more than price per seat.
• Seat pricing counts people. CodeRabbit bills for each developer who opens pull requests (one paid user is one seat). Our agents push as one person, so a whole team of agents is one seat. AI work adds volume without adding seats, and the vendors take it back through hourly limits and fair use instead.
• The allowances that run out are the ones inside the pipeline. CI minutes, review requests and hourly review limits are sized for a human pace, and AI agents use them up. The 3,000 included minutes went in four days, Copilot's 1,500 credits ran out, and CodeRabbit's hourly allowance shrank under fair use.
• The allowances people use directly had room to spare. Claude's plan carried about 2 billion tokens of coding work a month (a token is the unit a model counts text in, roughly three quarters of a word), and SuperGrok used about a quarter of its weekly limit. Both stayed well inside their limits, while the xAI credit sat unused.
• Every allowance that runs out needs a plan for the overflow. Add hardware, send the overflow to a free or cheaper tool, ask less often, or pay for extra usage with a cap.

The pricing page tells you what you'll pay. Only the usage tells you what you get for it, and when you'll run out.

The biggest cost, and the server that answered it

Our biggest cost turned out to sit outside AI altogether. It was CI minutes. Those ran on GitHub's machines, the hosted runners GitHub bills by the minute once the included allowance is gone.

Before the change, our largest repo alone ran up about $83 of a $108 monthly GitHub bill. The account's 3,000 included minutes ran out in the first four days of a month. A few habits drove most of it:

• One background job, running every 20 minutes, would have used about 2,160 minutes a month on its own (72 runs a day, each billed as at least a minute).
• Scheduled sync jobs ran every six hours in three repos.
• Overlapping review lanes in two repos asked several AI reviewers about the same change.

Our server has 16 cores and 64 GB of RAM. Most of it sat idle. So it became the CI server for all nine repos, and MU/TH/UR became its controller:

• Same rules, every repo. Every repo carries a word-for-word copy of the same CI rules. Jobs run on the server by default, and when no runner is free a job waits for the server rather than quietly falling back to paid machines. Pull requests from forks (copies of a repo made by someone outside the team) still run on GitHub's machines, so a stranger's code never runs on the server.
• A fresh runner for every job. Each job gets a single-use runner, created for that job and deleted when it ends, so nothing carries over between jobs.
• One cap across all repos. Four places (slots where one job can run at a time) are shared by nine repos: two for normal CI, one kept for the largest repo, and one fast pass for documentation-only changes, which are about one merged pull request in eleven. Places go to the oldest waiting job; the loudest repo gets no priority.
• A flex lane when the queue backs up. A fifth place opens for any repo whose job has waited 20 minutes while every place is busy.
• A stop on resources. Before every start, MU/TH/UR checks free memory, cores and disk against set lines. Below any line, nothing new starts. She also ends stale sessions and hung test runs.

It has failure modes, and we planned for them. If the server is off or overloaded, jobs wait in GitHub's queue, for up to 24 hours, rather than falling back to paid machines. Jobs that need secrets (the passwords and keys a job uses) stay on GitHub's machines. And the controller itself has to be watched, which is why MU/TH/UR reports on her own health as well as everyone else's.

The hardware was already paid for — what it lacked was a controller, so that nine repos and a team of AI agents could share one machine without starving each other, or the people who use it.

What it saved, and what it cost in time

Saving money on CI had a price, and we paid it in time. Here are the CI jobs we timed on GitHub's machines and on ours:

Table 3. CI jobs timed on GitHub's hosted runners and on our own server; the local jobs billed nothing and ran about 2.7 times as long on average.

Run on GitHub, the 1,870 local jobs would have used more than the whole monthly allowance of 3,000 minutes by themselves. At the table's hosted rate of 2.3 billed minutes a job, they come to about 4,200. Locally they billed none, and a month's GitHub bill fell to $24.74 (the $4 plan and $20.74 of hosted minutes), against about $108 before. GitHub rounds every job up to a whole minute, which is why billed minutes run higher than minutes used.

Yes, it slowed us down, and the table shows where. A typical job waited a little over a minute for a runner instead of a few seconds, and ran for about 3 minutes instead of under half a minute. On average, a local job ran about 2.7 times as long: 4.3 minutes against 1.6. The 15-minute average wait comes from the busiest stretches, when every place was taken and jobs queued behind each other.

Could it scale? The numbers put most of the slowdown on capacity, and very little on the method. Most of the wait is jobs queuing for one of four places on one machine — exactly what another server would cut. A repo that rarely competed for a place ran its jobs in 3.2 minutes locally against 3.0 on GitHub. The repo with the longest test sweeps slowed the most. A second server, or more places on a bigger one, would shrink the queue and bring back the parallel runs a single machine has to take in turn.

Did quality slip to save money? It held: the same checks run locally as on GitHub, and they caught failures just the same. About 94 in 100 jobs pass on the first try, and fewer than 3 runs in 1,000 need a re-run.

The server has running costs of its own. A machine like ours running around the clock costs roughly $15 to $20 a month in electricity. That estimate assumes it averages about 150 watts, or about 110 kWh a month, at typical US rates. Someone also has to look after it; for us, MU/TH/UR does most of that looking after.

Local runners trade dollars for waiting. Once you know the exchange rate, you can decide which jobs earn GitHub's speed, which can wait their turn, and when another server pays for itself.

The smaller rules

Beyond the server, a handful of small rules keep spend in check across every repo. Each is a small idea on its own; together they close the leaks:

• One switch per budget, read by every repo. Copilot is asked for a review only while MU/TH/UR's switch for it is on, and every repo that uses Copilot checks that one switch. When the quota ran out, it went off everywhere at once.
• Nobody waits blind on a reviewer. Before any repo waits on a review, it asks MU/TH/UR, who answers from the reviewer's own stated time, its allowance or its usual reply time. Of 127 CodeRabbit requests made after its allowance ran out, 2 ran. Now nobody asks while the allowance is spent.
• Hosted runs are a monthly budget, never a one-off OK. Only MU/TH/UR or the owner may allow GitHub's machines, for any repo, and only while spend is within budget and on pace. She may catch up during the owner's quiet hours, but never over budget.
• Every scheduled job has to justify itself. Jobs that need secrets stay on GitHub's machines and run weekly unless their purpose needs more. The every-20-minutes job is gone.
• One retry, then fix it. A failed job is re-run once and then fixed; it's never pushed through with an empty commit or a close-and-reopen.
• The ledger suggests savings, and the owner decides. Its latest suggestions: move routine work to a cheaper Claude model (the ledger values that at about $109 a month at list API prices), start new sessions instead of carrying about 272,000 tokens of context, and have our local AI reviewer look once per pull request instead of once per push.

Each rule closes a small leak that an invoice would only show after the money was gone.

Where to start

You don't need MU/TH/UR to do this. You need a handful of numbers for every tool, kept in one place, and one rule for running CI on your own hardware. Here is the short list:

1. For every tool, record five numbers: the license price, the allowance, usage against that allowance, extra usage (whether it's on, its cap and what it has spent), and refusals.
2. Take them from the source rather than the invoice: GitHub's billing usage API for Actions and Copilot, each review service's usage dashboard, and each vendor's limits page for what the allowance really is.
3. Work out the cost of each useful result: per answered review, per CI job, per merged pull request.
4. For every allowance that runs out, decide what happens to the overflow before you hit it.
5. If you run CI on your own hardware, keep pull requests from forks and jobs that need secrets on GitHub's machines.

A spreadsheet updated once a week will show you most of what our ledger showed us, and it will show you before the invoice does.

The whole bill

Tracking every license and meter in one place did more than tell us what we spent. It showed which tools earned their price, where the bill really came from, and that a server we already owned could do the CI work for all nine repos.

If you're building production software with AI, the model license is only the first line of the bill. The lines under it are everything that checks the model's work, and they arrive one meter at a time.

Sources

The usage, review and CI figures come from our own ledger and from GitHub's records for the nine repositories, over the five months to mid-October 2026. Vendor prices and allowances were checked against the vendors' own pages on 16 October 2026:

• GitHub, Actions runner pricing, for per-minute rates and the rule that each job is rounded up to a whole minute.
  • docs.github.com · actions runner pricing
• GitHub, About billing for GitHub Actions, for the 3,000 minutes in GitHub Pro and for self-hosted runners being free.
  • docs.github.com · managing billing for github actions
• GitHub, Plans for GitHub Copilot, for the 1,500 monthly AI Credits in Copilot Pro.
  • docs.github.com · plans for github copilot
• GitHub Blog, GitHub Copilot is moving to usage-based billing, for the move from premium requests to AI Credits on 1 June 2026.
  • github.blog · github copilot is moving to usage based billing
• CodeRabbit, Plans, for the Team plan's price and its 8 reviews an hour per developer.
  • docs.coderabbit.ai · plans

Glossary