read

Estimated Reading Time: 9 minutes

TinyOrbit is the from-scratch agent harness at the heart of Chapter 6 of my book, Token by Token. This is the short story of what it is, how it works, and what happened when I pointed it at real-world software bugs.


The magic trick, explained

AI agents feel like magic. You type “fix this bug,” go make a hot chocolate, and come back to a patch. Surely there’s something enormous under the hood, a planning engine, a reasoning module, a small city of microservices?

There’s a loop.

That’s the secret. The model is the brain: it does the thinking. Everything around it is called a harness: the loop that keeps the conversation going, the tools the model is allowed to touch, and the safety rails that decide what it may touch. I wanted to prove how small that harness really is, to myself, and to the readers of my book, so I built one from scratch. It’s called TinyOrbit: a small pile of plain Python, one third-party dependency, no framework, small enough to read in an afternoon, which is more than I can say for my hot chocolate machine’s firmware.

TinyOrbit: the model is the brain, the harness is the hands and legs, and TinyOrbit orbits around it all

Look at the sketch. The head is the model, the brain. It reads the task, reasons about it, and decides what to do next. But a brain on its own can’t open a file or run a test. It can only think.

Everything below the neck is the harness, and the closest thing to it in us is the limbic system: the old, unglamorous wiring that sits between thought and action. It carries the brain’s intent out to the hands, feeds back what the senses find, remembers what happened a moment ago, and flinches before you touch the hot stove. It isn’t clever, but without it the cleverest brain just sits there.

Here’s how the two work together as an agent:

  1. Think. The model reads the conversation so far and picks a move: “read models.py,” or “run the tests.”
  2. Act. The harness carries that request to the hands, read, grep and glob in one, edit, write and bash in the other, and does exactly that, nothing more.
  3. Observe. Whatever comes back, file contents, test output, a stack trace, the harness hands straight back to the brain as new input.
  4. Repeat. Round and round, the orbit in TinyOrbit’s name, until the model answers without asking for a tool. That’s the loop saying “done.”

Brain plus limbic system is a person who gets things done. Model plus harness is an agent. And that hatched belt around the waist? That’s the flinch: the permission gate that stops and asks before anything risky.

Everything else is seatbelts.

The seatbelts are the interesting part

Here’s what building it taught me: the loop takes an hour. The trust takes the rest of your life.

TinyOrbit’s tools are deliberately boring, read, write, edit, bash, glob, grep. The interesting engineering, the part that makes it a harness rather than a demo, is in what wraps them:

  • Permissions, fail-closed. Reading files is free. Writing files or running shell commands? The agent stops and asks you first. An agent that can quietly run rm -rf is not an assistant, it’s a liability with good manners.
  • Staleness detection. If the agent read a file, and you edited it behind its back, it refuses to blindly overwrite your changes. (A courtesy some humans have yet to master.)
  • Interruption as a first-class citizen. Hit Ctrl+C mid-task and the agent doesn’t just die, it tidies up its half-finished tool calls so the conversation stays coherent. Every request the model makes gets an answer, even if that answer is “the human pulled the plug.”
  • Memory files. Drop a TINYORBIT.md in your project with your conventions, and the agent reads it on startup, the difference between a contractor who read the brief and one who’s about to repaint your load-bearing wall.

None of this is glamorous. All of it is the difference between a demo and a tool.

OK, but does it actually work?

Fair question. Vibes are not a benchmark, so I ran TinyOrbit (driving Claude Opus) against SWE-bench Verified, the industry-standard test where agents must fix real, historical bugs from real open-source projects, judged by the projects’ own test suites. No partial credit. The patch either passes or it doesn’t.

On a random sample of ten tasks, TinyOrbit went 10 for 10, for a grand total of $3.39 in API costs, less than the hot chocolate I drank while watching it work. On a deliberately cruel sample of ten hard tasks (the ones humans estimate at 1–4+ hours each), it went 7 for 10. For context, when I scored the strongest published harnesses on those same hard ten, the best resolved 8.

Cost per task
Estimated from the token ledger at Opus 5 list prices. An outlined bar is an unresolved task.
$0.0 $0.5 $1.0 $1.5 $2.0 django-10554 django__django-10554 (1-4 hours) cost $1.88 · 56 calls · 11.2 min cache read 1,760,529 · out 26,398 tokens unresolved $1.88 · 56 calls · unresolved sympy-13878 sympy__sympy-13878 (>4 hours) cost $1.77 · 60 calls · 11.2 min cache read 1,751,833 · out 23,199 tokens resolved $1.77 · 60 calls django-16560 django__django-16560 (1-4 hours) cost $1.27 · 50 calls · 3.7 min cache read 1,286,735 · out 13,988 tokens resolved $1.27 · 50 calls pylint-4551 pylint-dev__pylint-4551 (1-4 hours) cost $1.14 · 41 calls · 4.5 min cache read 834,912 · out 19,699 tokens resolved $1.14 · 41 calls astropy-13398 astropy__astropy-13398 (1-4 hours) cost $1.05 · 40 calls · 4.0 min cache read 892,836 · out 14,609 tokens unresolved $1.05 · 40 calls · unresolved sympy-17630 sympy__sympy-17630 (1-4 hours) cost $0.90 · 45 calls · 6.1 min cache read 827,553 · out 12,197 tokens resolved $0.90 · 45 calls sphinx-7590 sphinx-doc__sphinx-7590 (>4 hours) cost $0.72 · 38 calls · 3.0 min cache read 574,011 · out 11,544 tokens resolved $0.72 · 38 calls django-11885 django__django-11885 (1-4 hours) cost $0.70 · 30 calls · 5.8 min cache read 542,627 · out 10,438 tokens resolved $0.70 · 30 calls django-15916 django__django-15916 (15 min - 1 hour) cost $0.64 · 21 calls · 2.5 min cache read 424,028 · out 8,948 tokens resolved $0.64 · 21 calls xarray-6938 pydata__xarray-6938 (15 min - 1 hour) cost $0.53 · 31 calls · 13.7 min cache read 285,377 · out 6,644 tokens resolved $0.53 · 31 calls django-12193 django__django-12193 (<15 min fix) cost $0.48 · 24 calls · 1.3 min cache read 453,145 · out 3,971 tokens resolved $0.48 · 24 calls xarray-3993 pydata__xarray-3993 (1-4 hours) cost $0.46 · 30 calls · 2.0 min cache read 366,509 · out 6,753 tokens resolved $0.46 · 30 calls pytest-10356 pytest-dev__pytest-10356 (1-4 hours) cost $0.33 · 22 calls · 1.8 min cache read 212,124 · out 5,381 tokens unresolved $0.33 · 22 calls · unresolved sphinx-10449 sphinx-doc__sphinx-10449 (<15 min fix) cost $0.33 · 25 calls · 1.5 min cache read 242,369 · out 4,746 tokens resolved $0.33 · 25 calls django-14493 django__django-14493 (<15 min fix) cost $0.33 · 19 calls · 1.2 min cache read 214,692 · out 2,921 tokens resolved $0.33 · 19 calls django-11551 django__django-11551 (15 min - 1 hour) cost $0.28 · 21 calls · 1.1 min cache read 180,306 · out 4,592 tokens resolved $0.28 · 21 calls django-14672 django__django-14672 (15 min - 1 hour) cost $0.24 · 22 calls · 1.0 min cache read 174,542 · out 3,352 tokens resolved $0.24 · 22 calls pytest-5262 pytest-dev__pytest-5262 (<15 min fix) cost $0.21 · 20 calls · 1.0 min cache read 147,905 · out 3,042 tokens resolved $0.21 · 20 calls django-11299 django__django-11299 (<15 min fix) cost $0.19 · 16 calls · 0.7 min cache read 118,755 · out 2,704 tokens resolved $0.19 · 16 calls django-12143 django__django-12143 (15 min - 1 hour) cost $0.16 · 19 calls · 1.0 min cache read 103,056 · out 2,732 tokens resolved $0.16 · 19 calls
easy samplehard sample
Turns and wall-clock per task
Left: API calls, capped at 60. Right: agent minutes in the container, inflated by x86 emulation on repos with slow test suites.
0 20 40 60 API CALLS sympy-13878 sympy__sympy-13878 60 calls · 11.2 min · exit max_turns resolved cap django-10554 django__django-10554 56 calls · 11.2 min · exit completed unresolved django-16560 django__django-16560 50 calls · 3.7 min · exit completed resolved sympy-17630 sympy__sympy-17630 45 calls · 6.1 min · exit completed resolved pylint-4551 pylint-dev__pylint-4551 41 calls · 4.5 min · exit completed resolved astropy-13398 astropy__astropy-13398 40 calls · 4.0 min · exit completed unresolved sphinx-7590 sphinx-doc__sphinx-7590 38 calls · 3.0 min · exit completed resolved xarray-6938 pydata__xarray-6938 31 calls · 13.7 min · exit completed resolved django-11885 django__django-11885 30 calls · 5.8 min · exit completed resolved xarray-3993 pydata__xarray-3993 30 calls · 2.0 min · exit completed resolved sphinx-10449 sphinx-doc__sphinx-10449 25 calls · 1.5 min · exit completed resolved django-12193 django__django-12193 24 calls · 1.3 min · exit completed resolved django-14672 django__django-14672 22 calls · 1.0 min · exit completed resolved pytest-10356 pytest-dev__pytest-10356 22 calls · 1.8 min · exit completed unresolved django-11551 django__django-11551 21 calls · 1.1 min · exit completed resolved django-15916 django__django-15916 21 calls · 2.5 min · exit completed resolved pytest-5262 pytest-dev__pytest-5262 20 calls · 1.0 min · exit completed resolved django-14493 django__django-14493 19 calls · 1.2 min · exit completed resolved django-12143 django__django-12143 19 calls · 1.0 min · exit completed resolved django-11299 django__django-11299 16 calls · 0.7 min · exit completed resolved 0 5 10 15 AGENT MINUTES sympy__sympy-13878 60 calls · 11.2 min · exit max_turns resolved django__django-10554 56 calls · 11.2 min · exit completed unresolved django__django-16560 50 calls · 3.7 min · exit completed resolved sympy__sympy-17630 45 calls · 6.1 min · exit completed resolved pylint-dev__pylint-4551 41 calls · 4.5 min · exit completed resolved astropy__astropy-13398 40 calls · 4.0 min · exit completed unresolved sphinx-doc__sphinx-7590 38 calls · 3.0 min · exit completed resolved pydata__xarray-6938 31 calls · 13.7 min · exit completed resolved django__django-11885 30 calls · 5.8 min · exit completed resolved pydata__xarray-3993 30 calls · 2.0 min · exit completed resolved sphinx-doc__sphinx-10449 25 calls · 1.5 min · exit completed resolved django__django-12193 24 calls · 1.3 min · exit completed resolved django__django-14672 22 calls · 1.0 min · exit completed resolved pytest-dev__pytest-10356 22 calls · 1.8 min · exit completed unresolved django__django-11551 21 calls · 1.1 min · exit completed resolved django__django-15916 21 calls · 2.5 min · exit completed resolved pytest-dev__pytest-5262 20 calls · 1.0 min · exit completed resolved django__django-14493 19 calls · 1.2 min · exit completed resolved django__django-12143 19 calls · 1.0 min · exit completed resolved django__django-11299 16 calls · 0.7 min · exit completed resolved
easyhard
Tool calls per task, by tool
Counted from the full message history. Bash dominates; Glob was never used in either run.
0 20 40 60 sympy-13878 sympy__sympy-13878: Bash 36 of 70 tool calls Bash 36 sympy__sympy-13878: Edit 16 of 70 tool calls 16 sympy__sympy-13878: Read 18 of 70 tool calls 18 70 django-10554 django__django-10554: Bash 52 of 55 tool calls Bash 52 django__django-10554: Edit 2 of 55 tool calls django__django-10554: Read 1 of 55 tool calls 55 · unresolved django-16560 django__django-16560: Bash 19 of 49 tool calls Bash 19 django__django-16560: Edit 23 of 49 tool calls 23 django__django-16560: Read 7 of 49 tool calls 7 49 sympy-17630 sympy__sympy-17630: Bash 32 of 44 tool calls Bash 32 sympy__sympy-17630: Edit 4 of 44 tool calls sympy__sympy-17630: Read 8 of 44 tool calls 8 44 pylint-4551 pylint-dev__pylint-4551: Bash 21 of 41 tool calls Bash 21 pylint-dev__pylint-4551: Edit 13 of 41 tool calls 13 pylint-dev__pylint-4551: Read 7 of 41 tool calls 7 41 astropy-13398 astropy__astropy-13398: Bash 28 of 41 tool calls Bash 28 astropy__astropy-13398: Edit 6 of 41 tool calls 6 astropy__astropy-13398: Read 5 of 41 tool calls astropy__astropy-13398: Write 2 of 41 tool calls 41 · unresolved sphinx-7590 sphinx-doc__sphinx-7590: Bash 21 of 37 tool calls Bash 21 sphinx-doc__sphinx-7590: Edit 11 of 37 tool calls 11 sphinx-doc__sphinx-7590: Read 5 of 37 tool calls 37 xarray-6938 pydata__xarray-6938: Bash 20 of 30 tool calls Bash 20 pydata__xarray-6938: Edit 6 of 30 tool calls 6 pydata__xarray-6938: Read 3 of 30 tool calls pydata__xarray-6938: Grep 1 of 30 tool calls 30 django-11885 django__django-11885: Bash 18 of 29 tool calls Bash 18 django__django-11885: Edit 8 of 29 tool calls 8 django__django-11885: Read 3 of 29 tool calls 29 xarray-3993 pydata__xarray-3993: Bash 18 of 31 tool calls Bash 18 pydata__xarray-3993: Edit 8 of 31 tool calls 8 pydata__xarray-3993: Read 5 of 31 tool calls 31 sphinx-10449 sphinx-doc__sphinx-10449: Bash 16 of 24 tool calls Bash 16 sphinx-doc__sphinx-10449: Edit 6 of 24 tool calls 6 sphinx-doc__sphinx-10449: Read 2 of 24 tool calls 24 django-12193 django__django-12193: Bash 18 of 23 tool calls Bash 18 django__django-12193: Edit 2 of 23 tool calls django__django-12193: Read 3 of 23 tool calls 23 django-14672 django__django-14672: Bash 16 of 21 tool calls Bash 16 django__django-14672: Edit 3 of 21 tool calls django__django-14672: Read 2 of 21 tool calls 21 pytest-10356 pytest-dev__pytest-10356: Bash 13 of 21 tool calls Bash 13 pytest-dev__pytest-10356: Edit 5 of 21 tool calls pytest-dev__pytest-10356: Read 2 of 21 tool calls pytest-dev__pytest-10356: Write 1 of 21 tool calls 21 · unresolved django-11551 django__django-11551: Bash 13 of 20 tool calls Bash 13 django__django-11551: Edit 5 of 20 tool calls django__django-11551: Read 2 of 20 tool calls 20 django-15916 django__django-15916: Bash 10 of 23 tool calls Bash 10 django__django-15916: Edit 4 of 23 tool calls django__django-15916: Read 6 of 23 tool calls 6 django__django-15916: Grep 3 of 23 tool calls 23 pytest-5262 pytest-dev__pytest-5262: Bash 10 of 20 tool calls Bash 10 pytest-dev__pytest-5262: Edit 4 of 20 tool calls pytest-dev__pytest-5262: Read 2 of 20 tool calls pytest-dev__pytest-5262: Grep 3 of 20 tool calls pytest-dev__pytest-5262: Write 1 of 20 tool calls 20 django-14493 django__django-14493: Bash 8 of 18 tool calls Bash 8 django__django-14493: Edit 3 of 18 tool calls django__django-14493: Read 4 of 18 tool calls django__django-14493: Grep 3 of 18 tool calls 18 django-12143 django__django-12143: Bash 12 of 18 tool calls Bash 12 django__django-12143: Edit 2 of 18 tool calls django__django-12143: Read 2 of 18 tool calls django__django-12143: Grep 2 of 18 tool calls 18 django-11299 django__django-11299: Bash 8 of 15 tool calls Bash 8 django__django-11299: Edit 5 of 15 tool calls django__django-11299: Read 2 of 15 tool calls 15
BashEditReadGrepWrite

Before anyone prints a trophy: ten tasks is a small sample, the model deserves a large share of the credit, and the comparison isn’t apples-to-apples since published entries ran older models. The honest reading is model-plus-harness, not “my weekend project beats the industry.”

And the three misses? All three failed the same way, and it’s deliciously human: the agent wrote its own tests, passed its own tests, and confidently declared victory, without ever reproducing the actual bug first. It graded its own homework. Every engineer I’ve told this to has gone quiet for a second, because we’ve all met that developer. Some of us in the mirror.

Why this matters (and where the book comes in)

The lesson of TinyOrbit isn’t “look how clever.” It’s the opposite: this technology is understandable. The agent loop that powers the billion-dollar tools is something you can read, build, and own in an afternoon. Once you’ve seen it, AI agents stop being magic and start being engineering, and you can reason about when to trust them, where they’ll fail, and what the seatbelts should be.

That’s the whole philosophy of Token by Token: the best way to understand AI is to build it. Chapter 6 walks through the agent loop step by step, TinyOrbit for the full harness, plus two small agents that debate each other live, with hand-drawn diagrams and runnable code for every idea. Earlier chapters build everything underneath it: a neural network from a single neuron, an autograd engine from high-school calculus, GPT-2 from scratch, and a model that learns to reason.

The code: github.com/badlogicmanpreet/tinyorbit, including the full SWE-bench reports with every verdict and a wire-level trace of one task, stream event by stream event.

The book: Token by Token: A Journey Through AI, Mathematics, and Machine Intelligence, available on Amazon (UK · India), Kindle, and Notion Press.

Build one. It’s the fastest way to stop believing in magic, and start doing it.

Blog Logo

Manpreet & Renaira


Published

Image

The Tokens

Bonding through words, and the math behind them.
Manpreet & Renaira

Back to Overview