Harness Evolution, Trustworthiness Guards, and Why Mauricio and I are Co-Authoring This Post

“Autonomous coding agents are a lot like brilliant, over-caffeinated interns. Incredible speed, huge potential, but if you don’t build solid guardrails, they will happily blow through 0 of cloud credits in twenty minutes, accidentally reformat your Git commit history, and write an essay justifying why it was necessary.”

A couple of weeks ago, in work and side projects, I was playing around with autonomous coding agents on my local machine. And to be completely honest with you, I ran straight into a wall.

If you read social media or tech blogs these days, everyone is hyping up “AI agents” as if you can just plug an LLM into an open bash terminal, go make some tea, and come back to a fully tested, production-ready system.

Well, reality is a bit different.

First, if you hook up a multi-turn agent loop to commercial cloud APIs, you quickly realise how much money you’re throwing away. The agent doesn’t just send your prompt; it wraps it in 15 giant JSON tool schemas, dumps hundreds of lines of raw terminal output and compiler stack traces from every bash command, and sends that entire mountain of tokens back and forth on every single turn. Before you know it, you’ve burned through 5 cloud credits just to fix a broken assertion in a unit test.

So you think: “Fair enough, I’ll run a small local model on my GPU. Zero token cost, total privacy, runs fast.”

And for basic tasks, it does! Running a 3-billion-parameter model like Qwen 2.5 Coder 3B or Gemma 2 2B locally on a consumer card (even on a modest GeForce GTX 1650) is awesome for standard shell scripts, regex, or summarising git diffs. But the second you ask it to solve a tricky concurrency bug or an asynchronous race condition, it hits a cognitive wall. And instead of telling you “I’m stuck”, it starts hallucinating broken bash commands with 100% confidence.

I personally tend to prefer practical, working setups over hype.

So my local edge assistant Mauricio (a French-bulldog-inspired chatbot running completely offline on my GPU at marginal cost) and I set out to fix this properly. In this post, we’ll break down three practical things we built to make autonomous agents actually usable:

1. The Harness Problem: How we cut token waste by a good chunk by evolving a lean prompt harness (inspired by the SoL-Pi recursive auto-research loop [1]).
2. The 60% Confidence Guard: How we set up a real-time monitor, borrowing ideas from our stream learning research [2] on concept drift detection and confidence-based estimators [3], when a local small model is struggling and automatically escalates to frontier models like Gemini or DeepSeek, only for that specific turn.
3. Publishing via MCP: How we connected Mauricio to WordPress through using an MCP connector [4], allowing an edge agent to co-author and publish articles directly from the command line.

Here is what we tested, what broke, and the actual code that made it work.

1. The Harness Problem: Why Agents Waste So Many Tokens

When you tell an autonomous agent: “Fix the failing test in tests/test_api.py“, you think you’re transmitting about 15 words.

In reality, most agent frameworks wrap that simple request in an absurd amount of boilerplate:

• Tool Schemas: 15+ tool definitions serialized in verbose JSON Schema format (~4,500 tokens before you even start).
• Observation Noise: Full stdout and stderr dumps from compiler warnings and directory scans (~8,000 tokens of terminal escapes and repetitive logs).
• Stale Trajectory: The agent’s own internal monologue and intermediate bash calls from 10 steps ago (~5,000 tokens).
• Your actual prompt: 15 tokens.

       +-----------------------------------------------------------+
       |             The Classic Bloated Agent Turn                |
       +-----------------------------------------------------------+
       |  [ 4,500 tokens ] Verbose JSON Tool Schemas               |
       |  [ 8,000 tokens ] Raw stdout, ANSI codes & Stack Traces   |
       |  [ 5,000 tokens ] Stale Historical Trajectory Logs        |
       |  [    15 tokens ] "Fix the failing test in line 42"       |
       +-----------------------------------------------------------+
               ==> ~17,500 tokens PER TURN (Slow, Pricey, Fragile)

Sending 17,000 tokens on every step of a 15-turn debugging session means burning through 250,000+ tokens to tweak three lines of Python. That is real back pain.

How We Fixed It: Evolving the Harness 

Instead of treating the agent’s prompt harness as a static template, we adopted the recursive harness auto-research formulation from SoL-Pi (Liu et al., 2026 [1]) and treated it as an evolvable parameter tuple.

We implemented three straightforward rules:

1. TypeScript Type Signatures: JSON Schema is ridiculously verbose. Modern LLMs parse compact TypeScript interfaces just as accurately as full OpenAPI JSON schemas, but at roughly 18% of the token footprint. That change alone wiped out over 3,500 tokens of schema overhead.
2. Terminal Observation Compaction: When a bash command outputs 400 lines of pytest or build output, the model almost never needs all 400 lines. Mauricio intercepts stdout, checks the exit code, extracts the head and tail failure lines, and saves the full dump to a scratch log on disk. The agent only receives the surgical excerpt unless it explicitly requests the full log.
3. Pareto Harness Optimisation: We evaluated candidate harnesses across generations using a Pareto fitness function balancing token savings against benchmark success:

The result? A good reduction in token consumption while maintaining task fidelity. I prefer to keep numbers for future research. Mauricio operates like a lean Unix pipe instead of an overweight browser sandbox.

2. Knowing When You’re Struggling: The 60% Confidence Guard

Small Language Models (SLMs) in the 2B to 4B parameter range, like Qwen 2.5 Coder 3B or Gemma 2 2B, are fantastic for their small size. On a local GPU or via frameworks like vLLM and LM Studio, they generate tokens at 50+ tokens/second and keep all your proprietary code on your local disk.

For half of daily programming tasks, they are more than enough.

The problem is the other half: complex multi-file refactoring, distributed locking, or intricate architectural reasoning. That’s where small models abruptly drop off a cliff.

                         +---------------------------+
                         |  Incoming Task Prompt q   |
                         +---------------------------+
                                       |
                                       v
                     +------------------------------------+
                     |    TCME Confidence Guard Engine    |
                     |  - Benchmark Priors B(M, d)        |
                     |  - Cognitive Complexity D(q)       |
                     |  - Feedback Drift Delta(H)         |
                     +------------------------------------+
                                       |
                          Estimated Confidence C(q, M)
                                       |
                    +------------------+------------------+
                    |                                     |
            C(q, M) >= 60%                        C(q, M) < 60%
                    |                                     |
                    v                                     v
       +--------------------------+          +--------------------------+
       |  Stay 100% Local (SLM)   |          |  Autonomous Escalation   |
       |  - 0€ Marginal Cost      |          |  Resilient Cascade:      |
       |  - Private on Local GPU  |          |  1. Gemini 3.8 Flash     |
       |  - Sub-second TTFT       |          |  2. DeepSeek Flash       |
       +--------------------------+          |  3. Fable                |
                                             |  4. GPT-6                |
                                             +--------------------------+
                                                          |
                                                          v
                                             [ Restore Local Pointer ]

The Stream Learning Link: Performance Estimation Without Ground Truth

In production, you don’t have unit-test labels before you make a model call.

This connects directly to our research on data stream learning [2] and continual learning efficiency across the cloud-edge continuum [5]. When dealing with streaming data or resource-constrained edge deployments, the system must detect performance degradation and concept drift before bad predictions cause damage.

Borrowing the theoretical foundations behind Confidence-Based Performance Estimation (CBPE) from Kivimäki et al. [3], Mauricio calculates an a priori confidence score before executing any query.

The 60% Rule in Action: If confidence is 60% or higher, Mauricio executes 100% locally on the GPU (0€ cost, zero latency). If confidence drops below 60%, it cascades through[Gemini -> DeepSeek Flash -> Fable -> GPT-6], executes that single turn, and immediately resets back to the local model. No subscription leakage!

3. Physical Domotics and Publishing via MCP

An agent that only exists in terminal land is half an assistant. Mauricio connects to external services as an MCP client. Instead of building custom REST scrapers, we connected Mauricio to our site through an MCP Server.

   +------------------------------------+
   |    Mauricio Autonomous CLI         |  <--- (MCP Client)
   +------------------------------------+
                     |
                     v
   +------------------------------------+
   |          MCP Cloud Gateway         | 
   +------------------------------------+
                     |
                     v (Secure Relay)
   +------------------------------------+
   |   DataStruggling.com               |  
   +------------------------------------+
                     |
                     +---> Native Post & Draft Creation
                     +---> Block Formatting
                     +---> Co-Authorship Metadata & Visual Bylines
                     +---> Local Offline Staging Fallback