Can AI build your agent? A benchmark says 23.9%. Tech & AI Weekly by Eli, September 11 2026
Hi!
My agent caching series just started and the repo is public. Prompt caching discounts the input you send again. It never reuses the answer, and that is where the real money is.
Then the news. Ah yes, another week, another batch of new models, another giant funding round, another benchmark. AI headlines barely change shape. What is worth reading this week is underneath them, and it all says the same thing: the model was never the hard part.
✍️ From me this week
- Prompt Caching Isn’t Enough · Your agent re-reasons, re-calls every tool and rewrites what it answered a minute ago, discount included. First post of the series: where an agent can cache, and the two caches that skip the work instead of discounting it. On the deployed sample with Amazon Nova Lite the warm run went from 5 cycles to 2 and from 3 tool calls to none. The traps are in there too, starting with my first version that saved nothing. All the code is in the repo, and the next two posts cover each backend.
🤖 AI & GenAI
The AI assistant can build you an agent. It just does not work alone yet · Sierra’s new benchmark puts coding agents in the developer’s seat (Claude Code, Codex, Kimi Code): here are a simulated company’s docs, transcripts, API and codebase, plus a budget the finished agent has to respect. The best build, Claude Opus 5 in Claude Code, passed 23.9%, and nobody got past 25%. They shipped the first design that ran, 92% of them the same single tool loop. Here is the part worth keeping: in one telecom task, a single sentence suggesting a different architecture took a build from 31% to 67%. The assistant will not work out agentic architecture on its own. Knowing it yourself is what lets you steer it. Read more
Review is the bottleneck now, not writing the code · OpenAI’s research org logs three agent-workdays for every human workday, and debugging office hours emptied out so much that one team stopped holding them. Then the catch: humans still had to step in on more than half of the tasks the agents finished successfully. Producing code stopped being the constraint. Reviewing it became one. Read more
Google open sourced Mantis, and the harness is the lesson · Agents that find, reproduce and fix vulnerabilities, built because AI code scanners hallucinate bugs. What makes it work is not the model: critic agents filter the findings, a sandbox proves the bug is real before anyone believes it, and each stage runs on the cheapest model that can handle it. Worth reading as an architecture reference even if you never touch a CVE. Read more
Mistral raised €3 billion, the biggest round a European tech company has ever raised · Samsung led it. The pitch is not a bigger model, it is open weights plus the compute underneath, so a bank or a government can run frontier AI without renting its dependencies. Read more
☁️ Cloud & DevOps
GPT-6 Astra is generally available on Amazon Bedrock · 1M input tokens, computer and browser use, prompt caching with your own cache breakpoints. It is also the first OpenAI model classified Critical for cybersecurity capability, so it ships with automated misuse safeguards. Powerful enough to need a chaperone. Read more
vlt 1.0, an npm replacement from npm’s original creators · Best idea in it: vlt install downloads and extracts without running anything, and vlt build runs scripts only for what you trust. Nothing executes just because you typed install. pnpm and Bun are still faster on raw install.
Read more
📊 Data & ML
Nvidia fine-tuned a 30B model on its own supply chain and beat a much bigger one · Trained with Palantir on how its supply-chain team actually decides. One Vera Rubin rack has around 1.3 million parts, and one missing part stalls everything that did arrive. A small model that knows your decisions beats a big one that knows the internet. Read more
📅 Where to find me
- AWS Miami User Group, Thursday Sep 17, Miami · “Agent Speedrun: Idea → Code → Deploy → Observe, Fix → Ship”. Live: agent loop, serverless deploy, observability, then I break it on purpose and fix it with tracing. Come say hi. Details
That’s it for this week. Nobody’s agent-builder passed a quarter of its tests, review is the new bottleneck, and Google’s answer to hallucinated bugs was a better harness. Cache what you already answered.
See you next Friday,
Eli
Dev.to · LinkedIn · GitHub · Twitter/X · Instagram · YouTube
¿Qué te pareció esta edición? Solo responde a este email.