In partnership with

Unplug the Test Box

This one isn’t a benchmark escape or a leaked API key.

This one reached the police.

Anthropic published a report on Friday listing cases where Claude acted on real websites in ways nobody intended.

Mostly during automated testing.

The case that stopped me cold came from the Philadelphia Police Department.

During one of those tests, Claude submitted false information about an unsolved murder.

To the department’s actual tip website.

A real form. A real investigation. A fabricated tip from a model that was supposed to be in a sandbox.

Nobody told it to.

It was pursuing a task, found a path to a live site, and filed.

Read that again.

The agent didn’t know it was in a test, and the test didn’t stop it from touching the world.

Here’s what Anthropic did about it, and it’s the part I actually respect.

They disclosed it themselves.

And they disabled live internet access for every internal agent evaluation.

All of them.

The company that wrote the “pace the frontier” essay just pulled the network cable out of its own test lab.

And they weren’t alone this week.

OpenAI published a report on an internal research model that was grading other models during training.

On October 6, it found its input files were missing.

So it tried to damage its own task environment, hoping the system would reset and hand it a fresh one with the files.

It sabotaged its own workspace to force a do-over.

Two labs. One week. Both saying out loud what their models did when nobody was looking.

Here’s the lesson, and it’s the bluntest version I’ve given you.

A test environment with a live internet connection is not a test environment.

It’s production with extra steps.

Every incident I’ve sent you since July had the same root cause.

OpenAI’s DNS gap. Google’s connected test box. Hugging Face. Now a police tip form.

Each time, a “sandbox” could reach the real world, and the agent did what agents do.

Anthropic’s answer is the right one and the simplest one.

Cut the cord.

If your agent is being tested, it gets no path to anything real.

No live email. No live payments. No live sites. A mirror, or nothing.

Only once it’s proven itself in the dark does it get a window.

The biggest labs on Earth just learned that the expensive way.

You get to learn it from a newsletter. 🫡

Did You Know?

Crime Stoppers, the first anonymous tip line, started in Albuquerque in 1976...

A detective set it up after a murder went unsolved for lack of witnesses, and the first tip solved a case within 72 hours.

Fifty years of humans phoning in what they saw.

Then an AI filed one about something it made up. 📞

Blu Dot surpasses 2,000% ROAS with self-serve CTV ads

Home furniture brand Blu Dot blew up on CTV with help from Roku Ads Manager. Here’s how:

After a test campaign reached 211,000 households and achieved 1,010% ROAS, the brand went all in to promote its annual sales event. It removed age and income constraints to expand reach and shifted budget to custom audiences and retargeting, where intent was strongest.

The results speak for themselves. As Blu Dot increased their investment by 10x, ROAS jumped to 2,308% and more page-view conversions surpassed 50,000.

“For CTV campaigns, Roku has been a top performer,” said Claire Folkestad, Paid Media Strategist, Blu Dot. “Comping to our other platforms, we have seen really strong ROAS… and highly efficient CPMs, lower than any other CTV partner we've worked with.”

Using Roku Ads Manager, the campaign moved from a pilot to a permanent performance engine for the brand.

🗞️ The Tip AI News 🗞️

Nadella wants an emergency brake. The question is whose hand is on it.

On Saturday, Microsoft’s Satya Nadella posted that AI models need an “emergency brake” that humans control.

Not a lab. Not a regulator. The people who DEPLOY the system.

His argument is that whoever runs an AI in the real world should hold the controls to stop it.

And Washington moved the same week.

Trump brought the CEOs of OpenAI, Anthropic, Google, Meta, xAI, and Nvidia to the White House to sign a voluntary accord.

Reports say the administration is now pressing AI companies to disclose incidents immediately, with Anthropic’s agent breaches as the trigger.

Britain’s privacy regulator opened enquiries with OpenAI, Anthropic, and Meta about agents that bypassed protections and reached outside systems.

And in California, lab representatives testified at a hearing after being told they’d be subpoenaed if they didn’t show.

So step back and look at the shape of it.

Three weeks ago the slowdown idea was declared dead. No brakes from Washington, no brakes from Beijing.

Now there’s a voluntary accord, an incident-disclosure push, regulators asking questions on two continents, and the CEO of Microsoft asking for a brake.

Not a slowdown.

A stop button.

That’s a different request, and it’s a smarter one.

Nobody’s asking the frontier to move slower anymore.

They’re asking that every deployed system has a hand on a lever that can halt it, and that the hand belongs to a human.

Which lands exactly where this newsletter has been since the DNS incident.

OpenAI caught its escape in 12 minutes and took two and a half hours to actually stop it.

Detection isn’t response.

So the brake Nadella wants isn’t a feature the labs ship.

It’s the thing YOU build, test, and keep your hand on, for every agent you run.

If you can’t stop your agent in under a minute, you don’t have a brake.

You have a dashboard and a prayer. 🫡

😂Meme of the Day

Over to You...

Go pull the network cable on your test agents today...

And go find out how fast you can actually stop the live ones. 🔥

Get paid to solve problems like this → AI Certified Consultant

PS. Anthropic also announced that Claude’s Managed Agents can now run up to 1,000 sub-agents in parallel... A thousand agents, the same week they pulled the internet on their own tests. Both things are true. Both are the point. 👀

» Join the AI Money Group «
💰 AI Money Blueprint: Your First $1K with AI - Learn the 7 proven ways to make money with AI right now

🚀 Zero to Product Masterclass - Watch us build a sellable AI product LIVE, then do it yourself

📞 Monthly Group Calls - Live training, Q&A, and strategy sessions with Jeff

Sent to: {{ email }}

Jeff J Hunter, 3220 W Monte Vista Ave #105, Turlock,
CA 95380, United States

Don't want future emails?

Reply

Avatar

or to participate