Sponsored by
Visa Tested 22 AI Shoppers
Yesterday I told you two in five of your customers are bringing an AI to Black Friday.
Thursday Visa published how good those AIs are at the job.
Here’s what happened.
Visa built a benchmark and ran 22 AI models from 12 labs through 15 shopping tasks.
A TV, a laptop, a personal-care gadget.
The twist is that the fake customer didn’t hand over every requirement up front, the way real ones don’t.
To pass, the agent had to dig out every requirement and then make the right call: buy, or don’t.
The models failed 42.8% of runs.
Here’s the number to tape to your monitor.
In 54% of the failed runs, the agent bought something anyway.
Either the wrong product, or it stopped asking questions too early and hit purchase. 😳
Think about that.
The failure mode isn’t “the robot couldn’t find your product.”
It’s “the robot bought the wrong one with your customer’s money.”
The best model passed 82.2% of attempts.
The worst passed 28.9%.
Across all three attempts at the same task, the agents got it right every time in just 34.2% of cases.
Speaking of what you pay for...
The five most expensive models passed 67.1% of runs.
The five cheapest passed 50.2%.
Translation: the budget agent is the one that buys the wrong TV. 💰
Per Fortune, Google and Meta agents tied at the top, Claude came third at 76%, and OpenAI placed sixth at 64%.
Visa’s own survey found 23% of Americans trust an AI to make a payment for them.
Nuvei asked 1,000 Americans the same question earlier this year.
60% strongly prefer to approve every purchase themselves before paying.
Global Payments asked 8,000 shoppers across several markets in May.
42% worry an AI will buy the wrong thing on their behalf.
Visa just measured how often that worry comes true.
So the agent gets to find, compare and fill the cart.
The human keeps the card.
Let me explain what this means for your store.
Visa says the agents fail because they don’t uncover requirements before deciding.
That means the information on your product page is doing the agent’s homework.
If your sizing, compatibility, shipping cutoff and return policy live in an image or a dropdown, the agent can’t read it, assumes, and buys wrong.
Then your customer gets a return and you get a chargeback.
Fix the text and you fix the agent.
Speaking of product pages...
The agent reads your text and skips your images.
Your human customers do the opposite.
So the page needs to work both ways: plain text an agent can parse, and a visual that stops a thumb.
I put together a free field guide for making an AI video where your product flies out of a giant phone screen.
Seven copy-and-paste prompts, from the first image to the finished clip.
Give the robot the spec sheet.
Give the human the show. 😏
Did You Know?
Visa was called BankAmericard until 1976.
The company picked “Visa” because it’s pronounced almost the same in every language and means nothing in most of them.
Fifty years later it built a test to check whether an AI knows how to say “no.” 😂
A CRM so smart, it updates itself.
HubSpot's CRM is so smart, it now updates itself. Calls get logged and summarized the moment they end. New leads get researched and a first outreach drafted, automatically. Deals move forward on their own, with next steps flagged before you have to ask.
Even the one repetitive task you've been meaning to fix can now run itself, no code required. Customers using it are already seeing the difference.
This isn't the CRM you have to work hard to figure out. It's the one already working before you log in.
🗞️ The Tip AI News 🗞️
Anthropic says its AI now leads a quarter of the work of building its AI.
In September Anthropic published three internal metrics it wants every frontier lab to report.
The headline one: Claude “leads” about 26% of Anthropic’s AI research and development tasks, up from under 1% in February.
More than 90% of those tasks now involve AI at a collaborating level or above.
Here’s the catch Anthropic admits itself.
The system that decides what counts as “AI-led” was built by Claude reviewing its own work logs.
Here’s the outside check.
METR, an independent evaluator, now has embedded staff inside Anthropic with the same access as its internal risk teams and the right to publish what it finds.
Its reported estimate for how much AI is speeding up Anthropic’s progress is around 1.5x.
Anthropic’s own line is that it’s still short of a sustained, AI-driven doubling, which is the threshold in its safety framework that triggers stricter controls.
Here’s why it matters Monday.
The roadshow opens and investors get asked to pay for a company whose product is building the next version of the product.
A 1.5x multiplier makes a $2 trillion ask sound reasonable.
A self-graded one is what the SEC’s questions are for. 👀
😂Meme of the Day
Over to You...
Take Visa’s test yourself this weekend and see how your own instructions score.
If you’d have bought the wrong TV too, your product page was never the problem.🔥
Founder, AI Persona Method | TheTip.ai
Get paid to solve problems like this → AI Certified Consultant
PS.Visa’s cheapest five models passed half their runs. The priciest five passed two-thirds. That gap is the whole argument for paying $20 a month instead of $5, and you’ll see it again in Anthropic’s price range on Monday.
» Join the AI Money Group «
💰 AI Money Blueprint: Your First $1K with AI - Learn the 7 proven ways to make money with AI right now
🚀 Zero to Product Masterclass - Watch us build a sellable AI product LIVE, then do it yourself
📞 Monthly Group Calls - Live training, Q&A, and strategy sessions with Jeff
Sent to: {{ email }}
Jeff J Hunter, 3220 W Monte Vista Ave #105, Turlock,
CA 95380, United States
Don't want future emails?
