Wy AI Can't Read the Room
4 mins read
Part 1, Part 2 and Part 3 of this series ("The Unscalable Advantage") made a counterintuitive argument: AI is making scalability worthless. Value is migrating to what doesn't scale. The farmer's market, not the supermarket.
So here is a logical question: Why? Why can't AI just scale the farmer's market too? Why does the pattern hold?
The answer is a million years old.
The Paradox
In 1997, when Deep Blue beat Garry Kasparov at chess, there was a mild panic.
Then in 2016, AlphaGo beat Lee Sedol at Go - a game way more complex than chess, with more possible board positions than atoms in the universe, a game we thought required human intuition, not just calculation.
The panic then deepened. If machines could master both computation AND intuition, what was left for humans?
The answer, it turns out, is: almost everything that actually matters.
A toddler can walk across a room without falling. AlphaGo cannot. A five-year-old knows when someone is lying. AlphaGo has no idea. You can read a room in seconds - sense the tension, detect sarcasm, know when to speak and when to shut up. AlphaGo would fail every single one of those tasks.
This isn't a bug. It's the fundamental architecture of intelligence itself.
The roboticist Hans Moravec noticed this in the 1980s (read my piece here). What he found became known as Moravec's Paradox: the hard things for humans are easy for machines. The easy things for humans are nearly impossible for machines.
Chess? Go? Hard for humans, easy for AI. It's computation. Pattern matching. Machines crush it.
Walking without tripping? Easy for humans, hard for AI. Reading body language? Easy for humans, hard for AI. Knowing when your client is about to bail on a deal, even though they haven't said it yet? Easy for humans. Impossible for AI.
Here's why.
The things that feel hard to us - calculus, chess, formal logic - are recent inventions. A few hundred years old, maybe a few thousand. We have to learn them deliberately. They don't come naturally. They require effort, training, conscious thought.
The things that feel easy to us - walking, talking, recognizing faces, sensing danger, reading emotions - are ancient. A million years of evolution built them into our nervous system. We don't think about them. We just do them. A baby learns to walk without a manual.
You know when someone is angry without running an algorithm.
Because evolutionary time matters.
AI is phenomenal at the recent stuff. The codifiable bits of our world. The things we invented in the last few centuries that can be broken down into rules, steps, and formal procedures. Math. Chess. Go. Language translation. Data analysis. Code generation.
But AI struggles with the ancient stuff. The embodied parts. The things evolution spent millions of years optimizing that we now take for granted. Physical intuition. Social intelligence. Contextual judgment. Common sense.
And here's what most business leaders are missing: the ancient stuff is where your actual competitive advantage now lives.
Let's bring this back to the farmer's market.
The supermarket optimizes for the recent stuff. Inventory management. Supply chain logistics. Pricing algorithms. Checkout efficiency. All of it is codifiable. All of it is what I call mechanical intelligence. AI will dominate every single one of those dimensions.
The farmer's market runs on the ancient stuff. The vendor recognizes you. Remembers what you bought last week. Notices you're in a hurry today and speeds up the interaction. Sees you're browsing and gives you space. Knows the tomatoes that just came in are better than the ones on display and sets them aside for you.
None of that is a workflow. None of it is in a database. It's embodied intelligence. Social intelligence. Contextual judgment honed over evolutionary time.
You can't prompt that.
This is why "agentizing" your customer relationships is so dangerous. You're not just automating tasks. You're eliminating the conditions under which ancient intelligence operates. The presence. The eye contact. The micro-adjustments. The unspoken read of the situation.
An AI agent can send a perfectly crafted email. But it can't sense that the email should never have been sent in the first place.
An agent can analyze customer data. But it can't tell you that the data is missing the most important thing: the shift in tone on the last call that signals the relationship is fracturing.
Where to rebuild your business moat
Move your moat away from things that require computing power. Machines will always beat you there. Move your moat away from things that can be codified into playbooks. Those are commodities now, or they will be soon enough.
Rebuild around the things evolution gave you for free.
The things that feel so natural, you don't even think about them. The ability to read a room. To sense when something is off. To know what someone needs before they ask. To make a judgment call that no LLM context could capture.
That's not mechanical intelligence. That's human intelligence. And it's the only moat left that a better token budget can't replicate.
This is part 4 of The Unscalable Advantage - a series on what survival looks like for SMBs in the age of AI. The final piece will explore what this means in practice: how to build a business around what can't be scaled.