A few years ago, if someone could walk me through event sourcing or a service mesh in confident detail, I could reasonably assume they had built one. I can't assume that any more.
Today anyone can ask an AI assistant for a crisp explanation of almost any architecture topic — trade-offs, diagrams, vocabulary and all — and repeat it convincingly an hour later. As a principal solution architect I sit in a lot of interviews, design reviews and proposal discussions, and I've had to get better at a question I never used to ask so deliberately: is this person telling me what they know, or what they've read?
I want to be careful here, because this isn't a complaint about AI. I use these tools every day, and they make people learn faster. The problem is narrower: fluent explanation and real experience now sound much more alike than they used to. Telling them apart has become a skill.
What actually changed
Two things used to travel together, and they've come apart.
Knowledge — what a pattern is, what the options are, what the documentation says about trade-offs — has become cheap. An assistant will supply it in seconds.
Judgment — which trade-off matters in this system, what breaks at 2 a.m., what the documentation leaves out, when the textbook answer is wrong — is still expensive. You get it by making decisions and living with the consequences.
It's also easy to fool ourselves about our own judgment. In a 2025 randomized study, experienced open-source developers took about 19% longer on real tasks when they were allowed to use AI tools — while believing afterwards that the tools had made them roughly 20% faster. Confidence and evidence parted ways, even for experts, even about themselves. I find that a useful bit of humility to carry into any interview room.
Two axes: Evidence of delivery & operation (left to right) and Fluency of explanation (bottom to top).
- Knowledge talker: Articulate and current. Thin on specifics, consequences and failures.
- Practitioner-communicator: Explains clearly and can go several levels deep into what happened and why.
- Learner: Early in the journey, honest about gaps. Moves fast with the right mentors.
- Quiet practitioner: Has built and run real systems. Less polished — easy to underrate.
These axes describe evidence in a particular area, not a person's worth. Someone can be a practitioner-communicator in data platforms and a knowledge talker in front-end frameworks. All of us start as learners in something.
What borrowed knowledge sounds like
None of these is proof on its own. Together, they tell me the knowledge hasn't met reality yet.
- It stays generic. Best practices, but never attached to a specific system, team, traffic pattern or constraint.
- It's a shopping list. "We'd use Kafka, Kubernetes and a vector database" — a stack, not a decision with a rejected alternative.
- Nothing ever went wrong. No rollbacks, no redesigns, no regrets.
- Everything is equally certain. Real experience is lumpy: very sure about some things, openly unsure about others.
- The second follow-up question hurts. The first answer is polished. "How did you know?" or "what did that cost?" gets vague, or restarts from first principles.
- Numbers without units. "Much faster", rather than "p95 went from about 800 ms to under 200 ms once we removed the synchronous call".
What real experience sounds like
- Awkward constraints. A regulator, a vendor contract, a shared database, a team that couldn't be retrained in time.
- A trade-off with a loser. They can name what they didn't choose and what it would have been better at.
- Life after go-live. On-call load, cost surprises, the migration of the migration.
- "It depends" — followed by on what.
- A changed mind. A belief they used to hold, the evidence that changed it, and what they do differently now.
- Clear ownership. They separate what they decided from what the team did and what they only observed.
The ladder I climb in every conversation
I'm not trying to catch anyone out. I'm trying to move the conversation from concepts to evidence, one rung at a time.
- Claim: Tell me about a system where you used this.
- Context: What were the constraints — scale, team, deadline, regulation?
- Decision: What did you reject, and why?
- Consequence: What happened after go-live? What broke first?
- Evidence: How did you know it worked? What did you measure?
- Reflection: What would you do differently now?
A few other things I've found work well:
- Hand them a flawed design. A one-page architecture with two or three realistic problems in it. Practitioners tend to find the data and operational problems first; talkers tend to critique the technology choices.
- Change a constraint halfway. "Now assume that database is shared with three other teams and can't change schema this year." Watch how quickly the design adapts.
- Ask what it costs to run. Very few people who have only read about a system can estimate what drives its bill.
- Ask about the boring parts. Deployment, rollback, access control, data retention. They rarely show up in a generated summary. They always show up in real projects.
Don't make the opposite mistake
A fluency-based process doesn't only let talkers through. It screens out quiet practitioners — people working in their second or third language, people who think before they speak, people who just don't interview well. If the process rewards polish, it will select for polish.
So I give people time to think. I'll take a rough sketch over a polished speech. Where it can be shared, I ask for a real artifact — a design note, a postmortem, a pull request. And I try to judge the reasoning, not the accent or the speed.
This matters more now that remote interviews are being gamed. In 2025 several large technology and consulting employers publicly brought back at least one in-person interview round, citing candidates using AI tools during virtual interviews. I understand why. But the deeper fix isn't the room — it's asking questions that only experience can answer.
And to be clear: using AI to learn is not the problem. If someone used an assistant to get up to speed on a platform, then built something with it and can tell me what surprised them, that's real experience. The problem is only when generated knowledge stands in for experience that isn't there.
Working well with both on a team
Every team has people at different points on these axes. That's healthy — as long as decisions don't depend on who sounds most convincing.
- Make claims testable. A spike, a prototype or a benchmark before any significant decision. Fluent arguments are cheap; a prototype under realistic load isn't.
- Write decisions down. Decision records with alternatives, consequences and a review date turn persuasion into something you can check later.
- Pair on purpose. Put a strong explainer next to a strong practitioner. One learns the operational reality; the other's knowledge finally gets communicated.
- Review outcomes, not slides. In design reviews and retrospectives, spend more time on what happened than on what was proposed.
A note to myself, too
The same tools that make it easy to sound experienced make it easier to become experienced — if you use them that way. I use AI to prepare, to explore options, to get a first draft. Then I try to test what it told me: build the spike, read the source, measure the result, and notice where reality disagreed with the confident summary.
Those disagreements are where experience comes from. AI can make anyone sound knowledgeable. Only delivery and operation make anyone experienced.
Sources