Writing · Adapt Progress Evolve
Personal
Five Questions to Ask Before Hiring an AI Studio
Most studios added AI to their pitch decks after the last board meeting. Here's how to tell if they actually know what they're doing.
Filed · 4 min read
Every studio pitch deck has "AI" on the cover now. Half of them added it after the last board meeting, not before. That's the problem, really, and it's not a small one: the word has become a costume, not a competency, and companies evaluating these studios are increasingly just... buying the costume.
I get why. The pressure to "have an AI strategy" is real, the fear of being left behind is real, and there's a whole industry of studios happy to nod along while charging accordingly. But most of them can't tell you what model they'd use for your problem, let alone why. They just know the word tests well in a proposal.
So here's how to actually check.
Five questions to ask before you sign anything
First: which model, and why not the others? A studio with real capability will have an opinion, quickly, and it'll be specific to your data and your constraints, not a name they read on a leaderboard last week. If they hedge or say "we'll figure that out in discovery," fine, that's honest. If they claim certainty without asking you a single question about your data first, that's worse than honesty. That's theatre.
Second: what happens to your data once it's in their system? Not the marketing answer. The actual answer. Where does it sit, who can see it, does it train anything beyond your own instance. If they haven't thought about this, they haven't built anything real, because anyone who has, has been burned by not thinking about it early enough.
Third: how do you know when the model is wrong? This one trips almost everyone up. A credible studio has a monitoring plan before the build starts, not after the client complains. If the answer is vague, along the lines of "we'll keep an eye on it," ask what "it" means. Usually there's no answer under the answer.
Fourth: who owns the model and the outputs once the engagement ends? This sounds like a legal question and it partly is, but it's also a competence question. Studios that haven't thought about ownership haven't thought about what happens after launch, and what happens after launch is basically the whole point.
Fifth, and this is the one people forget: what's your rollback plan if this doesn't work? Not "if it fails catastrophically", just... if it underperforms, quietly, for a month. Anyone who hasn't got an answer hasn't shipped anything real before. That's not a guess, it's just how it goes.
Red flags in the proposal itself
You can tell a lot before the meeting even happens, honestly, just from the document.
Vague model selection is the big one. "We'll leverage the latest AI technology to..." is not a sentence, it's a placeholder someone forgot to finish. A studio that's actually done this before will name the model, name the alternative they considered, and tell you in one line why they didn't pick it. If there's no alternative mentioned at all, there was probably no real decision made, just a default.
No data strategy is the second flag, and it's the one people miss because it's often just... absent. Not wrong, not thin, just not there. A proposal that jumps straight from "problem" to "solution" without a section on what data exists, what shape it's in, and what needs to happen to it first, is a proposal written by people who haven't done the unglamorous part of the work yet. And the unglamorous part is most of the work.
Third: no governance mention, anywhere. This used to be optional. It isn't now. If a proposal doesn't touch on who's accountable when the model makes a bad call, who audits it, how often, and what the escalation path looks like, that's not an oversight, that's a studio that hasn't shipped anything into a regulated or scrutinised environment. Which might be fine for a toy. It's not fine for your business.
There's a fourth, smaller flag worth naming: timelines that don't move. Real AI work has uncertainty baked into it, always, because data is messy and models behave unpredictably against real inputs. A studio that gives you a fixed, confident timeline with no caveats either hasn't done this before or isn't telling you something.
What a credible proposal actually looks like
It's less exciting than the pitch, honestly, and that's sort of the point.
A credible engagement proposal starts with discovery, properly scoped, not as a courtesy chapter before the "real" work. It names the model choice and the reasoning, in plain language, without leaning on buzzwords to fill the gap where reasoning should be. It has a data section that reads like someone actually looked at your data, or at least asked hard questions about it, rather than assuming it'll be clean and available.
It includes a governance layer: who owns decisions, how outputs get checked, what the audit trail looks like. It has a monitoring plan that starts on day one, not after go-live. And it's honest about uncertainty, which sounds like a weakness but is actually the strongest signal you'll get, because confidence without honesty is just marketing wearing a lab coat.
None of this is thrilling. It won't make the deck sing. But it's the difference between a studio that's built something real and one that's building the appearance of having built something real, and those two things look identical right up until the point where they don't.
Anyway. Ask the five questions. Read the proposal twice. The studios worth hiring won't mind either.
Filed under Personal