AI Vendor Red Flags: What to Watch Before You Sign
AI vendors are not slowing down. If anything, the sales pressure has increased. Every week there is a new tool, a new pitch, and a new promise that this one will change everything.
Most of those promises are at least partially true. But some of them will cost you. And the difference between a smart AI investment and a painful one often comes down to what you spot before you sign.
This article is for IT leaders who are in active conversations with AI vendors. Not to make you cynical, but to make you sharper. The red flags below are real, they are common, and they are avoidable if you know what to look for.
Red Flag 1: They Cannot Explain Where Your Data Goes
This should be a non-starter, but it still happens. You ask the vendor how your data is handled and you get a vague answer about “enterprise-grade security” or a link to a generic privacy page.
That is not an answer.
Before you sign anything, you need to know: Does the vendor use your data to train their models? Where is your data stored, and in which regions? Who can access it internally? How long is it retained after you cancel? What happens to your prompts and the outputs generated by their system?
These are basic questions. If the vendor cannot answer them clearly in writing, that tells you something important. Either they have not built the compliance infrastructure to support enterprise customers, or they are not willing to be transparent about practices you might not like.
Ask for the data processing addendum. Read it. If one does not exist, walk away.
Red Flag 2: The Demo Works Perfectly on Their Data, Not Yours
Vendor demos are built to impress. The data is clean, the use case is simple, and the AI performs flawlessly. That is by design.
What matters is how the tool performs on your data, with your workflows, in your environment.
A vendor who is confident in their product will welcome a proof of concept using your actual data and your actual use cases. A vendor who pushes back on this, or who insists that their demo is representative enough, is telling you something.
Your IT environment is probably messier than the demo. Your data likely has inconsistencies, legacy formats, and gaps. The AI tool needs to handle that reality. If you have not tested it against real conditions, you do not know what you are buying.
Always insist on a structured pilot before full commitment. Set specific success criteria in advance. Measure against them. If the vendor cannot support this process, that is a red flag.
Red Flag 3: Lock-In Is Baked Into the Contract
AI vendor contracts have gotten more aggressive as the market has heated up. Some of the lock-in mechanisms are obvious. Others are subtle.
Watch for multi-year commitments with steep early termination fees. Watch for contracts that tie you to proprietary data formats that make migration painful. Watch for API structures that would require significant re-engineering if you ever needed to switch tools. Watch for clauses that auto-renew and require long notice windows to cancel.
The more a vendor’s business model depends on keeping you locked in rather than keeping you happy, the more carefully you should read the fine print.
Lock-in is not always bad. Sometimes a longer commitment comes with better pricing and it makes sense. But you should be choosing that tradeoff knowingly, not discovering it when you try to leave.
Before signing, ask your legal team to flag any clause that creates switching costs. Get clear on what “exit” looks like. If the vendor is unwilling to discuss portability and migration support, treat it as a warning.
Red Flag 4: The ROI Claims Are Vague or Impossible to Verify
“We save companies 20 hours per employee per week.” “Our customers see a 40 percent reduction in support tickets.” “AI pays for itself in 90 days.”
These numbers make for great slides. But when you ask how they were calculated, the answer often falls apart.
Be specific. Ask the vendor to walk you through exactly how a customer achieved the result they are citing. What was the baseline? What changed? How was the time savings measured? Can you talk to that customer directly?
If the numbers are real, the vendor should be able to show their work. If they deflect, generalize, or point you to a third-party analyst report funded by the vendor, that is a red flag.
This matters because AI tools have real productivity potential, but the results vary significantly based on use case, adoption, and implementation quality. A vendor who sells you on inflated ROI projections is setting you up for disappointment and setting themselves up for churn.
Build your own ROI case based on your specific use cases. Do not use the vendor’s numbers as your starting point.
Red Flag 5: They Downplay Integration Complexity
Getting AI to work inside your existing IT environment is almost always harder than the vendor makes it sound.
“It plugs right into your existing stack.” “Setup takes about a day.” “No IT resources required.”
These statements might be true in a sandbox environment with a single clean data source. They are rarely true in a real mid-market IT environment with a mix of legacy systems, multiple cloud platforms, and years of technical debt.
Watch for vendors who gloss over integration questions or hand you off to a professional services team to handle “a few configuration steps.” Those configuration steps often turn into months of work and significant cost.
Ask the vendor for a detailed integration map specific to your environment. Get a realistic timeline from their implementation team. Talk to references who have a similar stack to yours. Find out what actually broke during their rollout and how long it took to fix.
Implementation surprises are one of the most common reasons AI projects fail. A vendor who is honest about complexity upfront is worth more than one who sells you on simplicity and then disappears when things get hard.
Red Flag 6: Security Certifications Are Missing or Outdated
For most mid-market IT leaders, this is a compliance issue as much as a security issue. Your clients, your auditors, or your cyber insurance policy may require vendors in your environment to hold specific certifications.
Common ones to look for include SOC 2 Type II, ISO 27001, and depending on your industry, HIPAA compliance documentation or FedRAMP authorization.
SOC 2 Type I is a starting point but not sufficient on its own. Type I says the vendor designed controls appropriately. Type II says those controls have been tested over time and actually work. If a vendor only has Type I, ask why and when they expect Type II.
Also check the certification dates. A SOC 2 report from two years ago does not tell you much about a product that has changed significantly in the last 18 months. Ask for the most recent audit report and check what period it covers.
If a vendor cannot produce current, relevant certifications for an enterprise deployment, either they are not ready for enterprise customers or they are moving too fast to keep their compliance posture current. Neither is a good sign.
Red Flag 7: References Are Controlled and Curated
Every vendor can produce three happy customers who will tell you the product is great. What matters is how those references are selected.
If a vendor gives you references without any friction, that is a good sign. If they only offer to connect you with customers after lengthy internal review, or if the references they provide are all in very different industries from yours, be skeptical.
Ask specifically for references who are similar to you. Similar company size. Similar industry. Similar use case. If the vendor cannot find anyone who matches your profile, that tells you something about who their product actually works for.
When you talk to references, go beyond the prepared questions. Ask what surprised them about implementation. Ask what they would do differently. Ask whether they have thought about switching and what has stopped them. The most useful information often comes from what people say off-script.
Red Flag 8: The Vendor Cannot Tell You What the AI Cannot Do
This is one of the most revealing questions you can ask: “What does your AI get wrong? Where does it struggle?”
A vendor who answers with confidence and specificity is worth talking to. A vendor who pivots to features or tells you the model is “continuously improving” is telling you they either do not know their product’s limits or are not willing to be honest about them.
Every AI tool has failure modes. Hallucinations, gaps in domain-specific knowledge, weak performance on certain types of data, latency under load. These are not disqualifying, but you need to know about them so you can build appropriate guardrails and set realistic expectations for your users.
If you deploy an AI tool and your team discovers the failure modes on their own, that destroys trust fast. If you understand the limits upfront and communicate them clearly during rollout, you protect adoption and give yourself a chance to build the right feedback loops.
What to Do With This List
Use these red flags as a structured checklist during your next vendor evaluation. Not to talk yourself out of a good tool, but to slow down at the right moments and ask the right questions.
The best AI vendors welcome hard questions. They know their product has limits and they are transparent about them. They have done the compliance work. They are willing to pilot before you commit. And they can show you real results from customers who look like you.
That is the bar. Hold every vendor to it.
Catch Advisors helps IT leaders cut through vendor noise and make smarter technology decisions. If you are evaluating AI tools and want a second opinion before you sign, visit catchadvisors.com.