Testing companies are starting to admit their own problems
Start with mabl. They had 288 tests running on every code change, and they cut the gate to 17. The reasoning is arithmetic. Even if every test works 99% of the time, the chance that all 288 pass together is about 5%. The alarm went off so often that nobody believed it anymore.
Their conclusion is that the check cannot sit inside the agent that wrote the code. It has to come from outside it.
Then SmartBear, the company behind TestComplete, published its new report. 46% of teams said they have shipped AI-generated code that later failed. Nearly half is a large share. Of those teams, 69% still said they had a lot or complete confidence that the code was acting as it was created.
Even Katalon's CEO put it directly: AI "can't be accountable for what gets shipped."
So the industry broadly agrees. Code is leaving the door faster than ever, and testing is still a pile of scripts that somebody has to write, look after and rerun at midnight. That leaves a choice between testing less and shipping slower.
That is why we built SpecHawk with no scripts at all. Our agents explore your app, find every user flow, and test the steps of a Jira ticket before it goes live. It is a way to ship quickly and still be safe.
The data behind this post
- mabl, "Our PR gate has just 17 tests, and here's why" (Pratish Singh, 2 October 2026): mabl.com
- SmartBear's State of Software Quality and Testing 2026, as SD Times reported it (30 September 2026): sdtimes.com
- Katalon's launch of its True platform (April 2026): theprint.in