+91 98726 60544 hello@mitstech.co Mon–Sat · 09:00–18:30 IST

The real cost of skipping automated tests

IT Strategy By Mits Engineering Team 3 min read
The real cost of skipping automated tests

Automated testing is the first thing cut when a deadline tightens, and the reasoning is always the same: the feature is what the customer wants, tests are overhead, and we will add them later. It sounds like a trade of quality for speed. In practice it is a trade of speed later for speed now, at a poor exchange rate, and the bill arrives within months rather than years.

The mechanism is simple enough to be worth spelling out. Without a suite you trust, the only way to gain confidence before a release is to have people click through the application. That takes days, so releases get batched to avoid paying the cost too often. Batched releases contain more changes, so when something breaks the cause is harder to isolate. Harder isolation means longer outages and more caution, which means larger batches. Teams describe the end state as moving slowly because the system is complex; usually the system is normal and the feedback loop is broken.

Then there is where defects get found. A bug caught by a unit test is found in seconds by the person who wrote the code, with the context still in their head, and fixed in minutes. The same bug found in production involves a customer noticing, a support ticket, an engineer context-switching off other work, reproducing an issue from an incomplete description, fixing it under pressure, and an emergency release. The engineering time alone is an order of magnitude greater, before counting the customer's lost confidence.

The suite that pays for itself is not the one that chases a coverage percentage. It is layered deliberately: many fast unit tests around business logic and edge cases, a solid middle layer of integration and contract tests that catch the failures which actually reach production - a changed API response, a broken database assumption, a mishandled null - and a small, carefully maintained set of end-to-end journeys covering the handful of flows that must never break. Inverting that shape, with many slow browser tests and few unit tests, produces a suite that is expensive to run and painful to maintain.

One failure mode deserves particular attention because it destroys the value of everything else: flakiness. A suite that fails intermittently for reasons unrelated to the change teaches everyone to re-run it and, eventually, to ignore red builds. At that point you are paying the full cost of the tests and receiving none of the benefit - which is genuinely worse than having no tests, because the team believes it is protected. Flaky tests should be quarantined and fixed with the same seriousness as production bugs.

If you are inheriting a codebase with no tests, do not attempt to backfill coverage across the whole thing - it never finishes and the value is spread too thin to notice. Write tests for the next bug you fix, so it cannot return. Write tests around any area before you change it. And write end-to-end tests for the two or three journeys that would be a genuine emergency if they broke. Within a few months the protection concentrates exactly where the code changes most, which is where it is worth having.

For buyers rather than engineers, this translates into one question to ask any development partner: what does your CI pipeline run on a pull request, and can I see the coverage report from a comparable project? A firm doing this properly answers in seconds. A firm that does not will explain why their situation is different, and that explanation is the answer.

Need help with this? Explore our Software Development services. Learn more Back to all news

Keep reading

More on IT Strategy