Want To Release Faster? Address These Bottlenecks

Develop fast, release, learn, repeat. That’s essentially the (not-so-secret) innovation formula, right?

Most of us spend our time enhancing the products we have already released. We want to be innovative, releasing new features with the velocity of an unencumbered startup. Yet, we also have customers with quality expectations we need to meet.

Guidance on shortening release cycles often centers on adopting agile (or similar) development methodologies. But most companies are already there. This post addresses bottlenecks that emerge or still remain in agile-oriented processes.

Most DevOps processes have phases that look something like this:

Plan > Code > Build > Test > Release > Deploy > Operate > Monitor > (and back to Plan).

When attempting to accelerate releases, testing and debugging are two bottlenecks that slow everything down. Teams spend a lot of time creating, running, and fixing tests. When bugs are found, developers also spend significant time going through multiple deployment cycles.

For this post, I’ll highlight a couple of bottlenecks that slow down release cycles. I’ll also discuss how you can address them through faster, more stable automated testing and the right tools to pinpoint code-level malfunctions on the fly.

First, terminology

When I talk about faster releases, I don’t mean POCs or prototypes where the goal is to see whether something works to test a concept. They don’t need to scale and quality is a secondary concern.

Instead, I’m talking about releasing MVP (minimum viable product) features to your customers and then iterating with feedback to evolve them. Your customers expect functional quality. Bugs, even in an MVP feature, reflect poorly on the entire product.

Second, the case for quality

For many organizations, their application is their brand. When the application is useful and functioning, customer satisfaction rises, but when users encounter quality issues, they become dissatisfied and are more likely to choose alternatives.

The financial impacts can be significant. According to a 2018 Accenture survey, 48% of consumers have left a business site due to poor customer experience and purchased on another site.

Low quality can also impact customer retention. According to Frederick Reichheld, Bain & Co., it can be 6-7X more expensive to acquire a new customer than retain an existing customer. Boosting customer retention by as little as 5% can increase profits by 25-95%.

Bottlenecks to faster releases

There are many potential bottlenecks or inefficiencies in a release cycle such as a poorly defined spec, gaps in team collaboration, fruitless debugging, ineffectual testing, or manual steps between automated processes.

Let’s focus on two areas that slow down fast-moving teams: end-to-end testing and debugging. Why these? Typically, they are both very iterative and people-intensive.

Debugging can slow down releases in coding and testing phases. Debugging typically involves stopping the application, inserting a breakpoint, running the application, and digging through data to find the error.

Most organizations would agree that their software development teams spend a large portion of their time each week debugging issues that arise in new releases or on technical debt that slows down the time to release new software. When a new issue is found, it often requires backtracking in the release process, redeploying applications for further testing in lower environments, or constant loops between testers and developers where the typical response is often, “it works on my machine.” Through the use of a remote debugging solution like Rookout, teams can diagnose issues in the native environment where they occur (on-prem or in the cloud) and eliminate much of the back and forth time that happens between teams over the course of a release.

Testing can generate bottlenecks in two primary areas: the time it takes to author end-to-end tests, and time to maintain existing tests. Many QA automation engineers state that it takes two toand eight hours to write an end-to-end test on an open-source automation framework. QA teams also report spending 30-40% of their team’s time maintaining existing tests. Devs and automation engineers are critical and expensive resources. If they are writing tests, they aren’t writing features.

Failing tests create additional work to identify the root cause. Was it a bug in the application, a poorly defined test, or an element not found? Failed tests need to be fixed and rerun, adding time to release cycles.

When you couple a complete testing strategy with a powerful production-grade debugging solution like Rookout, your development and testing teams will have the ability to pinpoint and resolve new issues immediately and improve team velocity.

End-to-end tests

E2E tests manipulate the browser to simulate real user journeys. For example, an e-commerce application might use E2E tests to simulate a user searching for a product, viewing the results, clicking into details, adding an item to a cart, logging in, and completing a purchase.

The significant number of user journeys, devices, browsers, data (valid and invalid), as well as different network and server response times create a massive array of possibilities. Manual testing quickly becomes too slow and costly.

Yet, many organizations use manual testers rather than automating. In fact, in a June 2020 survey on E2E testing by Testim, 74.6% of respondents said less than 50% of their E2E tests were automated—including 12.7% who said none of their E2E tests were automated.

There are many commercial and open-source test automation frameworks for E2E testing. You can see a comparison of some popular ones in this blog.

A modern approach to E2E testing

Several functional, UI and E2E testing solutions use artificial intelligence to speed test authoring and increase stability.

Instead of coding tests that can take hours, a user flow is recorded and then configured in the tool. Now tests can be created and configured in less than 20 minutes, reducing the amount of time to achieve test coverage.

Here’s the cool part: these tools use AI to identify each element uniquely and lock it in. Rather than using CSS, propertyID, or XPath, tools like Testim capture information about the entire DOM to understand the application and the relationships between different elements. If one or more of the element’s attributes change due to an application update, the test will still find it, minimizing flakiness.

AI-based locators minimize test updates for minor code changes. Developers are free to experiment without breaking the UI tests. UI tests should work this way—simulating how a user would see the new button and act on it.

Work smarter to release faster

Release bottlenecks are caused by a variety of issues, such as slow test authoring and flaky tests that consume time and resources, as well as long and inefficient debugging cycles. Some organizations make up for inefficiencies by staffing up or using a variety of partial solutions—but that’s not a scalable model.

You need a combination of a test automation solution that enables fast authoring and adapts as your application changes, as well as a tool that allows developers to gain instant, laser-focused data from their code, allowing them to understand their code better and resolve issues faster.

The combination of proper testing and a production debugging tool will enable developers to be at the forefront of innovation. No more slow releases and wasted time. Fast innovation is the future, and it comes with this awesome pairing of two great tools.

AI-based test automation tools like Testim can help accelerate the authoring of functional and end-to-end tests with a record/configure/customize model. Faster authoring shrinks the testing phase and builds the coverage to catch regressions. The AI-based locators help improve the stability of the tests so that they don’t break with every code change. Rather than spending your time troubleshooting tests, you can focus on the application bugs.

Live production debuggers like Rookout can help speed up debugging by providing the data you need to diagnose a defect without interrupting the application. With a non-breaking breakpoint, developers are able to get the data they need from any line of code, even when their application is running live. This helps them skip the endless deployment cycles needed to get data to understand the source of their bug. Rather, they are able to build a deep understanding of the application to better interpret what’s happening in their code, fix it, and get back to shipping fast.