The Maverik Debugging Challenge
The Maverik customer loyalty card website was developed several years ago as a Java monolith. In 2018, Maverik began replacing the legacy Java monolith architecture with modern Java-based microservices running on a Kubernetes stack.
Before Rookout was introduced, during development and staging and while the new microservices architecture was not yet in production, Maverik developers used the traditional debugging approach: Using an APM (Application Performance Management) solution, they would identify where a problem might lie. Then, they would add log lines and re-release the code to capture debugging data required to pinpoint the issue’s root cause.
Adding logs entails a 5-hour rebuild/test/redeploy process, which significantly slows the debugging speed even in the current pre-production environment. This becomes an even bigger challenge when the new microservices solution is released to production — a more controlled environment where code-changes take even longer to approve and release.
Getting Started with Rookout
One early adopter at Maverik, however, chose to dig into Rookout even before the new system was released and was gratified to discover how quickly it allowed him to debug issues. Jerrie Pineda, Maverik’s Enterprise Software Architect, found that Rookout can ease Maverik’s refactoring effort by enabling him to resolve issues faster than ever before.
When Jerrie needs to identify the exact spot within the code where the problem occurs, he uses Rookout. He sets non-breaking breakpoints on the fly, without changing/pushing new code and without slowing down or stopping the stack. Rookout lets Jerrie easily grab variables and full stack trace data to pinpoint the origin of the problem, so he can quickly implement a fix.
Adding Rookout on top of our APM and logging solutions lets me accurately pinpoint the exact line I want to look at. That is extremely helpful and saves me time and effort!
“When it comes to troubleshooting,” explained Jerrie Pineda, “our APM tells us which services are misbehaving and we use a cloud-based log aggregator for historical logging. But there is lots of stuff we do not capture with these tools. Adding Rookout on top of our APM and logging solutions lets me accurately pinpoint the exact line I want to look at. That is extremely helpful and saves me time and effort!”
With the new microservices-based stack, developers can’t freely push code changes at will. Using Rookout’s non-breaking breakpoints is the quickest and simplest way to get data points from live systems. This makes Rookout ideal for the Maverik engineering team for debugging issues their customers flag in production.
Jerrie is excited about Rookout for production debugging but also for other environments,
Our dev team will continue to use Rookout even in staging since it is basically remote debugging. Starting a remote process and creating an environment takes 15 minutes, not hours and days like redeploying, but those 15 minutes of context switching can mean the difference between releasing a feature or hotfix today or tomorrow.
Results
When a new problem comes in, I can react more quickly by capturing data with Rookout. Rookout helps me get the debugging data I need in seconds instead of waiting for several hours. As a result, our mean time to resolution (MTTR) for most issues is slashed by up to 80%, which is huge for us.
The Bottom Line
Rookout simplifies and accelerates Maverik’s transition to Kubernetes. Before Rookout, troubleshooting Maverik’s code depended on adding log lines and waiting for several cycles of rebuild-test-redeploy. Now, Maverik developers can significantly speed up debugging of their Kubernetes based microservices both in pre-Production environments as well as in Production.
A Few Words About Maverik
Founded in 1928, Maverik has grown to become the largest independent convenience store chain in the Intermountain West, with 331 stores and over 5,000 employees in 10 states. As
Adventure’s First Stop, Maverik’s goal is to build the coolest convenience experience on the planet.
Rookout helps me get the debugging data I need in seconds instead of waiting for several hours. As a result, our mean time to resolution (MTTR) for most issues is slashed by up to 80%, which is huge for us.
Root Cause identification
Instant snapshot of issues in live systems
MTTR is slashed by 80%