Why We’ve Decided To Make An Ungated Rookout Sandbox

A long time ago (in April 2019), in a galaxy far far away (in Tel Aviv, Planet Earth) we launched a self-serve debugging option. Users could now sign up for Rookout without having to contact us first. We invested significant resources in online promotion and — just like any startup at our stage — we hoped to soon see legions of users signing up and using the tool to fetch data directly from their code. However, like many other startups in that initial stage, we soon discovered that many of the people who signed up had quickly “bounced”, never to be heard from again.

We had several theories as to why folks leave without as much as installing our SDK. Our suspicion was that perhaps, for some developers, installing an SDK isn’t as simple or straightforward as we had initially assumed. We figured developers would not easily trust a startup with what appears as having access to their code, even though we absolutely do not. Some members of the team believed the setup was more complex than we anticipated, while others suspected our product still had some quality issues that prevented a smooth setup experience.

User test it: do, or do not. There is no try

Naturally, we performed some user testing. We also reached out to some recent signups by email and chat and collected data using in-app surveys. We learned that some of the assumptions above were correct and we also gained a couple more insights.

Perhaps the most interesting thing we learned was that some people who signed up for our online offering don’t really understand what they had signed up for. Although they visited our web pages and watched an online video of a demo, some still thought they were about to install an IDE plugin, or that they were just about to use another web-based IDE. Some even believed they had signed up for an online coding school.

At this point, we were stumped. We worked so hard to provide a smooth, friendly onboarding experience, we fixed so many bugs and glitches, invested in the design for a brand new Welcome Wizard, and built a bunch of “workarounds” for tricky setup steps like setting up the Rookout SDK and fetching the developer’s code from their favorite git repo. And now, after months of hard work and frustration, we learn that some people don’t even know what they’re expected to get.

Saved by the Vader: “I find your lack of faith disturbing”

Happily, Liran, Rookout’s CTO, came up with a great metaphor. Like many things in our day-to-day conversations, it had to do with video games and Star Wars. He mentioned that in Star Wars: Battlefront, you get to play as Darth Vader during the tutorial phase. Other than being an epic experience that makes any mere mortal feel like the dark lord himself, it also has an important message: this is what you’ll be able to do when you make it through the game. It provides the motivation to work hard and sets expectations as to what you can achieve. What if we had a similar experience for new Rookout signups, just before we ask them to “work hard” at setting up and learning how to use a new tool? That’s when we came up with the “Rookout Sandbox” experience.

We set out to provide the ability to get a true, end-to-end taste of what Rookout can do for a developer, without all the hassle of installing, fetching code, and learning how to use a new tool. Just set a non-breaking breakpoint, trigger your app, and instantly get the data you need. Something so simple and straightforward, baby Yoda can do it without waking up from his (or her?) nap. Inspired by the gaming metaphor we took things a step further by adding a “quest log”: a set of hints, leading the user to check out some advanced Rookout abilities such as conditional breakpoints, rate limiting, log line formatting, and more.

Never tell me the odds

We launched the Sandbox experience a few months ago, and the initial feedback was highly positive. Users told us they were quickly able to understand what the product does. Some users even used the sandbox to demonstrate Rookout to their friends, allowing us to reach even further into the developer community and into the very center of our target audience.

Another significant change was when we made the Sandbox experience *ungated* – that is, users can simply enter the sandbox with a click of a button, without needing to sign up or provide us with their name and email address. When we made this change, weekly Sandbox usage increased by 300%.

Sandbox: a new hope

The sandbox experience has allowed us to overcome one of the biggest challenges for any newly-launched product: quickly demonstrating the value of the product with as little friction as possible. Of course, things are still not perfect. We keep iterating on our onboarding experience, collecting data and feedback from our users as we try and refine our setup flow. This time, however, we are certain that new users understand what they should expect to gain, and why they should try and gain it.

We hope you take a chance to try out our sandbox for yourself. You should be able to complete it in a couple of minutes and if for some reason you can’t — do let us know what stopped you! If nothing else works, we may end up adding Darth Vader and lightsabers. Or maybe baby Yoda. This is the way.

‍