Solving Customer Related Problems with R&D

As an R&D manager, there are many things on my mind that keep me up at night. These thoughts range anywhere from impossible research explorations, employee motivation, rising cloud costs, all the way to security incidents. However, there is one that sweeps all of them aside when they surface and that is solving customer facing issues. In its raw form, a customer issue means that someone who is paying you money can’t use your product or that your product isn’t doing what it’s supposed to do. And that someone can also decide to stop paying you money.

It’s a pretty simple equation: handle the customer issue or your company loses money. This sort of equation, when left unsolved, keeps me up at night. Since I like sleeping, I try to resolve our customer’s issues as fast as I can. And since I assume most of you do too, following are some of my strategies on how to handle those issues and sleep like a baby.

Admitting there is an issue

The basic essence of a customer issue is the fact that when you need to handle a customer’s issue, it is usually unplanned. I’m pretty sure no sane engineer ever released a version and awaited eagerly for that 4AM phone call from a customer. As unplanned tasks usually happen, we sometimes try to shrug them off or wave them off with disbelief. An instinctive reaction would be to say that “the customer is doing something wrong” or “the issue is unrelated to our product”. While these assumptions might be true on occasions, they don’t in any way change the fact that there is an issue that requires resolving. If the customer is doing something wrong or misusing our product, we need to be attentive and explain to the user how to properly use the product (and later on understand why he was doing what he did). If the customer thinks the issue is related to our product, then we need to prove to the customer why it isn’t related to our product. Waving hands and pointing fingers at other culprits without evidence won’t solve anything.

The first thing you have to do is admit there is an issue. Whether the customer approached you by chat or via a support representative doesn’t matter. A customer issue is an issue that requires resolution the moment the customer thinks it is an issue. And if there is an issue requiring R&D attention, you must assign an engineer who’s in charge. Most importantly, everybody needs to understand who the person is that’s responsible for handling the issue.

Start and move forward with information

Once you’ve admitted there is an issue and assigned an engineer, you’ll need to understand in which scope R&D needs to handle the issue. In our R&D team at Rookout, we have a very close relationship with our Account Managers, Support Engineers, and our Customer Success team. This sort of close relationship allows these teams to easily communicate their customers’ issues and their own concerns with us. There is no such thing as a ‘stupid question’, and we try to help them out whenever we can. But we always need additional information, and an account manager intimately knows their customer’s history and tech stack, which is seldom true for our R&D engineers.

When an issue is elevated to R&D, we need information to be able to take care of it. Being told that “customer X is having issue Y” contains no useful information for us. We need to understand how the customer has deployed and integrated Rookout, whether the customer has used Rookout before, what issues the customer has encountered in the past, what exactly the customer reported, and which steps the support engineer tried to take. Basically, we need to know everything.

Every piece of information is critical, whether it is past communication with the customer or even what the customer’s hunches and assumptions are. Without information and properly documenting everything, you are just a goldfish that has to do everything every time from scratch. The information is so important you need to make sure you document everything. Every step that you take, every communication with the customer, and every piece of data related to the issue must be documented. When handling an issue in Rookout, we document each step in a dedicated Slack channel for each customer and eventually write the information in the relevant Jira ticket.

Work together

Once you have the necessary information, you can devise a plan to handle the issue. If this is an issue the support engineer can handle alone, then just go ahead and give them the proper tools to remediate the issue. It might be just a link to the relevant documentation or some hand holding using a complex feature. Either way, you will need to work together. Remember that talking with customers isn’t your or your engineer’s day job.

The support engineer, account manager or whoever his day job is handling customers are the ones proficient in handling and communicating with customers. They will know how to get back to the customer and what to tell him. If needed, they will ask R&D to join a meeting with the customer or to write an email together. Since Rookout’s customers are engineers, it’s sometimes very tempting to send someone from R&D to talk with the customer (because we speak the same language, right?), but make sure you don’t send your engineers alone.

Whenever there is any doubt, there is no doubt

Since information is crucial to resolve the issue (or even understanding the issue), the information must be solid and concrete. Be doubtful, and don’t take anything for granted. No need to assume everybody is wrong, but when the behavior of your system is reflecting that the information isn’t right, dig deeper and re-ask questions. If some data needs to be collected again with more verbosity, then do it. Relying on false data will lead you into wild goose chases. If the customer tells you that his application is installed on a Linux machine and you observe that things are acting like on a Windows machine, then ask the customer to double-check it. Maybe someone changed something in the customer’s environment and they aren’t aware of it. If the customer tells you that they clicked on the “kebab menu” then maybe they got confused and actually clicked on the “hamburger button”. Simply ask the customer to “show me what you did” (because, let’s be real- we’ve all mixed up kebabs and hamburgers when hungry).

Choose your own battleground

Every minute the customer is talking with you, showing you things, or helping you to help him is time the customer has wasted. A cooperative customer is amazing, and debugging hands-on in the customer environment can often lead to a fast resolution. However, the customer’s time is precious and must not be wasted. Once you understand the issue and have all the information, go ahead and leave the customer alone and reproduce the problem in your own battleground (or playground, however, you call your dev environment). Additionally, fighting the battle in the customer’s playground leaves you without your own tools and freedom, as you are constrained by the customer and his environment.

Sometimes it is hard to reproduce the issue in a different environment (hey, that’s what we built Rookout for!), but if you need the customer to reproduce the issue, then get the customer to help you collect enough information to allow you to reproduce the problem. Understand exactly what the customer’s environment looks like, and try to mimic it. If the customer is having issues with your web app, then use the same browser and the same extensions they use. In other scenarios, sometimes you can also use the same public container images the customer uses. Don’t go blindly into the “reproduction game” and don’t be a ‘reenactor’ and commit yourself to it if you don’t have enough information. Keep in mind that you can’t reproduce without collecting all the information in advance.

Master the away game

If you haven’t managed to reproduce the issue, then you need to collect more data. Sometimes you’ll have to go through the cycle of collecting data, trying to reproduce with that data, understand that you need more data, collect more data and on and on. Make sure you have the right tools to collect the data. Remember that your customer’s time is precious, and you need to disturb him as minimally as possible. Telling them to upgrade so you can get information from more logs that you’ve just added can be quite a strain on the customer. Try to reduce friction with the customer as you progress by using the logs that you have, using your error reporting tool (Bugsnag or Sentry are my favorites), or collect whatever you need whenever you need it using Rookout (yeah, we also use Rookout to debug Rookout).

Fix the problem

It might sound like I’m stating the obvious. But seriously – make sure you fix the problem. Once you’ve found the root cause, fixing the issue for the customer might involve one of two paths. The first path can be a tactical solution. If the customer can refrain from encountering the issue by doing something different or changing a configuration, then go ahead and help them do just that (but don’t forget: document everything!). The second path can be the strategic solution of writing code to fix the bug. This tactical path can be chosen when either the strategic path will take too long to resolve or if the customer needs an immediate solution or is unable to upgrade or redeploy. The strategic path might not be taken immediately and might be prioritized later on by your product manager if the tactical path is acceptable. But if you expect other customers to face the issue – the strategic path is a no-brainer.

Make sure to follow through

Once you’ve decided how to handle and fix the issue, don’t forget where it all started. Everything began with the customer trying to do something and was unable to properly do it. You might have come a long way from that first time you’ve heard about that issue. Now, go ahead and take a look at what the customer reported. Make sure that everything works as the customer expects it to. Once you’ve made sure that you handled the issue, you can approach the customer (but remember, don’t go alone!) and ask the customer to verify that from their point of view, everything is resolved and they’re satisfied.

Solving customer issues makes your product better

The quality of your product depends on many factors, but what really pushes it forward are your customers. A product without customer issues is a product without customers. If your customers call you up and tell you about an issue they are experiencing, it means they love your product and want to use it. So give the love back to your customers and handle their issues as professionally as you can. Every time you handle and resolve an issue, your product improves, and ultimately, that helps me sleep more peacefully. 😉