It’s time to look into the future of logging solutions: cut logging costs with snapshots

Logging is ancient history. You know – old and outdated. At one time, it was the best method – like sending carrier pigeons to convey messages – but we live in an ever-changing world. 

Long gone are the days in which logging was the primary method when troubleshooting and debugging. Any developer who was written millions of log lines can attest to the fact that they’ve probably needed to access maybe 1% of those. And even more so, have had difficulty finding those same logs when they need to understand what’s going on.

It’s simply an antiquated method of working. It isn’t dynamic. It isn’t efficient, and even worse, it comes with a considerable cost.

A better solution didn’t exist – until today. Welcome to the blog that will change your {logging} world –  both time and spend – and how developers work in the development cycle.

Let’s talk about the old logging solutions 

Back in the day, the only way for a developer to understand what was happening in their code was to write copious amounts of log lines. These log lines – whether ERROR, WARNING, INFO, etc – gave them the information they needed regarding what was happening in their running code when an issue occurred. Without these logs, developers were essentially blind. They didn’t have the proper information to understand what was going on in their environment. This caused them to write more logs in the vain hope that the more logs they wrote, the deeper view they’d get. 

However, that process doesn’t take into consideration a few key challenges and difficulties:

1. Developers rarely know what they will need – so they write everything. They only use some of their logs when an issue occurs, meaning they end up writing a lot of data that is rarely used. Yet, they need to write them because they don’t know if they will need it in the future. That is what we call Logging FOMO – the fear of missing out on logs and the information they’ll provide.

2. They can’t always get what they want – we’re talking about finding the relevant logs. Often, developers find themselves hunting for that veritable needle in a haystack. Finding logs is challenging, depending on their structure and content. Structure-wise, creating the same one for every log line is difficult. Parsing them to look the same is a significant pain, and searching in unstructured data is even more so. Content-wise, even if the log exists and the developer is able to find it, it doesn’t always contain the information they need. 

3. Astronomic costs. That’s the whole reason you’re reading this article, right? Because you know you’re paying through the roof for your log storage but have no idea how to reduce it. Let’s explore this a bit further.

Logging Costs

Let’s call a spade a spade. Logs are the most expensive telemetry in the observability world. The costs are hard to control, as every developer can add a log line that has the ability to send your logging bill through the roof. Organizations can pay millions of dollars annually for log storage/usage. And to what end?

That would be due to the need for immediate data. Log storage is divided into two parts – hot storage and cold storage. Hot storage is the more expensive option of the two, and cold storage is only a tenth of the cost. As a quick refresher, hot storage is fast, easy-to-access data storage. On the other hand, cold storage is archival data that’s rarely accessed and usually consists of security and compliance data. Due to developers’ need for immediate data when an issue arises, most of their logs go into hot storage to be easily accessible when needed. However, the costs that hot storage generates are considerable. 

Reducing these costs is difficult. Not only does all the data that is sent need to be analyzed, but hard decisions need to be made regarding which logs can be removed, pushback from the developers needs to be handled, and to top it all off, all of this decision-making and execution is time-consuming.

If you yourself are paying through the roof for those log storage costs, take a moment to consider. How many log lines do you write on an hourly basis? How many of those need to go into hot storage to be easily accessed? And how often do you actually need those logs? Insane, right?

Step into the future: the 4th pillar of Observability

The new method of troubleshooting and production debugging is a game-changer. Get rid of the old, inefficient methods that are tying organizations down, wasting time, and making them bleed dollar bills left and right. Instead, look into the future with the tool that will get developers everything they need throughout the development cycle in one click.

Let’s explain. What we’re talking about is a tool like Rookout that allows developers to access the data they need, in real-time, wherever, whenever. More specifically, we’re talking about Snapshots. 

Snapshots offer comprehensive information that encompasses all the necessary details. Variables are recorded in their entirety, preserving their type information and exact representation. Objects are captured by individual attributes, and collections are all appropriately enumerated, alongside the stack trace and other global variables that are easily accessible.

As you can see, Snapshots give developers so much more information than logs do. By capturing the most relevant application state, they give developers a detailed, clear, high-fidelity image of what’s happening. Truly, snapshots are worth a thousand log lines. 

Rookout and its Snapshots are there for developers throughout the whole development life cycle. It doesn’t matter if they’re developing in a remote environment, running tests through their CI pipeline, monitoring a deployment, or handling a production issue. With snapshots, they get the information they need, in the right place, at the right time. 

There’s no more need to store logs for the convenience of hopefully receiving the necessary data. Now, get the exact data needed immediately when it’s needed. Do yourself a favor and see how your bills shrink dramatically by keeping only your ERROR logs and doing away with the rest, while, of course, keeping the minimum needed for compliance and security in cold storage. Throughout the software development lifecycle – and especially when you need to resolve an issue immediately – you have Rookout to give you a hand. Logs shouldn’t be your old tried-and-somewhat-trusted fallback solution anymore. Now, whenever you need to get a glimpse ‘under the hood’ of your software, no matter the place or time – Rookout is there for you.

Let’s zoom in to see:

TL;DR

Don’t be stuck in the old ways. Step into the future and save yourself tons of money along the way.

It’s time to save development time and frustration. Let’s look to the future and the new wave of tools that allow real-time data, anytime, anywhere. 
Want to hear more? Check out Irit’s interview, where she goes more in-depth on this exact topic.