Why keeping security logs got so expensive

Most organisations collect far more security logging than they actually keep. Not because the data is worthless, but because keeping it is too expensive. And that cost isn’t an accident - it’s built into the way most SIEM tools work.

Full-text search on everything has a price

Traditional SIEM tools treat every single log line as unstructured text to be fully indexed, so you can search freely across all of it. It sounds flexible, but it means storage and compute grow faster than the data itself. The more you collect, the more it costs - until the economics force you to discard logs after a few weeks. And just when an investigation needs the history that reaches furthest back, it’s gone.

Most searches look for the same things

In practice, the vast majority of security searches come down to a handful of fields: IP addresses, domains, file hashes, usernames, email addresses. The question is usually simple - “have we seen this before?” - and it deserves a fast, clear answer: no (move on) or maybe (dig deeper). When you know what your logs contain and what you’ll actually use them for, you don’t need to index everything to answer that.

Keep more, pay less

The alternative is to store logs cheaply - on object storage - and index only what you actually search on. Then you can keep years of history and still search through it, at a fraction of the cost. This is the thinking behind Dovre’s Archive: full history for hunting and investigation, without the storage bill spiralling out of control. Sovereignty is partly this, too - being able to afford to keep your own history, and to search it when it counts.