Lexicon
Log management
Log management is collecting, processing, storing, and searching log data at scale. How the pipeline works and why log costs dominate observability bills.
Definition
Log management is the practice of collecting, processing, storing, and searching log data at scale: every event record your applications and infrastructure emit, made findable when someone needs it. It is the oldest discipline in observability and still the highest volume one.
What it means in observability
Logs are where the detail lives. Metrics summarize and traces localize, but when an engineer needs to know exactly what a service said at 14:03:27, the answer is a log line. That fidelity is also the problem: logs dwarf the other signals in volume, which is why log management is where observability cost pain concentrates and where pricing models based on ingestion and indexing bite hardest. How an organization manages logs, what it keeps, for how long, at what fidelity, is usually a budget decision wearing an engineering costume.
How it works in practice
The pipeline has three stages. Collection gathers logs from files, containers, and platforms, through collectors and forwarders, increasingly as structured OpenTelemetry log records. Processing happens in flight: parsing text into attributes, normalizing severity into consistent log levels, enriching with context, and redacting sensitive values. Storage and search then make the volume usable, with retention policies per environment or team, full text and attribute queries, and pattern grouping that collapses millions of similar lines into a readable handful.
Where it gets hard
Volume growth is relentless and mostly involuntary, since much log output comes from dependencies and platforms rather than your own code. Under cost pressure teams drop DEBUG in production, sample INFO, and shorten retention, and each cut is invisible until an incident lands in the gap. Compliance pulls the other way, demanding some logs be kept for years. The result is a permanent tension between what engineering wants to keep and what the invoice allows.
Where Tsuga fits
Tsuga prices logs flat per GB with retention controls per signal, environment, and team, and processing routes handle parsing, enrichment, and redaction at ingest. Running inside your own cloud account, log volume also stops generating egress costs on its way out the door.
Related terms
- Data residencyData residency is the question of where data physically lives: which country or region stores it, and where it is processed along the way.
- Elastic ObservabilityElastic offers observability built on the Elasticsearch stack: log search at its core, with metrics, traces, and APM layered on, available self managed or as the hosted Elastic Cloud.
- Log levelsA log level is the severity label attached to a log line: DEBUG, INFO, WARN, ERROR, and FATAL in the common scheme.
- ObservabilityObservability is the ability to understand what is happening inside a system from the data it emits: its logs, metrics, and traces.
- Observability pipelineAn observability pipeline is the layer that moves telemetry from where it is produced to where it is stored, transforming it along the way: parsing, enriching, redacting, filtering, sampling, and routing to one or more destinations.
- SamplingSampling is the practice of keeping only a subset of telemetry, usually traces, so that volume and cost stay manageable.
- SplunkSplunk is the long standing leader in log analytics and SIEM, acquired by Cisco in 2024, offering log search through its SPL query language plus an observability suite spanning metrics and traces.
- TelemetryTelemetry is the data software and infrastructure emit about their own behavior.