SIEM and log management: collection, correlation and retention
In short: A SIEM gathers logs from across the organisation into one searchable place and runs rules that turn suspicious patterns into alerts. Its value depends on choosing the right log sources, keeping field names consistent and writing use cases tied to real risks.
What a SIEM does
A Security Information and Event Management (SIEM) platform is the central log brain of a Security Operations Centre (SOC). Every firewall, server, laptop and cloud service writes its own logs in its own format. On their own they are hard to search and easy to ignore. A SIEM pulls them together, makes them comparable, and watches for patterns that suggest an attack.
It has two jobs that are easy to confuse. Log management means collecting and storing logs reliably so you can search them later, for an investigation or an audit. Security analytics means running rules and models over those logs in near real time to raise alerts. Common platforms include Splunk, Microsoft Sentinel, Elastic Security and the open-source Wazuh.
From raw log to useful event
- Collection: agents on hosts, syslog from network devices, and API connectors for cloud and SaaS services.
- Parsing: breaking a line of text into fields such as time, user, source IP and action.
- Normalisation: mapping different names for the same thing (
src,SourceIp,client_addr) to one schema so one rule works across vendors. - Enrichment: adding context, such as the user's department, the asset's owner, or whether an IP address appears on a threat feed.
- Correlation: linking events. One failed login is noise; forty failures followed by a success from a new country is a story.
Time matters more than people expect. If devices are not synchronised with the Network Time Protocol (NTP), events appear out of order and correlation breaks.
Use cases and correlation rules
A use case is a written statement of what you want to detect, why, which logs you need and what the analyst should do. Start with your biggest risks rather than every rule the vendor ships. A simple example is a password-spraying or brute-force check. In Microsoft Sentinel, written in Kusto Query Language (KQL), it could look like this:
SecurityEvent
| where EventID == 4625
| summarize Failures = count() by Account, IpAddress, bin(TimeGenerated, 10m)
| where Failures > 20Good starter use cases include impossible travel, a new administrator account, audit logs being cleared, logins from disabled accounts, and large data transfers out of hours.
Retention, cost and common mistakes
| Decision | Things to weigh |
|---|---|
| Hot (searchable) retention | Fast search for recent investigations; usually the most expensive tier |
| Archive retention | Cheaper storage for older logs needed by law, contracts or long investigations |
| What to ingest | Many SIEMs charge by volume, so drop verbose debug logs and keep security-relevant events |
| Integrity | Restrict who can delete logs and alert if collection from a source stops |
Common mistakes: switching on every log source with no plan, keeping hundreds of noisy rules nobody tunes, failing to notice when a source silently stops sending, and not documenting what each rule is for.
Tip: build a free practice SIEM with Wazuh in your own virtual lab; see the Wazuh SIEM lab.
Practice
Why does normalisation matter in a SIEM?
Name two things a correlation rule might link to detect a compromised account.
A firewall stops sending logs and no one notices for a month. What control was missing?
Key takeaways
- A SIEM combines log management with real-time security analytics.
- Logs are collected, parsed, normalised and enriched before correlation.
- Write use cases from real risks, with clear analyst actions.
- Plan retention tiers and ingest volume to control cost.
- Monitor the SIEM itself: time sync, source health and rule quality.
Original content written by Mohamed Fayas for FM TecHub. © 2026 FM TecHub. You're welcome to share a link to this page; please don't copy the text elsewhere without permission.