Security logging architecture

Why design logging deliberately

Logs are the evidence for detecting attacks, investigating incidents and proving compliance. Without a plan, organisations either miss the logs they need or drown in expensive noise. A logging architecture defines sources, transport, format, storage, access and retention as one system, aligned with detection use cases. NIST SP 800-92 gives guidance on log management.

Choosing sources

Start with sources that answer the most important investigation questions: who signed in, what they did with privileges, what ran on endpoints, what left the network, and what changed in cloud control planes. Identity provider logs, EDR telemetry, DNS and firewall logs, and cloud audit trails such as AWS CloudTrail or Azure Activity Log are usually top priority. Add application logs that record security-relevant business events, such as payment changes or bulk exports.

Collection, format and time

• Transport: use agents or syslog over TLS, with buffering so logs are not lost if the collector is down. Plain UDP syslog can silently drop messages.
• Normalisation: parse logs into common fields, such as user, source IP and action, using a schema like the Elastic Common Schema or OCSF (Open Cybersecurity Schema Framework), so rules work across sources.
• Enrichment: add context such as asset owner, geolocation and threat intelligence.
• Time: synchronise every source with NTP and store timestamps in UTC. Incorrect clocks make timelines useless (see syslog and NTP).

An rsyslog client forwarding to a central collector over TLS might use:

global(DefaultNetstreamDriver="gtls"
       DefaultNetstreamDriverCAFile="/etc/ssl/certs/log-ca.pem")
action(type="omfwd" target="logs.example.com" port="6514" protocol="tcp"
       StreamDriver="gtls" StreamDriverMode="1" StreamDriverAuthMode="x509/name")

Storage, retention and protection

Set retention from legal, regulatory and investigation needs, and remember that intrusions are often found long after they start. Protect logs as evidence:

1. Send logs off the source host quickly so an attacker cannot delete them locally.
2. Store them in a separate account or tenant with write-once or immutable storage.
3. Restrict who can read and delete logs; log access to the logs themselves.
4. Alert when a source goes silent, since stopping logging is a common attacker step.
5. Avoid logging secrets, passwords or full card numbers; mask personal data where possible.

Feed the central store into your SIEM and build detections from known attacker behaviour.

Tip: start from detection use cases ("detect a new admin account created outside change windows"), then make sure the logs to support each one are collected. This keeps volume, and cost, under control.