Text processing on the command line: grep, awk, sed, cut and sort
In short: Most network and security data arrives as text: logs, CSV exports and show command output. A handful of tools, grep, cut, awk, sed, sort and uniq, chained with pipes, can filter, extract and count that data in seconds.
Small tools, chained together
The Unix approach is to give each tool one job and connect them with the pipe symbol |. The output of one command becomes the input of the next. Once you are comfortable with six tools you can answer questions like "which source addresses failed SSH logins most often today?" without opening a spreadsheet.
grep and cut
grep keeps lines that match a pattern. Useful options are -i (ignore case), -v (invert), -c (count), -E (extended regular expressions) and -r (search folders).
grep -i "err" /var/log/syslog # any line with ERR, error, Err...
grep -v "^#" /etc/ssh/sshd_config | grep -v "^$" # config without comments or blanks
grep -E "%LINK-3-UPDOWN|%LINEPROTO-5" switch.log # Cisco interface flaps
cut -d',' -f2,5 devices.csv # columns 2 and 5 of a CSVcut is quick but only works with a single fixed delimiter. When columns are separated by a varying number of spaces, as in most show commands, use awk instead.
awk: fields and simple logic
awk splits each line into fields ($1, $2 and so on) on whitespace by default and lets you add conditions and arithmetic.
# Interfaces that are down from 'show ip interface brief' saved to a file
awk '$5 == "down" {print $1, $2}' show_ip_int_brief.txt
# Total bytes per source IP from a CSV export (field 3 = src, field 7 = bytes)
awk -F',' 'NR > 1 {sum[$3] += $7} END {for (ip in sum) print sum[ip], ip}' flows.csv | sort -rn | headNR > 1 skips the header row. The END block runs once after all lines are read, which is where totals are printed.
sed, sort and uniq
# Mask anything that looks like an SNMP community before sharing a config
sed -E 's/(snmp-server community) [^ ]+/\1 <removed>/' router.cfg
# Top 10 source IPs with failed SSH logins
grep "Failed password" /var/log/auth.log |
grep -oE "from [0-9.]+" | awk '{print $2}' |
sort | uniq -c | sort -rn | head -n 10uniq only removes adjacent duplicates, so always sort before uniq -c. The final sort -rn orders the counts numerically, largest first. Note that sed prints the changed text to the screen; add -i only when you really want to edit the file in place, ideally with a backup suffix such as -i.bak.
Build pipelines one stage at a time. Run the first grep and look at the output, then add the next command and check again. When something goes wrong you will know exactly which stage caused it. Also be careful with regular expressions that are too loose: a pattern such as 10.1.1.1 also matches 10.1.1.10 and 10.1.1.100, and an unescaped dot matches any character. Anchor and escape patterns when precision matters, for example grep -w to match whole words only.
| Question | One-liner idea |
|---|---|
| How many errors today? | grep -c with a date pattern |
| Which users logged in? | grep then awk on the user field, sort -u |
| Tidy a config before review | sed to mask secrets, grep -v to drop comments |
Tip: these skills appear in practical exams and on the job when a SIEM is not available. Practise on your own lab logs; never copy customer logs to personal machines.
Practice
Why must you sort before uniq -c?
When is awk better than cut?
What does grep -v do?
Key takeaways
- Chain small tools with pipes.
- grep filters lines; cut and awk extract fields.
- awk can filter, sum and group with arrays.
- sed edits text; use -i carefully and keep a backup.
- sort before uniq -c, then sort -rn for top counts.
Original content written by Mohamed Fayas for FM TecHub. © 2026 FM TecHub. You're welcome to share a link to this page; please don't copy the text elsewhere without permission.