Parsing device output and logs with Python regular expressions

Why parse text at all

Many devices still answer in human-friendly text. To act on that output, a script needs it as data: which interfaces are down, which neighbours exist, how many errors a port has. A regular expression (regex) is a compact pattern language for finding and capturing parts of text. Python’s built-in re module provides it.

Core regex tokens

Always write patterns as raw strings (r"...") so Python does not treat backslashes as its own escape codes. Use re.search() to find the first match anywhere, re.match() to match only at the start, and re.finditer() to loop over every match. Named groups, written (?P<name>...), let you read results by name with m["name"].

import re

output = """Interface              IP-Address      OK? Method Status                Protocol
GigabitEthernet0/0     192.168.10.1    YES manual up                    up
GigabitEthernet0/1     unassigned      YES unset  administratively down down
Loopback0              10.255.0.1      YES manual up                    up"""

pattern = re.compile(
    r"^(?P<intf>\S+)\s+(?P<ip>\S+)\s+\w+\s+\w+\s+"
    r"(?P<status>up|down|administratively down)\s+(?P<proto>up|down)$",
    re.MULTILINE,
)

interfaces = [m.groupdict() for m in pattern.finditer(output)]
for i in interfaces:
    print(i["intf"], i["ip"], i["status"])

down = [i["intf"] for i in interfaces if i["proto"] != "up"]
print("Down:", down)

re.MULTILINE makes ^ and $ match at the start and end of every line, not only the whole string. The header line does not match, so it is skipped automatically.

Parsing logs

Syslog messages have a fairly regular shape, which makes them good regex targets. This example counts interface state changes per interface from a log file.

import re
from collections import Counter

log_re = re.compile(
    r"%LINEPROTO-5-UPDOWN: Line protocol on Interface (?P<intf>\S+), "
    r"changed state to (?P<state>up|down)"
)

flaps = Counter()
with open("syslog.txt", encoding="utf-8") as f:
    for line in f:
        m = log_re.search(line)
        if m:
            flaps[m["intf"]] += 1

for intf, count in flaps.most_common(5):
    print(intf, count)

Tip: build patterns against real sample output saved in a file, and test them before trusting them. An online tester is fine for learning, but never paste confidential logs into third-party websites.

Pitfalls and better alternatives

• Greedy matching: .* grabs as much as possible. Prefer specific tokens such as \S+, or the lazy form .*?.
• Escape special characters: a literal dot in an address is \.. Better still, capture \S+ and check it with the ipaddress module.
• Fragile patterns: output changes between software versions. Fail loudly when nothing matches instead of silently reporting "no problems".
• Slow patterns: nested repeats such as (a+)+ can take exponential time on some input (ReDoS, regular expression denial of service). Keep patterns simple, especially on input from untrusted sources.

Netmiko can call TextFSM for you with use_textfsm=True. When a device offers structured output, use it: parsing text should be a fallback, not the plan.