Reading and writing files in Python: text, CSV, JSON and YAML

Why files matter in automation

Scripts rarely work on data typed into them. Device lists live in spreadsheets, settings live in YAML, APIs return JSON, and results end up in reports. Being able to read and write these formats confidently is one of the most useful Python skills for an engineer.

Plain text files

Open files with a with block. It closes the file for you even if an error happens. Mode "r" reads, "w" overwrites, "a" appends. State encoding="utf-8" so the script behaves the same on Windows and Linux.

with open("show_run.txt", encoding="utf-8") as f:
    for line in f:
        if line.startswith("hostname"):
            print(line.strip())

with open("report.txt", "w", encoding="utf-8") as f:
    f.write("Audit complete\n")

Looping over the file object reads one line at a time, which keeps memory use low even for large logs.

CSV: spreadsheets in and out

CSV (comma-separated values) is the common ground between engineers and spreadsheets. csv.DictReader uses the header row as keys, so each row becomes a dictionary.

import csv

with open("devices.csv", newline="", encoding="utf-8") as f:
    devices = list(csv.DictReader(f))   # header: hostname,ip,site

for d in devices:
    print(d["hostname"], d["ip"])

with open("status.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.DictWriter(f, fieldnames=["hostname", "status"])
    writer.writeheader()
    writer.writerow({"hostname": "acc-sw1", "status": "ok"})

Tip: pass newline="" when opening CSV files, otherwise Windows can add blank lines between rows.

JSON and YAML

JSON is what most APIs send and receive. The built-in json module converts between JSON text and Python dicts and lists. YAML (YAML Ain't Markup Language) is easier for people to write and is used by Ansible, Nornir and many inventories. Install it with pip install pyyaml.

import json
import yaml

with open("facts.json", encoding="utf-8") as f:
    facts = json.load(f)

with open("facts_pretty.json", "w", encoding="utf-8") as f:
    json.dump(facts, f, indent=2)

with open("inventory.yaml", encoding="utf-8") as f:
    inventory = yaml.safe_load(f)

for name, data in inventory["devices"].items():
    print(name, data["ip"])

A matching inventory.yaml might look like this:

devices:
  core-sw1:
    ip: 192.168.10.1
    platform: cisco_ios
  acc-sw1:
    ip: 192.168.10.11
    platform: cisco_ios

Safety and common mistakes

• Use yaml.safe_load(), never plain yaml.load() on untrusted files. The full loader can build arbitrary Python objects, which is a code execution risk.
• Do not store passwords in inventory files. Keep secrets in environment variables or a vault, and keep the file in version control without them.
• Opening with "w" wipes the file first. Write to a new name if you are not sure.
• YAML is indentation-sensitive and treats some values specially (for example no may become a boolean in older parsers). Quote values that look ambiguous.