Files, APIs & Automation Scripts

Historically, Python developers wrote f = open('file.txt') and stitched paths together with messy string concatenation. In modern Python, pathlib.Path and the with context manager make file operations clean, cross-platform, and leak-proof.

user_id,login_time,status
101,08:30:12,success

Automation scripts constantly ingest comma-separated logs or export structured JSON payloads for monitoring dashboards. The standard library modules csv and json provide fast, zero-dependency tools.

user_id,username,role,active
1001,ada_lovelace,admin,True
1002,alan_turing,engineer,True
1003,grace_hopper,architect,False

import csv
from pathlib import Path

path = Path("users.csv")
with path.open("r", encoding="utf-8") as f:
    reader = csv.DictReader(f)
    for row in reader:
        print(row["username"], row["role"])

import json
from pathlib import Path

payload = {"status": "ok", "count": 42}
Path("output.json").write_text(
    json.dumps(payload, indent=2),
    encoding="utf-8"
)

Real-world automation almost always connects to an external web service: sending Slack notifications, triggering GitHub deployments, or polling internal microservices.

{
  "uptime_pct": 99.98,
  "active_workers": 8,
  "requests_per_sec": 1420
}

response = requests.get(
    "https://api.cloudmetrics.internal/api/v1/metrics",
    headers={"Authorization": f"Bearer {os.getenv('API_KEY')}"},
    timeout=5.0  # Crucial! Prevents indefinite hangs
)
response.raise_for_status()  # Raises HTTPError on 4xx/5xx
data = response.json()

Networks fail, rate limits happen, and APIs experience brief outages. A brittle script crashes and wakes up on-call engineers; a resilient script logs the hiccup, backs off, and succeeds on a subsequent attempt.

Before scheduling an automation script in production, review the four most common pitfalls that cause silent data corruption, credential leaks, or memory exhaustion.

API_KEY = "sk_live_99812491283"  # LEAK RISK!
# Checked into public git repository!

import os
from pathlib import Path

# Reads from environment variable or .env file
API_KEY = os.environ.get("API_KEY")
if not API_KEY:
    raise RuntimeError("API_KEY environment variable is required")

Test your mastery of file pointers, cross-platform path handling, JSON serialization, HTTP status verification, and memory-safe stream iteration. Score 5/5 to complete Chapter 10!

import json
from pathlib import Path

report = {"course": "Python", "complete": True}
path = Path("/tmp/course-report.json")
path.write_text(json.dumps(report))
loaded = json.loads(path.read_text())
print(loaded["course"], loaded["complete"])

1import json
2from pathlib import Path
3 
4report = {"course": "Python", "complete": True}
5path = Path("/tmp/course-report.json")
6path.write_text(json.dumps(report))
7loaded = json.loads(path.read_text())
8print(loaded["course"], loaded["complete"])

Run the code to see its output.

Run and customize the live file parser & JSON reporting pipeline in the Python Playground (code preloaded).

• pathlib.Path: Use object-oriented, cross-platform paths with /, .read_text(), and .write_text().
• Context Managers: Always open files with with path.open() as f: to guarantee closing and flushing.
• Lazy Line Streaming: Never call .read() on giant files; iterate directly with for line in f:.
• csv.DictReader & json: Ingest CSV headers as dictionary keys, and serialize JSON with json.dumps().
• API Reliability: Always set explicit timeout=... and call response.raise_for_status().
• Secret Safety: Pull API credentials from os.getenv('KEY'), never hardcoded in source.