Python Automation Scripts 2026: CLI Tools, File Processing & Cron

Key Takeaways

• Typer for CLI tools: Type hints become CLI arguments automatically — no argparse boilerplate.
• pathlib over os.path: Path.glob(), .read_text(), .write_text() — cleaner and safer.
• subprocess.run(check=True): Raises on non-zero exit codes — prevents silent failures.
• systemd timers over cron: Better logging, missed-run recovery, and status visibility.

---

Introduction

Direct Answer: How do I build Python automation scripts and CLI tools for Ubuntu servers in 2026?

For CLI tools: pip install typer rich then define a function with type-hinted parameters and decorate it with @app.command() — Typer generates the full CLI. For file processing: use pathlib.Path for filesystem operations (Path.glob('*.log'), .read_text(), .write_text()). For running shell commands: subprocess.run(['cmd', 'arg'], check=True, capture_output=True, text=True) — check=True raises on failure, capture_output=True captures output, text=True decodes bytes to str. For scheduling: create a systemd service and timer rather than a cron job on Ubuntu 24.04 — better logging and missed-run handling.

---

Part 1: CLI Tools with Typer

# backup_tool.py — a complete CLI backup tool
import typer
from pathlib import Path
from datetime import datetime
import subprocess
import shutil

app = typer.Typer(
    name="backup",
    help="Sovereign backup automation tool",
    add_completion=True
)

@app.command()
def create(
    source: Path = typer.Argument(..., help="Directory to back up"),
    dest: Path = typer.Option(Path("/var/backups"), "--dest", "-d", help="Backup destination"),
    compress: bool = typer.Option(True, "--compress/--no-compress", help="Compress with gzip"),
    verbose: bool = typer.Option(False, "--verbose", "-v")
):
    """Create a backup of the SOURCE directory."""
    if not source.exists():
        typer.echo(f"Error: {source} does not exist", err=True)
        raise typer.Exit(1)

    dest.mkdir(parents=True, exist_ok=True)
    timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
    archive_name = f"{source.name}-{timestamp}"
    archive_path = dest / archive_name

    typer.echo(f"Backing up: {source} → {archive_path}")

    if compress:
        # Create tar.gz
        result = subprocess.run(
            ["tar", "czf", str(archive_path) + ".tar.gz", "-C", str(source.parent), source.name],
            check=True, capture_output=True, text=True
        )
        archive_path = Path(str(archive_path) + ".tar.gz")
    else:
        shutil.copytree(source, archive_path)

    size = archive_path.stat().st_size / 1024 / 1024
    typer.echo(f"Done: {archive_path.name} ({size:.1f} MB)")

@app.command()
def list_backups(
    dest: Path = typer.Option(Path("/var/backups"), "--dest", "-d"),
    last: int = typer.Option(10, "--last", "-n", help="Show last N backups")
):
    """List recent backups."""
    backups = sorted(dest.glob("*.tar.gz"), key=lambda p: p.stat().st_mtime, reverse=True)
    for backup in backups[:last]:
        mtime = datetime.fromtimestamp(backup.stat().st_mtime).strftime("%Y-%m-%d %H:%M")
        size = backup.stat().st_size / 1024 / 1024
        typer.echo(f"{mtime}  {size:8.1f} MB  {backup.name}")

if __name__ == "__main__":
    app()

python3 backup_tool.py --help

Expected output:

 Usage: backup_tool.py [OPTIONS] COMMAND [ARGS]...

 Sovereign backup automation tool

╭─ Commands ──────────────────────────────────────────────────────────────╮
│ create         Create a backup of the SOURCE directory.                  │
│ list-backups   List recent backups.                                      │
╰─────────────────────────────────────────────────────────────────────────╯

python3 backup_tool.py create /etc --dest /tmp/test-backups --verbose
python3 backup_tool.py list-backups --dest /tmp/test-backups

---

Part 2: File Processing Pipeline

# log_analyzer.py — process server logs and generate summary report
from pathlib import Path
from collections import Counter, defaultdict
from datetime import datetime
import re

LOG_DIR = Path("/var/log/nginx")
REPORT_DIR = Path("/var/reports")

def parse_nginx_log(line: str) -> dict | None:
    """Parse a single nginx access log line."""
    pattern = r'(\S+) - - \[([^\]]+)\] "(\S+) (\S+) \S+" (\d+) (\d+)'
    match = re.match(pattern, line)
    if not match:
        return None
    ip, timestamp, method, path, status, bytes_sent = match.groups()
    return {
        "ip": ip,
        "method": method,
        "path": path,
        "status": int(status),
        "bytes": int(bytes_sent)
    }

def analyze_logs(log_dir: Path, date: str | None = None) -> dict:
    """Analyze all log files in the directory."""
    stats = {
        "total_requests": 0,
        "status_codes": Counter(),
        "top_paths": Counter(),
        "top_ips": Counter(),
        "error_paths": Counter(),
        "total_bytes": 0
    }

    pattern = f"access.log*" if not date else f"access.log.{date}*"
    log_files = sorted(log_dir.glob(pattern))

    for log_file in log_files:
        encoding = "utf-8"
        try:
            content = log_file.read_text(encoding=encoding, errors="replace")
        except Exception as e:
            print(f"Warning: could not read {log_file}: {e}")
            continue

        for line in content.splitlines():
            entry = parse_nginx_log(line)
            if not entry:
                continue

            stats["total_requests"] += 1
            stats["status_codes"][entry["status"]] += 1
            stats["top_paths"][entry["path"]] += 1
            stats["top_ips"][entry["ip"]] += 1
            stats["total_bytes"] += entry["bytes"]

            if entry["status"] >= 400:
                stats["error_paths"][f"{entry['status']} {entry['path']}"] += 1

    return stats

def generate_report(stats: dict, output_path: Path) -> None:
    """Write a summary report to a markdown file."""
    output_path.parent.mkdir(parents=True, exist_ok=True)
    date = datetime.now().strftime("%Y-%m-%d")

    lines = [
        f"# Nginx Access Log Report — {date}\n",
        f"**Total Requests:** {stats['total_requests']:,}",
        f"**Total Bytes Served:** {stats['total_bytes'] / 1024 / 1024:.1f} MB\n",
        "## Status Code Distribution",
        *[f"- HTTP {code}: {count:,}" for code, count in sorted(stats["status_codes"].items())],
        "\n## Top 10 Paths",
        *[f"- {path}: {count:,}" for path, count in stats["top_paths"].most_common(10)],
        "\n## Top 5 Client IPs",
        *[f"- {ip}: {count:,} requests" for ip, count in stats["top_ips"].most_common(5)],
        "\n## Top 10 Errors",
        *[f"- {path}: {count:,}" for path, count in stats["error_paths"].most_common(10)],
    ]
    output_path.write_text("\n".join(lines))
    print(f"Report saved: {output_path} ({output_path.stat().st_size:,} bytes)")

# Run
stats = analyze_logs(LOG_DIR)
generate_report(stats, REPORT_DIR / "nginx-report.md")

---

Part 3: Server Health Script

# server_health.py — sysadmin monitoring script
import subprocess
from pathlib import Path
from dataclasses import dataclass

@dataclass
class HealthCheck:
    name: str
    ok: bool
    message: str

def run_cmd(cmd: list[str]) -> str:
    """Run a shell command and return stdout. Returns error message on failure."""
    try:
        result = subprocess.run(cmd, check=True, capture_output=True, text=True, timeout=10)
        return result.stdout.strip()
    except subprocess.CalledProcessError as e:
        return f"ERROR: {e.stderr.strip()}"
    except subprocess.TimeoutExpired:
        return "ERROR: timeout"

def check_disk() -> HealthCheck:
    output = run_cmd(["df", "-h", "/"])
    # Parse "Use%" from df output
    for line in output.splitlines()[1:]:
        parts = line.split()
        usage_pct = int(parts[4].rstrip("%"))
        return HealthCheck(
            "Disk Usage",
            ok=usage_pct < 85,
            message=f"{usage_pct}% used on / ({parts[3]} available)"
        )

def check_memory() -> HealthCheck:
    output = run_cmd(["free", "-m"])
    lines = output.splitlines()
    mem = lines[1].split()
    total, used = int(mem[1]), int(mem[2])
    pct = (used / total) * 100
    return HealthCheck("Memory", ok=pct < 90, message=f"{pct:.0f}% used ({used}MB/{total}MB)")

def check_load() -> HealthCheck:
    output = Path("/proc/loadavg").read_text()
    load1 = float(output.split()[0])
    cpu_count = int(run_cmd(["nproc"]))
    normalized = load1 / cpu_count
    return HealthCheck("CPU Load", ok=normalized < 0.8, message=f"1min load: {load1:.2f} ({normalized*100:.0f}% of {cpu_count} cores)")

def check_service(service: str) -> HealthCheck:
    result = subprocess.run(["systemctl", "is-active", service], capture_output=True, text=True)
    active = result.stdout.strip() == "active"
    return HealthCheck(f"Service: {service}", ok=active, message="running" if active else "NOT RUNNING")

checks = [
    check_disk(),
    check_memory(),
    check_load(),
    check_service("nginx"),
    check_service("postgresql"),
]

print("=== SERVER HEALTH REPORT ===")
all_ok = True
for check in checks:
    status = "✓" if check.ok else "✗"
    print(f"  {status} {check.name}: {check.message}")
    if not check.ok:
        all_ok = False

print(f"\nOverall: {'HEALTHY' if all_ok else 'ISSUES DETECTED'}")

Expected output:

=== SERVER HEALTH REPORT ===
  ✓ Disk Usage: 34% used on / (47G available)
  ✓ Memory: 58% used (1831MB/3138MB)
  ✓ CPU Load: 1min load: 0.12 (6% of 2 cores)
  ✓ Service: nginx: running
  ✓ Service: postgresql: running

Overall: HEALTHY

---

Part 4: Schedule with systemd Timer

# Create the service
sudo tee /etc/systemd/system/server-health.service << 'EOF'
[Unit]
Description=Server Health Check
After=network.target

[Service]
Type=oneshot
ExecStart=/usr/bin/python3 /opt/scripts/server_health.py
StandardOutput=journal
StandardError=journal
User=ubuntu
EOF

# Create the timer (run every 5 minutes)
sudo tee /etc/systemd/system/server-health.timer << 'EOF'
[Unit]
Description=Run server health check every 5 minutes

[Timer]
OnBootSec=2min
OnUnitActiveSec=5min
Persistent=true

[Install]
WantedBy=timers.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable --now server-health.timer

# Check results
journalctl -u server-health.service --since "10 minutes ago" --no-pager

---

Conclusion

Python automation on Ubuntu 24.04 is built on four tools: Typer for CLIs, pathlib for file operations, subprocess for shell commands, and systemd timers for scheduling. These patterns appear throughout DevOps automation — backup scripts, log analysis, monitoring, and deployment scripts all use the same primitives.

---

People Also Ask

Should I use Typer or Click for Python CLI tools?

Typer is built on Click and is the better default in 2026 — it uses Python type hints to define arguments automatically, requires less boilerplate, and has better help text generation. Use Click directly if you need features Typer doesn’t expose or if you’re working with an existing Click codebase. For simple scripts with 1–3 arguments, argparse (standard library) is also a valid choice — no installation needed.

---

Part 4: Scheduling with systemd Timers

On Ubuntu 24.04, prefer systemd timers over cron for reliability and observability.

4.1 Create a systemd service

# /etc/systemd/system/python-automation.service
[Unit]
Description=Run Python automation script

[Service]
Type=simple
WorkingDirectory=/opt/automation
ExecStart=/usr/bin/python3 /opt/automation/report_generator.py
StandardOutput=journal
StandardError=journal
User=automation
Group=automation

4.2 Create a systemd timer

# /etc/systemd/system/python-automation.timer
[Unit]
Description=Run Python automation every night at 02:00

[Timer]
OnCalendar=*-*-* 02:00:00
Persistent=true
RandomizedDelaySec=15m

[Install]
WantedBy=timers.target

Reload systemd and enable the timer:

sudo systemctl daemon-reload
sudo systemctl enable --now python-automation.timer
sudo systemctl status python-automation.timer

systemd stores logs in the journal, so you can inspect failures with:

journalctl -u python-automation.service --since today

4.3 Why systemd timers are better than cron

• missed runs are recovered if Persistent=true
• logging is centralized in journalctl
• you can define calendar expressions and random delays
• service units can specify dependencies and environment files

Part 5: Safe File Handling

Robust scripts avoid race conditions, partial writes, and corrupted outputs.

5.1 Atomic writes

Write temporary files, then rename them.

from pathlib import Path
from tempfile import NamedTemporaryFile

def atomic_write(path: Path, data: str):
    with NamedTemporaryFile('w', delete=False, dir=path.parent) as tmp:
        tmp.write(data)
        temp_name = tmp.name
    Path(temp_name).replace(path)

This ensures the final file is either complete or unchanged.

5.2 File locking

If multiple processes may access the same files, use file locks.

from pathlib import Path
import fcntl

with open('/var/lock/myapp.lock', 'w') as lockfile:
    fcntl.flock(lockfile, fcntl.LOCK_EX)
    # perform work safely

This is essential for scripts run from timers or multiple CLI invocations.

5.3 Pathlib best practices

Use Path methods instead of string manipulation:

report_dir = Path('/var/reports')
log_files = report_dir.glob('*.log')

Avoid os.path.join unless you need compatibility with older Python versions.

Part 6: Logging and Error Handling

A good automation script records what it did and why.

6.1 Structured logging with Rich

from rich.console import Console
console = Console()

console.log("Starting automation run")

For more structured logs, use the built-in logging module.

6.2 Exit codes and exceptions

Use raise SystemExit(1) or sys.exit(1) on fatal errors. This makes systemd and CI detect failures.

import sys
try:
    run_task()
except Exception as exc:
    console.log(f"[red]Automation failed: {exc}[/red]")
    sys.exit(1)

6.3 Capturing subprocess output

Use subprocess.run(check=True, capture_output=True, text=True) for shell commands.

result = subprocess.run(
    ["rsync", "-av", "/data/", "/backup/"],
    check=True,
    capture_output=True,
    text=True
)
console.log(result.stdout)

If a command fails, CalledProcessError includes the return code and stderr.

Part 7: CLI Testing and Validation

Test your automation tools like any other application.

7.1 Use pytest to test CLI commands

from typer.testing import CliRunner
from backup_tool import app

runner = CliRunner()

def test_backup_help():
    result = runner.invoke(app, ["--help"])
    assert result.exit_code == 0
    assert "Create a backup" in result.stdout

7.2 Validate inputs

Typer automatically validates types, but you can add custom validators.

from pathlib import Path

def validate_source(path: Path) -> Path:
    if not path.exists():
        raise typer.BadParameter("Path does not exist")
    return path

7.3 Regression tests for automation workflows

Write tests for the workflow logic, not just the CLI surface. Simulate directories with tmp_path and verify output files are created correctly.

Part 8: Packaging and Distribution

For reusable scripts, package them and install in a virtual environment.

8.1 requirements-dev.txt

Keep separate dependency files:

# requirements.txt
rich
typer

# requirements-dev.txt
pytest
pytest-cov
pytest-watch

8.2 Setup with pyproject.toml

[project]
name = "automation-tools"
version = "0.1.0"

[project.scripts]
backup = "backup_tool:app"

Install with pip install -e . for local development.

Part 9: Secure Automation for Sovereign Servers

Keep automation scripts under your control, not in a shared SaaS. Use local git repositories for code and configuration.

9.1 Avoid hard-coded secrets

Read credentials from environment variables or local config files with restrictive permissions.

from dotenv import load_dotenv
load_dotenv('/etc/automation/.env')

9.2 Least-privilege execution

Run automation tasks as a dedicated user. Do not execute scripts as root unless absolutely necessary.

9.3 Audit and review automation scripts

Treat automation scripts as part of your infrastructure. Peer review them, and store them in the same repo as operational runbooks.

Part 10: Example Automation Pipeline

A complete sovereign pipeline might:

1. ingest logs nightly from /var/log
2. analyse and aggregate metrics
3. generate markdown and CSV reports
4. upload reports to a private intranet or internal dashboard
5. rotate old data and archive backups

The same patterns apply whether you build the pipeline with Typer, plain Python, or simple shell wrappers.

A strong automation stack is one you can run offline, inspect locally, and recover easily.

Part 11: Event-Driven Automation and Hooks

Beyond scheduled jobs, Python automation can be event-driven.

11.1 File system event listeners

Use watchdog or built-in polling to trigger scripts when files change.

from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

class LogHandler(FileSystemEventHandler):
    def on_created(self, event):
        print(f"New file: {event.src_path}")

observer = Observer()
observer.schedule(LogHandler(), path="/var/log", recursive=False)
observer.start()

11.2 Webhooks and local HTTP triggers

A lightweight local HTTP endpoint can start automation on demand.

Use FastAPI or Flask for a secure webhook listener and check a shared secret before running the job.

Part 12: Scheduling and Backoff Strategies

For automation that interacts with external systems, implement retries with exponential backoff.

import time

for attempt in range(5):
    try:
        do_remote_work()
        break
    except Exception as exc:
        delay = 2 ** attempt
        time.sleep(delay)
        if attempt == 4:
            raise

This keeps your automation resilient when network resources fluctuate.

Part 13: Testability and Local Developer Workflow

Make it easy for developers to run automation scripts locally.

13.1 Development mode

Use --dry-run and --verbose flags in CLI tools.

@app.command()
def run(dry_run: bool = typer.Option(False, '--dry-run')):
    if dry_run:
        typer.echo("Dry run mode — no changes will be made")

13.2 Local environment files

Keep .env.example in the repo and instruct developers to copy it to .env with safe defaults.

Part 14: Example Advanced Automation Workflows

14.1 Data pipeline orchestration

Build a pipeline that:

1. ingests raw files
2. validates schemas
3. transforms data
4. writes reports or uploads summaries

Use a Python script or a workflow tool such as dagster if the pipeline grows complex.

14.2 Configurable maintenance utilities

Create a CLI tool that can run multiple maintenance tasks with a single entry point:

python manage.py cleanup --log-dir /var/log --days 30
python manage.py backup --dest /var/backups

Part 15: Final Automation Security Checklist

• secrets are not hard-coded
• scripts run as a dedicated user
• outputs are written atomically
• failures are logged with details
• scheduled jobs are managed by systemd timers
• local developers can reproduce the environment
• external dependencies are pinned and audited
• the automation repo includes documentation and runbooks

Part 16: Long-Running Job Management

Some automation tasks run for minutes or hours. Manage them carefully.

16.1 Timeout and heartbeat patterns

Add timeouts to avoid orphaned jobs.

import signal

class TimeoutException(Exception):
    pass

def handler(signum, frame):
    raise TimeoutException()

signal.signal(signal.SIGALRM, handler)
signal.alarm(3600)

Use heartbeats or status files to let supervisors know a job is still alive.

16.2 Graceful shutdown

Handle SIGTERM and SIGINT so the script can clean up before exiting.

import signal

def shutdown(signum, frame):
    print('Shutting down gracefully')
    sys.exit(0)

signal.signal(signal.SIGTERM, shutdown)

Part 17: Dependency Injection and Modular Design

Design automation scripts as reusable modules.

17.1 Separate core logic from CLI

Keep business logic in functions and classes, and use Typer only for the CLI layer.

17.2 Configurable services

Pass database connections, file paths, and network clients as parameters.

def run_report(storage_path: Path, db_client):
    ...

This makes tests easier and your scripts more maintainable.

Part 18: Advanced CLI Patterns

Build modular, user-friendly command suites.

18.1 Subcommands and groups

Use Typer app groups for related tasks.

cli = typer.Typer()
maintenance = typer.Typer()
cli.add_typer(maintenance, name='maintenance')

@maintenance.command()
def cleanup(...):
    ...

18.2 Default commands and aliases

Provide helpful defaults and short aliases for frequent operations.

Part 19: Packaging for Local Execution

Make your automation tools easy to install.

19.1 Editable installs for development

Use pip install -e . to work locally without reinstalling.

19.2 Executable bundles

For isolated hosts, build a single executable with shiv or pyinstaller.

shiv -o automation-app.pyz -e backup_tool:main -r requirements.txt .

This can simplify deployment on systems where Python environments are not standard.

Part 20: Final Automation Operations Checklist

• long-running jobs have timeouts and graceful shutdown
• CLI functions are modular and testable
• systemd logs capture stdout and stderr
• automation scripts are packaged for local host deployment
• secrets are stored securely and not in source control
• scheduling is managed through systemd timers
• file writes are atomic and locked when needed
• local developers can run the same tools with a simple command
• the automation repository includes runbooks and documentation
• the stack is self-contained and does not depend on cloud automation services

Part 21: Local Metrics and Reporting

Capture metrics for automation success and failure.

21.1 Simple Prometheus metrics

Expose a local /metrics endpoint with prometheus_client in Python.

from prometheus_client import start_http_server, Counter
job_runs = Counter('automation_runs_total', 'Number of automation runs')
start_http_server(8000)

21.2 Summary reports

Generate end-of-day summaries that list completed jobs, errors, and elapsed time.

21.3 Alerts and local emails

If a local mail relay is available, send a summary email or a log entry when a job fails.

Part 22: Developer Onboarding and Documentation

Make it easy for new operators to understand the automation stack.

22.1 README and runbook

Include a README.md with commands to install dependencies, run tests, and start the automation service.

22.2 Example command sequences

Document the exact commands to bootstrap the local environment:

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python3 backup_tool.py create /etc --dest /tmp/backups

22.3 Local developer scripts

Provide make test and make run commands so the stack is easy to use.

Part 23: Final Automation Practicality Checklist

• job metrics are captured or exposed locally
• failure reports are sent to a local monitoring endpoint
• developer onboarding docs exist and are current
• command aliases and Makefile targets are available
• automation code is reviewed as part of infrastructure changes
• tooling is self-contained and does not rely on cloud-hosted CI for core operation

Part 24: Health Checks and Operational Visibility

Make automation scripts visible to operators.

24.1 Alive checks

If a script runs as a service, expose a simple health check file or HTTP endpoint that indicates success.

24.2 Logs and rotation

Ensure logs are rotated and archived. Use logrotate or journal retention to keep storage bounded.

24.3 Failure notifications

Send a local notification or write a clear error summary to a status file when automation fails.

Part 25: Maintenance and Version Control

Treat automation code as infrastructure.

25.1 Git-based ops

Store automation scripts in a local git repo with separate branches for maintenance changes. Review changes before merging.

25.2 Dependency audits

Run pip-audit or safety on your automation dependencies periodically.

uv run python -m pip audit

25.3 Pinning and compatibility

Pin dependencies in requirements.txt and test them after each update.

Part 26: Integration with Local Operations

Automation scripts should connect to the rest of the self-hosted stack.

26.1 Configured via environment

Read service endpoints, credentials, and runtime options from environment variables or a secure config file.

26.2 Local dashboard readiness

If you have a local dashboard, emit metrics in a format the dashboard can consume.

26.3 Operator handoff

Document how to restart automation services, inspect logs, and recover from failures. This is the final step in making the stack operational.

Part 27: Final Python Automation Operations Notes

Python automation scripts are not just code; they are part of the operational surface area of your server. Make them observable, self-testing, and easy to run locally. When you treat automation as infrastructure, you build a sovereign system that can be inspected, audited, and maintained independently of hosted CI/CD services.

Part 28: Sustaining Python Automation in Production

Treat your automation scripts as part of your platform. Schedule regular reviews of dependency versions, security checks, and failure modes. This ensures the automation remains reliable, auditable, and aligned with your sovereign operations.

Part 29: Continuous Improvement for Automation

Review your automation scripts quarterly. Remove obsolete tasks, consolidate overlapping commands, and ensure the toolchain still matches your server environment. Continuous improvement keeps automation relevant and prevents drift from becoming technical debt.

Part 30: Practical Review Cycle

Schedule a short review every quarter. Confirm that the automation scripts still match the live server environment, update dependencies safely, and remove workarounds that were only meant for temporary fixes.

Part 31: Lightweight Review Summary

Keep a short review checklist in the repository so future maintainers can verify health, dependencies, and operational assumptions quickly.

Further Reading

• Python 3.12 Getting Started Guide 2026 — virtual environments before installing packages
• Bash Scripting Guide 2026 — shell scripting complement to Python automation
• Cron Jobs and systemd Timers on Ubuntu 24.04 — scheduling reference

Tested on: Ubuntu 24.04 LTS (Hetzner CX22). Python 3.12.3, Typer 0.12.4. Last verified: April 30, 2026.