A cronjob framework on GitHub Actions: turning 150-line workflows into 10-line specs

Every small platform team I’ve worked with has the same drawer of “things that need to run on a schedule”. A nightly Postgres backup. A weekly cleanup of stale ECS task definitions. An hourly check that pulls billing data from a SaaS API and pushes it as Prometheus metrics. A handful of one-off scripts somebody wrote during an incident and never quite removed.

These jobs share three properties. They’re small. They’re operationally identical (run in a container, pull some secrets, talk to AWS, exit cleanly). They’re invisible until they break. The first time one of them quietly stops running for a week, you stop trusting the others.

The natural place to put them is GitHub Actions, which already runs your CI/CD and which has cron triggers built in. The unnatural part is that a correct cronjob workflow on GitHub Actions is about 150 lines of YAML — secrets retrieval, AWS auth, Docker build, ECR push, role assumption, container run, log forwarding, success/failure metrics. Hand-writing N of them for N jobs is how you end up with N subtly different broken workflows.

This article is about the templating layer that flips that ratio: each new job is a 10-15 line spec file, and a build script generates the underlying workflow from it. The framework is small (a few hundred lines of Python and Jinja templates) but the leverage is significant.

The shape of a spec

Adding a new cronjob means adding one file under specfiles/:

---
name: prod_pg_dump_to_s3
repository: company/platform-tools
dockerfile: dockerfiles/postgres_backup.Dockerfile
crontab: "37 2 * * *"
github_runner: high-priority
docker_aws_role: backup_postgres_role
secrets:
  - name: POSTGRES
    source: prod/postgres/admin
  - name: S3_BUCKET
    source: prod/pg_dump/bucket_name
template: docker

That spec produces a complete .github/workflows/prod_pg_dump_to_s3.yml and a corresponding IAM policy fragment. The author of the new cronjob never touches the workflow YAML directly. They write a Dockerfile (which they’d have to do anyway), they write the spec, they commit, the framework does the rest.

A detail that matters for understanding the rest of the article: the Dockerfile lives in the cronjob author’s own repository, not in this one. The spec’s repository field names that source repo, and the generated workflow checks it out at runtime via a bot token, builds the image, pushes it to ECR (one ECR repository per cronjob, with a lifecycle policy that keeps only the last three images), and runs the resulting container. The runtime also compares the source repo’s last commit date against the most recent image tag and skips the rebuild when the source hasn’t changed — most cronjobs are code-stable for weeks or months between actual releases, and skipping the build there is a real saving.

The template field selects between two flavours: docker for containerised jobs, bash for inline shell scripts. Most jobs use docker because most of them want a real Python or Go runtime and access to a curated set of libraries. The bash template exists for the small percentage of jobs that genuinely don’t need a container — a one-line aws s3 cp ..., a quick API ping.

Decision: code-generated workflows, with the directory as state

The build script (build_workflows.py) walks specfiles/, renders each spec through a Jinja template, and writes the result to .github/workflows/. Critically, it also deletes any cronjob workflow file that doesn’t have a matching spec — but only ones it generated, never hand-authored workflows like ci.yml or release.yml that live in the same directory. The marker is a header comment that the renderer always emits and the reaper always checks for.

GENERATED_HEADER = "# generated by build_workflows.py — do not edit by hand\n"

def _is_generated(path: pathlib.Path) -> bool:
    try:
        return path.read_text(encoding="utf-8").startswith(GENERATED_HEADER)
    except OSError:
        return False

def regenerate_workflows() -> None:
    spec_files = sorted(SPECS_DIR.glob("*.yml"))
    expected = {spec.stem + ".yml" for spec in spec_files}

    for spec_path in spec_files:
        spec = yaml.safe_load(spec_path.read_text())
        if spec["name"] != spec_path.stem:
            raise ValueError(
                f"spec name {spec['name']!r} must match filename {spec_path.stem!r}"
            )
        template = ENV.get_template(f"run_{spec['template']}.yml.jj2")
        output = GENERATED_HEADER + template.render(**spec)
        (WORKFLOWS_DIR / f"{spec['name']}.yml").write_text(output)

    # Reap orphan generated files only — never touch hand-authored workflows.
    for existing in WORKFLOWS_DIR.glob("*.yml"):
        if existing.name in expected:
            continue
        if _is_generated(existing):
            existing.unlink()

This is the bit that matters most for long-term hygiene. The first time a cronjob gets renamed or deleted, somebody forgets to remove the old workflow file and you end up with a ghost job that runs forever, charging compute against an account whose owner left two years ago. With the build script as the source of truth and the generated-marker as the reaper’s gate, a deleted spec deletes its workflow and nothing else.

The build script runs in CI on every PR. If a contributor edits a generated workflow YAML directly, the next CI run regenerates it from the spec and the change is lost. That’s the desired behaviour: the cronjob workflows are derived artefacts; hand-authored ones live alongside them, untouched.

Decision: secrets passed by name, never written to disk

Secrets handling is the part of the workflow that’s easiest to get wrong in a way you don’t notice. AWS Secrets Manager values come back as JSON; the GitHub Action that fetches them expands the JSON into multiple environment variables, with names like POSTGRES_HOST, POSTGRES_USER, POSTGRES_PASSWORD derived from the JSON keys. The number of resulting env vars is unknown until runtime — but the spec does declare which secrets the job asked for, so the framework knows by name what to pass through.

The temptation is to iterate os.environ with a denylist for “things that look sensitive” and write the rest to a tempfile. Don’t. The runner’s environment contains GITHUB_TOKEN, ACTIONS_ID_TOKEN_REQUEST_TOKEN, ACTIONS_RUNTIME_TOKEN, and a long list of GitHub-injected credentials that no cronjob has any business seeing. A denylist is open-by-default; an allowlist driven by the spec is closed-by-default. The framework uses the allowlist, and never materialises a value to disk:

def docker_env_args(declared_secrets: list[str]) -> list[str]:
    """Emit `--env NAME` for each declared secret. Docker resolves the value
    from the runner's environment at exec time. We never iterate os.environ,
    never write secret values to a file, and never quote values into a shell."""
    args = []
    for var_name in declared_secrets:
        if var_name not in os.environ:
            raise RuntimeError(
                f"declared secret {var_name!r} missing from runner env"
            )
        args.extend(["--env", var_name])
    return args

The job then runs:

docker run "${ENV_ARGS[@]}" --rm "$IMAGE"

docker run --env NAME (no =value) tells Docker to look up the value in its own environment and forward it to the container — no tempfile, no --env-file parsing rules to get wrong, no shell quoting. The author of the cronjob declares which secrets the job needs in the spec; everything else stays out of the container.

Decision: every cronjob emits the same three metrics

A cronjob whose only signal is “it ran without erroring” is a cronjob you don’t trust. The minimum useful instrumentation is did it run, did it finish, when’s the next one. Every workflow generated by the framework emits three InfluxDB line-protocol records on completion, posted to Grafana’s InfluxDB-compatible write endpoint — picked because line protocol is the easiest format to construct from a shell template (one printf per line, no JSON encoding, no auth-header gymnastics):

heartbeats,job=prod_pg_dump_to_s3,state=success cronjob_status=1i  <ns_timestamp>
heartbeats,job=prod_pg_dump_to_s3,state=ended   cronjob_status=0i  <ns_timestamp>
heartbeats,job=prod_pg_dump_to_s3              cronjob_next_run=<unix_ts>i  <ns_timestamp>

The next_run is computed from the cron expression at the moment the job exits, and emitted alongside the heartbeat so the alerting layer doesn’t need to parse cron itself. A Grafana dashboard turns this into a “last seen” view per job; an alert fires when now - last_seen exceeds next_run - last_seen by more than 50%. If a job is supposed to run hourly and we haven’t heard from it in ninety minutes, somebody gets paged.

The crucial property: the metrics emit is in the workflow template, not in the job’s own code. An author cannot accidentally ship a cronjob that doesn’t emit them. If the job exits non-zero, the success line is missing; if the job hangs, the ended line is missing; either way the alert fires.

Decision: IAM policies live next to the cronjobs that need them

Most cronjobs need bespoke AWS permissions. The naive path is one giant policy attached to the runner role, which grows monotonically and becomes impossible to audit. The framework instead uses a Pulumi convention: each cronjob can declare a _policy.py file under its directory, which exports an IAM policy:

# cronjobs/cw_metrics_to_prometheus/_policy.py
from pulumi_aws import iam

exported_cronjob_policy = iam.Policy(
    "CronjobReadCloudwatchMetrics",
    policy={
        "Version": "2012-10-17",
        "Statement": [{
            "Effect": "Allow",
            "Action": [
                "cloudwatch:GetMetricStatistics",
                "cloudwatch:GetMetricData",
                "cloudwatch:ListMetrics",
            ],
            "Resource": "*",
        }],
    },
)

The Pulumi orchestrator imports each _policy.py module by path and reads the exported_cronjob_policy symbol — explicit, not metaprogramming. A reviewer reading the policy file alongside the spec sees exactly what permissions the new cronjob is asking for, in the same PR. The policy is scoped to the cronjob, not to the platform.

What this doesn’t give us today is one IAM role per cronjob. The exported policies are aggregated and attached to a single deploy role, so at runtime a cronjob technically holds the union of every cronjob’s permissions. That’s the obvious next refactor — emit one role per spec, with the OIDC trust policy scoped to that workflow’s name — and the spec already carries enough information to drive it. The same refactor would also let the runner assume an instance role for AWS at the workflow level, which would let us retire the static AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY secrets the workflows currently authenticate with. Until that ships, the discipline is in the per-cronjob policy file: a reviewer sees what each job asks for, even if the runtime can’t yet enforce that boundary.

What we considered instead

The obvious 2026 alternative is GitHub’s workflow_call (reusable workflows): write one shared workflow with all the runtime logic, then have each cronjob be a tiny caller workflow that sets the schedule and the inputs. Same author surface area, no code generation, no reaper, no marker comments, no bot token needed to write back into .github/workflows/. Composite actions cover a similar piece of the same idea for sub-job logic, and the framework already uses them for the cron parser and the Honeybadger notifier — so the alternatives weren’t unfamiliar territory.

We didn’t take that path because the spec file isn’t only the input to the workflow — it’s also the canonical registry that the Pulumi side walks. pulumi/_<spec>.py is paired to specfiles/<spec>.yml by filename, and the Pulumi orchestrator imports each _*.py it finds. Removing a cronjob is one PR that deletes both files; with workflow_call we’d still need a registry that Pulumi could walk, just maintained separately from the workflows themselves. The build script also emits a cronjob_scheduled heartbeat to Grafana at generation time, which the missing-job alert query depends on; that signal would have to live somewhere else if the workflows weren’t generated.

The cost of the choice, paid honestly: the build-and-commit-back loop needs a bot token with the workflows permission because the default GITHUB_TOKEN isn’t allowed to update workflow files. The first day of the framework’s git history is mostly fighting that loop, and the README still calls the bot-token requirement a “nasty trick”. If we were starting today and the registry argument didn’t apply — if Pulumi weren’t in the picture, or if the build-time scheduled-status emit weren’t load-bearing — workflow_call would be the answer.

The shape of the framework

The pattern — spec files plus a generator plus a curated runtime template — appears in dbt’s project structure, in Helm chart templates, and in Kubernetes Operators: a declarative author-facing surface, a generated implementation surface, and a single curated runtime. The author writes intent; the platform writes execution.

The framework is about a thousand lines of Python and templates altogether, and the leverage isn’t in the code volume but in the discipline it enforces — every cronjob, regardless of who wrote it, runs the same way, emits the same metrics, fails the same way, and is observable from the same dashboard. The author doesn’t have to remember any of that and only has to write the spec, which is the whole reason the author-facing API was the first thing designed and the last thing changed.