Cron → Apache Airflow

Most enterprises have hundreds — sometimes thousands — of cron jobs scattered across servers, VMs, and containers. These jobs have no central visibility, no dependency management, no alerting, and no retry logic. BatchFoundry consolidates crontab-based workloads into Airflow DAGs, adding proper dependency chains, monitoring, alerting, and retry policies. We start with a full crontab inventory across your estate before writing a single DAG.

Concept Mapping

The Hard Parts We Handle

• Discovering all crontabs across server estate
• Inferring implicit dependencies between separate cron jobs
• Translating non-standard cron expressions (@reboot, @yearly)
• Environment variable and PATH differences between cron and Airflow worker
• Consolidating jobs that share output files as implicit triggers

Crontab entry to DAG

# /etc/crontab entries

# Extract nightly at 01:00
0 1 * * 1-5 etluser /opt/etl/extract.sh >> /var/log/extract.log

# Load at 02:00 (assumes extract done)
0 2 * * 1-5 etluser /opt/etl/load.sh >> /var/log/load.log

# Apache Airflow DAG
from airflow.decorators import dag
from airflow.operators.bash import BashOperator
from pendulum import datetime

@dag(
    schedule="0 1 * * 1-5",
    start_date=datetime(2025,1,1),
    catchup=False,
    default_args={"retries": 2},
)
def nightly_etl():
    extract = BashOperator(task_id="extract", bash_command="/opt/etl/extract.sh")
    load = BashOperator(task_id="load", bash_command="/opt/etl/load.sh")
    extract >> load   # explicit dependency

nightly_etl()

Related Migrations