Redwood RunMyJobs → Apache Airflow

Redwood RunMyJobs (now Redwood Workload Automation) is a cloud-native enterprise scheduler with a process-chain model. Its process definitions, chains, and schedule definitions are well-structured and amenable to automated extraction. BatchFoundry maps RunMyJobs process chains to Airflow DAGs, handling parameter passing, calendar definitions, and cross-chain dependencies.

Concept Mapping

The Hard Parts We Handle

• RunMyJobs REST API extraction for job inventory
• Process chain parameter binding → Airflow templating
• Cross-chain dependencies → ExternalTaskSensor
• RunMyJobs calendar and exception definitions
• Conditional branching in chains → BranchPythonOperator
• Queue and agent mapping to Airflow workers

RunMyJobs process chain to DAG

/* RunMyJobs Process Chain (simplified JSON) */
{
  "chain": "DAILY_ETL",
  "schedule": "0 1 * * MON-FRI",
  "steps": [
    { "name": "EXTRACT", "process": "etl.Extract" },
    { "name": "LOAD",    "process": "etl.Load",
      "dependsOn": ["EXTRACT"], "condition": "SUCCESS" }
  ]
}

# Apache Airflow DAG
from airflow.decorators import dag
from airflow.operators.bash import BashOperator
from pendulum import datetime

@dag(schedule="0 1 * * 1-5", start_date=datetime(2025,1,1), catchup=False)
def daily_etl():
    extract = BashOperator(task_id="extract", bash_command="/opt/etl/extract.sh")
    load = BashOperator(task_id="load", bash_command="/opt/etl/load.sh")
    extract >> load

daily_etl()

Related Migrations