Broadcom ESP Workload Automation → Apache Airflow

ESP Workload Automation (formerly CA ESP) is a mainframe-native scheduler known for its event-driven processing model and tracking group hierarchy. ESP's job streams and predecessor/successor relationships map well to Airflow DAGs, but its event and trigger model requires careful translation. BatchFoundry handles the full ESP job stream catalog extraction and DAG generation.

Concept Mapping

The Hard Parts We Handle

• ESP event-driven triggers → Airflow Sensors
• Tracking group hierarchy → nested TaskGroups
• SYMBOL variable resolution at job stream level
• Cross-application predecessor chains
• ESP calendar definitions → Airflow timetables
• ABEND/RESTART logic → retry policies

ESP job stream to DAG

/* ESP Job Stream (simplified) */
APPL DAILY-ETL SCHD DAILY
  JDEFN EXTRACT
    JCL  PROD.JCL(EXTRACTP)
  JDEFN LOAD
    JCL  PROD.JCL(LOADP)
    PRED EXTRACT

# Apache Airflow DAG
from airflow.decorators import dag
from airflow.operators.bash import BashOperator
from pendulum import datetime

@dag(schedule="@daily", start_date=datetime(2025,1,1), catchup=False)
def daily_etl():
    extract = BashOperator(task_id="extract", bash_command="/opt/replatformed/extractp.sh")
    load = BashOperator(task_id="load", bash_command="/opt/replatformed/loadp.sh")
    extract >> load

daily_etl()

Related Migrations