Replace bundled MFT with Apache NiFi — and let Airflow do the orchestration
Most enterprise schedulers ship with their own file-transfer modules. When you move to Airflow, those modules don't come with you. BatchFoundry pairs Airflow with Apache NiFi for an open, auditable, vendor-neutral MFT plane.
The hidden MFT in your scheduler
Most teams underestimate how much file movement is buried inside their scheduler's own MFT module. Common examples include Control-M MFT, Stonebranch UAC MFT, IBM IWS paired with IBM Sterling Connect:Direct, AutoSys with CA File Transfer, OpCon with SMA Connect, and JAMS with built-in SFTP/FTPS tasks. When you migrate to Airflow, those file moves don't disappear — they need a home. Airflow's SFTPOperator is fine for low volume; for serious throughput, partner-managed transfers, retry guarantees, and lineage, you want a dedicated MFT engine.
Why Apache NiFi
- Open source, Apache 2.0 — no vendor lock-in
- Visual flow designer that batch-ops engineers can read without training
- Supports SFTP, FTPS, S3, Azure Blob, GCS, HDFS, JMS, Kafka, HTTP, and 200+ processors
- Built-in provenance for every byte transferred — required for SOX, HIPAA, PCI
- Clusters horizontally; plays well with Airflow via NifiOperator community provider
The reference architecture
1. Airflow DAG triggers a NiFi flow via REST API
2. Apache NiFi handles transfer + transformation + provenance
3. NiFi posts a webhook back to Airflow on completion
4. Airflow DAG advances to the next task
NiFi clusters live alongside Airflow workers in the same Kubernetes namespace or VPC.
What BatchFoundry delivers
- Inventory of every file movement buried in the scheduler's MFT module
- Apache NiFi flow templates per partner and per protocol
- Airflow DAG pattern that triggers and tracks NiFi flows via REST API
- Provenance-export configuration for SOX / HIPAA / PCI compliance
- Parallel-run cutover with byte-level diff verification
What we won't tell you
If you have a single low-volume FTP move, you don't need NiFi — SFTPOperator in Airflow is enough. We will tell you when NiFi is overkill.
Get a free MFT inventory
We'll map every file movement in your current scheduler before recommending NiFi.
Get a free MFT inventory of your current scheduler