Difference between revisions of "Airflow scheduler"
Jump to navigation
Jump to search
| Line 12: | Line 12: | ||
* <code>[[airflow webserver]]</code> | * <code>[[airflow webserver]]</code> | ||
* <code>[[airflow dag-processor]]</code> | * <code>[[airflow dag-processor]]</code> | ||
| + | |||
| + | https://airflow.apache.org/docs/apache-airflow/stable/concepts/dagfile-processing.html | ||
| + | The DagFileProcessorManager runs user codes. As a result, you can decide to run it as a standalone process in a different host than the scheduler process. If you decide to run it as a standalone process, you need to set this configuration: AIRFLOW__SCHEDULER__STANDALONE_DAG_PROCESSOR=True and run the airflow dag-processor CLI command, otherwise, starting the scheduler process (airflow scheduler) also starts the DagFileProcessorManager | ||
== See also == | == See also == | ||
Revision as of 15:06, 16 November 2022
Related
https://airflow.apache.org/docs/apache-airflow/stable/concepts/dagfile-processing.html The DagFileProcessorManager runs user codes. As a result, you can decide to run it as a standalone process in a different host than the scheduler process. If you decide to run it as a standalone process, you need to set this configuration: AIRFLOW__SCHEDULER__STANDALONE_DAG_PROCESSOR=True and run the airflow dag-processor CLI command, otherwise, starting the scheduler process (airflow scheduler) also starts the DagFileProcessorManager
See also
- Airflow Scheduler,
airflow scheduler, airflow scheduler --daemon, --help, airflow-scheduler airflow [ info | celery | config | db upgrad | config list | db init | scheduler | webserver | users | dags | kubernetes | pools | roles | tasks | variables | cheat-sheet | kereberos | plugins | rotate-fernet-key | sync-perm ] airflow --help- Apache Airflow, Installation, configuration
airflow.cfg, monitoring, Amazon Managed Workflows for Apache Airflow (MWAA),airflow, Airflow Scheduler, providers, Workflows, Airflow executors, Airflow versions, Datadog integration
Advertising:
