Difference between revisions of "Airflow scheduler"
Jump to navigation
Jump to search
| (3 intermediate revisions by the same user not shown) | |||
| Line 15: | Line 15: | ||
* <code>[[airflow webserver]]</code> | * <code>[[airflow webserver]]</code> | ||
* <code>[[airflow dag-processor]]</code> | * <code>[[airflow dag-processor]]</code> | ||
| + | * [[Airflow workers]] | ||
https://airflow.apache.org/docs/apache-airflow/stable/concepts/dagfile-processing.html | https://airflow.apache.org/docs/apache-airflow/stable/concepts/dagfile-processing.html | ||
| − | The DagFileProcessorManager runs user codes. As a result, you can decide to run it as a standalone process in a different host than the scheduler process. If you decide to run it as a standalone process, you need to set this configuration: AIRFLOW__SCHEDULER__STANDALONE_DAG_PROCESSOR=True and run the airflow dag-processor CLI command, otherwise, starting the scheduler process (airflow scheduler) also starts the DagFileProcessorManager | + | The DagFileProcessorManager runs user codes. As a result, you can decide to run it as a standalone process in a different host than the scheduler process. If you decide to run it as a standalone process, you need to set this configuration: [[AIRFLOW__SCHEDULER__STANDALONE_DAG_PROCESSOR]]=True and run the airflow dag-processor CLI command, otherwise, starting the scheduler process (airflow scheduler) also starts the DagFileProcessorManager |
== See also == | == See also == | ||
| + | * {{airflow dags}} | ||
* {{Airflow Scheduler}} | * {{Airflow Scheduler}} | ||
* {{airflow cmd}} | * {{airflow cmd}} | ||
| − | |||
[[Category:Airflow]] | [[Category:Airflow]] | ||
Latest revision as of 17:25, 18 January 2023
Starting the scheduler
Related[edit]
https://airflow.apache.org/docs/apache-airflow/stable/concepts/dagfile-processing.html The DagFileProcessorManager runs user codes. As a result, you can decide to run it as a standalone process in a different host than the scheduler process. If you decide to run it as a standalone process, you need to set this configuration: AIRFLOW__SCHEDULER__STANDALONE_DAG_PROCESSOR=True and run the airflow dag-processor CLI command, otherwise, starting the scheduler process (airflow scheduler) also starts the DagFileProcessorManager
See also[edit]
airflow dags [ show | unpause | list | list-jobs | state | backfill ]- Airflow Scheduler,
airflow scheduler, airflow scheduler --daemon, --help, airflow-scheduler airflow [ info | celery | config | db upgrad | config list | db init | scheduler | webserver | users | dags | kubernetes | pools | roles | tasks | variables | cheat-sheet | kereberos | plugins | rotate-fernet-key | sync-perm ] airflow --help
Advertising:
