Difference between revisions of "Airflow scheduler"

From wikieduonline
Jump to navigation Jump to search
 
(3 intermediate revisions by the same user not shown)
Line 15: Line 15:
 
* <code>[[airflow webserver]]</code>
 
* <code>[[airflow webserver]]</code>
 
* <code>[[airflow dag-processor]]</code>
 
* <code>[[airflow dag-processor]]</code>
 +
* [[Airflow workers]]
  
 
https://airflow.apache.org/docs/apache-airflow/stable/concepts/dagfile-processing.html
 
https://airflow.apache.org/docs/apache-airflow/stable/concepts/dagfile-processing.html
The DagFileProcessorManager runs user codes. As a result, you can decide to run it as a standalone process in a different host than the scheduler process. If you decide to run it as a standalone process, you need to set this configuration: AIRFLOW__SCHEDULER__STANDALONE_DAG_PROCESSOR=True and run the airflow dag-processor CLI command, otherwise, starting the scheduler process (airflow scheduler) also starts the DagFileProcessorManager
+
The DagFileProcessorManager runs user codes. As a result, you can decide to run it as a standalone process in a different host than the scheduler process. If you decide to run it as a standalone process, you need to set this configuration: [[AIRFLOW__SCHEDULER__STANDALONE_DAG_PROCESSOR]]=True and run the airflow dag-processor CLI command, otherwise, starting the scheduler process (airflow scheduler) also starts the DagFileProcessorManager
  
 
== See also ==
 
== See also ==
 +
* {{airflow dags}}
 
* {{Airflow Scheduler}}
 
* {{Airflow Scheduler}}
 
* {{airflow cmd}}
 
* {{airflow cmd}}
* {{airflow}}
 
  
 
[[Category:Airflow]]
 
[[Category:Airflow]]

Latest revision as of 17:25, 18 January 2023



Starting the scheduler

Related[edit]

https://airflow.apache.org/docs/apache-airflow/stable/concepts/dagfile-processing.html The DagFileProcessorManager runs user codes. As a result, you can decide to run it as a standalone process in a different host than the scheduler process. If you decide to run it as a standalone process, you need to set this configuration: AIRFLOW__SCHEDULER__STANDALONE_DAG_PROCESSOR=True and run the airflow dag-processor CLI command, otherwise, starting the scheduler process (airflow scheduler) also starts the DagFileProcessorManager

See also[edit]

Advertising: