Difference between revisions of "Airflow celery worker"
Jump to navigation
Jump to search
| (7 intermediate revisions by one other user not shown) | |||
| Line 4: | Line 4: | ||
[[airflow celery]] worker | [[airflow celery]] worker | ||
airflow celery worker -q yourqueue1,yourqueue2 | airflow celery worker -q yourqueue1,yourqueue2 | ||
| + | |||
| + | |||
| + | == Errors == | ||
| + | [[airflow celery]] worker | ||
| + | airflow command error: argument GROUP_OR_COMMAND: celery subcommand works only with [[CeleryExecutor]], [[CeleryKubernetesExecutor]] and executors derived from them, your current executor: [[SequentialExecutor]], subclassed from: [[BaseExecutor]], see help above. | ||
| + | |||
| + | Solution: change [[Airflow executors|executor]] in [[airflow.cfg]] to [[CeleryExecutor]] | ||
| + | |||
| + | |||
| + | [[airflow celery]] worker | ||
| + | Traceback (most recent call last): | ||
| + | File "/usr/local/bin/airflow", line 5, in <module> | ||
| + | from airflow.__main__ import main | ||
| + | File "/usr/local/lib/python3.10/dist-packages/airflow/__init__.py", line 34, in <module> | ||
| + | from airflow import settings | ||
| + | File "/usr/local/lib/python3.10/dist-packages/airflow/settings.py", line 37, in <module> | ||
| + | from airflow.configuration import AIRFLOW_HOME, WEBSERVER_CONFIG, conf # NOQA F401 | ||
| + | File "/usr/local/lib/python3.10/dist-packages/airflow/configuration.py", line 1650, in <module> | ||
| + | conf.validate() | ||
| + | File "/usr/local/lib/python3.10/dist-packages/airflow/configuration.py", line 321, in validate | ||
| + | self._validate_config_dependencies() | ||
| + | File "/usr/local/lib/python3.10/dist-packages/airflow/configuration.py", line 418, in _validate_config_dependencies | ||
| + | raise AirflowConfigException(f"error: cannot use sqlite with the {self.get('core', 'executor')}") | ||
| + | airflow.exceptions.AirflowConfigException: error: cannot use [[sqlite]] with the [[CeleryExecutor]] | ||
== Related == | == Related == | ||
| − | + | * <code>[[celery worker]]</code> | |
== See also == | == See also == | ||
| + | * {{airflow celery}} | ||
* {{airflow cmd}} | * {{airflow cmd}} | ||
* {{Airflow}} | * {{Airflow}} | ||
Latest revision as of 11:11, 20 December 2022
airflow celery worker airflow celery worker -q yourqueue1,yourqueue2
Errors[edit]
airflow celery worker airflow command error: argument GROUP_OR_COMMAND: celery subcommand works only with CeleryExecutor, CeleryKubernetesExecutor and executors derived from them, your current executor: SequentialExecutor, subclassed from: BaseExecutor, see help above. Solution: change executor in airflow.cfg to CeleryExecutor
airflow celery worker Traceback (most recent call last): File "/usr/local/bin/airflow", line 5, in <module> from airflow.__main__ import main File "/usr/local/lib/python3.10/dist-packages/airflow/__init__.py", line 34, in <module> from airflow import settings File "/usr/local/lib/python3.10/dist-packages/airflow/settings.py", line 37, in <module> from airflow.configuration import AIRFLOW_HOME, WEBSERVER_CONFIG, conf # NOQA F401 File "/usr/local/lib/python3.10/dist-packages/airflow/configuration.py", line 1650, in <module> conf.validate() File "/usr/local/lib/python3.10/dist-packages/airflow/configuration.py", line 321, in validate self._validate_config_dependencies() File "/usr/local/lib/python3.10/dist-packages/airflow/configuration.py", line 418, in _validate_config_dependencies raise AirflowConfigException(f"error: cannot use sqlite with the {self.get('core', 'executor')}") airflow.exceptions.AirflowConfigException: error: cannot use sqlite with the CeleryExecutor
Related[edit]
See also[edit]
airflow celery: [ flower | worker | stop | --help ]airflow [ info | celery | config | db upgrad | config list | db init | scheduler | webserver | users | dags | kubernetes | pools | roles | tasks | variables | cheat-sheet | kereberos | plugins | rotate-fernet-key | sync-perm ] airflow --help- Apache Airflow, Installation, configuration
airflow.cfg, monitoring, Amazon Managed Workflows for Apache Airflow (MWAA),airflow, Airflow Scheduler, providers, Workflows, Airflow executors, Airflow versions, Datadog integration
Advertising:
