Difference between revisions of "Airflow celery worker"

From wikieduonline
Jump to navigation Jump to search
(Created page with "{{lc}} == See also == * {{airflow cmd}} * {{Airflow}} Category:Apache Category:Airflow Category:Big data")
 
 
(9 intermediate revisions by one other user not shown)
Line 1: Line 1:
 
{{lc}}
 
{{lc}}
  
 +
 +
[[airflow celery]] worker
 +
airflow celery worker -q yourqueue1,yourqueue2
 +
 +
 +
== Errors ==
 +
[[airflow celery]] worker
 +
airflow command error: argument GROUP_OR_COMMAND: celery subcommand works only with [[CeleryExecutor]], [[CeleryKubernetesExecutor]] and executors derived from them, your current executor: [[SequentialExecutor]], subclassed from: [[BaseExecutor]], see help above.
 +
 +
Solution: change [[Airflow executors|executor]] in [[airflow.cfg]] to [[CeleryExecutor]]
 +
 +
 +
[[airflow celery]] worker
 +
Traceback (most recent call last):
 +
  File "/usr/local/bin/airflow", line 5, in <module>
 +
    from airflow.__main__ import main
 +
  File "/usr/local/lib/python3.10/dist-packages/airflow/__init__.py", line 34, in <module>
 +
    from airflow import settings
 +
  File "/usr/local/lib/python3.10/dist-packages/airflow/settings.py", line 37, in <module>
 +
    from airflow.configuration import AIRFLOW_HOME, WEBSERVER_CONFIG, conf  # NOQA F401
 +
  File "/usr/local/lib/python3.10/dist-packages/airflow/configuration.py", line 1650, in <module>
 +
    conf.validate()
 +
  File "/usr/local/lib/python3.10/dist-packages/airflow/configuration.py", line 321, in validate
 +
    self._validate_config_dependencies()
 +
  File "/usr/local/lib/python3.10/dist-packages/airflow/configuration.py", line 418, in _validate_config_dependencies
 +
    raise AirflowConfigException(f"error: cannot use sqlite with the {self.get('core', 'executor')}")
 +
airflow.exceptions.AirflowConfigException: error: cannot use [[sqlite]] with the [[CeleryExecutor]]
 +
 +
== Related ==
 +
* <code>[[celery worker]]</code>
  
 
== See also ==
 
== See also ==
 +
* {{airflow celery}}
 
* {{airflow cmd}}
 
* {{airflow cmd}}
 
* {{Airflow}}
 
* {{Airflow}}

Latest revision as of 11:11, 20 December 2022


airflow celery worker
airflow celery worker -q yourqueue1,yourqueue2


Errors[edit]

airflow celery worker
airflow command error: argument GROUP_OR_COMMAND: celery subcommand works only with CeleryExecutor, CeleryKubernetesExecutor and executors derived from them, your current executor: SequentialExecutor, subclassed from: BaseExecutor, see help above.

Solution: change executor in airflow.cfg to CeleryExecutor



airflow celery worker
Traceback (most recent call last):
 File "/usr/local/bin/airflow", line 5, in <module>
   from airflow.__main__ import main
 File "/usr/local/lib/python3.10/dist-packages/airflow/__init__.py", line 34, in <module>
   from airflow import settings
 File "/usr/local/lib/python3.10/dist-packages/airflow/settings.py", line 37, in <module>
   from airflow.configuration import AIRFLOW_HOME, WEBSERVER_CONFIG, conf  # NOQA F401
 File "/usr/local/lib/python3.10/dist-packages/airflow/configuration.py", line 1650, in <module>
   conf.validate()
 File "/usr/local/lib/python3.10/dist-packages/airflow/configuration.py", line 321, in validate
   self._validate_config_dependencies()
 File "/usr/local/lib/python3.10/dist-packages/airflow/configuration.py", line 418, in _validate_config_dependencies
   raise AirflowConfigException(f"error: cannot use sqlite with the {self.get('core', 'executor')}")
airflow.exceptions.AirflowConfigException: error: cannot use sqlite with the CeleryExecutor

Related[edit]

See also[edit]

Retrieved from "https://www.wikieduonline.com/index.php?title=Airflow_celery_worker&oldid=210371"

Advertising: