Difference between revisions of "Datadog Airflow integration"
Jump to navigation
Jump to search
| Line 28: | Line 28: | ||
webserver | [2022-12-05 14:02:36,136] {base.py:317} INFO - Statsd buffering is disabled | webserver | [2022-12-05 14:02:36,136] {base.py:317} INFO - Statsd buffering is disabled | ||
webserver | [2022-12-05 14:02:36,136] {base.py:335} INFO - Statsd periodic buffer flush is disabled | webserver | [2022-12-05 14:02:36,136] {base.py:335} INFO - Statsd periodic buffer flush is disabled | ||
| + | |||
| + | == Errors == | ||
| + | * <code>[[ERROR airflow.stats Could not configure StatsClient: No module named 'datadog', using DummyStatsLogger instead]]</code> | ||
| + | * <code>[[Error submitting packet]]: [Errno 111] Connection refused, dropping the packet and closing the socket, "level": "WARNING</code> | ||
== Related == | == Related == | ||
| Line 35: | Line 39: | ||
* [[Airflow]] <code>[[base_url]]</code> default value <code>http://localhost:8080</code> | * [[Airflow]] <code>[[base_url]]</code> default value <code>http://localhost:8080</code> | ||
* [[DogStatsD]]: <code>[[pip install]] 'apache-airflow[statsd]'</code> | * [[DogStatsD]]: <code>[[pip install]] 'apache-airflow[statsd]'</code> | ||
| − | |||
* <code>[[DD_DOGSTATSD_MAPPER_PROFILES]]</code> | * <code>[[DD_DOGSTATSD_MAPPER_PROFILES]]</code> | ||
| − | |||
== See also == | == See also == | ||
Revision as of 12:31, 7 December 2022
TO_MERGE: Datadog Airflow
- Kubernetes: https://airflow.apache.org/docs/helm-chart/stable/production-guide.html#datadog
- Windows/Linux https://docs.datadoghq.com/integrations/airflow/
- Official documentation with
dogstatsd_mapper_profile:configuration for the agent: https://app.datadoghq.eu/integrations/airflow
statsd_datadog_enabledstatsd_datadog_tags
Airflow monitoring
[metrics] statsd_on = True statsd_host = localhost statsd_port = 8125 statsd_prefix = airflow #statsd_allow_list = scheduler,executor,dagrun statsd_datadog_enabled = False
Logs
triggerer | [2022-12-05 14:02:35,051] {base.py:317} INFO - Statsd buffering is disabled triggerer | [2022-12-05 14:02:35,052] {base.py:335} INFO - Statsd periodic buffer flush is disabled triggerer | [2022-12-05 14:02:35,053] {base.py:317} INFO - Statsd buffering is disabled triggerer | [2022-12-05 14:02:35,053] {base.py:335} INFO - Statsd periodic buffer flush is disabled
scheduler | [2022-12-05 14:02:35,223] {base.py:317} INFO - Statsd buffering is disabled scheduler | [2022-12-05 14:02:35,234] {base.py:335} INFO - Statsd periodic buffer flush is disabled scheduler | [2022-12-05 14:02:35,235] {base.py:317} INFO - Statsd buffering is disabled scheduler | [2022-12-05 14:02:35,235] {base.py:335} INFO - Statsd periodic buffer flush is disabled
webserver | [2022-12-05 14:02:36,135] {base.py:317} INFO - Statsd buffering is disabled webserver | [2022-12-05 14:02:36,135] {base.py:335} INFO - Statsd periodic buffer flush is disabled webserver | [2022-12-05 14:02:36,136] {base.py:317} INFO - Statsd buffering is disabled webserver | [2022-12-05 14:02:36,136] {base.py:335} INFO - Statsd periodic buffer flush is disabled
Errors
ERROR airflow.stats Could not configure StatsClient: No module named 'datadog', using DummyStatsLogger insteadError submitting packet: [Errno 111] Connection refused, dropping the packet and closing the socket, "level": "WARNING
Related
airflow.cfgairflow config list | grep statsd8125 port- Airflow
base_urldefault valuehttp://localhost:8080 - DogStatsD:
pip install 'apache-airflow[statsd]' DD_DOGSTATSD_MAPPER_PROFILES
See also
- Datadog, Datadog agent, Forwarder, Spans, DBM, standard attribute list, facets, log explorer, Terraform datadog, Airflow,
dog, agent, DogStatsD, Airflow integration, Datadog pipelines,app.datadoghq.eu, Datadog Cloud SIEM, Cloud Security Management, Datadog AWS Integration, Datadog Monitors,dd-trace-go, DataDog Continuous Profiler, CoScreen, Kubernetes APM, flex logs, Datadog notifications
Advertising:
