Difference between revisions of "Datadog Airflow integration"
Jump to navigation
Jump to search
| Line 37: | Line 37: | ||
* <code>[[ERROR airflow.stats Could not configure StatsClient: No module named 'datadog', using DummyStatsLogger instead]]</code> | * <code>[[ERROR airflow.stats Could not configure StatsClient: No module named 'datadog', using DummyStatsLogger instead]]</code> | ||
* <code>[[DD_DOGSTATSD_MAPPER_PROFILES]]</code> | * <code>[[DD_DOGSTATSD_MAPPER_PROFILES]]</code> | ||
| + | * <code>[[Error submitting packet]]: [Errno 111] Connection refused, dropping the packet and closing the socket, "level": "WARNING"</code> | ||
== See also == | == See also == | ||
Revision as of 12:30, 7 December 2022
TO_MERGE: Datadog Airflow
- Kubernetes: https://airflow.apache.org/docs/helm-chart/stable/production-guide.html#datadog
- Windows/Linux https://docs.datadoghq.com/integrations/airflow/
- Official documentation with
dogstatsd_mapper_profile:configuration for the agent: https://app.datadoghq.eu/integrations/airflow
statsd_datadog_enabledstatsd_datadog_tags
Contents
Airflow monitoring
[metrics] statsd_on = True statsd_host = localhost statsd_port = 8125 statsd_prefix = airflow #statsd_allow_list = scheduler,executor,dagrun statsd_datadog_enabled = False
Logs
triggerer | [2022-12-05 14:02:35,051] {base.py:317} INFO - Statsd buffering is disabled triggerer | [2022-12-05 14:02:35,052] {base.py:335} INFO - Statsd periodic buffer flush is disabled triggerer | [2022-12-05 14:02:35,053] {base.py:317} INFO - Statsd buffering is disabled triggerer | [2022-12-05 14:02:35,053] {base.py:335} INFO - Statsd periodic buffer flush is disabled
scheduler | [2022-12-05 14:02:35,223] {base.py:317} INFO - Statsd buffering is disabled scheduler | [2022-12-05 14:02:35,234] {base.py:335} INFO - Statsd periodic buffer flush is disabled scheduler | [2022-12-05 14:02:35,235] {base.py:317} INFO - Statsd buffering is disabled scheduler | [2022-12-05 14:02:35,235] {base.py:335} INFO - Statsd periodic buffer flush is disabled
webserver | [2022-12-05 14:02:36,135] {base.py:317} INFO - Statsd buffering is disabled webserver | [2022-12-05 14:02:36,135] {base.py:335} INFO - Statsd periodic buffer flush is disabled webserver | [2022-12-05 14:02:36,136] {base.py:317} INFO - Statsd buffering is disabled webserver | [2022-12-05 14:02:36,136] {base.py:335} INFO - Statsd periodic buffer flush is disabled
Related
airflow.cfgairflow config list | grep statsd8125 port- Airflow
base_urldefault valuehttp://localhost:8080 - DogStatsD:
pip install 'apache-airflow[statsd]' ERROR airflow.stats Could not configure StatsClient: No module named 'datadog', using DummyStatsLogger insteadDD_DOGSTATSD_MAPPER_PROFILESError submitting packet: [Errno 111] Connection refused, dropping the packet and closing the socket, "level": "WARNING"
See also
- Datadog, Datadog agent, Forwarder, Spans, DBM, standard attribute list, facets, log explorer, Terraform datadog, Airflow,
dog, agent, DogStatsD, Airflow integration, Datadog pipelines,app.datadoghq.eu, Datadog Cloud SIEM, Cloud Security Management, Datadog AWS Integration, Datadog Monitors,dd-trace-go, DataDog Continuous Profiler, CoScreen, Kubernetes APM, flex logs, Datadog notifications
Advertising:
