Difference between revisions of "Datadog Airflow integration"
Jump to navigation
Jump to search
Line 4: | Line 4: | ||
* Windows/Linux https://docs.datadoghq.com/integrations/airflow/ | * Windows/Linux https://docs.datadoghq.com/integrations/airflow/ | ||
* Official documentation with <code>[[dogstatsd_mapper_profile:]]</code> configuration for the agent: https://app.datadoghq.eu/integrations/airflow | * Official documentation with <code>[[dogstatsd_mapper_profile:]]</code> configuration for the agent: https://app.datadoghq.eu/integrations/airflow | ||
+ | * https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html | ||
<code>[[statsd_datadog_enabled]]</code> | <code>[[statsd_datadog_enabled]]</code> |
Revision as of 12:23, 8 December 2022
TO_MERGE: Datadog Airflow
- Kubernetes: https://airflow.apache.org/docs/helm-chart/stable/production-guide.html#datadog
- Windows/Linux https://docs.datadoghq.com/integrations/airflow/
- Official documentation with
dogstatsd_mapper_profile:
configuration for the agent: https://app.datadoghq.eu/integrations/airflow - https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html
statsd_datadog_enabled
statsd_datadog_tags
Airflow monitoring
[metrics] statsd_on = True statsd_host = localhost statsd_port = 8125 statsd_prefix = airflow #statsd_allow_list = scheduler,executor,dagrun statsd_datadog_enabled = False
Logs
triggerer | [2022-12-05 14:02:35,051] {base.py:317} INFO - Statsd buffering is disabled triggerer | [2022-12-05 14:02:35,052] {base.py:335} INFO - Statsd periodic buffer flush is disabled triggerer | [2022-12-05 14:02:35,053] {base.py:317} INFO - Statsd buffering is disabled triggerer | [2022-12-05 14:02:35,053] {base.py:335} INFO - Statsd periodic buffer flush is disabled
scheduler | [2022-12-05 14:02:35,223] {base.py:317} INFO - Statsd buffering is disabled scheduler | [2022-12-05 14:02:35,234] {base.py:335} INFO - Statsd periodic buffer flush is disabled scheduler | [2022-12-05 14:02:35,235] {base.py:317} INFO - Statsd buffering is disabled scheduler | [2022-12-05 14:02:35,235] {base.py:335} INFO - Statsd periodic buffer flush is disabled
webserver | [2022-12-05 14:02:36,135] {base.py:317} INFO - Statsd buffering is disabled webserver | [2022-12-05 14:02:36,135] {base.py:335} INFO - Statsd periodic buffer flush is disabled webserver | [2022-12-05 14:02:36,136] {base.py:317} INFO - Statsd buffering is disabled webserver | [2022-12-05 14:02:36,136] {base.py:335} INFO - Statsd periodic buffer flush is disabled
Errors
ERROR airflow.stats Could not configure StatsClient: No module named 'datadog', using DummyStatsLogger instead
Error submitting packet: [Errno 111] Connection refused, dropping the packet and closing the socket, "level": "WARNING
Related
airflow.cfg
airflow config list | grep statsd
8125 port
- Airflow
base_url
default valuehttp://localhost:8080
- DogStatsD:
pip install 'apache-airflow[statsd]'
DD_DOGSTATSD_MAPPER_PROFILES
See also
- Datadog, Datadog agent, Forwarder, Spans, DBM, standard attribute list, facets, log explorer, Terraform datadog, Airflow,
dog, agent, DogStatsD
, Airflow integration, Datadog pipelines,app.datadoghq.eu
, Datadog Cloud SIEM, Cloud Security Management, Datadog AWS Integration, Datadog Monitors,dd-trace-go
, DataDog Continuous Profiler, CoScreen, Kubernetes APM, flex logs, Datadog notifications
Advertising: