magnusfurugard opened a new issue #19044:
URL: https://github.com/apache/airflow/issues/19044
### Official Helm Chart version
1.2.0 (latest released)
### Apache Airflow version
2.1.4 (latest released)
### Kubernetes Version
v.1.21.2
### Helm Chart configuration
```
statsd:
enabled: false
env:
# Enable statsd here, disable chart default.
- name: AIRFLOW__METRICS__STATSD_ON
value: true
- name: AIRFLOW__METRICS__HOST
value: <service name for statsd>
- name: AIRFLOW__METRICS__STATSD_PORT
value: <port for statsd>
```
### Docker Image customisations
None.
### What happened
Setting `AIRFLOW__METRICS__HOST` seem to have no effect, even if we set
`statsd.enabled=false` in the chart and enable it through the same config
(`AIRFLOW__METRICS__STATSD_ON`). Airflow will continue to try to resolve the
default host, `<name>-statsd`-service, even if set `AIRFLOW__METRICS__HOST`.
### What you expected to happen
I expected Airflow to honor the `AIRFLOW__METRICS__HOST` variable and
resolve the service as specified there.
### How to reproduce
### Step 1 - vanilla install
Install the helm-chart with the following `values.yml`
```yml
# airflow-values.yml
statsd:
enabled: true # this is the default
```
```bash
helm install -f airflow-values.yml example apache-airflow/airflow --version
1.2.0 --timeout 1200s
```
### Step 2 - Replace statsd with custom statsd
We want to disable the chart-built-in statsd and use our own instead. To
accommodate, set up _any_ statsd deployment in the same namespace. In my case,
I use the prometheus exporter one but it can be absolutely anything.
```bash
# Install statsd.
helm repo add prometheus-community
https://prometheus-community.github.io/helm-charts
helm repo update
helm install -f statsd-values.yml statsd
prometheus-community/prometheus-statsd-exporter --timeout 1200s
# Chart-default is to accept udp/tcp on port 9125. Above config also creates
a service for us called `statsd-prometheus-statsd-exporter`.
```
... And update Airflow to communicate with the new statsd
```yml
# airflow-values.yml
# Update Airflow to use the new statsd
statsd:
enabled: false
env:
- name: AIRFLOW__METRICS__STATSD_ON
value: true
- name: AIRFLOW__METRICS__HOST
value: statsd-prometheus-statsd-exporter
- name: AIRFLOW__METRICS__STATSD_PORT
value: 9125
```
```bash
helm upgrade -f airflow-values.yml example apache-airflow/airflow --version
1.2.0 --timeout 1200s
```
### Step 3 - Airflow fails to resolve the new host
Run any DAG from within Airflow, or restart the webserver to trigger an
error, displaying that it's unable to resolve the service;
```
[2021-10-18 14:22:35,668] {stats.py:359} ERROR - Could not configure
StatsClient: [Errno -2] Name or service not known, using DummyStatsLogger
instead.
```
### Step 4 - Rename the new statsd-service to match what Airflow is looking
for
We can fix this by overriding the name of the new service to match that
which we _know_ Airflow is looking for (`<name>-statsd`).
```yml
# statsd-values.yml
# Remember we set the name of our helm install to `example` in step 1.
# This toggle alters the name of the service for our custom statsd to be
whatever we put below.
fullnameOverride: example-statsd
```
```bash
helm upgrade -f statsd-values.yml statsd
prometheus-community/prometheus-statsd-exporter --timeout 1200s
```
**Airflow is now able to log metrics to our custom statsd-client.**
And just to showcase that these variables are not picked up, let's set them
to something that we know does not exist.
```yml
# airflow-values.yml
statsd:
enabled: false
env:
- name: AIRFLOW__METRICS__STATSD_ON
value: true
- name: AIRFLOW__METRICS__HOST
value: whatever-you-want-really
- name: AIRFLOW__METRICS__STATSD_PORT
value: 3123 # anything numeric
```
```bash
helm upgrade -f airflow-values.yml example apache-airflow/airflow --version
1.2.0 --timeout 1200s
```
**Airflow is _still_ able to log metrics to our custom statsd-client.**
### Anything else
I ran this with `executor=KubernetesExecutor`, but I doubt it has any
effect. Made releases and upgrades faster.
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
