kongdewen opened a new issue, #41124:
URL: https://github.com/apache/airflow/issues/41124
### Apache Airflow version
2.9.3
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
we are deploying to Airflow into a IPv6 only K8s cluster. when we enabled
Flower, and added the following in the airflow.cfg:
```
[celery]
flower_host = [::]
flower_url_prefix =
worker_concurrency = 16
```
Flower pod get into `CrashLoopBackOff` state, with following error in the log
```
[2024-07-30T15:37:07.638+0000] {command.py:168} INFO - Visit me at
http://[::]:5555
[2024-07-30T15:37:07.660+0000] {command.py:176} INFO - Broker:
redis://:**@airflow-redis:6379/0
[2024-07-30T15:37:07.663+0000] {command.py:177} INFO - Registered tasks:
['airflow.providers.celery.executors.celery_executor_utils.execute_command',
'celery.accumulate',
'celery.backend_cleanup',
'celery.chain',
'celery.chord',
'celery.chord_unlock',
'celery.chunks',
'celery.group',
'celery.map',
'celery.starmap']
Traceback (most recent call last):
File "/home/airflow/.local/bin/airflow", line 8, in <module>
sys.exit(main())
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/__main__.py", line
58, in main
args.func(args)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/cli_config.py",
line 49, in command
return func(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/cli.py", line
114, in wrapper
return f(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/celery/cli/celery_command.py",
line 61, in wrapper
providers_configuration_loaded(func)(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/providers_configuration_loader.py",
line 55, in wrapped_function
return func(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/celery/cli/celery_command.py",
line 98, in flower
_run_command_with_daemon_option(
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/celery/cli/celery_command.py",
line 47, in _run_command_with_daemon_option
run_command_with_daemon_option(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/daemon_utils.py",
line 85, in run_command_with_daemon_option
callback()
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/celery/cli/celery_command.py",
line 99, in <lambda>
args=args, process_name="flower", callback=lambda:
celery_app.start(options)
File
"/home/airflow/.local/lib/python3.8/site-packages/celery/app/base.py", line
371, in start
celery.main(args=argv, standalone_mode=False)
File "/home/airflow/.local/lib/python3.8/site-packages/click/core.py",
line 1078, in main
rv = self.invoke(ctx)
File "/home/airflow/.local/lib/python3.8/site-packages/click/core.py",
line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/airflow/.local/lib/python3.8/site-packages/click/core.py",
line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/airflow/.local/lib/python3.8/site-packages/click/core.py",
line 783, in invoke
return __callback(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.8/site-packages/click/decorators.py", line
33, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/airflow/.local/lib/python3.8/site-packages/flower/command.py",
line 57, in flower
flower_app.start()
File "/home/airflow/.local/lib/python3.8/site-packages/flower/app.py",
line 72, in start
self.listen(self.options.port, address=self.options.address,
File "/home/airflow/.local/lib/python3.8/site-packages/tornado/web.py",
line 2211, in listen
server.listen(
File
"/home/airflow/.local/lib/python3.8/site-packages/tornado/tcpserver.py", line
183, in listen
sockets = bind_sockets(
File
"/home/airflow/.local/lib/python3.8/site-packages/tornado/netutil.py", line
105, in bind_sockets
socket.getaddrinfo(address, port, family, socket.SOCK_STREAM, 0, flags),
File "/usr/local/lib/python3.8/socket.py", line 918, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
```
without ` flower_host = [::]`, flower pod will be running fine. any one
experience this as well? and how to fix this?
### What you think should happen instead?
_No response_
### How to reproduce
converted helm template into yaml as following
```
---
# Source: airflow/templates/flower/flower-serviceaccount.yaml
######################################
## Airflow Flower ServiceAccount
######################################
apiVersion: v1
kind: ServiceAccount
automountServiceAccountToken: true
metadata:
name: airflow-flower
labels:
tier: airflow
component: flower
release: airflow
chart: "airflow-1.14.0"
heritage: Helm
---
# Source: airflow/templates/flower/flower-service.yaml
################################
## Airflow Flower Service Component
#################################
apiVersion: v1
kind: Service
metadata:
name: airflow-flower
labels:
tier: airflow
component: flower
release: airflow
chart: "airflow-1.14.0"
heritage: Helm
spec:
type: ClusterIP
selector:
tier: airflow
component: flower
release: airflow
ipFamilies:
- IPv6
ports:
- name: flower-ui
port: 5555
---
# Source: airflow/templates/flower/flower-deployment.yaml
################################
## Airflow Flower Deployment
#################################
apiVersion: apps/v1
kind: Deployment
metadata:
name: airflow-flower
labels:
tier: airflow
component: flower
release: airflow
chart: "airflow-1.14.0"
heritage: Helm
spec:
replicas: 1
selector:
matchLabels:
tier: airflow
component: flower
release: airflow
template:
metadata:
labels:
tier: airflow
component: flower
release: airflow
annotations:
checksum/airflow-config: xxx
checksum/flower-secret: xxx
nodeSelector:
{}
affinity:
[]
tolerations:
[]
topologySpreadConstraints:
[]
serviceAccountName: airflow-flower
restartPolicy: Always
securityContext:
runAsUser: 50000
fsGroup: 0
containers:
- name: flower
image: docker.io/airflow:2.9.3-python3.8
imagePullPolicy: IfNotPresent
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
args:
- bash
- -c
- |-
exec \
airflow celery flower
resources:
{}
volumeMounts:
- name: config
mountPath: "/opt/airflow/airflow.cfg"
subPath: airflow.cfg
readOnly: true
- name: config
mountPath: "/opt/airflow/config/airflow_local_settings.py"
subPath: airflow_local_settings.py
readOnly: true
ports:
- name: flower-ui
containerPort: 5555
livenessProbe:
failureThreshold: 10
exec:
command:
- curl
- localhost:5555
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 5
readinessProbe:
failureThreshold: 10
exec:
command:
- curl
- localhost:5555
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 5
envFrom:
[]
env:
# Hard Coded Airflow Envs
- name: AIRFLOW__CORE__FERNET_KEY
valueFrom:
secretKeyRef:
name: airflow-fernet-key
key: fernet-key
- name: AIRFLOW_HOME
value: /opt/airflow
# For Airflow <2.3, backward compatibility; moved to [database]
in 2.3
- name: AIRFLOW__CORE__SQL_ALCHEMY_CONN
valueFrom:
secretKeyRef:
name: airflow-metadata
key: connection
- name: AIRFLOW__DATABASE__SQL_ALCHEMY_CONN
valueFrom:
secretKeyRef:
name: airflow-metadata
key: connection
- name: AIRFLOW_CONN_AIRFLOW_DB
valueFrom:
secretKeyRef:
name: airflow-metadata
key: connection
- name: AIRFLOW__WEBSERVER__SECRET_KEY
valueFrom:
secretKeyRef:
name: airflow-webserver-secret-key
key: webserver-secret-key
- name: AIRFLOW__CELERY__BROKER_URL
valueFrom:
secretKeyRef:
name: airflow-broker-url
key: connection
# Dynamically created environment variables
# Dynamically created secret envs
# Extra env
volumes:
- name: config
configMap:
name: airflow-config
```
### Operating System
Centos 9
### Versions of Apache Airflow Providers
_No response_
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]