potiuk commented on code in PR #51586:
URL: https://github.com/apache/airflow/pull/51586#discussion_r2145098824
##########
airflow-core/src/airflow/config_templates/config.yml:
##########
@@ -679,6 +679,39 @@ database:
type: integer
example: ~
default: "10000"
+ check_db_discovery:
Review Comment:
I am not sure we really want it - this seems to be band-aid for
misconfiguration of K8S and it only works for one specific case (database
lookup) which is likely one of many cases where temporary blip in DNS might
cause even more problems. Any time you communicate with another POD - you get
DNS lookup, any time you want to send message to broker you get DNS lookup,
etc. etc.
Adding 4 new configuration options to the list of few 100s we already have
to handle this single case seems quite wrong.
Instead one should look at their K8S configuration and see why there are DNS
errors and eliminate them. From the experience of others - the problem is that
in many installation k8s DNS is configured with fractional CPU (often 0.1 =
100ms) - and the way how k8s slicing works in this case is that if during 0.1
s you DNS is busy, it will stop responding for the raminder of the second. And
the way to fix it is to configure your DNS to not have fractional CPU
allocation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
