GitHub user potiuk added a comment to the discussion: Scaling Airflow 3 on EKS — API server OOMs, PgBouncer saturation, and health check flakiness at 8K concurrent tasks
Yep - but it should be measured - because it's not "absolutely" needed for Postgres Metadata DB as it was in Airflow 2. Postgres **can** handle many connections - and since it's not open/close all the time. PGBouncer was really **necessary** and helped in Airflow 2 as each task opened and closed connections continuously with every task, which is quite different use scenario. I do not have a "perfect" answer when and how, but this is more "deployment" question rather than "Recommendation resulting from Airflow Architecture". GitHub link: https://github.com/apache/airflow/discussions/62117#discussioncomment-18000179 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
