GitHub user raphaelauv added a comment to the discussion: Scaling Airflow 3 on EKS — API server OOMs, PgBouncer saturation, and health check flakiness at 8K concurrent tasks
Yes the user is free to use any result_backend datastore. Since the celery provider by default use the airflow database ( and same for the helm chart ) , for users with many workers the pgbouncer of the helm chart can still be useful in this case. GitHub link: https://github.com/apache/airflow/discussions/62117#discussioncomment-17999337 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
