[
https://issues.apache.org/jira/browse/AIRFLOW-6983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17055897#comment-17055897
]
Igor Khrol commented on AIRFLOW-6983:
-------------------------------------
[~ash], we have 3 workers with 16 worker_concurrency giving us 3 * 16 = 48
tasks.
Also we are using few `Variable.get` and `BaseHook.get_connection` calls for
DAG instantiation so the amount of short living DB connections per each DAG
load is not acceptable. Having this PR in place would allow us to use pool with
1-2 connections.
> Disabled connection pool in CLI might be harmful
> ------------------------------------------------
>
> Key: AIRFLOW-6983
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6983
> Project: Apache Airflow
> Issue Type: Improvement
> Components: database
> Affects Versions: 1.10.9
> Reporter: Igor Khrol
> Assignee: Igor Khrol
> Priority: Trivial
>
> Tasks are executed in CLI mode when connection pool to the database is
> disabled.
> `settings.configure_orm(disable_connection_pool=True)`
>
> While one task is run, multiple DB communications are happening while a
> separate connection is allocated for each of them.
>
> It results in DB failures.
>
> Default behavior might be ok but it worth making it configurable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
