[
https://issues.apache.org/jira/browse/BEAM-9629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Boris Shilov updated BEAM-9629:
-------------------------------
Description:
Greetings,
I am using JdbcIO via the Scala wrappers provided in the Scio project. I am
trying to read a few dozen tables in parallel from MySQL, but above 8
concurrent SELECT operations the pipeline freezes. With help of the Scio
maintainers we've been able to isolate the issue as likely originating in
JdbcIO running out of connections in the connection pool and idling
indefinitely. The issue occurs both on the Direct Runner and Dataflow.
Please see linked issue for more context:
https://github.com/spotify/scio/issues/2774
was:
Greetings,
I am using JdbcIO via the Scala wrappers provided in the Scio project. I am
trying to read a few dozen tables in parallel from MySQL, but above 8
concurrent read operations the pipeline freezes. With help of the Scio
maintainers we've been able to isolate the issue as likely originating in
JdbcIO running out of connections in the connection pool and idling
indefinitely. The issue occurs both on the Direct Runner and Dataflow.
Please see linked issue for more context:
https://github.com/spotify/scio/issues/2774
> JdbcIO seems to run out of connections in the connection pool and freezes
> pipeline
> ----------------------------------------------------------------------------------
>
> Key: BEAM-9629
> URL: https://issues.apache.org/jira/browse/BEAM-9629
> Project: Beam
> Issue Type: Bug
> Components: io-java-jdbc
> Affects Versions: 2.19.0
> Environment: Dataflow, Direct Runner on macOS Catalina.
> Reporter: Boris Shilov
> Priority: Major
> Labels: performance
>
> Greetings,
> I am using JdbcIO via the Scala wrappers provided in the Scio project. I am
> trying to read a few dozen tables in parallel from MySQL, but above 8
> concurrent SELECT operations the pipeline freezes. With help of the Scio
> maintainers we've been able to isolate the issue as likely originating in
> JdbcIO running out of connections in the connection pool and idling
> indefinitely. The issue occurs both on the Direct Runner and Dataflow.
> Please see linked issue for more context:
> https://github.com/spotify/scio/issues/2774
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
