GitHub user jrthe42 opened a pull request: https://github.com/apache/flink/pull/6301
[FLINK-9794] [jdbc] JDBCOutputFormat does not consider idle connection and multithreads synchronization ## What is the purpose of the change This pull request fix bugs in original implementation of `JDBCOutputFormat`, which does not consider idle connection and multithreads synchronization . - The Connection was established when JDBCOutputFormat is opened, and will be used all the time. But if this connection lies idle for a long time, the database will force close the connection, thus errors may occur. - The flush() method is called when batchCount exceeds the threshold, but it is also called while snapshotting state. So two threads may modify upload and batchCount, but without synchronization. ## Brief change log - Using a Timer to test the jdbc connection periodically and keep it alive - Add synchronization for batch operation ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable) You can merge this pull request into a Git repository by running: $ git pull https://github.com/jrthe42/flink fix-jdbcoutputformat Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/6301.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #6301 ---- commit d0c34811e3a30ebc97fd784ebf2fdd3f2085358b Author: jrthe42 <jrthe42@...> Date: 2018-07-11T01:50:29Z Using a Timer to test the jdbc connection periodically and keep it alive If jdbc connction lies idle for a long time, the database will force close the connetion. Keep this connection valid using a timer. commit d08938350bb3d78a324562e14d24a0892a052b5f Author: jrthe42 <jrthe42@...> Date: 2018-07-11T01:58:40Z [FLINK-9794] JDBCOutputFormat does not consider idle connection and multithreads synchronization ---- ---