Since the connection pool is per-TM, and a given job may span numerous TMs, I think you're imagining a per-TM cleanup.
Your idea of using reference counting seems like the best option. Maintain a static count of open instances. Use a synchronization block to manage the count and to conditionally open and close the connection pool. On Mon, Dec 18, 2017 at 1:58 AM, Fabian Hueske <fhue...@gmail.com> wrote: > Hi Andrew, > > I'm not aware of such a plan. > Another way to address such issues is to run multiple TaskManagers with a > single slot. In that case, only one subtask is executed per TM processes. > > Best, Fabian > > 2017-12-15 22:23 GMT+01:00 Andrew Roberts <arobe...@fuze.com>: > >> Hello, >> >> I’m writing a Flink operator that connects to a database, and running it >> in parallel results in issues due to the singleton nature of the connection >> pool in the library I’m working with. The operator needs to close the >> connection pool when it’s done, but only when ALL parallel instances are >> done. If one subtask finishes first, then it closes the pool out from under >> the subtasks that are still working. Currently, I’m using a >> reference-counting hack that’s pretty brittle and unsatisfying. Are there >> any plans to add a user-definable job-level cleanup interface? >> >> Thanks >> >> Andrew >> -- >> *Confidentiality Notice: The information contained in this e-mail and any >> attachments may be confidential. If you are not an intended recipient, you >> are hereby notified that any dissemination, distribution or copying of >> this >> e-mail is strictly prohibited. If you have received this e-mail in error, >> please notify the sender and permanently delete the e-mail and any >> attachments immediately. You should not retain, copy or use this e-mail or >> any attachment for any purpose, nor disclose all or any part of the >> contents to any other person. Thank you.* >> > >
