Re: Reducing database connection with JdbcIO

with BQ service the exception will be thrown, but my job will be restored from latest checkpoint. So I will not generate uuid for same message twice. In case of jdbcio it is possible to get uuid for same mesage twice( in case of multiple io it might be a problem). Aleksandr. 14. märts 2018 10:37

Re: Reducing database connection with JdbcIO

es to include a GBK before writing. Unfortunately I can't recommend a long-term reliable solution for the moment. On Wed, Mar 14, 2018 at 12:57 PM Aleksandr <aleksandr...@gmail.com> wrote: > Hello, > How many times will the setup per node be called? Is it possible to limit > pardo i

Re: Reducing database connection with JdbcIO

Hello, How many times will the setup per node be called? Is it possible to limit pardo intances in google dataflow? Aleksandr. 14. märts 2018 9:22 PM kirjutas kuupäeval "Eugene Kirpichov" < kirpic...@google.com>: "Jdbcio will create for each prepared sta

Re: Reducing database connection with JdbcIO

of auto scaling you will get to many connections to sql. Our solution was to create connection pool in setup and get connection and return back to pool in processElement. Best Regards, Aleksandr Gortujev. 14. märts 2018 8:52 PM kirjutas kuupäeval "Jean-Baptiste Onofré" < j...@nanthrax

Re: Reducing database connection with JdbcIO

Hello, We did own jdbcio with thread pool per jwm (using lazy initialization in @Setup). In processElement we are getting/freeing connection. Best Regards, Aleksandr Gortujev. 14. märts 2018 12:49 PM kirjutas kuupäeval "Derek Chan" <derek...@gmail.com >: Hi, We are new to Be

Re: IllegalStateException when combining 3 streams?

Hello, You can try put PCollection after flatten into same global window with triggers as it was before flattening. Best regards Aleksandr Gortujev 3. nov 2017 11:04 AM kirjutas kuupäeval "Artur Mrozowski" <art...@gmail.com >: Hi, I am on second week of our PoC with Beam and I

Re: PipelineTest with TestStreams: unable to serialize

Hello, Probably error is in your tuple tag classes, which are anonymous classes. It means that your test is trying to serialise testpipeline. Best regards Aleksandr Gortujev 3. nov 2017 3:33 PM kirjutas kuupäeval "Matthias Baetens" < matthias.baet...@datatonic.com>: Hi all

Re: Infinite retry in streaming - is there a workaround?

Hello Derek, There no general solution for failing bundle. Some kind of dataflow errors you can fix using dataflow update feature. Another solution is to catch exceptions in ParDo function. 25. okt 2017 9:42 PM kirjutas kuupäeval "Griselda Cuevas" : Hi Derek, yes you can use

Re: [VOTE] [DISCUSSION] Remove support for Java 7

+1 17. okt 2017 7:17 PM kirjutas kuupäeval "Ismaël Mejía" : We have discussed recently in the developer mailing list about the idea of removing support for Java 7 on Beam. There are multiple reasons for this: - Java 7 has not received public updates for almost two years and