Re: DirectRunner in test - await completion of workers threads?

2018-04-10 Thread Ismaël Mejía
It seems there is still an issue with teardown not being called in failed tasks, just created BEAM-4040 to track it. On Thu, Apr 5, 2018 at 4:45 PM, Tim Robertson wrote: > Will do - I'll report the result on https://github.com/apache/beam/pull/4905 > > On Thu, Apr 5, 2018 at 11:45 AM, Ismaël Mejí

Re: DirectRunner in test - await completion of workers threads?

2018-04-05 Thread Tim Robertson
Will do - I'll report the result on https://github.com/apache/beam/pull/4905 On Thu, Apr 5, 2018 at 11:45 AM, Ismaël Mejía wrote: > For info, Romain's PR was merged today, can you confirm if this fixes > the issue Tim. > > On Sun, Apr 1, 2018 at 9:21 PM, Tim Robertson > wrote: > > Thanks all.

Re: DirectRunner in test - await completion of workers threads?

2018-04-05 Thread Ismaël Mejía
For info, Romain's PR was merged today, can you confirm if this fixes the issue Tim. On Sun, Apr 1, 2018 at 9:21 PM, Tim Robertson wrote: > Thanks all. > > I went with what I outlined above, which you can see in this test. > https://github.com/timrobertson100/beam/blob/BEAM-3848/sdks/java/io/solr

Re: DirectRunner in test - await completion of workers threads?

2018-04-01 Thread Tim Robertson
Thanks all. I went with what I outlined above, which you can see in this test. https://github.com/timrobertson100/beam/blob/BEAM-3848/sdks/java/io/solr/src/test/java/org/apache/beam/sdk/io/solr/SolrIOTest.java#L285 That forms part of this PR https://github.com/apache/beam/pull/4956 I'll monitor

Re: DirectRunner in test - await completion of workers threads?

2018-04-01 Thread Jean-Baptiste Onofré
Indeed. It's exactly what Romain's PR is about. Regards JB Le 1 avr. 2018 à 19:33, à 19:33, Reuven Lax a écrit: >Correct - teardown is currently run in the direct runner, but >asynchronously. I believe Romain's pending PRs should solve this for >your >use case. > >On Sun, Apr 1, 2018 at 3:13 AM

Re: DirectRunner in test - await completion of workers threads?

2018-04-01 Thread Reuven Lax
Correct - teardown is currently run in the direct runner, but asynchronously. I believe Romain's pending PRs should solve this for your use case. On Sun, Apr 1, 2018 at 3:13 AM Tim Robertson wrote: > Thanks for confirming Romain - also for the very fast reply! > > I'll continue with the workarou

Re: DirectRunner in test - await completion of workers threads?

2018-04-01 Thread Tim Robertson
Thanks for confirming Romain - also for the very fast reply! I'll continue with the workaround and reference BEAM-3409 inline as justification. I'm trying to wrap this up before travel next week, but if I get a chance I'll try and run this scenario (BEAM-3848) with your patch. On Sun, Apr 1, 20

Re: DirectRunner in test - await completion of workers threads?

2018-04-01 Thread Romain Manni-Bucau
Hi I have the same blocker and created https://github.com/apache/beam/pull/4790 and https://github.com/apache/beam/pull/4965 to solve part of it Le 1 avr. 2018 11:35, "Tim Robertson" a écrit : Hi devs I'm working on SolrIO tests for failure scenarios (i.e. an exception will come out of the