Re: HiveIO

Hi Tim, I pushed the HBaseIO on github, I will do the same later today for HiveIO. I will let you know. Thanks ! Regards JB On 12/15/2016 02:39 AM, Tim Taschke wrote: Great to see that there is progress on this! Like written on the user mailing list, I would also be interested in

Re: Jenkins build is still unstable: beam_PostCommit_Java_RunnableOnService_Dataflow #1805

Jira filed: https://issues.apache.org/jira/browse/BEAM-1153 PR of a forward fix: https://github.com/apache/incubator-beam/pull/1608 PR of a partially rollback fix: https://github.com/apache/incubator-beam/pull/1611 On Tue, Dec 13, 2016 at 4:13 PM, Pei He wrote: > Thanks Dan

Re: HiveIO

For ref, I just created a JIRA so people can track the progress/contribute to the progress of HiveIO. https://issues.apache.org/jira/browse/BEAM-1158 On Wed, Dec 7, 2016 at 5:39 PM, Jean-Baptiste Onofré wrote: > Yes that's the first idea ;) > > Regards > JB⁣​ > > On Dec 7,

Re: New testSideInputsWithMultipleWindows and should DoFnRunner explode if DoFn contains a side input ?

Yes, this is a bug in SimplerDoFnRunner (or maybe some clarity on whether or not it owns this) not the Spark runner. FWIW the test is definitely correct, and runners-core has had this bug for a while. It is https://issues.apache.org/jira/browse/BEAM-1149 and I'm on it. On Wed, Dec 14, 2016 at

Re: New testSideInputsWithMultipleWindows and should DoFnRunner explode if DoFn contains a side input ?

Yes, it should explode if it has side inputs as well. On Wed, Dec 14, 2016 at 11:03 AM, Amit Sela wrote: > Hi all, > > Yesterday a new test was added to ParDoTest suite: > "testSideInputsWithMultipleWindows". > To the best of my understanding, it's meant to test sideInputs

Re: Jenkins build is still unstable: beam_PostCommit_Java_RunnableOnService_Spark #409

This relates to my question in Dev list. On Wed, Dec 14, 2016 at 9:07 PM Kenneth Knowles wrote: > This is still https://issues.apache.org/jira/browse/BEAM-1149. We recently > added a test for it. The actual behavior has been broken for everyone for a > while. It is

Re: Jenkins build is still unstable: beam_PostCommit_Java_RunnableOnService_Spark #409

This is still https://issues.apache.org/jira/browse/BEAM-1149. We recently added a test for it. The actual behavior has been broken for everyone for a while. It is half-fixed by Eugene K. (some DoFnRunners) but not all. On Wed, Dec 14, 2016 at 10:51 AM, Apache Jenkins Server <

New testSideInputsWithMultipleWindows and should DoFnRunner explode if DoFn contains a side input ?

Hi all, Yesterday a new test was added to ParDoTest suite: "testSideInputsWithMultipleWindows". To the best of my understanding, it's meant to test sideInputs for elements in multiple windows (unexploded). The Spark runner uses the DoFnRunner (Simple) to process DoFns, and it will explode

Re: Hosting data stores for IO Transform testing

Hi Stephen, the purpose of having in a specific module is to share resources and apply the same behavior from IT perspective and be able to have IT "cross" IO (for instance, reading from JMS and sending to Kafka, I think that's the key idea for integration tests). For instance, in Karaf, we

Re: HBase IO

Any update on which branch the preview for HBase IO might be available in?

Re: Hosting data stores for IO Transform testing

Hi Etienne, thanks for following up and answering my questions. re: where to store integration tests - having them all in a separate module is an interesting idea. I couldn't find JB's comments about moving them into a separate module in the PR - can you share the reasons for doing so? The IO

Re: Hbase IO preview

Hi Andrew, I have a protobuf issue on this IO that I would like to address. Sorry, I didn't have time to work on it this week. I do my best to push something work-able asap. Regards JB On 12/14/2016 03:18 PM, Andrew Hoblitzell wrote: Any update on which branch the preview for HBase IO