Alex/Gleb can someone of you please add the detailed instructions that worked for you in some section of cwiki. I have the impression that this will benefit us all at some point.
Thanks, On Tue, Feb 18, 2020 at 9:46 AM Alex Van Boxel <a...@vanboxel.be> wrote: > Thanks everyone. This really helped a lot. I used Gleb's tip to make it > work. Successfully validated my Pull Requests against Dataflow! > > _/ > _/ Alex Van Boxel > > > On Mon, Feb 17, 2020 at 11:55 PM Brian Hulette <bhule...@google.com> > wrote: > >> I think if you update past [1] this will go away. We had to build a new >> worker to use with builds on master after [2]. You should be fine running >> from master as long as you aren't using a commit between those two (merge >> commits are 6818560 and bde3031, respectively). Setting the Dataflow worker >> jar would work too. >> >> [1] https://github.com/apache/beam/pull/10861 >> [2] https://github.com/apache/beam/pull/10790 >> >> On Mon, Feb 17, 2020 at 2:14 AM Gleb Kanterov <g...@spotify.com> wrote: >> >>> You need to pass custom Dataflow worker jar. One of the ways of doing >>> that is adding it as a dependency, and using following code snippet: >>> >>> opts.setDataflowWorkerJar( >>> BatchDataflowWorker.class >>> .getProtectionDomain() >>> .getCodeSource() >>> .getLocation() >>> .toString()); >>> opts.setWorkerHarnessContainerImage(""); >>> >>> Coming with the disclaimer that it isn't for production :) >>> >>> On Mon, Feb 17, 2020 at 8:34 AM Alex Van Boxel <a...@vanboxel.be> wrote: >>> >>>> Yes, running it manually with the normal parameters as I do for >>>> production Dataflow. I'm probably a bit ignorant on that, and I >>>> probably need to provide my own worker. >>>> >>>> Thanks for the hint... I'll dive into that. >>>> >>>> _/ >>>> _/ Alex Van Boxel >>>> >>>> >>>> On Mon, Feb 17, 2020 at 8:16 AM Reuven Lax <re...@google.com> wrote: >>>> >>>>> Are you running things manually? This probably means you are using an >>>>> out-of-date Dataflow worker. I believe that all tests on Jenkins will >>>>> build >>>>> the Dataflow worker from head to prevent exactly this problem. >>>>> >>>>> On Sun, Feb 16, 2020 at 11:10 PM Alex Van Boxel <a...@vanboxel.be> >>>>> wrote: >>>>> >>>>>> Digging further in the traces, it seems like a result of changes to >>>>>> the model: >>>>>> >>>>>> Caused by: java.lang.ClassNotFoundException: >>>>>> org.apache.beam.model.pipeline.v1.StandardWindowFns$SessionsPayload$Enum >>>>>> >>>>>> I see changes by Lukasz Cwik. Will this be a problem for the release? >>>>>> >>>>>> _/ >>>>>> _/ Alex Van Boxel >>>>>> >>>>>> >>>>>> On Sun, Feb 16, 2020 at 12:11 PM Alex Van Boxel <a...@vanboxel.be> >>>>>> wrote: >>>>>> >>>>>>> Hey, >>>>>>> >>>>>>> I'm testing my own PR's against Dataflow, something I've done in the >>>>>>> past with success seem to fail now. I get this error: >>>>>>> >>>>>>> java.lang.NoClassDefFoundError: Could not initialize class >>>>>>> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.construction.WindowingStrategyTranslation >>>>>>> >>>>>>> 1. >>>>>>> >>>>>>> >>>>>>> Am I doing something wrong? >>>>>>> >>>>>>> _/ >>>>>>> _/ Alex Van Boxel >>>>>>> >>>>>>
