I look at the gradle task definition to see what additional flags are being passed whenever trying to rerun/repro an issue. Many of our integration tests require additional flags/experiments which are unique on a runner per runner basis.
On Wed, Feb 19, 2020 at 9:15 AM Alex Van Boxel <a...@vanboxel.be> wrote: > That's a great idea. I'll do, as some changes needed to be done to the > gradle files as well > > On Wed, Feb 19, 2020, 17:52 Ismaël Mejía <ieme...@gmail.com> wrote: > >> Alex/Gleb can someone of you please add the detailed instructions that >> worked for you in some section of cwiki. >> I have the impression that this will benefit us all at some point. >> >> Thanks, >> >> >> >> On Tue, Feb 18, 2020 at 9:46 AM Alex Van Boxel <a...@vanboxel.be> wrote: >> >>> Thanks everyone. This really helped a lot. I used Gleb's tip to make it >>> work. Successfully validated my Pull Requests against Dataflow! >>> >>> _/ >>> _/ Alex Van Boxel >>> >>> >>> On Mon, Feb 17, 2020 at 11:55 PM Brian Hulette <bhule...@google.com> >>> wrote: >>> >>>> I think if you update past [1] this will go away. We had to build a new >>>> worker to use with builds on master after [2]. You should be fine running >>>> from master as long as you aren't using a commit between those two (merge >>>> commits are 6818560 and bde3031, respectively). Setting the Dataflow worker >>>> jar would work too. >>>> >>>> [1] https://github.com/apache/beam/pull/10861 >>>> [2] https://github.com/apache/beam/pull/10790 >>>> >>>> On Mon, Feb 17, 2020 at 2:14 AM Gleb Kanterov <g...@spotify.com> wrote: >>>> >>>>> You need to pass custom Dataflow worker jar. One of the ways of doing >>>>> that is adding it as a dependency, and using following code snippet: >>>>> >>>>> opts.setDataflowWorkerJar( >>>>> BatchDataflowWorker.class >>>>> .getProtectionDomain() >>>>> .getCodeSource() >>>>> .getLocation() >>>>> .toString()); >>>>> opts.setWorkerHarnessContainerImage(""); >>>>> >>>>> Coming with the disclaimer that it isn't for production :) >>>>> >>>>> On Mon, Feb 17, 2020 at 8:34 AM Alex Van Boxel <a...@vanboxel.be> >>>>> wrote: >>>>> >>>>>> Yes, running it manually with the normal parameters as I do for >>>>>> production Dataflow. I'm probably a bit ignorant on that, and I >>>>>> probably need to provide my own worker. >>>>>> >>>>>> Thanks for the hint... I'll dive into that. >>>>>> >>>>>> _/ >>>>>> _/ Alex Van Boxel >>>>>> >>>>>> >>>>>> On Mon, Feb 17, 2020 at 8:16 AM Reuven Lax <re...@google.com> wrote: >>>>>> >>>>>>> Are you running things manually? This probably means you are using >>>>>>> an out-of-date Dataflow worker. I believe that all tests on Jenkins will >>>>>>> build the Dataflow worker from head to prevent exactly this problem. >>>>>>> >>>>>>> On Sun, Feb 16, 2020 at 11:10 PM Alex Van Boxel <a...@vanboxel.be> >>>>>>> wrote: >>>>>>> >>>>>>>> Digging further in the traces, it seems like a result of changes to >>>>>>>> the model: >>>>>>>> >>>>>>>> Caused by: java.lang.ClassNotFoundException: >>>>>>>> org.apache.beam.model.pipeline.v1.StandardWindowFns$SessionsPayload$Enum >>>>>>>> >>>>>>>> I see changes by Lukasz Cwik. Will this be a problem for the >>>>>>>> release? >>>>>>>> >>>>>>>> _/ >>>>>>>> _/ Alex Van Boxel >>>>>>>> >>>>>>>> >>>>>>>> On Sun, Feb 16, 2020 at 12:11 PM Alex Van Boxel <a...@vanboxel.be> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hey, >>>>>>>>> >>>>>>>>> I'm testing my own PR's against Dataflow, something I've done in >>>>>>>>> the past with success seem to fail now. I get this error: >>>>>>>>> >>>>>>>>> java.lang.NoClassDefFoundError: Could not initialize class >>>>>>>>> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.construction.WindowingStrategyTranslation >>>>>>>>> >>>>>>>>> 1. >>>>>>>>> >>>>>>>>> >>>>>>>>> Am I doing something wrong? >>>>>>>>> >>>>>>>>> _/ >>>>>>>>> _/ Alex Van Boxel >>>>>>>>> >>>>>>>>
