Are your DoFn's idempotent and don't rely on ordering of elements? Do you use any triggers?
Lots of things that can non-determinism to your output, need more details about what your pipeline does. Using smaller input datasets can help you track down the source of non-determinism. On Wed, Jul 13, 2016 at 3:09 PM, amir bahmanyari <[email protected]> wrote: > Hi Colleagues, > I am getting random results for: > - exact same data input > - exact same app binary > - exact same Flink cluster instances > Everything fixed, just repeat of running the something. > Every-time, I get a different result while data doesn't change, code > doesn't change, logic to calculate results is exact same... > > Is Beam "parallelism" playing a role due to something "un-usual" in my > code? > What could the "un-usual" be in the app that may make the Beam pipleline > produces different results for exact same "everything"? > Than+regards, > Amir- >
