The dataflow java worker version wasn't updated on the branch as in past releases ... should it be? https://issues.apache.org/jira/browse/BEAM-3815
On Thu, Mar 8, 2018 at 1:40 PM Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > Can still be provided as a generic one (like the an offset or key based > one) but good enough for now, right, was just surprising to not see it when > checking the breakage. > > Le 8 mars 2018 22:05, "Eugene Kirpichov" <kirpic...@google.com> a écrit : > > All SDF-related method annotations in DoFn are marked @Experimental. I > guess that should apply to RestrictionTracker too, but I wouldn't be too > worried about that, since it only makes sense to use in the context of > those methods. > > On Thu, Mar 8, 2018 at 12:36 PM Romain Manni-Bucau <rmannibu...@gmail.com> > wrote: > >> Hmm, does sdf api misses some @Experimental then? >> >> To clarify: for waitUntilFinish I'm fine with the 2.4 as this but cant +1 >> or +0 since none of my tests pass reliably in current state without a retry >> strategy making the call useless. >> >> Le 8 mars 2018 21:02, "Reuven Lax" <re...@google.com> a écrit : >> >>> Does Nexmark use SerializableCoder? >>> >>> >>> On Thu, Mar 8, 2018 at 10:42 AM Robert Bradshaw <rober...@google.com> >>> wrote: >>> >>>> I put the validation checklist spreadsheet is up at >>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit?ts=5a1c7310#gid=1663314475 >>>> >>>> Regarding the direct runner regression on query 10, this is >>>> understandable given how mutation detection has been changed for >>>> serializable coders (and should be tracked, probably fixed by avoiding >>>> SerializableCoder). It should not affect other runners. Could you file a >>>> bug? >>>> >>>> Regarding waitUntilFinish, this is a bug but not a blocker--it's been >>>> this way since teardown was introduced. There are many nice-to-haves that >>>> one could merge from master to the release branch, but we've seen where >>>> that trend leads. >>>> >>>> Regarding the backwards incompatible changes in restriction tracker, >>>> this is (as I understand it) a change to the experimental SDF API. Eugene, >>>> do you want to comment on this? >>>> >>>> >>>> >>>> On Thu, Mar 8, 2018 at 2:07 AM Ismaël Mejía <ieme...@gmail.com> wrote: >>>> >>>>> I confirm that the new release fixes both problems reported previously: >>>>> >>>>> - python package name >>>>> - nexmark query 10 mutability issue with the direct runner. >>>>> >>>>> One extra regression is that the the fix produced a way longer >>>>> execution time on the query. >>>>> Not sure if a blocker but worth tracking. >>>>> >>>>> Query 10 - Batch/Bounded >>>>> Version Runtime(sec) Events(/sec) Results >>>>> 2.3.0 3.6 27609.1 1 >>>>> 2.4.0 30.8 3244.3 1 >>>>> >>>>> Query 10 - Streaming/Unbounded >>>>> Version Runtime(sec) Events(/sec) Results >>>>> 2.3.0 6.3 15873.0 1 >>>>> 2.4.0 101.1 989.4 1 >>>>> >>>>> On Thu, Mar 8, 2018 at 8:54 AM, Romain Manni-Bucau >>>>> <rmannibu...@gmail.com> wrote: >>>>> > -1: >>>>> > a) still consider waitUntilFinish broken and a big blocker >>>>> > b) restrictiontracker api changed and is not backward compatible >>>>> > ( >>>>> https://github.com/apache/beam/commit/e0034314ad196d2274cef9831ed63e090bf4d4c1#diff-098d7247eb1e9d9423bfa2ae2da38a9d >>>>> ) >>>>> > >>>>> > with workarounds and fixes for these two issues the other parts work >>>>> (spark, >>>>> > flink, direct runner, java core) on my projects >>>>> > >>>>> > >>>>> > >>>>> > Romain Manni-Bucau >>>>> > @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book >>>>> > >>>>> > 2018-03-08 6:26 GMT+01:00 Robert Bradshaw <rober...@google.com>: >>>>> >> >>>>> >> Hi everyone, >>>>> >> >>>>> >> Please review and vote on the release candidate #2 for the version >>>>> 2.4.0, >>>>> >> as follows: >>>>> >> [ ] +1, Approve the release >>>>> >> [ ] -1, Do not approve the release (please provide specific >>>>> comments) >>>>> >> >>>>> >> The complete staging area is available for your review, which >>>>> includes: >>>>> >> * JIRA release notes [1], >>>>> >> * the official Apache source release to be deployed to >>>>> dist.apache.org >>>>> >> [2], >>>>> >> which is signed with the key with fingerprint BDC9 89B0 1BD2 A463 >>>>> 6010 >>>>> >> A1CA 8F15 5E09 610D 69FB [3], >>>>> >> * all artifacts to be deployed to the Maven Central Repository [4], >>>>> >> * source code tag "v2.4.0-RC2" [5], >>>>> >> * website pull request listing the release and publishing the API >>>>> >> reference >>>>> >> manual [6]. >>>>> >> * Java artifacts were built with Maven 3.2.5 and OpenJDK 1.8.0_112. >>>>> >> * Python artifact are deployed along with the source release to the >>>>> >> dist.apache.org [2]. If I am able to figure out how to build the >>>>> wheels, I >>>>> >> will post them there as well. >>>>> >> >>>>> >> The vote will be open for at least 72 hours. It is adopted by >>>>> majority >>>>> >> approval, with at least 3 PMC affirmative votes. >>>>> >> >>>>> >> Thanks, >>>>> >> - Robert >>>>> >> >>>>> >> [1] >>>>> >> >>>>> >> >>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12342682&projectId=12319527 >>>>> >> [2] https://dist.apache.org/repos/dist/dev/beam/2.4.0/ >>>>> >> [3] https://dist.apache.org/repos/dist/dev/beam/KEYS >>>>> >> [4] >>>>> https://repository.apache.org/content/repositories/orgapachebeam-1030/ >>>>> >> [5] https://github.com/apache/beam/tree/v2.4.0-RC2 >>>>> >> [6] https://github.com/apache/beam-site/pull/398 >>>>> > >>>>> > >>>>> >>>> >
