Here's an end-of-day update on migration work: * Snapshot unsigned dailies and signed release builds are working (!!). PR/5048 [1] merges changes from Luke's branch * python precommit failing... will investigate python precommit Monday * All Precommits are gradle only * All Postcommits except performance tests and Java_JDK_Versions_Test use gradle (after PR/5047 [2] merged) * Nightly snapshot release using gradle is ready; needs PR/5048 to be merged before switching * ValidatesRunner_Spark failing consistently; investigating
Thanks for another productive day of hacking. I'll pick up again on Monday. [1] https://github.com/apache/beam/pull/5048 [2] https://github.com/apache/beam/pull/5047 On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > Why building a zip per runner which its stack and just pointing out on > that zip and let beam lazy load the runner: > > --runner=LazyRunner --lazyRunnerDir=... --lazyRunnerOptions=... (or the > fromSystemProperties() if it gets merged a day ;)) > > Le 6 avr. 2018 20:21, "Kenneth Knowles" <k...@google.com> a écrit : > >> I'm working on finding a solution for launching the Nexmark suite with >> each runner. This doesn't have to be done via Gradle, but we anyhow need >> built artifacts that don't require user classpath intervention. >> >> It looks to me like the examples are also missing this - they have >> separate configuration e.g. sparkRunnerPreCommit but that is overspecified >> compared to a free-form launching of a main() program with a runner profile. >> >> On Fri, Apr 6, 2018 at 11:09 AM Lukasz Cwik <lc...@google.com> wrote: >> >>> Romain, are you talking about the profiles that exist as part of the >>> archetype examples? >>> >>> If so, then those still exist and haven't been changed. If not, can you >>> provide a link to the profile in a pom file to be clearer? >>> >>> On Fri, Apr 6, 2018 at 12:40 PM Romain Manni-Bucau < >>> rmannibu...@gmail.com> wrote: >>> >>>> Hi Scott, >>>> >>>> is it right that 2 doesn't handle the hierachy anymore and that it >>>> doesn't handle profiles for runners as it is currently with maven? >>>> >>>> >>>> Romain Manni-Bucau >>>> @rmannibucau <https://twitter.com/rmannibucau> | Blog >>>> <https://rmannibucau.metawerx.net/> | Old Blog >>>> <http://rmannibucau.wordpress.com> | Github >>>> <https://github.com/rmannibucau> | LinkedIn >>>> <https://www.linkedin.com/in/rmannibucau> | Book >>>> <https://www.packtpub.com/application-development/java-ee-8-high-performance> >>>> >>>> 2018-04-06 18:32 GMT+02:00 Scott Wegner <sweg...@google.com>: >>>> >>>>> I wanted to start a thread to summarize the current state of Gradle >>>>> migration. We've made lots of good progress so far this week. Here's the >>>>> status from what I can tell-- please add or correct anything I missed: >>>>> >>>>> * Release artifacts can be built and published for Snapshot and >>>>> officlal releases [1] >>>>> * Gradle-generated releases have been validated with the the Apache >>>>> Beam archetype generation quickstart; still needs additional validation. >>>>> * Generated release pom files have correct project metadata [2] >>>>> * The python pre-commits are now working in Gradle [3] >>>>> * Ismaël has started a collaborative doc of Gradle tips [4] as we all >>>>> learn the new system-- please add your own. This will eventually feed into >>>>> official documentation on the website. >>>>> * Łukasz Gajowy is working on migrating performance testing framework >>>>> [5] >>>>> * Daniel is working on updating documentation to refer to Gradle >>>>> instead of maven >>>>> >>>>> If I missed anything, please add it to this thread. >>>>> >>>>> The general roadmap we're working towards is: >>>>> (a) Publish release artifacts with Gradle (SNAPSHOT and signed >>>>> releases) >>>>> (b) Postcommits migrated to Gradle >>>>> (c) Migrate documentation from maven to Gradle >>>>> (d) Migrate perfkit suites to use Gradle >>>>> >>>>> For those of you that are hacking: thanks for your help so far! >>>>> Progress is being roughly tracked on the Kanban [6]; please make sure the >>>>> issues assigned to you are up-to-date. Many of the changes are staged on >>>>> lukecwik's local branch [7]; we'll work on merging them back soon. >>>>> >>>>> >>>>> [1] https://github.com/lukecwik/incubator-beam/pull/7 >>>>> [2] https://github.com/lukecwik/incubator-beam/pull/3 >>>>> [3] https://github.com/apache/beam/pull/5032 >>>>> [4] >>>>> https://docs.google.com/document/d/1wR56Jef3XIPwj4DFzQKznuGPM3JDfRDVkxzeDlbdVSQ/edit >>>>> [5] https://github.com/apache/beam/pull/5003 >>>>> [6] >>>>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=242 >>>>> [7] https://github.com/lukecwik/incubator-beam/tree/gradle >>>>> -- >>>>> >>>>> >>>>> Got feedback? http://go/swegner-feedback >>>>> >>>> >>>> -- Got feedback? http://go/swegner-feedback
