+1 gradle not being mainstream - and even more in beam scope than other java scope - it must stay simple. I know that personally it takes me x2 or 3 to say "ok ill fork" a project using gradle, in particular with custom scripts and not only parent/module scripts.
If needed we can write custom plugins to simply it and hide the complexity and weird task naming can move to "standard" names with options/parameters maybe? Le 9 avr. 2018 20:27, "Kenneth Knowles" <k...@google.com> a écrit : > My phrasing sounded a bit too much like adding a blocking condition. I > absolutely don't want to do that. Our maven build was far from clear, and > had lots of tech debt and spooky action at a distance, > > I just want to emphatically agree with JB's sentiment that we have an > opportunity to improve it before it ossifies in the current state. Maybe we > won't really have time, and that's OK too and we can just improve as we go. > > Luke - I know we have different standards of clarity but I doubt we'll > have specific disagreements on cleanups to the build as they happen.* I was > mostly listing elementary programming concepts that apply to our build and > tend to get forgotten in configs & build files. Being "natural gradle" is > also important. Let's see how it goes. > > Kenn > > *FWIW I did know that we have some string-munged identifiers already, but > mostly I am speaking from experience beyond & before Beam too. I'll likely > tidy the examples build.gradle as I finish up nexmark in a similar way. > > On Mon, Apr 9, 2018 at 10:50 AM Lukasz Cwik <lc...@google.com> wrote: > >> JB, learning the build system in a project is hopefully avoided by most >> users if the contribution guide can clearly explain what users need to do. >> But for everyone else who wants to change a dependency version or add a >> dependency it should be as simple as copy/paste (which I believe it already >> is). Note that people who want to do anything more complicated like add new >> jenkins job configurations for new integration test targets needs to learn >> how the build system works, how maven profiles work, how to plumb the >> required set of attributes from Jenkins into the test target via the build >> system. >> >> Kenn, I have to disagree with a lot in 1 and 2. For users who are >> unfamiliar with Maven, we didn't setup the Maven build in such a way that >> anyone unfamiliar with Maven knew what was going on or try to use concepts >> unnatural to Maven in an attempt to make it seem easier. I believe we >> should stick with Gradle/Groovy conventions. Users who are not familiar >> with how Gradle/Groovy works will either ask the community or look at >> StackOverflow and doing things like passing the project around into >> functions is extremely uncommon compared to using the current context and >> applying closures to it. For users who are familiar with Gradle, the builds >> should be natural Gradle. >> >> *- If you are going to refer to an identifier, it should have an explicit >> point of origin (not strings smashed together in a loop)* >> I assume your referring to how the examples java precommit is done. It is >> the only case of it to my knowledge and extra hands to migrate it to be >> like how the validates runner tests run would be useful. Filed >> https://issues.apache.org/jira/browse/BEAM-4033 >> >> >> On Mon, Apr 9, 2018 at 1:24 PM Kenneth Knowles <k...@google.com> wrote: >> >>> Huge +1 to the idea of investing in simplification and polish of the >>> Gradle files before considering the migration complete. >>> >>> 1. build.gradle files should be as close to straight-line configuration >>> as possible: >>> - You should be able to understand a module's build easily, locally, >>> without knowing Groovy >>> - As little Groovy programming as possible, focused on providing >>> well-defined utilities >>> >>> 2. Explicit > implicit, especially for config files >>> - Ditto about being able to understand a module's build locally >>> - Minimize globally inherited config, in favor of explicitly calling >>> utilities >>> - Passing current project to a utility is better than trying to make >>> something the "closure" >>> - If you are going to refer to an identifier, it should have an >>> explicit point of origin (not strings smashed together in a loop) >>> >>> When in doubt, a little redundancy to improve readability is worth it in >>> this context. >>> >>> Kenn >>> >>> On Mon, Apr 9, 2018 at 9:47 AM Jean-Baptiste Onofré <j...@nanthrax.net> >>> wrote: >>> >>>> Hi all, >>>> >>>> I did multiple gradle build since last week and I would like to share >>>> one of my concern: it's about the communities. >>>> >>>> If I think our users won't see any change for them due to Gradle build >>>> (I think that most of our users will still use Maven with artifacts >>>> provided by Gradle), I'm more concerned by the dev community and the >>>> contribution. >>>> >>>> Maven is well known and straight forward for a large part of potential >>>> contributors. I think we have to keep in mind that we still have to grow >>>> up our contributors community. >>>> >>>> Today, maybe I'm wrong, but I have the feeling that gradle build is not >>>> straight forward (build.gradle includes build_rules.gradle, gathering >>>> all taks all together). >>>> >>>> I would like to add a task in the gradle "migration" process: simplify >>>> the gradle structure and files, and document this. >>>> >>>> I know we already have a Jira about the documentation part, but I would >>>> like to "polish" and use a clean structure for the Gradle resources. As >>>> already quickly discussed, I think that having one gradle file per tasks >>>> in the .gradle directory would be helpful. >>>> >>>> The goal is really to simplify the contribution. >>>> >>>> Do you agree if I add a Jira about "Gradle polish" ? >>>> Thoughts ? >>>> >>>> Regards >>>> JB >>>> >>>> On 07/04/2018 04:52, Scott Wegner wrote: >>>> > Here's an end-of-day update on migration work: >>>> > >>>> > * Snapshot unsigned dailies and signed release builds are working >>>> (!!). >>>> > PR/5048 [1] merges changes from Luke's branch >>>> > * python precommit failing... will investigate python precommit >>>> Monday >>>> > * All Precommits are gradle only >>>> > * All Postcommits except performance tests and Java_JDK_Versions_Test >>>> > use gradle (after PR/5047 [2] merged) >>>> > * Nightly snapshot release using gradle is ready; needs PR/5048 to be >>>> > merged before switching >>>> > * ValidatesRunner_Spark failing consistently; investigating >>>> > >>>> > Thanks for another productive day of hacking. I'll pick up again on >>>> Monday. >>>> > >>>> > [1] https://github.com/apache/beam/pull/5048 >>>> > [2] https://github.com/apache/beam/pull/5047 >>>> > >>>> > >>>> > On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau >>>> > <rmannibu...@gmail.com <mailto:rmannibu...@gmail.com>> wrote: >>>> > >>>> > Why building a zip per runner which its stack and just pointing >>>> out >>>> > on that zip and let beam lazy load the runner: >>>> > >>>> > --runner=LazyRunner --lazyRunnerDir=... --lazyRunnerOptions=... >>>> (or >>>> > the fromSystemProperties() if it gets merged a day ;)) >>>> > >>>> > Le 6 avr. 2018 20:21, "Kenneth Knowles" <k...@google.com >>>> > <mailto:k...@google.com>> a écrit : >>>> > >>>> > I'm working on finding a solution for launching the Nexmark >>>> > suite with each runner. This doesn't have to be done via >>>> Gradle, >>>> > but we anyhow need built artifacts that don't require user >>>> > classpath intervention. >>>> > >>>> > It looks to me like the examples are also missing this - they >>>> > have separate configuration e.g. sparkRunnerPreCommit but that >>>> > is overspecified compared to a free-form launching of a main() >>>> > program with a runner profile. >>>> > >>>> > On Fri, Apr 6, 2018 at 11:09 AM Lukasz Cwik <lc...@google.com >>>> > <mailto:lc...@google.com>> wrote: >>>> > >>>> > Romain, are you talking about the profiles that exist as >>>> > part of the archetype examples? >>>> > >>>> > If so, then those still exist and haven't been changed. If >>>> > not, can you provide a link to the profile in a pom file >>>> to >>>> > be clearer? >>>> > >>>> > On Fri, Apr 6, 2018 at 12:40 PM Romain Manni-Bucau >>>> > <rmannibu...@gmail.com <mailto:rmannibu...@gmail.com>> >>>> wrote: >>>> > >>>> > Hi Scott, >>>> > >>>> > is it right that 2 doesn't handle the hierachy anymore >>>> > and that it doesn't handle profiles for runners as it >>>> is >>>> > currently with maven? >>>> > >>>> > >>>> > Romain Manni-Bucau >>>> > @rmannibucau <https://twitter.com/rmannibucau> | Blog >>>> > <https://rmannibucau.metawerx.net/> | Old Blog >>>> > <http://rmannibucau.wordpress.com> | Github >>>> > <https://github.com/rmannibucau> | LinkedIn >>>> > <https://www.linkedin.com/in/rmannibucau> | Book >>>> > <https://www.packtpub.com/ >>>> application-development/java-ee-8-high-performance> >>>> > >>>> > 2018-04-06 18:32 GMT+02:00 Scott Wegner >>>> > <sweg...@google.com <mailto:sweg...@google.com>>: >>>> > >>>> > I wanted to start a thread to summarize the >>>> current >>>> > state of Gradle migration. We've made lots of good >>>> > progress so far this week. Here's the status from >>>> > what I can tell-- please add or correct anything I >>>> > missed: >>>> > >>>> > * Release artifacts can be built and published for >>>> > Snapshot and officlal releases [1] >>>> > * Gradle-generated releases have been validated >>>> with >>>> > the the Apache Beam archetype generation >>>> quickstart; >>>> > still needs additional validation. >>>> > * Generated release pom files have correct project >>>> > metadata [2] >>>> > * The python pre-commits are now working in >>>> Gradle [3] >>>> > * Ismaël has started a collaborative doc of Gradle >>>> > tips [4] as we all learn the new system-- please >>>> add >>>> > your own. This will eventually feed into official >>>> > documentation on the website. >>>> > * Łukasz Gajowy is working on migrating >>>> performance >>>> > testing framework [5] >>>> > * Daniel is working on updating documentation to >>>> > refer to Gradle instead of maven >>>> > >>>> > If I missed anything, please add it to this >>>> thread. >>>> > >>>> > The general roadmap we're working towards is: >>>> > (a) Publish release artifacts with Gradle >>>> (SNAPSHOT >>>> > and signed releases) >>>> > (b) Postcommits migrated to Gradle >>>> > (c) Migrate documentation from maven to Gradle >>>> > (d) Migrate perfkit suites to use Gradle >>>> > >>>> > For those of you that are hacking: thanks for your >>>> > help so far! Progress is being roughly tracked on >>>> > the Kanban [6]; please make sure the issues >>>> assigned >>>> > to you are up-to-date. Many of the changes are >>>> > staged on lukecwik's local branch [7]; we'll work >>>> on >>>> > merging them back soon. >>>> > >>>> > >>>> > [1] https://github.com/lukecwik/ >>>> incubator-beam/pull/7 >>>> > [2] https://github.com/lukecwik/ >>>> incubator-beam/pull/3 >>>> > [3] https://github.com/apache/beam/pull/5032 >>>> > [4] >>>> > https://docs.google.com/document/d/ >>>> 1wR56Jef3XIPwj4DFzQKznuGPM3JDfRDVkxzeDlbdVSQ/edit >>>> > [5] https://github.com/apache/beam/pull/5003 >>>> > [6] >>>> > https://issues.apache.org/ >>>> jira/secure/RapidBoard.jspa?rapidView=242 >>>> > >>>> > [7] >>>> > https://github.com/lukecwik/ >>>> incubator-beam/tree/gradle >>>> > -- >>>> > >>>> > >>>> > Got feedback? http://go/swegner-feedback >>>> > >>>> > >>>> > -- >>>> > >>>> > >>>> > Got feedback? http://go/swegner-feedback >>>> >>>
