Le 9 avr. 2018 21:04, "Lukasz Cwik" <lc...@google.com> a écrit :

Romain, can you clarify by the weird task naming? (Give some examples using
our current project and Gradle and what you would have expected.)


Sure

https://github.com/apache/beam/blob/fb6ba3bfd43605a0f4944828b5f19e75840fa7aa/examples/java/build.gradle#L78
for instance

It is way more natural to have one generic command you can select a runner
(including a csv value) than N commands for this kind of things.

Gradle allows to define custom tasks and "scopes". This must not be used at
Xtrem until justifiable. We now have a lot of configurations not making
much sense too until you scripted yourself the script or know beam well.
This doesnt enable contribs :(.

Hope it is clearer


On Mon, Apr 9, 2018 at 2:49 PM Romain Manni-Bucau <rmannibu...@gmail.com>
wrote:

> +1 gradle not being mainstream - and even more in beam scope than other
> java scope - it must stay simple. I know that personally it takes me x2 or
> 3 to say "ok ill fork" a project using gradle, in particular with custom
> scripts and not only parent/module scripts.
>
> If needed we can write custom plugins to simply it and hide the complexity
> and weird task naming can move to "standard" names with options/parameters
> maybe?
>
> Le 9 avr. 2018 20:27, "Kenneth Knowles" <k...@google.com> a écrit :
>
>> My phrasing sounded a bit too much like adding a blocking condition. I
>> absolutely don't want to do that. Our maven build was far from clear, and
>> had lots of tech debt and spooky action at a distance,
>>
>> I just want to emphatically agree with JB's sentiment that we have an
>> opportunity to improve it before it ossifies in the current state. Maybe we
>> won't really have time, and that's OK too and we can just improve as we go.
>>
>> Luke - I know we have different standards of clarity but I doubt we'll
>> have specific disagreements on cleanups to the build as they happen.* I was
>> mostly listing elementary programming concepts that apply to our build and
>> tend to get forgotten in configs & build files. Being "natural gradle" is
>> also important. Let's see how it goes.
>>
>> Kenn
>>
>> *FWIW I did know that we have some string-munged identifiers already, but
>> mostly I am speaking from experience beyond & before Beam too. I'll likely
>> tidy the examples build.gradle as I finish up nexmark in a similar way.
>>
>> On Mon, Apr 9, 2018 at 10:50 AM Lukasz Cwik <lc...@google.com> wrote:
>>
>>> JB, learning the build system in a project is hopefully avoided by most
>>> users if the contribution guide can clearly explain what users need to do.
>>> But for everyone else who wants to change a dependency version or add a
>>> dependency it should be as simple as copy/paste (which I believe it already
>>> is). Note that people who want to do anything more complicated like add new
>>> jenkins job configurations for new integration test targets needs to learn
>>> how the build system works, how maven profiles work, how to plumb the
>>> required set of attributes from Jenkins into the test target via the build
>>> system.
>>>
>>> Kenn, I have to disagree with a lot in 1 and 2. For users who are
>>> unfamiliar with Maven, we didn't setup the Maven build in such a way that
>>> anyone unfamiliar with Maven knew what was going on or try to use concepts
>>> unnatural to Maven in an attempt to make it seem easier. I believe we
>>> should stick with Gradle/Groovy conventions. Users who are not familiar
>>> with how Gradle/Groovy works will either ask the community or look at
>>> StackOverflow and doing things like passing the project around into
>>> functions is extremely uncommon compared to using the current context and
>>> applying closures to it. For users who are familiar with Gradle, the builds
>>> should be natural Gradle.
>>>
>>> *- If you are going to refer to an identifier, it should have an
>>> explicit point of origin (not strings smashed together in a loop)*
>>> I assume your referring to how the examples java precommit is done. It
>>> is the only case of it to my knowledge and extra hands to migrate it to be
>>> like how the validates runner tests run would be useful. Filed
>>> https://issues.apache.org/jira/browse/BEAM-4033
>>>
>>>
>>> On Mon, Apr 9, 2018 at 1:24 PM Kenneth Knowles <k...@google.com> wrote:
>>>
>>>> Huge +1 to the idea of investing in simplification and polish of the
>>>> Gradle files before considering the migration complete.
>>>>
>>>> 1. build.gradle files should be as close to straight-line configuration
>>>> as possible:
>>>>  - You should be able to understand a module's build easily, locally,
>>>> without knowing Groovy
>>>>  - As little Groovy programming as possible, focused on providing
>>>> well-defined utilities
>>>>
>>>> 2. Explicit > implicit, especially for config files
>>>>  - Ditto about being able to understand a module's build locally
>>>>  - Minimize globally inherited config, in favor of explicitly calling
>>>> utilities
>>>>  - Passing current project to a utility is better than trying to make
>>>> something the "closure"
>>>>  - If you are going to refer to an identifier, it should have an
>>>> explicit point of origin (not strings smashed together in a loop)
>>>>
>>>> When in doubt, a little redundancy to improve readability is worth it
>>>> in this context.
>>>>
>>>> Kenn
>>>>
>>>> On Mon, Apr 9, 2018 at 9:47 AM Jean-Baptiste Onofré <j...@nanthrax.net>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I did multiple gradle build since last week and I would like to share
>>>>> one of my concern: it's about the communities.
>>>>>
>>>>> If I think our users won't see any change for them due to Gradle build
>>>>> (I think that most of our users will still use Maven with artifacts
>>>>> provided by Gradle), I'm more concerned by the dev community and the
>>>>> contribution.
>>>>>
>>>>> Maven is well known and straight forward for a large part of potential
>>>>> contributors. I think we have to keep in mind that we still have to
>>>>> grow
>>>>> up our contributors community.
>>>>>
>>>>> Today, maybe I'm wrong, but I have the feeling that gradle build is not
>>>>> straight forward (build.gradle includes build_rules.gradle, gathering
>>>>> all taks all together).
>>>>>
>>>>> I would like to add a task in the gradle "migration" process: simplify
>>>>> the gradle structure and files, and document this.
>>>>>
>>>>> I know we already have a Jira about the documentation part, but I would
>>>>> like to "polish" and use a clean structure for the Gradle resources. As
>>>>> already quickly discussed, I think that having one gradle file per
>>>>> tasks
>>>>> in the .gradle directory would be helpful.
>>>>>
>>>>> The goal is really to simplify the contribution.
>>>>>
>>>>> Do you agree if I add a Jira about "Gradle polish" ?
>>>>> Thoughts ?
>>>>>
>>>>> Regards
>>>>> JB
>>>>>
>>>>> On 07/04/2018 04:52, Scott Wegner wrote:
>>>>> > Here's an end-of-day update on migration work:
>>>>> >
>>>>> > * Snapshot unsigned dailies and signed release builds are working
>>>>> (!!).
>>>>> > PR/5048 [1] merges changes from Luke's branch
>>>>> >    * python precommit failing... will investigate python precommit
>>>>> Monday
>>>>> > * All Precommits are gradle only
>>>>> > * All Postcommits except performance tests and Java_JDK_Versions_Test
>>>>> > use gradle (after PR/5047 [2] merged)
>>>>> > * Nightly snapshot release using gradle is ready; needs PR/5048 to be
>>>>> > merged before switching
>>>>> > * ValidatesRunner_Spark failing consistently; investigating
>>>>> >
>>>>> > Thanks for another productive day of hacking. I'll pick up again on
>>>>> Monday.
>>>>> >
>>>>> > [1] https://github.com/apache/beam/pull/5048
>>>>> > [2] https://github.com/apache/beam/pull/5047
>>>>> >
>>>>> >
>>>>> > On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau
>>>>> > <rmannibu...@gmail.com <mailto:rmannibu...@gmail.com>> wrote:
>>>>> >
>>>>> >     Why building a zip per runner which its stack and just pointing
>>>>> out
>>>>> >     on that zip and let beam lazy load the runner:
>>>>> >
>>>>> >     --runner=LazyRunner --lazyRunnerDir=... --lazyRunnerOptions=...
>>>>> (or
>>>>> >     the fromSystemProperties() if it gets merged a day ;))
>>>>> >
>>>>> >     Le 6 avr. 2018 20:21, "Kenneth Knowles" <k...@google.com
>>>>> >     <mailto:k...@google.com>> a écrit :
>>>>> >
>>>>> >         I'm working on finding a solution for launching the Nexmark
>>>>> >         suite with each runner. This doesn't have to be done via
>>>>> Gradle,
>>>>> >         but we anyhow need built artifacts that don't require user
>>>>> >         classpath intervention.
>>>>> >
>>>>> >         It looks to me like the examples are also missing this - they
>>>>> >         have separate configuration e.g. sparkRunnerPreCommit but
>>>>> that
>>>>> >         is overspecified compared to a free-form launching of a
>>>>> main()
>>>>> >         program with a runner profile.
>>>>> >
>>>>> >         On Fri, Apr 6, 2018 at 11:09 AM Lukasz Cwik <
>>>>> lc...@google.com
>>>>> >         <mailto:lc...@google.com>> wrote:
>>>>> >
>>>>> >             Romain, are you talking about the profiles that exist as
>>>>> >             part of the archetype examples?
>>>>> >
>>>>> >             If so, then those still exist and haven't been changed.
>>>>> If
>>>>> >             not, can you provide a link to the profile in a pom file
>>>>> to
>>>>> >             be clearer?
>>>>> >
>>>>> >             On Fri, Apr 6, 2018 at 12:40 PM Romain Manni-Bucau
>>>>> >             <rmannibu...@gmail.com <mailto:rmannibu...@gmail.com>>
>>>>> wrote:
>>>>> >
>>>>> >                 Hi Scott,
>>>>> >
>>>>> >                 is it right that 2 doesn't handle the hierachy
>>>>> anymore
>>>>> >                 and that it doesn't handle profiles for runners as
>>>>> it is
>>>>> >                 currently with maven?
>>>>> >
>>>>> >
>>>>> >                 Romain Manni-Bucau
>>>>> >                 @rmannibucau <https://twitter.com/rmannibucau> |
>>>>> Blog
>>>>> >                 <https://rmannibucau.metawerx.net/> | Old Blog
>>>>> >                 <http://rmannibucau.wordpress.com> | Github
>>>>> >                 <https://github.com/rmannibucau> | LinkedIn
>>>>> >                 <https://www.linkedin.com/in/rmannibucau> | Book
>>>>> >                 <https://www.packtpub.com/
>>>>> application-development/java-ee-8-high-performance>
>>>>> >
>>>>> >                 2018-04-06 18:32 GMT+02:00 Scott Wegner
>>>>> >                 <sweg...@google.com <mailto:sweg...@google.com>>:
>>>>> >
>>>>> >                     I wanted to start a thread to summarize the
>>>>> current
>>>>> >                     state of Gradle migration. We've made lots of
>>>>> good
>>>>> >                     progress so far this week. Here's the status from
>>>>> >                     what I can tell-- please add or correct anything
>>>>> I
>>>>> >                     missed:
>>>>> >
>>>>> >                     * Release artifacts can be built and published
>>>>> for
>>>>> >                     Snapshot and officlal releases [1]
>>>>> >                     * Gradle-generated releases have been validated
>>>>> with
>>>>> >                     the the Apache Beam archetype generation
>>>>> quickstart;
>>>>> >                     still needs additional validation.
>>>>> >                     * Generated release pom files have correct
>>>>> project
>>>>> >                     metadata [2]
>>>>> >                     * The python pre-commits are now working in
>>>>> Gradle [3]
>>>>> >                     * Ismaël has started a collaborative doc of
>>>>> Gradle
>>>>> >                     tips [4] as we all learn the new system-- please
>>>>> add
>>>>> >                     your own. This will eventually feed into official
>>>>> >                     documentation on the website.
>>>>> >                     * Łukasz Gajowy is working on migrating
>>>>> performance
>>>>> >                     testing framework [5]
>>>>> >                     * Daniel is working on updating documentation to
>>>>> >                     refer to Gradle instead of maven
>>>>> >
>>>>> >                     If I missed anything, please add it to this
>>>>> thread.
>>>>> >
>>>>> >                     The general roadmap we're working towards is:
>>>>> >                     (a) Publish release artifacts with Gradle
>>>>> (SNAPSHOT
>>>>> >                     and signed releases)
>>>>> >                     (b) Postcommits migrated to Gradle
>>>>> >                     (c) Migrate documentation from maven to Gradle
>>>>> >                     (d) Migrate perfkit suites to use Gradle
>>>>> >
>>>>> >                     For those of you that are hacking: thanks for
>>>>> your
>>>>> >                     help so far! Progress is being roughly tracked on
>>>>> >                     the Kanban [6]; please make sure the issues
>>>>> assigned
>>>>> >                     to you are up-to-date. Many of the changes are
>>>>> >                     staged on lukecwik's local branch [7]; we'll
>>>>> work on
>>>>> >                     merging them back soon.
>>>>> >
>>>>> >
>>>>> >                     [1] https://github.com/lukecwik/
>>>>> incubator-beam/pull/7
>>>>> >                     [2] https://github.com/lukecwik/
>>>>> incubator-beam/pull/3
>>>>> >                     [3] https://github.com/apache/beam/pull/5032
>>>>> >                     [4]
>>>>> >                     https://docs.google.com/document/d/
>>>>> 1wR56Jef3XIPwj4DFzQKznuGPM3JDfRDVkxzeDlbdVSQ/edit
>>>>> >                     [5] https://github.com/apache/beam/pull/5003
>>>>> >                     [6]
>>>>> >                     https://issues.apache.org/
>>>>> jira/secure/RapidBoard.jspa?rapidView=242
>>>>> >
>>>>> >                     [7]
>>>>> >                     https://github.com/lukecwik/
>>>>> incubator-beam/tree/gradle
>>>>> >                     --
>>>>> >
>>>>> >
>>>>> >                     Got feedback? http://go/swegner-feedback
>>>>> >
>>>>> >
>>>>> > --
>>>>> >
>>>>> >
>>>>> > Got feedback? http://go/swegner-feedback
>>>>>
>>>>

Reply via email to