There is a fake module xxx_test which should have the right classpath but
since idea compilation is messed up you will still have to run
:module:cleanTest :module:test --tests org...MyTest.myMethod, even with
idea which leads to the same latency as the command line :(

Le 13 avr. 2018 22:23, "Eugene Kirpichov" <kirpic...@google.com> a écrit :

> While we're talking about running tests in IntelliJ with Gradle...
> Anybody got advice on how to run a single NeedsRunner test in
> sdks-java-core, say, ParDoTest? With Maven, I used to just run the test in
> IntelliJ and specify "runners-direct-java" as the classpath; with Gradle,
> the best I got is to manually run the direct runner's needsRunnerTests task
> with specifying --tests=..., but it takes a long time, and IntelliJ treats
> that as just a gradle task run, not as a test run.
>
> On Fri, Apr 13, 2018 at 11:14 AM Reuven Lax <re...@google.com> wrote:
>
>> Is there a Jira for this 3 second delay? Also you're initial complaint
>> was not about the 3 second delay, so it wasn't clear that's what you were
>> complaining about.
>>
>> Reuven
>>
>> On Fri, Apr 13, 2018 at 4:42 AM Romain Manni-Bucau <rmannibu...@gmail.com>
>> wrote:
>>
>>> When you launch a test with gradle runner it launches gradle which makes
>>> loose 3s on a very fast computer and more on a slower (6 on my personal one
>>> which is already fast but not as much as my work one). We are 5 to see that
>>> regression at least. So there is a reason to not use the gradle runner if
>>> possible cause when you work and need to debug you are just stucked (that
>>> is why i switched back to mvn after 15mn, i was loosing to much time).
>>>
>>> Switching back to native idea test run would fix it but tests just dont
>>> work this way for me whatever setup i do :( - missing resources IIRC in
>>> idea out dir.
>>>
>>> Le 13 avr. 2018 00:07, "Reuven Lax" <re...@google.com> a écrit :
>>>
>>> I also don't quite understand what your question is, and it appears like
>>> Dan spent considerable time trying to reproduce your issue. For the record,
>>> I have had no issues running tests via Gradle in IntelliJ for the past few
>>> weeks.
>>>
>>> Reuven
>>>
>>> On Thu, Apr 12, 2018 at 9:47 PM Daniel Oliveira <danolive...@google.com>
>>> wrote:
>>>
>>>> Sorry Romain, I'm not quite sure what you're asking. Can you clarify?
>>>>
>>>> On Thu, Apr 12, 2018 at 12:22 PM Romain Manni-Bucau <
>>>> rmannibu...@gmail.com> wrote:
>>>>
>>>>> Well you are the only one to not have the drawbacks to use it so maybe
>>>>> dont do it? I know Luke is in holidays but anyone else with the knowledge
>>>>> of why we nees that noise compared to idea native tooling/flow?
>>>>>
>>>>> Le 12 avr. 2018 20:16, "Daniel Oliveira" <danolive...@google.com> a
>>>>> écrit :
>>>>>
>>>>>> Ah, I did not. Thanks Romain.
>>>>>>
>>>>>> I tried it again, restarting in between, and still had no
>>>>>> differences. Since it seems like there's no reason not to use "Gradle 
>>>>>> Test
>>>>>> Runner", I'll mention it in the contributor's guide.
>>>>>>
>>>>>> On Thu, Apr 12, 2018 at 10:31 AM Romain Manni-Bucau <
>>>>>> rmannibu...@gmail.com> wrote:
>>>>>>
>>>>>>> @Daniel: did you restart in between? Otherwise it does nothing. One
>>>>>>> launches JunitCoreRunner from idea and the other a gradle command.
>>>>>>>
>>>>>>> Le 12 avr. 2018 19:24, "Daniel Oliveira" <danolive...@google.com> a
>>>>>>> écrit :
>>>>>>>
>>>>>>>> I think it depends on what exactly switching to "Gradle Test
>>>>>>>> Runner" from "Platform Test Runner" does. I tried it out on my machine 
>>>>>>>> and
>>>>>>>> they seem to act identically to each other. The IntelliJ documentation 
>>>>>>>> says
>>>>>>>> it determines what API to use to run the tests
>>>>>>>> <https://www.jetbrains.com/help/idea/runner.html>, so maybe it's
>>>>>>>> usefulness depends on the user's machine, in which case a note about 
>>>>>>>> that
>>>>>>>> would be useful. Something like: "If your IDE has trouble running 
>>>>>>>> tests via
>>>>>>>> IDEA shortcuts, try the following steps: [...]"
>>>>>>>>
>>>>>>>> On Thu, Apr 12, 2018 at 3:29 AM Alexey Romanenko <
>>>>>>>> aromanenko....@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Daniel, actually I did run it with default IDEA JUnit test runner.
>>>>>>>>> Then, in “Settings > Build, Execution, Deployment > Build Tools > 
>>>>>>>>> Gradle >
>>>>>>>>> Runner" I selected “Gradle Test Runner” in “Run tests using” 
>>>>>>>>> selectbox and
>>>>>>>>> it works ok when I run my tests with IDEA shortcuts. So, probably, we
>>>>>>>>> should add this details on https://beam.apache.org/
>>>>>>>>> contribute/intellij/ too.
>>>>>>>>> What do you think?
>>>>>>>>>
>>>>>>>>> WBR,
>>>>>>>>> Alexey
>>>>>>>>>
>>>>>>>>> On 11 Apr 2018, at 21:17, Daniel Oliveira <danolive...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Alexey, are you referring to tests run with "./gradlew
>>>>>>>>> :beam-runners-direct-java:needsRunnerTests"? That command works
>>>>>>>>> fine for me in both versions of IDEA, but I believe the same tests 
>>>>>>>>> fail if
>>>>>>>>> you run them directly through "./gradlew test".
>>>>>>>>>
>>>>>>>>> However, I am having issues with a bunch of validatesRunner tests,
>>>>>>>>> mostly be caused by 
>>>>>>>>> :beam-runners-google-cloud-dataflow-java:validatesRunner.
>>>>>>>>> Not sure if it's because of a code change or a gradle config, I'll 
>>>>>>>>> keep
>>>>>>>>> looking into it.
>>>>>>>>>
>>>>>>>>> On Wed, Apr 11, 2018 at 11:01 AM Romain Manni-Bucau <
>>>>>>>>> rmannibu...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> I got tests running rrconfiguring gradle (which was setup for
>>>>>>>>>> another project but seems beam didnt like it) but latency is still 
>>>>>>>>>> "high"
>>>>>>>>>> using gradle runner for tests (like Etienne said ~3s on an i7 with 
>>>>>>>>>> 16G vs a
>>>>>>>>>> few ms with default idea test runner, would be great to solve that).
>>>>>>>>>>
>>>>>>>>>> I also find the integration quite fishy cause configurations are
>>>>>>>>>> customs so idea is kind of forced to propose your modukes 3 times at 
>>>>>>>>>> least
>>>>>>>>>> when you select the classpath (x_test being generally the working 
>>>>>>>>>> one).
>>>>>>>>>>
>>>>>>>>>> Also the false positive you get if you forget a cleanX is a bit
>>>>>>>>>> annoying, maybe we should force a clean for test or at least when 
>>>>>>>>>> there is
>>>>>>>>>> a --tests to avoid gradle to not run it cause there is no diff.
>>>>>>>>>>
>>>>>>>>>> So it works but dev productivity is reduced a lot and it became
>>>>>>>>>> slow enough to think if you should do a contribution or not - at 
>>>>>>>>>> least for
>>>>>>>>>> me.
>>>>>>>>>>
>>>>>>>>>> Le 11 avr. 2018 19:37, "Alexey Romanenko" <
>>>>>>>>>> aromanenko....@gmail.com> a écrit :
>>>>>>>>>>
>>>>>>>>>>> I’ve managed to import a project as it’s described in
>>>>>>>>>>> documentation (starting from empty project) using Idea 2018 and run 
>>>>>>>>>>> unit
>>>>>>>>>>> tests successfully.
>>>>>>>>>>> For some reasons, tests, that use DirectRunner to run a
>>>>>>>>>>> pipeline, were failed.
>>>>>>>>>>>
>>>>>>>>>>> WBR,
>>>>>>>>>>> Alexey
>>>>>>>>>>>
>>>>>>>>>>> On 11 Apr 2018, at 19:01, Daniel Oliveira <
>>>>>>>>>>> danolive...@google.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi everyone, I was the one who initially wrote the PR with Idea
>>>>>>>>>>> instructions <https://github.com/apache/beam-site/pull/414>. I
>>>>>>>>>>> was using 2017.3 as well while writing it so all the instructions 
>>>>>>>>>>> were
>>>>>>>>>>> tested on that version. I'll try testing the instructions on 2018 
>>>>>>>>>>> to see if
>>>>>>>>>>> I can reproduce the issues people are having.
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Apr 11, 2018 at 9:51 AM Lukasz Cwik <lc...@google.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I use 2017.3 and it has been reliable for me. I haven't tried
>>>>>>>>>>>> 2018 yet.
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Apr 11, 2018 at 11:30 AM Romain Manni-Bucau <
>>>>>>>>>>>> rmannibu...@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Any of you using the idea 2018? the import works for me but
>>>>>>>>>>>>> then it is
>>>>>>>>>>>>> not as smooth as it seems for you. I'm just trying to see if
>>>>>>>>>>>>> it is a
>>>>>>>>>>>>> procedure thing or a version issue.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Romain Manni-Bucau
>>>>>>>>>>>>> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2018-04-11 17:28 GMT+02:00 Kenneth Knowles <k...@google.com>:
>>>>>>>>>>>>> > The only reason I did "empty project then add a module"
>>>>>>>>>>>>> procedure was to get
>>>>>>>>>>>>> > all the IntelliJ files outside the source tree. IIRC
>>>>>>>>>>>>> directly creating from
>>>>>>>>>>>>> > existing sources didn't give the necessary configuration
>>>>>>>>>>>>> options. If you
>>>>>>>>>>>>> > don't care about being able to `git clean` then you can do
>>>>>>>>>>>>> the shorter
>>>>>>>>>>>>> > version. Also the particular UI for creation might have
>>>>>>>>>>>>> improved since I
>>>>>>>>>>>>> > tried it. I'll do it again.
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > On the pom, since it is only generated for machine reading,
>>>>>>>>>>>>> why care if the
>>>>>>>>>>>>> > parent is inlined? I actually prefer to avoid coupling with
>>>>>>>>>>>>> things that you
>>>>>>>>>>>>> > have to go and look up. Using inheritance is OK for human
>>>>>>>>>>>>> edited poms
>>>>>>>>>>>>> > (actually IMO it is still a mistake) but it doesn't change
>>>>>>>>>>>>> the semantics of
>>>>>>>>>>>>> > a shipped pom if they are all immutable, which they should
>>>>>>>>>>>>> be. It is better
>>>>>>>>>>>>> > to have all the info right there. Is there an actually
>>>>>>>>>>>>> effective difference?
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Kenn
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > On Wed, Apr 11, 2018 at 6:53 AM Etienne Chauchot <
>>>>>>>>>>>>> echauc...@apache.org>
>>>>>>>>>>>>> > wrote:
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> Hi all,
>>>>>>>>>>>>> >> I just tested gradle environment from a fresh source clone
>>>>>>>>>>>>> with this
>>>>>>>>>>>>> >> procedure with just a tiny change: I used "new project from
>>>>>>>>>>>>> existing
>>>>>>>>>>>>> >> sources" rather than create empty project and then add
>>>>>>>>>>>>> module.
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> It works fine and junit runs from intellij also work.  with
>>>>>>>>>>>>> gradle we pay
>>>>>>>>>>>>> >> a 2s delay (on my machine) before running the actual test
>>>>>>>>>>>>> for each run. This
>>>>>>>>>>>>> >> delay seems quite constant no matter the module: I have run
>>>>>>>>>>>>> all the tests at
>>>>>>>>>>>>> >> once in  runner-core and a single test in another module
>>>>>>>>>>>>> with a similar
>>>>>>>>>>>>> >> delay.
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> I also tested a debug session from intellij and it runs
>>>>>>>>>>>>> fine also.
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> I'll do some more testing and keep you posted.
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> I still have some concerns about the potential difficulty
>>>>>>>>>>>>> for new
>>>>>>>>>>>>> >> contributors to have to learn gradle in addition to Beam
>>>>>>>>>>>>> itself comparing to
>>>>>>>>>>>>> >> maven which is more mainstream for java developers. That
>>>>>>>>>>>>> could be
>>>>>>>>>>>>> >> discouraging for ex for part-time contributors
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> Etienne
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> Le mardi 10 avril 2018 à 14:38 +0000, Lukasz Cwik a écrit :
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> beam-site PR/414 updates the instructions for using
>>>>>>>>>>>>> Intellij and how to
>>>>>>>>>>>>> >> import a module:
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> 1. Create an empty IntelliJ project outside of the Beam
>>>>>>>>>>>>> source tree.
>>>>>>>>>>>>> >> 2. Under Project Structure > Project, select a Project SDK.
>>>>>>>>>>>>> >> 3. Under Project Structure > Modules, click the + sign to
>>>>>>>>>>>>> add a module and
>>>>>>>>>>>>> >>    select "Import Module".
>>>>>>>>>>>>> >>     1. Select the directory containing the Beam source tree.
>>>>>>>>>>>>> >>     2. Tick the "Import module from external model" button
>>>>>>>>>>>>> and select
>>>>>>>>>>>>> >> Gradle
>>>>>>>>>>>>> >>        from the list.
>>>>>>>>>>>>> >>     3. Tick the following boxes.
>>>>>>>>>>>>> >>        * Use auto-import
>>>>>>>>>>>>> >>        * Create separate module per source set
>>>>>>>>>>>>> >>        * Store generated project files externally
>>>>>>>>>>>>> >>        * Use default gradle wrapper
>>>>>>>>>>>>> >> 4. Delegate build actions to Gradle by going to Settings >
>>>>>>>>>>>>> Build,
>>>>>>>>>>>>> >> Execution,
>>>>>>>>>>>>> >>    Deployment > Build Tools > Gradle and checking "Delegate
>>>>>>>>>>>>> IDE build/run
>>>>>>>>>>>>> >>    actions to gradle".
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré <
>>>>>>>>>>>>> j...@nanthrax.net>
>>>>>>>>>>>>> >> wrote:
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> That's a very important issue for contribution. Up to now,
>>>>>>>>>>>>> I used Maven
>>>>>>>>>>>>> >> for setup IntelliJ (and it works just fine). If we remove
>>>>>>>>>>>>> the pom.xml,
>>>>>>>>>>>>> >> we have to support Eclipse and IntelliJ "smoothly".
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> Let me try in IntelliJ.
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> Regards
>>>>>>>>>>>>> >> JB
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> On 10/04/2018 15:21, Romain Manni-Bucau wrote:
>>>>>>>>>>>>> >> > You dont have issue due to the build setup with that
>>>>>>>>>>>>> option. I get:
>>>>>>>>>>>>> >> >
>>>>>>>>>>>>> >> > avr. 10, 2018 3:20:10 PM
>>>>>>>>>>>>> >> > org.apache.beam.runners.direct.DirectTransformExecutor
>>>>>>>>>>>>> run
>>>>>>>>>>>>> >> > GRAVE: Error occurred within
>>>>>>>>>>>>> >> > org.apache.beam.runners.direct.DirectTransformExecutor@
>>>>>>>>>>>>> 66761b7a
>>>>>>>>>>>>> >> > com.google.common.util.concurrent.ExecutionError:
>>>>>>>>>>>>> >> > java.lang.NoClassDefFoundError:
>>>>>>>>>>>>> net/bytebuddy/NamingStrategy
>>>>>>>>>>>>> >> >
>>>>>>>>>>>>> >> > ?
>>>>>>>>>>>>> >> >
>>>>>>>>>>>>> >> > Romain Manni-Bucau
>>>>>>>>>>>>> >> > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>>>>>>>>>>>>> >> >
>>>>>>>>>>>>> >> >
>>>>>>>>>>>>> >> > 2018-04-10 15:13 GMT+02:00 Lukasz Cwik <lc...@google.com
>>>>>>>>>>>>> >:
>>>>>>>>>>>>> >> >> I have found that the simplest setup is to delegate the
>>>>>>>>>>>>> build/test
>>>>>>>>>>>>> >> >> actions
>>>>>>>>>>>>> >> >> to Gradle. This allows you to run unit tests very easily
>>>>>>>>>>>>> and since its
>>>>>>>>>>>>> >> >> in
>>>>>>>>>>>>> >> >> the same manner that Gradle would have, you know that if
>>>>>>>>>>>>> its passing it
>>>>>>>>>>>>> >> >> will
>>>>>>>>>>>>> >> >> pass on the command line and on Jenkins. Here is one
>>>>>>>>>>>>> site that
>>>>>>>>>>>>> >> >> discusses how
>>>>>>>>>>>>> >> >> to set this up:
>>>>>>>>>>>>> >> >>
>>>>>>>>>>>>> >> >> http://mrhaki.blogspot.com/2016/11/gradle-goodness-
>>>>>>>>>>>>> delegate-build-and-run.html
>>>>>>>>>>>>> >> >>
>>>>>>>>>>>>> >> >>
>>>>>>>>>>>>> >> >> On Tue, Apr 10, 2018 at 8:45 AM Romain Manni-Bucau
>>>>>>>>>>>>> >> >> <rmannibu...@gmail.com>
>>>>>>>>>>>>> >> >> wrote:
>>>>>>>>>>>>> >> >>>
>>>>>>>>>>>>> >> >>> What's the plan to make idea supporting gradle on beam
>>>>>>>>>>>>> project? Do we
>>>>>>>>>>>>> >> >>> import the workaround mentionned in
>>>>>>>>>>>>> >> >>> https://youtrack.jetbrains.com/issue/IDEA-175172?
>>>>>>>>>>>>> >> >>> For the ones who didn't see this issue in action: idea
>>>>>>>>>>>>> will compile in
>>>>>>>>>>>>> >> >>> out/ instead of build/ and you will just miss all the
>>>>>>>>>>>>> resources you
>>>>>>>>>>>>> >> >>> need like some SPI registration which are used by all
>>>>>>>>>>>>> our registrar =>
>>>>>>>>>>>>> >> >>> no way to run tests in idea without hacking the
>>>>>>>>>>>>> configuration quite
>>>>>>>>>>>>> >> >>> deeply :(
>>>>>>>>>>>>> >> >>>
>>>>>>>>>>>>> >> >>> Romain Manni-Bucau
>>>>>>>>>>>>> >> >>> @rmannibucau |  Blog | Old Blog | Github | LinkedIn |
>>>>>>>>>>>>> Book
>>>>>>>>>>>>> >> >>>
>>>>>>>>>>>>> >> >>>
>>>>>>>>>>>>> >> >>> 2018-04-10 10:08 GMT+02:00 Etienne Chauchot <
>>>>>>>>>>>>> echauc...@apache.org>:
>>>>>>>>>>>>> >> >>>> As a gradle beginner, I could not agree more !
>>>>>>>>>>>>> >> >>>> +1
>>>>>>>>>>>>> >> >>>> Etienne
>>>>>>>>>>>>> >> >>>> Le lundi 09 avril 2018 à 18:47 +0200, Jean-Baptiste
>>>>>>>>>>>>> Onofré a écrit :
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>> Hi all,
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>> I did multiple gradle build since last week and I
>>>>>>>>>>>>> would like to share
>>>>>>>>>>>>> >> >>>> one of my concern: it's about the communities.
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>> If I think our users won't see any change for them due
>>>>>>>>>>>>> to Gradle
>>>>>>>>>>>>> >> >>>> build
>>>>>>>>>>>>> >> >>>> (I think that most of our users will still use Maven
>>>>>>>>>>>>> with artifacts
>>>>>>>>>>>>> >> >>>> provided by Gradle), I'm more concerned by the dev
>>>>>>>>>>>>> community and the
>>>>>>>>>>>>> >> >>>> contribution.
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>> Maven is well known and straight forward for a large
>>>>>>>>>>>>> part of
>>>>>>>>>>>>> >> >>>> potential
>>>>>>>>>>>>> >> >>>> contributors. I think we have to keep in mind that we
>>>>>>>>>>>>> still have to
>>>>>>>>>>>>> >> >>>> grow
>>>>>>>>>>>>> >> >>>> up our contributors community.
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>> Today, maybe I'm wrong, but I have the feeling that
>>>>>>>>>>>>> gradle build is
>>>>>>>>>>>>> >> >>>> not
>>>>>>>>>>>>> >> >>>> straight forward (build.gradle includes
>>>>>>>>>>>>> build_rules.gradle, gathering
>>>>>>>>>>>>> >> >>>> all taks all together).
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>> I would like to add a task in the gradle "migration"
>>>>>>>>>>>>> process:
>>>>>>>>>>>>> >> >>>> simplify
>>>>>>>>>>>>> >> >>>> the gradle structure and files, and document this.
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>> I know we already have a Jira about the documentation
>>>>>>>>>>>>> part, but I
>>>>>>>>>>>>> >> >>>> would
>>>>>>>>>>>>> >> >>>> like to "polish" and use a clean structure for the
>>>>>>>>>>>>> Gradle resources.
>>>>>>>>>>>>> >> >>>> As
>>>>>>>>>>>>> >> >>>> already quickly discussed, I think that having one
>>>>>>>>>>>>> gradle file per
>>>>>>>>>>>>> >> >>>> tasks
>>>>>>>>>>>>> >> >>>> in the .gradle directory would be helpful.
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>> The goal is really to simplify the contribution.
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>> Do you agree if I add a Jira about "Gradle polish" ?
>>>>>>>>>>>>> >> >>>> Thoughts ?
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>> Regards
>>>>>>>>>>>>> >> >>>> JB
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>> On 07/04/2018 04:52, Scott Wegner wrote:
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>> Here's an end-of-day update on migration work:
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>> * Snapshot unsigned dailies and signed release builds
>>>>>>>>>>>>> are working
>>>>>>>>>>>>> >> >>>> (!!).
>>>>>>>>>>>>> >> >>>> PR/5048 [1] merges changes from Luke's branch
>>>>>>>>>>>>> >> >>>>     * python precommit failing... will investigate
>>>>>>>>>>>>> python precommit
>>>>>>>>>>>>> >> >>>> Monday
>>>>>>>>>>>>> >> >>>> * All Precommits are gradle only
>>>>>>>>>>>>> >> >>>> * All Postcommits except performance tests and
>>>>>>>>>>>>> Java_JDK_Versions_Test
>>>>>>>>>>>>> >> >>>> use gradle (after PR/5047 [2] merged)
>>>>>>>>>>>>> >> >>>> * Nightly snapshot release using gradle is ready;
>>>>>>>>>>>>> needs PR/5048 to be
>>>>>>>>>>>>> >> >>>> merged before switching
>>>>>>>>>>>>> >> >>>> * ValidatesRunner_Spark failing consistently;
>>>>>>>>>>>>> investigating
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>> Thanks for another productive day of hacking. I'll
>>>>>>>>>>>>> pick up again on
>>>>>>>>>>>>> >> >>>> Monday.
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>> [1] https://github.com/apache/beam/pull/5048
>>>>>>>>>>>>> >> >>>> [2] https://github.com/apache/beam/pull/5047
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>> On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau
>>>>>>>>>>>>> >> >>>> <rmannibu...@gmail.com <mailto:rmannibu...@gmail.com>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>      Why building a zip per runner which its stack and
>>>>>>>>>>>>> just pointing
>>>>>>>>>>>>> >> >>>> out
>>>>>>>>>>>>> >> >>>>      on that zip and let beam lazy load the runner:
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>      --runner=LazyRunner --lazyRunnerDir=...
>>>>>>>>>>>>> --lazyRunnerOptions=...
>>>>>>>>>>>>> >> >>>> (or
>>>>>>>>>>>>> >> >>>>      the fromSystemProperties() if it gets merged a
>>>>>>>>>>>>> day ;))
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>      Le 6 avr. 2018 20:21, "Kenneth Knowles" <
>>>>>>>>>>>>> k...@google.com
>>>>>>>>>>>>> >> >>>>      <mailto:k...@google.com>> a écrit :
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>          I'm working on finding a solution for
>>>>>>>>>>>>> launching the Nexmark
>>>>>>>>>>>>> >> >>>>          suite with each runner. This doesn't have to
>>>>>>>>>>>>> be done via
>>>>>>>>>>>>> >> >>>> Gradle,
>>>>>>>>>>>>> >> >>>>          but we anyhow need built artifacts that don't
>>>>>>>>>>>>> require user
>>>>>>>>>>>>> >> >>>>          classpath intervention.
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>          It looks to me like the examples are also
>>>>>>>>>>>>> missing this -
>>>>>>>>>>>>> >> >>>> they
>>>>>>>>>>>>> >> >>>>          have separate configuration e.g.
>>>>>>>>>>>>> sparkRunnerPreCommit but
>>>>>>>>>>>>> >> >>>> that
>>>>>>>>>>>>> >> >>>>          is overspecified compared to a free-form
>>>>>>>>>>>>> launching of a
>>>>>>>>>>>>> >> >>>> main()
>>>>>>>>>>>>> >> >>>>          program with a runner profile.
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>          On Fri, Apr 6, 2018 at 11:09 AM Lukasz Cwik
>>>>>>>>>>>>> >> >>>> <lc...@google.com
>>>>>>>>>>>>> >> >>>>          <mailto:lc...@google.com>> wrote:
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>              Romain, are you talking about the
>>>>>>>>>>>>> profiles that exist as
>>>>>>>>>>>>> >> >>>>              part of the archetype examples?
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>              If so, then those still exist and haven't
>>>>>>>>>>>>> been changed.
>>>>>>>>>>>>> >> >>>> If
>>>>>>>>>>>>> >> >>>>              not, can you provide a link to the
>>>>>>>>>>>>> profile in a pom file
>>>>>>>>>>>>> >> >>>> to
>>>>>>>>>>>>> >> >>>>              be clearer?
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>              On Fri, Apr 6, 2018 at 12:40 PM Romain
>>>>>>>>>>>>> Manni-Bucau
>>>>>>>>>>>>> >> >>>>              <rmannibu...@gmail.com <mailto:
>>>>>>>>>>>>> rmannibu...@gmail.com>>
>>>>>>>>>>>>> >> >>>> wrote:
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>                  Hi Scott,
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>                  is it right that 2 doesn't handle the
>>>>>>>>>>>>> hierachy
>>>>>>>>>>>>> >> >>>> anymore
>>>>>>>>>>>>> >> >>>>                  and that it doesn't handle profiles
>>>>>>>>>>>>> for runners as
>>>>>>>>>>>>> >> >>>> it is
>>>>>>>>>>>>> >> >>>>                  currently with maven?
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>                  Romain Manni-Bucau
>>>>>>>>>>>>> >> >>>>                  @rmannibucau <https://twitter.com/
>>>>>>>>>>>>> rmannibucau> |
>>>>>>>>>>>>> >> >>>> Blog
>>>>>>>>>>>>> >> >>>>                  <https://rmannibucau.metawerx.net/>
>>>>>>>>>>>>> | Old Blog
>>>>>>>>>>>>> >> >>>>                  <http://rmannibucau.wordpress.com> |
>>>>>>>>>>>>> Github
>>>>>>>>>>>>> >> >>>>                  <https://github.com/rmannibucau> |
>>>>>>>>>>>>> LinkedIn
>>>>>>>>>>>>> >> >>>>                  <https://www.linkedin.com/in/
>>>>>>>>>>>>> rmannibucau> | Book
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>> <https://www.packtpub.com/
>>>>>>>>>>>>> application-development/java-ee-8-high-performance>
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>                  2018-04-06 18:32 GMT+02:00 Scott
>>>>>>>>>>>>> Wegner
>>>>>>>>>>>>> >> >>>>                  <sweg...@google.com <mailto:
>>>>>>>>>>>>> sweg...@google.com>>:
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>                      I wanted to start a thread to
>>>>>>>>>>>>> summarize the
>>>>>>>>>>>>> >> >>>> current
>>>>>>>>>>>>> >> >>>>                      state of Gradle migration. We've
>>>>>>>>>>>>> made lots of
>>>>>>>>>>>>> >> >>>> good
>>>>>>>>>>>>> >> >>>>                      progress so far this week. Here's
>>>>>>>>>>>>> the status
>>>>>>>>>>>>> >> >>>> from
>>>>>>>>>>>>> >> >>>>                      what I can tell-- please add or
>>>>>>>>>>>>> correct anything
>>>>>>>>>>>>> >> >>>> I
>>>>>>>>>>>>> >> >>>>                      missed:
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>                      * Release artifacts can be built
>>>>>>>>>>>>> and published
>>>>>>>>>>>>> >> >>>> for
>>>>>>>>>>>>> >> >>>>                      Snapshot and officlal releases [1]
>>>>>>>>>>>>> >> >>>>                      * Gradle-generated releases have
>>>>>>>>>>>>> been validated
>>>>>>>>>>>>> >> >>>> with
>>>>>>>>>>>>> >> >>>>                      the the Apache Beam archetype
>>>>>>>>>>>>> generation
>>>>>>>>>>>>> >> >>>> quickstart;
>>>>>>>>>>>>> >> >>>>                      still needs additional validation.
>>>>>>>>>>>>> >> >>>>                      * Generated release pom files
>>>>>>>>>>>>> have correct
>>>>>>>>>>>>> >> >>>> project
>>>>>>>>>>>>> >> >>>>                      metadata [2]
>>>>>>>>>>>>> >> >>>>                      * The python pre-commits are now
>>>>>>>>>>>>> working in
>>>>>>>>>>>>> >> >>>> Gradle
>>>>>>>>>>>>> >> >>>> [3]
>>>>>>>>>>>>> >> >>>>                      * Ismaël has started a
>>>>>>>>>>>>> collaborative doc of
>>>>>>>>>>>>> >> >>>> Gradle
>>>>>>>>>>>>> >> >>>>                      tips [4] as we all learn the new
>>>>>>>>>>>>> system-- please
>>>>>>>>>>>>> >> >>>> add
>>>>>>>>>>>>> >> >>>>                      your own. This will eventually
>>>>>>>>>>>>> feed into
>>>>>>>>>>>>> >> >>>> official
>>>>>>>>>>>>> >> >>>>                      documentation on the website.
>>>>>>>>>>>>> >> >>>>                      * Łukasz Gajowy is working on
>>>>>>>>>>>>> migrating
>>>>>>>>>>>>> >> >>>> performance
>>>>>>>>>>>>> >> >>>>                      testing framework [5]
>>>>>>>>>>>>> >> >>>>                      * Daniel is working on updating
>>>>>>>>>>>>> documentation to
>>>>>>>>>>>>> >> >>>>                      refer to Gradle instead of maven
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>                      If I missed anything, please add
>>>>>>>>>>>>> it to this
>>>>>>>>>>>>> >> >>>> thread.
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>                      The general roadmap we're working
>>>>>>>>>>>>> towards is:
>>>>>>>>>>>>> >> >>>>                      (a) Publish release artifacts
>>>>>>>>>>>>> with Gradle
>>>>>>>>>>>>> >> >>>> (SNAPSHOT
>>>>>>>>>>>>> >> >>>>                      and signed releases)
>>>>>>>>>>>>> >> >>>>                      (b) Postcommits migrated to Gradle
>>>>>>>>>>>>> >> >>>>                      (c) Migrate documentation from
>>>>>>>>>>>>> maven to Gradle
>>>>>>>>>>>>> >> >>>>                      (d) Migrate perfkit suites to use
>>>>>>>>>>>>> Gradle
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>                      For those of you that are
>>>>>>>>>>>>> hacking: thanks for
>>>>>>>>>>>>> >> >>>> your
>>>>>>>>>>>>> >> >>>>                      help so far! Progress is being
>>>>>>>>>>>>> roughly tracked
>>>>>>>>>>>>> >> >>>> on
>>>>>>>>>>>>> >> >>>>                      the Kanban [6]; please make sure
>>>>>>>>>>>>> the issues
>>>>>>>>>>>>> >> >>>> assigned
>>>>>>>>>>>>> >> >>>>                      to you are up-to-date. Many of
>>>>>>>>>>>>> the changes are
>>>>>>>>>>>>> >> >>>>                      staged on lukecwik's local branch
>>>>>>>>>>>>> [7]; we'll
>>>>>>>>>>>>> >> >>>> work on
>>>>>>>>>>>>> >> >>>>                      merging them back soon.
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>                      [1]
>>>>>>>>>>>>> >> >>>> https://github.com/lukecwik/incubator-beam/pull/7
>>>>>>>>>>>>> >> >>>>                      [2]
>>>>>>>>>>>>> >> >>>> https://github.com/lukecwik/incubator-beam/pull/3
>>>>>>>>>>>>> >> >>>>                      [3] https://github.com/apache/
>>>>>>>>>>>>> beam/pull/5032
>>>>>>>>>>>>> >> >>>>                      [4]
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>> https://docs.google.com/document/d/
>>>>>>>>>>>>> 1wR56Jef3XIPwj4DFzQKznuGPM3JDfRDVkxzeDlbdVSQ/edit
>>>>>>>>>>>>> >> >>>>                      [5] https://github.com/apache/
>>>>>>>>>>>>> beam/pull/5003
>>>>>>>>>>>>> >> >>>>                      [6]
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>> https://issues.apache.org/jira/secure/RapidBoard.jspa?
>>>>>>>>>>>>> rapidView=242
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>                      [7]
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>> https://github.com/lukecwik/incubator-beam/tree/gradle
>>>>>>>>>>>>> >> >>>>                      --
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>                      Got feedback?
>>>>>>>>>>>>> http://go/swegner-feedback
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >> >>>>
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>

Reply via email to