FYI, I did a new attempt and it works fine (pretty long). Previous try
failed.
Regards
JB
On 10/04/2018 19:52, Kenneth Knowles wrote:
I've been on Idea+Gradle for ~two months, around the time I added
https://github.com/apache/beam/pull/4583 and
https://github.com/apache/beam/pull/4626 to make the import require zero
user work. I have no fear of deleting my project any time and re-importing.
I agree with not having auto-import on. It is just too slow. I can't
remember if it was importing too often due to build outputs or if it was
just that I was messing with the build.gradle files. Anyhow it doesn't
really add much value.
The gradle runner _is_ able to use submodules and run individual tests
methods, and all that.
Kenn
On Tue, Apr 10, 2018 at 9:31 AM Romain Manni-Bucau
<rmannibu...@gmail.com <mailto:rmannibu...@gmail.com>> wrote:
Runner a test doesnt have the right classpath (idea uses out/ instead
of build/) then when you switch on gradle runner the launching uses
gradle which is not able to use submodules directly but reconsider the
whole project which is quite slow for normal dev iterations
compare to just run the test with the right classpath and a fast
compile step if needed. I lost literally 1h for something simple with
that tooling, this is way too much to be acceptable on my side since
I'm sadly not paid to work on beam (one day maybe ;)).
Romain Manni-Bucau
@rmannibucau | Blog | Old Blog | Github | LinkedIn | Book
2018-04-10 18:27 GMT+02:00 Reuven Lax <re...@google.com
<mailto:re...@google.com>>:
> Romain,
>
> Can you detail what's not working. I switched my IntelliJ over to
Gradle
> about two weeks ago, and haven't had any trouble.
>
> Reuven
>
> On Tue, Apr 10, 2018 at 4:20 PM Romain Manni-Bucau
<rmannibu...@gmail.com <mailto:rmannibu...@gmail.com>>
> wrote:
>>
>> Ok, didn't find a way to make it working properly (only workaround
>> with direct commands and no good idea integration for
debugging). I'm
>> back with maven, if anyone knows how to properly solve it let's
do it.
>> If not I think JB point is to consider more than any other criteria.
>>
>> Romain Manni-Bucau
>> @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book
>>
>>
>> 2018-04-10 16:41 GMT+02:00 Romain Manni-Bucau
<rmannibu...@gmail.com <mailto:rmannibu...@gmail.com>>:
>> > side note: do NOT use auto-import until you are sure you can,
it locks
>> > regularly on beam (pby too big for idea?) and makes idea ready
to be
>> > killed :(
>> >
>> > Romain Manni-Bucau
>> > @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book
>> >
>> >
>> > 2018-04-10 16:40 GMT+02:00 Jean-Baptiste Onofré
<j...@nanthrax.net <mailto:j...@nanthrax.net>>:
>> >> It's what I did, I'm trying a complete reload now (maybe this
step
>> >> failed).
>> >>
>> >> On 10/04/2018 16:38, Lukasz Cwik wrote:
>> >>>
>> >>> beam-site PR/414 updates the instructions for using Intellij
and how
>> >>> to
>> >>> import a module:
>> >>>
>> >>> 1. Create an empty IntelliJ project outside of the Beam
source tree.
>> >>> 2. Under Project Structure > Project, select a Project SDK.
>> >>> 3. Under Project Structure > Modules, click the + sign to
add a module
>> >>> and
>> >>> select "Import Module".
>> >>> 1. Select the directory containing the Beam source tree.
>> >>> 2. Tick the "Import module from external model" button
and select
>> >>> Gradle
>> >>> from the list.
>> >>> 3. Tick the following boxes.
>> >>> * Use auto-import
>> >>> * Create separate module per source set
>> >>> * Store generated project files externally
>> >>> * Use default gradle wrapper
>> >>> 4. Delegate build actions to Gradle by going to Settings >
Build,
>> >>> Execution,
>> >>> Deployment > Build Tools > Gradle and checking "Delegate IDE
>> >>> build/run
>> >>> actions to gradle".
>> >>>
>> >>> On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré
<j...@nanthrax.net <mailto:j...@nanthrax.net>
>> >>> <mailto:j...@nanthrax.net <mailto:j...@nanthrax.net>>> wrote:
>> >>>
>> >>> That's a very important issue for contribution. Up to
now, I used
>> >>> Maven
>> >>> for setup IntelliJ (and it works just fine). If we
remove the
>> >>> pom.xml,
>> >>> we have to support Eclipse and IntelliJ "smoothly".
>> >>>
>> >>> Let me try in IntelliJ.
>> >>>
>> >>> Regards
>> >>> JB
>> >>>
>> >>> On 10/04/2018 15:21, Romain Manni-Bucau wrote:
>> >>> > You dont have issue due to the build setup with that
option. I
>> >>> get:
>> >>> >
>> >>> > avr. 10, 2018 3:20:10 PM
>> >>> >
org.apache.beam.runners.direct.DirectTransformExecutor run
>> >>> > GRAVE: Error occurred within
>> >>> >
org.apache.beam.runners.direct.DirectTransformExecutor@66761b7a
>> >>> > com.google.common.util.concurrent.ExecutionError:
>> >>> > java.lang.NoClassDefFoundError:
net/bytebuddy/NamingStrategy
>> >>> >
>> >>> > ?
>> >>> >
>> >>> > Romain Manni-Bucau
>> >>> > @rmannibucau | Blog | Old Blog | Github | LinkedIn |
Book
>> >>> >
>> >>> >
>> >>> > 2018-04-10 15:13 GMT+02:00 Lukasz Cwik
<lc...@google.com <mailto:lc...@google.com>
>> >>> <mailto:lc...@google.com <mailto:lc...@google.com>>>:
>> >>> >> I have found that the simplest setup is to delegate the
>> >>> build/test actions
>> >>> >> to Gradle. This allows you to run unit tests very
easily and
>> >>> since its in
>> >>> >> the same manner that Gradle would have, you know
that if its
>> >>> passing it will
>> >>> >> pass on the command line and on Jenkins. Here is one
site that
>> >>> discusses how
>> >>> >> to set this up:
>> >>> >>
>> >>>
>> >>>
>> >>>
http://mrhaki.blogspot.com/2016/11/gradle-goodness-delegate-build-and-run.html
>> >>> >>
>> >>> >>
>> >>> >> On Tue, Apr 10, 2018 at 8:45 AM Romain Manni-Bucau
>> >>> <rmannibu...@gmail.com <mailto:rmannibu...@gmail.com>
<mailto:rmannibu...@gmail.com <mailto:rmannibu...@gmail.com>>>
>> >>> >> wrote:
>> >>> >>>
>> >>> >>> What's the plan to make idea supporting gradle on beam
>> >>> project?
>> >>> Do we
>> >>> >>> import the workaround mentionned in
>> >>> >>> https://youtrack.jetbrains.com/issue/IDEA-175172?
>> >>> >>> For the ones who didn't see this issue in action:
idea will
>> >>> compile in
>> >>> >>> out/ instead of build/ and you will just miss all the
>> >>> resources
>> >>> you
>> >>> >>> need like some SPI registration which are used by
all our
>> >>> registrar =>
>> >>> >>> no way to run tests in idea without hacking the
configuration
>> >>> quite
>> >>> >>> deeply :(
>> >>> >>>
>> >>> >>> Romain Manni-Bucau
>> >>> >>> @rmannibucau | Blog | Old Blog | Github | LinkedIn
| Book
>> >>> >>>
>> >>> >>>
>> >>> >>> 2018-04-10 10:08 GMT+02:00 Etienne Chauchot
>> >>> <echauc...@apache.org <mailto:echauc...@apache.org>
<mailto:echauc...@apache.org <mailto:echauc...@apache.org>>>:
>> >>>
>> >>> >>>> As a gradle beginner, I could not agree more !
>> >>> >>>> +1
>> >>> >>>> Etienne
>> >>> >>>> Le lundi 09 avril 2018 à 18:47 +0200,
Jean-Baptiste Onofré a
>> >>> écrit :
>> >>> >>>>
>> >>> >>>> Hi all,
>> >>> >>>>
>> >>> >>>> I did multiple gradle build since last week and I
would like
>> >>> to share
>> >>> >>>> one of my concern: it's about the communities.
>> >>> >>>>
>> >>> >>>> If I think our users won't see any change for them
due to
>> >>> Gradle build
>> >>> >>>> (I think that most of our users will still use
Maven with
>> >>> artifacts
>> >>> >>>> provided by Gradle), I'm more concerned by the dev
community
>> >>> and the
>> >>> >>>> contribution.
>> >>> >>>>
>> >>> >>>> Maven is well known and straight forward for a
large part of
>> >>> potential
>> >>> >>>> contributors. I think we have to keep in mind that
we still
>> >>> have to grow
>> >>> >>>> up our contributors community.
>> >>> >>>>
>> >>> >>>> Today, maybe I'm wrong, but I have the feeling
that gradle
>> >>> build is not
>> >>> >>>> straight forward (build.gradle includes
build_rules.gradle,
>> >>> gathering
>> >>> >>>> all taks all together).
>> >>> >>>>
>> >>> >>>> I would like to add a task in the gradle "migration"
>> >>> process:
>> >>> simplify
>> >>> >>>> the gradle structure and files, and document this.
>> >>> >>>>
>> >>> >>>> I know we already have a Jira about the
documentation part,
>> >>> but I would
>> >>> >>>> like to "polish" and use a clean structure for the
Gradle
>> >>> resources. As
>> >>> >>>> already quickly discussed, I think that having one
gradle
>> >>> file
>> >>> per tasks
>> >>> >>>> in the .gradle directory would be helpful.
>> >>> >>>>
>> >>> >>>> The goal is really to simplify the contribution.
>> >>> >>>>
>> >>> >>>> Do you agree if I add a Jira about "Gradle polish" ?
>> >>> >>>> Thoughts ?
>> >>> >>>>
>> >>> >>>> Regards
>> >>> >>>> JB
>> >>> >>>>
>> >>> >>>> On 07/04/2018 04:52, Scott Wegner wrote:
>> >>> >>>>
>> >>> >>>> Here's an end-of-day update on migration work:
>> >>> >>>>
>> >>> >>>> * Snapshot unsigned dailies and signed release
builds are
>> >>> working (!!).
>> >>> >>>> PR/5048 [1] merges changes from Luke's branch
>> >>> >>>> * python precommit failing... will investigate
python
>> >>> precommit
>> >>> >>>> Monday
>> >>> >>>> * All Precommits are gradle only
>> >>> >>>> * All Postcommits except performance tests and
>> >>> Java_JDK_Versions_Test
>> >>> >>>> use gradle (after PR/5047 [2] merged)
>> >>> >>>> * Nightly snapshot release using gradle is ready;
needs
>> >>> PR/5048 to be
>> >>> >>>> merged before switching
>> >>> >>>> * ValidatesRunner_Spark failing consistently;
investigating
>> >>> >>>>
>> >>> >>>> Thanks for another productive day of hacking. I'll
pick up
>> >>> again on
>> >>> >>>> Monday.
>> >>> >>>>
>> >>> >>>> [1] https://github.com/apache/beam/pull/5048
>> >>> >>>> [2] https://github.com/apache/beam/pull/5047
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau
>> >>> >>>> <rmannibu...@gmail.com
<mailto:rmannibu...@gmail.com> <mailto:rmannibu...@gmail.com
<mailto:rmannibu...@gmail.com>>
>> >>> <mailto:rmannibu...@gmail.com
<mailto:rmannibu...@gmail.com> <mailto:rmannibu...@gmail.com
<mailto:rmannibu...@gmail.com>>>>
>> >>> wrote:
>> >>> >>>>
>> >>> >>>> Why building a zip per runner which its stack
and just
>> >>> pointing out
>> >>> >>>> on that zip and let beam lazy load the runner:
>> >>> >>>>
>> >>> >>>> --runner=LazyRunner --lazyRunnerDir=...
>> >>> --lazyRunnerOptions=... (or
>> >>> >>>> the fromSystemProperties() if it gets merged
a day ;))
>> >>> >>>>
>> >>> >>>> Le 6 avr. 2018 20:21, "Kenneth Knowles"
<k...@google.com <mailto:k...@google.com>
>> >>> <mailto:k...@google.com <mailto:k...@google.com>>
>> >>> >>>> <mailto:k...@google.com
<mailto:k...@google.com> <mailto:k...@google.com
<mailto:k...@google.com>>>> a
>> >>> écrit :
>> >>> >>>>
>> >>> >>>> I'm working on finding a solution for
launching the
>> >>> Nexmark
>> >>> >>>> suite with each runner. This doesn't have
to be
>> >>> done
>> >>> via Gradle,
>> >>> >>>> but we anyhow need built artifacts that don't
>> >>> require
>> >>> user
>> >>> >>>> classpath intervention.
>> >>> >>>>
>> >>> >>>> It looks to me like the examples are also
missing
>> >>> this - they
>> >>> >>>> have separate configuration e.g.
>> >>> sparkRunnerPreCommit
>> >>> but that
>> >>> >>>> is overspecified compared to a free-form
launching
>> >>> of
>> >>> a main()
>> >>> >>>> program with a runner profile.
>> >>> >>>>
>> >>> >>>> On Fri, Apr 6, 2018 at 11:09 AM Lukasz Cwik
>> >>> <lc...@google.com <mailto:lc...@google.com>
<mailto:lc...@google.com <mailto:lc...@google.com>>
>> >>> >>>> <mailto:lc...@google.com
<mailto:lc...@google.com>
>> >>> <mailto:lc...@google.com <mailto:lc...@google.com>>>>
>> >>> wrote:
>> >>> >>>>
>> >>> >>>> Romain, are you talking about the
profiles that
>> >>> exist as
>> >>> >>>> part of the archetype examples?
>> >>> >>>>
>> >>> >>>> If so, then those still exist and
haven't been
>> >>> changed. If
>> >>> >>>> not, can you provide a link to the
profile in a
>> >>> pom file to
>> >>> >>>> be clearer?
>> >>> >>>>
>> >>> >>>> On Fri, Apr 6, 2018 at 12:40 PM Romain
>> >>> Manni-Bucau
>> >>> >>>> <rmannibu...@gmail.com
<mailto:rmannibu...@gmail.com>
>> >>> <mailto:rmannibu...@gmail.com
<mailto:rmannibu...@gmail.com>> <mailto:rmannibu...@gmail.com
<mailto:rmannibu...@gmail.com>
>> >>> <mailto:rmannibu...@gmail.com
<mailto:rmannibu...@gmail.com>>>>
>> >>> >>>> wrote:
>> >>> >>>>
>> >>> >>>> Hi Scott,
>> >>> >>>>
>> >>> >>>> is it right that 2 doesn't handle the
>> >>> hierachy anymore
>> >>> >>>> and that it doesn't handle
profiles for
>> >>> runners as it is
>> >>> >>>> currently with maven?
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> Romain Manni-Bucau
>> >>> >>>> @rmannibucau
>> >>> <https://twitter.com/rmannibucau> | Blog
>> >>> >>>>
<https://rmannibucau.metawerx.net/> | Old
>> >>> Blog
>> >>> >>>>
<http://rmannibucau.wordpress.com> | Github
>> >>> >>>> <https://github.com/rmannibucau>
| LinkedIn
>> >>> >>>>
<https://www.linkedin.com/in/rmannibucau> |
>> >>> Book
>> >>> >>>>
>> >>> >>>>
>> >>> >>>>
>> >>>
>> >>>
>> >>>
<https://www.packtpub.com/application-development/java-ee-8-high-performance>
>> >>> >>>>
>> >>> >>>> 2018-04-06 18:32 GMT+02:00 Scott
Wegner
>> >>> >>>> <sweg...@google.com
<mailto:sweg...@google.com>
>> >>> <mailto:sweg...@google.com <mailto:sweg...@google.com>>
<mailto:sweg...@google.com <mailto:sweg...@google.com>
>> >>>
>> >>> <mailto:sweg...@google.com <mailto:sweg...@google.com>>>>:
>> >>> >>>>
>> >>> >>>> I wanted to start a thread to
summarize
>> >>> the current
>> >>> >>>> state of Gradle migration.
We've made
>> >>> lots of good
>> >>> >>>> progress so far this week.
Here's the
>> >>> status from
>> >>> >>>> what I can tell-- please add
or correct
>> >>> anything I
>> >>> >>>> missed:
>> >>> >>>>
>> >>> >>>> * Release artifacts can be
built and
>> >>> published for
>> >>> >>>> Snapshot and officlal
releases [1]
>> >>> >>>> * Gradle-generated releases
have been
>> >>> validated with
>> >>> >>>> the the Apache Beam archetype
>> >>> generation
>> >>> quickstart;
>> >>> >>>> still needs additional
validation.
>> >>> >>>> * Generated release pom files
have
>> >>> correct project
>> >>> >>>> metadata [2]
>> >>> >>>> * The python pre-commits are now
>> >>> working
>> >>> in Gradle
>> >>> >>>> [3]
>> >>> >>>> * Ismaël has started a
collaborative
>> >>> doc
>> >>> of Gradle
>> >>> >>>> tips [4] as we all learn the new
>> >>> system--
>> >>> please add
>> >>> >>>> your own. This will
eventually feed
>> >>> into
>> >>> official
>> >>> >>>> documentation on the website.
>> >>> >>>> * Łukasz Gajowy is working on
migrating
>> >>> performance
>> >>> >>>> testing framework [5]
>> >>> >>>> * Daniel is working on updating
>> >>> documentation to
>> >>> >>>> refer to Gradle instead of maven
>> >>> >>>>
>> >>> >>>> If I missed anything, please
add it to
>> >>> this thread.
>> >>> >>>>
>> >>> >>>> The general roadmap we're working
>> >>> towards
>> >>> is:
>> >>> >>>> (a) Publish release artifacts
with
>> >>> Gradle
>> >>> (SNAPSHOT
>> >>> >>>> and signed releases)
>> >>> >>>> (b) Postcommits migrated to
Gradle
>> >>> >>>> (c) Migrate documentation
from maven to
>> >>> Gradle
>> >>> >>>> (d) Migrate perfkit suites to use
>> >>> Gradle
>> >>> >>>>
>> >>> >>>> For those of you that are
hacking:
>> >>> thanks
>> >>> for your
>> >>> >>>> help so far! Progress is
being roughly
>> >>> tracked on
>> >>> >>>> the Kanban [6]; please make
sure the
>> >>> issues assigned
>> >>> >>>> to you are up-to-date. Many
of the
>> >>> changes are
>> >>> >>>> staged on lukecwik's local
branch [7];
>> >>> we'll work on
>> >>> >>>> merging them back soon.
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> [1]
>> >>> >>>> https://github.com/lukecwik/incubator-beam/pull/7
>> >>> >>>> [2]
>> >>> >>>> https://github.com/lukecwik/incubator-beam/pull/3
>> >>> >>>> [3]
>> >>> https://github.com/apache/beam/pull/5032
>> >>> >>>> [4]
>> >>> >>>>
>> >>> >>>>
>> >>> >>>>
>> >>>
>> >>>
>> >>>
https://docs.google.com/document/d/1wR56Jef3XIPwj4DFzQKznuGPM3JDfRDVkxzeDlbdVSQ/edit
>> >>> >>>> [5]
>> >>> https://github.com/apache/beam/pull/5003
>> >>> >>>> [6]
>> >>> >>>>
>> >>> >>>>
>> >>>
>> >>>
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=242
>> >>> >>>>
>> >>> >>>> [7]
>> >>> >>>>
>> >>> >>>> https://github.com/lukecwik/incubator-beam/tree/gradle
>> >>> >>>> --
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> Got feedback?
>> >>> http://go/swegner-feedback
>> >>> >>>>
>> >>> >>>>
>> >>>
>> >>
