Runner a test doesnt have the right classpath (idea uses out/ instead of build/) then when you switch on gradle runner the launching uses gradle which is not able to use submodules directly but reconsider the whole project which is quite slow for normal dev iterations compare to just run the test with the right classpath and a fast compile step if needed. I lost literally 1h for something simple with that tooling, this is way too much to be acceptable on my side since I'm sadly not paid to work on beam (one day maybe ;)).
Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book 2018-04-10 18:27 GMT+02:00 Reuven Lax <re...@google.com>: > Romain, > > Can you detail what's not working. I switched my IntelliJ over to Gradle > about two weeks ago, and haven't had any trouble. > > Reuven > > On Tue, Apr 10, 2018 at 4:20 PM Romain Manni-Bucau <rmannibu...@gmail.com> > wrote: >> >> Ok, didn't find a way to make it working properly (only workaround >> with direct commands and no good idea integration for debugging). I'm >> back with maven, if anyone knows how to properly solve it let's do it. >> If not I think JB point is to consider more than any other criteria. >> >> Romain Manni-Bucau >> @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book >> >> >> 2018-04-10 16:41 GMT+02:00 Romain Manni-Bucau <rmannibu...@gmail.com>: >> > side note: do NOT use auto-import until you are sure you can, it locks >> > regularly on beam (pby too big for idea?) and makes idea ready to be >> > killed :( >> > >> > Romain Manni-Bucau >> > @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book >> > >> > >> > 2018-04-10 16:40 GMT+02:00 Jean-Baptiste Onofré <j...@nanthrax.net>: >> >> It's what I did, I'm trying a complete reload now (maybe this step >> >> failed). >> >> >> >> On 10/04/2018 16:38, Lukasz Cwik wrote: >> >>> >> >>> beam-site PR/414 updates the instructions for using Intellij and how >> >>> to >> >>> import a module: >> >>> >> >>> 1. Create an empty IntelliJ project outside of the Beam source tree. >> >>> 2. Under Project Structure > Project, select a Project SDK. >> >>> 3. Under Project Structure > Modules, click the + sign to add a module >> >>> and >> >>> select "Import Module". >> >>> 1. Select the directory containing the Beam source tree. >> >>> 2. Tick the "Import module from external model" button and select >> >>> Gradle >> >>> from the list. >> >>> 3. Tick the following boxes. >> >>> * Use auto-import >> >>> * Create separate module per source set >> >>> * Store generated project files externally >> >>> * Use default gradle wrapper >> >>> 4. Delegate build actions to Gradle by going to Settings > Build, >> >>> Execution, >> >>> Deployment > Build Tools > Gradle and checking "Delegate IDE >> >>> build/run >> >>> actions to gradle". >> >>> >> >>> On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré <j...@nanthrax.net >> >>> <mailto:j...@nanthrax.net>> wrote: >> >>> >> >>> That's a very important issue for contribution. Up to now, I used >> >>> Maven >> >>> for setup IntelliJ (and it works just fine). If we remove the >> >>> pom.xml, >> >>> we have to support Eclipse and IntelliJ "smoothly". >> >>> >> >>> Let me try in IntelliJ. >> >>> >> >>> Regards >> >>> JB >> >>> >> >>> On 10/04/2018 15:21, Romain Manni-Bucau wrote: >> >>> > You dont have issue due to the build setup with that option. I >> >>> get: >> >>> > >> >>> > avr. 10, 2018 3:20:10 PM >> >>> > org.apache.beam.runners.direct.DirectTransformExecutor run >> >>> > GRAVE: Error occurred within >> >>> > org.apache.beam.runners.direct.DirectTransformExecutor@66761b7a >> >>> > com.google.common.util.concurrent.ExecutionError: >> >>> > java.lang.NoClassDefFoundError: net/bytebuddy/NamingStrategy >> >>> > >> >>> > ? >> >>> > >> >>> > Romain Manni-Bucau >> >>> > @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book >> >>> > >> >>> > >> >>> > 2018-04-10 15:13 GMT+02:00 Lukasz Cwik <lc...@google.com >> >>> <mailto:lc...@google.com>>: >> >>> >> I have found that the simplest setup is to delegate the >> >>> build/test actions >> >>> >> to Gradle. This allows you to run unit tests very easily and >> >>> since its in >> >>> >> the same manner that Gradle would have, you know that if its >> >>> passing it will >> >>> >> pass on the command line and on Jenkins. Here is one site that >> >>> discusses how >> >>> >> to set this up: >> >>> >> >> >>> >> >>> >> >>> http://mrhaki.blogspot.com/2016/11/gradle-goodness-delegate-build-and-run.html >> >>> >> >> >>> >> >> >>> >> On Tue, Apr 10, 2018 at 8:45 AM Romain Manni-Bucau >> >>> <rmannibu...@gmail.com <mailto:rmannibu...@gmail.com>> >> >>> >> wrote: >> >>> >>> >> >>> >>> What's the plan to make idea supporting gradle on beam >> >>> project? >> >>> Do we >> >>> >>> import the workaround mentionned in >> >>> >>> https://youtrack.jetbrains.com/issue/IDEA-175172? >> >>> >>> For the ones who didn't see this issue in action: idea will >> >>> compile in >> >>> >>> out/ instead of build/ and you will just miss all the >> >>> resources >> >>> you >> >>> >>> need like some SPI registration which are used by all our >> >>> registrar => >> >>> >>> no way to run tests in idea without hacking the configuration >> >>> quite >> >>> >>> deeply :( >> >>> >>> >> >>> >>> Romain Manni-Bucau >> >>> >>> @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book >> >>> >>> >> >>> >>> >> >>> >>> 2018-04-10 10:08 GMT+02:00 Etienne Chauchot >> >>> <echauc...@apache.org <mailto:echauc...@apache.org>>: >> >>> >> >>> >>>> As a gradle beginner, I could not agree more ! >> >>> >>>> +1 >> >>> >>>> Etienne >> >>> >>>> Le lundi 09 avril 2018 à 18:47 +0200, Jean-Baptiste Onofré a >> >>> écrit : >> >>> >>>> >> >>> >>>> Hi all, >> >>> >>>> >> >>> >>>> I did multiple gradle build since last week and I would like >> >>> to share >> >>> >>>> one of my concern: it's about the communities. >> >>> >>>> >> >>> >>>> If I think our users won't see any change for them due to >> >>> Gradle build >> >>> >>>> (I think that most of our users will still use Maven with >> >>> artifacts >> >>> >>>> provided by Gradle), I'm more concerned by the dev community >> >>> and the >> >>> >>>> contribution. >> >>> >>>> >> >>> >>>> Maven is well known and straight forward for a large part of >> >>> potential >> >>> >>>> contributors. I think we have to keep in mind that we still >> >>> have to grow >> >>> >>>> up our contributors community. >> >>> >>>> >> >>> >>>> Today, maybe I'm wrong, but I have the feeling that gradle >> >>> build is not >> >>> >>>> straight forward (build.gradle includes build_rules.gradle, >> >>> gathering >> >>> >>>> all taks all together). >> >>> >>>> >> >>> >>>> I would like to add a task in the gradle "migration" >> >>> process: >> >>> simplify >> >>> >>>> the gradle structure and files, and document this. >> >>> >>>> >> >>> >>>> I know we already have a Jira about the documentation part, >> >>> but I would >> >>> >>>> like to "polish" and use a clean structure for the Gradle >> >>> resources. As >> >>> >>>> already quickly discussed, I think that having one gradle >> >>> file >> >>> per tasks >> >>> >>>> in the .gradle directory would be helpful. >> >>> >>>> >> >>> >>>> The goal is really to simplify the contribution. >> >>> >>>> >> >>> >>>> Do you agree if I add a Jira about "Gradle polish" ? >> >>> >>>> Thoughts ? >> >>> >>>> >> >>> >>>> Regards >> >>> >>>> JB >> >>> >>>> >> >>> >>>> On 07/04/2018 04:52, Scott Wegner wrote: >> >>> >>>> >> >>> >>>> Here's an end-of-day update on migration work: >> >>> >>>> >> >>> >>>> * Snapshot unsigned dailies and signed release builds are >> >>> working (!!). >> >>> >>>> PR/5048 [1] merges changes from Luke's branch >> >>> >>>> * python precommit failing... will investigate python >> >>> precommit >> >>> >>>> Monday >> >>> >>>> * All Precommits are gradle only >> >>> >>>> * All Postcommits except performance tests and >> >>> Java_JDK_Versions_Test >> >>> >>>> use gradle (after PR/5047 [2] merged) >> >>> >>>> * Nightly snapshot release using gradle is ready; needs >> >>> PR/5048 to be >> >>> >>>> merged before switching >> >>> >>>> * ValidatesRunner_Spark failing consistently; investigating >> >>> >>>> >> >>> >>>> Thanks for another productive day of hacking. I'll pick up >> >>> again on >> >>> >>>> Monday. >> >>> >>>> >> >>> >>>> [1] https://github.com/apache/beam/pull/5048 >> >>> >>>> [2] https://github.com/apache/beam/pull/5047 >> >>> >>>> >> >>> >>>> >> >>> >>>> On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau >> >>> >>>> <rmannibu...@gmail.com <mailto:rmannibu...@gmail.com> >> >>> <mailto:rmannibu...@gmail.com <mailto:rmannibu...@gmail.com>>> >> >>> wrote: >> >>> >>>> >> >>> >>>> Why building a zip per runner which its stack and just >> >>> pointing out >> >>> >>>> on that zip and let beam lazy load the runner: >> >>> >>>> >> >>> >>>> --runner=LazyRunner --lazyRunnerDir=... >> >>> --lazyRunnerOptions=... (or >> >>> >>>> the fromSystemProperties() if it gets merged a day ;)) >> >>> >>>> >> >>> >>>> Le 6 avr. 2018 20:21, "Kenneth Knowles" <k...@google.com >> >>> <mailto:k...@google.com> >> >>> >>>> <mailto:k...@google.com <mailto:k...@google.com>>> a >> >>> écrit : >> >>> >>>> >> >>> >>>> I'm working on finding a solution for launching the >> >>> Nexmark >> >>> >>>> suite with each runner. This doesn't have to be >> >>> done >> >>> via Gradle, >> >>> >>>> but we anyhow need built artifacts that don't >> >>> require >> >>> user >> >>> >>>> classpath intervention. >> >>> >>>> >> >>> >>>> It looks to me like the examples are also missing >> >>> this - they >> >>> >>>> have separate configuration e.g. >> >>> sparkRunnerPreCommit >> >>> but that >> >>> >>>> is overspecified compared to a free-form launching >> >>> of >> >>> a main() >> >>> >>>> program with a runner profile. >> >>> >>>> >> >>> >>>> On Fri, Apr 6, 2018 at 11:09 AM Lukasz Cwik >> >>> <lc...@google.com <mailto:lc...@google.com> >> >>> >>>> <mailto:lc...@google.com >> >>> <mailto:lc...@google.com>>> >> >>> wrote: >> >>> >>>> >> >>> >>>> Romain, are you talking about the profiles that >> >>> exist as >> >>> >>>> part of the archetype examples? >> >>> >>>> >> >>> >>>> If so, then those still exist and haven't been >> >>> changed. If >> >>> >>>> not, can you provide a link to the profile in a >> >>> pom file to >> >>> >>>> be clearer? >> >>> >>>> >> >>> >>>> On Fri, Apr 6, 2018 at 12:40 PM Romain >> >>> Manni-Bucau >> >>> >>>> <rmannibu...@gmail.com >> >>> <mailto:rmannibu...@gmail.com> <mailto:rmannibu...@gmail.com >> >>> <mailto:rmannibu...@gmail.com>>> >> >>> >>>> wrote: >> >>> >>>> >> >>> >>>> Hi Scott, >> >>> >>>> >> >>> >>>> is it right that 2 doesn't handle the >> >>> hierachy anymore >> >>> >>>> and that it doesn't handle profiles for >> >>> runners as it is >> >>> >>>> currently with maven? >> >>> >>>> >> >>> >>>> >> >>> >>>> Romain Manni-Bucau >> >>> >>>> @rmannibucau >> >>> <https://twitter.com/rmannibucau> | Blog >> >>> >>>> <https://rmannibucau.metawerx.net/> | Old >> >>> Blog >> >>> >>>> <http://rmannibucau.wordpress.com> | Github >> >>> >>>> <https://github.com/rmannibucau> | LinkedIn >> >>> >>>> <https://www.linkedin.com/in/rmannibucau> | >> >>> Book >> >>> >>>> >> >>> >>>> >> >>> >>>> >> >>> >> >>> >> >>> <https://www.packtpub.com/application-development/java-ee-8-high-performance> >> >>> >>>> >> >>> >>>> 2018-04-06 18:32 GMT+02:00 Scott Wegner >> >>> >>>> <sweg...@google.com >> >>> <mailto:sweg...@google.com> <mailto:sweg...@google.com >> >>> >> >>> <mailto:sweg...@google.com>>>: >> >>> >>>> >> >>> >>>> I wanted to start a thread to summarize >> >>> the current >> >>> >>>> state of Gradle migration. We've made >> >>> lots of good >> >>> >>>> progress so far this week. Here's the >> >>> status from >> >>> >>>> what I can tell-- please add or correct >> >>> anything I >> >>> >>>> missed: >> >>> >>>> >> >>> >>>> * Release artifacts can be built and >> >>> published for >> >>> >>>> Snapshot and officlal releases [1] >> >>> >>>> * Gradle-generated releases have been >> >>> validated with >> >>> >>>> the the Apache Beam archetype >> >>> generation >> >>> quickstart; >> >>> >>>> still needs additional validation. >> >>> >>>> * Generated release pom files have >> >>> correct project >> >>> >>>> metadata [2] >> >>> >>>> * The python pre-commits are now >> >>> working >> >>> in Gradle >> >>> >>>> [3] >> >>> >>>> * Ismaël has started a collaborative >> >>> doc >> >>> of Gradle >> >>> >>>> tips [4] as we all learn the new >> >>> system-- >> >>> please add >> >>> >>>> your own. This will eventually feed >> >>> into >> >>> official >> >>> >>>> documentation on the website. >> >>> >>>> * Łukasz Gajowy is working on migrating >> >>> performance >> >>> >>>> testing framework [5] >> >>> >>>> * Daniel is working on updating >> >>> documentation to >> >>> >>>> refer to Gradle instead of maven >> >>> >>>> >> >>> >>>> If I missed anything, please add it to >> >>> this thread. >> >>> >>>> >> >>> >>>> The general roadmap we're working >> >>> towards >> >>> is: >> >>> >>>> (a) Publish release artifacts with >> >>> Gradle >> >>> (SNAPSHOT >> >>> >>>> and signed releases) >> >>> >>>> (b) Postcommits migrated to Gradle >> >>> >>>> (c) Migrate documentation from maven to >> >>> Gradle >> >>> >>>> (d) Migrate perfkit suites to use >> >>> Gradle >> >>> >>>> >> >>> >>>> For those of you that are hacking: >> >>> thanks >> >>> for your >> >>> >>>> help so far! Progress is being roughly >> >>> tracked on >> >>> >>>> the Kanban [6]; please make sure the >> >>> issues assigned >> >>> >>>> to you are up-to-date. Many of the >> >>> changes are >> >>> >>>> staged on lukecwik's local branch [7]; >> >>> we'll work on >> >>> >>>> merging them back soon. >> >>> >>>> >> >>> >>>> >> >>> >>>> [1] >> >>> >>>> https://github.com/lukecwik/incubator-beam/pull/7 >> >>> >>>> [2] >> >>> >>>> https://github.com/lukecwik/incubator-beam/pull/3 >> >>> >>>> [3] >> >>> https://github.com/apache/beam/pull/5032 >> >>> >>>> [4] >> >>> >>>> >> >>> >>>> >> >>> >>>> >> >>> >> >>> >> >>> https://docs.google.com/document/d/1wR56Jef3XIPwj4DFzQKznuGPM3JDfRDVkxzeDlbdVSQ/edit >> >>> >>>> [5] >> >>> https://github.com/apache/beam/pull/5003 >> >>> >>>> [6] >> >>> >>>> >> >>> >>>> >> >>> >> >>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=242 >> >>> >>>> >> >>> >>>> [7] >> >>> >>>> >> >>> >>>> https://github.com/lukecwik/incubator-beam/tree/gradle >> >>> >>>> -- >> >>> >>>> >> >>> >>>> >> >>> >>>> Got feedback? >> >>> http://go/swegner-feedback >> >>> >>>> >> >>> >>>> >> >>> >> >>
