Daniel, actually I did run it with default IDEA JUnit test runner. Then, in “Settings > Build, Execution, Deployment > Build Tools > Gradle > Runner" I selected “Gradle Test Runner” in “Run tests using” selectbox and it works ok when I run my tests with IDEA shortcuts. So, probably, we should add this details on https://beam.apache.org/contribute/intellij/ <https://beam.apache.org/contribute/intellij/> too. What do you think?
WBR, Alexey > On 11 Apr 2018, at 21:17, Daniel Oliveira <[email protected]> wrote: > > Alexey, are you referring to tests run with "./gradlew > :beam-runners-direct-java:needsRunnerTests"? That command works fine for me > in both versions of IDEA, but I believe the same tests fail if you run them > directly through "./gradlew test". > > However, I am having issues with a bunch of validatesRunner tests, mostly be > caused by :beam-runners-google-cloud-dataflow-java:validatesRunner. Not sure > if it's because of a code change or a gradle config, I'll keep looking into > it. > > On Wed, Apr 11, 2018 at 11:01 AM Romain Manni-Bucau <[email protected] > <mailto:[email protected]>> wrote: > I got tests running rrconfiguring gradle (which was setup for another project > but seems beam didnt like it) but latency is still "high" using gradle runner > for tests (like Etienne said ~3s on an i7 with 16G vs a few ms with default > idea test runner, would be great to solve that). > > I also find the integration quite fishy cause configurations are customs so > idea is kind of forced to propose your modukes 3 times at least when you > select the classpath (x_test being generally the working one). > > Also the false positive you get if you forget a cleanX is a bit annoying, > maybe we should force a clean for test or at least when there is a --tests to > avoid gradle to not run it cause there is no diff. > > So it works but dev productivity is reduced a lot and it became slow enough > to think if you should do a contribution or not - at least for me. > > Le 11 avr. 2018 19:37, "Alexey Romanenko" <[email protected] > <mailto:[email protected]>> a écrit : > I’ve managed to import a project as it’s described in documentation (starting > from empty project) using Idea 2018 and run unit tests successfully. > For some reasons, tests, that use DirectRunner to run a pipeline, were failed. > > WBR, > Alexey > >> On 11 Apr 2018, at 19:01, Daniel Oliveira <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hi everyone, I was the one who initially wrote the PR with Idea instructions >> <https://github.com/apache/beam-site/pull/414>. I was using 2017.3 as well >> while writing it so all the instructions were tested on that version. I'll >> try testing the instructions on 2018 to see if I can reproduce the issues >> people are having. >> >> On Wed, Apr 11, 2018 at 9:51 AM Lukasz Cwik <[email protected] >> <mailto:[email protected]>> wrote: >> I use 2017.3 and it has been reliable for me. I haven't tried 2018 yet. >> >> On Wed, Apr 11, 2018 at 11:30 AM Romain Manni-Bucau <[email protected] >> <mailto:[email protected]>> wrote: >> Any of you using the idea 2018? the import works for me but then it is >> not as smooth as it seems for you. I'm just trying to see if it is a >> procedure thing or a version issue. >> >> Romain Manni-Bucau >> @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book >> >> >> 2018-04-11 17:28 GMT+02:00 Kenneth Knowles <[email protected] >> <mailto:[email protected]>>: >> > The only reason I did "empty project then add a module" procedure was to >> > get >> > all the IntelliJ files outside the source tree. IIRC directly creating from >> > existing sources didn't give the necessary configuration options. If you >> > don't care about being able to `git clean` then you can do the shorter >> > version. Also the particular UI for creation might have improved since I >> > tried it. I'll do it again. >> > >> > On the pom, since it is only generated for machine reading, why care if the >> > parent is inlined? I actually prefer to avoid coupling with things that you >> > have to go and look up. Using inheritance is OK for human edited poms >> > (actually IMO it is still a mistake) but it doesn't change the semantics of >> > a shipped pom if they are all immutable, which they should be. It is better >> > to have all the info right there. Is there an actually effective >> > difference? >> > >> > Kenn >> > >> > On Wed, Apr 11, 2018 at 6:53 AM Etienne Chauchot <[email protected] >> > <mailto:[email protected]>> >> > wrote: >> >> >> >> Hi all, >> >> I just tested gradle environment from a fresh source clone with this >> >> procedure with just a tiny change: I used "new project from existing >> >> sources" rather than create empty project and then add module. >> >> >> >> It works fine and junit runs from intellij also work. with gradle we pay >> >> a 2s delay (on my machine) before running the actual test for each run. >> >> This >> >> delay seems quite constant no matter the module: I have run all the tests >> >> at >> >> once in runner-core and a single test in another module with a similar >> >> delay. >> >> >> >> I also tested a debug session from intellij and it runs fine also. >> >> >> >> I'll do some more testing and keep you posted. >> >> >> >> I still have some concerns about the potential difficulty for new >> >> contributors to have to learn gradle in addition to Beam itself comparing >> >> to >> >> maven which is more mainstream for java developers. That could be >> >> discouraging for ex for part-time contributors >> >> >> >> Etienne >> >> >> >> Le mardi 10 avril 2018 à 14:38 +0000, Lukasz Cwik a écrit : >> >> >> >> beam-site PR/414 updates the instructions for using Intellij and how to >> >> import a module: >> >> >> >> 1. Create an empty IntelliJ project outside of the Beam source tree. >> >> 2. Under Project Structure > Project, select a Project SDK. >> >> 3. Under Project Structure > Modules, click the + sign to add a module and >> >> select "Import Module". >> >> 1. Select the directory containing the Beam source tree. >> >> 2. Tick the "Import module from external model" button and select >> >> Gradle >> >> from the list. >> >> 3. Tick the following boxes. >> >> * Use auto-import >> >> * Create separate module per source set >> >> * Store generated project files externally >> >> * Use default gradle wrapper >> >> 4. Delegate build actions to Gradle by going to Settings > Build, >> >> Execution, >> >> Deployment > Build Tools > Gradle and checking "Delegate IDE build/run >> >> actions to gradle". >> >> >> >> On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré <[email protected] >> >> <mailto:[email protected]>> >> >> wrote: >> >> >> >> That's a very important issue for contribution. Up to now, I used Maven >> >> for setup IntelliJ (and it works just fine). If we remove the pom.xml, >> >> we have to support Eclipse and IntelliJ "smoothly". >> >> >> >> Let me try in IntelliJ. >> >> >> >> Regards >> >> JB >> >> >> >> On 10/04/2018 15:21, Romain Manni-Bucau wrote: >> >> > You dont have issue due to the build setup with that option. I get: >> >> > >> >> > avr. 10, 2018 3:20:10 PM >> >> > org.apache.beam.runners.direct.DirectTransformExecutor run >> >> > GRAVE: Error occurred within >> >> > org.apache.beam.runners.direct.DirectTransformExecutor@66761b7a >> >> > com.google.common.util.concurrent.ExecutionError: >> >> > java.lang.NoClassDefFoundError: net/bytebuddy/NamingStrategy >> >> > >> >> > ? >> >> > >> >> > Romain Manni-Bucau >> >> > @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book >> >> > >> >> > >> >> > 2018-04-10 15:13 GMT+02:00 Lukasz Cwik <[email protected] >> >> > <mailto:[email protected]>>: >> >> >> I have found that the simplest setup is to delegate the build/test >> >> >> actions >> >> >> to Gradle. This allows you to run unit tests very easily and since its >> >> >> in >> >> >> the same manner that Gradle would have, you know that if its passing it >> >> >> will >> >> >> pass on the command line and on Jenkins. Here is one site that >> >> >> discusses how >> >> >> to set this up: >> >> >> >> >> >> http://mrhaki.blogspot.com/2016/11/gradle-goodness-delegate-build-and-run.html >> >> >> >> >> >> <http://mrhaki.blogspot.com/2016/11/gradle-goodness-delegate-build-and-run.html> >> >> >> >> >> >> >> >> >> On Tue, Apr 10, 2018 at 8:45 AM Romain Manni-Bucau >> >> >> <[email protected] <mailto:[email protected]>> >> >> >> wrote: >> >> >>> >> >> >>> What's the plan to make idea supporting gradle on beam project? Do we >> >> >>> import the workaround mentionned in >> >> >>> https://youtrack.jetbrains.com/issue/IDEA-175172 >> >> >>> <https://youtrack.jetbrains.com/issue/IDEA-175172>? >> >> >>> For the ones who didn't see this issue in action: idea will compile in >> >> >>> out/ instead of build/ and you will just miss all the resources you >> >> >>> need like some SPI registration which are used by all our registrar => >> >> >>> no way to run tests in idea without hacking the configuration quite >> >> >>> deeply :( >> >> >>> >> >> >>> Romain Manni-Bucau >> >> >>> @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book >> >> >>> >> >> >>> >> >> >>> 2018-04-10 10:08 GMT+02:00 Etienne Chauchot <[email protected] >> >> >>> <mailto:[email protected]>>: >> >> >>>> As a gradle beginner, I could not agree more ! >> >> >>>> +1 >> >> >>>> Etienne >> >> >>>> Le lundi 09 avril 2018 à 18:47 +0200, Jean-Baptiste Onofré a écrit : >> >> >>>> >> >> >>>> Hi all, >> >> >>>> >> >> >>>> I did multiple gradle build since last week and I would like to share >> >> >>>> one of my concern: it's about the communities. >> >> >>>> >> >> >>>> If I think our users won't see any change for them due to Gradle >> >> >>>> build >> >> >>>> (I think that most of our users will still use Maven with artifacts >> >> >>>> provided by Gradle), I'm more concerned by the dev community and the >> >> >>>> contribution. >> >> >>>> >> >> >>>> Maven is well known and straight forward for a large part of >> >> >>>> potential >> >> >>>> contributors. I think we have to keep in mind that we still have to >> >> >>>> grow >> >> >>>> up our contributors community. >> >> >>>> >> >> >>>> Today, maybe I'm wrong, but I have the feeling that gradle build is >> >> >>>> not >> >> >>>> straight forward (build.gradle includes build_rules.gradle, gathering >> >> >>>> all taks all together). >> >> >>>> >> >> >>>> I would like to add a task in the gradle "migration" process: >> >> >>>> simplify >> >> >>>> the gradle structure and files, and document this. >> >> >>>> >> >> >>>> I know we already have a Jira about the documentation part, but I >> >> >>>> would >> >> >>>> like to "polish" and use a clean structure for the Gradle resources. >> >> >>>> As >> >> >>>> already quickly discussed, I think that having one gradle file per >> >> >>>> tasks >> >> >>>> in the .gradle directory would be helpful. >> >> >>>> >> >> >>>> The goal is really to simplify the contribution. >> >> >>>> >> >> >>>> Do you agree if I add a Jira about "Gradle polish" ? >> >> >>>> Thoughts ? >> >> >>>> >> >> >>>> Regards >> >> >>>> JB >> >> >>>> >> >> >>>> On 07/04/2018 04:52, Scott Wegner wrote: >> >> >>>> >> >> >>>> Here's an end-of-day update on migration work: >> >> >>>> >> >> >>>> * Snapshot unsigned dailies and signed release builds are working >> >> >>>> (!!). >> >> >>>> PR/5048 [1] merges changes from Luke's branch >> >> >>>> * python precommit failing... will investigate python precommit >> >> >>>> Monday >> >> >>>> * All Precommits are gradle only >> >> >>>> * All Postcommits except performance tests and Java_JDK_Versions_Test >> >> >>>> use gradle (after PR/5047 [2] merged) >> >> >>>> * Nightly snapshot release using gradle is ready; needs PR/5048 to be >> >> >>>> merged before switching >> >> >>>> * ValidatesRunner_Spark failing consistently; investigating >> >> >>>> >> >> >>>> Thanks for another productive day of hacking. I'll pick up again on >> >> >>>> Monday. >> >> >>>> >> >> >>>> [1] https://github.com/apache/beam/pull/5048 >> >> >>>> <https://github.com/apache/beam/pull/5048> >> >> >>>> [2] https://github.com/apache/beam/pull/5047 >> >> >>>> <https://github.com/apache/beam/pull/5047> >> >> >>>> >> >> >>>> >> >> >>>> On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau >> >> >>>> <[email protected] <mailto:[email protected]> >> >> >>>> <mailto:[email protected] <mailto:[email protected]>>> wrote: >> >> >>>> >> >> >>>> Why building a zip per runner which its stack and just pointing >> >> >>>> out >> >> >>>> on that zip and let beam lazy load the runner: >> >> >>>> >> >> >>>> --runner=LazyRunner --lazyRunnerDir=... --lazyRunnerOptions=... >> >> >>>> (or >> >> >>>> the fromSystemProperties() if it gets merged a day ;)) >> >> >>>> >> >> >>>> Le 6 avr. 2018 20:21, "Kenneth Knowles" <[email protected] >> >> >>>> <mailto:[email protected]> >> >> >>>> <mailto:[email protected] <mailto:[email protected]>>> a écrit : >> >> >>>> >> >> >>>> I'm working on finding a solution for launching the Nexmark >> >> >>>> suite with each runner. This doesn't have to be done via >> >> >>>> Gradle, >> >> >>>> but we anyhow need built artifacts that don't require user >> >> >>>> classpath intervention. >> >> >>>> >> >> >>>> It looks to me like the examples are also missing this - >> >> >>>> they >> >> >>>> have separate configuration e.g. sparkRunnerPreCommit but >> >> >>>> that >> >> >>>> is overspecified compared to a free-form launching of a >> >> >>>> main() >> >> >>>> program with a runner profile. >> >> >>>> >> >> >>>> On Fri, Apr 6, 2018 at 11:09 AM Lukasz Cwik >> >> >>>> <[email protected] <mailto:[email protected]> >> >> >>>> <mailto:[email protected] <mailto:[email protected]>>> wrote: >> >> >>>> >> >> >>>> Romain, are you talking about the profiles that exist as >> >> >>>> part of the archetype examples? >> >> >>>> >> >> >>>> If so, then those still exist and haven't been changed. >> >> >>>> If >> >> >>>> not, can you provide a link to the profile in a pom file >> >> >>>> to >> >> >>>> be clearer? >> >> >>>> >> >> >>>> On Fri, Apr 6, 2018 at 12:40 PM Romain Manni-Bucau >> >> >>>> <[email protected] <mailto:[email protected]> >> >> >>>> <mailto:[email protected] <mailto:[email protected]>>> >> >> >>>> wrote: >> >> >>>> >> >> >>>> Hi Scott, >> >> >>>> >> >> >>>> is it right that 2 doesn't handle the hierachy >> >> >>>> anymore >> >> >>>> and that it doesn't handle profiles for runners as >> >> >>>> it is >> >> >>>> currently with maven? >> >> >>>> >> >> >>>> >> >> >>>> Romain Manni-Bucau >> >> >>>> @rmannibucau <https://twitter.com/rmannibucau >> >> >>>> <https://twitter.com/rmannibucau>> | >> >> >>>> Blog >> >> >>>> <https://rmannibucau.metawerx.net/ >> >> >>>> <https://rmannibucau.metawerx.net/>> | Old Blog >> >> >>>> <http://rmannibucau.wordpress.com >> >> >>>> <http://rmannibucau.wordpress.com/>> | Github >> >> >>>> <https://github.com/rmannibucau >> >> >>>> <https://github.com/rmannibucau>> | LinkedIn >> >> >>>> <https://www.linkedin.com/in/rmannibucau >> >> >>>> <https://www.linkedin.com/in/rmannibucau>> | Book >> >> >>>> >> >> >>>> >> >> >>>> >> >> >>>> <https://www.packtpub.com/application-development/java-ee-8-high-performance >> >> >>>> >> >> >>>> <https://www.packtpub.com/application-development/java-ee-8-high-performance>> >> >> >>>> >> >> >>>> 2018-04-06 18:32 GMT+02:00 Scott Wegner >> >> >>>> <[email protected] <mailto:[email protected]> >> >> >>>> <mailto:[email protected] <mailto:[email protected]>>>: >> >> >>>> >> >> >>>> I wanted to start a thread to summarize the >> >> >>>> current >> >> >>>> state of Gradle migration. We've made lots of >> >> >>>> good >> >> >>>> progress so far this week. Here's the status >> >> >>>> from >> >> >>>> what I can tell-- please add or correct anything >> >> >>>> I >> >> >>>> missed: >> >> >>>> >> >> >>>> * Release artifacts can be built and published >> >> >>>> for >> >> >>>> Snapshot and officlal releases [1] >> >> >>>> * Gradle-generated releases have been validated >> >> >>>> with >> >> >>>> the the Apache Beam archetype generation >> >> >>>> quickstart; >> >> >>>> still needs additional validation. >> >> >>>> * Generated release pom files have correct >> >> >>>> project >> >> >>>> metadata [2] >> >> >>>> * The python pre-commits are now working in >> >> >>>> Gradle >> >> >>>> [3] >> >> >>>> * Ismaël has started a collaborative doc of >> >> >>>> Gradle >> >> >>>> tips [4] as we all learn the new system-- please >> >> >>>> add >> >> >>>> your own. This will eventually feed into >> >> >>>> official >> >> >>>> documentation on the website. >> >> >>>> * Łukasz Gajowy is working on migrating >> >> >>>> performance >> >> >>>> testing framework [5] >> >> >>>> * Daniel is working on updating documentation to >> >> >>>> refer to Gradle instead of maven >> >> >>>> >> >> >>>> If I missed anything, please add it to this >> >> >>>> thread. >> >> >>>> >> >> >>>> The general roadmap we're working towards is: >> >> >>>> (a) Publish release artifacts with Gradle >> >> >>>> (SNAPSHOT >> >> >>>> and signed releases) >> >> >>>> (b) Postcommits migrated to Gradle >> >> >>>> (c) Migrate documentation from maven to Gradle >> >> >>>> (d) Migrate perfkit suites to use Gradle >> >> >>>> >> >> >>>> For those of you that are hacking: thanks for >> >> >>>> your >> >> >>>> help so far! Progress is being roughly tracked >> >> >>>> on >> >> >>>> the Kanban [6]; please make sure the issues >> >> >>>> assigned >> >> >>>> to you are up-to-date. Many of the changes are >> >> >>>> staged on lukecwik's local branch [7]; we'll >> >> >>>> work on >> >> >>>> merging them back soon. >> >> >>>> >> >> >>>> >> >> >>>> [1] >> >> >>>> https://github.com/lukecwik/incubator-beam/pull/7 >> >> >>>> <https://github.com/lukecwik/incubator-beam/pull/7> >> >> >>>> [2] >> >> >>>> https://github.com/lukecwik/incubator-beam/pull/3 >> >> >>>> <https://github.com/lukecwik/incubator-beam/pull/3> >> >> >>>> [3] https://github.com/apache/beam/pull/5032 >> >> >>>> <https://github.com/apache/beam/pull/5032> >> >> >>>> [4] >> >> >>>> >> >> >>>> >> >> >>>> >> >> >>>> https://docs.google.com/document/d/1wR56Jef3XIPwj4DFzQKznuGPM3JDfRDVkxzeDlbdVSQ/edit >> >> >>>> >> >> >>>> <https://docs.google.com/document/d/1wR56Jef3XIPwj4DFzQKznuGPM3JDfRDVkxzeDlbdVSQ/edit> >> >> >>>> [5] https://github.com/apache/beam/pull/5003 >> >> >>>> <https://github.com/apache/beam/pull/5003> >> >> >>>> [6] >> >> >>>> >> >> >>>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=242 >> >> >>>> <https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=242> >> >> >>>> >> >> >>>> [7] >> >> >>>> >> >> >>>> https://github.com/lukecwik/incubator-beam/tree/gradle >> >> >>>> <https://github.com/lukecwik/incubator-beam/tree/gradle> >> >> >>>> -- >> >> >>>> >> >> >>>> >> >> >>>> Got feedback? http://go/swegner-feedback >> >> >>>> <http://go/swegner-feedback> >> >> >>>> >> >> >>>> >> >> >> > >
