I agree with the points made that our Gradle configuration is too complicated (learning Groovy and BeamModulePlugin come to mind).
But I mainly write for Python SDK, and I've done experiments in the past with Bazel builds, which was a much better fit for Python as an incremental and parallel build and test system. On Thu, Oct 11, 2018 at 4:56 AM Alexey Romanenko <aromanenko....@gmail.com> wrote: > I agree with all previous points related to Gradle-related problems. Among > all others, I’d highlight IDE integration issue since it can have real > negative impact on attracting new contributors and, finally, community > grow. The more obstacles there are to start code contribution, the more > chances that it will distract newcomers. > > In this way I see two points to do: > 1) Continue working on IDE integration improvement. I hope there are no > principal barriers for that and it’s only a question of applied forces, > otherwise, it can be a very serious issue. > 2) Improve our “Get started” documentation for contributors to make the > process of first-time contribution as smooth and simple as possible. > > > On 11 Oct 2018, at 00:22, Tim Robertson <timrobertson...@gmail.com> wrote: > > Thank you JB for starting this discussion. > > Others comment on many of these points far better than I can, but my > experience is similar to JB. > > 1. IDEA integration (and laptop slowing like crazy) being the biggest > contributor to my feeling of being unproductive > 2. Not knowing the correct way to modify the build scripts which I put > down to my own limitations > > It seems we also need to help build Gradle expertise in our community, so >> that those that are motivated are empowered to contribute. > > > Nicely phrased. +1 > > > > On Wed, Oct 10, 2018 at 7:15 PM Scott Wegner <sc...@apache.org> wrote: > >> > Perhaps we should go through and prioritize (and add missing items to) >> BEAM-4045 >> >> +1. It's hard to know where to start when there's such a laundry list of >> tasks. If you're having build issues, will you make sure it is represented >> in BEAM-4045, and "Vote" for the issues that you believe are the highest >> priority? >> >> I agree that the Gradle build is far from perfect (my top gripes are IDE >> integration and parallel/incremental build support). I believe that we're >> capable of making our build great, and continuing our investment in Gradle >> would be a shorter path than changing course again. Remember that our Maven >> build also had it's share of issues, which is why we as a community voted >> to replace it [1][2]. >> >> It seems we also need to help build Gradle expertise in our community, so >> that those that are motivated are empowered to contribute. Does anybody >> have a good "Getting Started with Gradle" guide they recommend? Perhaps we >> could also link to it from the website/wiki. >> >> [1] >> https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E >> [2] >> https://lists.apache.org/thread.html/bd399ecb17cd211be7c6089b562c09ba9116649c9eabe3b609606a3b@%3Cdev.beam.apache.org%3E >> >> On Wed, Oct 10, 2018 at 2:40 AM Robert Bradshaw <rober...@google.com> >> wrote: >> >>> Some rough stats (because I was curious): The gradle files have been >>> edited by ~79 unique contributors over 696 distinct commits, whereas the >>> maven ones were edited (over a longer time period) by ~130 unique >>> contributors over 1389 commits [1]. This doesn't capture how much effort >>> was put into these edits, but neither is restricted to a small set of >>> experts. >>> >>> Regarding "friendly for other languages" I don't think either is >>> necessarily easy to learn, but my impression is that the maven learning >>> curve shallower for those already firmly embedded in the Java ecosystem >>> (perhaps due to leveraging existing familiarity, and perhaps some due to >>> the implicit java-centric conventions that maven assumed about your >>> project), whereas with gradle at least I could keep pulling on the string >>> to unwind things to the bottom. The "I just want to build/test X without >>> editing/viewing the build files" seemed more natural with Gradle (e.g. I >>> can easily list all tasks). >>> >>> That being said, I don't think everyone needs to understand the full >>> build system. It's important that there be a critical mass that do (we have >>> that for both, and if we can simplify to improve this that'd be great), >>> it's easy enough to do basic changes (e.g. add a dependency, again I don't >>> think the barrier is sufficiently different for either), and works well out >>> of the box for someone who just wants to look up a command on the website >>> and edit code (the CLI is an improvement with Gradle, but it's clear that >>> (java) IDE support is a significant regression). >>> >>> Personally, I don't know much about IDE configuration (admittedly the >>> larger issue), but one action item I can take on is trying to eliminate the >>> need to do a "git clean" after building certain targets (assuming I can >>> reproduce this). >>> >>> Perhaps we should go through and prioritize (and add missing items to) >>> BEAM-4045 >>> https://issues.apache.org/jira/issues/?jql=parent%20%3D%20BEAM-4045%20ORDER%20BY%20priority%20DESC >>> ? There's always a long tail with this kind of thing, and looking at the >>> whole list can be daunting, but putting it in the correct order and >>> knocking off the top N items could possibly go a long way. >>> >>> - Robert >>> >>> [1] The commands I ran were (with and without the uniq) >>> >>> $ find . -name 'build.gradle' | xargs git log | grep Author: | grep -o >>> '[^< ]*@' | sort | uniq | wc >>> $ find . -name 'pom.xml' | xargs git log | grep Author: | grep -o '[^< >>> ]*@' | sort | uniq | wc >>> >>> On Wed, Oct 10, 2018 at 10:31 AM Etienne Chauchot <echauc...@apache.org> >>> wrote: >>> >>>> Hi all, >>>> I must admit that I agree on the status especially regarding 2 points: >>>> 1. new contributors obstacles: gradle learning curve might be too long >>>> for spare-time contributors, also complex scripted build takes time to >>>> understand comparing to self-descriptive one. >>>> 2. IDE integration kind of slows down development. >>>> >>>> Now, regarding how we improve the situation, I think we need to discuss >>>> and identify tasks and tackle them all together even if they are not sexy >>>> tasks as Ismaël mentioned. >>>> >>>> Etienne >>>> >>>> Le mardi 09 octobre 2018 à 10:04 +0200, Jean-Baptiste Onofré a écrit : >>>> >>>> Hi guys, >>>> >>>> >>>> I know that's a hot topic, but I have to bring this discussion on the >>>> table. >>>> >>>> >>>> Some months ago, we discussed about migrating our build from Maven to >>>> >>>> Gradle. One of the key expected improvement was the time to build. >>>> >>>> We proposed to do a PoC to evaluate the impacts and improvements, but >>>> >>>> this PoC was actually directly a migrate on master. >>>> >>>> >>>> Now, I would like to bring facts here: >>>> >>>> >>>> 1. Build time >>>> >>>> On my machine, the build time is roughly 1h15. It's pretty long, and >>>> >>>> regarding what the build is doing, I don't see huge improvement provided >>>> >>>> by Gradle. >>>> >>>> 2. Build reliability >>>> >>>> Even worse, most of the time, we need to use --no-parallel and >>>> >>>> --no-daemon to have a reliable build (it's basically recommended for >>>> >>>> release). It has an impact on build time, and we loose part of Gradle >>>> >>>> benefits. >>>> >>>> 3. Release and repositories >>>> >>>> Even if couple of releases has been performed with Gradle, it's not >>>> >>>> obvious to see improvements around artifacts handling. I got my >>>> >>>> repository polluted twice (that's part of the trick Gradle is doing to >>>> >>>> speed up the build dealing around the repository). >>>> >>>> 4. IDE integration >>>> >>>> We already had some comments on the mailing lists about the IDE >>>> >>>> integration. Clearly, the situation is not good on that front too. The >>>> >>>> integration on IDE (especially IntelliJ) is not good enough right now. >>>> >>>> >>>> We are working hard to grow up the community, and from a contributor >>>> >>>> perspective, our build system is not good today IMHO. >>>> >>>> As a contributor, I resumed my work on some PRs, and I'm spending so >>>> >>>> much time of the build, largely more than working on the PRs code itself. >>>> >>>> >>>> So, obviously, the situation is not perfect, at least from a contributor >>>> >>>> perspective. >>>> >>>> >>>> The purpose of this thread is not again to have a bunch of replied >>>> >>>> ending nowhere. I would like to be more "pushy" and let's try to be >>>> >>>> concrete. So basically, we only have two options: >>>> >>>> >>>> 1. Improve the build, working hard on Gradle front. Not sure if it makes >>>> >>>> such sense from a contributor perspective, as Maven is really well known >>>> >>>> from most of contributors (and easier to start with IMHO). >>>> >>>> 2. Back on Maven. That's clearly my preferred approach. IDE integration >>>> >>>> is better, Maven is well known from the contributors as already said. >>>> >>>> The effort is not so huge. We tried to use Gradle, we don't have the >>>> >>>> expected results now, that's not a problem, it's part of a project >>>> lifetime. >>>> >>>> >>>> Thoughts ? >>>> >>>> >>>> Regards >>>> >>>> JB >>>> >>>> >>>> >>>> >> >> -- >> >> >> >> >> Got feedback? tinyurl.com/swegner-feedback >> > >
smime.p7s
Description: S/MIME Cryptographic Signature
