Hi, William,
> > but I don't know that it's really relevant to building > > NetBeans. It is nice, and does the kind of parallelization you're > > after, but it's more like a meta-build system - a build system for > > running other build systems. FWIW, NetBSd/SmartOS's pkgsrc is also > > similar, but makefile-based and more aggressive about rebuilding > > dependencies. > > Guess you missed that my Netbeans ebuilds wrap nothing... They call > javac directly. NO ant, maven, gradle etc. It CAN wrap other build > systems, or it can BE the build system :) > Great. Got a link to this work? > In my case portage IS the build system for Netbeans. Its very trivial. > Probably the most simplistic way to build Netbeans. > Gentoo-prefix (portable gentoo build system that uses a custom $PREFIX - https://wiki.gentoo.org/wiki/Project:Prefix might be useful for running on other OS's [but probably not Windows]). It pretty well as long as you're not trying to install something that needs to create OS-level users or things like that. > > If you were looking at replacing the Ant based system, the first > > place I'd look is Maven, since there is already support for building > > modules with it, so a lot of the heavy lifting is already done. > > Maven is dead... Nothing new should be done in Maven. There is little > reason to use Maven over Gradle. The build aspect of Maven itself is > reason enough not to use it. > > Plexus and Modello are underlying to how Maven is built. It seems > nothing uses that stuff. I bet few here even do anything with it. > https://github.com/codehaus-plexus > https://github.com/codehaus-plexus/modello > > There are enough things that generate Java code from something else, > IMHO way to many. At least Gradle uses Groovy which other stuff uses. > Nothing uses the technology that Maven uses to build itself. > > Ant and Gradle are trivial to build compared to Maven, its junk IMHO. > We'll have to agree to disagree there. In the comments on my blog linked earlier, you'll find a comment added be me a couple of years ago: I spent a few hours the other day in Gradleware's offices, debating build system philosophy with Hans Doktor, the father of Gradle, before we moved on to more productive topics. To me the bottom line is this: There is a lot of badly engineered stuff out there. If your goal is to have the net amount of badly engineered stuff go down, you find out what are common ailments, and try to design systems that make those ailments an impossibility. If your goal is to make money, you write something that mitigates the harm that bad habits do, so people can keep practicing them. Nobody actually wants to hear *You are thinking about your own software in a way that does not reflect reality* - they'd much rather hear *Poor baby, did the mean engineer say you got things wrong? We'll make it so you can go right on doing what you've been doing*. Bottom line: People do dumb things when building software. And they'll pay money to someone who will facilitate letting them keep doing dumb things, because they imagine that is cheaper than ripping the dumb things out. If you would prefer not to do dumb things, don't let something in the door that's designed to make that easy. I get that there's a lot of money to be made helping people shoot themselves in the foot and feel less pain. But I do not want to shoot myself in the foot, or anyone else to - the world is a better place with less of that. These are my basic problems with Gradle: 1. Scriptable build systems increase the probability that your build will, over time - Slowly become more and more unmaintainable - Require the dreaded four hours of "how to set your machine up so builds work" that, pre-Maven, was the norm 2. The ability to script, or have ad-hoc conditionals in a build script means the only way a tool can figure out what it does is to run it and hope it will do the same thing the next time One of the things an IDE needs to be able to do is to look at a description of what to build, *reason* about it, and *make the minimal modification that will change its behavior in some way the user wants.* That is impossible with Ant (and is the reason NetBeans Ant projects create the dreaded inscrutable nbproject/build-impl.xml). It is impossible with Gradle. It *is* possible with Maven - precisely because if you are compiling Java, you are using the compiler plugin; if you're running tests, you're using the surefire plugin; etc. The key difference is that a build system should *describe what to do*, not *describe how to do it.* It's the same difference as the difference between SQL and looking things up in a wad of memory. The SQL can be optimized, because it doesn't say "read five bytes at offset 3027, another five bytes at offset 4052", it says "find the last names of people named Joe" and lets the back end decide how to do it, and anything in between it and the back end can do transforms on that SQL and actually know *exactly how it's changing the semantics.* If the build system *describes what to do*, you can build tools that reason about that - that is why there is so much excellent tooling around SQL, or HTTP - that's what lets you build higher level tools, such as IDEs that, say, can add building a new kind of artifact, or signing JARs or whatever into a build and *not break anything*, or things that convert from one kind of project to another, or adapt builds into a packaging system such as Portage or Pkgsrc. Anyway, unless someone here is going to actually do the work of converting the NetBeans build system to Gradle, none of this is terribly relevant. > > If you're looking to parallelize building modules, that probably > > could be done either with the existing Ant system, or possibly with > > Maven - I don't know off the top of my head something for > > parallelizing a multi module build, but it's likely to be a solved > > problem. > > IMHO the ant build system is a bit bulky and has source/targets in to > many places. I am not anti-ant, I do still have legacy projects in ant. > But anything new is Gradle. I see few reasons really to remain with ant. > I see absolutely no reason to ever use Maven for anything that is not > already Maven. > > XML is dead, and we should kill it ASAP. Ant and Maven continue this > XML legacy, most the rest of the world has moved passed horrendous XML. > I'll agree that the industry had an unhealthy fetish with XML, and both Ant and Maven are victims of it. I know Maven did add some sort of Groovy thing a few years back - and anyway, there are an infinity of config file markup languages (YAML, etc.) that could be adapted to replace XML there. > > IMO, Gradle is a step backward for build systems - scriptability > > leads to fragile systems, and ones that are impossible for tools to > > reason about. > > > > https://timboudreau.com/blog/maven/read > > Compared to Netbeans heavy ant modifications... > https://github.com/apache/incubator-netbeans/tree/ > master/nbbuild/antsrc/org/netbeans/nbbuild > > Netbeans ant build can be fragile. It was next to impossible to wrap > its ant build system in Gentoo's portage. Agreed, the Ant setup is pretty bespoke. I imagine you ran into it wanting to build all of its own dependencies. > Such that it was easier to > bypass it entirely. I believe the result of what I am working with is > considerably less complex. No java code, no extending a build system, > no plugins etc. > Interesting. Given the way Portage builds each package under /var/tmp/portage/$TYPE/$PACKAGE, did you avoid having a separate netbeans source checkout for each module you build? > > Just building Netbeans using my ebulids is SLOW... Speed was not > the goal. Flexibility on usage, being able to create your own meta > package, project build on Netbeans. Or a user specific IDE with only > modules they care about etc. If someone only wants what they need, it > could be less than building via existing build system. The harness > stuff. > In theory, you *should* be able to get portage to parallelize building multiple modules for free, since it's managing the build order based on the dependency graph. With a shared checkout, you'd have to be very careful to ensure two concurrent builds don't ever alter the same files at the same time, though. Without it, yeah, performance is going to be awful. > IMHO faster hardware is not always the answer to a speed solution. Code > reduction and efficiency etc could be more of a factor. Combine that > with faster hardware. You have a real winner :) > You're not thinking closely about what building software is really *doing*. By a huge margin, the majority of the work is reading and writing files - that's literally what a compiler does. Computationally, what compilers do is fairly cheap (though it can be memory intensive). Test it yourself - efficiency is great, but speeding up I/O or minimizing it is going to swamp microoptimizations by orders of magnitude. If you want to optimize a build, start by looking for I/O that is done multiple times identically. > Take portage, systems have gotten faster, but Gentoos portage slower > IMHO due to bad coding and maybe use of Python vs C or other. > Interesting that others stuff like Bazel and Buck also use mostly Java > and some python. Rather than all python. > The main thing I've noticed slowing down Portage is the size of the things commonly in use, and the build systems those things use. Chromium is a monster; electron builds most of chromium as well. Seems like the mainstreaming of things that embed v8 is the biggest culprit; and build/dependency-management systems like Maven, Gradle, Cargo, facilitate people writing things that have vast dependency trees. Great for reuse, sucks if you build it from source. > For that matter, there is probably a lot of low hanging fruit in > > improving performance of the existing Ant build - I don't know that > > much effort has ever been put into that. > > Netbeans build is not that slow IMHO. Though could be faster. I think > the main reason to move past Ant is its mostly yesterdays technology. > IMHO Ant was Java's 1st generation build system. Maven 2nd generation, > and Gradle is 3rd and current. > I'd say Gradle is more like 1.5 :-) > Given the rise of Meson, that may pave the way for more Gradle usage. > The one thing Gradle has almost nothing else does is gradle wrapper, > gradlew. The ability to install and run itself from a minimal script. > That alone is super useful for an environment that is not ready for a > project build. Get your JDK, run gradlew, and your off to the races. > Or: Gradle devs can't be bothered with keeping compatibility with themselves, and gradlew is the hack to work around that :-) > But the work to switch Netbeans to Gradle would be considerable. Hard > to say if thats worth it. Other than moving things forward vs remaining > the same. But it is some what moot. > I'd say keep looking. Your ebuild stuff sounds neat, though. It's probably too esoteric a toolchain to consider as a replacement - you'd like it, I'd like it, and the rest of the world would scream "why do I have to install the bones of another operating system under my real one just to build this thing?" -Tim -- http://timboudreau.com
