Re: Future plans for Autotools
On Wed, 2021-01-20 at 17:15 -0500, Zack Weinberg wrote: > As a starting point, I wrote up a "strengths, weaknesses, > opportunities, and threats" analysis for Autotools -- this is a > standard project management technique, if you're not familiar with > it, there's a nice writeup in the draft of the book my friend and > colleague Sumana Harihareswara is writing [ > https://changeset.nyc/resources/getting-unstuck-sampler-offer.html ]. Thank you for sharing, Zack (and Sumana). This looks very thoughtful and worth all of us reading at some point. I for one have no intention of migrating away from the Autotools for any of my projects. My personal experience is that every time someone tries to develop a replacement, they tend to reinvent it and in a poorer way. -- Kip Warner -- Senior Software Engineer OpenPGP signed/encrypted mail preferred https://www.thevertigo.com signature.asc Description: This is a digitally signed message part
Re: Future plans for Autotools
* On 2021 20 Jan 17:33 -0600, Bob Friesenhahn wrote: > Autotools is in great danger of becoming irrelevant, at least for new > software development. A lot of people feel hostile toward it. This is quite true. As a co-maintainer of a library project that uses Autoconf, Automake, and Libtool, I've been asked point blank after the most recent major release why we have not switched to cmake. My answer was that I don't have the time nor the desire after spending a considerable amount of time getting my head around Autotools enough to improve and maintain the project's build system with a modicum of sanity. The project builds on any POSIX platform that Autotools can support and MS Windows. Portability of the build system is vital to this project and that is the second priority of the build system. I've built projects from cmake and the Qt equivalent and while their build definition files often seem simple, it appears to me there is a lot that goes on behind the scenes. One strength of the Autotools is that Autoconf and Automake are almost infinitely malleable. Each supports dropping shell syntax in configure.ac or make syntax in a Makefile.am respectively. There just simply need to be ways for the project maintain to extend the build system in ways specific to that project. I've never cared to investigate cmake to know whether it is as capable thus I am unsure if cmake trades control for simplicity. > It seems to me that Autoconf is too difficult to work on. There is too much > to become expert at in order for new volunteers to make an impact. The same > is true for libtool. > > In my opinion, a new "language" designed specifically to meet the needs of > Autoconf should be developed and Autoconf should be re-implemented using it. > There should be no more need to run external utilities like 'sed', or 'awk' > or other things which can be implemented internally in a way which does not > suffer from the white space and delimiter issues that shell script does. I'm not here to defend m4, on the contrary, my life would continue on just fine if I never had to look at m4 ever again. The very few times I have tried have not been pleasant. On the other hand, is there any enthusiasm for reimplementing sed, awk, or other such utilities since Perl did it over 30 years ago and other scripting languages have done so since? The good thing about Automake is that though it is written in Perl, the implementation language doesn't seem to bleed through into the syntax of Makefile.am. With Autoconf, so long as the maintainer can stick with the provided macros or third party ones, life is tolerable. On the rare occasion that I started looking at enhancing a macro, I began questioning some life choices! On the other hand, I am not so quick to abandon shell syntax. It is the lowest common denominator on POSIX like systems. That said, perhaps a reasonable approach would be to target a somewhat older POSIX specification and abandon support for shells that cannot reach even that low bar. As an aside, I have worked to ensure that any shell script that I write is free of Bashisms even though they can make writing scripts easier. > It seems that the core precept that Automake should produce portable > makefiles should be re-evaluated if most systems provide GNU make, or can > easily be updated have it. At the moment the project(s) that seem to be working toward replacing all things GNU do allow the later installation of GNU software. Will such projects eventually become hostile to even the premise of installing GPL software, let alone GNU software, ala iOS? If so, my concerns about support for other make implementations likely become moot as the project I'm involved with is GPL/LGPL licensed. > There is a fork of Automake which was re-done to > be based on GNU Make. This assumes that makefiles written in GNU make can > noticeably improve things. What negative impact would relying solely on GNU make have on third party projects intending to build on platforms where the installation of GNU make may become discouraged? While I've not read of any public plans of such, the actions of FreeBSD make me curious if they might head in that direction eventually. > The support for 'distcheck' is excellent. > > Regardless, thanks for your ideas and the red alert. Agreed on both of these counts, Bob. - Nate -- "The optimist proclaims that we live in the best of all possible worlds. The pessimist fears this is true." Web: https://www.n0nb.us Projects: https://github.com/N0NB GPG fingerprint: 82D6 4F6B 0E67 CD41 F689 BBA6 FB2C 5130 D55A 8819 signature.asc Description: PGP signature
Re: Future plans for Autotools
It seems better not to start another language. with already lack of resources, that will further dilate available resources, and hard to compete with other tools already us9ng Python's mature ecosystem On Wed, Jan 20, 2021 at 3:32 PM Bob Friesenhahn < bfrie...@simple.dallas.tx.us> wrote: > > In my opinion, a new "language" designed specifically to meet the > needs of Autoconf should be developed and Autoconf should be > re-implemented using it. There should be no more need to run external > utilities like 'sed', or 'awk' or other things which can be > implemented internally in a way which does not suffer from the white > space and delimiter issues that shell script does. > -- > Bob Friesenhahn >
Re: Future plans for Autotools
On Wed, 20 Jan 2021, Zack Weinberg wrote: Now we've all had a while to recover from the long-awaited Autoconf 2.70 release, I'd like to start a conversation about where the Autotools in general might be going in the future. Clearly any future development depends on finding people who will do the work, but before we worry about that I think we might want to figure out what work we _want_ to do. Zack, your text was excellent and I agree with all of it. Autotools is in great danger of becoming irrelevant, at least for new software development. A lot of people feel hostile toward it. It seems to me that Autoconf is too difficult to work on. There is too much to become expert at in order for new volunteers to make an impact. The same is true for libtool. In my opinion, a new "language" designed specifically to meet the needs of Autoconf should be developed and Autoconf should be re-implemented using it. There should be no more need to run external utilities like 'sed', or 'awk' or other things which can be implemented internally in a way which does not suffer from the white space and delimiter issues that shell script does. It seems that the core precept that Automake should produce portable makefiles should be re-evaluated if most systems provide GNU make, or can easily be updated have it. There is a fork of Automake which was re-done to be based on GNU Make. This assumes that makefiles written in GNU make can noticeably improve things. I like your idea of supporting other underlying build tools besides 'make'. Make's dependence on matching rules and file time stamps is problematic and it does not scale. It is unfortunate that GNU produced a much more powerful 'make' tool (a paradigm invented in 1976), but not a new build tool with fresh ideas to improve build quality and reduce build times on modern systems. The macro definitions provided by Autoconf have been proven by the test of time and allow the underlying implementation to be changed. Only surrounding shell script might need to be changed if the underlying implementation changes. The support for 'distcheck' is excellent. Regardless, thanks for your ideas and the red alert. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt
Re: Future plans for Autotools
Thanks for writing all of this. I'm writing from the perspective of a long-term user of the autotools. A discussion like the one you have started will likely attract many opinions. Some will be contradictory. However, somebody in the end will have to decide. The challenge seems to be to evolve the system with regard to backward compatibility. You mention the use of many languages, like m4. This brings me to my first suggestion: make automake work without autoconf. This would potentially allow configuration scripts (if there is to be such a thing) to be written in a language that doesn't use m4. The purpose of autoconf, and of configure scripts, could be clarified. The manual says, "Autoconf is a tool for producing shell scripts that automatically configure software source code packages to adapt to many kinds of Posix-like systems." Posix-like systems, and thus portability, is implied to be a major issue. Many would not care about portability, especially when starting a small personal project. (This was my reaction when I first started using the autotools.) I see the purpose of a configure script to be two-fold: * Allowing a user to make choices about what features are built, and how the package is built (e.g. installation paths) * Adapting to operating system variants The first will always be relevant, even if you don't care about the second. I think configuration scripts on GNU-like systems would always have a heavy element of shell script in them, even if m4 or autoconf macros were abandoned. There would be no need to move so far away from this. If automake could generate its own configure script without autoconf, then the interface of the GNU Build System could continue unchanged for people building software. It would also simplify matters for people learning to use the autotools, that they could just create a single Makefile.am to describe their project (like a CMakefile for cmake or .pro file for qmake), rather than having to create Makefile.am and configure.ac. It could be made easier to get started with automake. For example, "AUTOMAKE_OPTIONS=foreign" should be the default; otherwise automake refuses to finish if required files aren't found in the project. There's a political problem here to persuade people of this. The documentation could also be improved. I remember somebody was complaining about this page: https://www.gnu.org/software/automake/manual/html_node/Program-and-Library-Variables.html and asking what "maude" meant - it turned out it was the name of the dog or something of the person who first wrote the program, who didn't want it to change. So think getting consensus is impossible for changes that are needed to make the program simpler, easier and better. There's a vacuum of responsibility where nobody wants to step on others' feet. Forking the project would go nowhere as nobody would use the fork. Your idea about adding support for new programming languages is good and if someone did this along with other changes, it could stimulate adoption of the other changes. It would be nice if all the code required to support a new language could occur as a self-contained "module" within automake. On Wed, Jan 20, 2021 at 10:15 PM Zack Weinberg wrote: > > Now we've all had a while to recover from the long-awaited Autoconf > 2.70 release, I'd like to start a conversation about where the > Autotools in general might be going in the future. Clearly any future > development depends on finding people who will do the work, but before > we worry about that I think we might want to figure out what work we > _want_ to do. > > As a starting point, I wrote up a "strengths, weaknesses, > opportunities, and threats" analysis for Autotools -- this is a > standard project management technique, if you're not familiar with it, > there's a nice writeup in the draft of the book my friend and > colleague Sumana Harihareswara is writing [ > https://changeset.nyc/resources/getting-unstuck-sampler-offer.html ]. > I'm going to paste the full text of this analysis below, please reply > inline. You can also read it on my blog at > https://www.owlfolio.org/development/autoconf-swot/ . > > zw > > > I’ve been a contributor to GNU projects for many years, notably both > GCC and GNU libc, and recently I led the effort to make the first > release of Autoconf since 2012 (release announcement for Autoconf > 2.70). For background and context, see the LWN article my colleague > Sumana Harihareswara of Changeset Consulting wrote. > > Autoconf not having made a release in eight years is a symptom of a > deeper problem. Many GNU projects, including all of the other > components of the Autotools (Automake, Libtool, Gnulib, etc.) and the > software they depend upon (GNU M4, GNU Make, etc.) have seen a steady > decline in both contributor enthusiasm and user base over the past > decade. I include myself in the group of declining enthusiasts; I > would not have done the work leading up to the Autoconf 2.70 release > if I
Future plans for Autotools
Now we've all had a while to recover from the long-awaited Autoconf 2.70 release, I'd like to start a conversation about where the Autotools in general might be going in the future. Clearly any future development depends on finding people who will do the work, but before we worry about that I think we might want to figure out what work we _want_ to do. As a starting point, I wrote up a "strengths, weaknesses, opportunities, and threats" analysis for Autotools -- this is a standard project management technique, if you're not familiar with it, there's a nice writeup in the draft of the book my friend and colleague Sumana Harihareswara is writing [ https://changeset.nyc/resources/getting-unstuck-sampler-offer.html ]. I'm going to paste the full text of this analysis below, please reply inline. You can also read it on my blog at https://www.owlfolio.org/development/autoconf-swot/ . zw I’ve been a contributor to GNU projects for many years, notably both GCC and GNU libc, and recently I led the effort to make the first release of Autoconf since 2012 (release announcement for Autoconf 2.70). For background and context, see the LWN article my colleague Sumana Harihareswara of Changeset Consulting wrote. Autoconf not having made a release in eight years is a symptom of a deeper problem. Many GNU projects, including all of the other components of the Autotools (Automake, Libtool, Gnulib, etc.) and the software they depend upon (GNU M4, GNU Make, etc.) have seen a steady decline in both contributor enthusiasm and user base over the past decade. I include myself in the group of declining enthusiasts; I would not have done the work leading up to the Autoconf 2.70 release if I had not been paid to do it. (I would like to say thank you to the project funders: Bloomberg, Keith Bostic, and the GNU Toolchain Fund of the FSF.) The Autotools are in particularly bad shape due to the decline in contributor enthusiasm. Preparation for the Autoconf 2.70 release took almost twice as long as anticipated; I made five beta releases between July and December 2020, and merged 157 patches, most of them bugfixes. On more than one occasion I was asked why I was going to the trouble—isn’t Autoconf (and the rest of the tools by implication) thoroughly obsolete? Why doesn’t everyone switch to something newer, like CMake or Meson? (See the comments on Sumana’s LWN article for examples.) I personally don’t think that the Autotools are obsolete, or even all that much more difficult to work with than some of the alternatives, but it is a fair question. Should development of the Autotools continue? If they are to continue, we need to find people who have the time and the inclination (and perhaps also the funding) to maintain them steadily, rather than in six-month release sprints every eight years. We also need a proper roadmap for where further development should take these projects. As a starting point for the conversation about whether the projects should continue, and what the roadmap should be, I was inspired by Sumana’s book in progress on open source project management (sample chapters are available from her website) to write up a strengths, weaknesses, opportunities, and threats analysis of Autotools. This inventory can help us figure out how to build on new opportunities, using the Autotools’ substantial strengths, and where to invest to guard against threats and shore up current weaknesses. Followup discussion should go to the Autoconf mailing list. Strengths In summary: as the category leader for decades, the Autotools benefit from their architectural approach, interoperability, edge case coverage, standards adherence, user trust, and existing install base. Autoconf’s feature-based approach to compiled-code portability scales better than lists of system quirks. The Autotools carry 30+ years’ worth of embedded knowledge about portability traps for C programs and shell-based build scripting on Unix (and to a lesser extent Windows and others), including variants of Unix that no other comparable configuration tool supports. Autoconf and Automake support cross-compilation better than competing build systems. Autoconf and Automake support software written in multiple languages better than some competing build systems (but see below). Autoconf is very extensible, and there are lots of third-party macros available. Tarball releases produced by Autotools have fewer build dependencies than tarball releases produced by competing tools. Tarball releases produced by Autotools have a predictable, standardized (literally; it’s a key aspect of the GNU Coding Standards) interface for setting build-time options, building them, testing them, and installing them. Automake tries very hard to generate Makefiles that will work with any Make implementation, not just GNU make, and not even just (GNU or BSD) make. The Autotools have excellent reference-level documentation (better than CMake and Meson’s). As they are GNU projects, users can
