Re: portability of xargs
On Tuesday 2022-02-15 07:16, Daniel Herring wrote: > > Maybe a next-generation configuration tool should start by defining interfaces > for user interactions and build tools. This would allow CLI and easy GUI and > IDE users, integration with multiple build systems, static and dynamic > probing, > ... Obligatory https://xkcd.com/927/ response.
Re: portability of xargs
Meson is a candidate for such a next-gen config system. It is in python, which does not quite qualify as usable during early uplift/bootstrap, but there are C ports in progress, see e.g. https://sr.ht/~lattis/muon/ In the meantime, I agree that find is more portable than cards ;-) - Dan Daniel Herring schrieb am Mo., 14. Feb. 2022, 22:17: > Hi Mike, > > It often makes sense to change a project that uses Autotools rather than > modifying the Autotools. Can this argument-splitting behavior be > documented as expected? Are no workarounds available? > > Replacing "overly fancy but proven shell script" with "dependency on a new > tool" may replace an obscure problem for one project with an intractable > problem that breaks portability (to obscure platforms) for all projects. > I have distant memories of working through similar problems by tracing > configure scripts and makefiles, redirecting the problematic command lines > to files, and monkey-patching them before execution. Tedious but doable > due to the simplicity of the tooling, even in a cross compile. Getting a > "functional" xargs would have been MUCH harder in that environment. > > IMO, you are touching on the fundamental issue of Autotools. Portable > shell scripting is one of the worst programming environments available > today, and it hinders progress and adoption, but it is the mostly widely > available bootstrapping environment available. Autotools embodies decades > of hard-won experience, and people rightly want to use it on new projects, > but they are weighed down by all the baggage of history, even when it is > not even remotely relevant. > > As the years go by, this tradeoff is getting less justifiable. I hope a > better bootstrapping environment will come along. Something that ports > the spirit of Autotools to a much better language. Maybe a subset of > Python that can bootstrap from shell? Many would argue for cmake or one > of the newer build systems, but they all seem to miss one or more > fundamentals of the Autotools. At a minimum, they should address the > concerns about imake in the Autoconf FAQ. ;) > > > https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.71/html_node/Why-Not-Imake.html > > Maybe a next-generation configuration tool should start by defining > interfaces for user interactions and build tools. This would allow CLI > and easy GUI and IDE users, integration with multiple build systems, > static and dynamic probing, ... > > -- Daniel > > > On Mon, 14 Feb 2022, Mike Frysinger wrote: > > > context: https://bugs.gnu.org/53340 > > > > how portable is xargs ? like, beyond POSIX, as autoconf & automake both > > support non-POSIX compliant systems. i want to use it in its simplest > > form: `echo $var | xargs rm -f`. > > > > automake jumps through some hoops to try and limit the length of > generated > > command lines, like deleting output objects in a non-recursive build. > it's > > not perfect -- it breaks arguments up into 40 at a time (akin to xargs > -n40) > > and assumes that it won't have 40 paths with long enough names to exceed > the > > command line length. it also has some logic where it's deleting paths by > > globs, but the process to partition the file list into groups of 40 > happens > > before the glob is expanded, so there are cases where it's 40 globs that > can > > expand into many many more files and then exceed the command line length. > > > > if i can assume `xargs` is available, then i think i can replace all of > this > > custom logic with a simple call to that. > > -mike > > > >
Re: portability of xargs
Hi Mike, It often makes sense to change a project that uses Autotools rather than modifying the Autotools. Can this argument-splitting behavior be documented as expected? Are no workarounds available? Replacing "overly fancy but proven shell script" with "dependency on a new tool" may replace an obscure problem for one project with an intractable problem that breaks portability (to obscure platforms) for all projects. I have distant memories of working through similar problems by tracing configure scripts and makefiles, redirecting the problematic command lines to files, and monkey-patching them before execution. Tedious but doable due to the simplicity of the tooling, even in a cross compile. Getting a "functional" xargs would have been MUCH harder in that environment. IMO, you are touching on the fundamental issue of Autotools. Portable shell scripting is one of the worst programming environments available today, and it hinders progress and adoption, but it is the mostly widely available bootstrapping environment available. Autotools embodies decades of hard-won experience, and people rightly want to use it on new projects, but they are weighed down by all the baggage of history, even when it is not even remotely relevant. As the years go by, this tradeoff is getting less justifiable. I hope a better bootstrapping environment will come along. Something that ports the spirit of Autotools to a much better language. Maybe a subset of Python that can bootstrap from shell? Many would argue for cmake or one of the newer build systems, but they all seem to miss one or more fundamentals of the Autotools. At a minimum, they should address the concerns about imake in the Autoconf FAQ. ;) https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.71/html_node/Why-Not-Imake.html Maybe a next-generation configuration tool should start by defining interfaces for user interactions and build tools. This would allow CLI and easy GUI and IDE users, integration with multiple build systems, static and dynamic probing, ... -- Daniel On Mon, 14 Feb 2022, Mike Frysinger wrote: context: https://bugs.gnu.org/53340 how portable is xargs ? like, beyond POSIX, as autoconf & automake both support non-POSIX compliant systems. i want to use it in its simplest form: `echo $var | xargs rm -f`. automake jumps through some hoops to try and limit the length of generated command lines, like deleting output objects in a non-recursive build. it's not perfect -- it breaks arguments up into 40 at a time (akin to xargs -n40) and assumes that it won't have 40 paths with long enough names to exceed the command line length. it also has some logic where it's deleting paths by globs, but the process to partition the file list into groups of 40 happens before the glob is expanded, so there are cases where it's 40 globs that can expand into many many more files and then exceed the command line length. if i can assume `xargs` is available, then i think i can replace all of this custom logic with a simple call to that. -mike
Re: portability of xargs
On 2/14/22 20:03, Mike Frysinger wrote:
are the corner cases known ?
I don't know of a catalog, no.
These days you might have better luck with "find ... -exec ... {} +".
Although standardized more recently than xargs, my vague impression is
that there's less variation among implementations. It works only on file
names, though.
Re: portability of xargs
On 14 Feb 2022 19:53, Paul Eggert wrote: > On 2/14/22 19:45, Mike Frysinger wrote: > > how portable is xargs ? > > It can be a porting problem, unfortunately. There are several corner > cases that various implementations don't get right. I expect this is why > the GNU Coding Standards exclude xargs from the list of programs that > 'configure' and Makefile rules can use. are the corner cases known ? if it's such that xargs doesn't always correctly limit itself to the system limit based on other factors, i can live with that assuming that the -n option is reliable. the trouble here is that in order to implement its own xargs logic, Automake has to expand things exactly once. atm, it happens when invoking rm, but if we want to partition things up, they have to be expanded first once, and then passed to the command directly. without shell arrays or other fancy tools, this is difficult to pull off :(. -mike signature.asc Description: PGP signature
Re: portability of xargs
On 2/14/22 19:45, Mike Frysinger wrote: how portable is xargs ? It can be a porting problem, unfortunately. There are several corner cases that various implementations don't get right. I expect this is why the GNU Coding Standards exclude xargs from the list of programs that 'configure' and Makefile rules can use.
portability of xargs
context: https://bugs.gnu.org/53340 how portable is xargs ? like, beyond POSIX, as autoconf & automake both support non-POSIX compliant systems. i want to use it in its simplest form: `echo $var | xargs rm -f`. automake jumps through some hoops to try and limit the length of generated command lines, like deleting output objects in a non-recursive build. it's not perfect -- it breaks arguments up into 40 at a time (akin to xargs -n40) and assumes that it won't have 40 paths with long enough names to exceed the command line length. it also has some logic where it's deleting paths by globs, but the process to partition the file list into groups of 40 happens before the glob is expanded, so there are cases where it's 40 globs that can expand into many many more files and then exceed the command line length. if i can assume `xargs` is available, then i think i can replace all of this custom logic with a simple call to that. -mike signature.asc Description: PGP signature
bug#19579: Long lines, short awk
On 14 Feb 2022 01:14, Daniel Richard G. wrote: > On Mon, 2022 Feb 7 23:03-05:00, Mike Frysinger wrote: > > can you provide an example that reproduces the issue ? or link to the > > package you're working on (assuming it's OSS) ? i checked my own > > projects and it didn't create a single line, it generated a bunch of > > independent rm commands. i looked through the automake source and > > couldn't quickly identify something that would explain the behavior > > you're seeing. > > I am currently using Automake 1.16.3, which no longer exhibits the > issue originally described. As far as I can tell, this was fixed as a > side effect of Git commit 08849db8, made just ten days before my > initial report. ok, thanks for checking -mike signature.asc Description: PGP signature
