Re: [PATCH] core-count: A new program to count the number of cpu cores
On 31/05/11 01:28, James Youngman wrote: > [ CC += bug-findutils, += Paolo, -= bug-coreutils ] > > 2009/11/3 Pádraig Brady : >> Pádraig Brady wrote: >>> Paolo Bonzini wrote: >>>> Maybe we want a --parallel option (too bad -p is taken) for xargs that >>>> forces the creation of the number of processes passed with -P or taken >>>> from nproc (for example by starting "md5sum $1 $5 $9 ...", "md5sum $2 $6 >>>> $10 ...", etc.)? >>>> That would be an interesting alternative to this core-count proposal... >>> >>> I'm not sure what you mean here. >>> I already suggested to the xargs maintainer that `xargs -P` >>> should be equivalent to xargs -P$(nproc). >>> `nproc` as an external command would still be useful though. >> >> Here's a patch for that. >> It needs to be updated to reference the new gnulib >> when Bruno's nproc update hits gnulib. > > I'm sorry, I totally missed this email because although it was sent to > me it got buried under a ton of gnulib stuff. > > As I understand things the intent of the patch is to (optionally I > suppose) make xargs keener to keep the CPUs busy (or whatever else > we're parallelising over I guess), even if it needs to launch > processes with short command lines to do so? That is With -P N, if > there are less than N running children and xargs reads an argument it > should launch a new child process? If that's the approximate idea, I > think that there could be use cases for this. > > Could we discuss this aspect on bug-findutils instead? Hence I > changed the subject and the CC list... The patch presented was flawed due to optional parameter processing. It was just to do support `xargs -P $(nproc)` without needing `nproc`. Also discussed was changing distribution of input to children. I.E. by adding a --parallel option which on a machine with 4 available cores would: $ seq 1 13 | xargs --parallel 1 5 9 13 2 6 10 3 7 11 4 8 12 Note -P $(nproc) is implicit in the above. I also discussed a mechanism using non blocking reading of input, to more efficiently split the input parameters across children. memory definitely hazy on this one :) cheers, Pádraig.
SIGPIPE warning
I noticed xargs output a warning about SIGPIPE $ yes 1234 | xargs -n1 | head -n1 1234 xargs: /bin/echo: terminated by signal 13 That's not normally the case on the shell: $ yes 1234 | head -n1 1234 Nor in other process management utils: $ timeout 1 yes 1234 | head -n1 1234 Could we supress that warning? At least for SIGPIPE? cheers, Pádraig.
Re: SIGPIPE warning
On 04/20/2013 02:59 PM, James Youngman wrote: > On Fri, Mar 9, 2012 at 10:41 PM, Pádraig Brady wrote: >> >> I noticed xargs output a warning about SIGPIPE >> >> $ yes 1234 | xargs -n1 | head -n1 >> 1234 >> xargs: /bin/echo: terminated by signal 13 >> >> That's not normally the case on the shell: >> >> $ yes 1234 | head -n1 >> 1234 >> >> Nor in other process management utils: >> >> $ timeout 1 yes 1234 | head -n1 >> 1234 >> >> Could we supress that warning? >> At least for SIGPIPE? > > > > Hmm, POSIX > (http://pubs.opengroup.org/onlinepubs/009695399/utilities/xargs.html) > seems to require it: > > CONSEQUENCES OF ERRORS > >If a command line meeting the specified requirements cannot be > assembled, the utility cannot be invoked, an invocation of the utility > is terminated by a signal, or an invocation of the utility exits with > exit status 255, the xargs utility shall write a diagnostic message > and exit without processing any remaining input. > > > In this case the utility (echo) is indeed being terminated by a > signal.But IMO the report that echo was killed by a signal isn't > helpful to the user in this case. I think that "yes" gets away with > this since it is not standardised by POSIX. > > If you can think of a way for xargs still to be POSIX-compliant while > suppressing the message, that would probably be a good choice.Any > ideas? Well to the letter of the law it would break POSIX "compliance". Though this is in the context of "ERRORS", whereas SIGPIPE could be thought of normal process flow, and is handled as such elsewhere. I'd be inclined to only apply the letter of the law if the POSIXLY_CORRECT env variable was set. Also it might be something to bring up with POSIX. thanks, Pádraig.
Fwd: bug#28079: find -type l -xtype and recursive symbolic links
Forwarding to the appropriate list... Forwarded Message Subject: bug#28079: find -type l -xtype and recursive symbolic links Resent-Date: Sun, 13 Aug 2017 16:44:01 + Resent-From: Poor Yorick Resent-CC: bug-coreut...@gnu.org Date: Sun, 13 Aug 2017 14:06:19 +0300 From: Poor Yorick To: 28...@debbugs.gnu.org When a find expression includes both the -type and -xtype options and a symbolic link that points to itself is encountered (a recursive symlink), find fails with the error message, "Too many levels of symbolic links". the -L option combined with the -type option also triggers the error: find -L . -type l One reason to use both -type and -xtype is to skip broken symbolic links: -type l -not -xtype l It would be useful in this case for a symbolic link that points to itself to be considered broken by failing the "-xtype l" test. If the bug was fixed without also making this adjustment, there would be no way to exclude symbolic links that point to themselves. One alternative would be to introduce a new type specifically for a symbolic link that points to itself. Another alternative would be to introduce some sort of meta option to pass information about the current file to an option. Maybe it would look like this: -meta -lname name -- Yorick
Re: bug#29033: [platform-testers] new snapshot available: gzip-1.8.32-4606
On 12/11/17 15:22, Bernhard Voelker wrote: > On 11/11/2017 03:30 PM, Bruno Haible wrote: >> Hi Pádraig, Bernhard, >> >> You may have seen that Paul made the gnulib module 'year2038' [1] useful for >> all >> platforms, especially Linux/i386. >> >> I think it would be a good idea for coreutils and findutils to include this >> module, >> in order to raise awareness of the year 2038 problem for those people who >> build >> binaries that will be in use for a long time (e.g. embedded and busybox >> environments). >> >> Bruno >> >> [1] >> https://www.gnu.org/software/gnulib/manual/html_node/Avoiding-the-year-2038-problem.html > > Good idea - patch attached. +1 thanks!
