A NOTE has been added to this issue. ====================================================================== https://austingroupbugs.net/view.php?id=1801 ====================================================================== Reported By: mohd_akram Assigned To: ====================================================================== Project: Issue 8 drafts Issue ID: 1801 Category: Shell and Utilities Type: Enhancement Request Severity: Editorial Priority: normal Status: Resolved Name: Mohamed Akram Organization: User Reference: Section: xargs Page Number: 3600-3601 Line Number: 123162, 123252 Final Accepted Text: See https://austingroupbugs.net/view.php?id=1801#c6657. Resolution: Accepted As Marked Fixed in Version: ====================================================================== Date Submitted: 2024-01-25 21:39 UTC Last Modified: 2024-02-21 00:20 UTC ====================================================================== Summary: xargs: add -P option ====================================================================== Relationships ID Summary ---------------------------------------------------------------------- related to 0001811 xargs: add -P option to FUTURE DIRECTIO... ======================================================================
---------------------------------------------------------------------- (0006670) gabravier (reporter) - 2024-02-21 00:20 https://austingroupbugs.net/view.php?id=1801#c6670 ---------------------------------------------------------------------- > For example, how are the args read from stdin supposed to be apportioned amongst the parallel invocations of the utility? One might expect that perhaps the idea is to read enough args until the command line limit is reached, and then start reading more for the next parallel invocation. But that means that if there happen to be not all that many args, then perhaps no parallelism would be achieved, when if less args had been allocated to each invocation, more parallel instances could have been invoked. All of GNU findutils, FreeBSD, OpenBSD, Illumos, BusyBox and Toybox read enough arguments until the command line limit is reached, i.e. `seq 1 100 | xargs -P10 -n70` prints in 2 processes (the first using 70 arguments and the second using 30 arguments) and thus results in two lines of output, rather than being printed in 10 lines as would be expected if split between 10 processes. > Beyond that, "execute ... concurrently" might be read as meaning that all the (up to maxprocs) invvocations should be made to run at the same time. To achieve that, the implementation would need to start them all at the same time, otherwise the first started might finish before the last is ready to commence, so they wouldn't be running concurrently. I doubt that's what is intended, but perhaps. I don't exactly get what you mean by "run at the same time". Are you implying this would require an xargs implementation to use some kind of implementation-specific system call that would execve an array of processes all at the same time ? (I am not aware of any system call that would accomplish this). I don't really understand what this objection is about, exactly, or how any such thing would be observable to any program. None of the implementations am I looking at do anything particularly unique in this regard, from what I can see. > And even more, how are these parallel invocations expected to interact with the CONSEQUENCES OF ERRORS section of the spec (page 3603 in D4 of I8). That is, if an invocation of utility does exit 255, or is killed by a signal, how is xargs supposed to terminate without processing any further input, when it has already processed more, and started more invocations of the utility? For xargs to "undo processing" of further input if it has already processed it would require xargs to implement time-travel. I have not yet been able to find an implementation with this capability, so I would recommend against trying to impose such a requirement, although of course xargs must stop processing any input it has not processed yet. > And what is to be done with those other invocations still running when one exits in one of those ways - are they to be killed, orphaned, or is xargs to wait for them to finish before terminating? In that latter case, should more diagnostic messages be written if more of the invocations also exit 255, or via a signal? Pretty much every implementation differs in this regard: - GNU findutils writes a diagnostic and waits for other invocations to finish before terminating. If another invocation also exits with 255, it writes another diagnostic and then proceeds to immediately invoke undefined behavior by calling exit within an atexit handler. On my machine that results in it exiting after printing that second diagnostic (leaving the remaining invocations orphaned) - FreeBSD writes a diagnostic and waits for other invocations to finish before terminating, and prints more diagnostics if other invocations also exit with 255 - OpenBSD and illumos write a diagnostic and immediately exit, leaving the other invocations orphaned - BusyBox has the same behavior as GNU findutils has on my machine (prints diagnostic, waits, prints another diagnostic if another invocation also exits with 255 and exits then) but manages to do so without invoking undefined behavior, at least - Toybox writes a diagnostic and then proceeds to continue execution as though everything is fine (it does this without -P too so that seems clearly just non-conforming...) I do agree the wording is not enough as-is, though. Personally I would recommend rewriting parts of the description to describe xargs as waiting until less than N invocations are currently running (where N is the value of -P, with a default of 1) before it starts another process, although I have no idea what exactly should be specified for cases where xargs would normally exit without processing any further input, given practically every single implementation behaves differently, except that I would exclude the behaviors from the GNU, Busybox and Toybox implementations, which seem clearly broken to me. Issue History Date Modified Username Field Change ====================================================================== 2024-01-25 21:39 mohd_akram New Issue 2024-01-25 21:39 mohd_akram Name => Mohamed Akram 2024-01-25 21:39 mohd_akram Section => xargs 2024-01-25 21:39 mohd_akram Page Number => 3600-3601 2024-01-25 21:39 mohd_akram Line Number => 123162, 123252 2024-02-15 16:47 geoffclare Relationship added related to 0001811 2024-02-15 16:52 Don Cragun Note Added: 0006657 2024-02-15 16:53 Don Cragun Status New => Resolved 2024-02-15 16:53 Don Cragun Resolution Open => Accepted As Marked 2024-02-15 16:55 Don Cragun Note Edited: 0006657 2024-02-15 16:55 Don Cragun Tag Attached: issue9 2024-02-15 16:59 Don Cragun Note Edited: 0006657 2024-02-15 17:01 Don Cragun Final Accepted Text => See https://austingroupbugs.net/view.php?id=1801#c6657. 2024-02-16 11:31 kre Note Added: 0006660 2024-02-21 00:20 gabravier Note Added: 0006670 ======================================================================
