Re: "builtin jobs" does not output to stdout.
On 2/14/23 7:09 PM, Koichi Murase wrote: I still don't think `builtin jobs' is equivalent to `jobs' because `jobs' can pick up a shell function. You're right; they're not equivalent. `builtin' guarantees you'll execute the builtin instead of any function replacement. I wish POSIX had picked it up instead of inventing `command'. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: "builtin jobs" does not output to stdout.
On 2/15/23 2:52 AM, Koichi Murase wrote:
two or three command substitutions are generally not considered "so
many command substitutions".
I can't reproduce a great deal of unresponsiveness with five or six
either, and my computer is very old too. I think this "delay" you
mentioned has more to do with the commands being substituted than bash.
You might hear of Fish shell is good, but which part of Fish shell is
considered good? They are just good for interactive behaviors. When we
focus on the language design, Fish shell is actually worse than Bash.
It's all about priorities; if your top priority is fashionable prompt
strings, you won't mind the inferior language.
Of course, there is no case where ${ list; } is "absolutely
necessary". But in that logic, the shell functions are not "absolutely
necessary", and even the command substitutions are not "absolutely
necessary" as everything can be in principle processed in combinations
of pipelines and { list; }, etc.
But they benefit the user tremendously.
OK, I understand what you try to say, (though I regard "no one ever"
and "any sense" as exaggerations; at least the developer who
introduced the feature should have had the reasoning and also should
have used it.
The developer doesn't count as an organic user. And ksh93 has been
around for three decades; if there were demand for the feature in
question, other shells would have copied it already.
I still think it benefits the users, even though it can be thought of
kind of syntax sugar for `list > tmp; var=$(< tmp)' (as managing the
temporary files properly is usually.non-trivial).
How many forks does that avoid anyway? Let's be realistic, the overhead
from one fork is not enough to warrant new syntax.
Re: "builtin jobs" does not output to stdout.
2023年2月15日(水) 1:20 Chet Ramey :
>
> On 2/13/23 6:43 AM, Koichi Murase wrote:
>
> > I guess just the support for ksh's ${ list; } [1] would make
> > everything simple and clear. One can simply call ${ jobs; }, ${ trap
> > -p; }, etc. without thinking about subshells.
> >
> > [1] https://lists.gnu.org/archive/html/help-bash/2020-05/msg00077.html
>
> The text about syntatic sugar still applies.
> > I haven't checked what POSIX says about the jobs in subshells,
>
> POSIX requires that the shell execution environment include what is
> essentially the jobs list;
Thank you for the information. This is what I wanted to find. Thank
you. (At that time, I just quickly searched for the job list, but I
couldn't find it. Also the definition of "a job" didn't seem to be
given in the standard, but I now found "Background Jobs" in XBD)
> https://www.austingroupbugs.net/view.php?id=1254
>
> discusses this extensively. However,
>
> "The jobs utility does not work as expected when it is operating in its own
> utility execution environment because that environment has no applicable
> jobs to manipulate."
>
>
> > but at
> > least Bash maintains a separate "job list" of a subshell (which is
> > accessible from the built-in command `jobs') regardless of whether the
> > "job control" is turned on or not.
>
> POSIX requires this:
>
> "The jobs utility is not dependent on the job control option, as are the
> seemingly related bg and fg utilities because jobs is useful for examining
> background jobs, regardless of the condition of job control."
Thank you for all the information.
--
Koichi
Re: "builtin jobs" does not output to stdout.
2023年2月15日(水) 0:51 Chet Ramey : > On 2/11/23 11:14 AM, Koichi Murase wrote: > > 2023年2月12日(日) 0:42 Robert Elz : > >> Why would you want to ever say "builtin jobs" though? > >> The jobs command has to be buikt in to work. > > > > `jobs' can be overwritten by a shell function. For example, when a > > user wants to modify the behavior of `jobs' for interactive uses, a > > typical solution is to override `jobs' with a shell function and call > > `builtin jobs' in the overriding function. > > > > I guessed you have asked this because `jobs' would be specified as a > > special built-in utility [XCU 2.14], which cannot be hidden by a > > shell-function name, but it doesn't seem to be the case actually; > > There's no way `jobs' would ever be a special builtin. It's always been > UPE shaded (User Portability). Job control wasn't even mandatory in > POSIX until Issue 6 (2004), and the specification that jobs can be run > when job control isn't enabled came well after that. Thank you for the information. I find it interesting to hear about the historical background. The statement about the special built-in was just my initial guess for the reason why kre seemed to claim that `builtin jobs' is equivalent to `jobs', but I just checked the standard and immediately knew my guess was incorrect. I still don't think `builtin jobs' is equivalent to `jobs' because `jobs' can pick up a shell function. > > I naively think that it is a valid request that `builtin jobs' would > > behave in the same way as `jobs' in this context. POSIX also states > > that the results are unspecified when the command name matches > > `builtin' according to [XCU 2.9.1 / Command Search and Execution / > > rule 1b]. > > Whether or not `builtin' is specified by POSIX isn't really relevant. That > list just means that some shells have implemented `builtin' as a builtin > command, so portable applications should take care with it. I just meant that even if Bash treats `builtin jobs' the same as `jobs' in these particular subshell contexts, it doesn't conflict with POSIX. So, POSIX seems to be irrelevant when we think about the behavior when `builtin' is involved.
Re: "builtin jobs" does not output to stdout.
2023年2月14日(火) 4:32 Oğuz :
> 13 Şubat 2023 Pazartesi tarihinde Koichi Murase
> yazdı:
>> One of the
>> cases that the fork cost of $() becomes a problem and that other
>> languages cannot be really used is the prompt setting `PS1' containing
>> command substitutions.
>
> Bash has many escape sequences to enrich your prompt strings. If they don't
> meet your needs and you find yourself embedding so many command substitutions
> in them that there is a discernible delay,
That is not my personal use case. In fact, my PS1 is just as simple as
PS1='[\[\]\u@\h\[\] \j \W]\$ '. Honestly, I
do more non-trivial and unusual stuff in my personal configurations,
but I'm personally satisfied with the current feature set of Bash
(with many supplementing shell functions) unless the long-lasting Bash
behaviors would be broken by the literal interpretation of a corner of
the standard that people haven't paid much attention historically.
I just naively think that ${ list; } would help normal users who
wouldn't like to do non-trivial stuff. There are many users who try to
embed multiple command substitutions to acquire git repository
information or other version-control-system information for the
current directory, the information of the current Python/Ruby virtual
environment, etc. The upstream Git itself prepares a utility intended
to be used in the command substitution embedded in PS1 as
/contrib/completion/git-prompt.sh in the git repository of git.
Embedding command substitutions for the VCS information and virtual
environment information in the prompt string is nothing wrong, and two
or three command substitutions are generally not considered "so many
command substitutions".
> perhaps it's time you give other shells a try. I hear fish is good.
For me, there are no other shells that are sufficiently matured for
daily uses and have essentially different languages and libraries that
are fully featured as Java, Python, C, etc.
You might hear of Fish shell is good, but which part of Fish shell is
considered good? They are just good for interactive behaviors. When we
focus on the language design, Fish shell is actually worse than Bash.
They tried to change superficial syntax like the keywords "esac",
"fi", etc., but they made a number of mistakes in the initial language
design, and many serious language defects are left unresolved for
compatibility. For example, it was impossible in Fish shell to pass
the multiline result of command substitutions as an argument to
another command until recently. They gave up and now have two
different command substitution syntaxes, () and $().
Instead, one might consider moving to other shells with languages
similar to POSIX sh or Bash, but in that case, the limitations of `sh'
languages still apply.
>> An example is a shell function: a shell
>> function can change the global variables (or previous-scope variables
>> with Bash's dynamic scoping) and, at the same time, can output data to
>> stdout. When one wants to capture the stdout of such a function while
>> keeping the changes made to the global variables, the command
>> substitution cannot be directly used.
>
>
> It's not a good example because there isn't much difference between `f() {
> list; }; x=$(f)' and `x=$(list)', they both read the output of `list' into
> `x'.
The assumption is the opposite. I'm not talking about converting
existing command substitutions to shell functions, of course. I'm
thinking the case where we first have a shell function that is not
necessarily intended for the command substitution but for general
uses, and then we would like to use its output in a particular place.
> When is it absolutely necessary to both retain the side effects of `list' and
> save its output to a variable?
Of course, there is no case where ${ list; } is "absolutely
necessary". But in that logic, the shell functions are not "absolutely
necessary", and even the command substitutions are not "absolutely
necessary" as everything can be in principle processed in combinations
of pipelines and { list; }, etc. The reason that we have shell
functions and command substitutions is that it makes it easier to
organize shell programs. For the same reason, functions are allowed to
return results with a side effect while it is not "absolutely
necessary" as one can rewrite the entire program in a purely
functional way. Then, when one wants to use the output of such a
function, it is useful to have ${ func; } while it is not "absolutely
necessary" because one can always reimplement the entire program so
that it doesn't have any functions with side effects.
>> Anyway, if there are no use cases or no benefits at all, ksh didn't
>> have the feature.
>
> Ksh has many features no one ever uses; some of them don't make any sense,
> some don't even work. Check out , it's a mess
> compared to bash.
OK, I understand what you try to say, (though I regard "no one ever"
and "any sense" as exaggerations; at least the
special built-in [Was: Re: "builtin jobs" does not output to stdout.]
Date:Tue, 14 Feb 2023 11:04:48 -0500
From:Chet Ramey
Message-ID: <8be447ca-5416-a64f-7e62-1e61102e8...@case.edu>
| "Special builtin" is not a particularly useful concept
I disagree - most of the time the difference doesn't matter much,
but that the special built-ins cause the shell to exit if they fail
means that script writers don't need to do things like
break || exit 99
just in case something is wrong with the script, and the break doesn't
happen in a loop, and so is an error. With it being a special built-in
the script simply exits at that point, otherwise if the script writer
hasn't tested for the error, the script just keeps running to the next
command, which is fairly clear is not what was intended.
That's somewhat esoteric, as things like break/continue/ ... rarely
ever fail.
But for "eval" it matters, consider
eval '| grep foo' || {
# code to handle what happens when foo isn't there
}
That's not likely as written, but could happen from
eval " $command | grep foo" || ...
if "$command" turns out to be unset or null at this point due to some
bug or other. That results in a syntax error from eval, which is a
special built-in failure, which should cause the shell to exit. Allowing
it to continue with just a failure status means the script believes that
the grep failed.
That is, I don't really think that (from an earlier message):
| the only rationale for why those particular ones were chosen
| is "this is what the Bourne shell did."
or perhaps stated better, yes, that is more or less why they were chosen, but
that makes it seem like some of the other (bizarre) things that the Bourne
shell did (probably as a code space saving mechanism), whereas I suspect
that this division was probably more carefully considered from the start.
The special built-in commands are commands that could almost be reserved
words, and often would be, but were implemented as commands instead (and
probably are better that way).
kre
Re: "builtin jobs" does not output to stdout.
On 2/14/23 2:39 PM, Dale R. Worley wrote: Chet Ramey writes: Yes. As described in later messages, there is a special case that allows the output of `jobs' to be piped. This came in at some point between bash-1.05 (March 1990) and bash-1.08 (May 1991). I never extended this special case to `builtin' or `command'. Is this (are these) special cases documented? I just looked through my (admittedly obsolete) bash man page and couldn't find any mention of special behavior of the "jobs" builtin. But presumably careful reading of the documentation of "command", "builtin", and "jobs" would show which cases work in which way. Nothing special. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: "builtin jobs" does not output to stdout.
Chet Ramey writes: > Yes. As described in later messages, there is a special case that allows > the output of `jobs' to be piped. This came in at some point between > bash-1.05 (March 1990) and bash-1.08 (May 1991). I never extended this > special case to `builtin' or `command'. Is this (are these) special cases documented? I just looked through my (admittedly obsolete) bash man page and couldn't find any mention of special behavior of the "jobs" builtin. But presumably careful reading of the documentation of "command", "builtin", and "jobs" would show which cases work in which way. Dale
Re: "builtin jobs" does not output to stdout.
On 2/13/23 6:43 AM, Koichi Murase wrote:
I guess just the support for ksh's ${ list; } [1] would make
everything simple and clear. One can simply call ${ jobs; }, ${ trap
-p; }, etc. without thinking about subshells.
[1] https://lists.gnu.org/archive/html/help-bash/2020-05/msg00077.html
The text about syntatic sugar still applies.
I haven't checked what POSIX says about the jobs in subshells,
POSIX requires that the shell execution environment include what is
essentially the jobs list;
https://www.austingroupbugs.net/view.php?id=1254
discusses this extensively. However,
"The jobs utility does not work as expected when it is operating in its own
utility execution environment because that environment has no applicable
jobs to manipulate."
but at
least Bash maintains a separate "job list" of a subshell (which is
accessible from the built-in command `jobs') regardless of whether the
"job control" is turned on or not.
POSIX requires this:
"The jobs utility is not dependent on the job control option, as are the
seemingly related bg and fg utilities because jobs is useful for examining
background jobs, regardless of the condition of job control."
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: "builtin jobs" does not output to stdout.
On 2/11/23 8:15 PM, Robert Elz wrote: Date:Sun, 12 Feb 2023 01:14:12 +0900 From:Koichi Murase Message-ID: | `jobs' can be overwritten by a shell function. For example, when a | user wants to modify the behavior of `jobs' for interactive uses, a | typical solution is to override `jobs' with a shell function and call | `builtin jobs' in the overriding function. The standard way to do that would be to use "command jobs" - but in either of those cases, it is possible that the way that the shell recognises the special case that allows the command in a sub-shell to output information from the parent shell, rather than the sub-shell's information may not be met. The same applies to the "trap" command which has similar issues. `command jobs' works (I had to remind myself that it does). It's another implementation artifact; it happened separately from the `jobs_hack'. It's because `command' runs the rest of the words through the execution path again, while `builtin' simply looks up and invokes a shell builtin directly. | I guessed you have asked this because `jobs' would be specified as a | special built-in utility [XCU 2.14], which cannot be hidden by a | shell-function name, but it doesn't seem to be the case actually; bash has very little belief in the concept of "special bui8lt-in". "Special builtin" is not a particularly useful concept, but bash embraces it in posix mode. Chet can decide if making bash handle more cases than it currently does is worth it or not for bash and its users. I think handling it in `jobs' and `command jobs' is enough for now. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: "builtin jobs" does not output to stdout.
On 2/11/23 11:14 AM, Koichi Murase wrote: 2023年2月12日(日) 0:42 Robert Elz : Why would you want to ever say "builtin jobs" though? The jobs command has to be buikt in to work. `jobs' can be overwritten by a shell function. For example, when a user wants to modify the behavior of `jobs' for interactive uses, a typical solution is to override `jobs' with a shell function and call `builtin jobs' in the overriding function. I guessed you have asked this because `jobs' would be specified as a special built-in utility [XCU 2.14], which cannot be hidden by a shell-function name, but it doesn't seem to be the case actually; There's no way `jobs' would ever be a special builtin. It's always been UPE shaded (User Portability). Job control wasn't even mandatory in POSIX until Issue 6 (2004), and the specification that jobs can be run when job control isn't enabled came well after that. The distinction between regular and special builtins has always seemed arbitrary -- special builtins are defined in terms of their effects, but the only rationale for why those particular ones were chosen is "this is what the Bourne shell did." Which is fine, but mostly an artifact of that implementation; no discussion of why or whether it's a good idea to carry that forward. `jobs' doesn't seem to be specified to be a special built-in utility. The 1988 Bourne shell didn't have it, so it can't be special. I naively think that it is a valid request that `builtin jobs' would behave in the same way as `jobs' in this context. POSIX also states that the results are unspecified when the command name matches `builtin' according to [XCU 2.9.1 / Command Search and Execution / rule 1b]. Whether or not `builtin' is specified by POSIX isn't really relevant. That list just means that some shells have implemented `builtin' as a builtin command, so portable applications should take care with it. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: "builtin jobs" does not output to stdout.
On 2/11/23 8:54 AM, 岡部将幸 wrote: Bash Version: 5.2 Patch Level: 15 Release Status: release Description: builtin command "jobs" behaves differntly depending on the presence or absence of "builtin". "jobs" outputs to stdout, but "builtin jobs" does not output to stdout. Repeat-By: "sleep 1000 &" "builtin jobs | cat" -> no output "jobs | cat" -> [1]+ Running Yes. As described in later messages, there is a special case that allows the output of `jobs' to be piped. This came in at some point between bash-1.05 (March 1990) and bash-1.08 (May 1991). I never extended this special case to `builtin' or `command'. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: "builtin jobs" does not output to stdout.
13 Şubat 2023 Pazartesi tarihinde Koichi Murase
yazdı:
> One of the
> cases that the fork cost of $() becomes a problem and that other
> languages cannot be really used is the prompt setting `PS1' containing
> command substitutions.
Bash has many escape sequences to enrich your prompt strings. If they don't
meet your needs and you find yourself embedding so many command
substitutions in them that there is a discernible delay, perhaps it's time
you give other shells a try. I hear fish is good.
An example is a shell function: a shell
> function can change the global variables (or previous-scope variables
> with Bash's dynamic scoping) and, at the same time, can output data to
> stdout. When one wants to capture the stdout of such a function while
> keeping the changes made to the global variables, the command
> substitution cannot be directly used.
It's not a good example because there isn't much difference between `f() {
list; }; x=$(f)' and `x=$(list)', they both read the output of `list' into
`x'. When is it absolutely necessary to both retain the side effects of
`list' and save its output to a variable?
Anyway, if there are no use cases or no benefits at all, ksh didn't
> have the feature.
>
Ksh has many features no one ever uses; some of them don't make any sense,
some don't even work. Check out , it's a mess
compared to bash.
--
Oğuz
Re: "builtin jobs" does not output to stdout.
2023年2月14日(火) 2:16 Oğuz :
> 13 Şubat 2023 Pazartesi tarihinde Koichi Murase
> yazdı:
>> Nevertheless, even if we forget about trap, jobs, etc., ${ list; }
>> alone is very useful as it's free from the fork cost
>
> Shells fork all the time. If your application can't afford the overhead,
> you've chosen the wrong language to write it in.
I agree that when we see such a request in this mailing list or in
help-bash, it would probably be the wrong choice of language in most
cases, more than 80% or 90%.
But it doesn't mean that it's always the wrong choice 100%. One of the
cases that the fork cost of $() becomes a problem and that other
languages cannot be really used is the prompt setting `PS1' containing
command substitutions. Even a 30--50ms delay of prompts is actually
noticeable by humans, but the delay can easily reach that threshold by
two or three command substitutions $() in slower systems. How can we
program Bash's prompt in a "more appropriate" language? We cannot
request Bash users to directly edit the C source of Bash and build
Bash binary from the source every time the user wants to change the
configuration slightly.
There is an attempt to replace the shell prompt with a single call of
$(external-cmd) and let the external command generate the entire
prompt, such as Starship written in Rust. I think that is one valid
option that a user can choose, but I don't think we should delete
every feature related to PS1 from Bash and totally switch to an
approach to ask users to write the prompt codes in another compiler
language from scratch.
>> and also can
>> affect the parent shell context when it wants to.
>
> This doesn't sound like an upside to me, I can't think of any use case where
> it wouldn't make the code more confusing.
The fact that one cannot think of the use cases doesn't prove that
there are no use cases at all. An example is a shell function: a shell
function can change the global variables (or previous-scope variables
with Bash's dynamic scoping) and, at the same time, can output data to
stdout. When one wants to capture the stdout of such a function while
keeping the changes made to the global variables, the command
substitution cannot be directly used. This can happen unless one
argues the global variable should not be changed by shell functions at
all. Of course, it is still possible to do the trick « eval
"$(something; declare -p global1 global2 etc)" », but this is
unnecessarily complicated compared to what to achieve.
Anyway, if there are no use cases or no benefits at all, ksh didn't
have the feature.
Re: "builtin jobs" does not output to stdout.
13 Şubat 2023 Pazartesi tarihinde Koichi Murase
yazdı:
>
> Nevertheless, even if we forget about trap, jobs, etc., ${ list; }
> alone is very useful as it's free from the fork cost
Shells fork all the time. If your application can't afford the overhead,
you've chosen the wrong language to write it in.
> and also can
> affect the parent shell context when it wants to.
This doesn't sound like an upside to me, I can't think of any use case
where it wouldn't make the code more confusing.
--
Oğuz
Re: "builtin jobs" does not output to stdout.
2023年2月13日(月) 22:00 Oğuz İsmail Uysal :
> On 2/13/23 2:43 PM, Koichi Murase wrote:
> > I guess just the support for ksh's ${ list; } [1] would make
> > everything simple and clear. One can simply call ${ jobs; }, ${ trap
> > -p; }, etc. without thinking about subshells.
> I don't see what difference that'd make.
Right, it doesn't change anything in the behavior, but I think that is
what Martin tried to think about, i.e., an alternative approach that
does the *same* thing but that can be understood more easily by users
without knowing the special rule of $(trap), $(jobs), etc.
Even though it is specified in the standard, that special rule is not
so natural that the users without knowledge can guess it. Compared to
that, the behaviors of ${ trap; }, ${ jobs; }, etc. would be naturally
understood without knowing the special rule as far as the user knows
what is ${ list; }.
> I think `builtin' should be fixed and the rest left alone.
I actually agree with it personally. It might not be so user-friendly
for general users, but the shells have behaved in this way, and the
rule is already in the standard. I don't think it is usuful to add
additional rules or builtins to handle it in an even different way.
Nevertheless, even if we forget about trap, jobs, etc., ${ list; }
alone is very useful as it's free from the fork cost and also can
affect the parent shell context when it wants to. I think it instead
needs to create a temporary file in the internal implementation, but
the file creation is much faster than the subshell fork usually. An
easy-to-understand idiom ${ jobs; } would be just a bonus of the
general ${ list; }.
--
Koichi
Re: "builtin jobs" does not output to stdout.
On 2/13/23 2:43 PM, Koichi Murase wrote:
I guess just the support for ksh's ${ list; } [1] would make
everything simple and clear. One can simply call ${ jobs; }, ${ trap
-p; }, etc. without thinking about subshells.
I don't see what difference that'd make. A subshell inherits its
parent's job list, and `$(trap ...)' expands to what `trap ...' would
output if it were run in the parent shell; these are well specified in
the standard.
I think `builtin' should be fixed and the rest left alone.
Re: "builtin jobs" does not output to stdout.
2023年2月13日(月) 13:29 Martin D Kealey :
> The contorted rules for reporting the state of the parent shell suggest
> that maybe the entire approach needs re-thinking.
Yeah, maybe we can rethink the approach and find an ideal shell
design. But even when we found it, we would need to continue to
maintain the current behavior along with the new one. With the new
approach, fewer users would (be supposed to) be confused, but the
subtleties with `builtin jobs' etc, would still remain unless we work
on it.
> Several approaches come to mind.
>
> 1. Create a built-in “parent” command where “parent ulimit” or “parent
> trap” answers based on the parent shell rather than the subshell itself.
> (Obviously it would be the snapshot taken when the processes forked, rather
> than the parent's *current* state, unless we want to get really fancy with
> some kind of IPC. I suspect this would not play well with the current
> "longjmp" implementation of subshells.)
>
> 2. Extend all the commands that report on the shell's internal state
> analogously to printf: take a '-v VAR' argument to which they write what
> would normally be their output, so there's no need for a subshell, and
> therefore doubt about which shell they're reporting on.
>
> 3. Provide tracking variables so that it's not necessary to use commands at
> all. e.g. "${BASH_ULIMIT[core]}" would report the maximum size of core dump
> (and "${BASH_ULIMIT[hard-core]}" would report the maximum value that the
> former can be set to), while "${BASH_TRAP[HUP]}" would match the command
> string from trap -p HUP (and [[ -v BASH_TRAP[HUP] ]] would tell you whether
> its unset or just empty).
>
> 4. For "jobs" only, always report the viewpoint of the nearest shell that
> has functioning job control - the one that actually uses setpgrp(). "fg",
> "bg", and any other command used with a job-spec should recognize that
> they're in the "wrong shell" and complain rather than simply being silent.
I guess just the support for ksh's ${ list; } [1] would make
everything simple and clear. One can simply call ${ jobs; }, ${ trap
-p; }, etc. without thinking about subshells.
[1] https://lists.gnu.org/archive/html/help-bash/2020-05/msg00077.html
In the above reply, Chet told us,
> if someone wants to take a shot at implementing it
> before I get to looking at it, I'd be happy to merge it in.
> I wonder how "job control enabled" should be reported in subshells. A
> simple reading of POSIX suggests that $- must contain 'm' in subshells of a
> session shell that has job control enabled, even though they don't actually
> do job control themselves. Maybe there should be an 'M' in $- when job
> control is actually active (in the top level session shell). What do others
> think about this?
Maybe I miss the point, but how can we use the "job control enabled"
state for the present issue? Or is it actually an independent
discussion?
I haven't checked what POSIX says about the jobs in subshells, but at
least Bash maintains a separate "job list" of a subshell (which is
accessible from the built-in command `jobs') regardless of whether the
"job control" is turned on or not. For example,
$ sleep 2023 & sleep 2024 &
[1] 2929313
[2] 2929314
$ jobs
[1]- Running sleep 2023 &
[2]+ Running sleep 2024 &
$ (sleep 2025 & sleep 2026 & jobs)
[1]- Running sleep 2025 &
[2]+ Running sleep 2026 &
$ jobs
[1]- Running sleep 2023 &
[2]+ Running sleep 2024 &
Or non-interactive shells without job control can also have a job list:
$ LANG=C bash -c 'echo $-; sleep 10 & jobs'
hBc
[1]+ Running sleep 10 &
This makes your suggestion 4 (report for the nearest shell with active
job control) non-trivial.
--
Koichi
Re: "builtin jobs" does not output to stdout.
The contorted rules for reporting the state of the parent shell suggest
that maybe the entire approach needs re-thinking.
Several approaches come to mind.
1. Create a built-in “parent” command where “parent ulimit” or “parent
trap” answers based on the parent shell rather than the subshell itself.
(Obviously it would be the snapshot taken when the processes forked, rather
than the parent's *current* state, unless we want to get really fancy with
some kind of IPC. I suspect this would not play well with the current
"longjmp" implementation of subshells.)
2. Extend all the commands that report on the shell's internal state
analogously to printf: take a '-v VAR' argument to which they write what
would normally be their output, so there's no need for a subshell, and
therefore doubt about which shell they're reporting on.
3. Provide tracking variables so that it's not necessary to use commands at
all. e.g. "${BASH_ULIMIT[core]}" would report the maximum size of core dump
(and "${BASH_ULIMIT[hard-core]}" would report the maximum value that the
former can be set to), while "${BASH_TRAP[HUP]}" would match the command
string from trap -p HUP (and [[ -v BASH_TRAP[HUP] ]] would tell you whether
its unset or just empty).
4. For "jobs" only, always report the viewpoint of the nearest shell that
has functioning job control - the one that actually uses setpgrp(). "fg",
"bg", and any other command used with a job-spec should recognize that
they're in the "wrong shell" and complain rather than simply being silent.
I wonder how "job control enabled" should be reported in subshells. A
simple reading of POSIX suggests that $- must contain 'm' in subshells of a
session shell that has job control enabled, even though they don't actually
do job control themselves. Maybe there should be an 'M' in $- when job
control is actually active (in the top level session shell). What do others
think about this?
- Martin
Re: "builtin jobs" does not output to stdout.
2023年2月12日(日) 10:15 Robert Elz : Thank you for the detailed explanations. I totally agree with you on all the points. > Chet can decide if making bash handle more cases than it currently does is > worth it or not for bash and its users. Yes, Chet will decide. -- Koichi
Re: "builtin jobs" does not output to stdout.
Date:Sun, 12 Feb 2023 01:14:12 +0900
From:Koichi Murase
Message-ID:
| `jobs' can be overwritten by a shell function. For example, when a
| user wants to modify the behavior of `jobs' for interactive uses, a
| typical solution is to override `jobs' with a shell function and call
| `builtin jobs' in the overriding function.
The standard way to do that would be to use "command jobs" - but in either
of those cases, it is possible that the way that the shell recognises the
special case that allows the command in a sub-shell to output information
from the parent shell, rather than the sub-shell's information may not be
met. The same applies to the "trap" command which has similar issues.
| I guessed you have asked this because `jobs' would be specified as a
| special built-in utility [XCU 2.14], which cannot be hidden by a
| shell-function name, but it doesn't seem to be the case actually;
bash has very little belief in the concept of "special bui8lt-in".
| POSIX also states that the results are unspecified when the command name
| matches `builtin' according to [XCU 2.9.1 / Command Search and Execution /
| rule 1b].
Yes, that's the same issue as with select [[ and more ... POSIX knows that
there are shells (like bash) which treat those differently than just being
normal commands, and so warns applications not to expect them to work as
normal commands.
There are no precise rules for when a shell should switch from parent shell
data to sub-shell data for commands like "trap" and "jobs" - the actual
expectation is very limited indeed, but shells can widen that, and allow
more cases through. Eg: if one does
j=jobs
$j | wc -l
then it is not required that shells give special treatment to the jobs
command that is run, it might report only the sub-shell's jobs in that
case (ie: here, none). That is, the shell can look at the text of the
command string when creating the sub-shell to decide what do do. But
it is allowed to do more - the NetBSD sh here doesn't make decisions about
things like this until the command is about to be run, after all expansions,
even after function calls, so for us
mjjobs() { jobs "$@" ; }
myjobs | wc -l
will still count the number of lines in the jobs output from the parent
shell, as "jobs" is the first simple command run by the sub-shell, and that
is what counts (for us). Other shells are likely to be more restricted.
Chet can decide if making bash handle more cases than it currently does is
worth it or not for bash and its users.
kre
Re: "builtin jobs" does not output to stdout.
2023年2月12日(日) 0:42 Robert Elz : > Why would you want to ever say "builtin jobs" though? > The jobs command has to be buikt in to work. `jobs' can be overwritten by a shell function. For example, when a user wants to modify the behavior of `jobs' for interactive uses, a typical solution is to override `jobs' with a shell function and call `builtin jobs' in the overriding function. I guessed you have asked this because `jobs' would be specified as a special built-in utility [XCU 2.14], which cannot be hidden by a shell-function name, but it doesn't seem to be the case actually; `jobs' doesn't seem to be specified to be a special built-in utility. Anyway, `jobs' can be hidden by a shell function at least in Bash. I naively think that it is a valid request that `builtin jobs' would behave in the same way as `jobs' in this context. POSIX also states that the results are unspecified when the command name matches `builtin' according to [XCU 2.9.1 / Command Search and Execution / rule 1b]. -- Koichi
Re: "builtin jobs" does not output to stdout.
root@localhost:~# ( sleep .1 & jobs ) | wc -c 41 On Sat, Feb 11, 2023, 3:57 PM 岡部将幸 wrote: > Configuration Information [Automatically generated, do not change]: > Machine: x86_64 > OS: linux-gnu > Compiler: gcc > Compilation CFLAGS: -g -O2 > uname output: Linux gold 5.8.0-63-generic #71-Ubuntu SMP Tue Jul 13 > 15:59:12 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux > Machine Type: x86_64-pc-linux-gnu > > Bash Version: 5.2 > Patch Level: 15 > Release Status: release > > Description: > builtin command "jobs" behaves differntly depending on > the presence or absence of "builtin". > "jobs" outputs to stdout, but "builtin jobs" does not > output to stdout. > > Repeat-By: > "sleep 1000 &" > "builtin jobs | cat" -> no output > "jobs | cat" -> [1]+ Running > > Fix: > [Description of how to fix the problem. If you don't know a > fix for the problem, don't include this section.] >
Re: "builtin jobs" does not output to stdout.
2023年2月11日(土) 23:57 岡部将幸 :
> Description:
> builtin command "jobs" behaves differntly depending on
> the presence or absence of "builtin".
> "jobs" outputs to stdout, but "builtin jobs" does not
> output to stdout.
This seems to be reproduced in all the Bash versions from 1.14.7 to
devel. The problem is related to the condition of whether subshell
builtins inherit the job information. In particular, this is related
to the following test in the source code [1]:
[1] https://git.savannah.gnu.org/cgit/bash.git/tree/execute_cmd.c?h=devel#n5236
$ sleep 1000 &
$ : | jobs
[1]+ Running sleep 1000 &
$ : | command jobs
[1]+ Running sleep 1000 &
$ echo "$(jobs)"
[1]+ Running sleep 1000 &
In these cases, the `jobs' builtin inherits the job information of the
main shell.
$ : | { jobs; }
$ (jobs)
In these cases, the `jobs' builtin shouldn't print the job information
of the main shell. So, the `jobs' builtin in subshells actually
inherits the job information only in a limited case, which is
implemented in [1]. Now we call the `jobs' builtin through the
`builtin' builtin in the reported code, so it bypasses the condition
tested in [1]. We can test many other combinations of such indirect
builtin calls, where it doesn't inherit the job information if at
least one `builtin' is contained in the chain:
$ : | command jobs
[1]+ Running sleep 1000 &
$ : | command command jobs
[1]+ Running sleep 1000 &
$ : | command command command jobs
[1]+ Running sleep 1000 &
$ : | builtin jobs
$ : | command builtin jobs
$ : | builtin builtin jobs
$ : | builtin command jobs
$ : | builtin builtin builtin jobs
--
Koichi
Re: "builtin jobs" does not output to stdout.
Date:Sat, 11 Feb 2023 22:54:47 +0900 From:=?UTF-8?B?5bKh6YOo5bCG5bm4?= Message-ID: | "jobs" outputs to stdout, but "builtin jobs" does not | output to stdout. Your test is flawed. Processes in a pipeline are in a subshell environment. That environment has no jobs, hence no output. But there is a special exemption for that, when 'jobs' is the first command in the subshell. Then it reports its parent's jobs table instead (this is to allow j=$(jobs) to work mainly). So jobs|whatever produces output, but builtin jobs|... does not. To test whether builtin jobs writes to stdout, use builtin jobs >/tmp/foo instead. No subshell there, so it should work. Why would you want to ever say "builtin jobs" though? The jobs command has to be buikt in to work. kre
