2026年6月10日(水) 8:54 Robert Elz <[email protected]>:
> But to me it looks as if the problems are caused by attempts to use
> the (csh derived ultimately) totally stupid \ expansions in prompts
A backslash in prompts is not stupid. It is a part of the protocol
defined by the ANSI standard (i.e., ANSI X3.64, ECMA-48, ISO/IEC 6429)
and ISO2022 (ECMA-35) encodings, and you can't claim those standards
are stupid.
> variable expansions for all the rest (creating new variables for the
> expansions which don't currently have one) - expanded at assignment
> time (inside "")
That doesn't work because a backslash at the end of the value of the
variable will escape the next character unexpectedly.
bs='\'
PS0="${bs}w "
Then you'll get «<current working directory> » instead of «\w ».
> or at executing time (inside '' or $'') and then just do the whole
> thing using var expansions (and command substitutions, and
> occasionally arithmetic) in the prompt definition
That doesn't work with «shopt -u promptvars». If it were the user's
personal setting, the user should just turn on `promptvars', but we
are now talking about interactive shell settings injected by the
terminal, which need to work with various user configurations.
> (and the promptvars option should just go away and always
> act as it does now when set).
OK, so, do you suggest removing «shopt -u promptvars»? Yeah, if that
will be accepted, that is ideal.
> There will always be issues when there are 2 different expansion
> schemes happening at the same time (more or less)
I totally agree, but we can't redesign this for backward
compatibility. Adding even another option to unify those expansions to
a single-pass conversion is even worse.
> when one sometimes wants one before the other, and other times the
> other order, it is all just a giant mess.
We consistently want post-promptvars removal. I don't think there are
cases where people want pre-promptvars removal.
> | Yes, we agree that they should simply vanish. We are talking about the
> | details of when they should vanish. Actually, the current behavior of
> | the devel branch fails to properly remove \x01 and \x02 embedded in a
> | command substitution, such as `$(prompt-framework git-status)'.
>
> Why would bash be removing those chars?
Because they are internally used for the markers of \[ and \]. I don't
know the reason that the markers are not embedded in a transparent
way, such that raw \1 and \2 are preserved by an escaping mechanism.
It's just for historical reasons, and we can't switch the behavior
now.
> What if I want to send \1 or \2 to the terminal for some reason?
There is no way. However, PS1 worked this way for a long time, and it
hasn't caused an issue because most terminals just ignore \1 and \2.
When \1 or \2 is received, some terminals might output an error
message of "unrecognized control character," or some terminals may
insert `?' or `^A'/`^B', or some representations for a control
character, but that's rare. In the first place, \1 and \2 are only
defined for the transmission layer by the standard, and their role is
not defined for upper layers.
> Ideally it should be possible to config (both bash & readline) to
> set what magic chars in the string are the "no position changes from
> here" and "back to normal output" values, so the user/script can set
> what ought to be used, rather than building anything in.
That shouldn't happen. A configuration for the magic adds significant
complications and solves nothing. First of all, in the interactive
shell settings, many independent settings are combined. If each
setting wants to specify a distinct magic character to Bash/Readline,
what would happen? Framework A sets \1\2 to Bash/Readline as the magic
pair, and then framework B sets \3\4 to Bash/Readline, which breaks
framework A. Even if framework B doesn't have a plan to send \3\4 to
the terminal, that doesn't mean that all frameworks don't try to send
\3\4 to the terminal, so framework B cannot set \3\4 to Bash/Readline
safely. Also, it breaks existing Bash settings that assume \1\2. Then,
it's much simpler to freeze the markers to be \1\2 rather than care
about the arbitrary possibilities of the marker strings.
> | To remove \x01 and \x02 properly, they should be removed at the very last
> | step, as we are requesting in the present thread.
>
> Those shouldn't be removed at all, I see nothing in the bash doc that
> says they are special.
However, that's how Readline worked for PS1 from at least bash-1.14.7:
bash-1.14.7$ grep -rn '#define RL_PROMPT_'
lib/readline/readline.h:282:#define RL_PROMPT_START_IGNORE '\001'
lib/readline/readline.h:283:#define RL_PROMPT_END_IGNORE '\002'
though I agree that they should actually be documented in the manual.
Currently, \1 and \2 are only mentioned for the Readline variables
«*-mode-string», but not mentioned for PS1 and PS2.
> How would you feel if bash simply decided to remove all \ chars, or
> spaces, or some other random character, just because someone decided
> that if that was done, some trick that they had invented would work?
> Totally absurd.
Of course, that's absurd, because that is an example intentionally
modified to be absurd. The situation is totally different. The
behavior in PS1 is consistent since the very early implementation of
Bash, and it's not a behavior that suddenly appeared. The characters
\1 and \2, which are effectively no-op in terminals, are different
from graphical characters and control characters associated with
functions in terminals.
> | We are talking about the detailed behavior of `what it should be
> | doing'. Both removing \[\] from the beginning and removing \x01\0x02
> | at the final step satisfy what you seem to request. You haven't
> | explained why they should be removed before the `promptvars'
> | processing instead of after `promptvars'.
>
> The issue is that, as I understand things, they aren't being removed at
> all - they simply are no longer being inserted where they once were, as
> they are not needed in PS0 or PS4 (in them, nothing there cares what chars
> in the string use char positions on the terminal, and which don't).
I don't think whether we call it "remove" or "not insert" is relevant.
They are equivalent. "When the `expand_prompt' or `promptvars'
processor sees \[\] or \1\2 in the input, one doesn't insert anything
in its output." This is what I mean by `remove'. I'm sorry if I
confused you, but if you don't like the terminology `remove', could
you please replace `remove' with `not insert' when reading the above
arguments?
> Any \[ \] in the string are simply deleted - and technically,
> perhaps even that shouldn't happen, as PS0 and PS4 don't need such
> things.
So, you mean deleting \[\] shouldn't happen in PS0 and PS4? That is
the behavior before commit 1e9f5e10 in the devel branch.
> | You seem to support
> | pre-`promptvars' removal in the current devel branch, but it actually
> | only removes the embedded markers halfway. To remove those markers,
> | including the ones emitted by command substitutions of a prompt
> | framework, one needs to remove them after `promptvars' instead of
> | pre-`promptvars' removal.
>
> As above, when the \ processing is done wrt when var expansions are done
> can never be done in a way which satisfies all uses, do it one way and
> some users won't be able to do what they want, do it the other way, and
> other users won't be able to do what they want, try to do things multiple
> times and no-one ends up happy, as nothing ever really works in a precdictable
> way any more.
>
> "I want this var/cmd-subst expanded before \ expansions, as
> I want to include \ expansions in its output, but this other var/.. expanded
> after \ expansions as it is to contain \ chars that aren't to be expanded".
> Nightmare.
>
> As I understand it (and assuming that the absurd \ things are to remain in
> prompt processing) bash has long documented the order in which those happen.
> And that is, \ expansions first, var (etc) expansions second (if enabled).
>
> If you cannot do what you want within those restrictions, then you need to
> find a completely different way, as changing it will break someone else's
> code.
No one is talking about exchanging the ordering of the \ processing
and the promptvars expansion. Thus, the above four paragraphs are
totally irrelevant. We are talking about the timing of removing the
markers. Let me clarify this for you: Before commit 1e9f5e10 (and in
Bash 5.3):
PS1 underwent (1) the \ processing, (2) the promptvars processing,
(3) (layout calculations if necessary and) deletion of \1\2.
PS0 underwent (1) the \ processing, (2) the promptvars processing.
We requested to add to PS0 a transform that is equivalent to (3),
i.e., expecting something like this:
PS1: (1) (2) (3),
PS0: (1) (2) (3).
However, after commit 1e9f5e10, the behavior became effectively
PS1: (1) (2) (3),
PS0: (1) (3) (2),
though, in the actual implementation, (1)+(3) is effectively combined
to directly produce the (1)+(3) result). In this thread, we are
requesting to correct (1) (3) (2) to (1) (2) (3) to match PS0's
behavior to PS1's behavior because that creates a difference between
PS1 and PS0 for how promptvars is applied to an identical setting.
> | > a simple \ already does that.
> | Unfortunately, no, when `shopt -s promptvars' is involved in Bash.
>
> I have promptvars set (since it is the default) and it works fine for me.
So you were just lucky. The issue doesn't always happen. It happens in
a specific situation. You've told that a simple \ can be used as the
generic quoting character, but when you want to quote a backslash
(which should appear literally in the data sent to the terminal), the
simple \\ has an issue. For example, consider a case where one wants
to output a literal string `[...]\$x '. Note that in a realistic
situation, the «[...]\» part would be actually something like
$'\e]<payload>\e\\'. How would you quote `\' and `$'? In your theory,
simple \ would do it:
PS1='[...]\\\$x '
However, this doesn't work (with the default setting of `shopt -s
promptvars'). You can confirm that yourself in your instance of Bash.
The correct answer is this:
PS1='[...]\\\\\\$x '
The above becomes «[...]\\\$x » after (1) the \ processing. Then, it
becomes «[...]\$x » after (2) the promptvars processing.
> If you mean it might not work the same without promptvars set
Yes, in addition to the complication I showed above, it is the
original issue that the proper number of backslashes depends on the
user's choice of `shopt -s/u promptvars'.
> (or in ancient bash when there was no such thing) then I simply
> don't care.
We are forced to care about such a situation because we provide
settings to the users, where some users might turn off promptvars.
Anyway, whether you personally care or not is irrelevant.
> ps: from what I gather, part of the issue is the desire to support ancient
> versions of bash, where promptvars didn't exist, and the \ expansions were
> all that were available.
That's the opposite. We are talking about the behavior of PS0, which
was introduced in Bash 4.4, where `shopt -u promptvars' were already
present.
--
Koichi
