Bug#1023429: pgrep/pkill: remove trailing 0x00 from matching?
Hey. Just for the records: At least as of now, ssh no longer seems to add the multiple 0x0 to it's cmdline. But I guess the "problem" (should it ever come back or exist for other programs) in pgrep/pkill still remains. Cheers, Chris.
Bug#1023429: pgrep/pkill: remove trailing 0x00 from matching?
Hey Craig.
On Tue, 2022-11-15 at 08:01 +1100, Craig Small wrote:
> >
> It can, but what we think the string is is not what the string
> actually is, I suspect.
> Each one of those 0x0 are delimiters, so if there was two of them at
> the end we would have:
> argv[0] DELIM argv[1] DELIM argv[2]
> Where DELIM is the " " delimiter ps uses and argv[2]=""
AFAIU the cmdline ist defined to be the fields, separated with 0x0 with
the last one terminated by 0x0 (so in other words, all are NULL
terminated strings).
For the matching, we need to make the fields one string, right? Because
BRE/ERE are not really defined for multiline/field matches and I guess
even if one makes something up, like one RE per field, it wouldn't be
so much useful in practise.
How does one reasonably make the multiple fields one true line without
any 0x0?
a) We concatenate simply, separated by space (which I guess is what's
done now and causes the issue?).
And this is already ambiguous if on wants perfect matching, cause:
Was "foo " actually:
foo "" ""
or was it rather:
foo " "
?
b) we try to escape/quote the fields, so if e.g. the 2nd one would be
we'd concatenate: '' (i.e. shell style quoting)
But that's obviously more complex, since the filed may also contain
quote characters.
c) Since (a) already looses the information how the fields actually
looked like, we could also just say, that any whitespace in fields
is effectively ignored, which means again, that one cannot
specifically match a e.g. 2nd field
But therefore, we could apply the following rule:
Only if a field in cmdline is non-empty, we actually append it
and a spearating whitespace to the string that we match against.
With the question left open, whether we strip any surrounding
whitespace from a field, i.e. does
foo " bar baz "
result in
"foo bar baz "
^
\- being the field spearatinspace
or in
"foo bar baz"
> The arglist is doesn't end with "[mux]" but "[mux] ". While
> looking odd, this is what the argument list actually is.
Well I think it's difficult to say what it "actually" is, since we
cannot really express it in ONE string without a separator characters
like 0x0 that is otherwise not allowed or without loosing information.
Right?
> Another strange thing, the proc(5) manpage says:
> /proc/[pid]/cmdline
> This read-only file holds the complete command line for
> the process, unless the process is a zombie. In the latter case,
> there is nothing in this file: that is, a read on this file will
> return 0 characters. The command-line arguments appear in this file
> as a
> set of strings separated by null bytes ('\0'), with a
> further null byte after the last string.
>
> But neither your or my example has that, its a space. That's the
> kernel doing something odd.
That I don't understand?
The (field-)separating (single) that you presumably add as a
convenience does not "really" exist in the command line. It's just
something that *might* have been used in the shell, that caused the
exec for the command, but the kernel never sees those.
The separator could have very well been always two or a .
Cheers,
Chris.
Bug#1023429: pgrep/pkill: remove trailing 0x00 from matching?
On Tue, 15 Nov 2022 at 00:36, Christoph Anton Mitterer <
cales...@scientia.org> wrote:
> $ hd /proc/19557/cmdline
> 73 73 68 3a 20 2f 68 6f 6d 65 2f 63 61 6c 65 73 |ssh:
> /home/cales|
> 0010 74 79 6f 2f 2e 73 73 68 2f 6d 75 78 2f 72 6f 6f
> |tyo/.ssh/mux/roo|
> 0020 74 40 6c 63 67 2d 6c 72 7a 2d 61 64 6d 69 6e 2e
> |t@lcg-lrz-admin.|
> 0030 67 72 69 64 2e 6c 72 7a 2e 64 65 3a 32 32 20 5b |
> grid.lrz.de:22 [|
> 0040 6d 75 78 5d 00|mux].|
> 0045
>
> That there's only exactly one trailing 0x0.
>
Ah, that might be the hint I needed.
> > That's how the argument list is stored. libproc parses it out into a
> > set of strings or one long string.
>
> Okay... but shouldn't it still be able to match?
>
It can, but what we think the string is is not what the string actually is,
I suspect.
Each one of those 0x0 are delimiters, so if there was two of them at the
end we would have:
argv[0] DELIM argv[1] DELIM argv[2]
Where DELIM is the " " delimiter ps uses and argv[2]=""
So when the cmdline is:
$ cat /proc/34724/cmdline | hd
73 73 68 3a 20 2f 68 6f 6d 65 2f 63 61 6c 65 73 |ssh:
/home/cales|
0010 74 79 6f 2f 2e 73 73 68 2f 6d 75 78 2f 72 6f 6f
|tyo/.ssh/mux/roo|
0020 74 40 6c 63 67 2d 6c 72 7a 2d 61 64 6d 69 6e 2e
|t@lcg-lrz-admin.|
0030 67 72 69 64 2e 6c 72 7a 2e 64 65 3a 32 32 20 5b |grid.lrz.de:22
[|
0040 6d 75 78 5d 00 00 00 00 00 00|mux]..|
004a
The arglist is doesn't end with "[mux]" but "[mux] ". While looking
odd, this is what the argument list actually is.
There is a difference between
If it does that strange not-matching again, try ending with " *" or
[[:space:]]*"
I found postgres does this too.
$ hd /proc/1291/cmdline
70 6f 73 74 67 72 65 73 3a 20 31 33 2f 6d 61 69 |postgres:
13/mai|
0010 6e 3a 20 44 65 62 69 61 6e 2d 6d 69 6e 65 74 65 |n:
Debian-minete|
0020 73 74 20 6d 69 6e 65 74 65 73 74 5f 77 6f 72 6c |st
minetest_worl|
0030 64 31 20 5b 6c 6f 63 61 6c 5d 20 69 64 6c 65 00 |d1 [local]
idle.|
0040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
||
*
0070 00 00 00 00 00 00 00 00 00 00|..|
007a
$ pgrep --full 'idle$'
$ pgrep --full 'idle[[:space:]]+$'
1075
1088
1089
1291
1293
1298
1300
1715
1720
2349
Another strange thing, the proc(5) manpage says:
/proc/[pid]/cmdline
This read-only file holds the complete command line for the
process, unless the process is a zombie. In the latter case, there is
nothing in this file: that is, a read on this file will return 0
characters. The command-line arguments appear in this file as a
set of strings separated by null bytes ('\0'), with a further
null byte after the last string.
But neither your or my example has that, its a space. That's the kernel
doing something odd.
- Craig
- Craig
Bug#1023429: pgrep/pkill: remove trailing 0x00 from matching?
Hey Craig
On Mon, 2022-11-14 at 19:25 +1100, Craig Small wrote:
> I'm not sure why it is not matching, my test script above works fine.
> That was with 3.3.17-5
I also just tried again, with 2:3.3.17-7.1, and oddly enough it worked
again.
$ ps ax | grep mux
19557 ?Ss 0:00 ssh:
/home/calestyo/.ssh/mux/r...@lcg-lrz-admin.grid.lrz.de:22 [mux]
$ pgrep --full --exact --euid "${LOGNAME}" --list-full -- "^ssh:
${HOME}/\.ssh/mux/.+ \[mux]$"
19557 ssh: /home/calestyo/.ssh/mux/r...@lcg-lrz-admin.grid.lrz.de:22 [mux]
The difference seems to be: ...
$ hd /proc/19557/cmdline
73 73 68 3a 20 2f 68 6f 6d 65 2f 63 61 6c 65 73 |ssh: /home/cales|
0010 74 79 6f 2f 2e 73 73 68 2f 6d 75 78 2f 72 6f 6f |tyo/.ssh/mux/roo|
0020 74 40 6c 63 67 2d 6c 72 7a 2d 61 64 6d 69 6e 2e |t@lcg-lrz-admin.|
0030 67 72 69 64 2e 6c 72 7a 2e 64 65 3a 32 32 20 5b |grid.lrz.de:22 [|
0040 6d 75 78 5d 00|mux].|
0045
That there's only exactly one trailing 0x0.
> That's how the argument list is stored. libproc parses it out into a
> set of strings or one long string.
Okay... but shouldn't it still be able to match?
Thanks,
Chris.
Bug#1023429: pgrep/pkill: remove trailing 0x00 from matching?
On Fri, 4 Nov 2022 at 12:09, Christoph Anton Mitterer
wrote:
> pgrep --full --exact --euid "${LOGNAME}" --list-full -- "^ssh:
> ${HOME}/\.ssh/mux/.+ \[mux]$"
>
$ ./blah [4565] &
[1] 769
$ pgrep --full --exact --list-full '^/bin/sh ./blah \[4565]$'
769 /bin/sh ./blah [4565]
$ cat blah
#!/bin/sh
sleep 100
I'm was also pretty sure that this script used to work (but no longer does
> now),
> but I've tested on some ancient Debian, and even there it fails.
>
I'm not sure why it is not matching, my test script above works fine. That
was with 3.3.17-5
The reason seems to be, that there are trailing 0x00.
>
That's how the argument list is stored. libproc parses it out into a set of
strings or one long string.
You can't really directly use grep because the library is parsing it, while
grep uses the raw input.
- Craig
Bug#1023429: pgrep/pkill: remove trailing 0x00 from matching?
Package: procps
Version: 2:3.3.17-7.1
Severity: wishlist
Hey.
I have a script that matches on any ssh channel multiplexing process
(and then kills it) after printing them.
It basically does:
pgrep --full --exact --euid "${LOGNAME}" --list-full -- "^ssh:
${HOME}/\.ssh/mux/.+ \[mux]$"
printf '\nPress return to kill these processes.\n'
read tmp
pkill --full --exact --euid "${LOGNAME}" -- "^ssh: ${HOME}/\.ssh/mux/.+
\[mux]$"
Where ".ssh/mux/" is simply my configured path.
I'm was also pretty sure that this script used to work (but no longer does now),
but I've tested on some ancient Debian, and even there it fails.
The reason seems to be, that there are trailing 0x00.
E.g. I have:
$ cat /proc/34724/cmdline | hd
73 73 68 3a 20 2f 68 6f 6d 65 2f 63 61 6c 65 73 |ssh: /home/cales|
0010 74 79 6f 2f 2e 73 73 68 2f 6d 75 78 2f 72 6f 6f |tyo/.ssh/mux/roo|
0020 74 40 6c 63 67 2d 6c 72 7a 2d 61 64 6d 69 6e 2e |t@lcg-lrz-admin.|
0030 67 72 69 64 2e 6c 72 7a 2e 64 65 3a 32 32 20 5b |grid.lrz.de:22 [|
0040 6d 75 78 5d 00 00 00 00 00 00|mux]..|
004a
$ pgrep --full --exact --euid "${LOGNAME}" --list-full -- "^ssh:
${HOME}/\.ssh/mux/.+ \[mux]$"
$
$ pgrep --full --exact --euid "${LOGNAME}" --list-full -- "^ssh:
${HOME}/\.ssh/mux/.+ \[mux].+$"
34724 ssh: /home/calestyo/.ssh/mux/r...@lcg-lrz-admin.grid.lrz.de:22 [mux]
$
works again, but is of course inferior, as it would posibly match undesired
processes.
In POSIX EREs it should never be possible to match 0x00.
Wouldn't it therefore make sense to e.g. strip any trailing 0x00 from cmdline
and e.g. print a warning if any non-trailing ones are encountered?
Or maybe add some commant line switches, one that allows to ignore trailing
0x00 one, that allows to ignore non-trailing ones.
Thanks,
Chris.
-- System Information:
Debian Release: bookworm/sid
APT prefers unstable-debug
APT policy: (500, 'unstable-debug'), (500, 'unstable')
merged-usr: no
Architecture: amd64 (x86_64)
Kernel: Linux 6.0.0-2-amd64 (SMP w/16 CPU threads; PREEMPT)
Kernel taint flags: TAINT_WARN
Locale: LANG=en_DE.UTF-8, LC_CTYPE=en_DE.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
Versions of packages procps depends on:
ii init-system-helpers 1.64
ii libc62.36-4
ii libncurses6 6.3+20220423-2
ii libncursesw6 6.3+20220423-2
ii libprocps8 2:3.3.17-7.1
ii libtinfo66.3+20220423-2
Versions of packages procps recommends:
ii psmisc 23.5-3
procps suggests no packages.
-- Configuration Files:
/etc/sysctl.conf changed [not included]
-- no debconf information
