Re: bash sockets: printf \x0a does TCP fragmentation
dirk+b...@testssl.sh wrote: > we discovered a strange phenomenon in the project testssl.sh: You are doing something that is quite unusual. You are using a shell script direction on a TCP socket. That isn't very common. More typically one would use a C program instead. So it isn't surprising that you are finding interactions that are not well known. > printf -- "$data" >&5 2>/dev/null Why is stderr discarded? That is almost always bad because it discards any errors that might occur. You probably shouldn't do this. What happens if $data contains % format strings? What happens if the format contains a sequence such as \c? This looks problematic. This is not a safe programming proctice. > does not do what it is intended. "Intent" is in the eye of the beholder. > "$data" is a ClientHello like > > '\x16\x03\x01\x2\x00\x01\x00\x1\xfc\x03\x03\x54\x51\x1e\x7a\xde\xad\xbe\xef\x31\x33\x07\x00\x00\x00\x00\x00\xcf\xbd\x39\x04\xcc\x16\x0a\...' > > Each \x0a like the last one causes a new TCP fragment to begin which can be > easily > spotted when using wireshark while running e.g. As Chet said the libc stdio library is probably doing line oriented buffering. The newline is causing a flush at that time. > One would assume that a bash socket connection cannot influence the TCP > fragmentation but obviously it does. One would be in error to assume this. > If there's a workaround, please let me know. (tried to add "%b" with no > effect). Otherwise I believe it's a bug. You can re-block the output stream using other tools such as 'cat' or 'dd'. Since you are concerned about block size then perhaps dd is the better of the two. | cat Or probably better: | dd status=none bs=1M Or use whatever block size you wish. The 'dd' program will read the input into its buffer and then output that block of data all in one write(2). That seems to be what you are wanting. Good luck! :-) Bob P.S. You can possibly use the 'stdbuf' command to control the output buffering depending upon the program. info stdbuf
Re: bash sockets: printf \x0a does TCP fragmentation
On 9/21/18 4:13 PM, dirk+b...@testssl.sh wrote: > > Hello there, > > we discovered a strange phenomenon in the project testssl.sh: > > After opening a TCP socket with a fd (here: 5), when writing to it, > it seems that > > printf -- "$data" >&5 2>/dev/null > > does not do what it is intended. "$data" is a ClientHello like > > '\x16\x03\x01\x2\x00\x01\x00\x1\xfc\x03\x03\x54\x51\x1e\x7a\xde\xad\xbe\xef\x31\x33\x07\x00\x00\x00\x00\x00\xcf\xbd\x39\x04\xcc\x16\x0a\...' > > Each \x0a like the last one causes a new TCP fragment to begin which can be > easily > spotted when using wireshark while running e.g. Newline? It's probably that stdout is line-buffered and the newline causes a flush, which results in a write(2). > If there's a workaround, please let me know. (tried to add "%b" with no > effect). Otherwise I believe it's a bug. How? Does the emitted output not correspond to what's passed to printf in some way? -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
bash sockets: printf \x0a does TCP fragmentation
Hello there, we discovered a strange phenomenon in the project testssl.sh: After opening a TCP socket with a fd (here: 5), when writing to it, it seems that printf -- "$data" >&5 2>/dev/null does not do what it is intended. "$data" is a ClientHello like '\x16\x03\x01\x2\x00\x01\x00\x1\xfc\x03\x03\x54\x51\x1e\x7a\xde\xad\xbe\xef\x31\x33\x07\x00\x00\x00\x00\x00\xcf\xbd\x39\x04\xcc\x16\x0a\...' Each \x0a like the last one causes a new TCP fragment to begin which can be easily spotted when using wireshark while running e.g. testssl.sh --assume-http -p testssl.sh Starting from the SSLv3 ClientHello the first reassembled packet ends with 0a. See also discussion @ https://github.com/drwetter/testssl.sh/pull/1113. One would assume that a bash socket connection cannot influence the TCP fragmentation but obviously it does. This behavior has a performance penalty and other strange effects, e.g. if the first segment is really small, some devices reject the ClientHello. If there's a workaround, please let me know. (tried to add "%b" with no effect). Otherwise I believe it's a bug. Cheers, Dirk PS: Would ulimit -b help?
Re: Which commit for a bug in 4.3.48 which is fixed in 4.4.23
On Fri, Sep 21, 2018 at 01:11:38PM +0200, Dr. Werner Fink wrote:
> Hi,
>
> with 4.3.48 the line
>
> T="";echo ">${T//*/ }<"
>
> leads to
>
> ><
>
> but with 4.4.23 the correct result is given back
>
> > <
>
> in the git repro I do not find any useful login entry for this
Check commit 34ec1876071b76d3654a418682e3f34ca9a72f1a:
+lib/glob/gmisc.c
+ - match_pattern_char, match_pattern_wchar: if passed an empty string,
+ return a match if the first character of the pattern is `*'
+
+subst.c
+ - pat_subst: change to allow empty strings to be replaced as long as
+ pattern matches empty string. Report and fix from isabella parakiss
+
In particular, this change to match_pattern:
@@ -4435,7 +4437,7 @@ match_pattern (string, pat, mtype, sp, ep)
size_t slen, plen, mslen, mplen;
#endif
- if (string == 0 || *string == 0 || pat == 0 || *pat == 0)
+ if (string == 0 || pat == 0 || *pat == 0)
return (0);
Re: Special parameter ?
> The thing that creates the background job IS a foreground pipeline. It's a > foreground pipeline that creates a background > job, if that makes sense. Sure. That's the mechanism to implement the background job. The problem is that the user that reads the manual is dealing with the "background" and "foreground" abstractions; for him, the most recently foreground command in my example is false. > I recommend to store the value of $? in a variable if you plan on using it > afterwards, since it's extremely easy to end > up overriding its value. Thanks, but I am not trying to implement anything. I am trying to fix the man page :) > You're right, but it might be less confusing if I removed the "foreground" > from the description. This could help. -- Enrique Soriano Escuela Superior de Ingeniería de Telecomunicación, Universidad Rey Juan Carlos. Tel: 914888762 http://gsyc.urjc.es/~esoriano
Re: suspend and ||
On 9/21/18 7:49 AM, esori...@gsyc.urjc.es wrote: > According to the manual: > > (I) Typing the suspend character (typically ^Z, Control-Z) > while a process is running causes that process to be > stopped and returns control to bash. > > (II) An OR list has the form > > command1 || command2 > > command2 is executed if and only if command1 > returns a non-zero exit status. The return status of > AND and OR lists is the exit status of the last > command executed in the list. > > In the following example: > > esoriano@omac:~$ sleep 10 || echo hey > ^Z > [5]+ Stopped sleep 10 > hey > esoriano@omac:~$ > > there are two different issues: > > (I) echo is executed when sleep is suspended, i.e., command2 is >executed before command1 exits. This is true, and it's the difference between a process, which is the object you suspend, and a shell command. When you suspend the sleep, it returns a status: 128+SIGTSTP. Since this is non-zero, the second part of the AND-OR list gets executed. The idiomatic way to do what you want is to get the command you want to run as a unit into a construct like (...) -- if you want to run it in the foreground -- so you have a single process that you can suspend. This has come up many times, in the context of compound commands like loops and command sequences. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Special parameter ?
On 9/21/18 11:17 AM, Eduardo A. Bustamante López wrote: > On Fri, Sep 21, 2018 at 04:17:11PM +0200, Enrique Soriano wrote: >>> You're seeing the status from the creation of the background job (which is >>> always 0), not from its completion. >> >> Ah, I see. >> >> Anyway, the behavior is not coherent with the manual page: in this >> case, $? has the status from the creation of the background job, >> that's not "the status of the most recently executed foreground >> pipeline". > > The thing that creates the background job IS a foreground pipeline. It's a > foreground pipeline that creates a background > job, if that makes sense. You're right, but it might be less confusing if I removed the "foreground" from the description. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Special parameter ?
On Fri, Sep 21, 2018 at 04:17:11PM +0200, Enrique Soriano wrote: > > You're seeing the status from the creation of the background job (which is > > always 0), not from its completion. > > Ah, I see. > > Anyway, the behavior is not coherent with the manual page: in this > case, $? has the status from the creation of the background job, > that's not "the status of the most recently executed foreground > pipeline". The thing that creates the background job IS a foreground pipeline. It's a foreground pipeline that creates a background job, if that makes sense. dualbus@ubuntu:~$ cat t.sh (exit 2) (exit 3) & pid=$! echo $? wait "$pid" echo $? dualbus@ubuntu:~$ bash t.sh 0 3 I recommend to store the value of $? in a variable if you plan on using it afterwards, since it's extremely easy to end up overriding its value.
Re: Special parameter ?
> You're seeing the status from the creation of the background job (which is > always 0), not from its completion. Ah, I see. Anyway, the behavior is not coherent with the manual page: in this case, $? has the status from the creation of the background job, that's not "the status of the most recently executed foreground pipeline". Regards
Re: Special parameter ?
On Fri, Sep 21, 2018 at 01:23:57PM +0200, esori...@gsyc.urjc.es wrote: > The man page of bash (Special Parameters section) says: > >? expands to the exit status of the most >recently executed foreground pipeline. > > Nevertheless, background commands also modify the > value of this variable. This is incorrect. The exit status of a background command can only be retrieved by calling "wait" (or if the background command itself chooses to write the forthcoming exit status to stdout, or to a file, etc.). > Example: > > esoriano@omac:~$ false > esoriano@omac:~$ sleep 2 & >(wait 3 seconds) > esoriano@omac:~$ echo $? > 0 > esoriano@omac:~$ > > In this example, the most recently executed foreground > command is false, so the value of $? should be 1. Under "Lists": If a command is terminated by the control operator &, the shell exe‐ cutes the command in the background in a subshell. The shell does not wait for the command to finish, and the return status is 0. You're seeing the status from the creation of the background job (which is always 0), not from its completion. Compare: wooledg:~$ (sleep 3; exit 47) & pid=$! [1] 19112 wooledg:~$ echo $? 0 wooledg:~$ [1]+ Exit 47 ( sleep 3; exit 47 ) wooledg:~$ echo $? 0 wooledg:~$ wait $pid; echo $? 47 Even after the background job completes, its exit status is never propagated into the foreground shell's $? parameter. The only way to retrieve the status is with a wait command which is given a PID or a job specifier.
suspend and ||
According to the manual: (I) Typing the suspend character (typically ^Z, Control-Z) while a process is running causes that process to be stopped and returns control to bash. (II) An OR list has the form command1 || command2 command2 is executed if and only if command1 returns a non-zero exit status. The return status of AND and OR lists is the exit status of the last command executed in the list. In the following example: esoriano@omac:~$ sleep 10 || echo hey ^Z [5]+ Stopped sleep 10 hey esoriano@omac:~$ there are two different issues: (I) echo is executed when sleep is suspended, i.e., command2 is executed before command1 exits. (II) the exit status of command1 (when the job is resumed) will be 0 and command2 is (was) executed. Configuration Information [Automatically generated, do not change]: Machine: x86_64 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu' -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I../. -I.././include -I.././lib -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/build/bash-vEMnMR/bash-4.4.18=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -Wno-parentheses -Wno-format-security uname output: Linux omac 4.15.0-34-generic #37-Ubuntu SMP Mon Aug 27 15:21:48 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Machine Type: x86_64-pc-linux-gnu Bash Version: 4.4 Patch Level: 19 Release Status: release Description: [Detailed description of the problem, suggestion, or complaint.] Repeat-By: [Describe the sequence of events that causes the problem to occur.] Fix: [Description of how to fix the problem. If you don't know a fix for the problem, don't include this section.]
Special parameter ?
The man page of bash (Special Parameters section) says: ? expands to the exit status of the most recently executed foreground pipeline. Nevertheless, background commands also modify the value of this variable. Example: esoriano@omac:~$ false esoriano@omac:~$ sleep 2 & (wait 3 seconds) esoriano@omac:~$ echo $? 0 esoriano@omac:~$ In this example, the most recently executed foreground command is false, so the value of $? should be 1. Configuration Information [Automatically generated, do not change]: Machine: x86_64 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu' -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I../. -I.././include -I.././lib -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/build/bash-vEMnMR/bash-4.4.18=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -Wno-parentheses -Wno-format-security uname output: Linux omac 4.15.0-34-generic #37-Ubuntu SMP Mon Aug 27 15:21:48 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Machine Type: x86_64-pc-linux-gnu Bash Version: 4.4 Patch Level: 19 Release Status: release Description: [Detailed description of the problem, suggestion, or complaint.] Repeat-By: [Describe the sequence of events that causes the problem to occur.] Fix: [Description of how to fix the problem. If you don't know a fix for the problem, don't include this section.]
Re: trap - inconsistent behaviour
On 9/20/18 7:39 PM, Jeremy Townshend wrote:
> Bash Version: 4.4
> Patch Level: 19
> Release Status: release
>
> Description:
> The behaviour of the "trap" builtin command changes merely by printing
> the
> list of commands associated with each signal (trap command issued
> without
> arguments).
>
> Repeat-By:
> Case 1: In a fresh terminal emulator issue the following:
>
> ( trap 'echo trapped >&2' HUP; { sleep 10 & sleepPID=$!; wait
> $sleepPID; } > >( sleep 1; kill -HUP $BASHPID; cat ) )
>
> Case 2: Then in a fresh terminal emulator issue the following:
>
> ( trap 'echo trapped >&2' HUP; { sleep 10 & sleepPID=$!; wait
> $sleepPID; } > >( trap; sleep 1; kill -HUP $BASHPID; cat ) )
>
> Expected outcome: either the trap is triggered in both cases or in
> neither
> since the only difference is the additional trap command with no
> arguments
> (for printing the list of commands associated with each signal). That
> is, in
> this case, "trapped" is expected to be printed in both cases or in
> neither
> case.
The trap command shouldn't be executed in either case. The process
substitution is executed in a subshell environment, which doesn't inherit
traps from its parent (though it inherits the trap strings; see
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_28).
The process substitution sends the SIGHUP to itself ($BASHPID), which
should cause it to terminate but not result in a signal being sent to its
parent. However, the parent is getting a SIGHUP from somewhere.
It seems like this behavior is the result of the shell being interactive.
I'll take a look.
Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Which commit for a bug in 4.3.48 which is fixed in 4.4.23
Hi,
with 4.3.48 the line
T="";echo ">${T//*/ }<"
leads to
><
but with 4.4.23 the correct result is given back
> <
in the git repro I do not find any useful login entry for this
Werner
--
"Having a smoking section in a restaurant is like having
a peeing section in a swimming pool." -- Edward Burr
signature.asc
Description: PGP signature
