Re: embedded newlines in shell function variable expansion
On Fri, Oct 9, 2020 at 11:11 AM Byrnes, Robert wrote: > > If I use this Makefile ... > > > bash$ cat Makefile > FOO := $(shell echo $(ENTRIES) ; ) > BAR := $(shell echo $(ENTRIES) ) > > all: > @echo FOO = $(FOO) > @echo BAR = $(BAR) > > .PHONY: all > > > ... and set ENTRIES with embedded newlines, then this happens: > > > bash$ make 'ENTRIES= > blartz > blurfl > ' > FOO = blartzblurfl > BAR = blartz blurfl > Good morning. i opened https://savannah.gnu.org/bugs/index.php?59247 and attached a patch. Thanks for your report and the example. regards, Dmitry
RE: embedded newlines in shell function variable expansion
> Hm, interesting. I guess it's possible we are stripping out newlines before > invoking the shell. I don't remember that being the case but it could be. > > I'm not exactly sure what your objective is ... We use a make variable that can be set to a whitespace-separated list of pathnames. These are passed to functions like $(notdir ...), and that broke when a script happened to use newlines instead of spaces as separators, because all of the pathnames were glued together into a single word when the newlines were stripped. We don't actually care about preserving newlines, and would be happy if they turned into spaces. But we do care about preserving whitespace that separates words. > ... but note that as written this probably won't do what you want, because if > you run: > > /bin/sh -c 'echo foo > bar' > > then it will echo "foo" and try to run the program "bar". I guess that particular case could be protected by quotes as $(shell "$(ENTRIES)"). -- Bob
Re: embedded newlines in shell function variable expansion
On Fri, 2020-10-09 at 18:00 +, Byrnes, Robert wrote: > [pid 144497] execve("/bin/sh", ["/bin/sh", "-c", "/bin/echo blartzblurfl ; > "], 0x7ffd98ee6bc0 /* 72 vars */) = 0 Hm, interesting. I guess it's possible we are stripping out newlines before invoking the shell. I don't remember that being the case but it could be. I'm not exactly sure what your objective is but note that as written this probably won't do what you want, because if you run: /bin/sh -c 'echo foo bar' then it will echo "foo" and try to run the program "bar". > For reference, here's the modified Makefile ... Thx but this is not needed to be included with all the messages... that's why we have an email archive :).
RE: embedded newlines in shell function variable expansion
> My suspicion is that it's a difference between your /bin/echo command and > your shell's builtin echo command. > > In the first case (with the ";") because there's a special character in the > command GNU make will run a shell like this: > >/bin/sh -c 'echo ... ; ' > > and the shell's built-in echo will be used (if it has one, which most shells > do). > > In the second case (without the ";"), there's no special character and so GNU > make runs the fast path which invokes the echo command directly, not using a > shell, and so it will use your PATH to find an "echo" program and run that. > > If you change both commands to use /bin/echo explicitly, do they work the > same way? The behavior doesn't change if I use /bin/echo for both commands. strace shows the subprocesses exactly as you describe, but the newlines have already been stripped (by make, I presume) before the shell is invoked: execve("/usr/bin/make", ["make", "ENTRIES=\nblartz\nblurfl\n"], 0x7ffdac2aff40 /* 72 vars */) = 0 [pid 144497] execve("/bin/sh", ["/bin/sh", "-c", "/bin/echo blartzblurfl ; "], 0x7ffd98ee6bc0 /* 72 vars */) = 0 [pid 144497] execve("/bin/echo", ["/bin/echo", "blartzblurfl"], 0x1631fb0 /* 71 vars */) = 0 [pid 144498] execve("/bin/echo", ["/bin/echo", "blartz\nblurfl\n"], 0x7ffd98ee6bc0 /* 72 vars */) = 0 For reference, here's the modified Makefile ... FOO := $(shell /bin/echo $(ENTRIES) ; ) BAR := $(shell /bin/echo $(ENTRIES) ) all: @echo FOO = $(FOO) @echo BAR = $(BAR) .PHONY: all And this is how to reproduce the behavior ... bash$ make 'ENTRIES= blartz blurfl ' FOO = blartzblurfl BAR = blartz blurfl Thanks, -- Bob
Re: embedded newlines in shell function variable expansion
On Fri, 2020-10-09 at 15:03 +, Byrnes, Robert wrote: > Why is the embedded whitespace removed in the first (FOO) case? It > seems related to the semicolon shell metacharacter ... My suspicion is that it's a difference between your /bin/echo command and your shell's builtin echo command. In the first case (with the ";") because there's a special character in the command GNU make will run a shell like this: /bin/sh -c 'echo ... ; ' and the shell's built-in echo will be used (if it has one, which most shells do). In the second case (without the ";"), there's no special character and so GNU make runs the fast path which invokes the echo command directly, not using a shell, and so it will use your PATH to find an "echo" program and run that. If you change both commands to use /bin/echo explicitly, do they work the same way?