Re: Trailing newlines disappear

2017-06-13 Thread Peter & Kelly Passchier
On 14/06/2560 03:52, Chet Ramey wrote: > You mean command substitution cutting off the input it reads at the > occurrence of a NUL? What I am really after is a shell option for command substitution not discarding trailing newlines. Peter

Re: Trailing newlines disappear

2017-06-13 Thread Chet Ramey
On 6/12/17 8:55 PM, Peter & Kelly Passchier wrote: > On 13/06/2560 02:54, Chet Ramey wrote: >> If you want to effectively change it to a newline, specify NUL as the >> line delimiter using the -d option (new in bash-4.4). > > Thanks, that sounds like a clean solution! > I only reverted to mapfile

Re: Trailing newlines disappear

2017-06-12 Thread Peter & Kelly Passchier
On 13/06/2560 02:54, Chet Ramey wrote: > If you want to effectively change it to a newline, specify NUL as the > line delimiter using the -d option (new in bash-4.4). Thanks, that sounds like a clean solution! I only reverted to mapfile because $(...) command substitution could not be made to work

Re: Trailing newlines disappear

2017-06-12 Thread Chet Ramey
On 6/9/17 1:40 PM, Peter & Kelly Passchier wrote: > On 09/06/2560 23:38, L A Walsh wrote: >> Chet Ramey wrote: >>> >>> Should mapfile silently drop the NULs? >> >> Maybe add a flag to ignore NUL bytes that could be used in the 'read' >> statement as well? If not specified, keep same behavior? >

Re: Trailing newlines disappear

2017-06-09 Thread Greg Wooledge
On Fri, Jun 09, 2017 at 08:58:05PM +0300, Pierre Gaston wrote: > Even if they don't realize it, few people would expect: > > var=$(wc -l file);echo "$var" > > to print 2 lines. imadev:~$ file=$'a\nb\nc\nd\ne'; touch "$file" imadev:~$ var=$(wc -l "$file"); echo "$var" 0 a b c d e > Trailing newl

Re: Trailing newlines disappear

2017-06-09 Thread Pierre Gaston
On Fri, Jun 9, 2017 at 8:40 PM, Peter & Kelly Passchier < peterke...@passchier.net> wrote: > On 09/06/2560 23:38, L A Walsh wrote: > > Chet Ramey wrote: > >> > >> Should mapfile silently drop the NULs? > > > > Maybe add a flag to ignore NUL bytes that could be used in the 'read' > > statement as

Re: Trailing newlines disappear

2017-06-09 Thread Peter & Kelly Passchier
On 09/06/2560 23:38, L A Walsh wrote: > Chet Ramey wrote: >> >> Should mapfile silently drop the NULs? > > Maybe add a flag to ignore NUL bytes that could be used in the 'read' > statement as well? If not specified, keep same behavior? That sounds like it might be useful. It might be more desir

Re: Trailing newlines disappear

2017-06-09 Thread PePa
On 09/06/2560 19:06, Greg Wooledge wrote: > imadev:~$ a=$(printf 'foo\0bar\nbaz\nquux\n'; printf x) a=${a%x} > bash: warning: command substitution: ignored null byte in input > imadev:~$ declare -p a > declare -- a="foobar > baz > quux > " > > imadev:~$ IFS= read -rd '' a < <(printf 'foo\0bar\nbaz

Re: Trailing newlines disappear

2017-06-09 Thread L A Walsh
Chet Ramey wrote: Should mapfile silently drop the NULs? Maybe add a flag to ignore NUL bytes that could be used in the 'read' statement as well? If not specified, keep same behavior?

Re: Trailing newlines disappear

2017-06-09 Thread Chet Ramey
On 6/9/17 9:36 AM, Greg Wooledge wrote: > If anything, the fact that $() drops NUL bytes is the surprising outlier. > But it's definitely too late to change that. Remember how many people > bitched and complained when 4.4 added a warning? How could I forget? But the complaining came because they

Re: Trailing newlines disappear

2017-06-09 Thread Greg Wooledge
On Fri, Jun 09, 2017 at 09:22:58AM -0400, Chet Ramey wrote: > On 6/9/17 8:06 AM, Greg Wooledge wrote: > > imadev:~$ mapfile -t a < <(printf 'foo\0bar\nbaz\nquux\n') > > imadev:~$ declare -p a > > declare -a a=([0]="foo" [1]="baz" [2]="quux") > > There's no mystery here. Mapfile reads lines delimit

Re: Trailing newlines disappear

2017-06-09 Thread Chet Ramey
On 6/9/17 8:06 AM, Greg Wooledge wrote: > Actually, it looks like the mapfile variant also has its own... let's > call it "idiosyncratic" NUL handling: > > imadev:~$ unset a > imadev:~$ mapfile -t a < <(printf 'foo\0bar\nbaz\nquux\n') > imadev:~$ declare -p a > declare -a a=([0]="foo" [1]="baz" [

Re: Trailing newlines disappear

2017-06-09 Thread Greg Wooledge
On Fri, Jun 09, 2017 at 08:06:28AM +0700, Peter & Kelly Passchier wrote: > On 09/06/2560 05:26, Eduardo Bustamante wrote: > > What's wrong with: > > > > IFS= read -rd '' foo < "$file" > > I think that's the winner! So long as you know the file doesn't contain any NUL bytes. The other variants w

Re: Trailing newlines disappear

2017-06-08 Thread Peter & Kelly Passchier
On 09/06/2560 05:26, Eduardo Bustamante wrote: > What's wrong with: > > IFS= read -rd '' foo < "$file" I think that's the winner!

Re: Trailing newlines disappear

2017-06-08 Thread Eduardo Bustamante
On Thu, Jun 8, 2017 at 3:56 PM, Greg Wooledge wrote: > On Thu, Jun 08, 2017 at 10:44:29PM +0200, Geir Hauge wrote: >> You can pick one of these instead: >> >> mapfile < "$file"; IFS= foo="${MAPFILE[*]}"; unset -v IFS >> >> or >> >> mapfile < "$file"; printf -v foo %s "${MAPFILE[@]}" > > or

Re: Trailing newlines disappear

2017-06-08 Thread Greg Wooledge
On Thu, Jun 08, 2017 at 10:44:29PM +0200, Geir Hauge wrote: > You can pick one of these instead: > > mapfile < "$file"; IFS= foo="${MAPFILE[*]}"; unset -v IFS > > or > > mapfile < "$file"; printf -v foo %s "${MAPFILE[@]}" or lambda() { local MAPFILE IFS=; mapfile < "$file"; foo="${

Re: Trailing newlines disappear

2017-06-08 Thread Peter & Kelly Passchier
On 09/06/2560 03:44, Geir Hauge wrote: > Greg already pointed out that this doesn't work. > > You can pick one of these instead: > > mapfile < "$file"; IFS= foo="${MAPFILE[*]}"; unset -v IFS > > or > > mapfile < "$file"; printf -v foo %s "${MAPFILE[@]}" > Yes, thanks. The second one lo

Re: Trailing newlines disappear

2017-06-08 Thread Geir Hauge
On Fri, Jun 09, 2017 at 03:34:39AM +0700, PePa wrote: > On 09/06/2560 02:14, Greg Wooledge wrote: > > Well, it leaves IFS changed, because you didn't revert or unset it, > > or run the entire thing in a function with local IFS. Also, I'd use > > "${MAPFILE[*]}" to reinforce that you are joining an

Re: Trailing newlines disappear

2017-06-08 Thread PePa
On 09/06/2560 02:14, Greg Wooledge wrote: > Well, it leaves IFS changed, because you didn't revert or unset it, > or run the entire thing in a function with local IFS. Also, I'd use > "${MAPFILE[*]}" to reinforce that you are joining an array into a string, > rather than expanding the array as a l

Re: Trailing newlines disappear

2017-06-08 Thread Greg Wooledge
ARGH! You dropped the list address! This was on bug-bash, right?? On Fri, Jun 09, 2017 at 03:14:10AM +0700, PePa wrote: > On 09/06/2560 02:14, Greg Wooledge wrote: > > Well, it leaves IFS changed, because you didn't revert or unset it, > > or run the entire thing in a function with local IFS. A

Re: Trailing newlines disappear

2017-06-08 Thread Greg Wooledge
On Fri, Jun 09, 2017 at 01:52:44AM +0700, Peter & Kelly Passchier wrote: > On 09/06/2560 00:42, Greg Wooledge wrote: > > foo=$(cat "$file"; printf x) foo=${foo%x} > > The workaround I came up with is: > mapfile <"$file"; IFS= foo=${MAPFILE[@]} > > This seems to be faster, but it probably has othe

Re: Trailing newlines disappear

2017-06-08 Thread Peter & Kelly Passchier
On 09/06/2560 00:42, Greg Wooledge wrote: > It's not a bug. This is how command substitution has worked since > the original Bourne shell. > > The workaround is to put something inside the command substitution, > so that the newlines aren't trailing any more, and then strip it away > afterward: >

Re: Trailing newlines disappear

2017-06-08 Thread Greg Wooledge
On Fri, Jun 09, 2017 at 12:38:19AM +0700, PePa wrote: > I would think this is a bug: > 4.3.48(1)-release> printf "q\n\n\n" >w > 4.3.48(1)-release> printf "$(cat w)" > q > 4.3.48(1)-release> > > Is there some workaround to somehow preserve the trailing newlines? It's not a bug. This is how comma

Trailing newlines disappear

2017-06-08 Thread PePa
I would think this is a bug: 4.3.48(1)-release> printf "q\n\n\n" >w 4.3.48(1)-release> cat w q 4.3.48(1)-release> printf "$(cat w)" q 4.3.48(1)-release> Is there some workaround to somehow preserve the trailing newlines? Peter