Re: Shell scripts: variable assignment within read loops
On Sun, 17 Aug 2008, David Wolfskill wrote: [snipped] > will assign to foo the value of the bar variable form the last record > read (in FreeBSD 6.3-STABLE, at least), the following fails to do so: > > foo="" > cat $filename | while read bar ... ; do >... > foo=$bar >... > done > echo $foo > > Well, that's not *quite* accurate:the assignment is done all right, but > in the latter case, it appears to be done in a subshell, so by the time > we get to the "echo" statement, any variable assignments from within the > read loop have vanished. You've already defined the reason behind this. Since subshells for parts of pipelines aren't guaranteed, you need something more clever. Assuming you're only interested in the _last_ value of foo, you could try this: foo=$( cat $filename | ( while read bar; do ... foo=$bar ... done echo $foo ) ) Cheers, jan -- jan grant, ISYS, University of Bristol. http://www.bris.ac.uk/ Tel +44 (0)117 3317661 http://ioctl.org/jan/ ( echo "ouroboros"; cat ) > /dev/fd/0 # it's like talking to yourself sometimes ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Shell scripts: variable assignment within read loops
David Wolfskill wrote:
> foo=""
> cat $filename | while read bar ... ; do
>...
> foo=$bar
>...
> done
> echo $foo
>
A trick I've used to great advantage in bourne shell and bash for
passing multiple variables back is to produce small snippets of shell
script within a function, such as the following, for pulling in a bunch
of variables with a single program invocation for efficiency:
get_stats(){
stat -fc 'mount="%n" blksz="%S" total="%b" free_root="%f" \
free_user="%a"' "[EMAIL PROTECTED]"
}
get_stats "/" "/dev" "/tmp" | while read line
do
eval ${line}
# now mount, blksz, total, free_root, and free_user are set here.
printf "=> %s has %u free %u-byte blocks, out of %u\n" \
"${mount}" "${free_user}" "${blksz}" "${total}"
done
The function returns a series of lines that can be iterated with 'while
read', and evaluated individually for action. If the function returns
only a single line, it can be passed directly into eval:
eval $(get_stats "/")
As this shortcut does execute arbitrary code, however, there is always a
chance that it can be hijacked for nefarious purposes if the data source
is untrusted.
--
Fuzzy love,
-CyberLeo
Technical Administrator
CyberLeo.Net Webhosting
http://www.CyberLeo.Net
<[EMAIL PROTECTED]>
Furry Peace! - http://.fur.com/peace/
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Shell scripts: variable assignment within read loops
David Wolfskill wrote: > I am writing a (Bourne) shell script that is intended (among other > things) to obtain information from a command, such as: > > netstat -nibd -f inet > > by reading and parsing the output. > > However, the "obvious" (to me) approach of piping the output of the > command to the standard input of a "while read ..." statement turns out > to be not very useful; > [...] > Well, that's not *quite* accurate:the assignment is done all right, but > in the latter case, it appears to be done in a subshell, so by the time > we get to the "echo" statement, any variable assignments from within the > read loop have vanished. That's correct, as Giorgos has already pointed out. Most bourne shells execute all parts of a pipe except the first one in a subshell, so any assignments are lost. A common way is to echo things from within the subshell and capture them through command expansion, like this: foo=$( something | while read x; do whatever echo "value" done ) That will assign "value" to the variable foo. This only works for single variables, of course. If you want to assign to multiple variables, it gets a little more tricky. One way is to use single assignment like above, and then split it into several variables on a delimiter character. The following will split $foo on whitespace and assign the results to $1, $2, $3 etc., with the count in $#: set -- $foo You can split on any other character by setting the IFS variable of the shell appropriately. If you know in advance how many values you'll get, another possibility is to use a so-called "here document": read foo1 foo2 foo3 rest
Re: Shell scripts: variable assignment within read loops
On Mon, 18 Aug 2008 14:33:05 +0200, Polytropon <[EMAIL PROTECTED]> wrote: > As I thought while reading your message, awk seems to be > a good solution. Just a note: > > On Mon, 18 Aug 2008 06:29:03 +0300, Giorgos Keramidas <[EMAIL PROTECTED]> > wrote: >> Would you >> be ok with an awk(1) script instead of /bin/sh? It tends to be nicer >> for this sort of thing, i.e.: >> >> [...] >> $ netstat -nibd -f inet | awk -f david.awk > > You could start your awk skript with > > #!/usr/bin/awk > > and give it +x attribute, as well as adding the desired source > command "netstat -nibd -f inet" to the script, using awk's system() > function, so you can start it more easily or use it in combination > with other commands. > > % ./netstuff.awk That's an option, but piping *to* awk may be slightly trickier then. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Shell scripts: variable assignment within read loops
As I thought while reading your message, awk seems to be a good solution. Just a note: On Mon, 18 Aug 2008 06:29:03 +0300, Giorgos Keramidas <[EMAIL PROTECTED]> wrote: > Would you > be ok with an awk(1) script instead of /bin/sh? It tends to be nicer > for this sort of thing, i.e.: > > [...] > $ netstat -nibd -f inet | awk -f david.awk You could start your awk skript with #!/usr/bin/awk and give it +x attribute, as well as adding the desired source command "netstat -nibd -f inet" to the script, using awk's system() function, so you can start it more easily or use it in combination with other commands. % ./netstuff.awk -- Polytropon >From Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Shell scripts: variable assignment within read loops
On Mon, Aug 18, 2008 at 06:29:03AM +0300, Giorgos Keramidas wrote:
> ...
> You are right that feeding data to a looping construct through a pipe
> may run in a subshell. The ``Single UNIX Specification'' says
>
Ah; thanks for the confirmation.
>...
> What I usually do in similar shell scripts is something like:
>
> cat "${filename}" | sed -n -e '/foo/ s/bar/baz/' | \
> xargs -n1 blah
>
> This isn't exactly the same as assigning $foo to the results of the
> loop, but you can also use:
>
> foo=`cat $filename | while read bar ; do \
> stuff ...
> echo "$bar"
> more stuff...
> done`
Right; I had seen that type of construct in /etc/rc.d.* (which is where
I often look for samples of shell scripts that need to work reliably).
As you noticed, that won't quite do for what I'm trying to accomplish here.
> ...
> > As you see, I am circumventing the issue by writing to a transient
> > file. In the intended application, the script is to be used to gather
> > resource-utilization information; thus, I want its "footprint" to be
> > smaller, rather than larger. Granted, in my case, I would be writing
> > a tiny text file to a swap-backed tmpfs, but in production, I won't
> > have the luxury of knowing that in advance: the intent is that the
> > script must run on a minimal FreeBSD system, with no "ports" or other
> > 3rd-party software installed.
> >
> > Is there some other -- possibly better -- way to do this (using Bourne
> > shell scripting)?
>
> Ah, that's much better. Now I see what you are trying to do.
:-)
> Would you be ok with an awk(1) script instead of /bin/sh? It tends
> to be nicer for this sort of thing, i.e.:
Yes, awk(1) would be OK. I'll be more inclined to use it if I can
figure out a way to use it instead of sed(1) for a very different part
of the script. :-}
> $ expand david.awk | cat -n
> 1 #
> 2 # Gather the field names if this is a header-line.
> 3 #
> 4 $0 ~ /^Name/ {
> 5 for (k = 1; k <= NF; k++)
> 6 tag[k] = $k;
> 7 }
> 8
> 9 #
> 10 # For all other lines, just print the tagged field values.
> 11 #
> 12 $0 !~ /^Name/ {
> 13 name = $1;
> 14 for (k = 1; k <= NF; k++) {
> 15 if ($k == "-")
> 16 $k = "0";
> 17 printf "%s_%s: %s\n", tag[k], name, $k;
> 18 }
> 19 }
>
> $ netstat -nibd -f inet | awk -f david.awk
> Name_re0: re0
> Mtu_re0: 1500
> Network_re0: 192.168.1.0/2
> ...
Very cool; thank you very much! I will study that a bit
(I'd normally do this stuff in Perl, but in addition to the other issues
mentioned earlier, the script will be sleeping most of the time, but
wake up & spit out results periodically. The usual case will be every 5
minutes, but I plan to make use of it with radically shorter periods in
certain specialized environments -- such as every 5 seconds. And I
still want it to be low overhead. I also note in passing that in its
"production" environments, the script's standard output will be
redirected to append to a file on a different machine via an SSH tunnel.)
> With a bit of preprocessing, it may be possible to extract the network
> names and print the "(end) NICs: XXX XXX" part too.
Right -- much of the output I demonstrated was strictly for debugging/
expository purposes.
Thanks again, Giorgos!
Peace,
david
--
David H. Wolfskill [EMAIL PROTECTED]
Depriving a girl or boy of an opportunity for education is evil.
See http://www.catwhisker.org/~david/publickey.gpg for my public key.
pgpQZk6NuCeRc.pgp
Description: PGP signature
Re: Shell scripts: variable assignment within read loops
On Sun, 17 Aug 2008 18:33:28 -0700, David Wolfskill <[EMAIL PROTECTED]> wrote:
> I am writing a (Bourne) shell script that is intended (among other
> things) to obtain information from a command, such as:
>
> netstat -nibd -f inet
>
> by reading and parsing the output.
>
> However, the "obvious" (to me) approach of piping the output of the
> command to the standard input of a "while read ..." statement turns out
> to be not very useful; it appears that while
>
> foo=""
> while read bar ... ; do
>...
> foo=$bar
>...
> done <$filename
> echo $foo
>
> will assign to foo the value of the bar variable form the last record
> read (in FreeBSD 6.3-STABLE, at least), the following fails to do so:
>
> foo=""
> cat $filename | while read bar ... ; do
>...
> foo=$bar
>...
> done
> echo $foo
>
> Well, that's not *quite* accurate:the assignment is done all right, but
> in the latter case, it appears to be done in a subshell, so by the time
> we get to the "echo" statement, any variable assignments from within the
> read loop have vanished.
Hi David,
You are right that feeding data to a looping construct through a pipe
may run in a subshell. The ``Single UNIX Specification'' says
"... each command of a multi-command pipeline is in a subshell
environment; as an extension, however, any or all commands in a
pipeline may be executed in the current environment."
You can read the online text of the Single UNIX Specification at:
http://www.unix.org/single_unix_specification/
A simple 'registration' is required, and you need a browser that
supports cookies. But after that, you can follow the links to the shell
command language page at
http://www.opengroup.org/onlinepubs/95399/utilities/xcu_chap02.html
Near section `` 2.12 Shell Execution Environment'' you can find the text
I quoted.
What I usually do in similar shell scripts is something like:
cat "${filename}" | sed -n -e '/foo/ s/bar/baz/' | \
xargs -n1 blah
This isn't exactly the same as assigning $foo to the results of the
loop, but you can also use:
foo=`cat $filename | while read bar ; do \
stuff ...
echo "$bar"
more stuff...
done`
> Now here's a copy of the in-development script:
>
> #! /bin/sh
>
> cmd="netstat -nibd -f inet"
> ctr=0
> clist=""
> hlist=`$cmd | head -1`
> for f in $hlist; do
> ctr=$(( $ctr + 1 ))
> eval c$ctr=\"$f\"
> eval h_$f=c$ctr
> done
> cmax=$ctr
>
> t_file=`mktemp /tmp/X`
> $cmd | tail +2 >$t_file
> while read $hlist dummy; do
> if [ "$Name" = "lo0" ]; then
> continue
> fi
> for f in $hlist; do
> eval val=\"\$$f\"
> case $val in
> -) eval ${f}_$Name=0;;
> *) eval ${f}_$Name="$val";;
> esac;
> done
> nics="$Name $nics";
> done rm $t_file
> echo "(end) NICs: $nics"
> for n in $nics; do
> for f in $hlist; do
> eval "echo ${f}_$n: \$${f}_$n"
> done
> done
>
> exit 0
>
> And its output on my laptop:
>
> (end) NICs: ath0
> Name_ath0: ath0
> Mtu_ath0: 1500
> Network_ath0: 172.17
> Address_ath0: 172.17.1.30
> Ipkts_ath0: 725191
> Ierrs_ath0: 0
> Ibytes_ath0: 185144197
> Opkts_ath0: 821917
> Oerrs_ath0: 0
> Obytes_ath0: 74260936
> Coll_ath0: 0
> Drop_ath0: 0
>
> and (somewhat more interestingly) on my "firewall" machine:
>
> (end) NICs: dc0 de0 fxp0
> Name_dc0: dc0
> Mtu_dc0: 1500
> Network_dc0: 172.16.8/24
> Address_dc0: 172.16.8.1
> Ipkts_dc0: 2501577
> Ierrs_dc0: 0
> Ibytes_dc0: 215386153
> Opkts_dc0: 20269087
> Oerrs_dc0: 0
> Obytes_dc0: 2553930555
> Coll_dc0: 0
> Drop_dc0: 0
> Name_de0: de0
> Mtu_de0: 1500
> Network_de0: 63.193.123/24
> Address_de0: 63.193.123.122
> Ipkts_de0: 5936847
> Ierrs_de0: 0
> Ibytes_de0: 734092787
> Opkts_de0: 18557543
> Oerrs_de0: 0
> Obytes_de0: 2551089632
> Coll_de0: 0
> Drop_de0: 0
> Name_fxp0: fxp0
> Mtu_fxp0: 1500
> Network_fxp0: 172.17
> Address_fxp0: 172.17.0.1
> Ipkts_fxp0: 10013
> Ierrs_fxp0: 0
> Ibytes_fxp0: 1366082
> Opkts_fxp0: 1253115
> Oerrs_fxp0: 0
> Obytes_fxp0: 70429903
> Coll_fxp0: 0
> Drop_fxp0: 0
>
> As you see, I am circumventing the issue by writing to a transient
> file. In the intended application, the script is to be used to gather
> resource-utilization information; thus, I want its "footprint" to be
> smaller, rather than larger. Granted, in my case, I would be writing
> a tiny text file to a swap-backed tmpfs, but in production, I won't
> have the luxury of knowing that in advance: the intent is that the
> script must run on a minimal FreeBSD system, with no "ports" or other
> 3rd-party software installed.
>
> Is there some other -- possibly better -- way to do this (using Bourne
> shell scripting)?
Ah, that's much better. Now I see what you are trying to do. Would you
be ok with an awk(1) script instead of /bin/sh? It tends to be nicer
for this sort of thing, i.e.:
$ expand david.awk | cat -n
1 #
2 # Gather the field names
