Re: missing way to extract data out of data

[felix]

I think, memory footprint doesn't really matter when standard availlable ram
is lot away...

But *time required to ``fork''* could matter:

  QuickTest() {
local TIMEFORMAT='%R ( %U + %S )';
printf "%-10s: " "${1##*/}";
time for i in {1..1000};
do
res=$("$@");
done;
[ "$res" != "12" ] && echo "WARN: $1: '$res' != 12."
  }
  CacheHostName=$(&1;
done |
  sort -nk 3

After ~23 seconds, this print (on my host):

  dash  : 1.667 ( 1.206 + 0.519 )
  busybox   : 1.868 ( 1.264 + 0.665 )
  sed   : 2.150 ( 1.364 + 0.841 )
  bash  : 2.201 ( 1.520 + 0.719 )
  perl  : 2.608 ( 1.679 + 0.978 )
  awk   : 2.662 ( 1.666 + 1.041 )
  python: 9.881 ( 6.867 + 3.052 )

Of course, all this was called without modules, so this times represent
something like *minimal footprint when forked to*.

There is no more consideration of efficience when used, for sample, as
filter throught big volume of data, for making complex calculs, etc...

On Wed, Mar 24, 2021 at 10:52:16PM -0400, Dale R. Worley wrote:
> Andreas Schwab  writes:
> > $ size /usr/bin/perl /bin/bash
> >textdata bss dec hex filename
> > 2068661   27364 648 2096673  1ffe21 /usr/bin/perl
> > 1056850   22188   61040 1140078  11656e /bin/bash
> $ size /usr/bin/perl /bin/bash
>text  data bss dec hex filename
>8588   876   0946424f8 /usr/bin/perl
>  898672 36064   22840  957576   e9c88 /bin/bash
> 
> I do suspect that's because perl is using more loadable modules than
...

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: missing way to extract data out of data

[Eli Schwartz]

On 3/23/21 4:46 AM, Andreas Schwab wrote:
> On Mär 22 2021, Dale R. Worley wrote:
> 
>> Greg Wooledge  writes:
>>> Partly true.  seq(1) is a Linux thing, and was never part of any
>>> tradition, until Linux people started doing it.
>>
>> Huh.  I started with Ultrix, and then SunOS, but don't remember learning
>> seq at a later date.
> 
> According to , seq
> appeared in Version 8 AT UNIX.
> 
>> I've never tracked down why, but the Perl executable is a lot smaller
>> than the Bash executable.
> 
> Is it?
> 
> $ size /usr/bin/perl /bin/bash
>textdata bss dec hex filename
> 2068661   27364 648 2096673  1ffe21 /usr/bin/perl
> 1056850   22188   61040 1140078  11656e /bin/bash
> 
> Of course, a lot of perl is part of loadable modules.

How thoroughly do you want to cheat?

$ size /usr/bin/perl /bin/bash
   textdata bss dec hex filename
   3840 792  1646481228 /usr/bin/perl
 923139   22092   60800 1006031   f59cf /bin/bash
$ du -sh /usr/bin/perl /bin/bash
16K /usr/bin/perl
932K/bin/bash

Gosh, how does the perl interpreter define an entire language core in
16kb??? Pure magic.

Naturally, the loadable modules you mention are very significant either way.


$ pacman -Qi bash perl
Name: bash
Version : 5.1.004-1
[...]
Installed Size  : 8.19 MiB
[...]
Name: perl
Version : 5.32.1-1
[...]
Installed Size  : 57.63 MiB


And, for extras, it turns out most of the size of perl's no-modules
interpreter (that ever so tiny 16kb binary) is in

$ du -sh /usr/lib/perl5/5.32/core_perl/CORE/libperl.so
3.7M/usr/lib/perl5/5.32/core_perl/CORE/libperl.so
$ size /usr/lib/perl5/5.32/core_perl/CORE/libperl.so
   textdata bss dec hex filename
3604395   77020   25352 3706767  388f8f
/usr/lib/perl5/5.32/core_perl/CORE/libperl.so


Which is indeed much larger than bash (one binary, no libbash.so,
whether to count the separate libreadline.so in my vendor configuration
is debatable).

-- 
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User



OpenPGP_signature
Description: OpenPGP digital signature

Re: missing way to extract data out of data

[Dale R. Worley]

Andreas Schwab  writes:
>> I've never tracked down why, but the Perl executable is a lot smaller
>> than the Bash executable.
>
> Is it?
>
> $ size /usr/bin/perl /bin/bash
>textdata bss dec hex filename
> 2068661   27364 648 2096673  1ffe21 /usr/bin/perl
> 1056850   22188   61040 1140078  11656e /bin/bash
>
> Of course, a lot of perl is part of loadable modules.

On mine, I get:

$ size /usr/bin/perl /bin/bash
   textdata bss dec hex filename
   8588 876   0946424f8 /usr/bin/perl
 898672   36064   22840  957576   e9c88 /bin/bash

I do suspect that's because perl is using more loadable modules than
bash, but I don't know much about how the object code is organized and
I've never been motivated enough to track down the truth.

Dale

Re: missing way to extract data out of data

[Andreas Schwab]

On Mär 22 2021, Dale R. Worley wrote:

> Greg Wooledge  writes:
>> Partly true.  seq(1) is a Linux thing, and was never part of any
>> tradition, until Linux people started doing it.
>
> Huh.  I started with Ultrix, and then SunOS, but don't remember learning
> seq at a later date.

According to , seq
appeared in Version 8 AT UNIX.

> I've never tracked down why, but the Perl executable is a lot smaller
> than the Bash executable.

Is it?

$ size /usr/bin/perl /bin/bash
   textdata bss dec hex filename
2068661   27364 648 2096673  1ffe21 /usr/bin/perl
1056850   22188   61040 1140078  11656e /bin/bash

Of course, a lot of perl is part of loadable modules.

Andreas.

-- 
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: missing way to extract data out of data

[Dale R. Worley]

Greg Wooledge  writes:
> Partly true.  seq(1) is a Linux thing, and was never part of any
> tradition, until Linux people started doing it.

Huh.  I started with Ultrix, and then SunOS, but don't remember learning
seq at a later date.

> (Before POSIX, it involved using expr(1) for every increment, which
> is simply abominable.)

And let's not forget the original "glob"!

But my point is that using external programs to do minor processing
tasks has a long history in shells.

> but beyond a certain
> point, trying to force a shell to act like a Real Programming Language
> is just not reasonable.

I've never tracked down why, but the Perl executable is a lot smaller
than the Bash executable.  So it's likely that turning a shell into a
Real Programming Language is also likely to be unusually expensive.

Dale

Re: missing way to extract data out of data

[Greg Wooledge]

On Fri, Mar 19, 2021 at 09:10:06PM -0400, Dale R. Worley wrote:
> Alex fxmbsw7 Ratchev  writes:
> > yea well it does wonders, however was looking for a way without spawning
> > externals like gawk.. maybe in future there will be =)
> 
> Traditionally, shell scripts depended on external binaries to do a lot
> of the processing.  At the least, what newer shells do with "{NNN..MMM}"
> and "[[" used to be done by "seq" and "test" a/k/a "[".  And what can be
> done by the complex parameter expansions ${...%...} and ${...#...} was
> done by "sed".

Partly true.  seq(1) is a Linux thing, and was never part of any
tradition, until Linux people started doing it.  Of note, it does not
exist on any other systems.  Some BSDs have jot(1) which does a similar
thing, though.  Counting in POSIX sh involves a while loop, and i=$((i+1)).
(Before POSIX, it involved using expr(1) for every increment, which
is simply abominable.)

test or [ has been a builtin in every common shell for ages.  The days
of shells without a builtin test command are long gone.  The difference
between [[ and test is not really significant, if you're using the
operators that are shared between them (e.g. test -f "$f").

The % and # parameter expansions have been around since ksh88, and are
standardized in POSIX.  You may be thinking of ${var//old/new} which
is not part of POSIX, and would definitely have been done in sed(1).

While I'm nitpicking your details, you are correct about your overall
point, which is that shells are designed to use external tools to get
things done.  There's certainly a large grey area of tasks that can be
done by builtins or by external tools, of course, but beyond a certain
point, trying to force a shell to act like a Real Programming Language
is just not reasonable.

Re: missing way to extract data out of data

[Dale R. Worley]

Alex fxmbsw7 Ratchev  writes:
> yea well it does wonders, however was looking for a way without spawning
> externals like gawk.. maybe in future there will be =)

Traditionally, shell scripts depended on external binaries to do a lot
of the processing.  At the least, what newer shells do with "{NNN..MMM}"
and "[[" used to be done by "seq" and "test" a/k/a "[".  And what can be
done by the complex parameter expansions ${...%...} and ${...#...} was
done by "sed".

Dale

Re: missing way to extract data out of data

[Alex fxmbsw7 Ratchev]

ehe :))

you tell me rather a good pike beginnings resource

On Fri, Mar 19, 2021 at 9:42 AM Andreas Schwab 
wrote:

> On Mär 19 2021, Alex fxmbsw7 Ratchev wrote:
>
> > yea well it does wonders, however was looking for a way without spawning
> > externals like gawk.. maybe in future there will be =)
>
> You know where to get perl or python.
>
> Andreas.
>
> --
> Andreas Schwab, sch...@linux-m68k.org
> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
> "And now for something completely different."
>

Re: missing way to extract data out of data

[Andreas Schwab]

On Mär 19 2021, Alex fxmbsw7 Ratchev wrote:

> yea well it does wonders, however was looking for a way without spawning
> externals like gawk.. maybe in future there will be =)

You know where to get perl or python.

Andreas.

-- 
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: missing way to extract data out of data

[Alex fxmbsw7 Ratchev]

yea well it does wonders, however was looking for a way without spawning
externals like gawk.. maybe in future there will be =)

On Fri, Mar 19, 2021 at 2:02 AM Dale R. Worley  wrote:

> Alex fxmbsw7 Ratchev  writes:
> > there is way to crop data to wanted, by cropping the exclulsions away
> > but what about a way to extract data, eg @( .. ) match
> > not about using [[ =~ but only var internal stuff
> > .. or do i miss there something
>
> If you want to do a pattern-match against a string, then extract a part
> of it, a common way is to use sed.  For instance, to extract a word and
> remove spaces before and after it:
>
> $ STRING='  abc def   '
> $ WORD="$( <<<$STRING sed -e 's/^ *\([^ ]*\) *$/\1/' )"
> $ echo "$WORD"
>
> It is common to use common Unix utilities to modify data items within
> shell scripts.
>
> Dale
>

Re: missing way to extract data out of data

[Dale R. Worley]

Alex fxmbsw7 Ratchev  writes:
> there is way to crop data to wanted, by cropping the exclulsions away
> but what about a way to extract data, eg @( .. ) match
> not about using [[ =~ but only var internal stuff
> .. or do i miss there something

If you want to do a pattern-match against a string, then extract a part
of it, a common way is to use sed.  For instance, to extract a word and
remove spaces before and after it:

$ STRING='  abc def   '
$ WORD="$( <<<$STRING sed -e 's/^ *\([^ ]*\) *$/\1/' )"
$ echo "$WORD"

It is common to use common Unix utilities to modify data items within
shell scripts.

Dale

Re: missing way to extract data out of data

[Greg Wooledge]

On Thu, Mar 18, 2021 at 02:01:12PM +0100, Alex fxmbsw7 Ratchev wrote:
> there is way to crop data to wanted, by cropping the exclulsions away
> but what about a way to extract data, eg @( .. ) match
> not about using [[ =~ but only var internal stuff
> .. or do i miss there something

This is really a help-bash question, not a bug-bash question.  If you
show us your input, and your desired output, we might be able to give
you some suggestions on how to write your script.

missing way to extract data out of data

[Alex fxmbsw7 Ratchev]

there is way to crop data to wanted, by cropping the exclulsions away
but what about a way to extract data, eg @( .. ) match
not about using [[ =~ but only var internal stuff
.. or do i miss there something