Re: When do I use perl, awk, sed, grep, cut etc.. Was[OT] get all devices from a vendor from pci.ids

[Javier Barroso]

Hello,

On Fri, Jan 6, 2017 at 10:46 PM, Floris  wrote:
>>> So every Debian user has the perl command?
>>
>> Not only Debian users, the vast majority of linux / unix users have
>> perl installed (maybe now that android is here, this statement is not
>> true any more ...
>>
>> With awk:
>> awk -v vendor=0e11 'p == 1 && /^[^[:space:]]/ { p=0; } $0 ~ "^"vendor"
>> " {p=1;} p' /usr/share/misc/pci.ids
>>
>> With sed:
>> sed -ne '/^0e11/p' -e '/^0e11/,/^[^[:space:]]/ { /^[^[:space:]]/d ; p
>> }'  /usr/share/misc/pci.ids
>>
>> Wrapping to script which get an argument is easy
>>
>> Regards
>>
>
> Thanks for your answer, and the next question pops in.
>
> Is there a "rule" when I use perl, awk, sed, grep, cut etc. in a script?
> Often I find multiple solutions on the Internet to achieve the same output.
>
> A simple example:
>
> cut -d: -f2
> or
> awk -F: '{ print $2 }'
>
> Is the one better as the other (CPU/RAM usage), or is it just "taste"

I'm pretty sure that there are tools to help meassure ram / cpu used
by a shell script and then present result (max / min / media ...), but
I don't know any one (maybe memusg from a quick google search). I
would limit memory though ulimit -m to the memory that I would have to
run the script.

If a tool will not be executed thousands of times is irrelevant if it
is using cut / awk or another shell basic command. Why would you spend
more time on an script to optimize it if it won't run many times?
Even, if it will be executed by cron task, It is not important if it
takes one second, or ten seconds. Of course if it is a public script
is important, you don't know how many times it will be executed, the
less time take the script , less time are lost by future users ...

For a simple cpu benchmark, sorted by real and user % cpu:

$ time for i in {1..1000} ; do ./test_awk > /dev/null ; done

real0m33,671s
user0m28,384s
sys 0m1,228s
$ time for i in {1..1000} ; do ./test_bash > /dev/null ; done

real0m29,404s
user0m26,708s
sys 0m1,344s
$ time for i in {1..1000} ; do ./test_sed > /dev/null ; done

real0m16,765s
user0m12,452s
sys 0m0,820s
$ time for i in {1..1000} ; do ./test_perl > /dev/null ; done

real0m15,932s
user0m11,344s
sys 0m0,564s

# test_bash:

id="${1:-0e11}"
while IFS=\0 read line
do
if [[ "$line" =~ ^$id[[:space:]].* ]]
then
echo "$line"
p=1
continue
fi
if [ "$p" = 1 ] && [[ "$line" =~ ^[[:space:]] ]]
then
echo "$line"
fi
if [[ "$line" =~ ^[^[:space:]] ]] && [ "$p" = 1 ]
then
exit 0
fi

done  < /usr/share/misc/pci.ids

Regards,

When do I use perl, awk, sed, grep, cut etc.. Was[OT] get all devices from a vendor from pci.ids

[Floris]

So every Debian user has the perl command?

Not only Debian users, the vast majority of linux / unix users have
perl installed (maybe now that android is here, this statement is not
true any more ...

With awk:
awk -v vendor=0e11 'p == 1 && /^[^[:space:]]/ { p=0; } $0 ~ "^"vendor"
" {p=1;} p' /usr/share/misc/pci.ids

With sed:
sed -ne '/^0e11/p' -e '/^0e11/,/^[^[:space:]]/ { /^[^[:space:]]/d ; p
}'  /usr/share/misc/pci.ids

Wrapping to script which get an argument is easy

Regards



Thanks for your answer, and the next question pops in.

Is there a "rule" when I use perl, awk, sed, grep, cut etc. in a script?
Often I find multiple solutions on the Internet to achieve the same output.

A simple example:

cut -d: -f2
or
awk -F: '{ print $2 }'

Is the one better as the other (CPU/RAM usage), or is it just "taste"

Floris

Re: [OT] get all devices from a vendor from pci.ids

[Javier Barroso]

Hello,

On Thu, Jan 5, 2017 at 9:13 PM, Floris  wrote:
> Op Thu, 05 Jan 2017 21:07:10 +0100 schreef Nicolas George :
>
>
>> Le sextidi 16 nivôse, an CCXXV, Floris a écrit :
>>>
>>> Thanks! But I prefer a solution with "essential" Debian software/
>>> packages
>>
>>
>> ~ $ dpkg -S =perl
>> perl-base: /usr/bin/perl
>> ~ $ dpkg -s perl-base | head -n 2
>> Package: perl-base
>> Essential: yes
>
>
> Sorry, I only looked at the perl package
> dpkg -s perl
> Package: perl
> Status: install ok installed
> Priority: standard
> Section: perl
>
> So every Debian user has the perl command?
Not only Debian users, the vast majority of linux / unix users have
perl installed (maybe now that android is here, this statement is not
true any more ...

With awk:
awk -v vendor=0e11 'p == 1 && /^[^[:space:]]/ { p=0; } $0 ~ "^"vendor"
" {p=1;} p' /usr/share/misc/pci.ids

With sed:
sed -ne '/^0e11/p' -e '/^0e11/,/^[^[:space:]]/ { /^[^[:space:]]/d ; p
}'  /usr/share/misc/pci.ids

Wrapping to script which get an argument is easy

Regards

Re: [OT] get all devices from a vendor from pci.ids

[Floris]

Op Thu, 05 Jan 2017 21:07:10 +0100 schreef Nicolas George  
:



Le sextidi 16 nivôse, an CCXXV, Floris a écrit :
Thanks! But I prefer a solution with "essential" Debian software/  
packages


~ $ dpkg -S =perl
perl-base: /usr/bin/perl
~ $ dpkg -s perl-base | head -n 2
Package: perl-base
Essential: yes


Sorry, I only looked at the perl package
dpkg -s perl
Package: perl
Status: install ok installed
Priority: standard
Section: perl

So every Debian user has the perl command?

Floris

Re: [OT] get all devices from a vendor from pci.ids

[Floris]

Op Thu, 05 Jan 2017 18:13:41 +0100 schreef Javier Barroso  
:



Hello,


On Thu, Jan 5, 2017 at 6:00 PM, Floris  wrote:

Op Wed, 04 Jan 2017 17:27:55 +0100 schreef Floris :


(Not really a Debian question, but I know there are smart people on  
this

list.)

I want to search the pci.ids file (from the pciutils package) for all
Compaq devices.
Pci ID's 0e11: , 1032: and 10da: matches Compaq.
Unfortunately, only at a beginning of a list the id of the vendor is  
printed
in column 1. The id of the devices are printed in column 2, with an  
empty

first column.

(part of pci.ids)
...
0e11  Compaq Computer Corporation
0001  PCI to EISA Bridge
0002  PCI to ISA Bridge
0046  Smart Array 64xx
0e11 4091  Smart Array 6i
0e11 409a  Smart Array 641
0e11 409b  Smart Array 642
0e11 409c  Smart Array 6400
0e11 409d  Smart Array 6400 EM
0049  NC7132 Gigabit Upgrade Module
004a  NC6136 Gigabit Server Adapter
...
c000  Remote Insight Lights-Out Edition
f130  NetFlex-3/P ThunderLAN 1.0
f150  NetFlex-3/P ThunderLAN 2.3
0e55  HaSoTec GmbH
...

Is there an awk, grep and or sed solution?

Floris



an other solution would be something like:

# cat pci.ids | {magic-command} 0e11:004a
NC6136 Gigabit Server Adapter

what is the magic command?


Using perl:

perl -ne 'print if (/^0e11/.../^\S/) and (/^\s/) or (/^0e11/);'
/usr/share/misc/pci.ids



Thanks! But I prefer a solution with "essential" Debian software/ packages

Floris

Re: [OT] get all devices from a vendor from pci.ids

[Nicolas George]

Le sextidi 16 nivôse, an CCXXV, Floris a écrit :
> Thanks! But I prefer a solution with "essential" Debian software/ packages

~ $ dpkg -S =perl
perl-base: /usr/bin/perl
~ $ dpkg -s perl-base | head -n 2
Package: perl-base
Essential: yes


signature.asc
Description: Digital signature

Re: [OT] get all devices from a vendor from pci.ids

[Javier Barroso]

Hello,


On Thu, Jan 5, 2017 at 6:00 PM, Floris  wrote:
> Op Wed, 04 Jan 2017 17:27:55 +0100 schreef Floris :
>
>
>> (Not really a Debian question, but I know there are smart people on this
>> list.)
>>
>> I want to search the pci.ids file (from the pciutils package) for all
>> Compaq devices.
>> Pci ID's 0e11: , 1032: and 10da: matches Compaq.
>> Unfortunately, only at a beginning of a list the id of the vendor is printed
>> in column 1. The id of the devices are printed in column 2, with an empty
>> first column.
>>
>> (part of pci.ids)
>> ...
>> 0e11  Compaq Computer Corporation
>> 0001  PCI to EISA Bridge
>> 0002  PCI to ISA Bridge
>> 0046  Smart Array 64xx
>> 0e11 4091  Smart Array 6i
>> 0e11 409a  Smart Array 641
>> 0e11 409b  Smart Array 642
>> 0e11 409c  Smart Array 6400
>> 0e11 409d  Smart Array 6400 EM
>> 0049  NC7132 Gigabit Upgrade Module
>> 004a  NC6136 Gigabit Server Adapter
>> ...
>> c000  Remote Insight Lights-Out Edition
>> f130  NetFlex-3/P ThunderLAN 1.0
>> f150  NetFlex-3/P ThunderLAN 2.3
>> 0e55  HaSoTec GmbH
>> ...
>>
>> Is there an awk, grep and or sed solution?
>>
>> Floris
>>
>
> an other solution would be something like:
>
> # cat pci.ids | {magic-command} 0e11:004a
> NC6136 Gigabit Server Adapter
>
> what is the magic command?

Using perl:

perl -ne 'print if (/^0e11/.../^\S/) and (/^\s/) or (/^0e11/);'
/usr/share/misc/pci.ids

Re: [OT] get all devices from a vendor from pci.ids

[Floris]

Op Wed, 04 Jan 2017 17:27:55 +0100 schreef Floris :

(Not really a Debian question, but I know there are smart people on this  
list.)


I want to search the pci.ids file (from the pciutils package) for all  
Compaq devices.
Pci ID's 0e11: , 1032: and 10da: matches Compaq.  
Unfortunately, only at a beginning of a list the id of the vendor is  
printed in column 1. The id of the devices are printed in column 2, with  
an empty first column.


(part of pci.ids)
...
0e11  Compaq Computer Corporation
0001  PCI to EISA Bridge
0002  PCI to ISA Bridge
0046  Smart Array 64xx
0e11 4091  Smart Array 6i
0e11 409a  Smart Array 641
0e11 409b  Smart Array 642
0e11 409c  Smart Array 6400
0e11 409d  Smart Array 6400 EM
0049  NC7132 Gigabit Upgrade Module
004a  NC6136 Gigabit Server Adapter
...
c000  Remote Insight Lights-Out Edition
f130  NetFlex-3/P ThunderLAN 1.0
f150  NetFlex-3/P ThunderLAN 2.3
0e55  HaSoTec GmbH
...

Is there an awk, grep and or sed solution?

Floris



an other solution would be something like:

# cat pci.ids | {magic-command} 0e11:004a
NC6136 Gigabit Server Adapter

what is the magic command?

Floris

[OT] get all devices from a vendor from pci.ids

[Floris]

(Not really a Debian question, but I know there are smart people on this  
list.)


I want to search the pci.ids file (from the pciutils package) for all  
Compaq devices.
Pci ID's 0e11: , 1032: and 10da: matches Compaq.  
Unfortunately, only at a beginning of a list the id of the vendor is  
printed in column 1. The id of the devices are printed in column 2, with  
an empty first column.


(part of pci.ids)
...
0e11  Compaq Computer Corporation
0001  PCI to EISA Bridge
0002  PCI to ISA Bridge
0046  Smart Array 64xx
0e11 4091  Smart Array 6i
0e11 409a  Smart Array 641
0e11 409b  Smart Array 642
0e11 409c  Smart Array 6400
0e11 409d  Smart Array 6400 EM
0049  NC7132 Gigabit Upgrade Module
004a  NC6136 Gigabit Server Adapter
...
c000  Remote Insight Lights-Out Edition
f130  NetFlex-3/P ThunderLAN 1.0
f150  NetFlex-3/P ThunderLAN 2.3
0e55  HaSoTec GmbH
...

Is there an awk, grep and or sed solution?

Floris