Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-20 Thread Jamie Wilkinson
This one time, at band camp, Alex Samad wrote: On Tue, Dec 18, 2007 at 11:57:39PM +1100, Jamie Wilkinson wrote: This one time, at band camp, Jeff Waugh wrote: quote who=Robert Thorsby sed 's#[^,]*##g' input.txt | tr -d '\n' | wc -m Something like the following might be close:

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-20 Thread Jamie Wilkinson
This one time, at band camp, Norman Gaywood wrote: On Wed, Dec 19, 2007 at 12:46:51PM +1100, Scott Ragen wrote: [EMAIL PROTECTED] wrote on 19/12/2007 11:34:30 AM: Norman Gaywood wrote: perl -00 -ne 'print tr/,//' input.txt I nominate the perl soln as the winner so far: runs like a bat

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-20 Thread Jamie Wilkinson
This one time, at band camp, Jeff Waugh wrote: quote who=Jamie Wilkinson sed 's#[^,]*##g' input.txt | tr -d '\n' | wc -m Tuesday afternoon shell optimisation party! You want to count the total number of characters in a file, not including newlines, that are on lines that don't start

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-20 Thread Jamie Wilkinson
This one time, at band camp, Ian Wienand wrote: I'd guess the Python version is spending that time doing some extra copying because it causes a lot of page faults is really cache unfriendly. Python Instructions retired per L1 data cache access: 11.03 Instructions retired per L2 data cache access:

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-20 Thread Ian Wienand
On Thu, Dec 20, 2007 at 09:22:49PM +1100, Jamie Wilkinson wrote: I love Ians posts only because he includes L1 and L2 cache hits in every one. If only you would share the command that gave you these numbers. The numbers come from the CPU performance counters. I use the perfmon tools [1] to

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-20 Thread Alex Samad
On Thu, Dec 20, 2007 at 09:18:06PM +1100, Jamie Wilkinson wrote: This one time, at band camp, Alex Samad wrote: On Tue, Dec 18, 2007 at 11:57:39PM +1100, Jamie Wilkinson wrote: This one time, at band camp, Jeff Waugh wrote: quote who=Robert Thorsby sed 's#[^,]*##g' input.txt | tr -d

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Peter Miller
On Tue, 2007-12-18 at 17:55 +1100, Peter Hardy wrote: I've got a sixpack of beer for a working PostScript variant. :-) drop this in a file called count.ps %!PS /count { 0 { currentfile read { (,) 0 get eq { 1 add } if } { 20 string cvs print (\n) print stop } ifelse }

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Sam Gentle
On Dec 18, 2007 5:21 PM, Peter Hardy [EMAIL PROTECTED] wrote: On Tue, 2007-12-18 at 16:09 +1100, Jeff Waugh wrote: Here's a starting point. What's a more optimal way to perform this task? :-) sed 's#[^,]*##g' input.txt | tr -d '\n' | wc -m Tuesday afternoon shell optimisation party!

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Jamie Wilkinson
This one time, at band camp, Jeff Waugh wrote: Hi all, Here's a starting point. What's a more optimal way to perform this task? :-) sed 's#[^,]*##g' input.txt | tr -d '\n' | wc -m Tuesday afternoon shell optimisation party! You want to count the total number of characters in a file, not

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Jamie Wilkinson
This one time, at band camp, Jeff Waugh wrote: quote who=Robert Thorsby sed 's#[^,]*##g' input.txt | tr -d '\n' | wc -m Something like the following might be close: awk 'BEGIN{FS=,}{$0~,$:i=i+NF?i=i+NF-1}END{print(i)}' input.txt Close in what sense, the syntax error, the length, or the

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Jamie Wilkinson
This one time, at band camp, Peter Miller wrote: Cascade Premium, please. Zing! -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Jeff Waugh
quote who=Jamie Wilkinson sed 's#[^,]*##g' input.txt | tr -d '\n' | wc -m Tuesday afternoon shell optimisation party! You want to count the total number of characters in a file, not including newlines, that are on lines that don't start with a comma. That's an... interesting... reading

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Rick Welykochy
Jeff Waugh wrote: Does it have to be in shell? :-) No sir! But shell usually wins. On my 1 GHz / 1 GB powerbook, the python one-liner I just submitted runs 5 x faster than the original. But what does 'shell' really mean? python,perl, etc. are external to the shell as surely as sed,tr,wc.

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Rick Welykochy
Jamie Wilkinson wrote: This one time, at band camp, Jeff Waugh wrote: Hi all, Here's a starting point. What's a more optimal way to perform this task? :-) sed 's#[^,]*##g' input.txt | tr -d '\n' | wc -m Tuesday afternoon shell optimisation party! You want to count the total number of

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Rick Welykochy
Jeff Waugh wrote: Traditionally it has been quick command line combos, but there are always offerings in more conventional and esoteric forms. Such as brainf*ck this time. I enjoyed the bf and postscript the best so far. Any INTERCAL skilz out there? cheers rickw --

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Jeff Waugh
quote who=Rick Welykochy Jeff Waugh wrote: Does it have to be in shell? :-) No sir! But shell usually wins. On my 1 GHz / 1 GB powerbook, the python one-liner I just submitted runs 5 x faster than the original. Ah yes, well there are different definitions of optimisation, and all are

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Andrew Bennetts
Rick Welykochy wrote: Jamie Wilkinson wrote: This one time, at band camp, Jeff Waugh wrote: Hi all, Here's a starting point. What's a more optimal way to perform this task? :-) sed 's#[^,]*##g' input.txt | tr -d '\n' | wc -m Tuesday afternoon shell optimisation party! You want to

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Rick Welykochy
Andrew Bennetts wrote: If you want to avoid reading the whole file into memory: python -Sc import sys; print sum(l.count(',') for l in sys.stdin) input.txt Yes. That's much better. Slurping the input into memory doesn't scale well. And they said TIMTOWTDI applied only to perl. cheers

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Alex Samad
On Tue, Dec 18, 2007 at 11:57:39PM +1100, Jamie Wilkinson wrote: This one time, at band camp, Jeff Waugh wrote: quote who=Robert Thorsby sed 's#[^,]*##g' input.txt | tr -d '\n' | wc -m Something like the following might be close: awk

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Norman Gaywood
On Dec 18, 2007 4:47 PM, Jeff Waugh [EMAIL PROTECTED] wrote: quote who=Martin Visser perl -e 'while(){$a+=s/[,]//g};print $a\n' input.txt Do I win?? Oddly, perl very rarely wins these. ;-) This must come close: perl -00 -ne 'print tr/,//' input.txt -- Norman Gaywood, Systems

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Rick Welykochy
Norman Gaywood wrote: Oddly, perl very rarely wins these. ;-) This must come close: perl -00 -ne 'print tr/,//' input.txt Timing test: say the above takes time 1.0 then the following takes time 0.46 ... python -Sc import sys; print sys.stdin.read().count(',') both are much much faster

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Scott Ragen
[EMAIL PROTECTED] wrote on 19/12/2007 11:34:30 AM: Norman Gaywood wrote: Oddly, perl very rarely wins these. ;-) This must come close: perl -00 -ne 'print tr/,//' input.txt Timing test: say the above takes time 1.0 then the following takes time 0.46 ... python -Sc import sys;

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Norman Gaywood
On Wed, Dec 19, 2007 at 12:46:51PM +1100, Scott Ragen wrote: [EMAIL PROTECTED] wrote on 19/12/2007 11:34:30 AM: Norman Gaywood wrote: perl -00 -ne 'print tr/,//' input.txt I nominate the perl soln as the winner so far: runs like a bat of out hell and is the most easy to understand.

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Rick Welykochy
Scott Ragen wrote: I have to disagree. Whilst it may be fast, its not 100% correct. Most of the time it would probably work, but if there are any blank lines, it outputs the current count, and starts again. Consider the following file contents: --file contents-- this,is,the,first,line

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Rick Welykochy
Norman Gaywood wrote: There is also the slightly shorter, tending to perl ugly instead of perl neat: perl -0777 -pe '$_=tr/,//' input.txt Let's get rid of one character: perl -0777 -pe '$_=y/,//' input.txt cheers rickw -- _ Rick Welykochy || Praxis

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Matthew Hannigan
On Wed, Dec 19, 2007 at 12:34:02AM +1100, Rick Welykochy wrote: Jeff Waugh wrote: Does it have to be in shell? :-) No sir! But shell usually wins. On my 1 GHz / 1 GB powerbook, the python one-liner I just submitted runs 5 x faster than the original. But what does 'shell' really mean?

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Ian Wienand
On Wed, Dec 19, 2007 at 12:34:02AM +1100, Rick Welykochy wrote: No sir! But shell usually wins. On my 1 GHz / 1 GB powerbook, the python one-liner I just submitted runs 5 x faster than the original. I think C usually wins, the version below is 25 times faster than the python version (from

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Matthew Hannigan
On Wed, Dec 19, 2007 at 02:50:55PM +1100, Ian Wienand wrote: [ C version ] Here's one in lex; ripped off from the flex info page. I'd be interested in its performance compared to straight C. No doubt worse, just curious how much worse. $ cat count.l cat count.l int num_commas = 0; %% ,

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-18 Thread Ian Wienand
On Wed, Dec 19, 2007 at 02:51:34PM +1100, Matthew Hannigan wrote: Here's one in lex; ripped off from the flex info page. I'd be interested in its performance compared to straight C. No doubt worse, just curious how much worse. Similar to the Python version [EMAIL PROTECTED]:/tmp$

[SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Jeff Waugh
Hi all, Here's a starting point. What's a more optimal way to perform this task? :-) sed 's#[^,]*##g' input.txt | tr -d '\n' | wc -m Tuesday afternoon shell optimisation party! Thanks, - Jeff -- linux.conf.au 2008: Melbourne, Australiahttp://lca2008.linux.org.au/ It will

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Martin Barry
$quoted_author = Jeff Waugh ; Here's a starting point. What's a more optimal way to perform this task? :-) the first question was what is the task trying to achieve? :) sed 's#[^,]*##g' input.txt | tr -d '\n' | wc -m you appear to be counting the number of fields in a csv file but

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Jeff Waugh
quote who=Martin Barry $quoted_author = Jeff Waugh ; Here's a starting point. What's a more optimal way to perform this task? :-) the first question was what is the task trying to achieve? :) That's part of the challenge. sed 's#[^,]*##g' input.txt | tr -d '\n' | wc -m you

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Martin Visser
perl -e 'while(){$a+=s/[,]//g};print $a\n' input.txt Do I win?? On Dec 18, 2007 4:09 PM, Jeff Waugh [EMAIL PROTECTED] wrote: Hi all, Here's a starting point. What's a more optimal way to perform this task? :-) sed 's#[^,]*##g' input.txt | tr -d '\n' | wc -m Tuesday afternoon shell

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Scott Ragen
[EMAIL PROTECTED] wrote on 18/12/2007 04:09:15 PM: Hi all, Here's a starting point. What's a more optimal way to perform this task? :-) sed 's#[^,]*##g' input.txt | tr -d '\n' | wc -m Not the most graceful, but the following seems to work: grep -o ',' input.txt |wc -l

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Robert Thorsby
On 18/12/07 16:32:03, Jeff Waugh wrote: That's part of the challenge. sed 's#[^,]*##g' input.txt | tr -d '\n' | wc -m Something like the following might be close: awk 'BEGIN{FS=,}{$0~,$:i=i+NF?i=i+NF-1}END{print(i)}' input.txt Robert Thorsby Old timers will tell you what a pain unstable

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Lindsay Holmwood
On Dec 18, 2007 4:35 PM, Martin Visser [EMAIL PROTECTED] wrote: perl -e 'while(){$a+=s/[,]//g};print $a\n' input.txt Do I win?? No. Ironically, your solution is 3 characters longer. :-) Lindsay -- http://slug.org.au/ (the Sydney Linux Users Group) http://holmwood.id.au/~lindsay/ (me) --

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Jeff Waugh
quote who=Martin Visser perl -e 'while(){$a+=s/[,]//g};print $a\n' input.txt Do I win?? Oddly, perl very rarely wins these. ;-) - Jeff -- linux.conf.au 2008: Melbourne, Australiahttp://lca2008.linux.org.au/ Odd is good by the way. I knew normal in high school and normal

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread James Gray
On 18/12/2007, at 4:42 PM, Scott Ragen wrote: [EMAIL PROTECTED] wrote on 18/12/2007 04:09:15 PM: Hi all, Here's a starting point. What's a more optimal way to perform this task? :-) sed 's#[^,]*##g' input.txt | tr -d '\n' | wc -m Not the most graceful, but the following seems to

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Jeff Waugh
quote who=Scott Ragen grep -o ',' input.txt |wc -l Oh nice, grep -o! Clever! :-) - Jeff -- GNOME.conf.au 2008: Melbourne, Australia http://live.gnome.org/Melbourne2008 People keep asking me why we aren't married, and he says, 'Every time I am about to ask you, you do

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Jeff Waugh
quote who=Robert Thorsby sed 's#[^,]*##g' input.txt | tr -d '\n' | wc -m Something like the following might be close: awk 'BEGIN{FS=,}{$0~,$:i=i+NF?i=i+NF-1}END{print(i)}' input.txt Close in what sense, the syntax error, the length, or the output? ;-) - Jeff -- GNOME.conf.au 2008:

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Robert Thorsby
On 18/12/07 16:45:18, Robert Thorsby wrote: Something like the following might be close: awk 'BEGIN{FS=,}{$0~,$:i=i+NF?i=i+NF-1}END{print(i)}' input.txt Oops, I transposed the : and the ? in the conditional. Just shows what you can do when fingers outpace brain. Robert Thorsby Research is

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Jeff Waugh
quote who=James Gray Not the most graceful, but the following seems to work: grep -o ',' input.txt |wc -l Assuming we're using GNU grep we can leave the pipe off: grep -c -o ',' input.txt Hmm, unfortunately the -c misinterprets the count due to a weird interaction between -c and

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Lindsay Holmwood
On Dec 18, 2007 4:53 PM, Jeff Waugh [EMAIL PROTECTED] wrote: quote who=James Gray Not the most graceful, but the following seems to work: grep -o ',' input.txt |wc -l Assuming we're using GNU grep we can leave the pipe off: grep -c -o ',' input.txt Hmm, unfortunately the

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Robert Thorsby
On 18/12/07 16:50:17, Jeff Waugh wrote: Something like the following might be close: awk 'BEGIN{FS=,}{$0~,$:i=i+NF?i=i+NF-1}END{print(i)}' input.txt Close in what sense, the syntax error, the length, or the output? ;-) - Jeff Syntax error granted -- just keeping you on your toes :-)

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Jeff Waugh
quote who=Lindsay Holmwood Not a bug at all! grep -c only counts the number of matching lines, not the number of occurances of a pattern in a line. But when you stick -o in there... Hrrrmmm... I even switched their position on the command line to see if that changed the output. Total cargo

RE: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Roger Barnes
Here's a starting point. What's a more optimal way to perform this task? :-) sed 's#[^,]*##g' input.txt | tr -d '\n' | wc -m For starters, remount the partition containing input.txt with the noatime option and disable trackerd. :) Then, change the '*' to a '\+' in your regex. This saved

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Peter Hardy
On Tue, 2007-12-18 at 16:09 +1100, Jeff Waugh wrote: Here's a starting point. What's a more optimal way to perform this task? :-) sed 's#[^,]*##g' input.txt | tr -d '\n' | wc -m Tuesday afternoon shell optimisation party! How do you want it optimised? grep -o is the most readable. But

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Lindsay Holmwood
On Dec 18, 2007 4:35 PM, Martin Visser [EMAIL PROTECTED] wrote: perl -e 'while(){$a+=s/[,]//g};print $a\n' input.txt Ruby version: ruby -e p IO.read('input.txt').count(',') Lindsay -- http://slug.org.au/ (the Sydney Linux Users Group) http://holmwood.id.au/~lindsay/ (me) -- SLUG - Sydney

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Scott Ragen
[EMAIL PROTECTED] wrote on 18/12/2007 05:21:35 PM: On Tue, 2007-12-18 at 16:09 +1100, Jeff Waugh wrote: Here's a starting point. What's a more optimal way to perform thistask? :-) sed 's#[^,]*##g' input.txt | tr -d '\n' | wc -m Tuesday afternoon shell optimisation party! How do

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Peter Hardy
On Tue, 2007-12-18 at 17:05 +1100, Lindsay Holmwood wrote: On Dec 18, 2007 4:35 PM, Martin Visser [EMAIL PROTECTED] wrote: perl -e 'while(){$a+=s/[,]//g};print $a\n' input.txt Ruby version: I've got a sixpack of beer for a working PostScript variant. :-) -- Pete -- SLUG - Sydney

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Jeff Waugh
quote who=Peter Hardy How do you want it optimised? It doesn't matter either way -- almost all claims in these threads are educational in some form or another! :-) - Jeff -- GNOME.conf.au 2008: Melbourne, Australia http://live.gnome.org/Melbourne2008 Blessed are the cracked, for they

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Robert Thorsby
On 18/12/07 17:33:25, Scott Ragen wrote: This seems to work too: cat input.txt |tr -dC ',' |wc -c Use redirection to eliminate cat tr -dC ',' input.txt | wc -c Robert Thorsby Let me know if you don't receive this message. -- email signature tagline -- SLUG - Sydney Linux

Re: [SLUG] Tuesday afternoon shell command optimisation party!

2007-12-17 Thread Peter Chubb
tr -dc ',' input.txt | wc -c will count the number of commas in the input file. Peter C -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html