Re: [asterisk-users] Benchmarking AGI performance in C, PHP, and Perl
Kevin P. Fleming kpflem...@digium.com writes: OT: Take a look at 'systemd'; this is exactly what's happening there, and Fedora is likely to incorporate it into Fedora 16, and it will make its way into other distros after that. It was incorporated into Fedora 14, and it is the default in Fedora 15... /Benny (And yes it meant I couldn't boot after upgrading to Fedora 15. It couldn't handle that I had the cgroup file system mounted on /cgroup in fstab.) -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Benchmarking AGI performance in C, PHP, and Perl
On Mon, Jul 11, 2011 at 02:29:25PM -0700, Steve Edwards wrote: The second AGI, 'neutered-agi' is an AGI of 'production length' (around 1,600 lines) and supporting access to a MySQL database. The AGI is of 'production length' but still exits after reading the AGI environment variables because we are measuring program startup time. Perl's startup time also depends on the modules you load: The following is a simple loop that runs a perl program doing nothing (executing an empty statement) but firt loading a different set of modules for the task. The loop repeats it 1000 times: # No extra modules: $ time for i in `seq 1000`; do perl -e ''; done real0m5.093s user0m1.676s sys 0m2.524s # Asterisk::AGI: $ time for i in `seq 1000`; do perl -MAsterisk::AGI -e ''; done real0m9.400s user0m6.368s sys 0m2.060s # MySQL access: $ time for i in `seq 1000`; do perl -MDBI -MDBD::mysql -e ''; done real0m40.964s user0m33.122s sys 0m5.416s # Asterisk::AGI and MySQL access: $ time for i in `seq 1000`; do perl -MAsterisk::AGI -MDBI -MDBD::mysql -e ''; done real0m45.898s user0m36.798s sys 0m6.048s -- Tzafrir Cohen icq#16849755 jabber:tzafrir.co...@xorcom.com +972-50-7952406 mailto:tzafrir.co...@xorcom.com http://www.xorcom.com iax:gu...@local.xorcom.com/tzafrir -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Benchmarking AGI performance in C, PHP, and Perl
Steve Edwards wrote:
Also they tend to be used more by 'non-programmers' who get away with
'stupid' stuff like calling out to system() and piping a bunch of
commands together because they don't know how to use the language
properly :)
I'm not disparaging Perl programmers or the language. I'm just saying
it is easier to 'abuse' a language like Perl or PHP than C.
This Bash snippet was posted to the -users list a couple of years ago.
I'm not trying to embarrass the original programmer or trash his
skills, I'm just using this snippet as an example. We've all got
skeletons in our closets :)
while read line; do
epoch=`echo $line | cut -d '|' -f 1`
if [ $epoch -ge $start_epoch -a $epoch -le $end_epoch ]; then
echo $line
fi
done /var/log/asterisk/queue_log
Note the second line. It creates a couple (2 or 3) of processes to
extract the first field from the queue_log file. For every line in
the input file!
This kind of coding is easy in a scripting language. It would be way
more difficult in C. So much more difficult, a programmer with the
requisite skills to do it should recognize the inefficiency and do
it another way.
If the original programmer had a better grasp of the language, he
could have coded this line as:
epoch=${line:0:10}
Since the Epoch will be 10 digits for the next 300 years, I'd feel
relatively comfortable with this solution.
This single change reduced the execution time of his script by an
order of magnitude.
Recoding it in a language more appropriate to processing lots of data
(like C) would reduce the execution time to 1/3,000th of the original. And
yes, I did it and measured it.
A skilled programmer (like any craftsman) has many tools in his toolbox,
the experience to choose the right one, and the skill to use it well.
C is my sharpest tool so I tend to see everything through that lens, but
I'm learning to appreciate PHP and how it lets me represent some
programming problems clearly, quickly and sufficiently efficient.
Steve,
I recognized the code you posted. It's mine:
http://lists.digium.com/pipermail/asterisk-users/2009-September/237750.html
Thank goodness you didn't try to embarrass me. You just used my code as an
example of how a non-programmer would use a language, called piping commands
together stupid and an indication of not knowing how to use the language,
referred to it as a skeleton in my closet and something that a programmer
with the requisites skills and a better grasp of the language wouldn't do.
I guess I am just not a skilled enough programmer to try to help someone out
on this list without getting criticized by you at the time
http://lists.digium.com/pipermail/asterisk-users/2009-September/237755.html
and again almost two years later.
Maybe, just maybe, it's possible that some people recognize that your crusade
to replace all scripting with compiled C has its virtues, but it's not worth
their programming time to save a few seconds of execution time. You've been
at this for years and yet scripting languages still exist.
Just think how fast Linux would boot if all of the init scripts were
rewritten in C and compiled (they probably have some pipes that could be
removed, too!!). Of course, it's pretty nice to be able to easily read and
modify them, but execution time is all that's important, right?
Your example reduced the execution time of my script by a factor of 2,700!!
That's great, but do you really think if I posted C source code as a response
(I could if I wanted to) to the original poster that she would have had any
idea what to do with it? I'd wager that she'd spend more than the minute and
a half your optimization saved pondering it before responding, What do I do
with this? Instead, she copy-and-pasted my code, ran it, said thanks (can
you believe she didn't even mention that pipe I used!!), and went on to the
next thing. So which is really more efficient?
Not to mention that your optimization is technically incorrect because it
assumes the field is a fixed length. I'll admit, it's a valid assumption,
but we're arguing technicalities here. I wonder if you'll be able to admit
that sometimes optimizing a one-off script just isn't worth it?
My code also has the benefit of being self-documenting. Anyone can look at
it and understand exactly what it does (quite useful for a mailing list,
don't you think?). On the other hand, most people will be headed straight
to the Advanced Bash Scripting Guide when they see epoch=${line:0:10}.
I haven't posted here in a long time and I usually make it a policy not to
get involved in or start any flame wars. Furthermore, I've always thought
that your posts are intelligent and show a real understanding of the subject
matter and a technical prowess. But you're out of line using other people's
code as examples of bad programming techniques. Unless my code is in a
directory on your desktop labeled Bad (but in NO Way
Re: [asterisk-users] Benchmarking AGI performance in C, PHP, and Perl
Let's make this a Spider-man contest. The No-prize will be the satisfaction of seeing how it actually works. Write stevestest.agi in Perl, PHP and C. The program must load the AGI variables, do a MYSQL QUERY and a MYSQL INSERT. Post your source and results using this methodology: time for i in `seq 1000`; do ./stevestest.out ; done - C time for i in `seq 1000`; do php stevesetest.php '; done - PHP time for i in `seq 1000`; do perl stevestest.pl ; done Any takers? -Original Message- From: asterisk-users-boun...@lists.digium.com [mailto:asterisk-users-boun...@lists.digium.com] On Behalf Of Matt Riddell Sent: Monday, July 11, 2011 5:07 PM To: Asterisk Users Mailing List - Non-Commercial Discussion Subject: Re: [asterisk-users] Benchmarking AGI performance in C, PHP, and Perl On 12/07/11 9:29 AM, Steve Edwards wrote: Many times, I've made the statement that you can execute hundreds of AGIs written in C in the time it takes to load an interpreter and parse a script written in PHP or Perl. It would be interesting to see the same types of tests run against fast-agi - personally if I write an agi that will be called 1000 times I'm going to leave it running and have network requests against it rather than starting and stopping every time. Interesting tests nonetheless - I would have been pretty concerned if straight C wasn't faster :-) Mind if I post it to the Daily Asterisk News? -- Cheers, Matt Riddell ___ http://www.venturevoip.com/news.php (Daily Asterisk News) http://www.venturevoip.com/exchange.php (Full ITSP Solution) http://www.venturevoip.com/cc.php (Call Centre Solutions) -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Benchmarking AGI performance in C, PHP, and Perl
On 07/12/2011 09:33 AM, Matthew J. Roth wrote: Just think how fast Linux would boot if all of the init scripts were rewritten in C and compiled (they probably have some pipes that could be removed, too!!). Of course, it's pretty nice to be able to easily read and modify them, but execution time is all that's important, right? OT: Take a look at 'systemd'; this is exactly what's happening there, and Fedora is likely to incorporate it into Fedora 16, and it will make its way into other distros after that. -- Kevin P. Fleming Digium, Inc. | Director of Software Technologies Jabber: kflem...@digium.com | SIP: kpflem...@digium.com | Skype: kpfleming 445 Jan Davis Drive NW - Huntsville, AL 35806 - USA Check us out at www.digium.com www.asterisk.org -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Benchmarking AGI performance in C, PHP, and Perl
On Tue, Jul 12, 2011 at 10:06:12AM -0500, Kevin P. Fleming wrote: On 07/12/2011 09:33 AM, Matthew J. Roth wrote: Just think how fast Linux would boot if all of the init scripts were rewritten in C and compiled (they probably have some pipes that could be removed, too!!). Of course, it's pretty nice to be able to easily read and modify them, but execution time is all that's important, right? OT: Take a look at 'systemd'; this is exactly what's happening there, and Fedora is likely to incorporate it into Fedora 16, and it will make its way into other distros after that. Well, there are a number of separate optimizations in systemd: 1. Delayed loading of services (or even not loading them at all, if not needed. E.g.: don't load CUPS if nobody needs it. 2. Paralelized loading of services (though there have been other implemtnations of that. SuSE has had that for years: insserv). 3. Rewriting some scripts in C. Also note that udev has been going through this for quite some time as well. 4. A nice facility to get the timing information and show where the actual bottlenecks are. -- Tzafrir Cohen icq#16849755 jabber:tzafrir.co...@xorcom.com +972-50-7952406 mailto:tzafrir.co...@xorcom.com http://www.xorcom.com iax:gu...@local.xorcom.com/tzafrir -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Benchmarking AGI performance in C, PHP, and Perl
On Tue, 12 Jul 2011, Matthew J. Roth wrote: Just think how fast Linux would boot if all of the init scripts were rewritten in C and compiled (they probably have some pipes that could be removed, too!!). Of course, it's pretty nice to be able to easily read and modify them, but execution time is all that's important, right? Very probably the wrong place to duscuss this, however... I build embedded linux boxes for a variety of purposes (including asterisk), and I have to say that boot-time isn't always a factor. Really, when you have a box with an uptime of: # uptime 22:22:21 up 1374 days, 2:42, 2 users, load average: 0.00, 0.02, 0.00 who really cares how long it take to boot. The Linux kernel itself can boot in under a second - trouble is, it can take 15-20 seconds going through a PCs BIOS to get there, then it's init scripts, loading modules, etc. A lot of these are waiting on hardware, or the network coming up and compiling them isn't going to help much... In properly embedded systems you can get to application very fast (no init/scripts!), however in more general purpose systems, I feel it's diminishing returns time. I've also moved from sysv-rc to file-rc too - which completely serialises scripts - which might seem a backwards step, but it gives easier control and I'm happy that my PBXs go from cold to ready in under a minute, and my (getting old now) Acer Aspire One goes from cold to workable GUI in 45 seconds. I can live with that. Gordon -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Benchmarking AGI performance in C, PHP, and Perl
Tzafrir Cohen wrote: Well, there are a number of separate optimizations in systemd: 1. Delayed loading of services (or even not loading them at all, if not needed. E.g.: don't load CUPS if nobody needs it. 2. Paralelized loading of services (though there have been other implemtnations of that. SuSE has had that for years: insserv). 3. Rewriting some scripts in C. Also note that udev has been going through this for quite some time as well. 4. A nice facility to get the timing information and show where the actual bottlenecks are. Tzafrir, I kind of suspected that the init scripts weren't a great example because of systemd. It's already in Fedora 15, but I have to plead ignorance to its implementation. I thought it still used text-based service files, but I'm only familiar with it from reading the large number of posts it has generated on the Fedora users list. That's actually started to settle down and Gnome 3 has emerged as the new whipping boy. Since systemd has been proposed as a dependency of Gnome 3, this has the potential to create singularity that will destroy the universe. ; ) Personally, I'm not really excited about any of the optimizations that you listed. I can manage which services are started and the Linux boot sequence is pretty fast already. It's far quicker than the serial initialization of hardware that our servers go through at each boot. So for my use case, it can still be cited as an example of optimizing the wrong thing. In the long run, it's likely to save me very little time compared to the amount of time spent learning the new system. I'm sure it has its benefits and I'll come to appreciate them over time, but SysV init ain't broke (for me) so I'm not looking forward to fixing it. Regards, Matthew Roth InterMedia Marketing Solutions Software Engineer and Systems Developer -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Benchmarking AGI performance in C, PHP, and Perl
On Tue, 12 Jul 2011, Matthew J. Roth wrote:
I recognized the code you posted. It's mine:
Thank goodness you didn't try to embarrass me.
Thank you for acknowledging that it was not my intent.
You just used my code as an example of how a non-programmer would use
a language, called piping commands together stupid and an indication
of not knowing how to use the language, referred to it as a skeleton in
my closet and something that a programmer with the requisites skills and
a better grasp of the language wouldn't do.
Which part is not true? I did not say you were stupid, just that 1 line of
code is. Would you be more comfortable with 'ignorant?' After reading the
definitions at www.dictionary.com I think it may be more accurate and less
prone to misinterpretation but I don't know how to say a line of code is
ignorant.
I think you're being overly sensitive here.
I guess I am just not a skilled enough programmer to try to help someone
out on this list without getting criticized by you at the time
http://lists.digium.com/pipermail/asterisk-users/2009-September/237755.html
and again almost two years later.
A man that has caught his first fish can teach the man with an empty net.
You've been harboring this perceived offense in silence for 2 years? I've
re-read that post several times today and cannot see where I criticized
you or your code. Please point it out for me.
Maybe, just maybe, it's possible that some people recognize that your
crusade to replace all scripting with compiled C has its virtues, but
it's not worth their programming time to save a few seconds of execution
time. You've been at this for years and yet scripting languages still
exist.
Both compiled and scripting languages have their place. I have a box
wrench and a ratchet socket even though they both do kind of the same
thing.
I do a fair bit of scripting, just not on tasks that are repeated 500,000
times a day.
Just think how fast Linux would boot if all of the init scripts were
rewritten in C and compiled (they probably have some pipes that could be
removed, too!!). Of course, it's pretty nice to be able to easily read
and modify them, but execution time is all that's important, right?
If I rebooted my box 500,000 times a day, that may be relevant. My box
does execute 500,000 AGIs a day.
If I was playing the same MP3 500,000 times a day and complaining about
the performance might you suggest I could (or should) transcode the file
to SLIN, ULAW, PCM, etc?
On the other hand, most people will be headed straight to the Advanced
Bash Scripting Guide when they see epoch=${line:0:10}.
I consider that a win-win. I have fed them for the day and inspired them
to learn to fish better. Two birds, one stone in my book.
I replied to your code post 2 years ago because you warned the original
recipient that 'it may take a while to complete.'
To me, that's a call to action for a programmer or any kind of engineer.
Part of the reason I feel compelled to post 'improvements' is to 'pay it
forward' for the programmers that help me learn my craft. Also, if I let
it be, future readers will happily (ignorantly?) implement this construct
for eternity.
I'm reminded of one of my favorite jokes.
A husband is observing his wife is preparing their first holiday meal. She
takes the ham out of the fridge, sets it on the block, hacks off the last
4 inches and tosses the piece in the trash.
The husband, cognizant of his ignorance in 'the ways of the kitchen' asks
her why she did this.
She stops, looks quizzically at the ceiling and says I don't know. I do
it because my mom always did it.
The husband calls her mom. Her explanation is the same 'I don't know. I do
it because my mom always did it.
The husband calls the grandmother. Her explanation is 'I don't know why
they do that. I do it because my oven is too small.'
Almost a year ago, Anthony Messina posted a link to his fax gateway script
(now at http://messinet.com/trac/wiki/AsteriskFAXGateway). My hubris
deluded me to think I could make it better. I got 'skooled!'
I made a list of about 20 constructs I had no clue what they did and
headed out to the 'Advanced Bash-Scripting Guide'
(http://tldp.org/LDP/abs/html/)
I've incorporated some of what I learned and my Bash scripting is much
better for it.
I always try to read Tzafrir Cohen's posts because it's obvious he knows
way more than I do about Unix at a systems level.
Ditto for (in alphabetical order) Gordon Henderson, Kevin P. Fleming,
Steve Underwood, Tilghman Lesher, Tony Mountifield and many others. We sit
at the feet of giants.
But you're out of line using other people's code as examples of bad
programming techniques. Unless my code is in a directory on your desktop
labeled Bad (but in NO Way Embarrassing) Programming Examples it would
have been more efficient (see what I did there?) to come up with
something on your own.
My sincere apologies. I did not
Re: [asterisk-users] Benchmarking AGI performance in C, PHP, and Perl
Steve, Apology accepted. As I said in the original post, I hold you in high regard so your criticism was hard to take. I still think that the trade- off between readability and optimization is up for debate, but it's certainly nothing to hold a grudge over. I can tell you one thing for certain: I will think of you and that Bash substring-extraction construct every time I even consider piping something to cut. I might even remember that particular syntax without resorting to the ABS. I'm not making any promises about the others, though. Some of them are just nasty. ; ) http://tldp.org/LDP/abs/html/refcards.html#AEN22429 Regards, Matthew Roth InterMedia Marketing Solutions Software Engineer and Systems Developer -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Benchmarking AGI performance in C, PHP, and Perl
On Mon, Jul 11, 2011 at 06:45:08PM -0700, Steve Edwards wrote:
Also they tend to be used more by 'non-programmers' who get away with
'stupid' stuff like calling out to system() and piping a bunch of
commands together because they don't know how to use the language
properly :)
On Mon, 11 Jul 2011, cbul...@gmail.com wrote:
I understand your point but I don't share it There are a lot
Asterisk-Perl project working in production environment.
Then I didn't communicate my point clearly.
I'm not disparaging Perl programmers or the language. I'm just saying it
is easier to 'abuse' a language like Perl or PHP than C.
This Bash snippet was posted to the -users list a couple of years ago.
I'm not trying to embarrass the original programmer or trash his skills,
I'm just using this snippet as an example. We've all got skeletons in our
closets :)
while read line; do
epoch=`echo $line | cut -d '|' -f 1`
if [ $epoch -ge $start_epoch -a $epoch -le $end_epoch ]; then
echo $line
fi
done /var/log/asterisk/queue_log
Note the second line. It creates a couple (2 or 3) of processes to
extract the first field from the queue_log file. For every line in the
input file!
This kind of coding is easy in a scripting language. It would be way more
difficult in C. So much more difficult, a programmer with the requisite
skills to do it should recognize the inefficiency and do it another way.
If the original programmer had a better grasp of the language, he could
have coded this line as:
epoch=${line:0:10}
Since the Epoch will be 10 digits for the next 300 years, I'd feel
relatively comfortable with this solution.
For the record, I suspect you meant:
epoch=${line%%|*}
This actually does exactly what is written on the above command (without
having to assume the length of the field), and also does not use
bash-specific syntax and hence can be used with a faster shell like
dash (which Debian and Ubuntu have as /bin/sh by default).
This single change reduced the execution time of his script by an order
of magnitude.
Recoding it in a language more appropriate to processing lots of data
(like C) would reduce the execution time to 1/3,000th of the original.
And yes, I did it and measured it.
Actually, that mistake is easy to make in shell scripts. But even simple
script languages such as awk fare much better here:
awk -F'|' {
epoch=\$1;
if (epoch = $start_epoch epoch = $end_epoch) {
print epoch
}
} /var/log/asterisk/queue_log
How much time will it take you to write something as fast as this in C?
--
Tzafrir Cohen
icq#16849755 jabber:tzafrir.co...@xorcom.com
+972-50-7952406 mailto:tzafrir.co...@xorcom.com
http://www.xorcom.com iax:gu...@local.xorcom.com/tzafrir
--
_
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
New to Asterisk? Join us for a live introductory webinar every Thurs:
http://www.asterisk.org/hello
asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Benchmarking AGI performance in C, PHP, and Perl
On Mon, Jul 11, 2011 at 06:45:08PM -0700, Steve Edwards wrote:
while read line; do
epoch=`echo $line | cut -d '|' -f 1`
if [ $epoch -ge $start_epoch -a $epoch -le $end_epoch ]; then
echo $line
fi
done /var/log/asterisk/queue_log
[snipping snippy comments about improving the second line]
epoch=${line:0:10}
Since the Epoch will be 10 digits for the next 300 years, I'd feel
relatively comfortable with this solution.
On Wed, 13 Jul 2011, Tzafrir Cohen wrote:
For the record, I suspect you meant:
epoch=${line%%|*}
I didn't mean it, but I like it. Thanks.
Actually, that mistake is easy to make in shell scripts. But even simple
script languages such as awk fare much better here:
awk -F'|' {
epoch=\$1;
if (epoch = $start_epoch epoch = $end_epoch) {
print epoch
}
} /var/log/asterisk/queue_log
1) The 2 conditionals should be swapped.
2) I think you meant 'print \$0' instead of 'print epoch'
We'll leave defining start_epoch and end_epoch as an exercise for the
reader :)
How much time will it take you to write something as fast as this in C?
Well, the C version was still 10 times faster, but I concede it took a
whole lot longer to write.
The question always comes down to how many times are you going to use it
and how much will it cost to make it faster.
--
Thanks in advance,
-
Steve Edwards sedwa...@sedwards.com Voice: +1-760-468-3867 PST
Newline Fax: +1-760-731-3000
--
_
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
New to Asterisk? Join us for a live introductory webinar every Thurs:
http://www.asterisk.org/hello
asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users
[asterisk-users] Benchmarking AGI performance in C, PHP, and Perl
Many times, I've made the statement that you can execute hundreds of AGIs written in C in the time it takes to load an interpreter and parse a script written in PHP or Perl. Recently, a Doubting Thomas asked me to substantiate my claim. I suspect nobody has made the effort to implement an AGI of any reasonable size and function in multiple languages. I'm guessing it may not really be all that important and the results would be too task specific to be relevant. I suspect once an AGI is executing, the choice of source language is unimportant. I'll go out on a limb and say executing: select prompt_path from foo where bar; stream file prompt_path 1234* stream file you_entered say digits selected will execute in effectively the same time regardless of source language. Waiting for Asterisk to play a file or for your database to return a row is beyond the scope of your AGI. It's what you do between your AGI and database calls that will determine how much your choice of source language will impact the total execution time. Unless you're doing a lot of stuff in between these API calls, the only place you can make an impact is getting your code into memory and ready to execute. I 'wrote' 2 different AGIs in C, PHP, and Perl. The first AGI, 'null-agi' reads the AGI environment variables from STDIN and exits. To me, this is the bare minimum a program can do and call itself an AGI. Each AGI was less than 10 lines. The second AGI, 'neutered-agi' is an AGI of 'production length' (around 1,600 lines) and supporting access to a MySQL database. The AGI is of 'production length' but still exits after reading the AGI environment variables because we are measuring program startup time. For both AGIs, the C implementation used an AGI library I developed way too many years ago. The PHP implementation used PHPAGI. The Perl implementation used Asterisk::AGI. The C version of neutered-agi was based on a 'voicemail-like' AGI I wrote many years ago that stored the user credentials and messages in MySQL. The PHP version of neutered-agi was based on dialparties.agi (nicked from PIAF). dialparties.agi is only about 800 lines long, so I 'doubled it' by copying and pasting it into the same source file. While dialparties.agi does not use MySQL, MySQL is available in PHP without including additional header or class files. The Perl version of neutered-agi was based on agi-VDAD_ALL_outbound.agi (nicked from Vicidial). I ran the tests in 3 different environments: |---+++--| | CPU | RAM| CentOS | Asterisk | |---+++--| | Geode 500MHz | 256 MB |4.9 | 1.2.37 | | Atom D525 1.80GHz | 4 GB |5.6 | 1.8.4.1 | | Xeon 3.40GHz | 2 GB |4.8 | 1.2.40 | |---+++--| No swapping occurred during the tests. My dialplan executed each AGI 1,000 times to make the cumulative execution time more measurable. I wrote the dialplan in both 'inline dialplan' and an AEL 'for' loop. The initial execution times were the same so the test runs were made with the AEL version because it is more manageable. (5 lines versus 1,000 lines.) Here's the results for executing each AGI 1,000 times on each host in seconds: Geode: |--+--+--| | language | null-agi | neutered-agi | |--+--+--| | C|6 |6 | | PHP | 116 | 160 | | Perl | 99 | 639 | |--+--+--| Atom: |--+--+--| | language | null-agi | neutered-agi | |--+--+--| | C|6 |6 | | PHP | 52 | 65 | | Perl | 38 | 197 | |--+--+--| Xeon: |--+--+--| | language | null-agi | neutered-agi | |--+--+--| | C|2 |2 | | PHP | 40 | 47 | | Perl | 10 | 107 | |--+--+--| Summary: Geode - Perl / C: 106 Atom - Perl / C: 33 Xeon - Perl / C: 54 The C null-agi AGI was statically linked. I didn't have all the libraries needed to statically link neutered-agi on these boxes, but the dynamically linked versions of null-agi and neutered-agi took the same time to execute (16 seconds on the Geode) so I'm assuming statically linked versions of null-agi and neutered-agi would also take the same time to execute. It also helps support my original statement :) I guessing the Perl version of neutered-agi took a big hit from having to load the database code as well as the AGI framework (use DBI; use Asterisk::AGI;) while PHP only had to load the AGI framework (require_once phpagi.php;). I'll let you decide if the methodology is meaningful to your
Re: [asterisk-users] Benchmarking AGI performance in C, PHP, and Perl
On 12/07/11 9:29 AM, Steve Edwards wrote: Many times, I've made the statement that you can execute hundreds of AGIs written in C in the time it takes to load an interpreter and parse a script written in PHP or Perl. It would be interesting to see the same types of tests run against fast-agi - personally if I write an agi that will be called 1000 times I'm going to leave it running and have network requests against it rather than starting and stopping every time. Interesting tests nonetheless - I would have been pretty concerned if straight C wasn't faster :-) Mind if I post it to the Daily Asterisk News? -- Cheers, Matt Riddell ___ http://www.venturevoip.com/news.php (Daily Asterisk News) http://www.venturevoip.com/exchange.php (Full ITSP Solution) http://www.venturevoip.com/cc.php (Call Centre Solutions) -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Benchmarking AGI performance in C, PHP, and Perl
On 12/07/11 9:29 AM, Steve Edwards wrote: Many times, I've made the statement that you can execute hundreds of AGIs written in C in the time it takes to load an interpreter and parse a script written in PHP or Perl. Well, now that I know better, let's not perpetuate an ancient claim. 'Dozens' is more appropriate with current hardware. On Tue, 12 Jul 2011, Matt Riddell wrote: It would be interesting to see the same types of tests run against fast-agi - personally if I write an agi that will be called 1000 times I'm going to leave it running and have network requests against it rather than starting and stopping every time. Isn't every AGI going to be executed 1,000s of times over it's lifetime? 'Standalone' AGIs still have advantages in lower complexity and less impact on failure. If a bug takes out your fastagi daemon it can affect all calls. Unless the bits between the AGI and DB calls are significant, there should be no significant difference between source languages in a fastagi environment. One of my other objections to scripting languages are that they don't 'catch' stupid errors for me (like a misspelled variable name) as well as compiled languages. 'gcc -Wall' is my friend. Also they tend to be used more by 'non-programmers' who get away with 'stupid' stuff like calling out to system() and piping a bunch of commands together because they don't know how to use the language properly :) Mind if I post it to the Daily Asterisk News? You have my permission. -- Thanks in advance, - Steve Edwards sedwa...@sedwards.com Voice: +1-760-468-3867 PST Newline Fax: +1-760-731-3000 -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Benchmarking AGI performance in C, PHP, and Perl
Also they tend to be used more by 'non-programmers' who get away with 'stupid' stuff like calling out to system() and piping a bunch of commands together because they don't know how to use the language properly :) I understand your point but I don't share it There are a lot Asterisk-Perl project working in production environment. Anyway...Thanks for your research! Mind if I post it to the Daily Asterisk News? You have my permission. -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Benchmarking AGI performance in C, PHP, and Perl
On Mon, Jul 11, 2011 at 5:29 PM, Steve Edwards asterisk@sedwards.com wrote: Many times, I've made the statement that you can execute hundreds of AGIs written in C in the time it takes to load an interpreter and parse a script written in PHP or Perl. I've truly enjoyed this thread. And while startup time is certainly part of the equation, I'm curious whether you also recorded memory overhead while you were doing your benchmarks. In my personal observation of 'Perl running in production', there's a significant memory complexity associated with running lots of simultaneous Perl interpreters. I'm guessing the PHP overhead is smaller but still a factor. Because C lib is both statically compiled and essentially built into the system, there's no such 'waste' of running the interpreters. Along with the narrowing gap you saw by 'throwing hardware at the problem' that shrinks the difference between C and interpreted languages, I personally now work with a production asterisk environment where systems have dozens of GB of ram. As such, it's not entirely crazy to say 'so what' about the whole thing. Nonetheless, do you have any numbers to back up my theory? -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Benchmarking AGI performance in C, PHP, and Perl
On 11/07/11 23:42, Steve Edwards wrote: On 12/07/11 9:29 AM, Steve Edwards wrote: Many times, I've made the statement that you can execute hundreds of AGIs written in C in the time it takes to load an interpreter and parse a script written in PHP or Perl. Well, now that I know better, let's not perpetuate an ancient claim. 'Dozens' is more appropriate with current hardware. On Tue, 12 Jul 2011, Matt Riddell wrote: It would be interesting to see the same types of tests run against fast-agi - personally if I write an agi that will be called 1000 times I'm going to leave it running and have network requests against it rather than starting and stopping every time. Isn't every AGI going to be executed 1,000s of times over it's lifetime? 'Standalone' AGIs still have advantages in lower complexity and less impact on failure. If a bug takes out your fastagi daemon it can affect all calls. I'm pretty sure if you have a bug in your AGI code it's going to affect all calls whether its fastagi or not. Unless the bits between the AGI and DB calls are significant, there should be no significant difference between source languages in a fastagi environment. One of my other objections to scripting languages are that they don't 'catch' stupid errors for me (like a misspelled variable name) as well as compiled languages. 'gcc -Wall' is my friend. Also they tend to be used more by 'non-programmers' who get away with 'stupid' stuff like calling out to system() and piping a bunch of commands together because they don't know how to use the language properly :) Mind if I post it to the Daily Asterisk News? You have my permission. In a production environment any code will have gone through a testing phase, so your argument about a compiled language detecting stupid errors is pretty much irrelevant. Also most critical bugs will be in the script logic not the syntax. I'm also curious why you think the poor performance of a scripting language actually matters for AGI code? As stated most of the time will be spent by Asterisk streaming audio / waiting for a prompt. The few extra milliseconds a php script takes to start up are not going to be noticeable by any human listening to the call. Vince. -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Benchmarking AGI performance in C, PHP, and Perl
Also they tend to be used more by 'non-programmers' who get away with
'stupid' stuff like calling out to system() and piping a bunch of
commands together because they don't know how to use the language
properly :)
On Mon, 11 Jul 2011, cbul...@gmail.com wrote:
I understand your point but I don't share it There are a lot
Asterisk-Perl project working in production environment.
Then I didn't communicate my point clearly.
I'm not disparaging Perl programmers or the language. I'm just saying it
is easier to 'abuse' a language like Perl or PHP than C.
This Bash snippet was posted to the -users list a couple of years ago. I'm
not trying to embarrass the original programmer or trash his skills, I'm
just using this snippet as an example. We've all got skeletons in our
closets :)
while read line; do
epoch=`echo $line | cut -d '|' -f 1`
if [ $epoch -ge $start_epoch -a $epoch -le $end_epoch ]; then
echo $line
fi
done /var/log/asterisk/queue_log
Note the second line. It creates a couple (2 or 3) of processes to extract
the first field from the queue_log file. For every line in the input file!
This kind of coding is easy in a scripting language. It would be way more
difficult in C. So much more difficult, a programmer with the requisite
skills to do it should recognize the inefficiency and do it another way.
If the original programmer had a better grasp of the language, he could
have coded this line as:
epoch=${line:0:10}
Since the Epoch will be 10 digits for the next 300 years, I'd feel
relatively comfortable with this solution.
This single change reduced the execution time of his script by an order of
magnitude.
Recoding it in a language more appropriate to processing lots of data
(like C) would reduce the execution time to 1/3,000th of the original. And
yes, I did it and measured it.
A skilled programmer (like any craftsman) has many tools in his toolbox,
the experience to choose the right one, and the skill to use it well.
C is my sharpest tool so I tend to see everything through that lens, but
I'm learning to appreciate PHP and how it lets me represent some
programming problems clearly, quickly and sufficiently efficient.
--
Thanks in advance,
-
Steve Edwards sedwa...@sedwards.com Voice: +1-760-468-3867 PST
Newline Fax: +1-760-731-3000
--
_
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
New to Asterisk? Join us for a live introductory webinar every Thurs:
http://www.asterisk.org/hello
asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Benchmarking AGI performance in C, PHP, and Perl
On Mon, Jul 11, 2011 at 5:29 PM, Steve Edwards asterisk@sedwards.com wrote: Many times, I've made the statement that you can execute hundreds of AGIs written in C in the time it takes to load an interpreter and parse a script written in PHP or Perl. I can see I'm going to spend the rest of my days erasing 'hundreds' and replacing it with 'dozens.' Everybody is quoting the obsolete ratio :) On Mon, 11 Jul 2011, David Backeberg wrote: I've truly enjoyed this thread. And while startup time is certainly part of the equation, I'm curious whether you also recorded memory overhead while you were doing your benchmarks. No I did not. My test was a single channel executing 1,000 AGIs one after the other. In my personal observation of 'Perl running in production', there's a significant memory complexity associated with running lots of simultaneous Perl interpreters. I'm guessing the PHP overhead is smaller but still a factor. I tend to agree. Just the simple change of statically linking the C AGI reduced it's execution time from 16 to 6. That was for an AGI that didn't do anything useful, but imagine if every executable in your distribution was statically linked. (Check out http://sta.li/) Would the system be noticeably more responsive? Every command line you type creates a new process -- maybe a bunch. When I first started programming on Unix, I thought it was hot stuff to code several commands into the PROMPT_COMMAND environment variable. Every time I pressed ENTER all those commands had to be evaluated. Along with the narrowing gap you saw by 'throwing hardware at the problem' that shrinks the difference between C and interpreted languages, I personally now work with a production asterisk environment where systems have dozens of GB of ram. As such, it's not entirely crazy to say 'so what' about the whole thing. Certainly it is 'cheaper' to throw hardware instead of labor in most situations, but once you 'know' one way of coding is more efficient than another, it helps you make better implementation decisions. -- Thanks in advance, - Steve Edwards sedwa...@sedwards.com Voice: +1-760-468-3867 PST Newline Fax: +1-760-731-3000 -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Benchmarking AGI performance in C, PHP, and Perl
On 11/07/11 23:42, Steve Edwards wrote: 'Standalone' AGIs still have advantages in lower complexity and less impact on failure. If a bug takes out your fastagi daemon it can affect all calls. On Tue, 12 Jul 2011, Vincent Sweeney wrote: I'm pretty sure if you have a bug in your AGI code it's going to affect all calls whether its fastagi or not. I was thinking of a situation where not all calls follow the exact same path through the code and one of those paths caused the daemon to segfault. In a production environment any code will have gone through a testing phase, so your argument about a compiled language detecting stupid errors is pretty much irrelevant. Also most critical bugs will be in the script logic not the syntax. In a perfect world, we would all have regression testing in place to exercise every decision point and every value threshold in our code. I've never found a client willing to pay for that. It has been my experience with many scripting languages that they do not syntax check my code, warn me if I have too many or too few arguments to a function, warn of undeclared variables or warn of unused variables. Gcc does all of these for me. Will 'strict' or 'taint' do this for me? In AEL, a single missing semicolon can cause large blocks of code to disappear. Maybe I'm just more clumsy at the keyboard, but I have had characters appear and disappear within an editing session. I'm also curious why you think the poor performance of a scripting language actually matters for AGI code? As stated most of the time will be spent by Asterisk streaming audio / waiting for a prompt. The few extra milliseconds a php script takes to start up are not going to be noticeable by any human listening to the call. It matters once you know. It influences your decision making process. In my home Asterisk box, it just doesn't matter. The dual cores (plus hyperthreading) of an Atom D525 with 4GB of RAM can handle the occasional 2 or 3 simultaneous calls in its sleep. Think of a system handling hundreds of simultaneous calls, tens of thousands of calls per day. All those milliseconds, all the memory footprints, all the disk accesses reading the 'uses' and 'requires' will add up. Paraphrasing Senator Everett Dirksen, A billion (th of a second) here, a billion (th of a second) there, and pretty soon you're talking about real money.” Look back to the stats from my first post. Starting (not executing to completion) neutered-agi.pl (AKA agi-VDAD_ALL_outbound.agi) on the Geode took 2/3rds of a second. Obviously a Geode is the wrong platform for Vicidial, but that kind of delay can add up fast. My first Asterisk system is still in production and making money for the client. It was designed (by me) very simplistically. Each AGI did one thing and then moved on. So an incoming call would execute set-globalid, block-ani, lookup-dnis, update-channel-status, update-caller-status and write-cdr all before the caller heard the first prompt. All the AGIs were coded in C, all but 1 access the database and it all happens in the blink of an ear. I think I was lucky. I think if I had implemented the same design in Perl it would have fallen flat. If I was to re-implement the system today I would obviously merge all these steps into a single AGI, but it works well and the client won't pay to fix what isn't broken. -- Thanks in advance, - Steve Edwards sedwa...@sedwards.com Voice: +1-760-468-3867 PST Newline Fax: +1-760-731-3000-- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
