Re: performance testing - emulating real world use
Heh. Forgot to state that it does cost an arm and a leg but it's one of the few software packages that is worth considering paying money for IMO. However, with the economy being the way it is, it is possible to "rent" the software for a period of time but this is done by special arrangement on a case by case basis. If you ask, they may be willing to give you a copy of the software. The out-the-box install allows you to record and playback as well as load up to 10 (iirc) users which nicely lets you test out the functionality. I think I have a copy of it somewhere around here. If there is interest, I can get you in touch with the right people. --Jauder On Wed, 13 Mar 2002, Matt Sergeant wrote: > On Tue, 12 Mar 2002, Jauder Ho wrote: > > > > > Another application (commercial) is Mercury Interactive's LoadRunner. It > > actually records events and plays it back on "load generator" machines. > > It's fairly complex, has LOTs of knobs to turn and can load test quite a > > bit more than just web apps, I use it to load test/benchmark Oracle 11i > > for instance. The software is not cheap but definitely worth looking into > > if you are serious about testing. (www.merc-int.com) > > > > They also sell something called ActiveTest which may be more suited to a > > web applications. In this case, they will test your site for you using > > their hardware at a colo site. > > Before anyone even looks into this, be warned they quoted me £50,000 once > for LoadRunner. Needless to say I was flabbergasted (though their software > did look kinda cool). > > -- > > <:->Get a smart net > >
Re: performance testing - emulating real world use
Another application (commercial) is Mercury Interactive's LoadRunner. It actually records events and plays it back on "load generator" machines. It's fairly complex, has LOTs of knobs to turn and can load test quite a bit more than just web apps, I use it to load test/benchmark Oracle 11i for instance. The software is not cheap but definitely worth looking into if you are serious about testing. (www.merc-int.com) They also sell something called ActiveTest which may be more suited to a web applications. In this case, they will test your site for you using their hardware at a colo site. --Jauder On Tue, 12 Mar 2002, Andrew Ho wrote: > Heyas, > > BH>Anyone know of good guides or general info on > BH>performance testing and emulating real use of > BH>an application. > > As a general rule, it's easiest if you have a production system already > running. Record all information that you need to reproduce the requests > (typically, HTTP request headers and POST data if applicable), from a > production server and you can replay any amount of data on a sandboxed QA > environment. You can either eliminate or proportionally shorten the time > period between requests to space out load arbitrarily. > > This is extremely effective if you have enough real user data because > you're not inventing user load. You're using real user load. > > I don't know of any product that does this all at once, but it's not hard > to hack together. If your site is entirely GET based, you can probably > just make do with parsing access logs and turning those into requests. I > believe Apache::DumpHeaders might get you most of the way on the capturing > side if you need special headers, cookies, or POST information. > > Feeding scripts into commercial products like SilkPerformer will give you > the best client side testing and reports. However, a homebrew Perl user > agent will do okay, too. Unfortunately, ab doesn't support taking in a > custom URL workload. > > For a simple record/replay load test tool that works remarkably well, > check out the resource CD that ships with Windows 2000 and you will find > the Microsoft Web Stress Tester. It's free and GUI based and can record IE > sessions and replay them with an arbitrary number of threads. It uses > Access databases to hold the tests and results so you can probably use > Perl on Windows to populate it with your custom tests. > > Humbly, > > Andrew > > -- > Andrew Ho http://www.tellme.com/ [EMAIL PROTECTED] > Engineer [EMAIL PROTECTED] Voice 650-930-9062 > Tellme Networks, Inc. 1-800-555-TELLFax 650-930-9101 > -- > >
Re: perl5.6 (was: Shared variables, inner subs and "our")
Unfortunately it is a requirement for this project. We need to parse utf8 XML and munge other XML using that data. I seem to have everything working now although I had to fix some stuff in XML::DOM. --Jauder On Sun, 18 Mar 2001, Bogomolnyi Constantin wrote: > Hi , > Unicode is a way of pain . don't go this way . > specialy in 5.7.0 were it is broken . > 5.6.0 is well you know what . > There is many other systems to hold multilingual apps , that not require any > unicode . > > I use 5.7.0 in production (i386 , BSD 4.2) and it runs perfectly . Except > unicode > support. 5.6.0 has a lot of bugs (witch were fixed in 5.7.0) > > Best > Cb > > - Original Message - > From: "Jauder Ho" <[EMAIL PROTECTED]> > To: "Bogomolnyi Constantin" <[EMAIL PROTECTED]> > Sent: Monday, March 19, 2001 8:44 AM > Subject: Re: perl5.6 (was: Shared variables, inner subs and "our") > > > > > > Hello there, could you detail a little more about your Unicode experience > > with perl? I am currently evaluating what the best verion of perl to use > > for a multilingual application. Thanks. > > > > --Jauder > > > > On Fri, 16 Mar 2001, Bogomolnyi Constantin wrote: > > > > > Hi , > > > > > > You should probably try 5.7.0 witch is much more stable than 5.6.0 (you > > > should not try unicode stuff , whitch is quite buggy) > > > > > > I use 5.7.0 on all my production servers without any problems . > > > > > > Best > > > - Original Message - > > > From: "Wim Kerkhoff" <[EMAIL PROTECTED]> > > > To: "modperl" <[EMAIL PROTECTED]> > > > Sent: Saturday, March 17, 2001 6:20 AM > > > Subject: Re: perl5.6 (was: Shared variables, inner subs and "our") > > > > > > > > > > Stas Bekman wrote: > > > > > > > > > our() and other perl5.6 new APIs are too early to be endorsed, as > 5.6 is > > > > > not yet considered as a stable version for mod_perl production > sites, > > > > > therefore the guide barely touches on it. > > > > > > > > Would you recommend the use of perl5.6 with mod_perl? What you are > > > > saying is making me queasy. > > > > > > > > I'm asking because I've been having bad luck with Apache::Session and > > > > some other modules in some mod_perl aware applications. Things aren't > > > > comming out the data source properly, things are hanging, things > aren't > > > > locking/unlocking properly, etc. It could well be that the > applications > > > > I'm working with aren't using Sessions, Tie::Cache, etc properly. I > may > > > > downgrade to perl5.005 and give that a whirl... > > > > > > > > -- > > > > > > > > Regards, > > > > > > > > Wim Kerkhoff > > > > > > > > > > > > > >
VeryOT: help needed
Alright, it's crunch time (trying to help a coworker out) and I need to pick up some XML parsing with perl fast. So if anyone here has some good resources, they can point me at, it would be much appreciated. The problem I am trying to solve is as follows: I have two XML files, one is encoded in UTF8 and the other is in ASCII7. I need to parse the ASCII7 file first and if a tag exists in the UTF8 file replace the data in the ASCII7 file with the UTF8 info. Everything should then be output in a UTF8 format. Pointers to manipulating XML needed but I would take any additional help. Thanks. --Jauder
Re: [OT] The $64K question/server hardware
Heh, I put in a solution that was MUCH larger. 16 sun servers (14 E420R and 2 E6500s) just in the production environment. Total server count was somewhere in the 60s. One thing to note is that IIS on NT4 leaks memory like mad on a high volume site so you will have to keep rebooting the boxes. The solution put in place required both Sun and NT machines and the Sun boxes were infinitely more stable (even though they were using *ugh* Netscape Enterprise Server). But IIS on Windows 2000 appears to be much better. --Jauder On Wed, 2 Aug 2000 [EMAIL PROTECTED] wrote: > > Can we also include OS ? > > I'm particularly interested in performance figures of NT against *nix to > help my case to ditch NT in a project which has yet to go live. > > Simon Wilcox > Intranet Development Manager > Williams Lea Group > > > > From Ajit Deshpande <[EMAIL PROTECTED]>Date 22:43:28 1 August >2000 > > To: [EMAIL PROTECTED] > cc:(bcc: Simon Wilcox/BASE/WilliamsLea) > Fax to: > Subject: Re: [OT] The $64K question/server hardware > > > > On Tue, Aug 01, 2000 at 02:32:52PM -0700, ___cliff rayman___ wrote: > > [ .. snip .. ] > > Since this has potential for becoming another long thread with people > posting their stories and some other people flaming for not doing things > in a certain way, I have suggestions for this monster thread in the > making: > > 1. Everyone post information in the format at the end of this mail > 2. Send it in a private e-mail to me if confidentiality is a problem >(or pick someone you trust, more than me, and have them e-mail it > to me :) > 3. I will consolidate the information and dash it off to Ask or whoever >maintains the perl.apache.org > > What do ye think? > > Ajit > > Format for posting information: > > 1. Setup : >for example: >- Single modperl server on one host, apache 1.3.12, mod_proxy >- One modperl, one front-end proxy on one host, apache 1.3.12, > mod_perl 1.24 >- Multiple modperl and multiple hosts > > 2. Average hits/day . Peak hits/second. > > 3. CPU details >for example: >- Front end proxy on PII 333 >- Modperl-enabled httpd on PIII 550 >- MySql/Oracle DB on Quad Xeon PIII 550 (Wow! :) > > 4. Memory details: >for example: >- Front end proxy with 128 Mb >- Modperl-enabled httpd with 256 Mb >- MySql/Oracle DB with 2Gb > > 5. Storage details: >for example: >- 10,000 RPM Ultra-wide SCSI discs on all hosts >- Root partition for each server on 1.2 Gb IDE >- Session stuff on NFS mounted NetApp (!) >- D/B disk on EMC sub-system (!) > > 6. Kind of content: >- 70% dynamic, d/b driven, 20% static, 10% images >- All dynamic except images > > 7. Publication of the site: >- http://www.foo.com OR Author does not wish to divulge >- Intranet site, not accessible from public internet >- In production since Aug-99 >- Primary contact : [EMAIL PROTECTED] OR Author does not wish to divulge > > > > >
Re: [is it time for something other than html?] RE: Templating system
Okay, I think there is still some confusion here as to what I am shooting for. Ideally, I would like a system where the static parts of the page are cached. I have recieved several mails suggesting that items such as the header/footer can be turned into compiled print() statements as part of the caching mechanism, but my concern is that a) You have to wait for it to generate the first time every time the server is restarted. Granted, this is not too big of a deal. BlueMartini (a product I was working with) is based on top of Weblogic and was so slow that they had to create a tool to walk the tree to preload all the pages into memory. Ugh. b) HTML AFAIK only lets you set the pragma=no-cache on a per page level so the "static" portions of the page will not be cachable by systems such as Inktomi's Traffic Server. So essentially, what would be nice would be a system which is cache-friendly (not server side) which at the same time is able to present dynamic content. This I think would certainly help in the needless representation of data that happens all the time today. Imagine, you go to a site with similar look and feel and once the templates hit your browser, they can stay cached and only have the dynamic bits fed to you. Faster surfing is good :) --Jauder On 28 Jul 2000, David Hodgkinson wrote: > > Jauder Ho <[EMAIL PROTECTED]> writes: > > > The template may be kept in memory but it needs to be reparsed to insert > > real values, no? What I would like to see is a way to say the template is > > static (header/footer) and does not need to be reparse/regenerated each > > time and comparitively small portion of the page be dynamic. This way, you > > can also store the html on the front end web server or akamize it and > > concentrate on having the app servers provide only the actual data. > > If you want the resultant page to be cached, you can do this with a > scary mod_rewrite mode. > > -- > Dave Hodgkinson, http://www.hodgkinson.org > Editor-in-chief, The Highway Star http://www.deep-purple.com > Apache, mod_perl, MySQL, Sybase hired gun for, well, hire > - > >
Re: [is it time for something other than html?] RE: Templating system
Cool, I will definitely look further into this. Time to google... --Jauder On Thu, 27 Jul 2000, Kip Hampton wrote: > Jauder Ho wrote: > > > > XML+XSLT is an interesting combination > > but integrating that into a dynamic generator (perl based or other) is > > going to be nontrivial to say the very least. Is there anyone interested > > in exploring this? > > > Most of the hard work for this has been done by our own lovely and > talented Matt Sergeant by providing native support for Perl XSP in > AxKit. You add to your XML and XSP does > &function($param) and whatever else behind the scenes. Okay, so you have > to write the taglib to make this happen, but once it's done it's done > and you're *ML folks never have to deal with code at any level. And, > while adding new taglibs is not exactly trivial, it certainly isn't > rocket science, either. > > IMHO, XML+yourfavoritetransformer(s) is the answer to separating the > three worlds of code, content and design. Yes, we all want lighting > fast, dynamic, reusable widgets for our Web apps. I'm just becoming > increasingly convinced that HTML is one possible *output* of such a > system, not it's basic building block. > > > -kip > -- > "In this world you must be oh-so smart or oh-so pleasant. > Well, for years I was smart I recommend pleasant." > >--Elwood P. Dowd > >
Re: [is it time for something other than html?] RE: Templating system
The template may be kept in memory but it needs to be reparsed to insert real values, no? What I would like to see is a way to say the template is static (header/footer) and does not need to be reparse/regenerated each time and comparitively small portion of the page be dynamic. This way, you can also store the html on the front end web server or akamize it and concentrate on having the app servers provide only the actual data. --Jauder PS. I guess now that I think more about it, we can accomplish this with frames but that's a whole other story :) On Thu, 27 Jul 2000, Perrin Harkins wrote: > On Thu, 27 Jul 2000, Jauder Ho wrote: > > If there was somehow a way to cache say the template, leaving only the > > same dynamic portion uncached, it would certainly help things along > > quite a bit. If anyone knows of a good way of doing this I would > > certainly be interested in hearing it. > > I believe every single one of the solutions discussed caches templates, > most of them as compiled perl code. > - Perrin > >
RE: [is it time for something other than html?] RE: Templating s
No. The problem is that you cannot cache the subsequent page that is generated because of the dynamic component. The template is always going to remain the same (interface changes should be rare for a stable web site), but the content (say news headlines or personalized component) is going to be dynamic. It is not reasonable to expect to cache that entire page. So what I am saying is that using HTML for such purposes is fundamentally broken unless we can figure out to tag a region of an html file as non-cachable i.e. dynamic. And as I replied to another reply about this, embedding code in a HTML file strikes me as an inelegant solution plus it leads to all sorts of problems such as html "designers" that delete tags for whatever reason. I know that the correct solution is to lart them but we can't always do that no matter how much we would like to :). --Jauder On Thu, 27 Jul 2000 [EMAIL PROTECTED] wrote: > > I could go off about why HTML sucks for dynamic page caching. If there was > > somehow a way to cache say the template, leaving only the same dynamic > > portion uncached, it would certainly help things along quite a bit. If > > anyone knows of a good way of doing this I would certainly be interested > > in hearing it. > > There is. Use HTML::Embperl. > > I've been using it for several years now. I build objects for accessing any > data I need and then do simple things in my templates like: > > [+ $forum->title +] > > -- > Jason Bodnar + [EMAIL PROTECTED] + Team Linux > > If it'll make you feel any better, I've learned that life is one crushing > defeat after another until you just wish Flanders was dead. > > -- Homer Simpson >Homer and Apu > >
[is it time for something other than html?] RE: Templating system
Watching this discussion has been very interesting, I am all for separating the HTML and the code portions and have been unable to think of a good solution to this particular problem. I ran across smartworker (http://www.smartworker.org) a while ago and even though I have not had the time/chance to implement such a system, it certainly looks like an interesting method of doing things. XML+XSLT is an interesting combination but integrating that into a dynamic generator (perl based or other) is going to be nontrivial to say the very least. Is there anyone interested in exploring this? I could go off about why HTML sucks for dynamic page caching. If there was somehow a way to cache say the template, leaving only the same dynamic portion uncached, it would certainly help things along quite a bit. If anyone knows of a good way of doing this I would certainly be interested in hearing it. --Jauder
Re: mod_perl Programmers demand is going up...
Nice argument but there's a flaw in it :) Do you have any idea how hard it is to find a "skilled unix admin"? In fact (this is off topic but I'm desperate), if there is someone in the Seattle area that's looking, there is an immediate opening... --Jauder On Fri, 3 Dec 1999, Richard Dice wrote: > Jeffrey: > > > There can be no such thing as a shortage of mod_perl programmers. When > > someone says that they can't find mod_perl people, what they really mean > > is that they can't find mod_perl people for the price they are willing > > to pay. > > Well said! The term for this is "market clearing price", and it > certainly applies in this situation. There are certain very limited > situations where there truly can be a "shortage", but it only lasts > for a limited amount of time -- the time it takes for a "new generation" > of people to train themselves for the task. Perhaps 12-18 months? > (i.e. Take all people who are skilled unix sysadmins and programmers, > with a decent amount of Perl experience as well, and tell them that > they can earn a zillion dollars an hour if only they can become mod_perl > experts. They all take the year off from work as an investment, > become those experts... and suddenly find that, because everyone did > this, they can only make some fraction of a zillion an hour. Ah, > market economics. :-) ) > > I agree with your conclusion, too, that more and better documentation > is the most effective way to increase the supply of skilled mod_perl > people. > > Cheers, > Richard > > - > Richard Dice * Personal 514 816 9568 * Fax 514 816 9569 > Open Source Evangelist, HBE Software * http://www.hbesoftware.com > ShadNet Creator * http://shadnet.shad.ca/ * [EMAIL PROTECTED] > Occasional Writer, HotWired * http://www.hotwired.com/webmonkey/ > "squeeze the world 'til it's small enough to join us heel to toe" > - jesus jones > >
Re: New mod_perl RPM really needed? (was: sourcegarden plug)
If anyone really cares, I have a mod_perl rpm built at http://www.carumba.com/rpms/. It really is not that hard to do. Solaris packaging is much more of a pain in the butt. --Jauder On Wed, 6 Oct 1999, David Harris wrote: > > Young, Geoffrey S wrote: > > Thus, it might be worth mentioning that RPMs are a _bad_ idea for > > those just getting into mod_perl. That is, unless others have been more > > successful that I... > > I've got mod_perl running just fine with my own Apache package and RedHat's > mod_perl RPM. I understand that this keeps me from being able to use some stuff > like request chaining, but I've not had a need for it. I've also stayed away > from any mod_perl development environments (Embperl, Mason, whatver) and just > wrote the handlers all myself. > > I'm going to package all of my work up today into RPMS and publish them out to > the production servers, so I'm wondering if I should make my own mod_perl and > Apache RPM or stick with what I have working. I keep hearing that RPM's and > mod_perl are evil, but personally everything installed and worked without a > hitch. > > Oh, I remember that I had some trouble compiling libapreq, but copying a few > mod_perl header files into the system solved that without too much pain. > > A while ago in his "sourcegarden plug" thread Stas wrote: > > Jim, one of the "services" we _want_ to provide at mod_perl Source Garden > > (modperl.sourcegarden.org) is prebuilded mod_perl RPMs in its various > > configurations and for mainly used platforms. Taken that we are only a few > > folks who actually contributing to this project, this item is not on the > > top of our priorities. > > > > I wonder if you or someone else may want to step in, and to start creating > > correctly prebuilded SRPMs and RPMs, and if later feel that it's not > > challenging (which I've found quite challenging, YMMV), someone else will > > pick the falling flag. But it's definitely a good way to learn creating > > RPMs which are very usefull, contributing back to our community, (and > > fighting our mutual enemy :) > > I'm basically saying that I could do this and put together some mod_perl+Apache > and libapreq RPM's today.. but I'm just wondering if it's really needed. > > - David Harris >Principal Engineer, DRH Internet Services > > >
