*I don't understand this statement. Shouldn't node outperform Python when IO is the majority of the operations, no matter the threads*
The first thing to recognize is that your test is severely IO bound - 1% (or probably less) of time may be spent in the application tier, while the rest is spent waiting on the database - Node.js can't make the database respond faster, and if it doesn't have a high volume of requests to process, driven by JMeter, it is sitting around twiddling its thumbs. Node.js should handle your workload really well, but that doesn't really become relevant until you have a very large number of simultaneous requests coming in. *That makes sense. But I still would have expected a single threaded asynchronous program to beat a threaded synchronous program in the majority of the cases. This was not happening.* * * *You've described a test in which almost no actual processing happens in the application tier. Which means response time is almost entirely governed by DB latency, which, because of lack of indexes, has an extremely high variance. Why would an asynchrounous framework have any advantage in this scenario?* * * *How would you design such a test?* * * *Not really sure of your goals, so that's a hard question to answer. But its worth saying that async isn't a performance silver bullet - what it does is enforce an architectural pattern that reduces IO/blocking bottlenecks as your applications scales. It doesn't seem to me that your test creates enough request concurrency to cause meaningful blocking within python (as long as there are more threads to process requests, blocking is essentially irrelevant), and so I don't think you'll get the results you want. * On Tuesday, November 27, 2012 12:28:15 PM UTC-5, Barry Steyn wrote: > > Hi Joel > > >> Node can't really outperform if its getting the same number of client >> threads and they're all sitting around waiting for the DB to do a full >> table scan > > > I don't understand this statement. Shouldn't node outperform Python when > IO is the majority of the operations, no matter the threads. To answer your > question, I had 30 threads running for each server (both Node and Python) > with a loop of 100. > > As a side note, I would assume the variance in your response times is >> dependent on where the target row in the DB is. Since you have no index, >> the query would have to look at each row >> > > That makes sense. But I still would have expected a single threaded > asynchronous program to beat a threaded synchronous program in the majority > of the cases. This was not happening. > > How would you design such a test? > > > > > On Tue, Nov 27, 2012 at 11:08 AM, Joel Rolfe <[email protected]<javascript:> > > wrote: > >> How many client threads are you pushing from JMeter? Node can't really >> outperform if its getting the same number of client threads and they're all >> sitting around waiting for the DB to do a full table scan. As a side note, >> I would assume the variance in your response times is dependent on where >> the target row in the DB is. Since you have no index, the query would have >> to look at each row. If it finds it early, then quick response time. If >> it finds it near the end, then longer response time. That's my initial gut >> reaction to how your test would behave anyway. >> >> On Monday, November 26, 2012 10:15:57 PM UTC-5, Barry Steyn wrote: >>> >>> Hi All >>> >>> We attempted to benchmark nodejs against python by using the express >>> framework (for Node) versus flask framework (for Python). The idea was to >>> use basic auth on a database with ten thousand (10,000) usernames. The >>> database table that holds the usernames was not optimised in any way (i.e. >>> no indexes etc). >>> >>> We expected (and hoped) for NodeJS to outperform Python due to the heavy >>> IO load in the test. Unfortunately, the results were almost the same, with >>> Python slightly beating Node. We were also noticing that response times >>> across both were varying quite a bit (from ~50/sec to ~500/sec)... perhaps >>> there's something else we're missing. >>> >>> The setup was as follows: >>> >>> - Web server: NginX >>> - For Python: Uwsgi server with 30 threads >>> - For node: Just ran node app on the command line >>> - A micro instance of AWS running unbuntu 12.04 >>> >>> Why is Node's performance not blowing Python out of the water. Perhaps >>> we have done something wrong with our benchmarking test. Any ideas would be >>> appreciated. >>> >>> Barry And Niall >>> >>> -- >> Job Board: http://jobs.nodejs.org/ >> Posting guidelines: >> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines >> You received this message because you are subscribed to the Google >> Groups "nodejs" group. >> To post to this group, send email to [email protected]<javascript:> >> To unsubscribe from this group, send email to >> [email protected] <javascript:> >> For more options, visit this group at >> http://groups.google.com/group/nodejs?hl=en?hl=en >> > > -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
