One problem I've come across using xmlrpc (with Twisted) in a web environment, is that when traffic is high you run out of file handles on your server. 100 web server requests means 100 (or more) HTTP connections for HTTP/1.0 -- which is by far the easiest to implemenent. (I'm not sure Twisted's xmlrpc service supports HTTP/1.1!)
My experiences support what you are saying here. I successfully implemented a PyLucene search application ontop of twisted web. Performance is pretty good under normal conditions. When I stress the search application using http_load I get pretty good TPS. When I stress a simplier page I stop accepting after about 30 seconds, but after a short interval twisted web will start accepting connections again. Maybe twisted web2 can handle this better, but I am a little hesitant to delve too deep with twisted web and it's divmod incarnations. On first glance, things do not appear clean and stable enough for my purposes. And this is from a guy who tried to run PyLucene in CherryPy/TurboGears.
So let's say you have 4 round-robin web servers all hitting a central server. If each of those takes 500 hits, that means 2,000 connections to the web server. Over PB, it means 4 connections.
I need to do more research on PB. Are you saying that PB can function as a HTTP load balancer?
And as Pete Fein mentioned, xmlrpc serialization isn't always fast, especially for large, complex data structures.
There are always tradeoffs. A binary protocol can be blazing fast, but a major pain to develop with. Personally, I really like JSON. Straight up data is much easier to model across languages and serialize over the wire than function calls. Joe _______________________________________________ pylucene-dev mailing list [email protected] http://lists.osafoundation.org/mailman/listinfo/pylucene-dev
