Re: [Zope-dev] remote procedure calls to manage functions
On 10 Aug 2000 08:10:01 -0500, Jim Fulton wrote: Michel Pelletier wrote: Jim Fulton wrote: Michel Pelletier wrote: Also, ZClient is much faster i've found, probably due to the marshalling/unmarshalling necessary for xml-rpc. That's interesting..Hm. ZClient has to marshal. I suspect that xml-rpc wants some sort of optimization. I made a test script to make about 300 xmlrpc calls to various manage_ methods. I had to kill it after about a half-hour cuz it was takin so long. This sounds pretty fishy. Are you saying that you made less than 300 calls in half an hour? This doesn't seem right at all. I'm probably way off base here, but I use xml-rpc to transfer records to a Zope-based application from a bunch of Macs attached to ABI gene sequencers. The data is simple lists of lists and it *appears* to take about 2-3 seconds for the Macs to xml-encode and transfer the data as-is, but it takes typically *3 minutes* for the xmlrpc server to decode and rebuild the table data. This is for about 6000 records representing about 200k of text data (Sun Ultra 5 server). However, if I simply build a string representation of the table data, it takes about 3-4 seconds for the client to stringify, transfer the string data, and the server to reconstruct the table. Guess which method I use? The tests I ran using a SAX-based parser show it might parse and rebuild the data at least 50x faster than the current Python-only xmlrpclib. Now maybe something else is going on wrt manage* methods and xmlrpc, (I'm using external methods for managing the data transfer and am not storing the raw data in the Zope database), but my external methods-based calls rip along at a pretty good clip as long as I convert the data to a (few) string(s) before sending. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] remote procedure calls to manage functions
Jim Fulton wrote: It's not really favored, both are quite useful. xml-rpc is more for when you want two different system to interoperate. ZClient is very zope specific and probably gives you a bit more functionality than xml-rpc (because it has the 'remote object' abstraction). I don't think ZClient is really Zope specific. At least it's not supposed to be. It does have a richer API, including features like authentication and header support. The problem I have with ZClient is that the result of method calls is always a string - while with XML-RPC I can get lists/dictionaries/integers, if that's what the method returned. -- Itamar S.T. [EMAIL PROTECTED] Fingerprint = D365 7BE8 B81E 2B18 6534 025E D0E7 92DB E441 411C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] remote procedure calls to manage functions
Itamar Shtull-Trauring wrote: Jim Fulton wrote: It's not really favored, both are quite useful. xml-rpc is more for when you want two different system to interoperate. ZClient is very zope specific and probably gives you a bit more functionality than xml-rpc (because it has the 'remote object' abstraction). I don't think ZClient is really Zope specific. At least it's not supposed to be. It does have a richer API, including features like authentication and header support. The problem I have with ZClient is that the result of method calls is always a string - while with XML-RPC I can get lists/dictionaries/integers, if that's what the method returned. I didn't claim that it was better for RPC, but that it simply wasn't Zope specific and has some features that XML-RPC lacks. Frankly, I'd very much like to see the missing features in CML-RPC or SOAP. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! Technical Director (888) 344-4332http://www.python.org Digital Creationshttp://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] remote procedure calls to manage functions
On Wed, 09 Aug 2000 16:06:36 -0400, Jim Fulton [EMAIL PROTECTED] wrote: That's interesting..Hm. ZClient has to marshal. I suspect that xml-rpc wants some sort of optimization. xmlrpclib will by default use python's standard library xmllib.py for parsing, which is terribly slow. There is support for an optional C extension, which gave me a 12x speed increase. Check out the source for xmlrpclib.py for pointers. Toby Dickenson [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] remote procedure calls to manage functions
Jim Fulton wrote: Michel Pelletier wrote: I made a test script to make about 300 xmlrpc calls to various manage_ methods. I had to kill it after about a half-hour cuz it was takin so long. This sounds pretty fishy. Are you saying that you made less than 300 calls in half an hour? This doesn't seem right at all. My bad, my fingers dropped a zero. I was gonna say 3000. I don't know the exact number but I can find out, i just wrapped a for loop around the code that uploads the book content to zope.org; effectively uploading the book 100 times. I don't know the exact time either, but in the time it ran before i killed it I recompiled and reconfigured a linux kernel three times and had at least four cups of coffee. Hm. This says that the FileStorage _finish method was called 3852 times. This says that 3852 transactions were committed. That may be the number of calls it made before i killed it. Since transactions only get committed on write requests, then this suggests 3852 write requests. There should have been way more than 3852 requests to have this many write requests. You mean 300? First, it's not spending most of it's time in DT_InSV __getitem__, although it's spending more time in that method than in any other single method. The reason it spends so much time in that method is that the method gets called 700 thousand times. That is, it's used alot, because alot of in-tag special variables are used in your test. I'm not sure why though, I'm not executing any DTML, just uploading a bunch of structured text into File objects (and three DTML methods to view the book). This is also a pretty non-trivial method, so it's reasonable that it might take a while. All of this seems to me to be somewhat beside the point. The original comment was regarding ZClient vs XML-RPC. It would be more interesting to: - Compare ZClient and XML-RPC for some realistic tests, like yours, - Compare them for simple Python methods that don't do much but return some data, something like: def test(x,y,z): return 'hello world' * 100 - If there is a big difference, use the profiler to try to figure out why XML-RPC is taking so long. Ok, in my spare time i'll cobble up a much better benchmark. -Michel ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] remote procedure calls to manage functions
andrew wrote: Have been delving into ZPublisher.Client to make remote procedure calls under Zope 2.2.1. Did someone slip 2.2.1 without mentioning it?! I thought XML-RPC was now favoured over ZClient? cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] remote procedure calls to manage functions
Chris Withers wrote: andrew wrote: Have been delving into ZPublisher.Client to make remote procedure calls under Zope 2.2.1. Did someone slip 2.2.1 without mentioning it?! I thought XML-RPC was now favoured over ZClient? It's not really favored, both are quite useful. xml-rpc is more for when you want two different system to interoperate. ZClient is very zope specific and probably gives you a bit more functionality than xml-rpc (because it has the 'remote object' abstraction). Also, ZClient is much faster i've found, probably due to the marshalling/unmarshalling necessary for xml-rpc. -Michel ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] remote procedure calls to manage functions
Michel Pelletier wrote: Chris Withers wrote: andrew wrote: Have been delving into ZPublisher.Client to make remote procedure calls under Zope 2.2.1. Did someone slip 2.2.1 without mentioning it?! I thought XML-RPC was now favoured over ZClient? It's not really favored, both are quite useful. xml-rpc is more for when you want two different system to interoperate. ZClient is very zope specific and probably gives you a bit more functionality than xml-rpc (because it has the 'remote object' abstraction). I don't think ZClient is really Zope specific. At least it's not supposed to be. It does have a richer API, including features like authentication and header support. Also, ZClient is much faster i've found, probably due to the marshalling/unmarshalling necessary for xml-rpc. That's interesting..Hm. ZClient has to marshal. I suspect that xml-rpc wants some sort of optimization. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! Technical Director (888) 344-4332http://www.python.org Digital Creationshttp://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] remote procedure calls to manage functions
Jim Fulton wrote: Michel Pelletier wrote: I thought XML-RPC was now favoured over ZClient? It's not really favored, both are quite useful. xml-rpc is more for when you want two different system to interoperate. ZClient is very zope specific and probably gives you a bit more functionality than xml-rpc (because it has the 'remote object' abstraction). I don't think ZClient is really Zope specific. At least it's not supposed to be. It does have a richer API, including features like authentication and header support. My bad, you're right. Also, ZClient is much faster i've found, probably due to the marshalling/unmarshalling necessary for xml-rpc. That's interesting..Hm. ZClient has to marshal. I suspect that xml-rpc wants some sort of optimization. I made a test script to make about 300 xmlrpc calls to various manage_ methods. I had to kill it after about a half-hour cuz it was takin so long. Here's the profiler output: ncalls tottime percall cumtime percall filename:lineno(function) 702883/2151 108.9800.000 288.5800.134 DT_InSV.py:379(__getitem__) 7704 79.0500.010 243.9000.032 xmllib.py:201(goahead) 3852 42.3000.011 43.1600.011 FileStorage.py:706(_finish) 158831/158827 38.0100.000 70.1400.000 DT_Var.py:258(render) 732178 37.5800.000 49.4500.000 re.py:112(match) 495239/51912 36.6900.000 307.2000.006 HTTPRequest.py:741(__getitem__) 586827 29.0400.000 29.0400.000 re.py:335(group) 499291 22.9000.000 30.5300.000 re.py:95(search) 875347 19.5000.000 19.5000.000 re.py:290(__init__) 45981 14.0900.000 39.0000.001 xmllib.py:539(parse_starttag) 748592 10.6800.000 10.6800.000 exceptions.py:65(__init__) 5419249.4200.0009.4200.000 re.py:297(start) 4562739.1900.0009.1900.000 re.py:306(end) 1078318.5200.0008.5200.000 urllib.py:898(quote) 127647/107577.4000.000 300.0800.028 :0(?) 491137.1600.0008.2400.000 User.py:140(getRolesInContext) 127647/43026.3500.000 303.9100.071 DT_Util.py:325(eval) 459815.3600.000 19.8700.000 xmllib.py:615(parse_endtag) 1112585.2400.0007.0300.000 xmllib.py:699(handle_charref) 38565.1000.001 17.0200.004 BaseRequest.py:224(traverse) 2845154.5100.0004.5100.000 xmlrpclib.py:387(data) I think you're right about optimizing, why is it spending most of it's time in DT_InSV? -Michel ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
