Hi Peter, 2009/5/18 Peter Bienstman <[email protected]>: > > (Let's move this to devel) > > Good job, Ed! > > Some more questions: > > * did you compress the data? I guess xml is a bit more verbose, but > compression should probably remedy that I tried compression with json-rpc, but it didn't bring anything. The reason for that is both protocols are text and any binary data get converted to the text format(base64, I believe) before sending. It increases size and adds cpu load There is always possibility to use our own protocol, like anki does, but it wasn't the aim of my investigation. I mainly investigated difference between xml and json usage.
> * did you look at memory usage? A protocol that is fast on desktop machines, > but that requires the entire sync exchange to be kept in memory will be > useless for a mobile device. > No I didn't. If you look at the results you can find out that difference in speed between 100 cards/pack and 10000 cards/pack is not so big even when working through loopback, so there is no big need in sending the whole deck at once. > Peter > > On Sunday 17 May 2009 09:23:06 pm Ed Bartosh wrote: >> Hi, >> >> 2009/5/14 Damien Elmes <[email protected]>: >> > Processing single facts at a time will be memory efficient but very >> > slow. A more intelligent solution would be to place an upper bound on >> > the number of facts processed at once. >> >> 2009/5/14 Peter Bienstman <[email protected]>: >> >> > An alternative would >> >> > be an incremental JSON parser, but that may be complicated to >> >> > implement. >> >> >> >> Not that I know anything about this, but would there be a difference >> >> JSON/XML in that regard? >> > >> > I stumbled upon this: >> > >> > http://effbot.org/zone/element-iterparse.htm >> > >> > This seems like a very efficient way of doing an incremental XML parser. >> > Moreover, it has the advantage that the C-compiled cElementTree module is >> > available for the Python 2.5 installation on Windows Mobile, whereas JSON >> > would be a Python only separate install there, and therefore much slower. >> > >> > That seems to point rather strongly to XML over JSON, but I'd appreciate >> > feedback from people who have more experience in these matters. >> >> I sketched out test script for comparing xml-rpc and json-rpc >> transports to investigate this topic a bit deeper. Script sends cards >> and related facts from client to server using different protocols >> packing them into packs(list of cards and related facts). Protocol, >> amount of cards to send and amount of cards per pack are arguments of >> the script. Cards and facts are artificial, but their structure and >> field sizes were taken from current development version of Mnemosyne. >> >> I looked at the anki sources and used it's approach, but different >> protocol. I also used this implementation of SimpleJSONRPCServer: >> http://code.activestate.com/recipes/552751/ with small modification. >> It's included into the script to simplify its usage. The only external >> dependency is simplejson module, which you >> can install from pypi: http://pypi.python.org/pypi/simplejson/ >> >> A couple of command line examples: >> 1. run json-rpc client connecting to server on 192.168.255.17:8000 and >> sending 10000 cards. One pack contains 1000 cards and related facts. >> deck1 is the name of the deck. It should be different from one defined >> for server. >> synctest.py client json-rpc http://localhost:8000 deck1 10000 1000 >> 2. run json-rpc server listening on 192.168.255.17:8000: >> synctest.py server json-rpc 192.168.255.17 8000 deck2 10000 1000 >> deck2 is the name of the deck. It should be different from the one >> defined for client. >> Amount of cards and cards/pack is not used at the moment, because >> server only gets data from client but doesn't send anything. >> >> Feel free to play with the script and modify it whatever way you want. >> You can get the source from my master branch on github, or look at it >> and download from here: >> http://github.com/bartosh/pomni/blob/609c579e09f85e12cf4c8b832e00fd3f4a13a3 >>19/mnemosyne/mnemosyne/pomni/scripts/synctest.py >> >> Below is the link to test results table for two major cases: >> 1. server and client run on one host(X60s lenovo laptop) and >> communicate through loopback interface. Test results of this case have >> 'loopback' type in the table. >> 2. server run on the same laptop and client run on N810 handheld >> device (TI OMAP 2420, 400Mhz arm processor, 128Mb RAM). Both server >> and client in the same WLAN with almost no other traffic in it. This >> case has type 'WLAN 802.11g'. >> http://www.assembla.com/wiki/show/pomni/Sync_Test >> >> I'm not going to draw any conclusions in purpose. >> Instead looking forward to hearing your comments. > > ------------------------------------------------ > Peter Bienstman > Ghent University, Dept. of Information Technology > Sint-Pietersnieuwstraat 41, B-9000 Gent, Belgium > tel: +32 9 264 34 46, fax: +32 9 264 35 93 > WWW: http://photonics.intec.UGent.be > email: [email protected] > ------------------------------------------------ > > > > -- BR, Ed --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "mnemosyne-proj-users" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/mnemosyne-proj-users?hl=en -~----------~----~----~----~------~----~------~--~---
