Hi, 2009/5/14 Damien Elmes <[email protected]>: > > Processing single facts at a time will be memory efficient but very > slow. A more intelligent solution would be to place an upper bound on > the number of facts processed at once. >
2009/5/14 Peter Bienstman <[email protected]>: > >> > An alternative would >> > be an incremental JSON parser, but that may be complicated to >> > implement. >> >> Not that I know anything about this, but would there be a difference >> JSON/XML in that regard? > > I stumbled upon this: > > http://effbot.org/zone/element-iterparse.htm > > This seems like a very efficient way of doing an incremental XML parser. > Moreover, it has the advantage that the C-compiled cElementTree module is > available for the Python 2.5 installation on Windows Mobile, whereas JSON > would be a Python only separate install there, and therefore much slower. > > That seems to point rather strongly to XML over JSON, but I'd appreciate > feedback from people who have more experience in these matters. > I sketched out test script for comparing xml-rpc and json-rpc transports to investigate this topic a bit deeper. Script sends cards and related facts from client to server using different protocols packing them into packs(list of cards and related facts). Protocol, amount of cards to send and amount of cards per pack are arguments of the script. Cards and facts are artificial, but their structure and field sizes were taken from current development version of Mnemosyne. I looked at the anki sources and used it's approach, but different protocol. I also used this implementation of SimpleJSONRPCServer: http://code.activestate.com/recipes/552751/ with small modification. It's included into the script to simplify its usage. The only external dependency is simplejson module, which you can install from pypi: http://pypi.python.org/pypi/simplejson/ A couple of command line examples: 1. run json-rpc client connecting to server on 192.168.255.17:8000 and sending 10000 cards. One pack contains 1000 cards and related facts. deck1 is the name of the deck. It should be different from one defined for server. synctest.py client json-rpc http://localhost:8000 deck1 10000 1000 2. run json-rpc server listening on 192.168.255.17:8000: synctest.py server json-rpc 192.168.255.17 8000 deck2 10000 1000 deck2 is the name of the deck. It should be different from the one defined for client. Amount of cards and cards/pack is not used at the moment, because server only gets data from client but doesn't send anything. Feel free to play with the script and modify it whatever way you want. You can get the source from my master branch on github, or look at it and download from here: http://github.com/bartosh/pomni/blob/609c579e09f85e12cf4c8b832e00fd3f4a13a319/mnemosyne/mnemosyne/pomni/scripts/synctest.py Below is the link to test results table for two major cases: 1. server and client run on one host(X60s lenovo laptop) and communicate through loopback interface. Test results of this case have 'loopback' type in the table. 2. server run on the same laptop and client run on N810 handheld device (TI OMAP 2420, 400Mhz arm processor, 128Mb RAM). Both server and client in the same WLAN with almost no other traffic in it. This case has type 'WLAN 802.11g'. http://www.assembla.com/wiki/show/pomni/Sync_Test I'm not going to draw any conclusions in purpose. Instead looking forward to hearing your comments. -- BR, Ed --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "mnemosyne-proj-users" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/mnemosyne-proj-users?hl=en -~----------~----~----~----~------~----~------~--~---
