> 3) I need to dump this data (for all races, not just current about to > start race) to text files, store it as BLOBs in a DB *and* update real > time display in a wxpython windowed client.
A few important questions: 1) How real-time must the display be? (should update immediately after you get new XML data, or is it ok to update a few seconds later?). 2) How much data is being processed at peak? (100 records a second, 1000?) 3) Does your app need to share fetched data with other apps? If so, how? (read from db, download HTML, RPC, etc). 4) Does your app need to use data from previous executions? (eg: if you restart it, does it need to have a fully populated UI, or can it start from an empty UI and start updating as it downloads new XML updates). How you answer the above questionss determines what kind of algorithm will work best. David. PS: I suggest that you contact the people you're downloading the XML from if you haven't already. eg: it might be against their TOS to constantly scrape data (I assume not, since they provide XML). You don't want them to black-list your IP address ;-). Also, maybe they have ideas for efficient data retrieval (eg: RSS feeds). -- http://mail.python.org/mailman/listinfo/python-list
