Sub-Subject: Sad man seeks life on long weekend. I'm puttering around with XML-RPC, which is all relatively straight forward and such, and have come across an issue that I've struck before in the past and never adequately resolved, and that is the burning issue of today - "Scalable APIs".
I want to provide an access method for a large list of objects (say, Customers, or Accounts, or records of some type - they will be uniform types). If I provide a method that says "return list", then it may crunch away on a a very large dataset and occupy: 1) a large amount of bandwidth traversing the network (although if the data is required, the data is required), so this reason is a bit of a furfy. B) a large amount of time to transfer the result (variation on 1, but more important). III) a large amount of memory to store the result on the client. (also an inability to stick to a consistent numbering format, but that's the fault of the author's). B & III are the ones I'm most concerned about. If the results were driving (for example) a GUI listbox, I'd like the results to start populating that immediately rather than waiting for the entire result set to be returned. If I'm driving an export or conversion process then I don't need to store the entire result set on the client, I can process each record as it comes in. Cool, so some form of iterator would probably be ideal. Unfortunately, to use an iterator I would need to retain some form of state (insert dramatic chord: or would I?) on the server, which, like cliches, I try to avoid like the plague. I could probably "cheat" (although I'm not really sure that it's cheating) and provide an interface that not only returns the data in question, but some form of nonce that I could use to retrieve the next (or next 'x') records from. The nonce could probably even be the primary key of the next record, if it were against a database table that had primary keys; I tend to use views a lot (which may not have primary keys), and the result set may not even be the output of a database. So I was wondering if anyone has ever come across this problem, found any naff solutions, or sunk any thought into the problem? -- Regards, Daryl Tester, IOCANE Pty. Ltd. _______________________________________________ sapug mailing list [email protected] http://mail.python.org/mailman/listinfo/sapug
