I'm wondering if I should implement this as a utility function or try to fit it into DataContext.performQuery() via setPageSize().
I'm going to take a stab at implementing it as a helper function. Perhaps it can be retrofitted into IncrementalFaultList once it has matured. The advantage of doing it as a helper function is that it can easily be copy & pasted into the application for modification & adaptation to application access patterns. A helper function should be much more straightforward as opposed to the current implementation which is a bit messy in terms of the number of concepts that it introduces. Cayenne can try to come up with a good heuristic for generic access patterns when setPageSize() is used, but of course the application has special knowledge + at some point the application has to be adapted to fit an access pattern that gives reasonble memory usage & performance tradeofs. The current IncrementalFaultList is nice in that it stores everything in memory, it just fetches stuff one page at the time as is pretty clear from the documentation. The attached implementation goes in a very different direction, it does not try to keep things in memory, but is rather optimized for a single iteration in a linear fashion. Taking handling of large tables a step further, support could be added for iterating over queries where even the pk's won't fit into memory(heap could be limited or tables could be large). -- Øyvind Harboe http://www.zylin.com/zy1000.html ARM7 ARM9 XScale Cortex JTAG debugger and flash programmer
