On 2 Mar 2006, at 11:17, Kevin Windham wrote: On Mar 2, 2006, at 10:44 AM, Trausti Thor Johannsson wrote:
Currently, I have an Object that takes an sql command, queries, and then read all into a dictionary containing individual objects. 90.000 records, it does take time to read on, and create objects. I would think that reading in the whole shebang would cause out of memory or something bad happening, having millions of objects in memory. At least it would take an awful long time. What you would do is use a server side cursor if available on your DB server. If not you could fudge your own. Essentially you select on your table the objects you want. Then you would create a buffer of objects that were needed immediately maybe 100 or a thousand or whatever. Then in your object loader you'd build a mechanism for fetching the object from the server's cursor as needed in batches of the buffer size. Of course this wouldn't work well for every scenario, but for most scenarios this works quite well. Cursors are indeed your friend, grab just the data you need to output one row at a time, the client needn't hold much data at all. Who can look at 90,000 objects at once anyway. This is the crux of the matter, there is no scenario that I can think of that requires 30,000,000 records to be available to the user at once, there aren't any methods of display that allow one to comprehend that volume of data. You may want to aggregate some fields, you may even want to print out every row (ouch), but it should always be possible to filter data on the server and return a sub-set or pull back results with a cursor row by row. I've worked with databases with 100s of millions of rows in a table. With the right tools and an optimized database (indexes built properly, locking and memory parameters correctly tuned etc) and server hardware that can deal with it, you can easily work effectively with the data. REALbasic is more than capable of working with huge databases, as long as the database can return cursor data you're all set, if it supports random access cursors you're laughing, but it's always possible to get around it if it's not possible. I for one have been using the Data-On-Demand ListBox just recently, it's working pretty well for me, haven't stress tested it yet but I'm liking what I'm seeing. -- Ian M. Jones ___________________________________ IMiJ Software http://www.imijsoft.com http://www.ianmjones.net (blog) _______________________________________________ Unsubscribe or switch delivery mode: <http://www.realsoftware.com/support/listmanager/> Search the archives of this list here: <http://support.realsoftware.com/listarchives/lists.html>
