I don't think that anyone is crazy enough to load a whole, huge dataset into memory. People try to avoid this because you'd be spending a lot of time just to get the data in memory. And we're talking about a delay range in seconds or perhaps even minutes.
I do know that by using a memory mapped flat file, that you can use a whole file as if it's loaded into memory. It's a bit complex technique which I'm not really familiar with but apparantly it does provide a huge speed (since it doesn't really load all the data in memory) combined with a huge amount of records. But in your case you have dozens of SQL queries running to get all the data. So I think it would be better to prevent the user from expanding the whole tree with a shortcut or whatever. Or otherwise ask them if they really want to expand all, since it will be time- consuming. And if they still want to expand all, try to show a progressbar at the same time. And consider this: not even Google immediately returns all 500.000 results when you search for something. At the most, you get 100 records and an option to just to a next page. It would just be too slow for the user. And worse, once you have all those thousands of records in memory, you will have to deal with changes in the database. What if you have two users active at the same time, both looking at the same data. One makes a modification, but since the other has all data already in memory they won't notice the change since you never return to the database to get the 'fresh' data. In general, it's a very bad idea... Greetings, Lisa. --- In [email protected], "Jamie L. Mitchell" <[EMAIL PROTECTED]> wrote: > > Hello all: > > I am working with what will likely become a very large data set and a > Virtual Tree (TVirtualStringTree.) The attractive thing about this VT > is that it may have hundreds of thousands of nodes, but it only loads > the nodes with data at the point in time that it is needed - usually. > Due to the hierarchical structure of the tree, and the just-in-time > loading of the nodes throughout the hierarchy, I have a variety of SQL > Select statements, each filtered to bring in just the node data that > is needed. > > Unfortunately, there may be times that the user may decide to load the > whole tree at once. Such times include using the short-cut keys > (Ctrl-Shift-Plus) to load the whole tree, or when the user wants to do > a search through the entire tree looking for specific data. > > If a full load is triggered, and there is a large number of nodes, it > is possible that the app will run thousands of separate queries > against the data base during the load. This - to be mild - is a huge > time waster. I would like to take the special case where a full load > is going to happen and load the entire data set into memory and then > run the queries against the data in memory. I figure this should > reduce the network time and reduce load time by a huge amount. > > I am looking for an example where a data set is fully loaded into > memory and then used as a basis for subsequent queries. Anyone do > this? I am using Delphi 5, Adonis ADO, and both Access and SQL Server > back ends (I assume really large installations will be SQL Server.) > > Thanks > jamie > ----------------------------------------------------- Home page: http://groups.yahoo.com/group/delphi-en/ To unsubscribe: [EMAIL PROTECTED] Yahoo! Groups Links <*> To visit your group on the web, go to: http://groups.yahoo.com/group/delphi-en/ <*> To unsubscribe from this group, send an email to: [EMAIL PROTECTED] <*> Your use of Yahoo! Groups is subject to: http://docs.yahoo.com/info/terms/
