On 12 April 2010 07:35, Mike Caron <[email protected]> wrote: > > > Ralph Versteegen wrote: >> >> On 11 April 2010 13:49, <[email protected]> wrote: >>> >>> pkmnfrk >>> 2010-04-10 18:49:08 -0700 (Sat, 10 Apr 2010) >>> 333 >>> RELOAD now uses a hashtable to speed up working with the string table. >>> >>> I am aware that util.bas has a hashtable implementation, but RELOAD's >>> uses its private heap, if applicable. >>> >>> Also, added a MemoryUsage function, which tallies up all the memory used >>> on the aforementioned private heap. It's potentially very slow, so be >>> careful! >>> --- >>> U wip/reload.bas >>> U wip/reload.bi >>> U wip/reloadtest.bas >> >> Good god. THAT is NOT a hash function. You could have at least reused >> the string hashing function from util.bas, strhash. > > I actually forgot to check to see what yours was. I haven't had a chance to > profile this one. > >> While using a private heap is cool, I'm not convinced that it was >> worth the effort. The effect will certainly be dwarfed by the speed >> increases due to using zstrings, a hash table, and interning strings >> in the string table instead of copying them. Have you timed the >> difference? :) > > I have not. The problem is that for documents that are large enough to reap > the benefits of having a private heap, RELOAD is all but unusable! > > Which is strange, since I took precautions to the contrary...
You mean that it's unusably slow? Well I guess that explains why you're spending so much time optimising, but how slow is unusable? >> Because other than the difference memory allocation functions, I don't >> think I see any reason for a new hashtable implementation. > > In C++, we could template it, and pass in a memory allocating function. > Alas, poor -lang deprecated... There's only a single callocate and deallocate there, wouldn't have been hard to generalise it. > Anyway, the other reason I did it this way is because I don't like all that > byref stuff. I'd rather use pointers any day, since it makes it explicit > what is modifiable and what isn't. I think I made 'this' byref because it was intented that the end user would normally be using a HashTable, not a HashTable ptr. As for hash_iter, looking back I might disagree with my use of byref there. >> Also, I haven't picked through all this RELOAD stuff properly yet, but >> now that you're using a hashtable do you still need Doc.strings[]? > > Currently, I do, since the hashtable contains indicies into the string > table. I spit out the string table when I write the binary file, and read it > in again when I load it. Maybe I could do away with it, but it's not that > big a deal... sizeof(zstring ptr) * numStrings isn't that big a deal. OK, better/easier to leave it then. > -- > Mike _______________________________________________ Ohrrpgce mailing list [email protected] http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
