Hi Dustin. I have to go on my work now so I probably wont spend any time on this issue. Please, before you suggest, try some experiment to load more than 6 million records using the same API. I would be happy to hear how you do that. I now fully rely on Whalin's API, it can handle 14 million records without any problem.
Shi On Sun, Oct 17, 2010 at 1:45 AM, Dustin <[email protected]> wrote: > > On Oct 16, 9:28 pm, Shi Yu <[email protected]> wrote: >> Map<String,String> map1 = new HashMap<String,String>(); >> Map<String,String> map2 = new HashMap<String,String>(); >> Map<String,String> map3 = new HashMap<String,String>(); > > You're loading at least four million strings into two million hash > table entries on a 64-bit system. Each hash table entry contains a > pointer to a key, a pointer to a value, a pointer to another entry, > and an integer copy of the hash code that's there. That's a huge > amount of memory just to load up a generated set of data. > > If you instead either implemented an iterator as a generator to > dynamically do what that large map is doing or just used the > CacheLoader.push method in a way similar to how you were doing add > before, I suspect you'd have no problems and significantly less memory > consumption.
