I have also tried the CacheLoader API, it pops a java GC error. The thing I haven't tried is to separate 6 million records into several objects and try CacheLoader. But I don't think it should be that fragile and complicated. I have spent a whole day on this issue, now I just rely the hybrid approach to finish the work. But I would be very interested to hear any solution to solve this issue.
Shi On Sat, Oct 16, 2010 at 8:33 PM, Dustin <[email protected]> wrote: > > On Oct 16, 5:27 pm, Shi Yu <[email protected]> wrote: > >> I am really curious why spy cannot store up to 6 million... > > I'm quite sure you can add more than 6M items to something. Part of > the problem might be that you're adding to an in-memory queue as fast > as possible, not checking results, or slowing down when you blow past > maximum queue depths. That's not "normal" use of the simple cache > operation APIs. > > If you just want to go fast, use the CacheLoader API: > > > http://dustin.github.com/java-memcached-client/apidocs/net/spy/memcached/util/CacheLoader.html > > That will ensure you're balancing the speed of the fast loop in the > JVM against the time it takes to get over the network.
