On Tue, Sep 10, 2013 at 9:35 PM, Emmanuel Lécharny <[email protected]>wrote:
> Le 9/10/13 5:40 PM, Emmanuel Lécharny a écrit : > > Hi guys, > > > > yestrday and today, I was implementing a cache in Mavibot to replace the > > WeakReferences we were using previously. The rational was that with the > > WeakReferences, we were unable to inject more than 50 000 entries in the > > server (at this point, the GC is just going crazy trying to free some > > WeakReferences, and it slows down the server to a pint it's unusable...) > > > > So after a day fighting with an EhCache implementation in Mavibot, I was > > able to load 100K entries in the server. So far, so good, except that > > the performances are anything but good. I can add 26 entries per second, > > and fetch 555 entries per second. Worse than JDBM... > > > > Why is it so slow, especially for a search operation ? There are two > > reasons : > > - first, I configured the cache to store only 1000 elements (mainly > nodes) > > - second, when we try to update a leaf, we mostly have to load it from > > disk, as we rarely have it in memory > > - third, a leaf contains from 8 to 16 entries, and everytime we fetch a > > leaf from disk, we have to deserialize the Entries, which is extremely > > costly > > > > Fixing this third problem would save us a lot of time, and it's a matter > > of adding one level of indirection (the entries would be kept as byte[], > > and deserialized only when needed). > > > > If anyone has a better idea... > indeed : the pb is that we serialize/deserialize too late. It would be > I think you mean 'too _early_' > way better if we process serialized values until the point we return the > result to the user : > > - addition : we receive an object, we immediately serialize it into a > byte[] and process the addition using this byte[]. We don't anymore have > to deserialize all the values from the page we will add the new value, > they are all byte[] > - search : same thing, we don't deserialize the values until we return > it to the user. > > +1 > The gain will be huge ! > > -- > Regards, > Cordialement, > Emmanuel Lécharny > www.iktek.com > > -- Kiran Ayyagari http://keydap.com
