Thanks for helping narrow down the problem . . . I'm also considering putting in a check in the read method. If the keys don't match, I'll return null and log an error.
Aaron --- On Wed, 8/12/09, Tim Cronin <tim.cro...@autonomy.com> wrote: > From: Tim Cronin <tim.cro...@autonomy.com> > Subject: RE: index cache corruption > To: "JCS Users List" <jcs-users@jakarta.apache.org> > Date: Wednesday, August 12, 2009, 7:28 AM > Amen! > > Thanks for the quick response to my issues. > > > -----Original Message----- > From: Josh Szmajda [mailto:jos...@gmail.com] > > Sent: Wednesday, August 12, 2009 9:17 AM > To: JCS Users List > Subject: Re: index cache corruption > > Aaron I just wanted to say how thankful I am to have such a > committed > maintainer for JCS! Thanks for all your help, this project > is rock solid > because of you. :) > > On Wed, Aug 12, 2009 at 10:11 AM, Aaron Smuts <asm...@yahoo.com> > wrote: > > > I think I found it. During partial key removeall > the Disk Element > > Descriptor, which identifies the slot on disk, was > being added to the > > recylebin. Then the removeSingleItem was called > for each match. The > remove > > single item method also adds the item to the > recylebin. The > recyclebin is > > not a set. The spot would get entered > twice. This is very bad. > Here's > > why: > > > > The spot gets used. Another put uses the same > spot. Both keys in the > key > > map point to the same spot. A get for the first > key will pull the > data for > > the second. Not so good. > > > > I'm verifying this and I'll have a fix soon. > > > > Aaron > > > > --- On Wed, 8/12/09, Aaron Smuts <asm...@yahoo.com> > wrote: > > > > > From: Aaron Smuts <asm...@yahoo.com> > > > Subject: RE: index cache corruption > > > To: "JCS Users List" <jcs-users@jakarta.apache.org> > > > Date: Wednesday, August 12, 2009, 6:52 AM > > > > > > So were you getting back the wrong data or data > that should > > > have been deleted? > > > > > > Are you running a recent temp build? > > > > > > I'll dig into the partial key removal. This > should > > > probably deprecated for the remove matching > method. . . > > > . Either way, it should work. > > > > > > Aaron > > > > > > --- On Tue, 8/11/09, Tim Cronin <tim.cro...@autonomy.com> > > > wrote: > > > > > > > From: Tim Cronin <tim.cro...@autonomy.com> > > > > Subject: RE: index cache corruption > > > > To: "JCS Users List" <jcs-users@jakarta.apache.org> > > > > Date: Tuesday, August 11, 2009, 2:30 PM > > > > Good to know, thanks. > > > > > > > > Yes that's when we see it too, under high > load. > > > > > > > > For us it seems to happen around removes, we > use the > > > key: > > > > linkage and > > > > It happens to data that's linked and would > get cleared > > > with > > > > the key: > > > > call. > > > > > > > > > > > > -----Original Message----- > > > > From: Al Forbes [mailto:forbes...@googlemail.com] > > > > > > > > Sent: Tuesday, August 11, 2009 4:13 PM > > > > To: JCS Users List > > > > Subject: Re: index cache corruption > > > > > > > > Hi Tim, > > > > > > > > I had the same problem. It seems to happen > under high > > > load, > > > > with fairly > > > > large objects, but I could not reproduce it > offline. > > > I > > > > changed to using > > > > a > > > > JDBC cache and have not had a problem since > then. > > > > > > > > We still use disk caches for more static > data - mostly > > > read > > > > only, and > > > > the > > > > problem never happens there, so I assume > it's the > > > writes > > > > that cause the > > > > corruption. > > > > > > > > Regards > > > > Al > > > > > > > > 2009/8/11 Tim Cronin <tim.cro...@autonomy.com> > > > > > > > > > We initially were using indexed disk > cache but > > > ran > > > > into cache > > > > corruption > > > > > where the data returned for a key would > not be > > > the > > > > data associated for > > > > > that key. I haven't been able to come > up with a > > > good > > > > repro scenario... > > > > > > > > > > We had to work around it with the > following > > > code: > > > > > > > > > > /** > > > > > * is the cache element > not the > > > > correct object for the key > > > > > * @param key the > current key > > > > > * @param element the > current element > > > > associated with the key > > > > > * @param hasReadLock > whether caller > > > > has read lock > > > > > * @return true if key > mismatch els > > > > false > > > > > * @throws > InterruptedException if > > > > locking fails > > > > > */ > > > > > private boolean isCorrupt(String > key, > > > > ICacheElement element, boolean > > > > > hasReadLock) throws > InterruptedException > > > > > { > > > > > boolean corrupt = > > > > !key.equals(element.getKey()); > > > > > if (corrupt) > > > > > { > > > > > > mLogger.error("cache corruption!!! > > > > [" + (mCorruptionCounter++) + > > > > > "]"); > > > > > > > > > > if > (mLogger.isDebugEnabled()) > > > > > { > > > > > > mLogger.error("culprit > > > > stack...", new Exception("cache > > > > > corruption")); > > > > > } > > > > > > > > > > try > > > > > { > > > > > if > (hasReadLock) > > > > > { > > > > > > > > > mLock.readLock().release(); > > > > > } > > > > > > > > > mLock.writeLock().acquire(); > > > > > try > > > > > { > > > > > > > > > mLogger.error("purging " + key); > > > > > > mCache.remove(key); > > > > > > mCache.remove(key + > > > > ":"); > > > > > } > > > > > catch > (Exception e) > > > > > { > > > > > > > > > mLogger.error("failed to purge key " + key, > e); > > > > > } > > > > > try > > > > > { > > > > > > String k = > > > > (String)element.getKey(); > > > > > > > > > mLogger.error("purging " + k); > > > > > > mCache.remove(k); > > > > > > mCache.remove(k + > > > > ":"); > > > > > } > > > > > catch > (Exception e) > > > > > { > > > > > > > > > mLogger.error("failed to purge key " + > > > element.getKey(), > > > > e); > > > > > } > > > > > } > > > > > finally > > > > > { > > > > > if > (hasReadLock) > > > > > { > > > > > > > > > mLock.readLock().acquire(); > > > > > } > > > > > > > > > mLock.writeLock().release(); > > > > > } > > > > > } > > > > > return corrupt; > > > > > } > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: jcs-users-unsubscr...@jakarta.apache.org > > > > > For additional commands, e-mail: > jcs-users-h...@jakarta.apache.org > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: jcs-users-unsubscr...@jakarta.apache.org > > > > For additional commands, e-mail: jcs-users-h...@jakarta.apache.org > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: jcs-users-unsubscr...@jakarta.apache.org > > > For additional commands, e-mail: jcs-users-h...@jakarta.apache.org > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: jcs-users-unsubscr...@jakarta.apache.org > > For additional commands, e-mail: jcs-users-h...@jakarta.apache.org > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: jcs-users-unsubscr...@jakarta.apache.org > For additional commands, e-mail: jcs-users-h...@jakarta.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: jcs-users-unsubscr...@jakarta.apache.org For additional commands, e-mail: jcs-users-h...@jakarta.apache.org
