hi Adam, Thanks for your work. Cache layer is critical for ofbiz as ERP system.
Just one quick question: There are a lot of cache frameworks (Personally I like Apache JCS very much) out there. Why ofbiz has to have its own implement? In my opinion, we have two ways to reduce bugs. Write more test codes, or write less product codes. Maybe we can replace ofbiz components with some existing components (say cache) or separate ofbiz components as sub projects or just totally new project to attract more projects to use (say, entity engine). -- Regards, Michael Xu (xudong) On Mon, Mar 22, 2010 at 11:47 PM, Adam Heath <[email protected]> wrote: > So, for the past week, I've been working on automated test cases for > UtilCache. I can say that I've got the hardest parts done, namely, > ttl and ref based testing. > > While writing the test cases, I discovered several bugs. One of the > biggies was very bad LRU handling. This required some major > rewriting, so that ttl and ref clearing is no longer polled. There > are background threads that magically remove items as soon as they are > available for clearing. This can actually reduce memory, because now > as soon as tll or ref fires, the item gets removed from the cache, > instead of waiting until some time later when a get is called. > > Additionally, CacheLineTable is no longer around, I've merged it into > UtilCache directly. I consider any of the CacheLine classes to be > completely internal, and should not be used outside of UtilCache. > Since expiration and invalidation are no longer a polled system, those > methods actually no longer exist. So any such methods that were > involved with CacheLine management I am planning on removing. > > Below are the list of bugs that I have completely identified, and > fixed. There may be more, and my explanation may not be entirely > accurate. I should have this ready to be committed by the end of the > week. Afterwords, I'll finally start writing docs on all the > git/testing stuff I've been talking about. > > == > * BUG: LRU out of order > * BUG: containsKey() calls get(), which records an access > * BUG: clear() notes removal before calling clear, and calls > get, which records an access in LRU mode. > * BUG: With a positive expireTime, setting to any other positive > value has no effect. > * BUG: when an item was expired due to a ttl, or invalidated > when a soft ref was cleared, the disk store would have the > item removed as well. So the disk store actually gave no > benefit whatsoever. > * BUG: CacheLineTable stored CacheLine instances in the file > store, and didn't handle changing them to the proper reference > when reading from disk, if the reference type had been changed. > If the ref type was changed, then the disk-based ref type would > be changed, but just doing a fetch from disk wouldn't handle > the ref type change. It is possible that upon reading the > serialized CacheLine from disk, that a soft ref would be > immediately cleared. > * BUG: CacheLineTable synchronized on (this) when creating the > static, shared jdbmMgr; never noticed in production, as ofbiz > startup is single threaded. > * BUG: CacheLineTable created a global jdbmMgr, using the > per-cache fileSystemStore location. > * BUG: Using a fileStore without SoftReference makes no sense. > All objects might have a ttl attached, which caused memory and > disk access for put/remove, or, without a cache, the memory > store would always be the same as the disk store. > * BUG: CacheLineTable.remove called fileTable.remove, but didn't > call jdbmMgr.commit(). > * BUG: getCacheLineKeys(), and other collection based > meta-methods, didn't handle invalidation or expiration. > * FEATURE: CacheLineTable didn't add fetched values from disk > into memory, to take advantage of LRU and soft ref processing. >
