ivankelly opened a new pull request #913: [WIP] Refactor FileInfo locking and refcounting out IndexPersistenceMgr URL: https://github.com/apache/bookkeeper/pull/913 There is a number of bugs in index persistence mgr related to how refcounts are handled and how FileInfos are freed, due to the locking mechanism. These can result in the fencing bit being lost in ledgers, which is a serious issue. They also cause flakes in IndexPersistenceMgrTest#testEvictFileInfoWhenUnderlyingFileExists. For details see the discussion at: https://github.com/apache/bookkeeper/pull/513/files/8075b0#r156676238 There are two key problems. 1. FileInfos are flushed asynchronously on eviction. If another thread tries to read the FileInfo after the flush has been scheduled, but before it runs, it will read stale date (and we'll possibly lose the fence bit). 2. FileInfos can be closed while it is still references in the IndexPersistenceMgr. This means that it can be fenced, but this will very be flushed to disk because the FileInfo is already closed. The patch solves this by moving FileInfo locking and refcount into a separate class, FileInfoBackingCache. When either the write cache or read cache load a file info, it tries to load it from this backing cache. All writes to the the backing cache are done under a write lock. This class also takes care of reference counting. It hands out CachedFileInfo objects, which will be flushed to disk when all references are released.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
