If i'm understanding this suggestion correctly, the main change in observable behavior will be that actions performed by a "reader" will never block or invalidate actions performed by a "writer" -- writers on the other hand can still block eachother.
This seems like it might be the opposite of what most people would want: that opening "reader" threads for doing searches need to be fast, and if a writer thread has to wait a half second that's okay. I also don't believe this would "solve" the NFS issues with regards to the commit lock -- as i recall, the problem stems from NFS not being able to garuntee transactional order of file operations (ie: i open the commit lock file, i modify and close segments, i close/delete the commit file -- a remote NFS client might still see the orriginal segments file after the commit file is deleted. Your version file might suffer the same fate (with reader clients seeing V1==V2 because the whole file is a second stale) : Date: Thu, 24 Aug 2006 23:22:56 -0700 : From: Doron Cohen <[EMAIL PROTECTED]> : Reply-To: java-dev@lucene.apache.org : To: java-dev@lucene.apache.org : Subject: Re: Lock-less commits : : I would like to discuss an additional approach, that requires small changes : to current Lucene implementation. Here, the index version (currently in : segments file) is maintained in a separate file, and is used to synchronize : between readers and writers, without requiring readers to create/obtain any : lock files, and without requiring readers to write anything to disk. : : - Index version would be maintained in a separate, dedicated Version file - : (say .vsn) - one per index. : - Version file contains two occurrences of the version number - V1 and V2. : - In steady state, V1 == V2, but During update V1 == V2+1. : - Every commit would: : - obtain a write lock (as today), to guarantee single writer at a time. : - increments V1 in that file, using RandomAccessFile API (RAF). : notice: now V1 != V2. : - do the commit work (merge, delete, whatever). : - increments V2 in that file, using RAF. : notice: now, again, V1 == V2. : - release the write lock (as today) : - Every reader would read the version data in opposite order: : (1) read V2 from the version file, using RAF. : (2) read V1 from the version file, using RAF. : (3) if not V1==V2 wait some time, and try again (from step 1), until : V1==V2, or timeout and fail. : (4) initialize reader data (read segment infos, open files). : (5) read again V2 then V1 using RAF. : (6) if not V2==V1 or they changed from steps 1 and 2, try again (from : step 1), or timeout and fail. : : : A few points to notice: : - Using RAF protects from errors due to IO buffering. : - Only tiny amount of version data is being read/written using RAF, so : performance should not degrade. : - Readers are not writing any data, so they are faster (A reader that does : deleteDoc is a writer in this regard). : - The opposite read/write order of RAF operations, i.e. writing V1 and then : V2 by writer but reading V2 and then V1 by reader, protects from race : conditions between readers and writers that otherwise might have caused : reading corrupt data and concluding wrongly that the data is consistent : while in fact it is not. : - By using RAF and by the order of operations above, this scheme would work : also for NFS (excluding the write lock mechanism which remains an issue in : NFS). : - For backward compatibility with current index structure the code can fall : back to obtain a commit lock file if the .vsn file does not exist, and then : create that .vsn file if it still does not exist. : : Regards, : Doron : : : : --------------------------------------------------------------------- : To unsubscribe, e-mail: [EMAIL PROTECTED] : For additional commands, e-mail: [EMAIL PROTECTED] : -Hoss --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
