Hello, Together with the Infinispan Directory we developed such a LockFactory; I'd me more than happy if you wanted to add some pointers to it in the Lucene documention/readme. This depends on Infinispan for multiple-machines communication (JGroups, indirectly) but it's not required to use an Infinispan Directory, you could combine it with a Directory impl of choice. This was tested with the LockVerifyServer mentioned by Michael McCandless and also with some other tests inspired from it (in-VM for lower delay coordination and verify, while the LockFactory was forced to use real network communication).
While this is a technology preview and performance regarding the Directory code is still unknown, I believe the LockFactory was the most tested component. free to download and inspect (LGPL): http://anonsvn.jboss.org/repos/infinispan/trunk/lucene-directory/ Regards, Sanne 2009/11/27 Michael McCandless <luc...@mikemccandless.com>: > I think a LockFactory for Lucene that implemented the ideas you & > Marvin are discussing in LUCENE-1877, and/or the approach you > implemented in the H2 DB, would be a useful addition to Lucene! > > For many apps, the simple LockFactory impls suffice, but for apps > where multiple machines can become the writer, it gets hairy. Having > an always correct Lock impl for these apps would be great. > > Note that Lucene has some basic tools (in oal.store) for asserting > that a LockFactory is correct (see LockVerifyServer), so it's a useful > way to test that things are working from Lucene's standpoint. > > Mike > > On Fri, Nov 27, 2009 at 9:23 AM, Thomas Mueller > <thomas.tom.muel...@gmail.com> wrote: >> Hi, >> >> I'm wondering if your are interested in automatically releasing the >> write lock. See also my comments on >> https://issues.apache.org/jira/browse/LUCENE-1877 - I thought it's a >> problem worth solving, because it's also in the Lucene FAQ list at >> http://wiki.apache.org/lucene-java/LuceneFAQ#What_is_the_purpose_of_write.lock_file.2C_when_is_it_used.2C_and_by_which_classes.3F >> >> Unfortunately there seems to be no solution that 'always works', but >> delegating the task and responsibility to the application / to the >> user is problematic as well. For example, a user of the H2 database >> (that supports Lucene fulltext indexing) suggested to automatically >> remove the write.lock file whenever the file is there: >> http://code.google.com/p/h2database/issues/detail?id=141 - sounds a >> bit dangerous in my view. >> >> So, if you are interested to solve the problem, then maybe I can help. >> If not, then I will not bother you any longer :-) >> >> Regards, >> Thomas >> >> >> >>> > > shouldn't active code like that live in the application layer? >>> > Why? >>> You can all but guarantee that polling will work at the app layer >> >> The application layer may also run with low priority. In operating >> systems, it's usually the lower layer that have more 'rights' >> (priority), and not the higher levels (I'm not saying it should be >> like that in Java). I just think the application layer should not have >> to deal with write locks or removing write locks. >> >>> by the time the original process realizes that it doesn't hold the lock >>> anymore, the damage could already have been done. >> >> Yes, I'm not sure how to best avoid that (with any design). Asking the >> application layer or the user whether the lock file can be removed is >> probably more dangerous than trying the best in Lucene. >> >> Standby / hibernate: the question is, if the machine process is >> currently not running, does the process still hold the lock? I think >> no, because the machine might as well turned off. How to detect >> whether the machine is turned off versus in hibernate mode? I guess >> that's a problem for all mechanisms (socket / file lock / background >> thread). >> >> When a hibernated process wakes up again, he thinks he owns the lock. >> Even if the process checks before each write, it is unsafe: >> >> if (isStillLocked()) { >> write(); >> } >> >> The process could wake up after isStillLocked() but before write(). >> One protection is: The second process (the one that breaks the lock) >> would need to work on a copy of the data instead of the original file >> (it could delete / truncate the orginal file after creating a copy). >> On Windows, renaming the file might work (not sure); on Linux you >> probably need to copy the content to a new file. Like that, the awoken >> process can only destroy inactive data. >> >> The question is: do we need to solve this problem? How big is the >> risk? Instead of solving this problem completely, you could detect it >> after the fact without much overhead, and throw an exception saying: >> "data may be corrupt now". >> >> PID: With the PID, you could check if the process still runs. Or it >> could be another process with the same PID (is that possible?), or the >> same PID but a different machine (when using a network share). It's >> probably more safe if you can communicate with the lock owner (using >> TCP/IP or over the file system by deleting/creating a file). >> >> Unique id: The easiest solution is to use a UUID (a cryptographically >> secure random number). That problem _is_ solved (some systems have >> trouble generating entropy, but there are workarounds). If you anyway >> have a communication channel to the process, you could ask for this >> UUID. One you have a communication channel, you can do a lot >> (reference counting, safely transfer the lock,...). >> >>> If the server and the client can't access each other >> >> How to find out that the server is still running? My point is: I like >> to have a secure, automatic way to break the lock if the machine or >> process is stopped. And from my experience, native file locking is >> problematic for this. >> >> You could also combine solutions (such as: combine the 'open a server >> socket' solution with 'background thread' solution). I'm not sure if >> it's worth it to solve the 'hibernate' problem. >> >> Regards, >> Thomas >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-dev-h...@lucene.apache.org >> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-dev-h...@lucene.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
