Hi Borek, Thanks for your reply.
This seems to be the issue we are having - CLucene is NOT restricting write access to the index (i.e. it seems it is not handling multi-threading correctly). When we launch, for example, 2 processes simultaneously, the 2nd one crashes when trying to access the index (it seems as if the 1st one is not releasing the lock on the directory). Unfortunately, we can't add synchronisation code as we are shipping a version of CLucene which is integrated with Qt 4.7.3. Qt support has been unable to help so far so I was hoping there would be other CLucene users who have seen this behaviour and may have a fix/workaround for it. Thanks, Adrian > Hi, > > In CLucene only one process can write to the index at the same time. As far > as I know it is by intent. > The only solution is to synchronize the write access in your apllication (eg. > by boost::file_lock) > > Regards > > Borek > >> -----Original Message----- >> From: Adrian Jordan [mailto:adrian.jor...@iongeo.com] >> Sent: Thursday, December 19, 2013 12:31 PM >> To: clucene-developers@lists.sourceforge.net >> Subject: [CLucene-dev] IndexWriter concurrency >> >> Hello, >> >> We are using Qt Help (version 4.7.3) to provide documentation for our >> application. >> >> When we launch it, Qt creates an index directory based upon the Qt Help >> collection file, which is used by CLucene (bundled with the Qt Help >> Framework) to index the help contents. >> >> So we have for example: >> >> ../documentation/ >> help.qhc >> .help/ >> segments >> _6d.cfs >> deletable >> >> This setup is not working in our application, which consists of >> multiple processes running concurrently, as it causes one or more >> processes to crash when we startup. >> >> >From the GDB back trace from the core file that was produced, the >> CLucene code is crashing in: >> >> RAMDirectory::close (line 352) >> files.clear(); >> TransactionalRAMDirectory::close (line 209) >> RAMDirectory::close(); >> IndexWriter::close (line 165) >> ramDirectory->close(); >> >> This seems to point to an issue with concurrent access to the >> IndexWriter (related to SCOPED_LOCK_MUTEX? VoidMap?). I believe CLucene >> uses a write lock on this directory (.help in our case) that can only >> be access by one process at a time. >> >> We can reproduce it repeatedly be removing the files under the .help >> directory and restarting our application. >> >> Upgrading to the latest version of CLucene didn't resolve the issue (I >> can't even find out which version we are using!). >> >> So: >> >> 1/ Should CLucene support multiple processes/instances of the same >> process writing to this index directory? >> 2/ Is there setting we can use to try and resolve this (e.g increase >> the write lock timeout)? >> 3/ Is this a known issue and has it been fixed? >> >> I look forward to hearing from anyone with any ideas about this. >> >> Thanks, >> >> Adrian >> >> ________________________________ >> >> >> This email and any files transmitted with it are confidential and are >> intended solely for the use of the individual or entity to whom they >> are addressed. If you are not the original recipient or the person >> responsible for delivering the email to the intended recipient, be >> advised that you have received this email in error, and that any use, >> dissemination, forwarding, printing, or copying of this email is >> strictly prohibited. If you received this email in error, please >> immediately notify the sender and delete the original. >> >> >> ----------------------------------------------------------------------- >> ------- >> Rapidly troubleshoot problems before they affect your business. Most IT >> organizations don't have a clear picture of how application performance >> affects their revenue. With AppDynamics, you get 100% visibility into >> your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of >> AppDynamics Pro! >> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.cl >> ktrk >> _______________________________________________ >> CLucene-developers mailing list >> CLucene-developers@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/clucene-developers > ------------------------------------------------------------------------------ > Rapidly troubleshoot problems before they affect your business. Most IT > organizations don't have a clear picture of how application performance > affects their revenue. With AppDynamics, you get 100% visibility into your > Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! > http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk > _______________________________________________ > CLucene-developers mailing list > CLucene-developers@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/clucene-developers ________________________________ This email and any files transmitted with it are confidential and are intended solely for the use of the individual or entity to whom they are addressed. If you are not the original recipient or the person responsible for delivering the email to the intended recipient, be advised that you have received this email in error, and that any use, dissemination, forwarding, printing, or copying of this email is strictly prohibited. If you received this email in error, please immediately notify the sender and delete the original. ------------------------------------------------------------------------------ CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments & Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk _______________________________________________ CLucene-developers mailing list CLucene-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/clucene-developers
