Thank you for your reply. This isn't happening in a single environment. Our application is being used by various clients and this has been reported by multiple users - all of whom were running the earlier pylucene (v4.10) - without issues.
One thing to mention is that our earlier version used Python 2.7.15 (with pylucene 4.10) and now we are using Python 3.8.10 with Pylucene 6.5.0 - the indexing logic is the same... One other thing to note is that the issue described has (so far!) only occurred on MS Windows - none of our Linux customers have complained about this. Any ideas? Regards, Antony On Thu, 28 Apr 2022 at 17:00, Adrien Grand <jpou...@gmail.com> wrote: > Hi Anthony, > > This isn't something that you should try to fix programmatically, > corruptions indicate that something is wrong with the environment, > like a broken disk or corrupt RAM. I would suggest running a memtest > to check your RAM and looking at system logs in case they have > anything to tell about your disks. > > Can you also share the full stack trace of the exception? > > On Thu, Apr 28, 2022 at 10:26 AM Antony Joseph > <antony.dev.webm...@gmail.com> wrote: > > > > Hello, > > > > We are facing a strange situation in our application as described below: > > > > *Using*: > > > > - Python 3.8.10 > > - Pylucene 6.5.0 > > - Java 8 (1.8.0_181) > > - Runs on Linux and Windows (error seen on Windows) > > > > We suddenly get the following *error*: > > > > 2022-02-10 09:58:09.253215: ERROR : writer | Failed to get index > > (D:\i\202202) writer, Exception: > > org.apache.lucene.index.CorruptIndexException: Unexpected file read error > > while reading index. > > > (resource=BufferedChecksumIndexInput(MMapIndexInput(path="D:\i\202202\segments_fo"))) > > > > > > After this, no further indexing happens - trying to open the index for > > writing throws the above error - and the index writer does not open. > > > > FYI, our code contains the following *settings*: > > > > index_path = "D:\i\202202" > > index_directory = FSDirectory.open(Paths.get(index_path)) > > iconfig = IndexWriterConfig(wrapper_analyzer) > > iconfig.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND) > > iconfig.setRAMBufferSizeMB(16.0) > > writer = IndexWriter(index_directory, iconfig) > > > > > > *Repairing* > > We tried 'repairing' the index with the following command / tool: > > > > java -cp lucene-core-6.5.0.jar:lucene-backward-codecs-6.5.0.jar > > org.apache.lucene.index.CheckIndex "D:\i\202202" -exorcise > > > > This however returns saying "No problems found with the index." > > > > > > *Work around* > > We have to manually delete the problematic segment file: > > D:\i\202202\segments_fo > > after which the application starts again... until the next corruption. We > > can't spot a specific pattern. > > > > > > *Two questions:* > > > > 1. Can we handle this situation programmatically, so that no manual > > intervention is needed? > > 2. Any reason why we are facing the corruption issue in the first > place? > > > > > > Before this we were using Pylucene 4.10 and we didn't face this problem - > > the application logic is the same. > > > > Also, while the application runs on both Linux and Windows, so far we > have > > observed this situation only on various Windows platforms. > > > > Would really appreciate some assistance. Thanks in advance. > > > > Regards, > > Antony > > > > -- > Adrien > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > >
