[jira] [Commented] (LUCENE-7662) Index with missing files should throw CorruptIndexException
[
https://issues.apache.org/jira/browse/LUCENE-7662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862123#comment-15862123
]
ASF subversion and git services commented on LUCENE-7662:
-
Commit 8ad763555012cd7637238380b6bfc5b4d7254325 in lucene-solr's branch
refs/heads/branch_6x from Mike McCandless
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=8ad7635 ]
LUCENE-7662: throw CorruptIndexException if index files are missing
> Index with missing files should throw CorruptIndexException
> ---
>
> Key: LUCENE-7662
> URL: https://issues.apache.org/jira/browse/LUCENE-7662
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 6.4
>Reporter: Mike Drob
> Fix For: master (7.0), 6.5.0
>
> Attachments: LUCENE-7662.patch, LUCENE-7662.patch, LUCENE-7662.patch
>
>
> Similar to what we did in LUCENE-7592 for EOF, we should catch missing files
> and rethrow those as CorruptIndexException.
> If a particular codec can handle missing files, it should be proactive check
> for those optional files and not throw anything, so I think we can safely do
> this at SegmentReader or SegmentCoreReaders level.
> Stack trace copied from SOLR-10006:
> {noformat}
> Caused by: java.nio.file.NoSuchFileException:
> /Users/Erick/apache/solrVersions/trunk/solr/example/cloud/node3/solr/eoe_shard1_replica1/data/index/_1_Lucene50_0.doc
> at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
> at java.nio.channels.FileChannel.open(FileChannel.java:287)
> at java.nio.channels.FileChannel.open(FileChannel.java:335)
> at
> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:238)
> at
> org.apache.lucene.store.NRTCachingDirectory.openInput(NRTCachingDirectory.java:192)
> at
> org.apache.solr.core.MetricsDirectoryFactory$MetricsDirectory.openInput(MetricsDirectoryFactory.java:334)
> at
> org.apache.lucene.codecs.lucene50.Lucene50PostingsReader.(Lucene50PostingsReader.java:81)
> at
> org.apache.lucene.codecs.lucene50.Lucene50PostingsFormat.fieldsProducer(Lucene50PostingsFormat.java:442)
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.(PerFieldPostingsFormat.java:292)
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:372)
> at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:109)
> at org.apache.lucene.index.SegmentReader.(SegmentReader.java:74)
> at
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:143)
> at
> org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:195)
> at
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:103)
> at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:473)
> at
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:103)
> at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:79)
> at
> org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:39)
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1958)
> ... 12 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7662) Index with missing files should throw CorruptIndexException
[
https://issues.apache.org/jira/browse/LUCENE-7662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862119#comment-15862119
]
ASF subversion and git services commented on LUCENE-7662:
-
Commit f8e16e3c4e852bce20d0793d157963777ac8f07b in lucene-solr's branch
refs/heads/master from Mike McCandless
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f8e16e3 ]
LUCENE-7662: move CHANGES entry under 6.5.0
> Index with missing files should throw CorruptIndexException
> ---
>
> Key: LUCENE-7662
> URL: https://issues.apache.org/jira/browse/LUCENE-7662
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 6.4
>Reporter: Mike Drob
> Attachments: LUCENE-7662.patch, LUCENE-7662.patch, LUCENE-7662.patch
>
>
> Similar to what we did in LUCENE-7592 for EOF, we should catch missing files
> and rethrow those as CorruptIndexException.
> If a particular codec can handle missing files, it should be proactive check
> for those optional files and not throw anything, so I think we can safely do
> this at SegmentReader or SegmentCoreReaders level.
> Stack trace copied from SOLR-10006:
> {noformat}
> Caused by: java.nio.file.NoSuchFileException:
> /Users/Erick/apache/solrVersions/trunk/solr/example/cloud/node3/solr/eoe_shard1_replica1/data/index/_1_Lucene50_0.doc
> at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
> at java.nio.channels.FileChannel.open(FileChannel.java:287)
> at java.nio.channels.FileChannel.open(FileChannel.java:335)
> at
> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:238)
> at
> org.apache.lucene.store.NRTCachingDirectory.openInput(NRTCachingDirectory.java:192)
> at
> org.apache.solr.core.MetricsDirectoryFactory$MetricsDirectory.openInput(MetricsDirectoryFactory.java:334)
> at
> org.apache.lucene.codecs.lucene50.Lucene50PostingsReader.(Lucene50PostingsReader.java:81)
> at
> org.apache.lucene.codecs.lucene50.Lucene50PostingsFormat.fieldsProducer(Lucene50PostingsFormat.java:442)
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.(PerFieldPostingsFormat.java:292)
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:372)
> at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:109)
> at org.apache.lucene.index.SegmentReader.(SegmentReader.java:74)
> at
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:143)
> at
> org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:195)
> at
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:103)
> at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:473)
> at
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:103)
> at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:79)
> at
> org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:39)
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1958)
> ... 12 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7662) Index with missing files should throw CorruptIndexException
[
https://issues.apache.org/jira/browse/LUCENE-7662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862117#comment-15862117
]
ASF subversion and git services commented on LUCENE-7662:
-
Commit be007a6c1124103ed4b21051a4ec98d94a2f7efa in lucene-solr's branch
refs/heads/master from Mike McCandless
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=be007a6 ]
LUCENE-7662: throw CorruptIndexException if index files are missing
> Index with missing files should throw CorruptIndexException
> ---
>
> Key: LUCENE-7662
> URL: https://issues.apache.org/jira/browse/LUCENE-7662
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 6.4
>Reporter: Mike Drob
> Attachments: LUCENE-7662.patch, LUCENE-7662.patch, LUCENE-7662.patch
>
>
> Similar to what we did in LUCENE-7592 for EOF, we should catch missing files
> and rethrow those as CorruptIndexException.
> If a particular codec can handle missing files, it should be proactive check
> for those optional files and not throw anything, so I think we can safely do
> this at SegmentReader or SegmentCoreReaders level.
> Stack trace copied from SOLR-10006:
> {noformat}
> Caused by: java.nio.file.NoSuchFileException:
> /Users/Erick/apache/solrVersions/trunk/solr/example/cloud/node3/solr/eoe_shard1_replica1/data/index/_1_Lucene50_0.doc
> at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
> at java.nio.channels.FileChannel.open(FileChannel.java:287)
> at java.nio.channels.FileChannel.open(FileChannel.java:335)
> at
> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:238)
> at
> org.apache.lucene.store.NRTCachingDirectory.openInput(NRTCachingDirectory.java:192)
> at
> org.apache.solr.core.MetricsDirectoryFactory$MetricsDirectory.openInput(MetricsDirectoryFactory.java:334)
> at
> org.apache.lucene.codecs.lucene50.Lucene50PostingsReader.(Lucene50PostingsReader.java:81)
> at
> org.apache.lucene.codecs.lucene50.Lucene50PostingsFormat.fieldsProducer(Lucene50PostingsFormat.java:442)
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.(PerFieldPostingsFormat.java:292)
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:372)
> at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:109)
> at org.apache.lucene.index.SegmentReader.(SegmentReader.java:74)
> at
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:143)
> at
> org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:195)
> at
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:103)
> at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:473)
> at
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:103)
> at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:79)
> at
> org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:39)
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1958)
> ... 12 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7662) Index with missing files should throw CorruptIndexException
[
https://issues.apache.org/jira/browse/LUCENE-7662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15861905#comment-15861905
]
Michael McCandless commented on LUCENE-7662:
Thanks [~mdrob]!
bq. That is frustrating. I ran it 10 times and somehow never hit that or a
similar seed.
The joys of randomized tests! I'll review and push soon...
> Index with missing files should throw CorruptIndexException
> ---
>
> Key: LUCENE-7662
> URL: https://issues.apache.org/jira/browse/LUCENE-7662
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 6.4
>Reporter: Mike Drob
> Attachments: LUCENE-7662.patch, LUCENE-7662.patch, LUCENE-7662.patch
>
>
> Similar to what we did in LUCENE-7592 for EOF, we should catch missing files
> and rethrow those as CorruptIndexException.
> If a particular codec can handle missing files, it should be proactive check
> for those optional files and not throw anything, so I think we can safely do
> this at SegmentReader or SegmentCoreReaders level.
> Stack trace copied from SOLR-10006:
> {noformat}
> Caused by: java.nio.file.NoSuchFileException:
> /Users/Erick/apache/solrVersions/trunk/solr/example/cloud/node3/solr/eoe_shard1_replica1/data/index/_1_Lucene50_0.doc
> at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
> at java.nio.channels.FileChannel.open(FileChannel.java:287)
> at java.nio.channels.FileChannel.open(FileChannel.java:335)
> at
> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:238)
> at
> org.apache.lucene.store.NRTCachingDirectory.openInput(NRTCachingDirectory.java:192)
> at
> org.apache.solr.core.MetricsDirectoryFactory$MetricsDirectory.openInput(MetricsDirectoryFactory.java:334)
> at
> org.apache.lucene.codecs.lucene50.Lucene50PostingsReader.(Lucene50PostingsReader.java:81)
> at
> org.apache.lucene.codecs.lucene50.Lucene50PostingsFormat.fieldsProducer(Lucene50PostingsFormat.java:442)
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.(PerFieldPostingsFormat.java:292)
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:372)
> at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:109)
> at org.apache.lucene.index.SegmentReader.(SegmentReader.java:74)
> at
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:143)
> at
> org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:195)
> at
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:103)
> at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:473)
> at
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:103)
> at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:79)
> at
> org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:39)
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1958)
> ... 12 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7662) Index with missing files should throw CorruptIndexException
[
https://issues.apache.org/jira/browse/LUCENE-7662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858828#comment-15858828
]
Michael McCandless commented on LUCENE-7662:
Hmm something is still angry:
{noformat}
[junit4] Suite: org.apache.lucene.index.TestMissingIndexFiles
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestMissingIndexFiles -Dtests.method=testMissingDoc
-Dtests.seed=4D7CBCD6B337257 -Dtests.locale=de-CH -Dtests.timezone=Etc/GMT-10
-Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] FAILURE 0.04s J2 | TestMissingIndexFiles.testMissingDoc <<<
[junit4]> Throwable #1: junit.framework.AssertionFailedError: Expected
exception CorruptIndexException
[junit4]>at
__randomizedtesting.SeedInfo.seed([4D7CBCD6B337257:6405AAA9369B3658]:0)
[junit4]>at
org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2703)
[junit4]>at
org.apache.lucene.index.TestMissingIndexFiles.testMissingDoc(TestMissingIndexFiles.java:52)
[junit4]>at java.lang.Thread.run(Thread.java:745)
[junit4]>Suppressed: java.lang.RuntimeException:
MockDirectoryWrapper: cannot close: there are still 1 open files: {_0.cfs=1}
[junit4]>at
org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:841)
[junit4]>at
org.apache.lucene.index.TestMissingIndexFiles.testMissingDoc(TestMissingIndexFiles.java:53)
[junit4]>... 36 more
[junit4]>Caused by: java.lang.RuntimeException: unclosed
IndexInput: _0.cfs
[junit4]>at
org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:732)
[junit4]>at
org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:776)
[junit4]>at
org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.(Lucene50CompoundReader.java:78)
[junit4]>at
org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.getCompoundReader(Lucene50CompoundFormat.java:71)
[junit4]>at
org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:99)
[junit4]>at
org.apache.lucene.index.SegmentReader.(SegmentReader.java:74)
[junit4]>at
org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:62)
[junit4]>at
org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:54)
[junit4]>at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:666)
[junit4]>at
org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:77)
[junit4]>at
org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
[junit4]>at
org.apache.lucene.index.TestMissingIndexFiles.lambda$testMissingDoc$0(TestMissingIndexFiles.java:52)
[junit4]>at
org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2694)
[junit4]>at
org.apache.lucene.index.TestMissingIndexFiles.testMissingDoc(TestMissingIndexFiles.java:52)
[junit4]>... 36 more
{noformat}
> Index with missing files should throw CorruptIndexException
> ---
>
> Key: LUCENE-7662
> URL: https://issues.apache.org/jira/browse/LUCENE-7662
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 6.4
>Reporter: Mike Drob
> Attachments: LUCENE-7662.patch, LUCENE-7662.patch
>
>
> Similar to what we did in LUCENE-7592 for EOF, we should catch missing files
> and rethrow those as CorruptIndexException.
> If a particular codec can handle missing files, it should be proactive check
> for those optional files and not throw anything, so I think we can safely do
> this at SegmentReader or SegmentCoreReaders level.
> Stack trace copied from SOLR-10006:
> {noformat}
> Caused by: java.nio.file.NoSuchFileException:
> /Users/Erick/apache/solrVersions/trunk/solr/example/cloud/node3/solr/eoe_shard1_replica1/data/index/_1_Lucene50_0.doc
> at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
> at java.nio.channels.FileChannel.open(FileChannel.java:287)
> at java.nio.channels.FileChannel.open(FileChannel.java:335)
> at
>
[jira] [Commented] (LUCENE-7662) Index with missing files should throw CorruptIndexException
[
https://issues.apache.org/jira/browse/LUCENE-7662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858752#comment-15858752
]
Michael McCandless commented on LUCENE-7662:
Thanks [~mdrob], the new patch looks great, and +1 to do that test cleanup
here. I'll push soon!
> Index with missing files should throw CorruptIndexException
> ---
>
> Key: LUCENE-7662
> URL: https://issues.apache.org/jira/browse/LUCENE-7662
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 6.4
>Reporter: Mike Drob
> Attachments: LUCENE-7662.patch, LUCENE-7662.patch
>
>
> Similar to what we did in LUCENE-7592 for EOF, we should catch missing files
> and rethrow those as CorruptIndexException.
> If a particular codec can handle missing files, it should be proactive check
> for those optional files and not throw anything, so I think we can safely do
> this at SegmentReader or SegmentCoreReaders level.
> Stack trace copied from SOLR-10006:
> {noformat}
> Caused by: java.nio.file.NoSuchFileException:
> /Users/Erick/apache/solrVersions/trunk/solr/example/cloud/node3/solr/eoe_shard1_replica1/data/index/_1_Lucene50_0.doc
> at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
> at java.nio.channels.FileChannel.open(FileChannel.java:287)
> at java.nio.channels.FileChannel.open(FileChannel.java:335)
> at
> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:238)
> at
> org.apache.lucene.store.NRTCachingDirectory.openInput(NRTCachingDirectory.java:192)
> at
> org.apache.solr.core.MetricsDirectoryFactory$MetricsDirectory.openInput(MetricsDirectoryFactory.java:334)
> at
> org.apache.lucene.codecs.lucene50.Lucene50PostingsReader.(Lucene50PostingsReader.java:81)
> at
> org.apache.lucene.codecs.lucene50.Lucene50PostingsFormat.fieldsProducer(Lucene50PostingsFormat.java:442)
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.(PerFieldPostingsFormat.java:292)
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:372)
> at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:109)
> at org.apache.lucene.index.SegmentReader.(SegmentReader.java:74)
> at
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:143)
> at
> org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:195)
> at
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:103)
> at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:473)
> at
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:103)
> at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:79)
> at
> org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:39)
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1958)
> ... 12 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7662) Index with missing files should throw CorruptIndexException
[
https://issues.apache.org/jira/browse/LUCENE-7662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858534#comment-15858534
]
Mike Drob commented on LUCENE-7662:
---
Those are good suggestions, I'll get them into the next version of this patch.
Looking at the code in MockDirectoryWrapper, some of the "a random IOException"
stuff looks really hackish, especially where we are checking for string
messages to match. I'm uncomfortable with how brittle some of that is. We
already have FakeIOException available and I think it would be good to use that
instead in several places. Do you think we should handle that here, or I can
file a new issue for it.
> Index with missing files should throw CorruptIndexException
> ---
>
> Key: LUCENE-7662
> URL: https://issues.apache.org/jira/browse/LUCENE-7662
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 6.4
>Reporter: Mike Drob
> Attachments: LUCENE-7662.patch
>
>
> Similar to what we did in LUCENE-7592 for EOF, we should catch missing files
> and rethrow those as CorruptIndexException.
> If a particular codec can handle missing files, it should be proactive check
> for those optional files and not throw anything, so I think we can safely do
> this at SegmentReader or SegmentCoreReaders level.
> Stack trace copied from SOLR-10006:
> {noformat}
> Caused by: java.nio.file.NoSuchFileException:
> /Users/Erick/apache/solrVersions/trunk/solr/example/cloud/node3/solr/eoe_shard1_replica1/data/index/_1_Lucene50_0.doc
> at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
> at java.nio.channels.FileChannel.open(FileChannel.java:287)
> at java.nio.channels.FileChannel.open(FileChannel.java:335)
> at
> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:238)
> at
> org.apache.lucene.store.NRTCachingDirectory.openInput(NRTCachingDirectory.java:192)
> at
> org.apache.solr.core.MetricsDirectoryFactory$MetricsDirectory.openInput(MetricsDirectoryFactory.java:334)
> at
> org.apache.lucene.codecs.lucene50.Lucene50PostingsReader.(Lucene50PostingsReader.java:81)
> at
> org.apache.lucene.codecs.lucene50.Lucene50PostingsFormat.fieldsProducer(Lucene50PostingsFormat.java:442)
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.(PerFieldPostingsFormat.java:292)
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:372)
> at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:109)
> at org.apache.lucene.index.SegmentReader.(SegmentReader.java:74)
> at
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:143)
> at
> org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:195)
> at
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:103)
> at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:473)
> at
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:103)
> at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:79)
> at
> org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:39)
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1958)
> ... 12 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7662) Index with missing files should throw CorruptIndexException
[
https://issues.apache.org/jira/browse/LUCENE-7662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15857826#comment-15857826
]
Michael McCandless commented on LUCENE-7662:
Thanks [~mdrob]; I think this patch looks good, except it makes some tests
angry, e.g.:
{noformat}
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestLucene62SegmentInfoFormat -Dtests.method=testRandomExceptions
-Dtests.seed=F65CD1D4D104665D -Dtests.locale=zh -Dtests.timezone=Asia/Khandyga
-Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.03s J3 |
TestLucene62SegmentInfoFormat.testRandomExceptions <<<
[junit4]> Throwable #1: org.apache.lucene.index.CorruptIndexException:
Problem reading index. (resource=a random IOException (_e.cfe))
[junit4]>at
__randomizedtesting.SeedInfo.seed([F65CD1D4D104665D:9E73BF104F0A3FFD]:0)
[junit4]>at
org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:142)
[junit4]>at
org.apache.lucene.index.SegmentReader.(SegmentReader.java:74)
[junit4]>at
org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:143)
[junit4]>at
org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:195)
[junit4]>at
org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:103)
[junit4]>at
org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:473)
[junit4]>at
org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:103)
[junit4]>at
org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:563)
[junit4]>at
org.apache.lucene.index.BaseSegmentInfoFormatTestCase.testRandomExceptions(BaseSegmentInfoFormatTestCase.java:50)
[junit4]>at java.lang.Thread.run(Thread.java:745)
[junit4]> Caused by: java.nio.file.NoSuchFileException: a random
IOException (_e.cfe)
[junit4]>at
org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:575)
[junit4]>at
org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:744)
[junit4]>at
org.apache.lucene.store.Directory.openChecksumInput(Directory.java:119)
[junit4]>at
org.apache.lucene.store.MockDirectoryWrapper.openChecksumInput(MockDirectoryWrapper.java:1072)
[junit4]>at
org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.readEntries(Lucene50CompoundReader.java:105)
[junit4]>at
org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.(Lucene50CompoundReader.java:69)
[junit4]>at
org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.getCompoundReader(Lucene50CompoundFormat.java:71)
[junit4]>at
org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:99)
[junit4]>... 44 more
{noformat}
Maybe we just need to relax that base test case to accept the new
{{CorruptIndexExcpeption}} as well, and look to its cause to check the
exception message?
Also, I think it'd be a bit better to use our {{expectThrows}} method in the
test case, wrapped around the one line where you try to open an index reader,
instead of the {{@Test(expected = ...)}}, which would pass if
{{CorruptIndexException}} was hit anywhere in that test case?
> Index with missing files should throw CorruptIndexException
> ---
>
> Key: LUCENE-7662
> URL: https://issues.apache.org/jira/browse/LUCENE-7662
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 6.4
>Reporter: Mike Drob
> Attachments: LUCENE-7662.patch
>
>
> Similar to what we did in LUCENE-7592 for EOF, we should catch missing files
> and rethrow those as CorruptIndexException.
> If a particular codec can handle missing files, it should be proactive check
> for those optional files and not throw anything, so I think we can safely do
> this at SegmentReader or SegmentCoreReaders level.
> Stack trace copied from SOLR-10006:
> {noformat}
> Caused by: java.nio.file.NoSuchFileException:
> /Users/Erick/apache/solrVersions/trunk/solr/example/cloud/node3/solr/eoe_shard1_replica1/data/index/_1_Lucene50_0.doc
> at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
> at java.nio.channels.FileChannel.open(FileChannel.java:287)
> at
[jira] [Commented] (LUCENE-7662) Index with missing files should throw CorruptIndexException
[
https://issues.apache.org/jira/browse/LUCENE-7662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15843284#comment-15843284
]
Michael McCandless commented on LUCENE-7662:
+1
> Index with missing files should throw CorruptIndexException
> ---
>
> Key: LUCENE-7662
> URL: https://issues.apache.org/jira/browse/LUCENE-7662
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 6.4
>Reporter: Mike Drob
>
> Similar to what we did in LUCENE-7592 for EOF, we should catch missing files
> and rethrow those as CorruptIndexException.
> If a particular codec can handle missing files, it should be proactive check
> for those optional files and not throw anything, so I think we can safely do
> this at SegmentReader or SegmentCoreReaders level.
> Stack trace copied from SOLR-10006:
> {noformat}
> Caused by: java.nio.file.NoSuchFileException:
> /Users/Erick/apache/solrVersions/trunk/solr/example/cloud/node3/solr/eoe_shard1_replica1/data/index/_1_Lucene50_0.doc
> at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
> at java.nio.channels.FileChannel.open(FileChannel.java:287)
> at java.nio.channels.FileChannel.open(FileChannel.java:335)
> at
> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:238)
> at
> org.apache.lucene.store.NRTCachingDirectory.openInput(NRTCachingDirectory.java:192)
> at
> org.apache.solr.core.MetricsDirectoryFactory$MetricsDirectory.openInput(MetricsDirectoryFactory.java:334)
> at
> org.apache.lucene.codecs.lucene50.Lucene50PostingsReader.(Lucene50PostingsReader.java:81)
> at
> org.apache.lucene.codecs.lucene50.Lucene50PostingsFormat.fieldsProducer(Lucene50PostingsFormat.java:442)
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.(PerFieldPostingsFormat.java:292)
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:372)
> at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:109)
> at org.apache.lucene.index.SegmentReader.(SegmentReader.java:74)
> at
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:143)
> at
> org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:195)
> at
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:103)
> at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:473)
> at
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:103)
> at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:79)
> at
> org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:39)
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1958)
> ... 12 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
