Do you have 2 separate segments files in that 2nd index?

Which exact Lucene version is this?

Mike McCandless

http://blog.mikemccandless.com


On Thu, Sep 22, 2016 at 7:44 AM, Ziming Dong <dzm1016397...@gmail.com> wrote:
> I used checkIndex to recover second index though I lost many docs in index,
> but first index can't be read by checkIndex, error is
>
>> java -cp lucene-core-6.1.0.jar -ea:org.apache.lucene...
>> org.apache.lucene.index.CheckIndex /Volumes/HPT8_56T/infomall-index/index0
>> Opening index @ /Volumes/HPT8_56T/infomall-index/index0
>> ERROR: could not read any segments file in directory
>> org.apache.lucene.index.IndexFormatTooOldException: Format version is not
>> supported (resource
>> BufferedChecksumIndexInput(MMapIndexInput(path="/Volumes/HPT8_56T/infomall-index/index0/segments_5t3"))):
>> 0 (needs to be between 1071082519 and 1071082519). This version of Lucene
>> only supports indexes created with release 5.0 and later.
>>         at
>> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:295)
>>         at
>> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:284)
>>         at
>> org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:507)
>>         at
>> org.apache.lucene.index.CheckIndex.doCheck(CheckIndex.java:2595)
>>         at org.apache.lucene.index.CheckIndex.doMain(CheckIndex.java:2497)
>>         at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:2423)
>
>
>  I use NFS, but I set mount option as  mount -t nfs -o tcp,sync,retrans=10
> The index program has run 1 month without any problem before power failure.
>
> On Thu, Sep 22, 2016 at 6:06 PM, Michael McCandless
> <luc...@mikemccandless.com> wrote:
>>
>> Hmm I'm no longer so sure this is an IW bug: on commit we fsync the
>> pending_segments_N and then do an atomic rename to segments_N.
>>
>> Can you describe your IO system?  Is it possible it does not implement
>> fsync or atomic renames correctly?
>>
>> Also, your 2nd exception indices the segments_N file was intact but
>> the .cfs file was corrupt, which is also hard to explain unless fsync
>> isn't working on your IO system.
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>> On Thu, Sep 22, 2016 at 5:10 AM, Michael McCandless
>> <luc...@mikemccandless.com> wrote:
>> > Sorry for the slow reply here.  Curious that both of these exceptions
>> > are from IW.init.  I think this may be a real bug, caused by this:
>> >
>> > https://github.com/apache/lucene-solr/commit/981bfba841144d08df1d1a183d39fcd6f195ad56
>> >
>> > I'll see if I can make a standalone test case showing this.
>> >
>> > If you open those indices with an IndexReader instead, does it succeed?
>> >
>> > If you run CheckIndex, what does it report?
>> >
>> > Mike McCandless
>> >
>> > http://blog.mikemccandless.com
>> >
>> > On Wed, Sep 14, 2016 at 1:22 AM, Ziming Dong <dzm1016397...@gmail.com>
>> > wrote:
>> >> I have 6 machine and 6 index directories, each machine builds index
>> >> into
>> >> one index directory. After power failure last night, two of those
>> >> machine
>> >> can't start index program.
>> >>
>> >> one error is
>> >>
>> >>> INFO: 2016-09-14 12:31:38 [main]
>> >>> sewm.bdbox.search.InfomallIndexer$Builder:ignoreCollectionsFile(227):
>> >>> Loaded 2146 ignored collections from
>> >>> /mnt/HPT8_56T/infomall-index/index0/ignored_collections.txt
>> >>> ERROR: 2016-09-14 12:31:39 [main] sewm.bdbox.util.LogUtil:error(71):
>> >>> org.apache.lucene.index.IndexFormatTooOldException: Format version is
>> >>> not
>> >>> supported (resource
>> >>>
>> >>> BufferedChecksumIndexInput(MMapIndexInput(path="/mnt/HPT8_56T/infomall-index/index0/segments_5t3"))):
>> >>> 0 (needs to be between 1071082519 and 1071082519). This version of
>> >>> Lucene
>> >>> only supports indexes created with release 5.0 and later.
>> >>>         at
>> >>> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:295)
>> >>>         at
>> >>> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:284)
>> >>>         at
>> >>> org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:910)
>> >>>         at
>> >>> sewm.bdbox.search.InfomallIndexer.<init>(InfomallIndexer.java:60)
>> >>>         at
>> >>>
>> >>> sewm.bdbox.search.ThreadedInfomallIndexer.<init>(ThreadedInfomallIndexer.java:28)
>> >>>         at
>> >>>
>> >>> sewm.bdbox.search.ThreadedInfomallIndexer.<init>(ThreadedInfomallIndexer.java:21)
>> >>>         at
>> >>>
>> >>> sewm.bdbox.search.ThreadedInfomallIndexer$Builder.build(ThreadedInfomallIndexer.java:72)
>> >>>         at
>> >>>
>> >>> sewm.bdbox.search.ThreadedInfomallIndexer.main(ThreadedInfomallIndexer.java:129)
>> >>
>> >>
>> >> another is
>> >>
>> >> INFO: 2016-09-14 01:11:06 [main]
>> >>> sewm.bdbox.search.InfomallIndexer$Builder:ignoreCollectionsFile(227):
>> >>> Loaded 8575 ignored collections from
>> >>> /mnt/HPT8/infomall-index/index5/ignored_collections.txt
>> >>> ERROR: 2016-09-14 01:11:09 [main] sewm.bdbox.util.LogUtil:error(71):
>> >>> org.apache.lucene.index.CorruptIndexException: codec footer mismatch
>> >>> (file
>> >>> truncated?): actual footer=0 vs expected footer=-1071082520
>> >>>
>> >>> (resource=MMapIndexInput(path="/mnt/HPT8/infomall-index/index5/_1kqn.cfs"))
>> >>>         at
>> >>> org.apache.lucene.codecs.CodecUtil.validateFooter(CodecUtil.java:448)
>> >>>         at
>> >>>
>> >>> org.apache.lucene.codecs.CodecUtil.retrieveChecksum(CodecUtil.java:433)
>> >>>         at
>> >>>
>> >>> org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.<init>(Lucene50CompoundReader.java:86)
>> >>>         at
>> >>>
>> >>> org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.getCompoundReader(Lucene50CompoundFormat.java:71)
>> >>>         at
>> >>>
>> >>> org.apache.lucene.index.IndexWriter.readFieldInfos(IndexWriter.java:1016)
>> >>>         at
>> >>>
>> >>> org.apache.lucene.index.IndexWriter.getFieldNumberMap(IndexWriter.java:1033)
>> >>>         at
>> >>> org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:938)
>> >>>         at
>> >>> sewm.bdbox.search.InfomallIndexer.<init>(InfomallIndexer.java:60)
>> >>>         at
>> >>>
>> >>> sewm.bdbox.search.ThreadedInfomallIndexer.<init>(ThreadedInfomallIndexer.java:28)
>> >>>         at
>> >>>
>> >>> sewm.bdbox.search.ThreadedInfomallIndexer.<init>(ThreadedInfomallIndexer.java:21)
>> >>>         at
>> >>>
>> >>> sewm.bdbox.search.ThreadedInfomallIndexer$Builder.build(ThreadedInfomallIndexer.java:72)
>> >>>         at
>> >>>
>> >>> sewm.bdbox.search.ThreadedInfomallIndexer.main(ThreadedInfomallIndexer.java:129)
>> >>>
>> >>
>> >>
>> >> it seems 1071082519 is a special number.
>> >>
>> >> - -
>> >>
>> >> Ziming Dong
>> >> *http://suiyuan2009.github.io/ <http://suiyuan2009.github.io/>*
>
>
>
>
> --
>
> Ziming Dong
> http://suiyuan2009.github.io/
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org

Reply via email to