Re: org.apache.lucene.index.CorruptIndexException: checksum failed

2016-07-20 Thread Ziming Dong
I upgrade lucene to 6.1.0, hope it works, thank you very much! the mount( contain all data directories and index directories ) is shared by four different computers, but each index directory can be accessed only by one computer, we have four index directories. 2016-07-21 5:13 GMT+08:00 Michael Mc

Re: org.apache.lucene.index.CorruptIndexException: checksum failed

2016-07-29 Thread Ziming Dong
later. these two computer mount the same directory of a macmini use nfs, but their index directory is different. It seems nfs is the problem. On Thu, Jul 21, 2016 at 12:49 PM, Ziming Dong wrote: > I upgrade lucene to 6.1.0, hope it works, thank you very much! > > the mount( contain

lucene index program won't start after power failure

2016-09-13 Thread Ziming Dong
r.java:28) > at > sewm.bdbox.search.ThreadedInfomallIndexer.(ThreadedInfomallIndexer.java:21) > at > sewm.bdbox.search.ThreadedInfomallIndexer$Builder.build(ThreadedInfomallIndexer.java:72) > at > sewm.bdbox.search.ThreadedInfomallIndexer.main(ThreadedInfomallIndexer.java:129) > it seems 1071082519 is a special number. - - Ziming Dong *http://suiyuan2009.github.io/ <http://suiyuan2009.github.io/>*

Re: lucene index program won't start after power failure

2016-09-22 Thread Ziming Dong
> I'll see if I can make a standalone test case showing this. > > > > If you open those indices with an IndexReader instead, does it succeed? > > > > If you run CheckIndex, what does it report? > > > > Mike McCandless > > > > http://blog.mikem

Re: lucene index program won't start after power failure

2016-09-22 Thread Ziming Dong
M, Michael McCandless < luc...@mikemccandless.com> wrote: > Do you have 2 separate segments files in that 2nd index? > > Which exact Lucene version is this? > > Mike McCandless > > http://blog.mikemccandless.com > > > On Thu, Sep 22, 2016 at 7:44 AM, Ziming D

Re: lucene index program won't start after power failure

2016-09-22 Thread Ziming Dong
pen, if your IO system honors > fsync. > > What IO devices are used by the NFS server? > > NFS is not well tested and has several known problems with Lucene so > this is already risky ground... > > Mike McCandless > > http://blog.mikemccandless.com > > On Thu, Sep

Re: lucene index program won't start after power failure

2016-09-25 Thread Ziming Dong
your NFS client and > then to the Mac Mini's NFS server failed to actually move all written > bytes to durable storage. > > Can you reproduce this issue if you use a more well trodden IO system, > e.g. Linux with ext4 on a local IO device? > > Mike McCandless > > http://b

Re: lucene index program won't start after power failure

2016-09-25 Thread Ziming Dong
sorry to resend. I'll change IO to local. Is there anyway to recover first index? now it can not be opened by checkIndex, we are building index of 7 billion webpages, it costs much time to rebuild. On Sun, Sep 25, 2016 at 5:31 PM, Ziming Dong wrote: > I'll change IO to local. Is th

Re: lucene index program won't start after power failure

2016-09-29 Thread Ziming Dong
This is why Lucene is so careful about > writing a new one to a "pending" file, fsync'ing that, fsyncing the > directory, and doing an atomic rename, all before removing the older > segment files. > > Mike McCandless > > http://blog.mikemccandless.com > > &g

Re: lucene index program won't start after power failure

2016-09-29 Thread Ziming Dong
ad/ayl5q6rgtngeuoyy > > Just be careful! Make a backup copy of your index first! > > Mike McCandless > > http://blog.mikemccandless.com > > > On Thu, Sep 29, 2016 at 11:31 AM, Ziming Dong > wrote: > > do you mean `http://www.getopt.org/luke/`? > > > >