Hmm... so in fact it looks like Solr had done a number of commits already (especially, given how large your generation is -- the "cje" in segments_cje means there were a number of commits).
Were there any other exceptions leading up to this? Disk full? Anything unusual in your Solr configuration? Is there any chance that a 2nd Solr core attempted to access this same directory? Mike On Tue, May 26, 2009 at 9:33 PM, Wang Guangchen <guangchen...@gmail.com> wrote: > Hi Mike, > > The index is autoCommit every 1000 docs. I set this to increase the indexing > speed. What is the best configuration do you suggest for each commit cycle? > > Thank you very much for your help. > > Following is the original exception: > > java.lang.RuntimeException: java.io.FileNotFoundException: > /solr/example/data/index/segments_cje (No such file or directory) > at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1085) > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:561) > at > org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:121) > > at > org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:69) > at > org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:275) > at > org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:397) > > at > org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:108) > > at > org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:3709) > at > org.apache.catalina.core.StandardContext.start(StandardContext.java:4363) > at > org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791) > > at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771) > at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:525) > at > org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:627) > > at > org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:553) > at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:488) > at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1149) > at > org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:311) > at > org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117) > > at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053) > at org.apache.catalina.core.StandardHost.start(StandardHost.java:719) > at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045) > at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443) > at org.apache.catalina.core.StandardService.start(StandardService.java:516) > at org.apache.catalina.core.StandardServer.start(StandardServer.java:710) > at org.apache.catalina.startup.Catalina.start(Catalina.java:578) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288) > at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413) Caused by: > java.io.FileNotFoundException: > /mnt_APS/solr/solrHomeFull/data/index/segments_cje (No such file or > directory) > at java.io.RandomAccessFile.open(Native Method) > at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212) > at > org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:630) > at > org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:660) > > at > org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.<init>(NIOFSDirectory.java:76) > > at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:63) > at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:560) > at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:224) > at > org.apache.lucene.index.DirectoryIndexReader$1.doBody(DirectoryIndexReader.java:103) > > at > org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:688) > > at > org.apache.lucene.index.DirectoryIndexReader.open(DirectoryIndexReader.java:123) > at org.apache.lucene.index.IndexReader.open(IndexReader.java:316) > at org.apache.lucene.index.IndexReader.open(IndexReader.java:237) > at org.apache.solr.search.SolrIndexSearcher.<init> > > > > On Wed, May 27, 2009 at 2:32 AM, Michael McCandless < > luc...@mikemccandless.com> wrote: > >> It sounds like you had never committed while building the original >> index? Unfortunately, it's not easy to recover an index in that >> state. It's best to periodically commit if you're building such a >> large index. >> >> Do you have the original exception you hit? >> >> I'll fix CheckIndex to be more sane if it could not load any segments file. >> >> Mike >> >> On Tue, May 26, 2009 at 2:12 AM, Wang Guangchen <guangchen...@gmail.com> >> wrote: >> > Hi peter, >> > >> > Thank you very much for your quick reply. >> > >> > I tried the CheckIndex method. It can't work on my crashed index. >> > In the error message, it says the segments file in the directory is >> missing. >> > and when I use the -fix param, new segments file still can't be write. >> > I even try the CheckIndex without the assertion, it still can't work. >> > >> > >> > Do you know why this is happening ? Does it mean that the segment file >> can't >> > be rewrite at all? >> > >> > Btw, i am using the nightly build solr. >> > >> > following is the error messages: >> > >> > [r...@localhost lib]# java -cp lucene-core-2.9-dev.jar >> > org.apache.lucene.index.CheckIndex -fix /solr/example/data/index/ >> > >> > NOTE: testing will be more thorough if you run java with >> > '-ea:org.apache.lucene...', so assertions are enabled >> > >> > Opening index @ /solr/example/data/index/ >> > >> > ERROR: could not read any segments file in directory >> > java.io.FileNotFoundException: /solr/example >> > >> > /data/index/segments_cje (No such file or directory) >> > at java.io.RandomAccessFile.open(Native Method) >> > at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212) >> > at >> > >> org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:630) >> > at >> > >> org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:660) >> > at >> > org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:566) >> > at >> > org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:560) >> > at >> org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:224) >> > at >> > org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:292) >> > at >> > >> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:688) >> > at >> org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:289) >> > at >> > org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:258) >> > at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:678) >> > WARNING: 0 documents will be lost >> > >> > NOTE: will write new segments file in 5 seconds; this will remove 0 docs >> > from the index. THIS IS YOUR LAST CHANCE TO CTRL+C! >> > 5... >> > 4... >> > 3... >> > 2... >> > 1... >> > Writing... >> > Exception in thread "main" java.lang.NullPointerException >> > at >> org.apache.lucene.index.CheckIndex.fixIndex(CheckIndex.java:556) >> > at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:697) >> > >> > >> > Regards >> > >> > GC >> > >> > >> > >> > On Tue, May 26, 2009 at 12:49 AM, Peter Wolanin < >> peter.wola...@acquia.com>wrote: >> > >> >> you can use the lucene jar with solr to invoke the CheckIndex method - >> >> this will possibly allow you to recover if you pass the -fix param. >> >> >> >> You may lose some docs, however, so this is only viable if you can, >> >> for example, query to check what's missing. >> >> >> >> The command looks like (from the root of the solr svn checkout): >> >> >> >> java -ea:org.apache.lucene -cp lib/lucene-core-2.9-dev.jar >> >> org.apache.lucene.index.CheckIndex [path to index directory] >> >> >> >> For example, to check the example index: >> >> >> >> java -ea:org.apache.lucene -cp lib/lucene-core-2.9-dev.jar >> >> org.apache.lucene.index.CheckIndex example/solr/data/index/ >> >> >> >> -Peter >> >> >> >> On Mon, May 25, 2009 at 4:42 AM, Wang Guangchen <guangchen...@gmail.com >> > >> >> wrote: >> >> > Hi everyone, >> >> > >> >> > I have 8m docs to index, and each doc is around 50kb. The solr crashed >> in >> >> > the middle of indexing. error message said that one of the file in the >> >> data >> >> > directory is missing. I don't know why this is happened. >> >> > >> >> > So right now I have to find a way to recover the index to avoid >> re-index. >> >> Is >> >> > there anyone know any tools or method to recover the crashed index? >> >> Please >> >> > help. >> >> > >> >> > Thanks a lot. >> >> > >> >> > Regards >> >> > GC >> >> > >> >> >> >> >> >> >> >> -- >> >> Peter M. Wolanin, Ph.D. >> >> Momentum Specialist, Acquia. Inc. >> >> peter.wola...@acquia.com >> >> >> > >> >
