That NPE is happening inside Cassandra's sources; I think you need to trace what's happening there and how its FileBlock can be null?
It looks like it's a bug on how CassandraDirectory handles compound files (e.g. _0.cfs), which are somewhat tricky because it's a file that acts itself like a Directory, since it's essentially an archive, containing multiple files within it. Have a look at how the createSlicer method is implemented? Is it somehow failing to set the FileBlock? Mike McCandless http://blog.mikemccandless.com On Mon, Feb 17, 2014 at 3:50 AM, Jason Wee <peich...@gmail.com> wrote: > Hi Mike, > > Thank you. > > This exception is pretty clear that during lucene execute readInternal(...) > on _0.cfs and encountered an npe. The root cause is because the object > being read, FileBlock is null. As far as i can tell, it happen only during > reading _0.cfs but not on the index files that were read before ( that is, > for example, segments.gen, segments_1, _0.cfs). > > It's pretty mind boggling to understand without a better description on how > lucene read the file. Tried to search in google, lucene wiki, lucene source > repository on the lib and your blog but without much avail, could you give > some pointer or write a general description on what happened > after IndexReader reader = DirectoryReader.open(cassandraDirectory); ? > > 2014-02-17 16:40:48 CassandraDirectory [INFO] called length() and returning > 1034 > 2014-02-17 16:40:48 BufferedIndexInput [INFO] length = '1034' > 2014-02-17 16:40:48 CassandraDirectory [INFO] called length() and returning > 1034 > 2014-02-17 16:40:48 CassandraDirectory [TRACE] read internal to bytes with > offset 0 and length 309 > 2014-02-17 16:40:48 CassandraDirectory [INFO] fileDescriptor name = > '_0.cfs' fileLength = '1034' > 2014-02-17 16:40:48 CassandraDirectory [INFO] fileDescriptor length 1034 > fileDescriptor blockSize 1 > java.lang.NullPointerException > at > org.apache.lucene.store.CassandraDirectory$CassandraIndexInput.readInternal(CassandraDirectory.java:1850) > at > org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:178) > at > org.apache.lucene.store.Directory$SlicedIndexInput.readInternal(Directory.java:306) > at > org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:298) > at > org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:50) > at org.apache.lucene.store.DataInput.readInt(DataInput.java:84) > at > org.apache.lucene.store.BufferedIndexInput.readInt(BufferedIndexInput.java:202) > at org.apache.lucene.codecs.CodecUtil.checkHeader(CodecUtil.java:126) > at > org.apache.lucene.codecs.lucene46.Lucene46FieldInfosReader.read(Lucene46FieldInfosReader.java:56) > at > org.apache.lucene.index.SegmentReader.readFieldInfos(SegmentReader.java:214) > at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:94) > at > org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:62) > at > org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:843) > at > org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:52) > at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:66) > at org.apache.lucene.store.Search.<init>(Search.java:41) > at org.apache.lucene.store.Search.main(Search.java:34) > > > On Fri, Feb 14, 2014 at 7:14 PM, Michael McCandless < > luc...@mikemccandless.com> wrote: > >> This means Lucene was attempting to open _0.fnm but somehow got the >> contents of _0.cfs instead; seems likely that it's a bug in the >> Cassanda Directory implementation? Somehow it's opening the wrong >> file name? >> >> Mike McCandless >> >> http://blog.mikemccandless.com >> >> >> On Fri, Feb 14, 2014 at 3:13 AM, Jason Wee <peich...@gmail.com> wrote: >> > Hello, >> > >> > This is my first question to lucene mailing list, sorry if the question >> > sounds funny. >> > >> > I have been experimenting to store lucene index files on cassandra, >> > unfortunately the exception got overwhelmed. Below are the stacktrace. >> > >> > org.apache.lucene.index.CorruptIndexException: codec mismatch: actual >> > codec=CompoundFileWriterData vs expected codec=Lucene46FieldInfos >> > (resource: SlicedIndexInput(SlicedIndexInput(_0.fnm in >> > lucene-cassandra-desc) in lucene-cassandra-desc slice=31:340)) >> > at >> org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:140) >> > at org.apache.lucene.codecs.CodecUtil.checkHeader(CodecUtil.java:130) >> > at >> > >> org.apache.lucene.codecs.lucene46.Lucene46FieldInfosReader.read(Lucene46FieldInfosReader.java:56) >> > at >> > >> org.apache.lucene.index.SegmentReader.readFieldInfos(SegmentReader.java:214) >> > at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:94) >> > at >> > >> org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:62) >> > at >> > >> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:843) >> > at >> > >> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:52) >> > at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:66) >> > at org.apache.lucene.store.Search.<init>(Search.java:41) >> > at org.apache.lucene.store.Search.main(Search.java:34) >> > >> > I'm not sure what does it means, can anybody help? >> > >> > When I check the hex representation of _0.fnm in cassandra, and >> translated >> > to ascii. It is something like this: >> > >> ??l??Lucene46FieldInfos??????path?Q??????????????PerFieldPostingsFormat.format?Lucene41?PerFieldPostingsFormat.suffix?0?modified?Q??????????????PerFieldPostingsFormat.format?Lucene41?PerFieldPostingsFormat.suffix?0?contents????????????????PerFieldPostingsFormat.format?Lucene41?PerFieldPostingsFormat.suffix?0 >> > >> > It looks to me the expected codec is found in the _0.fnm file or am I >> wrong? >> > >> > Thank you and please let me know if you need additional information. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> >> --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
