Re:  OutOfMemoryError throwed by SimpleMergedSegmentWarmer

2014-09-13 Thread 308181687
uot;; Subject: Re: OutOfMemoryError throwed by SimpleMergedSegmentWarmer Norms are not stored sparsely by the default codec. So they take 1 byte per doc per indexed field regardless of whether that doc had that field. There is no setting to turn this off in IndexReader, though you could make

Re: OutOfMemoryError throwed by SimpleMergedSegmentWarmer

2014-09-13 Thread Michael McCandless
We have 8 million documents and our jvm heap is 5G.‍ > > > Thanks & Best Regards! > > > ‍ > > > > -- Original -- > From: "Michael McCandless";; > Date: Sat, Sep 13, 2014 06:29 PM > To: "Lucene Users"; &

Re:   OutOfMemoryError throwed by SimpleMergedSegmentWarmer

2014-09-13 Thread 308181687
ava-user"; Subject: Re: OutOfMemoryError throwed by SimpleMergedSegmentWarmer Hi, Mike ‍ In our use case, we have thousands of index fields, different kind of document have different fields. Do you meant that ‍norms field will consume large memory? Why?‍ If we decide to disabl

Re:  OutOfMemoryError throwed by SimpleMergedSegmentWarmer

2014-09-13 Thread 308181687
documents and our jvm heap is 5G.‍ Thanks & Best Regards! ‍ -- Original -- From: "Michael McCandless";; Date: Sat, Sep 13, 2014 06:29 PM To: "Lucene Users"; Subject: Re: OutOfMemoryError throwed by SimpleMergedSegmentWarmer The w

Re: OutOfMemoryError throwed by SimpleMergedSegmentWarmer

2014-09-13 Thread Michael McCandless
regardless of whether that doc had indexed that field), or increase HEAP to the JVM. Mike McCandless http://blog.mikemccandless.com On Sat, Sep 13, 2014 at 4:25 AM, 308181687 <308181...@qq.com> wrote: > Hi, all > we got an OutOfMemoryError throwed ‍by SimpleMergedSegmentWarmer. We use

OutOfMemoryError throwed by SimpleMergedSegmentWarmer

2014-09-13 Thread 308181687
Hi, all we got an OutOfMemoryError throwed ‍by SimpleMergedSegmentWarmer. We use lucene 4.7, and access index file by NRTCachingDirectory/MMapDirectory. Could any body give me a hand? Strack trace is as follows: org.apache.lucene.index.MergePolicy$MergeException