Wait, in fst.ByteStore I see "only" 5'485'824 -- does this mean 5485824 bytes, or ~5.2 MB? This is probably "correct", meaning this is the RAM to hold the terms index. But I can't see from your heap dump output where the other ~51.3 MB is being used by StandardDirectoryReader.
Mike McCandless http://blog.mikemccandless.com On Sun, Jun 15, 2014 at 2:16 PM, Clemens Wyss DEV <clemens...@mysign.ch> wrote: >> bytes (ie, ~56.5 MB) being used by the StandardDirectoryReader? > Right, namely in org.apache.lucene.util.fst.BytesStore's > > Class Name > | Shallow Heap | Retained Heap | Percentage > ---------------------------------------------------------------------------------------------------------------------------------------------- > org.apache.lucene.index.StandardDirectoryReader @ 0x783932460 > | 72 | 59'255'872 | 3.04% > |- org.apache.lucene.index.SegmentReader[24] @ 0x794089ee0 > | 112 | 59'190'960 | 3.03% > | |- org.apache.lucene.index.SegmentReader @ 0x788820f40 > | 72 | 16'905'072 | 0.87% > | | |- org.apache.lucene.index.SegmentCoreReaders @ 0x7910cacc8 > | 56 | 16'895'576 | 0.87% > | | | |- > org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader @ > 0x780661c50 | 24 | 16'864'864 | 0.86% > | | | | |- org.apache.lucene.codecs.BlockTreeTermsReader @ 0x7910cae50 > | 56 | 16'864'240 | 0.86% > | | | | | |- java.util.TreeMap @ 0x783902738 > | 48 | 16'858'472 | 0.86% > | | | | | | '- java.util.TreeMap$Entry @ 0x77ec5f9f8 > | 40 | 16'858'424 | 0.86% > | | | | | | |- java.util.TreeMap$Entry @ 0x77ec5fa20 > | 40 | 10'895'656 | 0.56% > | | | | | | | |- > org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader @ 0x78347fc08| > 72 | 5'497'328 | 0.28% > | | | | | | | | |- org.apache.lucene.util.fst.FST @ 0x788fe36d0 > | 104 | 5'497'208 | 0.28% > | | | | | | | | | |- org.apache.lucene.util.fst.BytesStore @ > 0x77ec5fe80 | 40 | 5'485'824 | 0.28% > | | | | | | | | | |- org.apache.lucene.util.fst.FST$Arc[128] @ > 0x786efec80 | 528 | 11'216 | 0.00% > | | | | | | | | | |- org.apache.lucene.util.BytesRef @ > 0x780664410 | 24 | 48 | 0.00% > | | | | | | | | | |- int[0] @ 0x7806643f8 > | 16 | 16 | 0.00% > | | | | | | | | | '- Total: 4 entries > | | | > ---------------------------------------------------------------------------------------------------------------------------------------------- > -----Ursprüngliche Nachricht----- > Von: Michael McCandless [mailto:luc...@mikemccandless.com] > Gesendet: Freitag, 13. Juni 2014 15:48 > An: Lucene Users > Betreff: Re: [lucene 4.6] NPE when calling IndexReader#openIfChanged > > On Fri, Jun 13, 2014 at 8:53 AM, Clemens Wyss DEV <clemens...@mysign.ch> > wrote: >> Thanks a lot! >>>"large text fields" >> What is a good limit (in characters) to switch from StringField to >> TextField? Do <Langugae>Analyzers (e.g. GermanAnalyzer) help a lot in >> reducing the size of an Index? > > It's more based on your app's requirements. StringField indexes everything > as a single token. > >>> Add XXXDocValuesField instead of e.g. StringField. >> Does this apply only for StringFields? Or for TextFields too? >> >>> Upgrade to the upcoming Lucene 4.9 >> we have not yet transitionen to Java 7/8 ... hopefully soon ;) >> >>> and take a heap dump and see what's using RAM >> Find attached a snippet from MemoryAnalyzer > > Does this say 59255872 bytes (ie, ~56.5 MB) being used by the > StandardDirectoryReader? > > I'm a little confused because I don't see which structures sum up to that > total. And I would expect the FST (terms index) to take more RAM. > > Mike McCandless > > http://blog.mikemccandless.com > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
