Re: Question about startup memory usage

On 11/14/2019 1:46 AM, Hongxu Ma wrote: Thank you @Shawn Heisey , you help me many times. My -xms=1G When restart solr, I can see the progress of memory increasing (from 1G to 9G, took near 10s). I have a guess: maybe solr is loading some needed files into heap

Re: Question about startup memory usage

What's your thoughts? thanks. From: Shawn Heisey Sent: Thursday, November 14, 2019 1:15 To: solr-user@lucene.apache.org Subject: Re: Question about startup memory usage On 11/13/2019 2:03 AM, Hongxu Ma wrote: > I have a solr-cloud cluster with a big collecti

Re: Question about startup memory usage

On 11/13/2019 2:03 AM, Hongxu Ma wrote: I have a solr-cloud cluster with a big collection, after startup (no any search/index operations), its jvm memory usage is 9GB (via top: RES). Cluster and collection info: each host: total 64G mem, two solr nodes with -xmx=15G collection: total 9B

Question about startup memory usage

Hi I have a solr-cloud cluster with a big collection, after startup (no any search/index operations), its jvm memory usage is 9GB (via top: RES). Cluster and collection info: each host: total 64G mem, two solr nodes with -xmx=15G collection: total 9B billion docs (but each doc is very small

Re: Question about memory usage and file handling

(1) no. The internal Ram buffer will pretty much limit the amount of heap used however. (2) You actually have several segments. “.cfs” stands for “Compound File”, see: https://lucene.apache.org/core/7_1_0/core/org/apache/lucene/codecs/lucene70/package-summary.html "An optional "virtual" file

Re: Question about memory usage and file handling

On 11/11/2019 1:40 PM, siddharth teotia wrote: I have a few questions about Lucene indexing and file handling. It would be great if someone can help with these. I had earlier asked these questions on gene...@lucene.apache.org but was asked to seek help here. This mailing list (solr-user) is

Question about memory usage and file handling

Hi All, I have a few questions about Lucene indexing and file handling. It would be great if someone can help with these. I had earlier asked these questions on gene...@lucene.apache.org but was asked to seek help here. (1) During indexing, is there any knob to tell the writer to use off-heap

Re: investigating high heap memory usage particularly on overseer / collection leaders

.0 > 24gb max heap memory > 13 collections, each with 500mb-2gb index (on disk) > > We are investigating high heap memory usage/spikes with our SOLR cluster > (details above). After rebooting the cluster, all three instances stay > under 2gb for about a day. Then sud

investigating high heap memory usage particularly on overseer / collection leaders

3-node SOLR 7.4.0 24gb max heap memory 13 collections, each with 500mb-2gb index (on disk) We are investigating high heap memory usage/spikes with our SOLR cluster (details above). After rebooting the cluster, all three instances stay under 2gb for about a day. Then suddenly, one instance

Re: Synonym filters memory usage

terrible. > > I would be really, really, really surprised if this is the case. The > Lucene people are very careful with memory usage and would hop on this in > an instant if true I’d guess. > > Best, > Erick > > > On Sep 30, 2019, at 5:27 AM, Andrea Gazzarini > wrote

Re: Synonym filters memory usage

with memory usage and would hop on this in an instant if true I’d guess. Best, Erick > On Sep 30, 2019, at 5:27 AM, Andrea Gazzarini wrote: > > That sounds really strange to me. > Segments are created gradually depending on changes applied to the index, > while the Schema should ha

Re: Synonym filters memory usage

so. While integrating a Thesaurus as synonyms.txt I saw massive memory usage. A heap dump and analysis with MemoryAnalyzer pointed out that the SynonymMap took 3 times a huge amount of memory, together with each opened index segment. Just try it and check that by yourself with heap dump

Re: Synonym filters memory usage

Yes, I think so. While integrating a Thesaurus as synonyms.txt I saw massive memory usage. A heap dump and analysis with MemoryAnalyzer pointed out that the SynonymMap took 3 times a huge amount of memory, together with each opened index segment. Just try it and check that by yourself with heap

Re: Synonym filters memory usage

mmm, ok for the core but are you sure things in this case are working per-segment? I would expect a FilterFactory instance per index, initialized at schema loading time. On 30/09/2019 09:04, Bernd Fehling wrote: And I think this is per core per index segment. 2 cores per instance, each core

Re: Synonym filters memory usage

On 30/09/2019 09:04, Bernd Fehling wrote: And I think this is per core per index segment. 2 cores per instance, each core with 3 index segments, sums up to 6 times the 2 SynonymMaps. Results in 12 times SynonymMaps. Regards Bernd Am 30.09.19 um 08:41 schrieb Andrea Gazzarini:   Hi,

Re: Synonym filters memory usage

And I think this is per core per index segment. 2 cores per instance, each core with 3 index segments, sums up to 6 times the 2 SynonymMaps. Results in 12 times SynonymMaps. Regards Bernd Am 30.09.19 um 08:41 schrieb Andrea Gazzarini: Hi, looking at the stateful nature of

Re: Synonym filters memory usage

Hi, looking at the stateful nature of SynonymGraphFilter/FilterFactory classes, the answer should be 2 times (one time per type instance). The SynonymMap, which internally holds the synonyms table, is a private member of the filter factory and it is loaded each time the factory needs to create a

Synonym filters memory usage

Hi, My concern is about memory used by synonym filter, especially if synonyms resources files are large. If in my schema, there are two field types "TypeSyno1" and "TypeSyno2" using synonym filter with the same synonyms files. For each of these two field types there are two fields Field1 type

Re: Determing Solr heap requirments and analyzing memory usage

> https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/util/ConcurrentLRUCache.java#L605. > Surprisingly, the query objects used as keys in the queryResultCache do not > implement Accountable as far as I can tell, and this lines up very well > wi

Re: Determing Solr heap requirments and analyzing memory usage

/solr/util/ConcurrentLRUCache.java#L605. Surprisingly, the query objects used as keys in the queryResultCache do not implement Accountable as far as I can tell, and this lines up very well with our observation of memory usage because in the heap dump we can also see that the keys in the cache

Re: Determing Solr heap requirments and analyzing memory usage

On 4/23/2019 11:48 AM, Brian Ecker wrote: I see. The other files I meant to attach were the GC log ( https://pastebin.com/raw/qeuQwsyd), the heap histogram ( https://pastebin.com/raw/aapKTKTU), and the screenshot from top ( http://oi64.tinypic.com/21r0bk.jpg). I have no idea what to do with

Re: Determing Solr heap requirments and analyzing memory usage

Thanks for your response. See below please for detailed responses. On Tue, Apr 23, 2019 at 6:04 PM Shawn Heisey wrote: > On 4/23/2019 6:34 AM, Brian Ecker wrote: > > What I’m trying to determine is (1) How much heap does > > this setup need before it stabilizes and stops crashing with OOM

Re: Determing Solr heap requirments and analyzing memory usage

On 4/23/2019 6:34 AM, Brian Ecker wrote: What I’m trying to determine is (1) How much heap does this setup need before it stabilizes and stops crashing with OOM errors, (2) can this requirement somehow be reduced so that we can use less memory, and (3) from the heap histogram, what is actually

Determing Solr heap requirments and analyzing memory usage

Hello, We are currently running into a situation where Solr (version 7.4) in slowly using up all available memory allocated to the heap, and then eventually hitting an OutOfMemory error. We have tried increasing the heap size and also tuning the GC settings, but this does not seem to solve the

Re: High CPU and Physical Memory Usage in solr with 4000 user load

Hi Shawn As asked, have attached the gc log and snapshot of top command TopCommandSlave1.jpg and regarding blocked threads, we are fetching facets and doing grouping with the main query and for those fields docValues were

Re: High CPU and Physical Memory Usage in solr with 4000 user load

On 2/9/2018 12:48 AM, rubi.hali wrote: As threads are blocked , our cpu is reaching to 90% even even when heap or os memory is not getting used at all. One of the BLOCKED Thread Snippet Most of the threads are blocked either on Jetty Connector or FieldCacheIMPL for DOC values.

Re: High CPU and Physical Memory Usage in solr with 4000 user load

Hi Shawn We tried one more round of testing after We increased CPU cores on each server from 4 to 16 which amounts to 64 cores in total against 4 slaves.But Still CPU Usage was higher.So We took the thread dumps on one of our slaves and found that threads were blocked. Have also attached them.

Re: High CPU and Physical Memory Usage in solr with 4000 user load

On 2/8/2018 12:41 AM, rubi.hali wrote: > We are using Solr-6.6.2 with one master and 4 Slaves each having 4 CPU core > and 16GB RAM. We were doing load testing with 4000 users and 800 odd search > keywords which resulted into 95% of CPU usage in less than 3 minutes and > affected our QUERY

High CPU and Physical Memory Usage in solr with 4000 user load

Hi We are using Solr-6.6.2 with one master and 4 Slaves each having 4 CPU core and 16GB RAM. We were doing load testing with 4000 users and 800 odd search keywords which resulted into 95% of CPU usage in less than 3 minutes and affected our QUERY Responses. There was spike in physical memory also

Re: 3 color jvm memory usage bar

On Thu, 2017-10-19 at 08:56 -0700, Nawab Zada Asad Iqbal wrote: > I see three colors in the JVM usage bar. Dark Gray, light Gray, > white. (left to right).  Only one dark and one light color made sense > to me (as i could interpret them as used vs available memory), but > there is light gray

Re: 3 color jvm memory usage bar

Thanks Erik I see three colors in the JVM usage bar. Dark Gray, light Gray, white. (left to right). Only one dark and one light color made sense to me (as i could interpret them as used vs available memory), but there is light gray between dark gray and white parts. Thanks Nawab On Thu, Oct

Re: 3 color jvm memory usage bar

Nawab: Images are stripped aggressively by the Apache mail servers, your attachment didn't come through. You'll have to put it somewhere and provide a link. Generally the lighter color in each bar is the available resource and the darker shade is used. Best, Erick On Thu, Oct 19, 2017 at 7:27

3 color jvm memory usage bar

Good morning, What do the 3 colors mean in this bar on Solr dashboard page? (please see attached) : Regards Nawab

Re: Solr 6.5.1 crashing when too many queries with error or high memory usage are queried

case and try to > > fix that. > > > > Can you share your key numbers here so we can do a qualified guess? > > > > -- > > Jan Høydahl, search solution architect > > Cominvent AS - www.cominvent.com > > > > > 2. jul. 2017 kl. 09.00 skrev Zheng Li

Re: Solr 6.5.1 crashing when too many queries with error or high memory usage are queried

e...@gmail.com>: > > > > Hi, > > > > I'm currently facing the issue whereby the Solr crashed when I have > issued > > too many queries with error or those with high memory usage, like JSON > > facet or Streaming expressions. > > > > What could be the issue here? > > > > I'm using Solr 6.5.1 > > > > Regards, > > Edwin > >

Re: Solr 6.5.1 crashing when too many queries with error or high memory usage are queried

m > 2. jul. 2017 kl. 09.00 skrev Zheng Lin Edwin Yeo <edwinye...@gmail.com>: > > Hi, > > I'm currently facing the issue whereby the Solr crashed when I have issued > too many queries with error or those with high memory usage, like JSON > facet or Streaming expressions.

Re: Solr 6.5.1 crashing when too many queries with error or high memory usage are queried

On Sun, 2017-07-02 at 15:00 +0800, Zheng Lin Edwin Yeo wrote: > I'm currently facing the issue whereby the Solr crashed when I have > issued too many queries with error or those with high memory usage, > like JSON facet or Streaming expressions. > > What could be the issue h

Re: Solr 6.5.1 crashing when too many queries with error or high memory usage are queried

Stack trace? Memory diagnostics from top(1)? What querys? On July 2, 2017 3:00:16 AM EDT, Zheng Lin Edwin Yeo <edwinye...@gmail.com> wrote: >Hi, > >I'm currently facing the issue whereby the Solr crashed when I have >issued >too many queries with error or those with high me

Solr 6.5.1 crashing when too many queries with error or high memory usage are queried

Hi, I'm currently facing the issue whereby the Solr crashed when I have issued too many queries with error or those with high memory usage, like JSON facet or Streaming expressions. What could be the issue here? I'm using Solr 6.5.1 Regards, Edwin

Re: Heap memory usage is -1 in UI

in services like zabbix. -- /Yago Riveiro On 23 Sep 2016, 01:08 +0100, Shawn Heisey <apa...@elyograg.org>, wrote: > On 9/22/2016 4:59 PM, Yago Riveiro wrote: > > The Heap Memory Usage in the UI it's always -1. There is some way to > > get the amount of heap that a core

Re: Heap memory usage is -1 in UI

On 9/22/2016 4:59 PM, Yago Riveiro wrote: > The Heap Memory Usage in the UI it's always -1. There is some way to > get the amount of heap that a core consumes? In all the versions that I have looked at, up to 6.0, this number is either entirely too small or -1. Looking into the code, thi

Re: Heap memory usage is -1 in UI

What version of Solr and which Operating System is that on? Regards, Alex On 23 Sep 2016 5:59 AM, "Yago Riveiro" <yago.rive...@gmail.com> wrote: > The Heap Memory Usage in the UI it's always -1. > > There is some way to get the amount of heap that a core consu

Heap memory usage is -1 in UI

The Heap Memory Usage in the UI it's always -1. There is some way to get the amount of heap that a core consumes? - Best regards -- View this message in context: http://lucene.472066.n3.nabble.com/Heap-memory-usage-is-1-in-UI-tp4297601.html Sent from the Solr - User mailing list archive

Re: Memory Usage increases by a lot during and after optimization .

S (or in your case, Shareable from Working) > >> reveals the actual memory being used, and I believe you can see this > >> actual number in the Private column, which is approximately the > >> difference between Working and Shareable. If I'm right, this means that > >> the

Re: Memory Usage increases by a lot during and after optimization .

ase, Shareable from Working) >reveals the actual memory being used, and I believe you can see this >actual number in the Private column, which is approximately the >difference between Working and Shareable. If I'm right, this means that >the actual memory usage is almost 14GB lower than Windo

Re: Memory Usage increases by a lot during and after optimization .

approximately the difference between Working and Shareable. If I'm right, this means that the actual memory usage is almost 14GB lower than Windows is reporting. If both of the top processes are Solr, I'm not sure why you have two Solr processes on one machine. One Solr instance can handle mu

Re: Memory Usage increases by a lot during and after optimization .

Hi Shawn, Thanks for your reply. I have uploaded the screenshot here https://www.dropbox.com/s/l5itfbaus1c9793/Memmory%20Usage.png?dl=0 Basically, Java(TM) Platform SE Library, which Solr is running on, is only using about 22GB currently. However, the memory usage at the top says it is using 73

Re: Memory Usage increases by a lot during and after optimization .

Hi Toke, I read the server's memory usage from the Task manager under Windows, Regards, Edwin On 4 January 2016 at 17:17, Toke Eskildsen <t...@statsbiblioteket.dk> wrote: > On Mon, 2016-01-04 at 10:05 +0800, Zheng Lin Edwin Yeo wrote: > > A) Before I start the optimizatio

Re: Memory Usage increases by a lot during and after optimization .

On 1/5/2016 9:59 AM, Zheng Lin Edwin Yeo wrote: > I have uploaded the screenshot here > https://www.dropbox.com/s/l5itfbaus1c9793/Memmory%20Usage.png?dl=0 > > Basically, Java(TM) Platform SE Library, which Solr is running on, is only > using about 22GB currently. However, t

Re: Memory Usage increases by a lot during and after optimization .

n Edwin Yeo wrote: > > I have uploaded the screenshot here > > https://www.dropbox.com/s/l5itfbaus1c9793/Memmory%20Usage.png?dl=0 > > > > Basically, Java(TM) Platform SE Library, which Solr is running on, is > only > > using about 22GB currently. However, the memory

Re: Memory Usage increases by a lot during and after optimization .

On Mon, 2016-01-04 at 10:05 +0800, Zheng Lin Edwin Yeo wrote: > A) Before I start the optimization, the server's memory usage > is consistent at around 16GB, when Solr startsup and we did some searching. How do you read this number? > However, when I click on the optimization button, t

Re: Memory Usage increases by a lot during and after optimization .

On 1/3/2016 7:05 PM, Zheng Lin Edwin Yeo wrote: > A) Before I start the optimization, the server's memory usage > is consistent at around 16GB, when Solr startsup and we did some searching. > However, when I click on the optimization button, the memory usage > increases gradually, unti

Re: Memory Usage increases by a lot during and after optimization .

memory usage is consistent at around 16GB, when Solr startsup and we did some searching. However, when I click on the optimization button, the memory usage increases gradually, until it reaches the maximum of 64GB which the server has. But this only happens to the collection with index of 200GB

Re: Memory Usage increases by a lot during and after optimization .

On 12/31/2015 8:03 PM, Zheng Lin Edwin Yeo wrote: >> But the problem I'm facing now is that during optimizing, the memory usage >> of the server hit the maximum of 64GB, and I believe the optimization could >> not be completed fully as there is not enough memory, so whe

Re: Memory Usage increases by a lot during and after optimization .

On 12/31/2015 8:03 PM, Zheng Lin Edwin Yeo wrote: > But the problem I'm facing now is that during optimizing, the memory usage > of the server hit the maximum of 64GB, and I believe the optimization could > not be completed fully as there is not enough memory, so when I check the >

Re: Memory Usage increases by a lot during and after optimization .

gt; >> wun...@wunderwood.org > >> http://observer.wunderwood.org/ (my blog) > >> > >>> On Dec 29, 2015, at 6:35 PM, Zheng Lin Edwin Yeo <edwinye...@gmail.com > > > >> wrote: > >>> > >>> Hi, > >>> > >>&g

Re: Memory Usage increases by a lot during and after optimization .

Hi Yonik, Yes, the plan is to do the optimizing at night after indexing, when there are lesser user who will use the system. But the problem I'm facing now is that during optimizing, the memory usage of the server hit the maximum of 64GB, and I believe the optimization could not be completed

Memory Usage increases by a lot during and after optimization .

Hi, I am facing a situation, when I do an optimization by clicking on the "Optimized" button on the Solr Admin Overview UI, the memory usage of the server increases gradually, until it reaches near the maximum memory available. There is 64GB of memory available in the server.

Re: Memory Usage increases by a lot during and after optimization .

facing a situation, when I do an optimization by clicking on the > "Optimized" button on the Solr Admin Overview UI, the memory usage of the > server increases gradually, until it reaches near the maximum memory > available. There is 64GB of memory available in the serve

Re: Memory Usage increases by a lot during and after optimization .

Zheng Lin Edwin Yeo <edwinye...@gmail.com> > wrote: > > > > Hi, > > > > I am facing a situation, when I do an optimization by clicking on the > > "Optimized" button on the Solr Admin Overview UI, the memory usage of the > > server increas

Re: Memory Usage increases by a lot during and after optimization .

Edwin Yeo <edwinye...@gmail.com> >> wrote: >>> >>> Hi, >>> >>> I am facing a situation, when I do an optimization by clicking on the >>> "Optimized" button on the Solr Admin Overview UI, the memory usage of the >>

Re: Memory Usage increases by a lot during and after optimization .

Some people also want to control when major segment merges happen, and optimizing at a known time helps prevent a major merge at an unknown time (which can be equivalent to an optimize/forceMerge). The benefits of optimizing (and having fewer segments to search across) will vary depending on the

Re: Memory Usage increases by a lot during and after optimization .

Thanks for the information. Another thing I like to confirm is, will the Java Heap size setting affect the optimization process or the memory usage? Is the any recommended setting that we can use, for an index size of 200GB? Regards, Edwin On 30 December 2015 at 11:07, Walter Underwood <

Re: Memory Usage increases by a lot during and after optimization .

Question: does anyone have example good merge settings for solrconfig ? To keep the number of segments small like 6? On Tue, Dec 29, 2015 at 8:49 PM, Yonik Seeley wrote: > Some people also want to control when major segment merges happen, and > optimizing at a known time

Re: Solr memory usage

is at 5 GB and other times it is at 10 GB (this is while I'm > running my search tests). What does such high fluctuation means? > > If it helps, Solr's "JVM-Memory" report states 2.5 GB usage when Solr is > first started and before I run any search on it. I'm taking this as

Re: Solr memory usage

t states 2.5 GB usage when Solr is > first started and before I run any search on it. I'm taking this as my > base startup memory usage. The heap usage at any particular instant in time (even right after startup) is nearly useless information. To reach any useful conclusions and change yo

RE: Solr memory usage

- > From:Steven White <swhite4...@gmail.com> > Sent: Wednesday 9th December 2015 15:56 > To: solr-user@lucene.apache.org > Subject: Re: Solr memory usage > > Thanks Erick!! Your summary and the blog by Uwe (thank you too Uwe) are > very helpful. > > A fo

Re: Solr memory usage

'm taking this as my base startup memory usage. Steve On Tue, Dec 8, 2015 at 3:17 PM, Erick Erickson <erickerick...@gmail.com> wrote: > You're doing nothing wrong, that particular bit of advice has > always needed a bit of explanation. > > Solr (well, actually Lucene) uses MMapDirec

Solr memory usage

Hi folks, My index size on disk (optimized) is 20 GB (single core, single index). I have a system with 64 GB of RAM. I start Solr with 24 GB of RAM. I have run load tests (up to 100 concurrent users) for hours where each user issuing unique searches (the same search is never executed again for

Re: Solr memory usage

You're doing nothing wrong, that particular bit of advice has always needed a bit of explanation. Solr (well, actually Lucene) uses MMapDirectory for much of the index structure which uses the OS memory rather than the JVM heap. See Uwe's excellent:

Re: Confusing SOLR 5 memory usage

Tom Evans tevans...@googlemail.com wrote: I do apologise for wasting anyone's time on this, the PEBKAC (my keyboard and chair unfortunately). When adding the new server to haproxy, I updated the label for the balancer entry to the new server, but left the host name the same, so the server that

Re: Confusing SOLR 5 memory usage

time? You should see a typical saw tooth. Markus -Original message- From:Tom Evans tevans...@googlemail.com Sent: Tuesday 21st April 2015 12:22 To: solr-user@lucene.apache.org Subject: Confusing SOLR 5 memory usage Hi all I have two SOLR 5 servers, one is the master and one

Re: Confusing SOLR 5 memory usage

Sent: Tuesday 21st April 2015 12:22 To: solr-user@lucene.apache.org Subject: Confusing SOLR 5 memory usage Hi all I have two SOLR 5 servers, one is the master and one is the slave. They both have 12 cores, fully replicated and giving identical results when querying them. The only difference

Confusing SOLR 5 memory usage

. When viewing the memory usage of the servers, the master server's JVM has 8.8GB RSS, but the slave only has 1.2GB RSS. Can someone hit me with the cluebat please? :) Cheers Tom

RE: Confusing SOLR 5 memory usage

Hi - what do you see if you monitor memory over time? You should see a typical saw tooth. Markus -Original message- From:Tom Evans tevans...@googlemail.com Sent: Tuesday 21st April 2015 12:22 To: solr-user@lucene.apache.org Subject: Confusing SOLR 5 memory usage Hi all I have

Re: High memory usage while querying with sort using cursor

Thanks Chris, that makes a lot of sense. On Wed, Mar 18, 2015 at 3:16 PM, Chris Hostetter hossman_luc...@fucit.org wrote: : A simple query on the collection: ../select?q=*:* works perfectly fine. : : But as soon as i add sorting, it crashes the nodes with OOM: :

Re: High memory usage while querying with sort using cursor

: A simple query on the collection: ../select?q=*:* works perfectly fine. : : But as soon as i add sorting, it crashes the nodes with OOM: : .../select?q=*:*sort=unique_id ascrows=0. if you don't have docValues=true on your unique_id field, then sorting rquires it to build up a large in memory

High memory usage while querying with sort using cursor

Hi all, My setup is as follows: *Collection* size: 32GB, 2 shards, replication factor: 2 (~16GB on each replica). Number of rows: 250million 4 *Solr* nodes: RAM: 30GB each. Heap size: 8GB. Version: 4.9.1 Besides the collection in question, the nodes have some other collections present. The

Re: Field collapsing memory usage

On Thu, 2015-01-22 at 22:52 +0100, Erick Erickson wrote: What do you think about folding this into the Solr (or Lucene?) code base? Or is it to specialized? (writing under the assumption that DVEnabler actually works as it should for everyone and not just us) Right now it is an explicit tool.

RE: Field collapsing memory usage

Norgorn [lsunnyd...@mail.ru] wrote: Nice, thanks! If u'd like to, I'll write our results with that amazing util. By all means, please do. Good as well as bad. Independent testing is needed to ensure proper working tools. - Toke Eskildsen

RE: Field collapsing memory usage

Nice, thanks! If u'd like to, I'll write our results with that amazing util. -- View this message in context: http://lucene.472066.n3.nabble.com/Field-collapsing-memory-usage-tp4181092p4181159.html Sent from the Solr - User mailing list archive at Nabble.com.

RE: Field collapsing memory usage

. Is there any way to make 'docValues=true' without reindexing? -- View this message in context: http://lucene.472066.n3.nabble.com/Field-collapsing-memory-usage-tp4181092p4181108.html Sent from the Solr - User mailing list archive at Nabble.com.

RE: Field collapsing memory usage

Norgorn [lsunnyd...@mail.ru] wrote: Is there any way to make 'docValues=true' without reindexing? Depends on how brave you are :-) We recently had the same need and made https://github.com/netarchivesuite/dvenabler To my knowledge that is the only existing tool for that task an as we are the

Re: Field collapsing memory usage

Toke: What do you think about folding this into the Solr (or Lucene?) code base? Or is it to specialized? Not sure one way or the other, just askin' Erick On Thu, Jan 22, 2015 at 3:47 AM, Toke Eskildsen t...@statsbiblioteket.dk wrote: Norgorn [lsunnyd...@mail.ru] wrote: Is there any way

RE: Field collapsing memory usage

that it turned on for your group field: We do grouping on indexes with 250M documents (and 200M+ unique values in the group field) without any significant memory overhead, using DocValues. Caveat: If you ask for very large result sets, the memory usage will be high. But only temporarily. - Toke Eskildsen

Field collapsing memory usage

/Field-collapsing-memory-usage-tp4181092.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr Memory Usage - How to reduce memory footprint for solr

And keep in mind that starving the OS of memory to give it to the JVM is an anti-pattern, see Uwe's excellent blog on MMapDirectory here: http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html Best, Erick On Wed, Jan 7, 2015 at 5:55 AM, Shawn Heisey apa...@elyograg.org wrote:

Re: Solr Memory Usage - How to reduce memory footprint for solr

On 1/6/2015 1:10 PM, Abhishek Sharma wrote: *Q* - I am forced to set Java Xmx as high as 3.5g for my solr app.. If i keep this low, my CPU hits 100% and response time for indexing increases a lot.. And i have hit OOM Error as well when this value is low.. Is this too high? If so, how can I

RE: Solr Memory Usage - How to reduce memory footprint for solr

. num. of Search Hits per sec - 10 [*IMP* - All search queries have faceting..] Faceting is often the reason for high memory usage. If you are not already doing so, do enable DocValues for the fields you are faceting on. If you have a lot of unique values in your facets (millions), you might

Solr Memory Usage - How to reduce memory footprint for solr

*Q* - I am forced to set Java Xmx as high as 3.5g for my solr app.. If i keep this low, my CPU hits 100% and response time for indexing increases a lot.. And i have hit OOM Error as well when this value is low.. Is this too high? If so, how can I reduce this? *Machine Details* 4 G RAM, SSD

Re: Solr Memory Usage

On Wed, 2014-10-29 at 23:37 +0100, Will Martin wrote: This command only touches OS level caches that hold pages destined for (or not) the swap cache. Its use means that disk will be hit on future requests, but in many instances the pages were headed for ejection anyway. It does not have

Solr Memory Usage

I am observing some weird behavior with how Solr is using memory. We are running both Solr and zookeeper on the same node. We tested memory settings on Solr Cloud Setup of 1 shard with 146GB index size, and 2 Shard Solr setup with 44GB index size. Both are running on similar beefy machines.

Re: Solr Memory Usage

On 10/29/2014 11:43 AM, Vijay Kokatnur wrote: I am observing some weird behavior with how Solr is using memory. We are running both Solr and zookeeper on the same node. We tested memory settings on Solr Cloud Setup of 1 shard with 146GB index size, and 2 Shard Solr setup with 44GB index

RE: Solr Memory Usage

Vijay Kokatnur [kokatnur.vi...@gmail.com] wrote: For the Solr Cloud setup, we are running a cron job with following command to clear out the inactive memory. It is working as expected. Even though the index size of Cloud is 146GB, the used memory is always below 55GB. Our response times are

RE: Solr Memory Usage

don't even research the matter. -Original Message- From: Toke Eskildsen [mailto:t...@statsbiblioteket.dk] Sent: Wednesday, October 29, 2014 3:06 PM To: solr-user@lucene.apache.org Subject: RE: Solr Memory Usage Vijay Kokatnur [kokatnur.vi...@gmail.com] wrote: For the Solr Cloud setup

RE: Solr Memory Usage

@lucene.apache.org' Subject: RE: Solr Memory Usage This command only touches OS level caches that hold pages destined for (or not) the swap cache. Its use means that disk will be hit on future requests, but in many instances the pages were headed for ejection anyway. It does not have anything whatsoever

Re: Solr Memory Usage

On 10/29/2014 1:05 PM, Toke Eskildsen wrote: We did have some problems on a 256GB machine churning terabytes of data through 40 concurrent Tika processes and into Solr. After some days, performance got really bad. When we did a top, we noticed that most of the time was used in the kernel

RE: Solr configuration, memory usage and MMapDirectory

for these, are these what are using up the memory? Thanks Si -Original Message- From: Shawn Heisey [mailto:apa...@elyograg.org] Sent: 06 October 2014 16:56 To: solr-user@lucene.apache.org Subject: Re: Solr configuration, memory usage and MMapDirectory On 10/6/2014 9:24 AM, Simon Fairey

Re: Solr configuration, memory usage and MMapDirectory

On 10/8/2014 4:02 AM, Simon Fairey wrote: I'm currently setting up jconsole but as I have to remotely monitor (no gui capability on the server) I have to wait before I can restart solr with a JMX port setup. In the meantime I looked at top and given the calculations you said based on your

RE: Solr configuration, memory usage and MMapDirectory

- From: Shawn Heisey [mailto:apa...@elyograg.org] Sent: 08 October 2014 21:09 To: solr-user@lucene.apache.org Subject: Re: Solr configuration, memory usage and MMapDirectory On 10/8/2014 4:02 AM, Simon Fairey wrote: I'm currently setting up jconsole but as I have to remotely monitor (no gui

  1   2   3   >