t; Sent: Tuesday, March 19, 2019 3:29:17 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr index slow response
>
> Indexing is CPU bound. If you have enough RAM, SSD disks, and enough client
> threads, you should be able to drive CPU to over 90%.
>
> Start with two cl
time.
I will try with Solr Could cluster, maybe get better speed there.
//Aaron
From: Walter Underwood
Sent: Tuesday, March 19, 2019 3:29:17 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr index slow response
Indexing is CPU bound. If you have enough RAM
7;s constantly happening,
>> almost half of the request has such long delay. The more document added in
>> one request the more delay it has.
>>
>>
>> From: Emir Arnautović
>> Sent: Tuesday, March 19, 2019 12:30:33 PM
>>
I'll second Emir's suggestion to try disabling swap. "I doubt swap would
affect it since there is such huge free memory." -- sounds reasonable, but
has not been my experience, and the stats you sent indicate that swap is in
fact being used. Also, note that in many cases setting vm.swappiness=0 is
n
Indexing is CPU bound. If you have enough RAM, SSD disks, and enough client
threads, you should be able to drive CPU to over 90%.
Start with two client threads per CPU. That allows one thread to be sending
data over the network while another is waiting for Solr to process the batch.
A couple of
Isn't there somthing about largePageTables which must be enabled
in JAVA and also supported by OS for such huge heaps?
Just a guess.
Am 19.03.19 um 15:01 schrieb Jörn Franke:
It could be an issue with jdk 8 that may not be suitable for such large heaps.
Have more nodes with smaller heaps (eg 3
It could be an issue with jdk 8 that may not be suitable for such large heaps.
Have more nodes with smaller heaps (eg 31 gb)
> Am 18.03.2019 um 11:47 schrieb Aaron Yingcai Sun :
>
> Hello, Solr!
>
>
> We are having some performance issue when try to send documents for solr to
> index. The rep
___
> From: Emir Arnautović
> Sent: Tuesday, March 19, 2019 1:00:19 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr index slow response
>
> If you start indexing with just a single thread/client, do you still see
> slow bulks?
>
> Emir
&g
Design Considerations. indexed fields. The number of indexed fields
greatly increases the following: Memory usage during indexing ; Segment merge
time
From: Emir Arnautović
Sent: Tuesday, March 19, 2019 1:00:19 PM
To: solr-user@lucene.apache.org
Subject: Re: Sol
; From: Emir Arnautović
> Sent: Tuesday, March 19, 2019 12:30:33 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr index slow response
>
> Just to add different perspective here: how do you send documents to Solr?
> Are those log lines from your client? Maybe it is not Sol
2019 12:30:33 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr index slow response
Just to add different perspective here: how do you send documents to Solr? Are
those log lines from your client? Maybe it is not Solr that is slow. Could it
be network or client itself. If you have some dry run on
re
> any other faster way to index such big amount of data?
>
>
> BRs
>
> //Aaron
>
>
> From: Walter Underwood
> Sent: Monday, March 18, 2019 4:59:20 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr index slow response
>
> Solr is not designed t
9 ms.
> 190318-162821.028-189979 DBG1:doc_count: 10 , doc_size: 584 KB, Res code:
> 200, QTime: 22800 ms, Request time: 22802 ms.
> 190318-162821.056-189948 DBG1:doc_count: 10 , doc_size: 670 KB, Res code:
> 200, QTime: 34193 ms, Request time: 34195 ms.
> 190318-162821.062-189983 D
18-162821.028-189979 DBG1:doc_count: 10 , doc_size: 584 KB, Res code:
> 200, QTime: 22800 ms, Request time: 22802 ms.
> 190318-162821.056-189948 DBG1:doc_count: 10 , doc_size: 670 KB, Res code:
> 200, QTime: 34193 ms, Request time: 34195 ms.
> 190318-162821.062-189983 DBG1:doc_count: 10 , do
s Ulicny
Sent: Monday, March 18, 2019 2:54:25 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr index slow response
One other thing to look at besides the heap is your commit settings. We've
experienced something similar, and changing commit settings alleviated the
issue.
Are you opening a sea
__
> From: Emir Arnautović
> Sent: Monday, March 18, 2019 2:19:19 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr index slow response
>
> Hi Aaron,
> Without looking too much into numbers, my bet would be that it is large
> heap that is causing issues.
___
> From: Emir Arnautović
> Sent: Monday, March 18, 2019 2:19:19 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr index slow response
>
> Hi Aaron,
> Without looking too much into numbers, my bet would be that it is large heap
> that is causi
142655.210-160208 DBG1:doc_count: 10 , doc_size: 605 KB, Res code:
> 200, QTime: 108 ms, Request time: 110 ms.
> 190318-142655.304-160208 DBG1:doc_count: 10 , doc_size: 481 KB, Res code:
> 200, QTime: 89 ms, Request time: 90 ms.
> 190318-142655.410-160208 DBG1:doc_count: 10 , doc_size: 4
time.
BRs
//Aaron
From: Emir Arnautović
Sent: Monday, March 18, 2019 2:19:19 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr index slow response
Hi Aaron,
Without looking too much into numbers, my bet would be that it is large heap
that is causing issues
olr-user@lucene.apache.org
Subject: Re: Solr index slow response
On Mon, 2019-03-18 at 10:47 +, Aaron Yingcai Sun wrote:
> Solr server is running on a quit powerful server, 32 cpus, 400GB RAM,
> while 300 GB is reserved for solr, [...]
300GB for Solr sounds excessive.
> Our applica
.0_144/jre/classes",
>> "classpath":"...",
>> "commandLineArgs":["-Xms100G",
>> "-Xmx300G",
>> "-DSTOP.PORT=8079",
>> "-DSTOP.KEY=..",
>> "-Dsolr.solr.home=.."
lr.home=..",
>"-Djetty.port=8983"],
> "startTime":"2019-03-18T09:35:27.892Z",
> "upTimeMS":9258422}},
> "system":{
>"name":"Linux",
>"arch":"amd64",
>"av
On Mon, 2019-03-18 at 10:47 +, Aaron Yingcai Sun wrote:
> Solr server is running on a quit powerful server, 32 cpus, 400GB RAM,
> while 300 GB is reserved for solr, [...]
300GB for Solr sounds excessive.
> Our application send 100 documents to solr per request, json encoded.
> the size is aro
t;:32,
"systemLoadAverage":14.72,
"version":"3.0.101-311.g08a8a9d-default",
"committedVirtualMemorySize":2547960700928,
"freePhysicalMemorySize":4530696192,
"freeSwapSpaceSize":3486846976,
"processCpuLoad":0.
Hi Aaron,
Which version of Solr? How did you configure your heap? Is it standalone Solr
or SolrCloud? A single server? Do you use some monitoring tool? Do you see some
spikes, pauses or CPU usage is constant?
Thanks,
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elast
Hello, Solr!
We are having some performance issue when try to send documents for solr to
index. The repose time is very slow and unpredictable some time.
Solr server is running on a quit powerful server, 32 cpus, 400GB RAM, while 300
GB is reserved for solr, while this happening, cpu usage is
26 matches
Mail list logo