Hi Shawn,
Thank you very much for your analysis. I currently don’t have multiple machines
to play with. I will try "one Solr instance and one ZK instance would be more
efficient on a single server” you suggested.
Thanks again,
Chuming
On Nov 4, 2018, at 7:56 PM, Shawn Heisey wrote:
> On 1
On 11/4/2018 8:38 AM, Chuming Chen wrote:
I have shared a tar ball with you (apa...@elyograg.org) from google drive. The
tar ball includes logs directories of 4 nodes, solrconfig.xml, solr.in.sh, and
screenshot of TOP command. The log files is about 1 day’s log. However, I
restarted the solr c
Hi Shawn,
I have shared a tar ball with you (apa...@elyograg.org) from google drive. The
tar ball includes logs directories of 4 nodes, solrconfig.xml, solr.in.sh, and
screenshot of TOP command. The log files is about 1 day’s log. However, I
restarted the solr cloud several times during that pe
Please see inline for my thoughts
Deepak
"The greatness of a nation can be judged by the way its animals are
treated. Please consider stopping the cruelty by becoming a Vegan"
+91 73500 12833
deic...@gmail.com
Facebook: https://www.facebook.com/deicool
LinkedIn: www.linkedin.com/in/deicool
"Pl
On 11/2/2018 1:38 PM, Chuming Chen wrote:
I am running a Solr cloud 7.4 with 4 shards and 4 nodes (JVM "-Xms20g
-Xmx40g”), each shard has 32 million documents and 32Gbytes in size.
A 40GB heap is probably completely unnecessary for an index of that
size. Does each machine have one replica on
Hi All,
I am running a Solr cloud 7.4 with 4 shards and 4 nodes (JVM "-Xms20g
-Xmx40g”), each shard has 32 million documents and 32Gbytes in size.
For a given query (I use complexphrase query), typically, the first time it
took a couple of seconds to return the first 20 docs. However, for the
Hi Shawn,
Thanks for your input and help.
what you just guessed is right, we run solr in jetty by using start.jar, the
parms is what I sent to you in my last mail.
about GC, I will check it carefully, thanks.
--
发自我的网易邮箱手机智能版
在 2016-05-11 21:32:33，"Shawn Heisey" 写道：
>On 5/10/2016 7:46 PM
On 5/10/2016 7:46 PM, lltvw wrote:
> the args used to start solr are as following, and upload my screen shot to
> http://www.yupoo.com/photos/qzone3927066199/96064170/, please help to take a
> look, thanks.
>
> -DSTOP.PORT=7989
> -DSTOP.KEY=
> -DzkHost=node1:2181,node2:2181,node3:2181/solr
> -Dso
Hi Shawn,
Thanks for your help.
the args used to start solr are as following, and upload my screen shot to
http://www.yupoo.com/photos/qzone3927066199/96064170/, please help to take a
look, thanks.
-DSTOP.PORT=7989
-DSTOP.KEY=
-DzkHost=node1:2181,node2:2181,node3:2181/solr
-Dsolr.solr.hom
On 5/9/2016 11:42 PM, lltvw wrote:
> By using jps command double check the parms used to start solr, i found that
> the max heap size already set to 10G. So I made a big mistake yesterday.
>
> But by using solr admin UI, I select the collection with performance problem,
> in the overview page I
Hi Toke,
The version i am using is 4.10, i do not know why by setting log to all, and
then recycle solr, i still could not get detailed log info. What is wrong.
Does debug info from solr admin UI make sense?
--
发自我的网易邮箱手机智能版
在 2016-05-10 16:25:34，"Toke Eskildsen" 写道：
>On Tue, 2016-05-10 at
On Tue, 2016-05-10 at 15:33 +0800, lltvw wrote:
> What log you mentioned, console log or something else. the version I am using
> is 4.10.
There should be a solr.log somewhere. If you have not changed the
default log levels, it should log all queries.
- Toke Eskildsen, State and University Libr
Hi Toke,
What log you mentioned, console log or something else. the version I am using
is 4.10.
--
发自我的网易邮箱手机智能版
在 2016-05-10 14:42:34，"Toke Eskildsen" 写道：
>On Tue, 2016-05-10 at 00:41 +0800, lltvw wrote:
>> Recently we setup a 4.10 solrcloud env with about 9000 doc indexed
>> in it，t
On Tue, 2016-05-10 at 00:41 +0800, lltvw wrote:
> Recently we setup a 4.10 solrcloud env with about 9000 doc indexed
> in it，this solrcloud with 12 shards, each shard on one separate
> machine, but when we try to search some infor on solrcloud, the
> response time is about 300ms.
Could you pr
Shawn,
By using jps command double check the parms used to start solr, i found that
the max heap size already set to 10G. So I made a big mistake yesterday.
But by using solr admin UI, I select the collection with performance problem,
in the overview page I find that the heap memory is about
On 5/9/2016 9:11 PM, lltvw wrote:
> You are right, the max heap is 512MB, thanks.
90 million documents split into 12 shards means 7.5 million documents
per shard.
With that many documents and a 512MB heap, you're VERY lucky if Solr
doesn't experience OutOfMemoryError problems -- which will make S
Hi shawn,
You are right, the max heap is 512MB, thanks.
--
发自我的网易邮箱手机智能版
在 2016-05-10 10:02:44，"Shawn Heisey" 写道：
>On 5/9/2016 4:41 PM, lltvw wrote:
>> Shawn, thanks.
>>
>> Each machine with 48G memory installed, and now with 20G free, I check JVM
>> heap size use solr admin UI, the heap
On 5/9/2016 4:41 PM, lltvw wrote:
> Shawn, thanks.
>
> Each machine with 48G memory installed, and now with 20G free, I check JVM
> heap size use solr admin UI, the heap size is about 20M.
What is the *max* heap? An unmodified install of Solr 5.x or later has
a max heap of 512MB.
In the admin
Shawn, thanks.
Each machine with 48G memory installed, and now with 20G free, I check JVM
heap size use solr admin UI, the heap size is about 20M.
--
发自我的网易邮箱手机智能版
在 2016-05-10 02:04:22，"Shawn Heisey" 写道：
>On 5/9/2016 10:52 AM, lltvw wrote:
>> Sorry, I missed the size of each shard, the s
On 5/9/2016 10:52 AM, lltvw wrote:
> Sorry, I missed the size of each shard, the size is about 3G each. Thanks.
>
> 在 2016-05-10 00:41:13，lltvw 写道：
>> Recently we setup a 4.10 solrcloud env with about 9000 doc indexed in
>> it，this solrcloud with 12 shards, each shard on one separate machine
Hi all,
Recently we setup a 4.10 solrcloud env with about 9000 doc indexed in
it，this solrcloud with 12 shards, each shard on one separate machine, but when
we try to search some infor on solrcloud, the response time is about 300ms.
Seems that the performance is not good, please advice ho
why you use 15 replicas?
more replicas more slower.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solrcloud-performance-issues-tp4186035p4188738.html
Sent from the Solr - User mailing list archive at Nabble.com.
Hi,
Did you say you have 150 servers in this cluster? And 10 shards for just
90M docs? If so, that 150 hosts sounds like too much for all other numbers
I see here. I'd love to see some metrics here. e.g. what happens with
disk IO around those commits? How about GC time/size info? Are JVM mem
Hi Vijay,
We're working on SOLR-6816 ... would love for you to be a test site for any
improvements we make ;-)
Curious if you've experimented with changing the mergeFactor to a higher
value, such as 25 and what happens if you set soft-auto-commits to
something lower like 15 seconds? Also, make s
Hi Erick,
We have following configuration of our solr cloud
1. 10 Shards
2. 15 replicas per shard
3. 9 GB of index size per shard
4. a total of around 90 mil documents
5. 2 collection viz search1 serving live traffic and search 2 for
indexing. We swap collection when indexing fin
search engn dev [sachinyadav0...@gmail.com] wrote:
> Yes, You are right my facet queries are for text analytic purpose.
Does this mean that facet calls are rare (at most one at a time)?
> Users will send boolean and spatial queries. current performance for spatial
> queries is 100qps with 150 con
;
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/SolrCloud-performance-issues-regarding-hardware-configuration-tp4147843p4148222.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
--
Himanshu Mehrotra
Download Our App[imag
l number shards
--
View this message in context:
http://lucene.472066.n3.nabble.com/SolrCloud-performance-issues-regarding-hardware-configuration-tp4147843p4148222.html
Sent from the Solr - User mailing list archive at Nabble.com.
search engn dev [sachinyadav0...@gmail.com] wrote:
> out of 700 million documents 95-97% values are unique approx.
That's quite a lot. If you are not already using DocValues for that, you should
do so.
So, each shard handles ~175M documents. Even with DocValues, there is an
overhead of just hav
gt; Above query throws OOM exception as soon as fire it to solr.
>
>
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/SolrCloud-performance-issues-regarding-hardware-configuration-tp4147843p4147871.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
:
http://lucene.472066.n3.nabble.com/SolrCloud-performance-issues-regarding-hardware-configuration-tp4147843p4147871.html
Sent from the Solr - User mailing list archive at Nabble.com.
From: search engn dev [sachinyadav0...@gmail.com]:
> 1 collection : 4 shards : each shard has one master and one replica
> total documents : 700 million
Are you using DocValues for your facet fields? What is the approximate number
of unique values in your facets and what is their type (string, nu
with 32 gb ram each?
--
View this message in context:
http://lucene.472066.n3.nabble.com/SolrCloud-performance-issues-regarding-hardware-configuration-tp4147843.html
Sent from the Solr - User mailing list archive at Nabble.com.
Be a bit careful here. 128G is lots of memory, you may encounter very long
garbage collection pauses. Just be aware that this may be happening later.
Best,
Erick
On Tue, Oct 22, 2013 at 5:04 PM, Tom Mortimer wrote:
> Just tried it with no other changes than upping the RAM to 128GB total, and
>
Just tried it with no other changes than upping the RAM to 128GB total, and
it's flying. I think that proves that RAM is good. =) Will implement
suggested changes later, though.
cheers,
Tom
On 22 October 2013 09:04, Tom Mortimer wrote:
> Boogie, Shawn,
>
> Thanks for the replies. I'm going to
Boogie, Shawn,
Thanks for the replies. I'm going to try out some of your suggestions
today. Although, without more RAM I'm not that optimistic..
Tom
On 21 October 2013 18:40, Shawn Heisey wrote:
> On 10/21/2013 9:48 AM, Tom Mortimer wrote:
>
>> Hi everyone,
>>
>> I've been working on an inst
On 10/21/2013 9:48 AM, Tom Mortimer wrote:
Hi everyone,
I've been working on an installation recently which uses SolrCloud to index
45M documents into 8 shards on 2 VMs running 64-bit Ubuntu (with another 2
identical VMs set up for replicas). The reason we're using so many shards
for a relativel
some sort of
MPIO in place (esp if you are using 1GB iscsi)
From: Tom Mortimer
Sent: Monday, October 21, 2013 08:48
To: solr-user@lucene.apache.org
Subject: SolrCloud performance in VM environment
Hi everyone,
I've been working on an install
Hi everyone,
I've been working on an installation recently which uses SolrCloud to index
45M documents into 8 shards on 2 VMs running 64-bit Ubuntu (with another 2
identical VMs set up for replicas). The reason we're using so many shards
for a relatively small index is that there are complex filte
difference in the
> > performance. I do see minor entry in the queryfiltercache rate which is a
> > meager 0.02.
> >
> > I'm really struggling to figure out the bottleneck, any known pain
> points I
> > should be checking ?
> >
> >
> >
> > --
> > View this message in context:
> http://lucene.472066.n3.nabble.com/SolrCloud-Performance-Issue-tp4095971p4096277.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
>
ge in context:
> http://lucene.472066.n3.nabble.com/SolrCloud-Performance-Issue-tp4095971p4096277.html
> Sent from the Solr - User mailing list archive at Nabble.com.
in context:
http://lucene.472066.n3.nabble.com/SolrCloud-Performance-Issue-tp4095971p4096277.html
Sent from the Solr - User mailing list archive at Nabble.com.
://lucene.472066.n3.nabble.com/SolrCloud-Performance-Issue-tp4095971p4096123.html
Sent from the Solr - User mailing list archive at Nabble.com.
Query result cache hit might be low due to using NOW in bf. NOW is always
translated to current time and that of course changes from ms to ms... :)
Primoz
From: Shamik Bandopadhyay
To: solr-user@lucene.apache.org
Date: 17.10.2013 00:14
Subject:SolrCloud Performance Issue
Hi,
I'm in the process of transitioning to SolrCloud from a conventional
Master-Slave model. I'm using Solr 4.4 and has set-up 2 shards with 1
replica each. I've 3 zookeeper ensemble. All the nodes are running on AWS
EC2 instances. Shards are on m1.xlarge and sharing a zookeeper instance
(mounte
ork.
Any pointers will be highly appreciated.
Regards,
Shamik
--
View this message in context:
http://lucene.472066.n3.nabble.com/SolrCloud-Performance-Issue-tp4095940.html
Sent from the Solr - User mailing list archive at Nabble.com.
rsday, January 17, 2013 10:12 AM
>Subject: Re: SolrCloud Performance for High Query Volume
>
>
>Hello Niran,
>
>
>> Now with the roughly the "same" schema and solrconfig configuration
>
>
>Can you be more specific about what was changed and how?
>
&
Hi all,
I'm currently in the process of doing some performance testing in preparations
for upgrading from Solr 3.6.1 to Solr 4.0. (We're badly in need of NRT
functionality)
Our existing deployment is not a typical deployment for Solr, as we use it to
search and facet on financial data such as
e will be better for this case?
>>>
>>> I am aware cloud makes loadbalancing and some other stuff largely
>> concerned
>>> about searching, rather than indexing, but for a frequently updated
>> system,
>>> does it still useful to set up a cloud environment?
>>>
>>&
set up a cloud environment?
> >
> > and are there some workarounds for indexing speed, other than the known
> ones
> > for solr, on cloud?
> >
> >
> >
> > -
> > Zeki ama calismiyor... Calissa yapar...
> > --
> > View this message in context:
> http://lucene.472066.n3.nabble.com/SolrCloud-Performance-Indexing-tp4022549.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
>
>
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
<http://www.griddynamics.com>
speed, other than the known ones
> for solr, on cloud?
>
>
>
> -
> Zeki ama calismiyor... Calissa yapar...
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/SolrCloud-Performance-Indexing-tp4022549.html
> Sent from the Solr - User mailing list archive at Nabble.com.
frequently updated system,
does it still useful to set up a cloud environment?
and are there some workarounds for indexing speed, other than the known ones
for solr, on cloud?
-
Zeki ama calismiyor... Calissa yapar...
--
View this message in context:
http://lucene.472066.n3.nabble.com/SolrCloud
52 matches
Mail list logo
