RE: documentCache not used in 4.3.1?
Yes, we are softCommit'ing every 1000ms, but that should be enough time to see metrics though, right? For example, I still get non-cumulative metrics from the other caches (which are also throw away). I've also curl/sampled enough that I probably should have seen a value by now. If anyone else can reproduce this on 4.3.1 I will feel less crazy :). Cheers, Tim -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Saturday, June 29, 2013 10:13 AM To: solr-user@lucene.apache.org Subject: Re: documentCache not used in 4.3.1? It's especially weird that the hit ratio is so high and you're not seeing anything in the cache. Are you perhaps soft committing frequently? Soft commits throw away all the top-level caches including documentCache I think Erick On Fri, Jun 28, 2013 at 7:23 PM, Tim Vaillancourt t...@elementspace.comwrote: Thanks Otis, Yeah I realized after sending my e-mail that doc cache does not warm, however I'm still lost on why there are no other metrics. Thanks! Tim On 28 June 2013 16:22, Otis Gospodnetic otis.gospodne...@gmail.com wrote: Hi Tim, Not sure about the zeros in 4.3.1, but in SPM we see all these numbers are non-0, though I haven't had the chance to confirm with Solr 4.3.1. Note that you can't really autowarm document cache... Otis -- Solr ElasticSearch Support -- http://sematext.com/ Performance Monitoring -- http://sematext.com/spm On Fri, Jun 28, 2013 at 7:14 PM, Tim Vaillancourt t...@elementspace.com wrote: Hey guys, This has to be a stupid question/I must be doing something wrong, but after frequent load testing with documentCache enabled under Solr 4.3.1 with autoWarmCount=150, I'm noticing that my documentCache metrics are always zero for non-cumlative. At first I thought my commit rate is fast enough I just never see the non-cumlative result, but after 100s of samples I still always get zero values. Here is the current output of my documentCache from Solr's admin for 1 core: - documentCache http://localhost:8983/solr/#/channels_shard1_replica2/plugins/cache?en try=documentCache - class:org.apache.solr.search.LRUCache - version:1.0 - description:LRU Cache(maxSize=512, initialSize=512, autowarmCount=150, regenerator=null) - src:$URL: https:/ /svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/ solr/core/src/java/org/apache/solr/search/LRUCache.java https://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/s olr/core/src/java/org/apache/solr/search/LRUCache.java $ - stats: - lookups:0 - hits:0 - hitratio:0.00 - inserts:0 - evictions:0 - size:0 - warmupTime:0 - cumulative_lookups:65198986 - cumulative_hits:63075669 - cumulative_hitratio:0.96 - cumulative_inserts:2123317 - cumulative_evictions:1010262 The cumulative values seem to rise, suggesting doc cache is working, but at the same time it seems I never see non-cumlative metrics, most importantly warmupTime. Am I doing something wrong, is this normal/by-design, or is there an issue here? Thanks for helping with my silly question! Have a good weekend, Tim
RE: Solr developer IRC channel
I agree with Yonik. It is great to see an IRC for Solr! Tim -Original Message- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: Monday, June 10, 2013 2:46 PM To: solr-user@lucene.apache.org Subject: Re: Solr developer IRC channel On Mon, Jun 10, 2013 at 5:32 PM, Otis Gospodnetic otis.gospodne...@gmail.com wrote: Mucho good! +1 Why unlogged though? Just curious. Personal preference give it a more informal / slightly more private feel. Some people don't want casual watercooler chat recorded publicized forever. -Yonik http://lucidworks.com
RE: SolrCloud Load Balancer weight
Cool! Having those values influenced by stats is a neat idea too. I'll get on that soon. Tim -Original Message- From: Mark Miller [mailto:markrmil...@gmail.com] Sent: Monday, June 03, 2013 5:07 PM To: solr-user@lucene.apache.org Subject: Re: SolrCloud Load Balancer weight On Jun 3, 2013, at 3:33 PM, Tim Vaillancourt t...@elementspace.com wrote: Should I JIRA this? Thoughts? Yeah - it's always been in the back of my mind - it's come up a few times - eventually we would like nodes to report some stats to zk to influence load balancing. - mark
RE: CSS appearing in Solr 4.2.1 logs
Thanks Chris! Somehow I managed to miss that ticket searching, thanks for looking for me. I will confirm the version I have and I am glad to hear this was reported and resolved! Cheers, Tim -Original Message- From: Chris Hostetter [mailto:hossman_luc...@fucit.org] Sent: Friday, April 12, 2013 2:53 PM To: solr-user@lucene.apache.org Subject: Re: CSS appearing in Solr 4.2.1 logs : This sounds crazy, but does anyone see strange CSS/HTML in their Solr 4.2.x : logs? are you sure you're running 4.2.1 and not 4.2? https://issues.apache.org/jira/browse/SOLR-4573 -Hoss
RE: [ANNOUNCE] Solr wiki editing change
I would also like to contribute to SolrCloud's wiki where possible. Please add myself (TimVaillancourt) when you have a chance. Cheers, Tim -Original Message- From: Trey Grainger [mailto:solrt...@gmail.com] Sent: Saturday, March 30, 2013 9:43 PM To: d...@lucene.apache.org Cc: solr-user@lucene.apache.org Subject: Re: [ANNOUNCE] Solr wiki editing change Please add TreyGrainger to the the contributors group. Thanks! -Trey On Sun, Mar 24, 2013 at 11:18 PM, Steve Rowe sar...@gmail.com wrote: The wiki at http://wiki.apache.org/solr/ has come under attack by spammers more frequently of late, so the PMC has decided to lock it down in an attempt to reduce the work involved in tracking and removing spam. From now on, only people who appear on http://wiki.apache.org/solr/ContributorsGroup will be able to create/modify/delete wiki pages. Please request either on the solr-user@lucene.apache.org or on d...@lucene.apache.org to have your wiki username added to the ContributorsGroup page - this is a one-time step. Steve - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: [WEBINAR] - Lucene/Solr 4 - A Revolution in Enterprise Search Technology
Too bad I missed it, thanks for putting this on! Are there any links to the recorded version? Cheers, Tim -Original Message- From: Erik Hatcher [mailto:erik.hatc...@gmail.com] Sent: Tuesday, March 26, 2013 6:24 PM To: solr-user@lucene.apache.org; java-u...@lucene.apache.org Subject: [WEBINAR] - Lucene/Solr 4 - A Revolution in Enterprise Search Technology Excuse the blatant marketing, though for the benefit of the community... http://programs.lucidworks.com/Solr4032013_signuppage.html Join me tomorrow/today (March 27) for a webinar on what's new and improved in Lucene and Solr 4. It's the last call to register. Help me break the webinar system by overloading it for the presentation! (and it'll be recorded and shared afterwards with those that registered) Erik -- [official description] Lucene/Solr 4 is a ground breaking shift from previous releases. Solr 4.0 dramatically improves scalability, performance, reliability, and flexibility. Lucene 4 has been extensively upgraded. It now supports near real-time (NRT) capabilities that allow indexed documents to be rapidly visible and searchable. Additional Lucene improvements include pluggable scoring, much faster fuzzy and wildcard querying, and vastly improved memory usage. The improvements in Lucene have automatically made Solr 4 substantially better. But Solr has also been considerably improved and magnifies these advances with a suite of new SolrCloud features that radically improve scalability and reliability. In this Webinar, you will learn: * What are the Key Feature Enhancements of Lucene/Solr 4, including the new distributed capabilities of SolrCloud * How to Use the Improved Administrative User Interface * How Sharding has been improved * What are the improvements to GeoSpatial Searches, Highlighting, Advanced Query Parsers, Distributed search support, Dynamic core management, Performance statistics, and searches for rare values, such as Primary Key Presenter: Erik Hatcher, Lucene/Solr Committer and PMC member Erik Hatcher is the co-author of Lucene in Action as well as co-author of Java Development with Ant. Erik has been an active member of the Lucene community, a leading Lucene/Solr committer, member of the Lucene/Solr Project Management Committee, member of the Apache Software Foundation as well as a frequent invited speaker at various industry events. Erik earned his B.S. in Computer Science from University of Virginia, Charlottesville, VA.
RE: clusterstate.json size
Wow. I'm guessing we may have a new Largest SolrCloud winner ;). Tim -Original Message- From: svamb...@gmail.com [mailto:svamb...@gmail.com] Sent: Saturday, March 30, 2013 11:56 AM To: solr-user@lucene.apache.org Cc: solr-user@lucene.apache.org Subject: Re: clusterstate.json size You can zip the file and send it to ZK Sent from my iPhone On 30-Mar-2013, at 1:47 PM, Mark Miller markrmil...@gmail.com wrote: 4.2.1 *should* return a decent response. How many nodes!? I didn't even see that size at 1000 shards! It's a zk sys prop to raise it - on the road now, so I'd try google. - mark Sent from my iPhone On Mar 30, 2013, at 1:49 PM, yriveiro yago.rive...@gmail.com wrote: Hi, Is there a size limitation for the clusterstate file? I can't create more collections for my cluster I have no error but the CREATE command not return any response. I read in the past that the max size for a file in zookeeper was 1MB, my clusterstate file has 1.1MB. It's possible be this the problem? If it's, how I can increase this limit? - Best regards -- View this message in context: http://lucene.472066.n3.nabble.com/clusterstate-json-size-tp4052598.h tml Sent from the Solr - User mailing list archive at Nabble.com.
RE: Basic auth on SolrCloud /admin/* calls
Yes, I should have mentioned this is under 4.2 Solr.
I sort of expected what I'm doing might be unsupported, but basically my
concern is under the current SOLR design, any client with connectivity to
SOLR's port can perform Admin-level API calls like create/drop Cores or
Collections.
I'm only aiming for '/solr/admin/*' calls to separate Application access from
the Administrative access logically, and not the non-admin calls like
'/update', although you can cause damage with '/update', too.
I may try to patch the code to send Basic auth credentials on internal calls
just for fun, but I'm thinking longer-term authentication should be
implemented/added to the SOLR codebase (for at least admin calls) vs playing
with security at the container level, and having the app inside the container
aware of it.
On the upside, in short testing I was able to get a Collection online using
Cores API only using curl calls w/basic auth. Only the Collections API is
affected due to it calls itself which do not have auth.
Cheers,
Tim
-Original Message-
From: Isaac Hebsh [mailto:isaac.he...@gmail.com]
Sent: Friday, March 29, 2013 12:37 AM
To: solr-user@lucene.apache.org
Subject: Re: Basic auth on SolrCloud /admin/* calls
Hi Tim,
Are you running Solr 4.2? (In 4.0 and 4.1, the Collections API didn't return
any failure message. see SOLR-4043 issue).
As far as I know, you can't tell Solr to use authentication credentials when
communicating other nodes. It's a bigger issue.. for example, if you want to
protect the /update requestHandler, so unauthorized users won't delete your
whole collection, it can interfere the replication process.
I think it's a necessary mechanism in production environment... I'm curious how
do people use SolrCloud in production w/o it.
On Fri, Mar 29, 2013 at 3:42 AM, Vaillancourt, Tim tvaillanco...@ea.comwrote:
Hey guys,
I've recently setup basic auth under Jetty 8 for all my Solr 4.x
'/admin/*' calls, in order to protect my Collections and Cores API.
Although the security constraint is working as expected ('/admin/*'
calls require Basic Auth or return 401), when I use the Collections
API to create a collection, I receive a 200 OK to the Collections API
CREATE call, but the background Cores API calls that are ran on the
Collection API's behalf fail on the Basic Auth on other nodes with a
401 code, as I should have foreseen, but didn't.
Is there a way to tell SolrCloud to use authentication on internal
Cores API calls that are spawned on Collections API's behalf, or is
this a new feature request?
To reproduce:
1. Implement basic auth on '/admin/*' URIs.
2. Perform a CREATE Collections API call to a node (which will
return 200 OK).
3. Notice all Cores API calls fail (Collection isn't created). See
stack trace below from the node that was issued the CREATE call.
The stack trace I get is:
org.apache.solr.common.SolrException: Server at http://HOST
HERE:8983/solrhttp://%3cHOST%20HERE%3e:8983/solr returned non ok
status:401, message:Unauthorized
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServe
r.java:373)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServe
r.java:181)
at
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHan
dler.java:169)
at
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHan
dler.java:135) at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439
) at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecu
tor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
java:918) at java.lang.Thread.run(Thread.java:662)
Cheers!
Tim
RE: Basic auth on SolrCloud /admin/* calls
Agreed, we don't have clients hitting Solr directly, it is used like a backend
database in our usage by intermediaries, similar to say MySQL. Although
restricting the access to Solr to fewer hosts is something, I still feel an
application has no business being able to perform admin level calls, at least
in my use case. This is being very nitpicky though.
We also open Solr's port to monitoring servers who shouldn't have access to
admin calls and thinking paranoid a compromised app using a single collection
could affect the entire cloud with admin call access.
Seeing the long term plan is to leave this feature at the container level
(which is totally valid), I think I'll continue with the basic auth approach I
attempted and see what I can dig up on past efforts. I'll be sure to share what
I've done.
Thanks Mark!
Tim
-Original Message-
From: Mark Miller [mailto:markrmil...@gmail.com]
Sent: Friday, March 29, 2013 1:04 PM
To: solr-user@lucene.apache.org
Subject: Re: Basic auth on SolrCloud /admin/* calls
This has always been the case with Solr. Solr's security model is that clients
should not have access to it - only trusted intermediaries should have access
to it. Otherwise, it should be locked down at a higher level. That's been the
case from day one and still is.
That said, someone did do some work on internode basic auth a while back, but
it didn't raise a ton of interest yet.
- Mark
On Mar 29, 2013, at 2:09 PM, Vaillancourt, Tim tvaillanco...@ea.com wrote:
Yes, I should have mentioned this is under 4.2 Solr.
I sort of expected what I'm doing might be unsupported, but basically my
concern is under the current SOLR design, any client with connectivity to
SOLR's port can perform Admin-level API calls like create/drop Cores or
Collections.
I'm only aiming for '/solr/admin/*' calls to separate Application access
from the Administrative access logically, and not the non-admin calls like
'/update', although you can cause damage with '/update', too.
I may try to patch the code to send Basic auth credentials on internal calls
just for fun, but I'm thinking longer-term authentication should be
implemented/added to the SOLR codebase (for at least admin calls) vs playing
with security at the container level, and having the app inside the container
aware of it.
On the upside, in short testing I was able to get a Collection online using
Cores API only using curl calls w/basic auth. Only the Collections API is
affected due to it calls itself which do not have auth.
Cheers,
Tim
-Original Message-
From: Isaac Hebsh [mailto:isaac.he...@gmail.com]
Sent: Friday, March 29, 2013 12:37 AM
To: solr-user@lucene.apache.org
Subject: Re: Basic auth on SolrCloud /admin/* calls
Hi Tim,
Are you running Solr 4.2? (In 4.0 and 4.1, the Collections API didn't return
any failure message. see SOLR-4043 issue).
As far as I know, you can't tell Solr to use authentication credentials when
communicating other nodes. It's a bigger issue.. for example, if you want to
protect the /update requestHandler, so unauthorized users won't delete your
whole collection, it can interfere the replication process.
I think it's a necessary mechanism in production environment... I'm curious
how do people use SolrCloud in production w/o it.
On Fri, Mar 29, 2013 at 3:42 AM, Vaillancourt, Tim
tvaillanco...@ea.comwrote:
Hey guys,
I've recently setup basic auth under Jetty 8 for all my Solr 4.x
'/admin/*' calls, in order to protect my Collections and Cores API.
Although the security constraint is working as expected ('/admin/*'
calls require Basic Auth or return 401), when I use the Collections
API to create a collection, I receive a 200 OK to the Collections API
CREATE call, but the background Cores API calls that are ran on the
Collection API's behalf fail on the Basic Auth on other nodes with a
401 code, as I should have foreseen, but didn't.
Is there a way to tell SolrCloud to use authentication on internal
Cores API calls that are spawned on Collections API's behalf, or is
this a new feature request?
To reproduce:
1. Implement basic auth on '/admin/*' URIs.
2. Perform a CREATE Collections API call to a node (which will
return 200 OK).
3. Notice all Cores API calls fail (Collection isn't created). See
stack trace below from the node that was issued the CREATE call.
The stack trace I get is:
org.apache.solr.common.SolrException: Server at http://HOST
HERE:8983/solrhttp://%3cHOST%20HERE%3e:8983/solr returned non ok
status:401, message:Unauthorized
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServ
e
r.java:373)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServ
e
r.java:181)
at
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHa
n
dler.java:169)
at
org.apache.solr.handler.component.HttpShardHandler$1.call
RE: Basic auth on SolrCloud /admin/* calls
Here we go:
https://issues.apache.org/jira/browse/SOLR-4470
Tim
-Original Message-
From: Vaillancourt, Tim [mailto:tvaillanco...@ea.com]
Sent: Friday, March 29, 2013 3:25 PM
To: solr-user@lucene.apache.org
Subject: RE: Basic auth on SolrCloud /admin/* calls
Agreed, we don't have clients hitting Solr directly, it is used like a backend
database in our usage by intermediaries, similar to say MySQL. Although
restricting the access to Solr to fewer hosts is something, I still feel an
application has no business being able to perform admin level calls, at least
in my use case. This is being very nitpicky though.
We also open Solr's port to monitoring servers who shouldn't have access to
admin calls and thinking paranoid a compromised app using a single collection
could affect the entire cloud with admin call access.
Seeing the long term plan is to leave this feature at the container level
(which is totally valid), I think I'll continue with the basic auth approach I
attempted and see what I can dig up on past efforts. I'll be sure to share what
I've done.
Thanks Mark!
Tim
-Original Message-
From: Mark Miller [mailto:markrmil...@gmail.com]
Sent: Friday, March 29, 2013 1:04 PM
To: solr-user@lucene.apache.org
Subject: Re: Basic auth on SolrCloud /admin/* calls
This has always been the case with Solr. Solr's security model is that clients
should not have access to it - only trusted intermediaries should have access
to it. Otherwise, it should be locked down at a higher level. That's been the
case from day one and still is.
That said, someone did do some work on internode basic auth a while back, but
it didn't raise a ton of interest yet.
- Mark
On Mar 29, 2013, at 2:09 PM, Vaillancourt, Tim tvaillanco...@ea.com wrote:
Yes, I should have mentioned this is under 4.2 Solr.
I sort of expected what I'm doing might be unsupported, but basically my
concern is under the current SOLR design, any client with connectivity to
SOLR's port can perform Admin-level API calls like create/drop Cores or
Collections.
I'm only aiming for '/solr/admin/*' calls to separate Application access
from the Administrative access logically, and not the non-admin calls like
'/update', although you can cause damage with '/update', too.
I may try to patch the code to send Basic auth credentials on internal calls
just for fun, but I'm thinking longer-term authentication should be
implemented/added to the SOLR codebase (for at least admin calls) vs playing
with security at the container level, and having the app inside the container
aware of it.
On the upside, in short testing I was able to get a Collection online using
Cores API only using curl calls w/basic auth. Only the Collections API is
affected due to it calls itself which do not have auth.
Cheers,
Tim
-Original Message-
From: Isaac Hebsh [mailto:isaac.he...@gmail.com]
Sent: Friday, March 29, 2013 12:37 AM
To: solr-user@lucene.apache.org
Subject: Re: Basic auth on SolrCloud /admin/* calls
Hi Tim,
Are you running Solr 4.2? (In 4.0 and 4.1, the Collections API didn't return
any failure message. see SOLR-4043 issue).
As far as I know, you can't tell Solr to use authentication credentials when
communicating other nodes. It's a bigger issue.. for example, if you want to
protect the /update requestHandler, so unauthorized users won't delete your
whole collection, it can interfere the replication process.
I think it's a necessary mechanism in production environment... I'm curious
how do people use SolrCloud in production w/o it.
On Fri, Mar 29, 2013 at 3:42 AM, Vaillancourt, Tim
tvaillanco...@ea.comwrote:
Hey guys,
I've recently setup basic auth under Jetty 8 for all my Solr 4.x
'/admin/*' calls, in order to protect my Collections and Cores API.
Although the security constraint is working as expected ('/admin/*'
calls require Basic Auth or return 401), when I use the Collections
API to create a collection, I receive a 200 OK to the Collections API
CREATE call, but the background Cores API calls that are ran on the
Collection API's behalf fail on the Basic Auth on other nodes with a
401 code, as I should have foreseen, but didn't.
Is there a way to tell SolrCloud to use authentication on internal
Cores API calls that are spawned on Collections API's behalf, or is
this a new feature request?
To reproduce:
1. Implement basic auth on '/admin/*' URIs.
2. Perform a CREATE Collections API call to a node (which will
return 200 OK).
3. Notice all Cores API calls fail (Collection isn't created). See
stack trace below from the node that was issued the CREATE call.
The stack trace I get is:
org.apache.solr.common.SolrException: Server at http://HOST
HERE:8983/solrhttp://%3cHOST%20HERE%3e:8983/solr returned non ok
status:401, message:Unauthorized
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request
Basic auth on SolrCloud /admin/* calls
Hey guys,
I've recently setup basic auth under Jetty 8 for all my Solr 4.x '/admin/*'
calls, in order to protect my Collections and Cores API.
Although the security constraint is working as expected ('/admin/*' calls
require Basic Auth or return 401), when I use the Collections API to create a
collection, I receive a 200 OK to the Collections API CREATE call, but the
background Cores API calls that are ran on the Collection API's behalf fail on
the Basic Auth on other nodes with a 401 code, as I should have foreseen, but
didn't.
Is there a way to tell SolrCloud to use authentication on internal Cores API
calls that are spawned on Collections API's behalf, or is this a new feature
request?
To reproduce:
1. Implement basic auth on '/admin/*' URIs.
2. Perform a CREATE Collections API call to a node (which will return 200
OK).
3. Notice all Cores API calls fail (Collection isn't created). See stack
trace below from the node that was issued the CREATE call.
The stack trace I get is:
org.apache.solr.common.SolrException: Server at http://HOST
HERE:8983/solrhttp://%3cHOST%20HERE%3e:8983/solr returned non ok status:401,
message:Unauthorized
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:373)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
at
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:169)
at
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:135)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
Cheers!
Tim
RE: Poll: Largest SolrCloud out there?
Considering the silence, I'll take the unofficial largest SolrCloud award until beaten :D: 2 VMWare VMs 4GB RAM/VM 4 Virtual CPUs 1000mb index Beat that :)!! Tim -Original Message- From: Otis Gospodnetic [mailto:otis.gospodne...@gmail.com] Sent: Thursday, February 28, 2013 12:00 AM To: solr-user@lucene.apache.org Subject: Re: Poll: Largest SolrCloud out there? I'd love to know, too. What we observed at Sematext was that 4.0 SolrCloud very very buggy and difficult, so I suspect there aren't many big Solr 4.0 based clusters out there. 4.1 is much better (thanks Mark Co.) and I'm looking forward to 4.2 in March. Also, based on the stats we have access to via SPM ( see http://sematext.com/spm/index.html ) I can tell you that ElasticSearch clusters are, on average, quite a bit bigger than Solr clusters in terms of nodes, which I find interesting, but not surprising -- if you look at http://blog.sematext.com/2013/02/25/poll-solr-cloud-or-not/ you'll see less than 40% of Solr users are SolrCloud users, which kind of explains it. Otis -- Solr ElasticSearch Support http://sematext.com/ On Tue, Feb 26, 2013 at 9:41 PM, Vaillancourt, Tim tvaillanco...@ea.comwrote: Hey guys, I wanted to see who's running SolrCloud out there, and at what scales? I'd start the thread off but I am merely at the RD phases. Cheers! Tim
RE: Repartition solr cloud
Fantastic! Thanks Eric. Tim -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Thursday, February 28, 2013 6:16 PM To: solr-user@lucene.apache.org Subject: Re: Repartition solr cloud In the works, high priority: https://issues.apache.org/jira/browse/SOLR-3755 Best Erick On Thu, Feb 28, 2013 at 8:13 PM, Vaillancourt, Tim tvaillanco...@ea.comwrote: Sort of off-topic, is there a way to do the reverse, ie: split indexes? This could be useful for people that would like to move to sharding from one core and could be interesting under SolrCloud. Cheers, Tim -Original Message- From: Mark Miller [mailto:markrmil...@gmail.com] Sent: Friday, February 22, 2013 5:30 PM To: solr-user@lucene.apache.org Subject: Re: Repartition solr cloud You could copy each shard to a single node and then use the merge index feature to merge them into one index and then start up a single Solr node on that. Use the same configs. - Mark On Feb 22, 2013, at 8:11 PM, Erol Akarsu eaka...@gmail.com wrote: I have a solr cloud 7 nodes, each has 2 shards. Now, I would like to build another solr server with only one core (no shards or replica) from whole cloud data. What is fastest and safest way to achieve this, making only one SOLR from solr cloud? I appreciate your answer Erol Akarsu
RE: Repartition solr cloud
Sort of off-topic, is there a way to do the reverse, ie: split indexes? This could be useful for people that would like to move to sharding from one core and could be interesting under SolrCloud. Cheers, Tim -Original Message- From: Mark Miller [mailto:markrmil...@gmail.com] Sent: Friday, February 22, 2013 5:30 PM To: solr-user@lucene.apache.org Subject: Re: Repartition solr cloud You could copy each shard to a single node and then use the merge index feature to merge them into one index and then start up a single Solr node on that. Use the same configs. - Mark On Feb 22, 2013, at 8:11 PM, Erol Akarsu eaka...@gmail.com wrote: I have a solr cloud 7 nodes, each has 2 shards. Now, I would like to build another solr server with only one core (no shards or replica) from whole cloud data. What is fastest and safest way to achieve this, making only one SOLR from solr cloud? I appreciate your answer Erol Akarsu
Poll: Largest SolrCloud out there?
Hey guys, I wanted to see who's running SolrCloud out there, and at what scales? I'd start the thread off but I am merely at the RD phases. Cheers! Tim
RE: Is it possible to manually select a shard leader in a running SolrCloud?
Thanks Mark, Sounds good. We are still at the load test stage and will see how this goes. I imagine this is more of a concerning in concept than in reality. :) Cheers, Tim -Original Message- From: Mark Miller [mailto:markrmil...@gmail.com] Sent: Thursday, February 21, 2013 7:52 PM To: solr-user@lucene.apache.org Subject: Re: Is it possible to manually select a shard leader in a running SolrCloud? The leader doesn't really do a lot more work than any of the replicas, so I don't think it's likely that important. If someone starts running into problems, that's usually when we start looking for solutions. - Mark On Feb 21, 2013, at 10:20 PM, Vaillancourt, Tim tvaillanco...@ea.com wrote: I sent this request to ServerA in this case, which became the leader of all shards. As far as I know you're supposed to issue this call to just one server as it issues the calls to the other leaders/replicas in the background, right? I am expecting the single collections API call to spread the leaders evenly across SOLR instances. Hopefully I am just doing/expecting something wrong :). Tim Vaillancourt -Original Message- From: Upayavira [mailto:u...@odoko.co.uk] Sent: Thursday, February 21, 2013 1:44 PM To: solr-user@lucene.apache.org Subject: Re: Is it possible to manually select a shard leader in a running SolrCloud? Which of your three hosts did you point this request at? Upayavira On Thu, Feb 21, 2013, at 09:13 PM, Vaillancourt, Tim wrote: Correction, I used this curl: curl -v 'http://HOST:8983/solr/admin/collections?action=CREATEname=testnumShards=3replicationFactor=2maxShardsPerNode=2' So 3 instances, 3 shards, 2 replicas per shard. ServerA becomes leader of all 3 shards in 4.1 with this call. Tim Vaillancourt -Original Message- From: Vaillancourt, Tim [mailto:tvaillanco...@ea.com] Sent: Thursday, February 21, 2013 11:27 AM To: solr-user@lucene.apache.org; markrmil...@gmail.com Subject: RE: Is it possible to manually select a shard leader in a running SolrCloud? Thanks Mark, The real driver for me wanting to promote a different leader is when I create a new Collection via the Collections API across a multi-server SolrCloud, the leader of each shard is always the same host, so you're right that I'm tackling the wrong problem with this request, although it would fix it for me. If I create the cores manually via the cores API, one-by-one, I am able to get what I expect, but when running this Collections API call on a 3 SOLR 4.1 instance, 3 shard setup, 1 server becomes the leader of all 3 shards, meaning it will get all the writes for everything (correct me if I am wrong). If so, this will not scale well with all writes to one node (or correct me if I am wrong)? curl -v 'http://HOST:8983/solr/admin/collections?action=CREATEname=testnumShards=3replicationFactor=1maxShardsPerNode=2' Currently on my 3 instance SOLR 4.1 setup, the above call creates the following: - ServerA is the leader of all 3 shards (the problem I want to address). - ServerB + ServerC are automagically replicas of the 3 leader shards on ServerA. So again, my issue is one server gets all the writes. Does anyone else encounter this? If so, I should spawn a separate thread on my specific issue. Cheers, Tim -Original Message- From: Mark Miller [mailto:markrmil...@gmail.com] Sent: Tuesday, February 19, 2013 8:44 PM To: solr-user@lucene.apache.org Subject: Re: Is it possible to manually select a shard leader in a running SolrCloud? You can't easily do it the way it's implemented in ZooKeeper. We would probably internally have to do the same thing - elect a new leader and drop him until the one we wanted came up. The main thing doing it internally would gain is that you could skip the elected guy from becoming the actual leader and just move on to the next candidate. Still some tricky corner cases to deal with and such as well. I think for most things you would use this to solve, there is probably an alternate thing that should be addressed. - Mark On Mon, Feb 18, 2013 at 4:15 PM, Vaillancourt, Tim tvaillanco...@ea.com wrote: Hey all, I feel having to unload the leader core to force an election is hacky, and as far as I know would still leave which node becomes the Leader to chance, ie I cannot guarantee NodeX becomes Leader 100% in all cases. Also, this imposes additional load temporarily. Is there a way to force the winner of the Election, and if not, is there a known feature-request for this? Cheers, Tim Vaillancourt -Original Message- From: Joseph Dale [mailto:joey.d...@gmail.com] Sent: Sunday, February 03, 2013 7:42 AM To: solr-user@lucene.apache.org Subject: Re: Is it possible to manually select a shard leader in a running SolrCloud? With solrclound all cores are collections. The collections API it just a wrapper to call
RE: Is it possible to manually select a shard leader in a running SolrCloud?
Yeah, exactly - although there are workarounds, this is probably worth a feature request whether or not it is turned down. I have created these 2 new feature requests (after not being able to find duplicates through searching): 1) Please add support for manual leader election/promotion: https://issues.apache.org/jira/browse/SOLR-4491 2) Please add support for Collection API CREATE method to evenly distribute leader roles among instances: https://issues.apache.org/jira/browse/SOLR-4492 Thanks Mark/all! Tim Vaillancourt -Original Message- From: Boogie Shafer [mailto:boo...@ebrary.com] Sent: Friday, February 22, 2013 12:05 PM To: solr-user@lucene.apache.org Subject: Re: Is it possible to manually select a shard leader in a running SolrCloud? i think the use case here is more of a management one. -wanting to explicitly configure a specific node as leader (the reasons for this could vary) -wanting to gracefully/safely move a leader role from a specific node without going thru an actual election process (as was mentioned previously, why introduce variability?)..some sanity checks ala role moves if nodes are actually in sync, etc. otherwise it stays put the most obvious situation where i could see this being beneficial would be for minimizing elections, and the transitions, when doing wide service restarts for config changes, etc (restart all the replicas, gracefully move the leader to a restarted replica cleanly and orderly once everything is back in sync, restart the now former leader) On Thu, Feb 21, 2013 at 7:51 PM, Mark Miller markrmil...@gmail.com wrote: The leader doesn't really do a lot more work than any of the replicas, so I don't think it's likely that important. If someone starts running into problems, that's usually when we start looking for solutions. - Mark On Feb 21, 2013, at 10:20 PM, Vaillancourt, Tim tvaillanco...@ea.com wrote: I sent this request to ServerA in this case, which became the leader of all shards. As far as I know you're supposed to issue this call to just one server as it issues the calls to the other leaders/replicas in the background, right? I am expecting the single collections API call to spread the leaders evenly across SOLR instances. Hopefully I am just doing/expecting something wrong :). Tim Vaillancourt -Original Message- From: Upayavira [mailto:u...@odoko.co.uk] Sent: Thursday, February 21, 2013 1:44 PM To: solr-user@lucene.apache.org Subject: Re: Is it possible to manually select a shard leader in a running SolrCloud? Which of your three hosts did you point this request at? Upayavira On Thu, Feb 21, 2013, at 09:13 PM, Vaillancourt, Tim wrote: Correction, I used this curl: curl -v 'http://HOST:8983/solr/admin/collections?action=CREATEname=testnumShards=3replicationFactor=2maxShardsPerNode=2' So 3 instances, 3 shards, 2 replicas per shard. ServerA becomes leader of all 3 shards in 4.1 with this call. Tim Vaillancourt -Original Message- From: Vaillancourt, Tim [mailto:tvaillanco...@ea.com] Sent: Thursday, February 21, 2013 11:27 AM To: solr-user@lucene.apache.org; markrmil...@gmail.com Subject: RE: Is it possible to manually select a shard leader in a running SolrCloud? Thanks Mark, The real driver for me wanting to promote a different leader is when I create a new Collection via the Collections API across a multi-server SolrCloud, the leader of each shard is always the same host, so you're right that I'm tackling the wrong problem with this request, although it would fix it for me. If I create the cores manually via the cores API, one-by-one, I am able to get what I expect, but when running this Collections API call on a 3 SOLR 4.1 instance, 3 shard setup, 1 server becomes the leader of all 3 shards, meaning it will get all the writes for everything (correct me if I am wrong). If so, this will not scale well with all writes to one node (or correct me if I am wrong)? curl -v 'http://HOST:8983/solr/admin/collections?action=CREATEname=testnumShards=3replicationFactor=1maxShardsPerNode=2' Currently on my 3 instance SOLR 4.1 setup, the above call creates the following: - ServerA is the leader of all 3 shards (the problem I want to address). - ServerB + ServerC are automagically replicas of the 3 leader shards on ServerA. So again, my issue is one server gets all the writes. Does anyone else encounter this? If so, I should spawn a separate thread on my specific issue. Cheers, Tim -Original Message- From: Mark Miller [mailto:markrmil...@gmail.com] Sent: Tuesday, February 19, 2013 8:44 PM To: solr-user@lucene.apache.org Subject: Re: Is it possible to manually select a shard leader in a running SolrCloud? You can't easily do it the way it's implemented in ZooKeeper. We would probably internally have to do the same thing - elect a new leader and drop him until the one we wanted came up. The main thing
RE: Is it possible to manually select a shard leader in a running SolrCloud?
Thanks Mark, The real driver for me wanting to promote a different leader is when I create a new Collection via the Collections API across a multi-server SolrCloud, the leader of each shard is always the same host, so you're right that I'm tackling the wrong problem with this request, although it would fix it for me. If I create the cores manually via the cores API, one-by-one, I am able to get what I expect, but when running this Collections API call on a 3 SOLR 4.1 instance, 3 shard setup, 1 server becomes the leader of all 3 shards, meaning it will get all the writes for everything (correct me if I am wrong). If so, this will not scale well with all writes to one node (or correct me if I am wrong)? curl -v 'http://HOST:8983/solr/admin/collections?action=CREATEname=testnumShards=3replicationFactor=1maxShardsPerNode=2' Currently on my 3 instance SOLR 4.1 setup, the above call creates the following: - ServerA is the leader of all 3 shards (the problem I want to address). - ServerB + ServerC are automagically replicas of the 3 leader shards on ServerA. So again, my issue is one server gets all the writes. Does anyone else encounter this? If so, I should spawn a separate thread on my specific issue. Cheers, Tim -Original Message- From: Mark Miller [mailto:markrmil...@gmail.com] Sent: Tuesday, February 19, 2013 8:44 PM To: solr-user@lucene.apache.org Subject: Re: Is it possible to manually select a shard leader in a running SolrCloud? You can't easily do it the way it's implemented in ZooKeeper. We would probably internally have to do the same thing - elect a new leader and drop him until the one we wanted came up. The main thing doing it internally would gain is that you could skip the elected guy from becoming the actual leader and just move on to the next candidate. Still some tricky corner cases to deal with and such as well. I think for most things you would use this to solve, there is probably an alternate thing that should be addressed. - Mark On Mon, Feb 18, 2013 at 4:15 PM, Vaillancourt, Tim tvaillanco...@ea.com wrote: Hey all, I feel having to unload the leader core to force an election is hacky, and as far as I know would still leave which node becomes the Leader to chance, ie I cannot guarantee NodeX becomes Leader 100% in all cases. Also, this imposes additional load temporarily. Is there a way to force the winner of the Election, and if not, is there a known feature-request for this? Cheers, Tim Vaillancourt -Original Message- From: Joseph Dale [mailto:joey.d...@gmail.com] Sent: Sunday, February 03, 2013 7:42 AM To: solr-user@lucene.apache.org Subject: Re: Is it possible to manually select a shard leader in a running SolrCloud? With solrclound all cores are collections. The collections API it just a wrapper to call the core api a million times with one command. to /solr/admin/cores?action=CREATEname=core1collection=core1shard=1 Basically your creating the shard again, after leader props have gone out. Solr will check ZK and find a core meeting that description, then simply get a copy of the index from the leader of that shard. On Feb 3, 2013, at 10:37 AM, Brett Hoerner br...@bretthoerner.com wrote: What is the inverse I'd use to re-create/load a core on another machine but make sure it's also known to SolrCloud/as a shard? On Sat, Feb 2, 2013 at 4:01 PM, Joseph Dale joey.d...@gmail.com wrote: To be more clear lets say bob it the leader of core 1. On bob do a /admin/cores?action=unloadname=core1. This removes the core/shard from bob, giving the other servers a chance to grab leader props. -Joey On Feb 2, 2013, at 11:27 AM, Brett Hoerner br...@bretthoerner.com wrote: Hi, I have a 5 server cluster running 1 collection with 20 shards, replication factor of 2. Earlier this week I had to do a rolling restart across the cluster, this worked great and the cluster stayed up the whole time. The problem is that the last node I restarted is now the leader of 0 shards, and is just holding replicas. I've noticed this node has abnormally high load average, while the other nodes (who have the same number of shards, but more leaders on average) are fine. First, I'm wondering if that loud could be related to being a 5x replica and 0x leader? Second, I was wondering if I could somehow flag single shards to re-elect a leader (or force a leader) so that I could more evenly distribute how many leader shards each physical server has running? Thanks. -- - Mark
RE: Is it possible to manually select a shard leader in a running SolrCloud?
Correction, I used this curl: curl -v 'http://HOST:8983/solr/admin/collections?action=CREATEname=testnumShards=3replicationFactor=2maxShardsPerNode=2' So 3 instances, 3 shards, 2 replicas per shard. ServerA becomes leader of all 3 shards in 4.1 with this call. Tim Vaillancourt -Original Message- From: Vaillancourt, Tim [mailto:tvaillanco...@ea.com] Sent: Thursday, February 21, 2013 11:27 AM To: solr-user@lucene.apache.org; markrmil...@gmail.com Subject: RE: Is it possible to manually select a shard leader in a running SolrCloud? Thanks Mark, The real driver for me wanting to promote a different leader is when I create a new Collection via the Collections API across a multi-server SolrCloud, the leader of each shard is always the same host, so you're right that I'm tackling the wrong problem with this request, although it would fix it for me. If I create the cores manually via the cores API, one-by-one, I am able to get what I expect, but when running this Collections API call on a 3 SOLR 4.1 instance, 3 shard setup, 1 server becomes the leader of all 3 shards, meaning it will get all the writes for everything (correct me if I am wrong). If so, this will not scale well with all writes to one node (or correct me if I am wrong)? curl -v 'http://HOST:8983/solr/admin/collections?action=CREATEname=testnumShards=3replicationFactor=1maxShardsPerNode=2' Currently on my 3 instance SOLR 4.1 setup, the above call creates the following: - ServerA is the leader of all 3 shards (the problem I want to address). - ServerB + ServerC are automagically replicas of the 3 leader shards on ServerA. So again, my issue is one server gets all the writes. Does anyone else encounter this? If so, I should spawn a separate thread on my specific issue. Cheers, Tim -Original Message- From: Mark Miller [mailto:markrmil...@gmail.com] Sent: Tuesday, February 19, 2013 8:44 PM To: solr-user@lucene.apache.org Subject: Re: Is it possible to manually select a shard leader in a running SolrCloud? You can't easily do it the way it's implemented in ZooKeeper. We would probably internally have to do the same thing - elect a new leader and drop him until the one we wanted came up. The main thing doing it internally would gain is that you could skip the elected guy from becoming the actual leader and just move on to the next candidate. Still some tricky corner cases to deal with and such as well. I think for most things you would use this to solve, there is probably an alternate thing that should be addressed. - Mark On Mon, Feb 18, 2013 at 4:15 PM, Vaillancourt, Tim tvaillanco...@ea.com wrote: Hey all, I feel having to unload the leader core to force an election is hacky, and as far as I know would still leave which node becomes the Leader to chance, ie I cannot guarantee NodeX becomes Leader 100% in all cases. Also, this imposes additional load temporarily. Is there a way to force the winner of the Election, and if not, is there a known feature-request for this? Cheers, Tim Vaillancourt -Original Message- From: Joseph Dale [mailto:joey.d...@gmail.com] Sent: Sunday, February 03, 2013 7:42 AM To: solr-user@lucene.apache.org Subject: Re: Is it possible to manually select a shard leader in a running SolrCloud? With solrclound all cores are collections. The collections API it just a wrapper to call the core api a million times with one command. to /solr/admin/cores?action=CREATEname=core1collection=core1shard=1 Basically your creating the shard again, after leader props have gone out. Solr will check ZK and find a core meeting that description, then simply get a copy of the index from the leader of that shard. On Feb 3, 2013, at 10:37 AM, Brett Hoerner br...@bretthoerner.com wrote: What is the inverse I'd use to re-create/load a core on another machine but make sure it's also known to SolrCloud/as a shard? On Sat, Feb 2, 2013 at 4:01 PM, Joseph Dale joey.d...@gmail.com wrote: To be more clear lets say bob it the leader of core 1. On bob do a /admin/cores?action=unloadname=core1. This removes the core/shard from bob, giving the other servers a chance to grab leader props. -Joey On Feb 2, 2013, at 11:27 AM, Brett Hoerner br...@bretthoerner.com wrote: Hi, I have a 5 server cluster running 1 collection with 20 shards, replication factor of 2. Earlier this week I had to do a rolling restart across the cluster, this worked great and the cluster stayed up the whole time. The problem is that the last node I restarted is now the leader of 0 shards, and is just holding replicas. I've noticed this node has abnormally high load average, while the other nodes (who have the same number of shards, but more leaders on average) are fine. First, I'm wondering if that loud could be related to being a 5x replica and 0x leader? Second, I was wondering if I could somehow flag single shards to re-elect a leader (or force
RE: Is it possible to manually select a shard leader in a running SolrCloud?
I sent this request to ServerA in this case, which became the leader of all shards. As far as I know you're supposed to issue this call to just one server as it issues the calls to the other leaders/replicas in the background, right? I am expecting the single collections API call to spread the leaders evenly across SOLR instances. Hopefully I am just doing/expecting something wrong :). Tim Vaillancourt -Original Message- From: Upayavira [mailto:u...@odoko.co.uk] Sent: Thursday, February 21, 2013 1:44 PM To: solr-user@lucene.apache.org Subject: Re: Is it possible to manually select a shard leader in a running SolrCloud? Which of your three hosts did you point this request at? Upayavira On Thu, Feb 21, 2013, at 09:13 PM, Vaillancourt, Tim wrote: Correction, I used this curl: curl -v 'http://HOST:8983/solr/admin/collections?action=CREATEname=testnumShards=3replicationFactor=2maxShardsPerNode=2' So 3 instances, 3 shards, 2 replicas per shard. ServerA becomes leader of all 3 shards in 4.1 with this call. Tim Vaillancourt -Original Message- From: Vaillancourt, Tim [mailto:tvaillanco...@ea.com] Sent: Thursday, February 21, 2013 11:27 AM To: solr-user@lucene.apache.org; markrmil...@gmail.com Subject: RE: Is it possible to manually select a shard leader in a running SolrCloud? Thanks Mark, The real driver for me wanting to promote a different leader is when I create a new Collection via the Collections API across a multi-server SolrCloud, the leader of each shard is always the same host, so you're right that I'm tackling the wrong problem with this request, although it would fix it for me. If I create the cores manually via the cores API, one-by-one, I am able to get what I expect, but when running this Collections API call on a 3 SOLR 4.1 instance, 3 shard setup, 1 server becomes the leader of all 3 shards, meaning it will get all the writes for everything (correct me if I am wrong). If so, this will not scale well with all writes to one node (or correct me if I am wrong)? curl -v 'http://HOST:8983/solr/admin/collections?action=CREATEname=testnumShards=3replicationFactor=1maxShardsPerNode=2' Currently on my 3 instance SOLR 4.1 setup, the above call creates the following: - ServerA is the leader of all 3 shards (the problem I want to address). - ServerB + ServerC are automagically replicas of the 3 leader shards on ServerA. So again, my issue is one server gets all the writes. Does anyone else encounter this? If so, I should spawn a separate thread on my specific issue. Cheers, Tim -Original Message- From: Mark Miller [mailto:markrmil...@gmail.com] Sent: Tuesday, February 19, 2013 8:44 PM To: solr-user@lucene.apache.org Subject: Re: Is it possible to manually select a shard leader in a running SolrCloud? You can't easily do it the way it's implemented in ZooKeeper. We would probably internally have to do the same thing - elect a new leader and drop him until the one we wanted came up. The main thing doing it internally would gain is that you could skip the elected guy from becoming the actual leader and just move on to the next candidate. Still some tricky corner cases to deal with and such as well. I think for most things you would use this to solve, there is probably an alternate thing that should be addressed. - Mark On Mon, Feb 18, 2013 at 4:15 PM, Vaillancourt, Tim tvaillanco...@ea.com wrote: Hey all, I feel having to unload the leader core to force an election is hacky, and as far as I know would still leave which node becomes the Leader to chance, ie I cannot guarantee NodeX becomes Leader 100% in all cases. Also, this imposes additional load temporarily. Is there a way to force the winner of the Election, and if not, is there a known feature-request for this? Cheers, Tim Vaillancourt -Original Message- From: Joseph Dale [mailto:joey.d...@gmail.com] Sent: Sunday, February 03, 2013 7:42 AM To: solr-user@lucene.apache.org Subject: Re: Is it possible to manually select a shard leader in a running SolrCloud? With solrclound all cores are collections. The collections API it just a wrapper to call the core api a million times with one command. to /solr/admin/cores?action=CREATEname=core1collection=core1shard=1 Basically your creating the shard again, after leader props have gone out. Solr will check ZK and find a core meeting that description, then simply get a copy of the index from the leader of that shard. On Feb 3, 2013, at 10:37 AM, Brett Hoerner br...@bretthoerner.com wrote: What is the inverse I'd use to re-create/load a core on another machine but make sure it's also known to SolrCloud/as a shard? On Sat, Feb 2, 2013 at 4:01 PM, Joseph Dale joey.d...@gmail.com wrote: To be more clear lets say bob it the leader of core 1. On bob do a /admin/cores?action
RE: Is it possible to manually select a shard leader in a running SolrCloud?
Hey all, I feel having to unload the leader core to force an election is hacky, and as far as I know would still leave which node becomes the Leader to chance, ie I cannot guarantee NodeX becomes Leader 100% in all cases. Also, this imposes additional load temporarily. Is there a way to force the winner of the Election, and if not, is there a known feature-request for this? Cheers, Tim Vaillancourt -Original Message- From: Joseph Dale [mailto:joey.d...@gmail.com] Sent: Sunday, February 03, 2013 7:42 AM To: solr-user@lucene.apache.org Subject: Re: Is it possible to manually select a shard leader in a running SolrCloud? With solrclound all cores are collections. The collections API it just a wrapper to call the core api a million times with one command. to /solr/admin/cores?action=CREATEname=core1collection=core1shard=1 Basically your creating the shard again, after leader props have gone out. Solr will check ZK and find a core meeting that description, then simply get a copy of the index from the leader of that shard. On Feb 3, 2013, at 10:37 AM, Brett Hoerner br...@bretthoerner.com wrote: What is the inverse I'd use to re-create/load a core on another machine but make sure it's also known to SolrCloud/as a shard? On Sat, Feb 2, 2013 at 4:01 PM, Joseph Dale joey.d...@gmail.com wrote: To be more clear lets say bob it the leader of core 1. On bob do a /admin/cores?action=unloadname=core1. This removes the core/shard from bob, giving the other servers a chance to grab leader props. -Joey On Feb 2, 2013, at 11:27 AM, Brett Hoerner br...@bretthoerner.com wrote: Hi, I have a 5 server cluster running 1 collection with 20 shards, replication factor of 2. Earlier this week I had to do a rolling restart across the cluster, this worked great and the cluster stayed up the whole time. The problem is that the last node I restarted is now the leader of 0 shards, and is just holding replicas. I've noticed this node has abnormally high load average, while the other nodes (who have the same number of shards, but more leaders on average) are fine. First, I'm wondering if that loud could be related to being a 5x replica and 0x leader? Second, I was wondering if I could somehow flag single shards to re-elect a leader (or force a leader) so that I could more evenly distribute how many leader shards each physical server has running? Thanks.
