Switch to new leader transparently?
Hi there, I've built a SolrCloud cluster from example, but I have some question. When I send query to one leader (say http://xxx.xxx.xxx.xxx:8983/solr/collection1) and no problem everything will be fine. When I shutdown that leader, the other replica( http://xxx.xxx.xxx.xxx:9983/solr/collection1) in the some shard will be new leader. The problem is: The application doesn't know new leader's location and still send request to http://xxx.xxx.xxx.xxx:8983/solr/collection1 and of course no response. How can I know new leader in my application? Are there any mechanism that application can send request to one fixed endpoint no matter who is leader? For example, application just send to http://xxx.xxx.xxx.xxx:8983/solr/collection1 even the real leader run on http://xxx.xxx.xxx.xxx:9983/solr/collection1 Please help on this or give me some key infomation to google it. Many thanks. Floyd
Re: Switch to new leader transparently?
You don't really need to direct any query specifically to a leader. It will automatically be routed to the right leader. You may put a load balancer on top to just fix the problem with querying a node that has gone away. Also, ZK aware SolrJ Java client that load-balances across all nodes in cluster. On Wed, Jul 10, 2013 at 2:52 PM, Floyd Wu floyd...@gmail.com wrote: Hi there, I've built a SolrCloud cluster from example, but I have some question. When I send query to one leader (say http://xxx.xxx.xxx.xxx:8983/solr/collection1) and no problem everything will be fine. When I shutdown that leader, the other replica( http://xxx.xxx.xxx.xxx:9983/solr/collection1) in the some shard will be new leader. The problem is: The application doesn't know new leader's location and still send request to http://xxx.xxx.xxx.xxx:8983/solr/collection1 and of course no response. How can I know new leader in my application? Are there any mechanism that application can send request to one fixed endpoint no matter who is leader? For example, application just send to http://xxx.xxx.xxx.xxx:8983/solr/collection1 even the real leader run on http://xxx.xxx.xxx.xxx:9983/solr/collection1 Please help on this or give me some key infomation to google it. Many thanks. Floyd -- Anshum Gupta http://www.anshumgupta.net
Re: Switch to new leader transparently?
You can define a CloudSolrServer as like that:
*private static CloudSolrServer solrServer;*
and then define the addres of your zookeeper host:
*private static String zkHost = localhost:9983;*
initialize your variable:
*solrServer = new CloudSolrServer(zkHost);*
You can get leader list as like:
*ClusterState clusterState =
cloudSolrServer.getZkStateReader().getClusterState();
ListReplica leaderList = new ArrayList();
for (Slice slice : clusterState.getSlices(collectionName)) {
leaderList.add(slice.getLeader()); /
}*
For querying you can try that:
*
*
*SolrQuery solrQuery = new SolrQuery();*
*//fill your **solrQuery variable here**
*
*QueryRequest queryRequest = new QueryRequest(solrQuery,
SolrRequest.METHOD.POST);
queryRequest.process(**solrServer**);*
CloudSolrServer uses LBHttpSolrServer by default. It's definiton is like
that: *LBHttpSolrServer or Load Balanced HttpSolrServer is just a wrapper
to CommonsHttpSolrServer. This is useful when you have multiple SolrServers
and query requests need to be Load Balanced among them. It offers automatic
failover when a server goes down and it detects when the server comes back
up.*
*
*
*
*
2013/7/10 Anshum Gupta ans...@anshumgupta.net
You don't really need to direct any query specifically to a leader. It will
automatically be routed to the right leader.
You may put a load balancer on top to just fix the problem with querying a
node that has gone away.
Also, ZK aware SolrJ Java client that load-balances across all nodes in
cluster.
On Wed, Jul 10, 2013 at 2:52 PM, Floyd Wu floyd...@gmail.com wrote:
Hi there,
I've built a SolrCloud cluster from example, but I have some question.
When I send query to one leader (say
http://xxx.xxx.xxx.xxx:8983/solr/collection1) and no problem everything
will be fine.
When I shutdown that leader, the other replica(
http://xxx.xxx.xxx.xxx:9983/solr/collection1) in the some shard will be
new
leader. The problem is:
The application doesn't know new leader's location and still send request
to http://xxx.xxx.xxx.xxx:8983/solr/collection1 and of course no
response.
How can I know new leader in my application?
Are there any mechanism that application can send request to one fixed
endpoint no matter who is leader?
For example, application just send to
http://xxx.xxx.xxx.xxx:8983/solr/collection1
even the real leader run on http://xxx.xxx.xxx.xxx:9983/solr/collection1
Please help on this or give me some key infomation to google it.
Many thanks.
Floyd
--
Anshum Gupta
http://www.anshumgupta.net
Re: Switch to new leader transparently?
Hi anshum Thanks for your response. My application is developed using C#, so I can't use CloudSolrServer with SolrJ. My problem is there is a setting in my application SolrUrl = http://xxx.xxx.xxx.xxx:8983/solr/collection1 When this Solr instance shutdown or crash, I have to change this setting. I've read source code of CloudSolrServer.java in SolrJ just few minutes ago. It seems to that CloudSolrServer first read cluster state from zk ( or some live node) to retrieve info and then use this info to decide which node to send request. Maybe I have to modify my application to mimic CloudSolrServer impl. Any idea? Floyd 2013/7/10 Anshum Gupta ans...@anshumgupta.net You don't really need to direct any query specifically to a leader. It will automatically be routed to the right leader. You may put a load balancer on top to just fix the problem with querying a node that has gone away. Also, ZK aware SolrJ Java client that load-balances across all nodes in cluster. On Wed, Jul 10, 2013 at 2:52 PM, Floyd Wu floyd...@gmail.com wrote: Hi there, I've built a SolrCloud cluster from example, but I have some question. When I send query to one leader (say http://xxx.xxx.xxx.xxx:8983/solr/collection1) and no problem everything will be fine. When I shutdown that leader, the other replica( http://xxx.xxx.xxx.xxx:9983/solr/collection1) in the some shard will be new leader. The problem is: The application doesn't know new leader's location and still send request to http://xxx.xxx.xxx.xxx:8983/solr/collection1 and of course no response. How can I know new leader in my application? Are there any mechanism that application can send request to one fixed endpoint no matter who is leader? For example, application just send to http://xxx.xxx.xxx.xxx:8983/solr/collection1 even the real leader run on http://xxx.xxx.xxx.xxx:9983/solr/collection1 Please help on this or give me some key infomation to google it. Many thanks. Floyd -- Anshum Gupta http://www.anshumgupta.net
Re: Switch to new leader transparently?
You can check the source code of LBHttpSolrServer and try to implement something like that as your own. 2013/7/10 Floyd Wu floyd...@gmail.com Hi anshum Thanks for your response. My application is developed using C#, so I can't use CloudSolrServer with SolrJ. My problem is there is a setting in my application SolrUrl = http://xxx.xxx.xxx.xxx:8983/solr/collection1 When this Solr instance shutdown or crash, I have to change this setting. I've read source code of CloudSolrServer.java in SolrJ just few minutes ago. It seems to that CloudSolrServer first read cluster state from zk ( or some live node) to retrieve info and then use this info to decide which node to send request. Maybe I have to modify my application to mimic CloudSolrServer impl. Any idea? Floyd 2013/7/10 Anshum Gupta ans...@anshumgupta.net You don't really need to direct any query specifically to a leader. It will automatically be routed to the right leader. You may put a load balancer on top to just fix the problem with querying a node that has gone away. Also, ZK aware SolrJ Java client that load-balances across all nodes in cluster. On Wed, Jul 10, 2013 at 2:52 PM, Floyd Wu floyd...@gmail.com wrote: Hi there, I've built a SolrCloud cluster from example, but I have some question. When I send query to one leader (say http://xxx.xxx.xxx.xxx:8983/solr/collection1) and no problem everything will be fine. When I shutdown that leader, the other replica( http://xxx.xxx.xxx.xxx:9983/solr/collection1) in the some shard will be new leader. The problem is: The application doesn't know new leader's location and still send request to http://xxx.xxx.xxx.xxx:8983/solr/collection1 and of course no response. How can I know new leader in my application? Are there any mechanism that application can send request to one fixed endpoint no matter who is leader? For example, application just send to http://xxx.xxx.xxx.xxx:8983/solr/collection1 even the real leader run on http://xxx.xxx.xxx.xxx:9983/solr/collection1 Please help on this or give me some key infomation to google it. Many thanks. Floyd -- Anshum Gupta http://www.anshumgupta.net
Re: Switch to new leader transparently?
Hi Furkan
I'm using C#, SolrJ won't help on this, but its impl is a good reference
for me. Thanks for your help.
by the way, how to fetch/get cluster state from zk directly in plain http
or tcp socket?
In my SolrCloud cluster, I'm using standalone zk to coordinate.
Floyd
2013/7/10 Furkan KAMACI furkankam...@gmail.com
You can define a CloudSolrServer as like that:
*private static CloudSolrServer solrServer;*
and then define the addres of your zookeeper host:
*private static String zkHost = localhost:9983;*
initialize your variable:
*solrServer = new CloudSolrServer(zkHost);*
You can get leader list as like:
*ClusterState clusterState =
cloudSolrServer.getZkStateReader().getClusterState();
ListReplica leaderList = new ArrayList();
for (Slice slice : clusterState.getSlices(collectionName)) {
leaderList.add(slice.getLeader()); /
}*
For querying you can try that:
*
*
*SolrQuery solrQuery = new SolrQuery();*
*//fill your **solrQuery variable here**
*
*QueryRequest queryRequest = new QueryRequest(solrQuery,
SolrRequest.METHOD.POST);
queryRequest.process(**solrServer**);*
CloudSolrServer uses LBHttpSolrServer by default. It's definiton is like
that: *LBHttpSolrServer or Load Balanced HttpSolrServer is just a wrapper
to CommonsHttpSolrServer. This is useful when you have multiple SolrServers
and query requests need to be Load Balanced among them. It offers automatic
failover when a server goes down and it detects when the server comes back
up.*
*
*
*
*
2013/7/10 Anshum Gupta ans...@anshumgupta.net
You don't really need to direct any query specifically to a leader. It
will
automatically be routed to the right leader.
You may put a load balancer on top to just fix the problem with querying
a
node that has gone away.
Also, ZK aware SolrJ Java client that load-balances across all nodes in
cluster.
On Wed, Jul 10, 2013 at 2:52 PM, Floyd Wu floyd...@gmail.com wrote:
Hi there,
I've built a SolrCloud cluster from example, but I have some question.
When I send query to one leader (say
http://xxx.xxx.xxx.xxx:8983/solr/collection1) and no problem
everything
will be fine.
When I shutdown that leader, the other replica(
http://xxx.xxx.xxx.xxx:9983/solr/collection1) in the some shard will
be
new
leader. The problem is:
The application doesn't know new leader's location and still send
request
to http://xxx.xxx.xxx.xxx:8983/solr/collection1 and of course no
response.
How can I know new leader in my application?
Are there any mechanism that application can send request to one fixed
endpoint no matter who is leader?
For example, application just send to
http://xxx.xxx.xxx.xxx:8983/solr/collection1
even the real leader run on
http://xxx.xxx.xxx.xxx:9983/solr/collection1
Please help on this or give me some key infomation to google it.
Many thanks.
Floyd
--
Anshum Gupta
http://www.anshumgupta.net
Re: Switch to new leader transparently?
By the this is not related to your question but this may help you for
connecting Solr via C#: http://solrsharp.codeplex.com/
2013/7/10 Floyd Wu floyd...@gmail.com
Hi Furkan
I'm using C#, SolrJ won't help on this, but its impl is a good reference
for me. Thanks for your help.
by the way, how to fetch/get cluster state from zk directly in plain http
or tcp socket?
In my SolrCloud cluster, I'm using standalone zk to coordinate.
Floyd
2013/7/10 Furkan KAMACI furkankam...@gmail.com
You can define a CloudSolrServer as like that:
*private static CloudSolrServer solrServer;*
and then define the addres of your zookeeper host:
*private static String zkHost = localhost:9983;*
initialize your variable:
*solrServer = new CloudSolrServer(zkHost);*
You can get leader list as like:
*ClusterState clusterState =
cloudSolrServer.getZkStateReader().getClusterState();
ListReplica leaderList = new ArrayList();
for (Slice slice : clusterState.getSlices(collectionName)) {
leaderList.add(slice.getLeader()); /
}*
For querying you can try that:
*
*
*SolrQuery solrQuery = new SolrQuery();*
*//fill your **solrQuery variable here**
*
*QueryRequest queryRequest = new QueryRequest(solrQuery,
SolrRequest.METHOD.POST);
queryRequest.process(**solrServer**);*
CloudSolrServer uses LBHttpSolrServer by default. It's definiton is like
that: *LBHttpSolrServer or Load Balanced HttpSolrServer is just a
wrapper
to CommonsHttpSolrServer. This is useful when you have multiple
SolrServers
and query requests need to be Load Balanced among them. It offers
automatic
failover when a server goes down and it detects when the server comes
back
up.*
*
*
*
*
2013/7/10 Anshum Gupta ans...@anshumgupta.net
You don't really need to direct any query specifically to a leader. It
will
automatically be routed to the right leader.
You may put a load balancer on top to just fix the problem with
querying
a
node that has gone away.
Also, ZK aware SolrJ Java client that load-balances across all nodes in
cluster.
On Wed, Jul 10, 2013 at 2:52 PM, Floyd Wu floyd...@gmail.com wrote:
Hi there,
I've built a SolrCloud cluster from example, but I have some
question.
When I send query to one leader (say
http://xxx.xxx.xxx.xxx:8983/solr/collection1) and no problem
everything
will be fine.
When I shutdown that leader, the other replica(
http://xxx.xxx.xxx.xxx:9983/solr/collection1) in the some shard will
be
new
leader. The problem is:
The application doesn't know new leader's location and still send
request
to http://xxx.xxx.xxx.xxx:8983/solr/collection1 and of course no
response.
How can I know new leader in my application?
Are there any mechanism that application can send request to one
fixed
endpoint no matter who is leader?
For example, application just send to
http://xxx.xxx.xxx.xxx:8983/solr/collection1
even the real leader run on
http://xxx.xxx.xxx.xxx:9983/solr/collection1
Please help on this or give me some key infomation to google it.
Many thanks.
Floyd
--
Anshum Gupta
http://www.anshumgupta.net
Re: Switch to new leader transparently?
Floyd:
The Apache Zookeeper project should have the relevant info
on how to get the state from ZK directly.
FWIW,
Erick
On Wed, Jul 10, 2013 at 6:41 AM, Furkan KAMACI furkankam...@gmail.com wrote:
By the this is not related to your question but this may help you for
connecting Solr via C#: http://solrsharp.codeplex.com/
2013/7/10 Floyd Wu floyd...@gmail.com
Hi Furkan
I'm using C#, SolrJ won't help on this, but its impl is a good reference
for me. Thanks for your help.
by the way, how to fetch/get cluster state from zk directly in plain http
or tcp socket?
In my SolrCloud cluster, I'm using standalone zk to coordinate.
Floyd
2013/7/10 Furkan KAMACI furkankam...@gmail.com
You can define a CloudSolrServer as like that:
*private static CloudSolrServer solrServer;*
and then define the addres of your zookeeper host:
*private static String zkHost = localhost:9983;*
initialize your variable:
*solrServer = new CloudSolrServer(zkHost);*
You can get leader list as like:
*ClusterState clusterState =
cloudSolrServer.getZkStateReader().getClusterState();
ListReplica leaderList = new ArrayList();
for (Slice slice : clusterState.getSlices(collectionName)) {
leaderList.add(slice.getLeader()); /
}*
For querying you can try that:
*
*
*SolrQuery solrQuery = new SolrQuery();*
*//fill your **solrQuery variable here**
*
*QueryRequest queryRequest = new QueryRequest(solrQuery,
SolrRequest.METHOD.POST);
queryRequest.process(**solrServer**);*
CloudSolrServer uses LBHttpSolrServer by default. It's definiton is like
that: *LBHttpSolrServer or Load Balanced HttpSolrServer is just a
wrapper
to CommonsHttpSolrServer. This is useful when you have multiple
SolrServers
and query requests need to be Load Balanced among them. It offers
automatic
failover when a server goes down and it detects when the server comes
back
up.*
*
*
*
*
2013/7/10 Anshum Gupta ans...@anshumgupta.net
You don't really need to direct any query specifically to a leader. It
will
automatically be routed to the right leader.
You may put a load balancer on top to just fix the problem with
querying
a
node that has gone away.
Also, ZK aware SolrJ Java client that load-balances across all nodes in
cluster.
On Wed, Jul 10, 2013 at 2:52 PM, Floyd Wu floyd...@gmail.com wrote:
Hi there,
I've built a SolrCloud cluster from example, but I have some
question.
When I send query to one leader (say
http://xxx.xxx.xxx.xxx:8983/solr/collection1) and no problem
everything
will be fine.
When I shutdown that leader, the other replica(
http://xxx.xxx.xxx.xxx:9983/solr/collection1) in the some shard will
be
new
leader. The problem is:
The application doesn't know new leader's location and still send
request
to http://xxx.xxx.xxx.xxx:8983/solr/collection1 and of course no
response.
How can I know new leader in my application?
Are there any mechanism that application can send request to one
fixed
endpoint no matter who is leader?
For example, application just send to
http://xxx.xxx.xxx.xxx:8983/solr/collection1
even the real leader run on
http://xxx.xxx.xxx.xxx:9983/solr/collection1
Please help on this or give me some key infomation to google it.
Many thanks.
Floyd
--
Anshum Gupta
http://www.anshumgupta.net
Re: Switch to new leader transparently?
Hi Floyd,
We use SolrNet to connect to Solr from a C# application. Since SolrNet is
not aware about SolrCloud or ZK, we use a Http load balancer in front of
the Solr nodes query via the load balancer url. You could use something
like HAProxy or Apache reverse proxy for load balancing.
On the other hand in order to write a ZK aware client in C# you could start
here: https://github.com/ewhauser/zookeeper/tree/trunk/src/dotnet
Regards,
Aloke
On Wed, Jul 10, 2013 at 4:11 PM, Furkan KAMACI furkankam...@gmail.comwrote:
By the this is not related to your question but this may help you for
connecting Solr via C#: http://solrsharp.codeplex.com/
2013/7/10 Floyd Wu floyd...@gmail.com
Hi Furkan
I'm using C#, SolrJ won't help on this, but its impl is a good reference
for me. Thanks for your help.
by the way, how to fetch/get cluster state from zk directly in plain http
or tcp socket?
In my SolrCloud cluster, I'm using standalone zk to coordinate.
Floyd
2013/7/10 Furkan KAMACI furkankam...@gmail.com
You can define a CloudSolrServer as like that:
*private static CloudSolrServer solrServer;*
and then define the addres of your zookeeper host:
*private static String zkHost = localhost:9983;*
initialize your variable:
*solrServer = new CloudSolrServer(zkHost);*
You can get leader list as like:
*ClusterState clusterState =
cloudSolrServer.getZkStateReader().getClusterState();
ListReplica leaderList = new ArrayList();
for (Slice slice : clusterState.getSlices(collectionName)) {
leaderList.add(slice.getLeader()); /
}*
For querying you can try that:
*
*
*SolrQuery solrQuery = new SolrQuery();*
*//fill your **solrQuery variable here**
*
*QueryRequest queryRequest = new QueryRequest(solrQuery,
SolrRequest.METHOD.POST);
queryRequest.process(**solrServer**);*
CloudSolrServer uses LBHttpSolrServer by default. It's definiton is
like
that: *LBHttpSolrServer or Load Balanced HttpSolrServer is just a
wrapper
to CommonsHttpSolrServer. This is useful when you have multiple
SolrServers
and query requests need to be Load Balanced among them. It offers
automatic
failover when a server goes down and it detects when the server comes
back
up.*
*
*
*
*
2013/7/10 Anshum Gupta ans...@anshumgupta.net
You don't really need to direct any query specifically to a leader.
It
will
automatically be routed to the right leader.
You may put a load balancer on top to just fix the problem with
querying
a
node that has gone away.
Also, ZK aware SolrJ Java client that load-balances across all nodes
in
cluster.
On Wed, Jul 10, 2013 at 2:52 PM, Floyd Wu floyd...@gmail.com
wrote:
Hi there,
I've built a SolrCloud cluster from example, but I have some
question.
When I send query to one leader (say
http://xxx.xxx.xxx.xxx:8983/solr/collection1) and no problem
everything
will be fine.
When I shutdown that leader, the other replica(
http://xxx.xxx.xxx.xxx:9983/solr/collection1) in the some shard
will
be
new
leader. The problem is:
The application doesn't know new leader's location and still send
request
to http://xxx.xxx.xxx.xxx:8983/solr/collection1 and of course no
response.
How can I know new leader in my application?
Are there any mechanism that application can send request to one
fixed
endpoint no matter who is leader?
For example, application just send to
http://xxx.xxx.xxx.xxx:8983/solr/collection1
even the real leader run on
http://xxx.xxx.xxx.xxx:9983/solr/collection1
Please help on this or give me some key infomation to google it.
Many thanks.
Floyd
--
Anshum Gupta
http://www.anshumgupta.net
Re: Switch to new leader transparently?
Thanks Aloke, I will do some research.
2013/7/10 下午9:45 於 Aloke Ghoshal alghos...@gmail.com 寫道：
Hi Floyd,
We use SolrNet to connect to Solr from a C# application. Since SolrNet is
not aware about SolrCloud or ZK, we use a Http load balancer in front of
the Solr nodes query via the load balancer url. You could use something
like HAProxy or Apache reverse proxy for load balancing.
On the other hand in order to write a ZK aware client in C# you could start
here: https://github.com/ewhauser/zookeeper/tree/trunk/src/dotnet
Regards,
Aloke
On Wed, Jul 10, 2013 at 4:11 PM, Furkan KAMACI furkankam...@gmail.com
wrote:
By the this is not related to your question but this may help you for
connecting Solr via C#: http://solrsharp.codeplex.com/
2013/7/10 Floyd Wu floyd...@gmail.com
Hi Furkan
I'm using C#, SolrJ won't help on this, but its impl is a good
reference
for me. Thanks for your help.
by the way, how to fetch/get cluster state from zk directly in plain
http
or tcp socket?
In my SolrCloud cluster, I'm using standalone zk to coordinate.
Floyd
2013/7/10 Furkan KAMACI furkankam...@gmail.com
You can define a CloudSolrServer as like that:
*private static CloudSolrServer solrServer;*
and then define the addres of your zookeeper host:
*private static String zkHost = localhost:9983;*
initialize your variable:
*solrServer = new CloudSolrServer(zkHost);*
You can get leader list as like:
*ClusterState clusterState =
cloudSolrServer.getZkStateReader().getClusterState();
ListReplica leaderList = new ArrayList();
for (Slice slice : clusterState.getSlices(collectionName)) {
leaderList.add(slice.getLeader()); /
}*
For querying you can try that:
*
*
*SolrQuery solrQuery = new SolrQuery();*
*//fill your **solrQuery variable here**
*
*QueryRequest queryRequest = new QueryRequest(solrQuery,
SolrRequest.METHOD.POST);
queryRequest.process(**solrServer**);*
CloudSolrServer uses LBHttpSolrServer by default. It's definiton is
like
that: *LBHttpSolrServer or Load Balanced HttpSolrServer is just a
wrapper
to CommonsHttpSolrServer. This is useful when you have multiple
SolrServers
and query requests need to be Load Balanced among them. It offers
automatic
failover when a server goes down and it detects when the server comes
back
up.*
*
*
*
*
2013/7/10 Anshum Gupta ans...@anshumgupta.net
You don't really need to direct any query specifically to a leader.
It
will
automatically be routed to the right leader.
You may put a load balancer on top to just fix the problem with
querying
a
node that has gone away.
Also, ZK aware SolrJ Java client that load-balances across all
nodes
in
cluster.
On Wed, Jul 10, 2013 at 2:52 PM, Floyd Wu floyd...@gmail.com
wrote:
Hi there,
I've built a SolrCloud cluster from example, but I have some
question.
When I send query to one leader (say
http://xxx.xxx.xxx.xxx:8983/solr/collection1) and no problem
everything
will be fine.
When I shutdown that leader, the other replica(
http://xxx.xxx.xxx.xxx:9983/solr/collection1) in the some shard
will
be
new
leader. The problem is:
The application doesn't know new leader's location and still send
request
to http://xxx.xxx.xxx.xxx:8983/solr/collection1 and of course no
response.
How can I know new leader in my application?
Are there any mechanism that application can send request to one
fixed
endpoint no matter who is leader?
For example, application just send to
http://xxx.xxx.xxx.xxx:8983/solr/collection1
even the real leader run on
http://xxx.xxx.xxx.xxx:9983/solr/collection1
Please help on this or give me some key infomation to google it.
Many thanks.
Floyd
--
Anshum Gupta
http://www.anshumgupta.net
