Re: Backup and restore Solr 8.11.2 collections and configsets in Zookeeper version: 3.7.0

2022-09-22 Thread Kaushal Shriyan
On Thu, Sep 22, 2022 at 4:06 PM Szalay-Bekő Máté 
wrote:

> Hello Kaushal!
>
> One of my Solr colleagues just mentioned a possible solution. What about
> using the solr admin script? With recursive option, like "solr zk -r"
>
> e.g.
>
> bin/solr zk cp -r file:/apache/confgs/whatever/conf zk:/configs/myconf -z
> 111.222.333.444:2181
>
>
> https://solr.apache.org/guide/8_11/solr-control-script-reference.html#copy-between-local-files-and-zookeeper-znodes
>
> Isn't this what you are looking for?
>
> Best regards,
> Máté
>

Thanks Mate and appreciate it. I will try it out and keep you posted.
Thanks in advance.

Best Regards,

Kaushal


Re: Zookeeper.sync() behavior

2022-09-22 Thread Enrico Olivelli
Keith

Il Gio 22 Set 2022, 15:03 Keith Turner  ha scritto:

> I have read some documentation about Zookeeper.sync() that makes me
> think there is no need to wait on its callback before reading data,
> but its not explicitly stated in what I have read. I am wondering if
> the following code is a reasonable way to get the latest data for
> '/a/b'?
>
>   ZooKeeper zooKeeper = null;
>
>   AsyncCallback.VoidCallback cb;
>   CompletableFuture returnCode = new CompletableFuture<>();
>   zooKeeper.sync("/a/b", (rc, path, ctx) -> returnCode.complete(rc), null);
>   zooKeeper.getData("/a/b",false, null);
>   if(returnCode.get() != 0) {
> //handle problem with sync
>   }
>
> Thanks
>

I think that generally you have to wait for 'sync' to complete
successfully, then you can call getData()

Otherwise  for instance if the error is ConnectionLoss you haven't really
sent the sync command to the server

I hope that helps

Enrico


> Keith
>


Zookeeper.sync() behavior

2022-09-22 Thread Keith Turner
I have read some documentation about Zookeeper.sync() that makes me
think there is no need to wait on its callback before reading data,
but its not explicitly stated in what I have read. I am wondering if
the following code is a reasonable way to get the latest data for
'/a/b'?

  ZooKeeper zooKeeper = null;

  AsyncCallback.VoidCallback cb;
  CompletableFuture returnCode = new CompletableFuture<>();
  zooKeeper.sync("/a/b", (rc, path, ctx) -> returnCode.complete(rc), null);
  zooKeeper.getData("/a/b",false, null);
  if(returnCode.get() != 0) {
//handle problem with sync
  }

Thanks

Keith


Re: Backup and restore Solr 8.11.2 collections and configsets in Zookeeper version: 3.7.0

2022-09-22 Thread Szalay-Bekő Máté
Hello Kaushal!

One of my Solr colleagues just mentioned a possible solution. What about
using the solr admin script? With recursive option, like "solr zk -r"

e.g.

bin/solr zk cp -r file:/apache/confgs/whatever/conf zk:/configs/myconf -z
111.222.333.444:2181

https://solr.apache.org/guide/8_11/solr-control-script-reference.html#copy-between-local-files-and-zookeeper-znodes

Isn't this what you are looking for?

Best regards,
Máté

On Fri, Sep 16, 2022 at 8:23 PM Shawn Heisey 
wrote:

> On 9/16/22 09:37, Szalay-Bekő Máté wrote:
> > But actually much better would be to do the backup and restore on Solr
> > level.
>
> Solr doesn't currently have this capability.  We do have functionality
> that can download index configs from ZK to the filesystem, but not all
> the cluster contents in ZK.
>
> If the ZK is dedicated to Solr, I believe you can copy the entire
> "version-2" directory from the ZK datadir and install it in new ZK nodes
> while they are down, then start them up.
>
> Thanks,
> Shawn
>
>


Re: Zookeeper leader election for client read and write requests

2022-09-22 Thread Szalay-Bekő Máté
Hello Kaushal,

>  1. What is the algorithm used to elect the new leader between the
remaining 2 followers?

There is a very high-level description of our internal ZooKeeper leader
election algorithm here:
https://zookeeper.apache.org/doc/current/zookeeperInternals.html#sc_leaderElection
I don't know if we have more detailed documentation. If you are interested
in the code, best to start here:
https://github.com/apache/zookeeper/blob/master/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/FastLeaderElection.java
Also we have many unit tests around leader election that can help to
understand the behaviour.


> 2. During the leader elections process in place, does the client see
a 503 service unavailable for all read or write requests?

 "503 service unavailable" is an HTTP error code, and on the ZooKeeper
Client interface we don't use HTTP but we use a (jute based) binary
protocol. In ZooKeeper, we have client sessions which can be kept alive for
some time even if they can not communicate with the server. E.g. if you set
client session timeout to 30 sec and there is a leader election in
ZooKeeper server that takes e.g. 10 seconds, then (as far as I remember)
the ZooKeeper client library should keep the session open so this should
not be visible for the applications using ZooKeeper. Of course no change
can be submitted (or no new session can be created) while the quorum has no
active leader, so I assume these operations will be blocked until the
internal leader election finishes in ZooKeeper. So one can expect longer
response time temporarily in case of a leader election.


>3. In an ensemble of 3 nodes with 1 leader and 2 followers. Is there a
way to see which node is serving read operations and which node is serving
write operations?

In ZooKeeper, the current leader is responsible to do all the modification
on the data, and all the changes made by the leader are synchronized to all
followers. The four-letter-word diagnostic interface (
https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_4lw) or the
HTTP admin API (
https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_adminserver)
can be used to find the current leader in the cluster. However, in
ZooKeeper the clients can be connected to any ZooKeeper Server in the
quorum (unless leaderServes config is explicitly disabled), and normally
all servers will accept both read and write operations. A client session is
handled by a server and if we send a write request, then this server will
make sure to play it through the current leader before sending back the
answer to the client. The client doesn't need to know who is the current
leader, it can communicate to any server. Usually we list all the ZooKeeper
servers when we initiate a new client session, so the client library can
fail-over and loadbalance.

In general, you might find useful to read our documentation:
https://zookeeper.apache.org/doc/current/zookeeperOver.html


Kind regards,
Máté


On Sat, Sep 17, 2022 at 6:27 PM Steph van Schalkwyk 
wrote:

> Just google leader election site:zookeeper.apache.org
>
>
> On Fri, Sep 16, 2022 at 7:39 PM Kaushal Shriyan 
> wrote:
>
> > Hi,
> >
> > I am running Zookeeper version: 3.7.0 ( 3 nodes -> 1 Leader and 2
> > Followers) on CentOS Linux release 7.9.2009 (Core). In an ensemble of 3
> > nodes with 1 leader and 2 followers, if the leader goes down then two
> > servers can elect a leader among themselves. I have the below questions.
> >
> >1. What is the algorithm used to elect the new leader between the
> >remaining 2 followers?
> >2. During the leader elections process in place, does the client see a
> >503 service unavailable for all read or write requests?
> >3. In an ensemble of 3 nodes with 1 leader and 2 followers. Is there a
> >way to see which node is serving read operations and which node is
> > serving
> >write operations?
> >
> > Please guide me. Any help will be highly appreciable. Thanks in advance.
> >
> > Best Regards,
> >
> > Kaushal
> >
>