Re: Confusion around client ports and dynamic reconfiguration

[Chris T.]

We run secure client port at 2182 and standard port 2181. This 2181 is in 
the dynamic config strings. Then we have a firewalld rule to block incoming 
traffic on 2181.
For us this works, no problem with Curator Ensemble tracker either if I 
recall. Our Curator based clients connect fine on the secure port and we 
run frequent dynamic reconfiguration.


Regards
Chris

On 18 November 2022 20:15:39 "Ruel, Ryan"  wrote:


Seems others have faced this same problem:
https://issues.apache.org/jira/browse/ZOOKEEPER-3577

/Ryan

On 11/18/22, 1:26 PM, "Ruel, Ryan"  wrote:

   Enrico,

   Sure, I can create a PR for any documentation change suggestions.

   I am still having some trouble with this, though.

   Using the dynamic reconfiguration command within the CLI tool, I was able 
   to specify a client port of 2281 in the server specification (the same port 
   as what I have in secureClientPort in zookeeper.conf). This was accepted by 
   the quorum and was working just fine.


   However, after installing a new ZooKeeper node and initially configuring 
   it, it's unable to bind to 2281.


   I'm running the most recent stable release (3.7.3).

   My configuration is as follows:

   zookeeper.conf (I have removed the irrelevant bits for brevity):
   secureClientPort=2281
   serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory
   ssl.quorum.keyStore.location=/etc/zookeeper/keys/quorum/keyStore.jks
   sslQuorum=true
   X509AuthenticationProvider.superUser=SUPERUSER
   ssl.quorum.trustStore.location=/etc/zookeeper/keys/quorum/trustStore.jks
   authProvider.=
   reconfigEnabled=true
   ssl.trustStore.password=
   dynamicConfigFile=/etc/zookeeper/zookeeper.conf.dynamic.3b0019

   zookeeper.conf.dynamic.3b0019:
   server.1=100.80.2.1:2888:3888:participant;0.0.0.0:2281
   server.2=100.80.2.2:2888:3888:participant;0.0.0.0:2281
   server.3=100.80.2.3:2888:3888:participant;0.0.0.0:2281

   logs:
   2022-11-18 18:07:26,990 [main] INFO  server.NettyServerCnxnFactory - bound 
   to port 2281
   2022-11-18 18:07:26,990 [main] INFO  server.NettyServerCnxnFactory - 
   binding to port 0.0.0.0/0.0.0.0:2281
   2022-11-18 18:07:26,995 [main] ERROR quorum.QuorumPeerMain - Unexpected 
   exception, exiting abnormally

   java.net.BindException: Address already in use
   at java.base/sun.nio.ch.Net.bind0(Native Method)
   at java.base/sun.nio.ch.Net.bind(Net.java:459)
   at java.base/sun.nio.ch.Net.bind(Net.java:448)
   at 
   java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
   at 
   io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:141)
   at 
   io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:562)
   at 
   io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1334)
   at 
   io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:506)
   at 
   io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:491)
   at 
   io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:973)

   at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:260)
   at 
io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:356)
   at 
   io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
   at 
   io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)

   at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503)
   at 
   io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)

   at 
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
   at 
   io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)

   at java.base/java.lang.Thread.run(Thread.java:829)

   So, it seems like it's trying to bind twice to port 2281 using Netty.

   If I remove "secureClientPort" from zookeeper.conf and restart, then 
   ZooKeeper starts up, but as you can see in the following logs connections 
   exception out (presumably because TLS is disabled):
   2022-11-18 18:23:57,851 [nioEventLoopGroup-4-1] WARN  
   server.NettyServerCnxn - Closing connection to /100.80.2.5:36609

   java.io.IOException: Len error 369296129
   at 
   org.apache.zookeeper.server.NettyServerCnxn.receiveMessage(NettyServerCnxn.java:521)
   at 
   org.apache.zookeeper.server.NettyServerCnxn.processMessage(NettyServerCnxn.java:374)
   at 
   org.apache.zookeeper.server.NettyServerCnxnFactory$CnxnChannelHandler.channelRead(NettyServerCnxnFactory.java:357)
   at 
   

Re: A question about the client side exception

[Chris T.]

Hi Yong,

When you run an operation on a node you will mostly get
ConnectionLossException, which you can retry. When the client connects to
ZK again after session expiration time is exceeded and it gets a session
expired notice, your node operation will also get a
SessionExpiredException. The client will then need to setup a new zookeeper
object + connection. If you're using an existing framework like Curator
this will be done for you.

If you're not using an existing framework and d.i.y. the session management
and you are going to use ephemeral nodes or node watchers then you need to
pay special attention to session expiration. If if so, let me know and I
can explain more.

Regards,

Chris






On Mon, Sep 5, 2022 at 9:29 AM Yong Zhang 
wrote:

> Hi guys,
>
> I saw client has three exceptions: AUTH_FAILED, SESSION_EXPIRED,
> and CONNECTION_LOSS. The question is about connection loss.
> Connection loss has various reason to cause. One scenario we saw is the
> client received the connection loss but the data updated successfully on
> the
> server, then we have a mismatch between the memory and zk server.  So
> our plan is to catch this error and retry it until an explicit error throws
> out. I
> want to know, if there has other exceptions we need to take care?
>
> And I read the code and find there isn’t a timeout exception and looks like
> most exceptions transform to the connection loss exception. Do I understand
> right?
>
> Look forward to any answer!
>
> Thank you
> Yong
>

Re: Can the leader of a Zookeeper be specifically selected at startup?

[Chris T.]

Hi George,
As a Zookeeper user or developer you should not have to care about which ZK 
is the leader.  What would be the use case of that?
Leader election is automatic and the clients can connect to any of the zk 
cluster members regardless of role.

Regards,
Chris


On 19 June 2022 17:01:15 "Heller, George A III CTR (USA)" 
 wrote:
We have 3 Zookeeper nodes and would like node 2 to always be the leader 
unless node 2 goes down. IF node 2 goes down, then either node 1 or node 3 
would be the leader.


Can this be done? If so, how would this be done?

Re: Info about Zookeeper 3.8.0

[Chris T.]

If you want to get rid of log4j1 in your existing Zookeeper installations, 
you could also take the following approach :


Reload4j is the direct continuation / replacement of log4j1.

Stop zookeeper.

From the zookeeper/lib directory, remove log4j and slf4j-log4j.

Download reload4j and slf4j-reload4j and drop them in /lib
Start zookeeper.
Done.

Regards
Chris


On 3 May 2022 19:21:29 Enrico Olivelli  wrote:


Gello

Il Mar 3 Mag 2022, 17:54 dilip anand  ha scritto:


Hi,



Zookeeper 3.8.0 is using what version of log4j and is this "3.8.0" a stable
release?



We are using logback in 3.8.0.
Zookeeper 3.8.0 is released as production ready.

If you are going to upgrade an existing cluster it is always a good
practice to test the upgrade in some staging environment with a
configuration similar to your production  environment

Best regards
Enrico






Regards,

Dilip

--
anand

Re: Bootstrapping a new cluster and using the reconfig feature

[Chris T.]

If you want to run a zookeeper cluster you have to start with at least 2 
members. From there you can scale up with the dynamic reconfig commands.

Regards
Chris

On 30 December 2021 16:40:40 Eric Edgar 
 wrote:



I am experimenting with zk and the reconfig feature and trying to
understand if I can start a single zk node and then reconfig/bootstrap the
other 2 nodes into the ensemble.  The reconfig command is throwing an error
that there isn't a quorum yet.  Is this line of thinking possible?  or do I
need to setup the first 3 nodes manually the first time?
I am basing this experiment off of this web page.
https://blog.container-solutions.com/dynamic-zookeeper-cluster-with-docker

/opt/zookeeper/zookeeper/bin/zkCli.sh -server 10.1.1.104:2181 reconfig -add
"server.2=10.1.1.40:2888:3888:participant;2181"
No quorum of new config is connected and up-to-date with the leader of last
commmitted config - try invoking reconfiguration after new servers are
connected and synced

/opt/zookeeper/zookeeper/bin/zkCli.sh -server 10.1.1.104:2181 config
server.1=10.1.1.104:2888:3888:participant;0.0.0.0:2181

cat ./zoo.cfg
autopurge.purgeInterval=1
initLimit=10
syncLimit=5
autopurge.snapRetainCount=6
tickTime=2000
dataDir=/mnt/zookeeper/data
reconfigEnabled=true
standaloneEnabled=false
dynamicConfigFile=/opt/zookeeper/zookeeper/conf/zoo.cfg.dynamic.16

What is the best solution for an unattended bootstrap setup of a new
cluster from scratch?


This was something that we were able to accomplish with exhibitor on older
versions of zookeeper in the past.

Re: zookeeper digest authentication

[Chris T.]

Hi,
I think you are referring to
zookeeper.allowSaslFailedClients
This is casually mentioned in the link you provided but not explained as a 
standalone option.

Regards
Chris



On 15 December 2021 08:14:19 Szalay-Bekő Máté  
wrote:



Hello Andrzej,

In ZooKeeper, the authentication is not enforced by default, meaning that
even if you fail to authenticate (or don't even provide any credentials)
you can still connect to ZooKeeper, but your session won't have any user
attached to it. So you will be able to see/modify only the ZNodes that are
granting permission to the "world" user. There are several server side
options to change this behaviour. I think you are looking for the
"enforce.auth.enabled=true" option, see here:
https://zookeeper.apache.org/doc/r3.7.0/zookeeperAdmin.html#sc_authOptions

(I remember there is some other option, which will disable the "fallback to
world user" behaviour (so terminating the session if you connect with wrong
credentials, but still let you connect without providing any credentials).
I remember seeing this in the code, but don't see it in the documentation.
If you would need this one, I can dig deeper.

Kind regards,
Máté

On Tue, Dec 14, 2021 at 2:20 PM Andrzej Trzeciak <
andrzej.trzec...@exelaonline.com> wrote:


Hi,

I’m having trouble implementing the simplest zookeeper (v 3.7.0)
authentication using just username and password and the ‘digest’ mechanism.

I tried various config properties, but none of them worked.

The problem is, that when I connect giving the wrong credentials I am
still being successfully authenticated instead of being rejected.

My setup below (including oprions I have tried, but didn’t work, so I
commented them:

*Zoo.cfg:*


#SASL




#authProvider.sasl=org.apache.zookeeper.server.auth.SASLAuthenticationProvider

#authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider

#requireClientAuthScheme=sasl

#sessionRequireClientSASLAuth=true

#set.acl=true

DigestAuthenticationProvider.enabled=true

enforce.auth.enabled=true

enforce.auth.schemes=digest

#SASL
END--



*Jaas_config:*

Server {

   org.apache.zookeeper.server.auth.DigestLoginModule required

   user_super="adminsecret"

   user_bob="bobsecret";

};

*Client code:*

CuratorFrameworkFactory.Builder builder = CuratorFrameworkFactory.builder()

.connectString(connectUris(zookeeper, "zookeeper:2181"))

.connectionStateErrorPolicy(connectionStateErrorPolicy)

.retryPolicy(retryPolicy)

.aclProvider(aclProvider)

.connectionTimeoutMs(1)

.sessionTimeoutMs(sessionTimeout);

if(zookeeperAuthEnabled){

builder.authorization("digest",
"kuku:adminsecret4".getBytes());

}

curatorClient = builder.build();

curatorClient.getConnectionStateListenable().addListener((c, s) ->
{

connectionState = s;

log.info(MessageFormat.format("CuratorState
[State={0},Connected={1}]", s.name(), s.isConnected()));

});

curatorClient.start();

try {

curatorClient.blockUntilConnected();

leaderLatch = initLeadership();

} catch (InterruptedException e) {

log.info(e);

}



As a result, when the application starts I a successful authentication and
a message in zookeeper console:

2021-12-14 14:08:45,854 [myid:] - INFO
[NIOWorkerThread-13:ZooKeeperServer@1623] - got auth packet /
192.168.43.169:49753

2021-12-14 14:08:45,854 [myid:] - INFO
[NIOWorkerThread-13:ZooKeeperServer@1642] - Session 0x1004d2f28d1:
auth success for scheme digest and address /192.168.43.169:49753





*Andrzej Trzeciak*
Senior System Engineer
[image: Exela Technologies]


Grudziądzka 46-48 • 87-100 Toruń • Poland

Tel. +48 573 251 507
exelatech.com

  •  About EXELA

  •  Instagram

Re: Zookeeper 3.5.5 RHEL Operating System Architecture Question

[Chris T.]

Hi Raymond,
A zookeeper distribution consists of a java program and some bash scripts. 
It's independent of cpu architecture.

Regards
Chris

On 2 December 2021 14:56:38 "Reed, Raymond J (CGI Federal)" 
 wrote:



Hello,

I am working with Zookeeper version 3.5.5, and I noticed that the 
administrator guide mentions that Linux/GNU is supported for client, 
server, etc, but not which versions of linux? It would seem like RHEL 7 and 
8 would be supported, but I am no t sure which architectures (x86-64, 
p-series,etc)?


Thank you for your time and patience.


Sincerely,
Raymond Reed

Proprietary/confidential information belonging to CGI Federal Inc. or its 
affiliates may be contained in this message. If you are not a recipient 
indicated or intended in this message (or responsible for the delivery of 
this message to such person), or if you think for any reason that this 
message may have been addressed to you in error, you may not use or copy or 
deliver this message to anyone else. In such case, you should destroy this 
message and are asked to notify the sender by reply email.

Re: Zookeeper cluster won't cluster

[Chris T.]

You need the following on each server:
The '. 1' refers to the myid number.



server.1=ip-10-0-0-65.ec2.internal:2888:3888
server.2=ip-10-0-1-234.ec2.internal:2888:3888
server.3=ip-10-0-2-207.ec2.internal:2888:3888


Regards
Chris

On 4 November 2021 17:39:34 Ken Tune  wrote:


Hi

I have 3 zookeeper nodes. Ports 2181, 2888 and 3888 are open between them.

My zoo.cfg has respectively

On host 1 ( with /var/lib/zookeeper/myid = 1)

server.1=ip-10-0-0-65.ec2.internal:2888:3888
server.1=ip-10-0-1-234.ec2.internal:2888:3888
server.1=ip-10-0-2-207.ec2.internal:2888:3888

On host 2 ( with /var/lib/zookeeper/myid = 2)

server.2=ip-10-0-0-65.ec2.internal:2888:3888
server.2=ip-10-0-1-234.ec2.internal:2888:3888
server.2=ip-10-0-2-207.ec2.internal:2888:3888

On host 3 ( with /var/lib/zookeeper/myid = 3)

server.3=ip-10-0-0-65.ec2.internal:2888:3888
server.3=ip-10-0-1-234.ec2.internal:2888:3888
server.3=ip-10-0-2-207.ec2.internal:2888:3888

Yet if I run the srvr command on any host I see

Zookeeper version: 3.7.0-e3704b390a6697bfdf4b0bef79e3da7a4f6bac4b, built on 
2021-03-17 09:46 UTC

Latency min/avg/max: 0/0.0858/11
Received: 2148
Sent: 2147
Connections: 2
Outstanding: 0
Zxid: 0x46
Mode: standalone
Node count: 28

And in the log file I see

2021-11-04 12:10:00,901 [myid:3] - WARN  [main:QuorumPeerMain@139] - Either 
no config or no quorum defined in config, running in standalone mode


What am I doing wrong? Note I am using 
https://github.com/sleighzy/ansible-zookeeper 
 and supplying the 
zookeeper_id to each of the hosts which results in the zoo.cfg / myid 
config described.


I have tried altering (for host 1) the zoo.cfg to

server.1=ip-10-0-0-65.ec2.internal:2888:3888
server.2=ip-10-0-1-234.ec2.internal:2888:3888
server.3=ip-10-0-2-207.ec2.internal:2888:3888

But this results in

2021-11-04 16:28:05,324 [myid:1] - WARN  
[NIOWorkerThread-4:NIOServerCnxn@380] - Close of session 0x0

java.io.IOException: ZooKeeperServer not running
at org.apache.zookeeper.server.NIOServerCnxn.readLength(NIOServerCnxn.java:554)
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:339)

What do I do to do to get the cluster to cluster.

With thanks

Re: Questions about network segmentation problems

[Chris T.]

1: It will close the client port and will remain unavailable for clients
until it can form or join a quorum (majority).
2: No, see above.
3+4: They will keep trying to connect to the Zookeeper servers in the
connection string until they find one that works. The exact messages you
get depend on the client application or framework you are using. For
example Apache Curator framework or the internal client implementations of
SOLR or KAFKA all have different behaviour and messages messages. Something
like Connection State Lost, Client Connection timed out, Attempting
reconnect etc...

Regards,

Chris


On Tue, Apr 28, 2020 at 5:47 PM Vincent Ngan  wrote:

> Hi,
>
> I would like to know what will happen to ZooKeeper servers and the clients
> connected to them when a network segmentation occurs.
>
> Supposing a network segmentation happens. One of the ZK servers
> looses contact with all the other ZK servers. This ZK server is still but
> it should know that it is not among the majority of a quorum. Then,
>
>1. What will happen to this isolated ZK server?
>2. Will it still function and serve client requests?
>3. If there are clients also located in the same isolated segment and
>are currently connected to this ZK server, what will happen to these
>clients?
>4. What errors code and messages will these clients detect?
>
> Best regards,
> VN
>

Re: [curator][lock-recipe] implementing locks with just ephemeral nodes

[Chris T.]

I have a type of locking mechanism using ephemerals via Curator
framework, they are created with with creatingParentsIfNeeded. The
whole tree disappears when the last ephemeral leaf node is removed.
Parent nodes have an IP address as value.


Code snip from our class:

CreateModable> localCreateMethod
= createMethod.get();
if (localCreateMethod == null) {
CreateBuilderMain createBuilder = cfClient.create();
CreateModable>
tempCreateMethod = createBuilder.creatingParentContainersIfNeeded();
createMethod.compareAndSet(null, tempCreateMethod);
localCreateMethod = createMethod.get();
}

try {
localCreateMethod.withMode(CreateMode.EPHEMERAL).inBackground(backgroundCallback).forPath(basePath,
getEncryptedBytes(commandNode.getNodeBytes()));
} catch (Exception e) {
log.error("Exception: ", e);
}



[zk: localhost:2181(CONNECTED) 1] get /GROUP_V7POC03/ephemerals/inquirylock
10.194.54.93
[zk: localhost:2181(CONNECTED) 2] stat /GROUP_V7POC03/ephemerals/inquirylock
cZxid = 0xf20001914e
ctime = Tue Feb 04 11:11:45 CET 2020
mZxid = 0xf200019169
mtime = Tue Feb 04 11:11:48 CET 2020
pZxid = 0xf20001918b
cversion = 2
dataVersion = 3
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 12
numChildren = 2
[zk: localhost:2181(CONNECTED) 3] get /GROUP_V7POC03/ephemerals/inquirylock
10.194.54.93
[zk: localhost:2181(CONNECTED) 4] get /GROUP_V7POC03/ephemerals

[zk: localhost:2181(CONNECTED) 5] stat
/GROUP_V7POC03/ephemerals/inquirylock/lrv1705f.europe.intranet:v7poc03
cZxid = 0xf200019189
ctime = Tue Feb 04 11:11:51 CET 2020
mZxid = 0xf200019189
mtime = Tue Feb 04 11:11:51 CET 2020
pZxid = 0xf200019189
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x10022c4e19c000b
dataLength = 83
numChildren = 0



On 4 February 2020 15:58:39 Jordan Zimmerman 
wrote:

>> can I implement a lock recipe which just
>> tries to create ephemeral node without any persistent nodes.
>
> Ephemeral nodes cannot have any children. Unless you create your lock at
> the root ("/") you'd need some parent persistent node. Is this a big
> problem for you? Unless you have 1000s of unique lock paths there shouldn't
> be any problem with persistent parent node.
>
> -Jordan

Re: 'zkServer.sh status' connects only to local node ?

[Chris T.]

Hi Prashant,

correct it only checks the local node. We built a script that parses the
dynamic config file and goes over all the nodes to report their status. You
can do this easily from shell  by enabling adminserver functionality and
issuing curl http://yourzookeepernode.com:8081/commands/stats  and grabbing
the server status from the output. Added bonus is that a call to 'stats' is
much faster than spinning up a jvm for each check (which zkServer.sh will
do) and will report a zookeeper that is running but attempting to connect
to it's peers (no quorum yet) as 'not currently serving connections'
whereas zkserver.sh status simply states it could not connect without
telling you the program is in fact running but not yet open for business.



Regards,

Chris

On Fri, Dec 13, 2019 at 8:19 AM prashantkumar dhotre <
prashantkumardho...@gmail.com> wrote:

> Hi,
> Could you please help me with a basic query regarding 'zkServer.sh status'
> ?
> From zkServer.sh, it appears that this cmd tries to connect to server at
> local node and get info.
> If the local server is down, does it connect to next available server (as
> specified in config file )  and returns status ?
> Or is this cmd intended to only report status of local server and not
> status of zookeeper quorum ?
> My use case is, my client app needs to connect to zookeeper and create
> znode.
> connect routine ( which does zookeeper_init2) has retries and in case of
> connect failure, it sleeps and retries for number of times.
> But if zookeeper quorum is down (zookeeper not running ) then I don't want
> to even attempt connection ( hence avoid wait delay in connect api)
> Regards,
> Prashant
>

Re: Disable non-secured client port

[Chris T.]

Great, thanks.

On Thu, Dec 12, 2019 at 3:53 PM Szalay-Bekő Máté 
wrote:

> Hi Chris,
>
> The short answer is yes, all you need to do is don't specify the clientPort
> in the zoo.cfg.
>
> An example for the zoo.cfg in this case:
>
> tickTime=3000
> initLimit=10
> syncLimit=5
> dataDir=/data/zkdata
>
> secureClientPort=2191
> serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory
> ssl.keyStore.location=/etc/zookeeper/ssl/server.jks
> ssl.keyStore.password=password
> ssl.trustStore.location=/etc/zookeeper/ssl/servertrust.jks
> ssl.trustStore.password=password
>
> These are the main options you can choose from, when configuring client
> SSL:
>
> - option 1: only unsecure (this is when you specify only the clientPort in
> the zoo.cfg)
> - option 2: only SSL (I think this is what you want... this happens if you
> don't specify clientPort in the zoo.cfg, but you specify the
> secureClientPort. Of course in this case you also have to set a few other
> parameters, like setting the serverCnxnFactory to netty and defining the
> ssl.keyStore / ssl.trustStore locations and passwords - see the example
> above)
> - option 3: both secure and unsecure client connections on different server
> ports (this happens when you set both the clientPort and
> the secureClientPort in the zoo.cfg to different ports)
> - option 4: both secure and unsecure client connections using the same
> server port (this happens when you set the 'client.portUnification=true' in
> the config file and set the clientPort in the zoo.cfg. In this case the
> clientPort will be able to serve both unsecure and SSL connections. Of
> course, in this case you still have to set the other SSL related
> parameters, like the serverCnxnFactory and the ssl.keyStore /
> ssl.trustStore locations and passwords)
>
> AFAIK, the option 4 is not available yet in any released 3.5 version. It
> will be part of the first 3.6 release candidate soon. Also there is a
> backport PR for it just under review (
> https://github.com/apache/zookeeper/pull/1174) so it's likely to be in the
> next 3.5.7 release, if / whenever that will happen.
>
> Regards,
> Mate
>
> On Thu, Dec 12, 2019 at 2:31 PM Chris T.  wrote:
>
> > Hi,
> >
> > we're working on securing our clusters. For quorum, once we set
> > sslQuorum=true, there is no more unsecured quorum port.
> > Is such an option available or in the works for the client port as well ?
> > Or is ssl client port always in addition to unsecured client port ?
> >
> > Regards,
> >
> > Chris
> >
>

Disable non-secured client port

[Chris T.]

Hi,

we're working on securing our clusters. For quorum, once we set
sslQuorum=true, there is no more unsecured quorum port.
Is such an option available or in the works for the client port as well ?
Or is ssl client port always in addition to unsecured client port ?

Regards,

Chris

Re: About ZooKeeper Dynamic Reconfiguration

[Chris T.]

Please reply to my private mail address from now.

On Thu, Oct 10, 2019 at 5:01 AM Gao,Wei  wrote:

> Hi Chris,
> I received your codes about zookeeper balancer. It seems that there are a
> few java class files missing. They include:
> nl.ing.profileha.util.EventCreator;
> nl.ing.profileha.util.FailsafeTriggeredException;
> nl.ing.profileha.util.StringUtils;
> nl.ing.profileha.util.Validator;
> nl.ing.profileha.util.shell.SystemCommandExecutorWithTimeout;
> nl.ing.profileha.zoomonitor.LocalConfig;
> nl.ing.profileha.util.httpGetRequester;
> ACLMode.java;
> ZookeeperTreeCache.java;
>
> Would you please send these class file to me?
> Really appreciate for your kindness!
> Thanks
>
>
>
>
> --
> Sent from: http://zookeeper-user.578899.n2.nabble.com/
>

Re: About ZooKeeper Dynamic Reconfiguration

[Chris T.]

I sent it again, please check.

On Wed, Oct 9, 2019 at 6:31 AM Gao,Wei  wrote:

> Hi oo4load,
>   Where did you sent it to? Through this site or directly sent to my email?
> I received your pseudo codes last week just like this shown below:
>
> buildDatacenterAndServerModel(configurationFile) {
>   enum zookeeperRole PARTICIPANT, OBSERVER, NONE, DOWN
>   object datacenter has servers
>   object server has zookeeperRole configuredRole, zookeeperRole activeRole
>   parse(configurationFile) into (datacenter, servers);
> }
> shiftMajority(designatedSurvivorDatacenter) {
>
>
> designatedSurvivorDatacenter.someObserver.dynamicReconfigure(server=PARTICIPANT)
>   otherDatacenter.someParticipant.dynamicReconfigure(server=OBSERVER)
> }
> balanceServerRoles() {
>   if (designatedSurvivorDatacenter.hasMinimumQuorum)
>   someParticipant.dynamicReconfigure(server=OBSERVER)
>   if (quorumSize.aboveSafeLimit)
>   someObserver.dynamicReconfigure(server=PARTICIPANT)
>   //This is a lot more complicated than 2 simple commands, you need an
> algorithm or define several scenarios.
> }
>
>
> main() {
>  buildDatacenterAndServerModel(configurationFile);
>   while (IamLeader) {
> parse(zk.getData(“/zookeeper/config”)) into servers.configuredRole;
> foreach(server) getServerRole(“server:8081/commands/stat”) into
> servers.activeRole;
>
> foreach(server.activeRole=DOWN)  dynamicReconfigure(server=OBSERVER);
>   server.setConfiguredRole(OBSERVER);
>
> if(designatedSurvivorDatacenter != datacenter.hasMajority)
>shiftMajority(designatedSurvivorDatacenter);
>balanceServerRoles();
>   }
> }
>
> If this above is not what you mean, would you please send it again?
> Really appreciate for your kindness!
>
>
>
>
>
> --
> Sent from: http://zookeeper-user.578899.n2.nabble.com/
>

Re: About ZooKeeper Dynamic Reconfiguration

[Chris T.]

I sent it 1 week ago.

On Tue, Oct 8, 2019 at 10:08 AM Gao,Wei  wrote:

> Hi oo4load,
>   If it is convenient to you, I would like to get the actual code from you
> about the zookeeper cluster balancer implementation. My email address is:
> wei@arcserve.com
> Thank you again.
>
>
>
> --
> Sent from: http://zookeeper-user.578899.n2.nabble.com/
>

Re: About ZooKeeper Dynamic Reconfiguration

[Chris T.]

Let me write this from memory. :)

We have the following:

-A running zookeeper cluster with adminserver enabled
-One or more balancer client processes (one per datacenter), of which one
has a master role through some leader election. The master does the work,
the others do nothing.
-In our case, we work with a designated survivor datacenter (has 3
participants and the other non survivor datacenter has 2 participant and 1
observer ) , and the balancer always resides in the designated survivor
datacenter. This is not a requirement, due to above leader election.
-A balancer client configuration file with all predefined Zookeeper servers
(use it for building the client connection string and generating the server
list). Each predefined server under normal condition has a running
Zookeeper in either participant or observer role.

Balancer design:

buildDatacenterAndServerModel(configurationFile) {
  enum zookeeperRole PARTICIPANT, OBSERVER, NONE, DOWN
  object datacenter has servers
  object server has zookeeperRole configuredRole, zookeeperRole activeRole
  parse(configurationFile) into (datacenter, servers);
}

shiftMajority(designatedSurvivorDatacenter) {

designatedSurvivorDatacenter.someObserver.dynamicReconfigure(server=PARTICIPANT)
  otherDatacenter.someParticipant.dynamicReconfigure(server=OBSERVER)
}


balanceServerRoles() {
  if (designatedSurvivorDatacenter.hasMinimumQuorum)
someParticipant.dynamicReconfigure(server=OBSERVER)
  if (quorumSize.aboveSafeLimit)
someObserver.dynamicReconfigure(server=PARTICIPANT)
  //This is a lot more complicated than 2 simple commands, you need an
algorithm or define several scenarios.
}


main() {
 buildDatacenterAndServerModel(configurationFile);
  while (iAmLeader) {
parse(zk.getData(/zookeeper/config)) into servers.configuredRole
foreach(server) getServerRole(server:8081/commands/stat) into
servers.activeRole

foreach (server.activeRole=DOWN) dynamicReconfigure(server=OBSERVER) ;
server.setConfiguredRole(OBSERVER)

if(designatedSurvivorDatacenter != datacenter.hasMajority)
shiftMajority(designatedSurvivorDatacenter)

balanceServerRoles()
  }
}



Hope this helps. If you need more details, I can check the actual code
coming week.

On Fri, Sep 27, 2019 at 5:06 AM Gao,Wei  wrote:

> Hi oo4load,
>   Could you please tell me how to implements this to avoid the problem
> above?
> Thanks
>
>
>
> --
> Sent from: http://zookeeper-user.578899.n2.nabble.com/
>