Hello Mate,
thank you for your reply.
I just deployed it in our Dev-Environment, which has three nodes. But
when updating the first node it has the same problem.
The problem with not using 0.0.0.0 is that the configuration of Docker
only promotes the internal network to the containers and if I add the
internal IP to the config it just won't work as all three docker-hosts
have the same internal network and only port-mappings allow the services
to be reached from outside world. The FQDN doesn't match the internal IP
either, so the only solution for me is to use 0.0.0.0.
I set it to false as it's a rolling update of the three nodes. Will
change them to true on two more rolling restarts then.
That's what I don't understand either. Even adding the "Server" to the
jaas.conf didn't fix that exception.
Here are the Configs and Logs (with the Server-part included):
jaas.conf:
QuorumServer {
org.apache.zookeeper.server.auth.DigestLoginModule required
user_zookeeper="test";
};
QuorumClient {
org.apache.zookeeper.server.auth.DigestLoginModule required
username="zookeeper"
password="test";
};
Server {
org.apache.zookeeper.server.auth.DigestLoginModule required
user_zookeeper="test"
};
Client {
org.apache.zookeeper.server.auth.DigestLoginModule required
username="zookeeper"
password="test";
};
zoo.cfg:
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/mnt/zk_data
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
dataLogDir=/mnt/zk_data_log
autopurge.snapRetainCount=3
autopurge.purgeInterval=24
quorum.auth.enableSasl=true
quorum.auth.learnerRequireSasl=false
quorum.auth.serverRequireSasl=false
quorum.auth.learner.loginContext=QuorumLearner
quorum.auth.server.loginContext=QuorumServer
quorum.cnxn.threads.size=20
authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
secureClientPort=2281
server.1=0.0.0.0:2888:3888
server.2=kafkad02.x.azure.com:2888:3888
server.3=kafkad03.x.azure.com:2888:3888
Server-Log:
Using config: /opt/zookeeper-cluster/zookeeper/bin/../conf/zoo.cfg
Feb 11, 2020 18:43:53 +0000 [1 1] com.newrelic INFO: New Relic Agent:
Loading configuration file "/opt/zookeeper-cluster/newrelic/./newrelic.yml"
Feb 11, 2020 18:43:53 +0000 [1 1] com.newrelic INFO: Using default
collector host: collector.newrelic.com
Feb 11, 2020 18:43:53 +0000 [1 1] com.newrelic INFO: New Relic Agent:
Writing to log file: /opt/zookeeper-cluster/newrelic/logs/newrelic_agent.log
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by
com.newrelic.weave.weavepackage.NewClassAppender
(file:/opt/zookeeper-cluster/newrelic/newrelic.jar) to method
java.net.URLClassLoader.addURL(java.net.URL)
WARNING: Please consider reporting this to the maintainers of
com.newrelic.weave.weavepackage.NewClassAppender
WARNING: Use --illegal-access=warn to enable warnings of further illegal
reflective access operations
WARNING: All illegal access operations will be denied in a future release
2020-02-11 18:43:59,257 [myid:] - INFO [main:QuorumPeerConfig@136] -
Reading configuration from:
/opt/zookeeper-cluster/zookeeper/bin/../conf/zoo.cfg
2020-02-11 18:43:59,477 [myid:] - INFO
[main:QuorumPeer$QuorumServer@185] - Resolved hostname:
kafkad02.x.azure.com to address: kafkad02.x.azure.com/1.2.3.4
2020-02-11 18:43:59,477 [myid:] - INFO
[main:QuorumPeer$QuorumServer@185] - Resolved hostname: 0.0.0.0 to
address: /0.0.0.0
2020-02-11 18:43:59,666 [myid:] - INFO
[main:QuorumPeer$QuorumServer@185] - Resolved hostname:
kafkad03.x.azure.com to address: kafkad03.x.azure.com/1.2.3.5
2020-02-11 18:43:59,666 [myid:] - INFO [main:QuorumPeerConfig@398] -
Defaulting to majority quorums
2020-02-11 18:43:59,677 [myid:1] - INFO [main:DatadirCleanupManager@78]
- autopurge.snapRetainCount set to 3
2020-02-11 18:43:59,677 [myid:1] - INFO [main:DatadirCleanupManager@79]
- autopurge.purgeInterval set to 24
2020-02-11 18:43:59,732 [myid:1] - INFO
[PurgeTask:DatadirCleanupManager$PurgeTask@138] - Purge task started.
2020-02-11 18:43:59,749 [myid:1] - INFO [main:QuorumPeerMain@130] -
Starting quorum peer
2020-02-11 18:43:59,788 [myid:1] - INFO [main:ServerCnxnFactory@117] -
Using org.apache.zookeeper.server.NIOServerCnxnFactory as server
connection factory
2020-02-11 18:43:59,804 [myid:1] - INFO
[PurgeTask:DatadirCleanupManager$PurgeTask@144] - Purge task completed.
'.20-02-11 18:43:59,826 [myid:1] - ERROR [main:ServerCnxnFactory@210] -
No JAAS configuration section named 'Server' was foundin
'/opt/zookeeper-cluster/zookeeper/conf/jaas.conf
2020-02-11 18:43:59,827 [myid:1] - ERROR [main:QuorumPeerMain@92] -
Unexpected exception, exiting abnormally
java.io.IOException: No JAAS configuration section named 'Server' was
foundin '/opt/zookeeper-cluster/zookeeper/conf/jaas.conf
'.
at
org.apache.zookeeper.server.ServerCnxnFactory.configureSaslLogin(ServerCnxnFactory.java:211)
at
org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:82)
at
org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:133)
at
org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:114)
at
org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:81)
Best regards
Sebastian
On 11-Feb-20 9:56 PM, Szalay-Bekő Máté wrote:
Hello Sebastian,
In general I think you did configure ZooKeeper just fine. A few remarks:
- I am not sure how ZooKeeper server-server authentication is expected to
work when you only use a single server. Would you mind trying to start a
Quorum with e.g. 3 servers?
- also, I think it is a good idea to avoid using 0.0.0.0 as hostname,
especially if you would run the cluster on multiple different servers /
docker containers. Try using the fully qualified domain name for a
multi-server setup, or if you just test multiple ZooKeeper servers on the
same machine, then just use 127.0.0.1. (maybe it has no effect in the
current case, but for SSL or for dynamic reconfig it might be bad to use
0.0.0.0. Also I remember problems with rolling restarts when using 0.0.0.0
in the config)
- is there a reason why you set 'quorum.auth.learnerRequireSasl' and
'quorum.auth.serverRequireSasl' to false? Using false is usually good idea
during rolling upgrade, but if you start a new cluster and want to use
server-server authentication, then you can just set them to true.
I don't understand why you got the exception " No JAAS configuration
section named 'Server' was found" ... Setting the loginContext should have
fixed that. If you still see the same issue with the 3 server setup, then
can you please share the config files, the command how you start ZooKeeper
and also the log files with us, so that we can look deeper?
Kind regards,
Mate
On Tue, Feb 11, 2020 at 2:56 AM Sebastian Schmitz <
sebastian.schm...@propellerhead.co.nz> wrote:
Hello,
I'm currently looking into enabling the Auth between Zookeeper-Servers
and found this documentation:
https://cwiki.apache.org/confluence/display/ZOOKEEPER/Server-Server+mutual+authentication
However, when I use the config from the document (for Digest-MD5) I get
this exception in Zookeeper 3.4.14 and also 3.5.6, which I tried because
I thought using latest version could help:
java.io.IOException: No JAAS configuration section named 'Server' was
found in '/opt/zookeeper-cluster/zookeeper/conf/jaas.conf
And of course that's right, because there's only QuorumServer and
QuorumClient in the jaas.conf:
jaas.conf:
QuorumServer {
org.apache.zookeeper.server.auth.DigestLoginModule required
user_zookeeper="test";
};
QuorumClient {
org.apache.zookeeper.server.auth.DigestLoginModule required
username="zookeeper"
password="test";
};
I also tried renaming the QuorumServer to just "Server". No change.
My zoo.cfg:
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/mnt/zk_data
clientPort=2181
dataLogDir=/mnt/zk_data_log
autopurge.snapRetainCount=3
autopurge.purgeInterval=24
quorum.auth.enableSasl=true
quorum.auth.learnerRequireSasl=false
quorum.auth.serverRequireSasl=false
quorum.auth.learner.loginContext=QuorumLearner
quorum.auth.server.loginContext=QuorumServer
quorum.cnxn.threads.size=20
authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
secureClientPort=2281
server.1=0.0.0.0:2888:3888
Any idea what I could try? Or maybe there's some better document on how
to achieve this?
Thank you
Sebastian
--
DISCLAIMER
This email contains information that is confidential and which
may be
legally privileged. If you have received this email in error please
notify the sender immediately and delete the email.
This email is intended
solely for the use of the intended recipient and you may not use or
disclose this email in any way.
--
DISCLAIMER
This email contains information that is confidential and which
may be
legally privileged. If you have received this email in error please
notify the sender immediately and delete the email.
This email is intended
solely for the use of the intended recipient and you may not use or
disclose this email in any way.
