Re: Enabling Auth between Zookeeper Servers
Hey, I also just tried using 3.5.7, but same problem... Best regards Sebastian On 17-Feb-20 11:34 AM, Sebastian Schmitz wrote: Hi Mate, that's what I also tried. I copied it to the /opt/zookeeper-cluster/-folder and got the same exception just with the new path. So, if that config works on your side it might be my environment then!? Maybe it's a problem with the base-image openjdk:11-jre-stretch which I use for the container... I'll try using the openjdk:8u222-jre you're using. Best regards Sebastian On 17-Feb-20 9:19 AM, Szalay-Bekő Máté wrote: Hi Sebastian, It's strange indeed... I also see the owner is root. That should work in docker usually, given that you run the zookeeper process with the root user. Maybe copying it to a different folder? I see that the conf folder has different owner, maybe the java security library doesn't like that? But honestly, I don't have any useful explanation. Good luck! Mate On Sun, Feb 16, 2020, 20:06 Sebastian Schmitz < sebastian.schm...@propellerhead.co.nz> wrote: Hey Mate, now it gets really weird. I get the file not found exception: '.20-02-16 18:27:50,530 [myid:1] - ERROR [main:ServerCnxnFactory@246] - No JAAS configuration section named 'Server' was found in '/opt/zookeeper-cluster/zookeeper/conf/jaas.conf java.lang.SecurityException: java.io.IOException: /opt/zookeeper-cluster/zookeeper/conf/jaas.conf (No such file or directory) at java.base/sun.security.provider.ConfigFile$Spi.(Unknown Source) at java.base/sun.security.provider.ConfigFile.(Unknown Source) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source) at java.base/java.lang.Class.newInstance(Unknown Source) at java.base/javax.security.auth.login.Configuration$2.run(Unknown Source) at java.base/javax.security.auth.login.Configuration$2.run(Unknown Source) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.login.Configuration.getConfiguration(Unknown Source) at org.apache.zookeeper.server.ServerCnxnFactory.configureSaslLogin(ServerCnxnFactory.java:210) at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:646) at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:143) at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:106) at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:64) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:128) at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82) Caused by: java.io.IOException: /opt/zookeeper-cluster/zookeeper/conf/jaas.conf (No such file or directory) at java.base/sun.security.provider.ConfigFile$Spi.ioException(Unknown Source) at java.base/sun.security.provider.ConfigFile$Spi.init(Unknown Source) ... 18 more 2020-02-16 18:27:50,566 [myid:1] - ERROR [main:ZooKeeperServerMain@83] - Unexpected exception, exiting abnormally java.io.IOException: No JAAS configuration section named 'Server' was found in '/opt/zookeeper-cluster/zookeeper/conf/jaas.conf '. at org.apache.zookeeper.server.ServerCnxnFactory.configureSaslLogin(ServerCnxnFactory.java:247) at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:646) at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:143) at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:106) at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:64) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:128) at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82) So I checked the image: root@2eeeb625500c:/opt/zookeeper-cluster/zookeeper/conf# ls -la total 32 drwxr-xr-x 2 1010 1011 4096 Feb 16 18:27 . drwxr-xr-x 12 root root 4096 Feb 16 18:27 .. -rw-r--r-- 1 1010 1011 535 Jan 30 12:18 configuration.xsl -rw-r--r-- 1 root root 600 Feb 13 18:32 jaas.conf -rw-r--r-- 1 root root 101 Feb 11 00:05 java.env -rw-r--r-- 1 1010 1011 2712 Feb 14 05:49 log4j.properties -rw-r--r-- 1 root root 1255 Feb 16 18:27 zoo.cfg -rw-r--r-- 1 1010 1011 922 Feb 14 05:49 zoo_sample.cfg And tried to output the file it states in
Re: Enabling Auth between Zookeeper Servers
Hi Mate, that's what I also tried. I copied it to the /opt/zookeeper-cluster/-folder and got the same exception just with the new path. So, if that config works on your side it might be my environment then!? Maybe it's a problem with the base-image openjdk:11-jre-stretch which I use for the container... I'll try using the openjdk:8u222-jre you're using. Best regards Sebastian On 17-Feb-20 9:19 AM, Szalay-Bekő Máté wrote: Hi Sebastian, It's strange indeed... I also see the owner is root. That should work in docker usually, given that you run the zookeeper process with the root user. Maybe copying it to a different folder? I see that the conf folder has different owner, maybe the java security library doesn't like that? But honestly, I don't have any useful explanation. Good luck! Mate On Sun, Feb 16, 2020, 20:06 Sebastian Schmitz < sebastian.schm...@propellerhead.co.nz> wrote: Hey Mate, now it gets really weird. I get the file not found exception: '.20-02-16 18:27:50,530 [myid:1] - ERROR [main:ServerCnxnFactory@246] - No JAAS configuration section named 'Server' was found in '/opt/zookeeper-cluster/zookeeper/conf/jaas.conf java.lang.SecurityException: java.io.IOException: /opt/zookeeper-cluster/zookeeper/conf/jaas.conf (No such file or directory) at java.base/sun.security.provider.ConfigFile$Spi.(Unknown Source) at java.base/sun.security.provider.ConfigFile.(Unknown Source) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source) at java.base/java.lang.Class.newInstance(Unknown Source) at java.base/javax.security.auth.login.Configuration$2.run(Unknown Source) at java.base/javax.security.auth.login.Configuration$2.run(Unknown Source) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.login.Configuration.getConfiguration(Unknown Source) at org.apache.zookeeper.server.ServerCnxnFactory.configureSaslLogin(ServerCnxnFactory.java:210) at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:646) at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:143) at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:106) at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:64) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:128) at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82) Caused by: java.io.IOException: /opt/zookeeper-cluster/zookeeper/conf/jaas.conf (No such file or directory) at java.base/sun.security.provider.ConfigFile$Spi.ioException(Unknown Source) at java.base/sun.security.provider.ConfigFile$Spi.init(Unknown Source) ... 18 more 2020-02-16 18:27:50,566 [myid:1] - ERROR [main:ZooKeeperServerMain@83] - Unexpected exception, exiting abnormally java.io.IOException: No JAAS configuration section named 'Server' was found in '/opt/zookeeper-cluster/zookeeper/conf/jaas.conf '. at org.apache.zookeeper.server.ServerCnxnFactory.configureSaslLogin(ServerCnxnFactory.java:247) at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:646) at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:143) at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:106) at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:64) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:128) at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82) So I checked the image: root@2eeeb625500c:/opt/zookeeper-cluster/zookeeper/conf# ls -la total 32 drwxr-xr-x 2 1010 1011 4096 Feb 16 18:27 . drwxr-xr-x 12 root root 4096 Feb 16 18:27 .. -rw-r--r-- 1 1010 1011 535 Jan 30 12:18 configuration.xsl -rw-r--r-- 1 root root 600 Feb 13 18:32 jaas.conf -rw-r--r-- 1 root root 101 Feb 11 00:05 java.env -rw-r--r-- 1 1010 1011 2712 Feb 14 05:49 log4j.properties -rw-r--r-- 1 root root 1255 Feb 16 18:27 zoo.cfg -rw-r--r-- 1 1010 1011 922 Feb 14 05:49 zoo_sample.cfg And tried to output the file it states in the error: root@2eeeb625500c:/opt/zookeeper-cluster/zookeeper/conf# cat /opt/zookeeper-cluster/zookeeper/conf/jaas.conf QuorumServer { org.apache.zookeeper.server.auth.DigestLoginModule
Re: Enabling Auth between Zookeeper Servers
Hi Sebastian, It's strange indeed... I also see the owner is root. That should work in docker usually, given that you run the zookeeper process with the root user. Maybe copying it to a different folder? I see that the conf folder has different owner, maybe the java security library doesn't like that? But honestly, I don't have any useful explanation. Good luck! Mate On Sun, Feb 16, 2020, 20:06 Sebastian Schmitz < sebastian.schm...@propellerhead.co.nz> wrote: > Hey Mate, > > now it gets really weird. I get the file not found exception: > > '.20-02-16 18:27:50,530 [myid:1] - ERROR [main:ServerCnxnFactory@246] - > No JAAS configuration section named 'Server' was found in > '/opt/zookeeper-cluster/zookeeper/conf/jaas.conf > java.lang.SecurityException: java.io.IOException: > /opt/zookeeper-cluster/zookeeper/conf/jaas.conf > (No such file or directory) > at > java.base/sun.security.provider.ConfigFile$Spi.(Unknown Source) > at java.base/sun.security.provider.ConfigFile.(Unknown > Source) > at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native > > Method) > at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown > > Source) > at > java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown > > Source) > at java.base/java.lang.reflect.Constructor.newInstance(Unknown > Source) > at java.base/java.lang.Class.newInstance(Unknown Source) > at > java.base/javax.security.auth.login.Configuration$2.run(Unknown Source) > at > java.base/javax.security.auth.login.Configuration$2.run(Unknown Source) > at java.base/java.security.AccessController.doPrivileged(Native > Method) > at > java.base/javax.security.auth.login.Configuration.getConfiguration(Unknown > Source) > at > > org.apache.zookeeper.server.ServerCnxnFactory.configureSaslLogin(ServerCnxnFactory.java:210) > at > > org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:646) > at > > org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:143) > at > > org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:106) > at > > org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:64) > at > > org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:128) > at > > org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82) > Caused by: java.io.IOException: > /opt/zookeeper-cluster/zookeeper/conf/jaas.conf > (No such file or directory) > at > java.base/sun.security.provider.ConfigFile$Spi.ioException(Unknown Source) > at java.base/sun.security.provider.ConfigFile$Spi.init(Unknown > Source) > ... 18 more > 2020-02-16 18:27:50,566 [myid:1] - ERROR [main:ZooKeeperServerMain@83] - > Unexpected exception, exiting abnormally > java.io.IOException: No JAAS configuration section named 'Server' was > found in '/opt/zookeeper-cluster/zookeeper/conf/jaas.conf > '. > at > > org.apache.zookeeper.server.ServerCnxnFactory.configureSaslLogin(ServerCnxnFactory.java:247) > at > > org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:646) > at > > org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:143) > at > > org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:106) > at > > org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:64) > at > > org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:128) > at > > org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82) > > So I checked the image: > > root@2eeeb625500c:/opt/zookeeper-cluster/zookeeper/conf# ls -la > total 32 > drwxr-xr-x 2 1010 1011 4096 Feb 16 18:27 . > drwxr-xr-x 12 root root 4096 Feb 16 18:27 .. > -rw-r--r-- 1 1010 1011 535 Jan 30 12:18 configuration.xsl > -rw-r--r-- 1 root root 600 Feb 13 18:32 jaas.conf > -rw-r--r-- 1 root root 101 Feb 11 00:05 java.env > -rw-r--r-- 1 1010 1011 2712 Feb 14 05:49 log4j.properties > -rw-r--r-- 1 root root 1255 Feb 16 18:27 zoo.cfg > -rw-r--r-- 1 1010 1011 922 Feb 14 05:49 zoo_sample.cfg > > And tried to output the file it states in the error: > > root@2eeeb625500c:/opt/zookeeper-cluster/zookeeper/conf# cat > /opt/zookeeper-cluster/zookeeper/conf/jaas.conf > QuorumServer { > org.apache.zookeeper.server.auth.DigestLoginModule required > user_zookeeper="test"; > }; > QuorumClient { > org.apache.zookeeper.server.auth.DigestLoginModule required > username="zookeeper" > password="test"; > }; > Server { > org.apache.zookeeper.server.auth.DigestLoginModule
Re: Enabling Auth between Zookeeper Servers
Hey Mate, now it gets really weird. I get the file not found exception: '.20-02-16 18:27:50,530 [myid:1] - ERROR [main:ServerCnxnFactory@246] - No JAAS configuration section named 'Server' was found in '/opt/zookeeper-cluster/zookeeper/conf/jaas.conf java.lang.SecurityException: java.io.IOException: /opt/zookeeper-cluster/zookeeper/conf/jaas.conf (No such file or directory) at java.base/sun.security.provider.ConfigFile$Spi.(Unknown Source) at java.base/sun.security.provider.ConfigFile.(Unknown Source) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source) at java.base/java.lang.Class.newInstance(Unknown Source) at java.base/javax.security.auth.login.Configuration$2.run(Unknown Source) at java.base/javax.security.auth.login.Configuration$2.run(Unknown Source) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.login.Configuration.getConfiguration(Unknown Source) at org.apache.zookeeper.server.ServerCnxnFactory.configureSaslLogin(ServerCnxnFactory.java:210) at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:646) at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:143) at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:106) at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:64) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:128) at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82) Caused by: java.io.IOException: /opt/zookeeper-cluster/zookeeper/conf/jaas.conf (No such file or directory) at java.base/sun.security.provider.ConfigFile$Spi.ioException(Unknown Source) at java.base/sun.security.provider.ConfigFile$Spi.init(Unknown Source) ... 18 more 2020-02-16 18:27:50,566 [myid:1] - ERROR [main:ZooKeeperServerMain@83] - Unexpected exception, exiting abnormally java.io.IOException: No JAAS configuration section named 'Server' was found in '/opt/zookeeper-cluster/zookeeper/conf/jaas.conf '. at org.apache.zookeeper.server.ServerCnxnFactory.configureSaslLogin(ServerCnxnFactory.java:247) at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:646) at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:143) at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:106) at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:64) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:128) at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82) So I checked the image: root@2eeeb625500c:/opt/zookeeper-cluster/zookeeper/conf# ls -la total 32 drwxr-xr-x 2 1010 1011 4096 Feb 16 18:27 . drwxr-xr-x 12 root root 4096 Feb 16 18:27 .. -rw-r--r-- 1 1010 1011 535 Jan 30 12:18 configuration.xsl -rw-r--r-- 1 root root 600 Feb 13 18:32 jaas.conf -rw-r--r-- 1 root root 101 Feb 11 00:05 java.env -rw-r--r-- 1 1010 1011 2712 Feb 14 05:49 log4j.properties -rw-r--r-- 1 root root 1255 Feb 16 18:27 zoo.cfg -rw-r--r-- 1 1010 1011 922 Feb 14 05:49 zoo_sample.cfg And tried to output the file it states in the error: root@2eeeb625500c:/opt/zookeeper-cluster/zookeeper/conf# cat /opt/zookeeper-cluster/zookeeper/conf/jaas.conf QuorumServer { org.apache.zookeeper.server.auth.DigestLoginModule required user_zookeeper="test"; }; QuorumClient { org.apache.zookeeper.server.auth.DigestLoginModule required username="zookeeper" password="test"; }; Server { org.apache.zookeeper.server.auth.DigestLoginModule required user_zookeeper="test"; }; Client { org.apache.zookeeper.server.auth.DigestLoginModule required username="zookeeper" password="test"; }; The weird part now is that the access is set exactly the same as the zoo.cfg which it can read without problems. Also changing the access to 666 doesn't change anything. And using your config doesn't help either: jaas.conf: QuorumServer { org.apache.zookeeper.server.auth.DigestLoginModule required user_zookeeper="test"; }; QuorumLearner { org.apache.zookeeper.server.auth.DigestLoginModule required username="zookeeper" password="test"; }; Server {
Re: question on ZAB protocol
Sorry for misunderstood. I think the client could not receive the 'error' message or even if it receive any ack, the ack should be 'undeterminated' 发自我的 iPhone > 在 2020年2月16日,10:35,jonefeewang 写道: > > Norbert Kalmar-2 wrote >> Hi, >> >> A would not have confirmed in this case to the client the write. Sending >> ACK means the followers have written the transaction to disc. Leader (in >> this case A) still needs to send COMMIT message to the followers. >> It goes like this: >> - LEADER(A) receives a write, so it creates a transaction and send it to >> all FOLLOWERs. >> - FOLLOWERs receive the transaction and writes it to disc (txnlog). It >> does >> NOT apply to the datatree. >> - After writing to disc FOLLOWERs send ACK to LEADER(A) (Nothing at this >> point is acknowledged to the client) >> - After LEADER(A) receives quorum of ACK, then, and only then will it >> apply >> to the datatree and send COMMIT message to all FOLLOWERs to do the same. >> And also ACK to client that the write is complete. And at this point the >> data sent by the client is saved in the txnlogs of the quorum. >> >> Hope this helps, >> >> Regards, >> Norbert >> >> On Sat, Feb 15, 2020 at 5:20 AM > >> hnwyllmm@ > >> wrote: >> >>> How do you know A has sent the ack to client before he die ? >>> >>> 发自我的 iPhone >>> 在 2020年2月15日,09:15,jonefeewang > >> jonefeewang@ > >> 写道: I also have the same question like this below: let's say we have nodes A B C D E, now A is the leader A broadcasts <1,1>, it reaches B, then A, B die, C D E elect someone, the new system is going to throw away <1,1> since it does not know its existence, right? start from scratch, A broadcasts<1,1> , it reaches all, all send ACK to A, but A dies before receiving the ACK, then BCDE elects someone, and the new leader sees <1,1> in log, so it broadcasts <1,1> to BCDE, which all commit it. now if we look back, when A dies, the client should get a "write failure", but now after BCDE relection, the written value does get into the system ??? the client and the cluster has an inconsistent view >>> ?? -- Sent from: http://zookeeper-user.578899.n2.nabble.com/ >>> >>> > > > Sorry, I think I need to make the question more clear : > > 1. A broadcasts<1,1> , it reaches all, all send ACK to A > 2. A dies before receiving the ACK, > 3. BCDE elects someone, and the new leader sees <1,1> in log, so it > broadcasts <1,1> to BCDE, which all commit it. > > now if we look back, when A dies, the client should get a "write > failure", but now after BCDE relection, the written value does get into the > system 。 > > so in the last, the client got a write error(probably think this write did > not succeed), but the server clusters did write this value in their log and > datatree. > > so the client and the cluster has an inconsistent view. > > > > > -- > Sent from: http://zookeeper-user.578899.n2.nabble.com/