[jira] [Comment Edited] (ZOOKEEPER-4276) Serving only with secureClientPort fails
[ https://issues.apache.org/jira/browse/ZOOKEEPER-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17780132#comment-17780132 ] Abhilash Kishore edited comment on ZOOKEEPER-4276 at 10/27/23 5:10 AM: --- [~andor] Can you please please try this config? {code:java} tickTime=2000 initLimit=10 syncLimit=5 dataDir=./andor-5560-ubuntu:2181/data secureClientPort=2181 clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty ... serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory ... server.1=andor-5560-ubuntu:3181:4181;2181 # or server.1=andor-5560-ubuntu:3181:4181:participant;0.0.0.0:2181 server.2=andor-5560-ubuntu:3182:4182;2181 server.3=andor-5560-ubuntu:3183:4183:participant;0.0.0.0:2181 {code} I believe this issue happens when `secureClientPort` is declared in zoo.cfg in addition to client port in the server entry (e.g., 2181 in "server.1=andor-5560-ubuntu:3181:4181;2181") [ZooKeeper reconfig doc|https://zookeeper.apache.org/doc/current/zookeeperReconfig.html] recommends specifying client port in the dynamic config server entry (and not declare separate clientPort and clientPortAddress). But it doesn't talk about how to indicate to ZK server that the client port (2181) in "server.1=andor-5560-ubuntu:3181:4181;2181" entry should be treated as secureClientPort and not just clientPort. I believe there's no way to do this currently. So for a TLS only cluster, we *have* to specify `secureClientPort=2181` in zoo.cfg. In such cases, ZK server should skip binding to the client port in "server.1=andor-5560-ubuntu:3181:4181;2181" as non-TLS client port. It can/should still do some validation to ensure the client port in server entry matches clientPort or secureClientPort and is not completely arbitrary. was (Author: abhilash1in): [~andor] Can you please please try this config? {code:java} tickTime=2000 initLimit=10 syncLimit=5 dataDir=./andor-5560-ubuntu:2181/data secureClientPort=2181 clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty ... serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory ... server.1=andor-5560-ubuntu:3181:4181;2181 # or server.1=andor-5560-ubuntu:3181:4181:participant;0.0.0.0:2181 server.2=andor-5560-ubuntu:3182:4182;2181 server.3=andor-5560-ubuntu:3183:4183:participant;0.0.0.0:2181 {code} I believe this issue happens when `secureClientPort` is declared in zoo.cfg in addition to client port in the server entry (e.g., 2181 in "server.1=andor-5560-ubuntu:3181:4181;2181") [ZooKeeper reconfig doc|https://zookeeper.apache.org/doc/current/zookeeperReconfig.html] recommends specifying client port in the dynamic config server entry (and not declare separate clientPort and clientPortAddress). But it doesn't talk about how to indicate to ZK server that the client port (2181) in "server.1=andor-5560-ubuntu:3181:4181;2181" entry should be treated as secureClientPort and not just clientPort. I believe there's no way to do this currently. So, we still need to specify `secureClientPort=2181` in zoo.cfg and if specified, ZK server should ignore the client port in "server.1=andor-5560-ubuntu:3181:4181;2181" (it can still do some validation to ensure the port in this entry matches clientPort or secureClientPort and is not completely arbitrary). > Serving only with secureClientPort fails > > > Key: ZOOKEEPER-4276 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4276 > Project: ZooKeeper > Issue Type: Bug > Components: server >Affects Versions: 3.7.0, 3.5.8, 3.6.2, 3.8.0 >Reporter: Kei Kori >Priority: Major > Labels: pull-request-available > Time Spent: 2h > Remaining Estimate: 0h > > clientPort in zoo.cfg is forcefully complemented from client address by > QuorumPeerConfig#setupClientPort even though secureClientPort is set and > matches with client address' port. > Because of this behavior, in case rolling update with replacing clientPort to > secureClientPort in the same port number following [Upgrading existing > non-TLS cluster with no > downtime|https://zookeeper.apache.org/doc/r3.7.0/zookeeperAdmin.html#Upgrading+existing+nonTLS+cluster] > conflicts and gets errors below. > {code} > 2021-03-29 23:21:58,638 - INFO [main:NettyServerCnxnFactory@590] - binding > to port /0.0.0.0:2281 > 2021-03-29 23:21:58,748 - INFO [main:NettyServerCnxnFactory@595] - bound to > port 2281 > 2021-03-29 23:21:58,749 - INFO [main:NettyServerCnxnFactory@590] - binding > to port 0.0.0.0/0.0.0.0:2281 > 2021-03-29 23:21:58,753 - ERROR [main:QuorumPeerMain@101] - Unexpected > exception, exiting abnormally > java.net.BindException: Address already in use > {code} > QuorumPeerConfig#setupClientPort should complement only when both clientPort > and secureClientPort are empty, and
[jira] [Comment Edited] (ZOOKEEPER-4276) Serving only with secureClientPort fails
[ https://issues.apache.org/jira/browse/ZOOKEEPER-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17780132#comment-17780132 ] Abhilash Kishore edited comment on ZOOKEEPER-4276 at 10/27/23 5:06 AM: --- [~andor] Can you please please try this config? {code:java} tickTime=2000 initLimit=10 syncLimit=5 dataDir=./andor-5560-ubuntu:2181/data secureClientPort=2181 clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty ... serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory ... server.1=andor-5560-ubuntu:3181:4181;2181 # or server.1=andor-5560-ubuntu:3181:4181:participant;0.0.0.0:2181 server.2=andor-5560-ubuntu:3182:4182;2181 server.3=andor-5560-ubuntu:3183:4183:participant;0.0.0.0:2181 {code} I believe this issue happens when `secureClientPort` is declared in zoo.cfg in addition to client port in the server entry (e.g., 2181 in "server.1=andor-5560-ubuntu:3181:4181;2181") [ZooKeeper reconfig doc|https://zookeeper.apache.org/doc/current/zookeeperReconfig.html] recommends specifying client port in the dynamic config server entry (and not declare separate clientPort and clientPortAddress). But it doesn't talk about how to indicate to ZK server that the client port (2181) in "server.1=andor-5560-ubuntu:3181:4181;2181" entry should be treated as secureClientPort and not just clientPort. I believe there's no way to do this currently. So, we still need to specify `secureClientPort=2181` in zoo.cfg and if specified, ZK server should ignore the client port in "server.1=andor-5560-ubuntu:3181:4181;2181" (it can still do some validation to ensure the port in this entry matches clientPort or secureClientPort and is not completely arbitrary). was (Author: abhilash1in): [~andor] Can you please please try this config? {code:java} tickTime=2000 initLimit=10 syncLimit=5 dataDir=./andor-5560-ubuntu:2181/data secureClientPort=2181 clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty ... serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory ... server.1=andor-5560-ubuntu:3181:4181;2181 # or server.1=andor-5560-ubuntu:3181:4181:participant;0.0.0.0:2181 server.2=andor-5560-ubuntu:3182:4182;2181 server.3=andor-5560-ubuntu:3183:4183:participant;0.0.0.0:2181 {code} I believe this issue happens when `secureClientPort` is declared in zoo.cfg in addition to client port in the server entry (e.g., 2181 in "server.1=andor-5560-ubuntu:3181:4181;2181") However, [ZooKeeper reconfig doc|https://zookeeper.apache.org/doc/current/zookeeperReconfig.html] recommends specifying client port in the dynamic config server entry (and not declare separate clientPort and clientPortAddress). But it doesn't talk about how to indicate to ZK server that the client port (2181) in "server.1=andor-5560-ubuntu:3181:4181;2181" entry should be treated as secureClientPort and not just clientPort. I believe there's no way to do this currently. So, we still need to specify `secureClientPort=2181` in zoo.cfg and if specified, ZK server should ignore the client port in "server.1=andor-5560-ubuntu:3181:4181;2181" (it can still do some validation to ensure the port in this entry matches clientPort or secureClientPort and is not completely arbitrary). > Serving only with secureClientPort fails > > > Key: ZOOKEEPER-4276 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4276 > Project: ZooKeeper > Issue Type: Bug > Components: server >Affects Versions: 3.7.0, 3.5.8, 3.6.2, 3.8.0 >Reporter: Kei Kori >Priority: Major > Labels: pull-request-available > Time Spent: 2h > Remaining Estimate: 0h > > clientPort in zoo.cfg is forcefully complemented from client address by > QuorumPeerConfig#setupClientPort even though secureClientPort is set and > matches with client address' port. > Because of this behavior, in case rolling update with replacing clientPort to > secureClientPort in the same port number following [Upgrading existing > non-TLS cluster with no > downtime|https://zookeeper.apache.org/doc/r3.7.0/zookeeperAdmin.html#Upgrading+existing+nonTLS+cluster] > conflicts and gets errors below. > {code} > 2021-03-29 23:21:58,638 - INFO [main:NettyServerCnxnFactory@590] - binding > to port /0.0.0.0:2281 > 2021-03-29 23:21:58,748 - INFO [main:NettyServerCnxnFactory@595] - bound to > port 2281 > 2021-03-29 23:21:58,749 - INFO [main:NettyServerCnxnFactory@590] - binding > to port 0.0.0.0/0.0.0.0:2281 > 2021-03-29 23:21:58,753 - ERROR [main:QuorumPeerMain@101] - Unexpected > exception, exiting abnormally > java.net.BindException: Address already in use > {code} > QuorumPeerConfig#setupClientPort should complement only when both clientPort > and secureClientPort are empty, and allow serving zookeeper server only with > secure client
[jira] [Comment Edited] (ZOOKEEPER-4276) Serving only with secureClientPort fails
[ https://issues.apache.org/jira/browse/ZOOKEEPER-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17780132#comment-17780132 ] Abhilash Kishore edited comment on ZOOKEEPER-4276 at 10/27/23 5:06 AM: --- [~andor] Can you please please try this config? {code:java} tickTime=2000 initLimit=10 syncLimit=5 dataDir=./andor-5560-ubuntu:2181/data secureClientPort=2181 clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty ... serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory ... server.1=andor-5560-ubuntu:3181:4181;2181 # or server.1=andor-5560-ubuntu:3181:4181:participant;0.0.0.0:2181 server.2=andor-5560-ubuntu:3182:4182;2181 server.3=andor-5560-ubuntu:3183:4183:participant;0.0.0.0:2181 {code} I believe this issue happens when `secureClientPort` is declared in zoo.cfg in addition to client port in the server entry (e.g., 2181 in "server.1=andor-5560-ubuntu:3181:4181;2181") However, [ZooKeeper reconfig doc|https://zookeeper.apache.org/doc/current/zookeeperReconfig.html] recommends specifying client port in the dynamic config server entry (and not declare separate clientPort and clientPortAddress). But it doesn't talk about how to indicate to ZK server that the client port (2181) in "server.1=andor-5560-ubuntu:3181:4181;2181" entry should be treated as secureClientPort and not just clientPort. I believe there's no way to do this currently. So, we still need to specify `secureClientPort=2181` in zoo.cfg and if specified, ZK server should ignore the client port in "server.1=andor-5560-ubuntu:3181:4181;2181" (it can still do some validation to ensure the port in this entry matches clientPort or secureClientPort and is not completely arbitrary). was (Author: abhilash1in): [~andor] Can you please please try this config? {code:java} tickTime=2000 initLimit=10 syncLimit=5 dataDir=./andor-5560-ubuntu:2181/data secureClientPort=2181 clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty ... serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory ... server.1=andor-5560-ubuntu:3181:4181;2181 # or server.1=andor-5560-ubuntu:3181:4181:participant;0.0.0.0:2181 server.2=andor-5560-ubuntu:3182:4182;2181 server.3=andor-5560-ubuntu:3183:4183:participant;0.0.0.0:2181 {code} I believe this issue happens when `secureClientPort` is declared in zoo.cfg in addition to client port in the server entry. However, [ZooKeeper reconfig doc|https://zookeeper.apache.org/doc/current/zookeeperReconfig.html] recommends specifying client port in the dynamic config server entry (and not declare separate clientPort and clientPortAddress). But it doesn't talk about how to indicate to ZK server that the client port (2181) in "server.1=andor-5560-ubuntu:3181:4181;2181" entry should be treated as secureClientPort and not just clientPort. I believe there's no way to do this currently. So, we still need to specify `secureClientPort=2181` in zoo.cfg and if specified, ZK server should ignore the client port in "server.1=andor-5560-ubuntu:3181:4181;2181" (it can still do some validation to ensure the port in this entry matches clientPort or secureClientPort and is not completely arbitrary). > Serving only with secureClientPort fails > > > Key: ZOOKEEPER-4276 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4276 > Project: ZooKeeper > Issue Type: Bug > Components: server >Affects Versions: 3.7.0, 3.5.8, 3.6.2, 3.8.0 >Reporter: Kei Kori >Priority: Major > Labels: pull-request-available > Time Spent: 2h > Remaining Estimate: 0h > > clientPort in zoo.cfg is forcefully complemented from client address by > QuorumPeerConfig#setupClientPort even though secureClientPort is set and > matches with client address' port. > Because of this behavior, in case rolling update with replacing clientPort to > secureClientPort in the same port number following [Upgrading existing > non-TLS cluster with no > downtime|https://zookeeper.apache.org/doc/r3.7.0/zookeeperAdmin.html#Upgrading+existing+nonTLS+cluster] > conflicts and gets errors below. > {code} > 2021-03-29 23:21:58,638 - INFO [main:NettyServerCnxnFactory@590] - binding > to port /0.0.0.0:2281 > 2021-03-29 23:21:58,748 - INFO [main:NettyServerCnxnFactory@595] - bound to > port 2281 > 2021-03-29 23:21:58,749 - INFO [main:NettyServerCnxnFactory@590] - binding > to port 0.0.0.0/0.0.0.0:2281 > 2021-03-29 23:21:58,753 - ERROR [main:QuorumPeerMain@101] - Unexpected > exception, exiting abnormally > java.net.BindException: Address already in use > {code} > QuorumPeerConfig#setupClientPort should complement only when both clientPort > and secureClientPort are empty, and allow serving zookeeper server only with > secure client port. -- This message was sent by Atlassian
[jira] [Updated] (ZOOKEEPER-4764) Tune the log of refuse session request.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ZOOKEEPER-4764: -- Labels: pull-request-available (was: ) > Tune the log of refuse session request. > --- > > Key: ZOOKEEPER-4764 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4764 > Project: ZooKeeper > Issue Type: Improvement > Components: server >Affects Versions: 3.7.2, 3.8.3, 3.9.1 >Reporter: Yan Zhao >Priority: Trivial > Labels: pull-request-available > Fix For: 3.7.3, 3.8.4, 3.9.2 > > Time Spent: 10m > Remaining Estimate: 0h > > The log: > Refusing session request for client as it has seen zxid our last zxid is 0x0 > client must try another server (org.apache.zookeeper.server.ZooKeeperServer) > We would better print the sessionId in the content. > After improvement: > Refusing session(0xab) request for client as it has seen zxid our last zxid > is 0x0 client must try another server > (org.apache.zookeeper.server.ZooKeeperServer) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ZOOKEEPER-4764) Tune the log of refuse session request.
Yan Zhao created ZOOKEEPER-4764: --- Summary: Tune the log of refuse session request. Key: ZOOKEEPER-4764 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4764 Project: ZooKeeper Issue Type: Improvement Components: server Affects Versions: 3.9.1, 3.8.3, 3.7.2 Reporter: Yan Zhao Fix For: 3.7.3, 3.8.4, 3.9.2 The log: Refusing session request for client as it has seen zxid our last zxid is 0x0 client must try another server (org.apache.zookeeper.server.ZooKeeperServer) We would better print the sessionId in the content. After improvement: Refusing session(0xab) request for client as it has seen zxid our last zxid is 0x0 client must try another server (org.apache.zookeeper.server.ZooKeeperServer) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (ZOOKEEPER-4276) Serving only with secureClientPort fails
[ https://issues.apache.org/jira/browse/ZOOKEEPER-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17780132#comment-17780132 ] Abhilash Kishore commented on ZOOKEEPER-4276: - [~andor] Can you please please try this config? {code:java} tickTime=2000 initLimit=10 syncLimit=5 dataDir=./andor-5560-ubuntu:2181/data secureClientPort=2181 clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty ... serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory ... server.1=andor-5560-ubuntu:3181:4181;2181 # or server.1=andor-5560-ubuntu:3181:4181:participant;0.0.0.0:2181 server.2=andor-5560-ubuntu:3182:4182;2181 server.3=andor-5560-ubuntu:3183:4183:participant;0.0.0.0:2181 {code} I believe this issue happens when `secureClientPort` is declared in zoo.cfg in addition to client port in the server entry. However, [ZooKeeper reconfig doc|https://zookeeper.apache.org/doc/current/zookeeperReconfig.html] recommends specifying client port in the dynamic config server entry (and not declare separate clientPort and clientPortAddress). But it doesn't talk about how to indicate to ZK server that the client port (2181) in "server.1=andor-5560-ubuntu:3181:4181;2181" entry should be treated as secureClientPort and not just clientPort. I believe there's no way to do this currently. So, we still need to specify `secureClientPort=2181` in zoo.cfg and if specified, ZK server should ignore the client port in "server.1=andor-5560-ubuntu:3181:4181;2181" (it can still do some validation to ensure the port in this entry matches clientPort or secureClientPort and is not completely arbitrary). > Serving only with secureClientPort fails > > > Key: ZOOKEEPER-4276 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4276 > Project: ZooKeeper > Issue Type: Bug > Components: server >Affects Versions: 3.7.0, 3.5.8, 3.6.2, 3.8.0 >Reporter: Kei Kori >Priority: Major > Labels: pull-request-available > Time Spent: 2h > Remaining Estimate: 0h > > clientPort in zoo.cfg is forcefully complemented from client address by > QuorumPeerConfig#setupClientPort even though secureClientPort is set and > matches with client address' port. > Because of this behavior, in case rolling update with replacing clientPort to > secureClientPort in the same port number following [Upgrading existing > non-TLS cluster with no > downtime|https://zookeeper.apache.org/doc/r3.7.0/zookeeperAdmin.html#Upgrading+existing+nonTLS+cluster] > conflicts and gets errors below. > {code} > 2021-03-29 23:21:58,638 - INFO [main:NettyServerCnxnFactory@590] - binding > to port /0.0.0.0:2281 > 2021-03-29 23:21:58,748 - INFO [main:NettyServerCnxnFactory@595] - bound to > port 2281 > 2021-03-29 23:21:58,749 - INFO [main:NettyServerCnxnFactory@590] - binding > to port 0.0.0.0/0.0.0.0:2281 > 2021-03-29 23:21:58,753 - ERROR [main:QuorumPeerMain@101] - Unexpected > exception, exiting abnormally > java.net.BindException: Address already in use > {code} > QuorumPeerConfig#setupClientPort should complement only when both clientPort > and secureClientPort are empty, and allow serving zookeeper server only with > secure client port. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (ZOOKEEPER-4753) Explicit handling of DIGEST-MD5 vs GSSAPI in quorum auth
[ https://issues.apache.org/jira/browse/ZOOKEEPER-4753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779790#comment-17779790 ] Damien Diederen commented on ZOOKEEPER-4753: Hi [~xiaotong.wang], {quote}we need verify the server host when we use SASL/Kerberos {quote} Yes. (I also have additional improvements queued regarding this topic, but the changes you mention were in fact preliminary to fixing [https://zookeeper.apache.org/security.html#CVE-2023-44981]. The other changes were not included as not strictly part of the security fix.) {quote}it's better to verify if current authentication is Kerberos or not, but now we check it with isDigestAuthn and use entry.getLoginModuleName().equals(DigestLoginModule.class.getName()) {quote} Yes; this is unfortunate. Would you know of a better method to detect the SASL mechanism in use? What we really want here is to conditionalize on {{DIGEST-MD5}} or {{{}GSSAPI{}}}. {quote}we rewrite DigestLoginModule to make sure user paasword are storage with encrypted our new DigestLoginModule required user{~}hd{~}=encode("testpwd") it will incompatible when we upgrade {quote} Indeed. (I was afraid I would hear about something like that… and there we are :) Is your custom digest module a subclass of the ZooKeeper one, or an unrelated object? {quote}Is there a better way to fix this issue {quote} As mentioned above: I would love it if we could just look up whether {{DIGEST-MD5}} or {{GSSAPI}} is in use. Ideas welcome! In any case, I will keep your case into account when submitting the updated patch—worst case, you will have to explicitly disable the principal check. In the meantime, you are not affected by CVE-2023-44981 if using DIGEST-MD5. HTH, -D > Explicit handling of DIGEST-MD5 vs GSSAPI in quorum auth > > > Key: ZOOKEEPER-4753 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4753 > Project: ZooKeeper > Issue Type: Improvement > Components: server >Affects Versions: 3.9.0 >Reporter: Damien Diederen >Assignee: Damien Diederen >Priority: Major > Fix For: 3.7.2, 3.8.3, 3.9.1 > > > The SASL-based quorum authorizer does not explicitly distinguish between the > DIGEST-MD5 and GSSAPI mechanisms: it is simply relying on {{NameCallback}} > and {{PasswordCallback}} for authentication with the former and examining > Kerberos principals in {{AuthorizeCallback}} for the latter. > It turns out that some SASL/DIGEST-MD5 configurations cause authentication > and authorization IDs not to match the expected format, and the > DIGEST-MD5-based portions of the quorum test suite to fail with obscure > errors. (They can be traced to failures to join the quorum, but only by > looking into detailed logs.) > We can use the login module name to determine whether DIGEST-MD5 or GSSAPI is > used, and relax the authentication ID check for the former. As a cleanup, we > can keep the password-based credential map empty when Kerberos principals are > expected. Finally, we can adapt tests to ensure "weirdly-shaped" credentials > only cause authentication failures in the GSSAPI case. -- This message was sent by Atlassian Jira (v8.20.10#820010)