Re: Proposed: Leader communication should listen on specified IP, not wildcard address
Hi Jared, I was referring to InetSocketAddress(port) (or InetSocketAddress(port, null)), which uses a wild-card IP address. If you decide to make this optional, couldn't you can use Java property to select which InetSocketAddress() constructor to use? As mentioned earlier, I agree that keeping it optional does not buy us much. But it might be good to get a vote on this. -Vishal On Fri, Aug 27, 2010 at 10:58 AM, Jared Cantwell wrote: > Hi Vishal, > > Each server uses a different configuration file, so config files for > different servers can be different. If you were in a situation where you > would _want_ to bind to the wildcard interface, then you would need > different configuration files anyway. For example > > Assume Server 1 has two interfaces, one for each of 192.168.0.1 and > 192.168.0.2. > Server 1 config file - > server.1=*:3181:4181 > server.2=192.169.0.3:3181:4181 > server.3=192.169.0.4:3181:4181 > -- > > Server 2 config file - > server.1=192.168.0.1:3181:4181 <--- note the ip > server.2=192.169.0.3:3181:4181 > server.3=192.169.0.4:3181:4181 > -- > > Server 3 config file - > server.1=192.168.0.2:3181:4181 <--- note the ip > server.2=192.169.0.3:3181:4181 > server.3=192.169.0.4:3181:4181 > -- > > In this situation, where two servers are communicating with you through > different addresses, server 1 needs to bind to *. However, in the normal > case where everyone has the same config file, every server will communicate > with server 1 on the same address, so server 1 has no need to bind to any > other address. > > To me it seems like another config option would tell server 1 to bind to > wildcard (or a host or whatever), but then the config files of the servers > would still have to be different... > > Does that make sense? > > Jared > > > On Fri, Aug 27, 2010 at 10:49 AM, Vishal K wrote: > > > Hi Jared, > > > > If we keep this optional, then the servers would not need to bind to any > > specific interface. We will just use the current implementation > > (InetSocketAddress(port) with wildcard IP). > > You are right, keeping it optional does not buy us much. However, I did > not > > understand your solution of using "server.#:0.0.0.0[:port][:port]" in the > > config file. > > How will a peer know the IP address of other peers? > > > > -Vishal > > > > On Fri, Aug 27, 2010 at 10:20 AM, Jared Cantwell > > wrote: > > > > > Thanks for the response Vishal. I agree that it should be used for > both > > > FLE > > > and normal communication (both server ports). I believe the diff I > sent > > > incorporates that thinking, although I may not have been clear. I > > > considered that it be optional, but cases where you would want it to > bind > > > on > > > all interfaces seem rare to me, since it would require specializing the > > > config files for each server so that two different servers communicated > > > with > > > a third server on different ports. Users who would want this to be > > > optional > > > could achieve the same effect by specifying their own > > > "server.#=address[:port][:[port]" entry as > > > "server.#:0.0.0.0[:port][:port]". > > > > > > I guess my proposal is that the address specified in the server.# > should > > be > > > used. Advanced configurations can specify 0:0:0:0 to get the current > > > behavior. This way there wouldn't need to be another option to say > that > > > (which would need customized anyway in such configurations). > > > > > > ~Jared > > > > > > On Fri, Aug 27, 2010 at 9:45 AM, Vishal K > wrote: > > > > > > > Hi Jared, > > > > > > > > I like the idea of allowing communication with ZK on only the IP > > > addresses > > > > specified in the config file. This is useful when you are having a > host > > > > with > > > > multiple IP address and/or when you want to have traffic isolation. > In > > > our > > > > setup, we will use this feature for traffic isolation. We have > multiple > > > > interfaces on the server, and we want to route certain type of > traffic > > > > through certain interfaces. > > > > > > > > Why not have this feature for both FLE and leader-follower > > communication? > > > I > > > > would suggest to keep this optional (add a property) since most users > > > would > > > > prefer to have the ability to reach the servers from all interfaces. > > > > > > > > -Vishal > > > > > > > > On Thu, Aug 26, 2010 at 1:34 PM, Jared Cantwell < > > > jared.cantw...@gmail.com > > > > >wrote: > > > > > > > > > Hello, > > > > > > > > > > My project currently has the need to specify the local address that > > is > > > > used > > > > > for leader communication (and not use the default of listening on > all > > > > > interfaces). This is similar to the clientPortAddress parameter > that > > > was > > > > > recently added. After reviewing the code, we can't think of a > reason > > > why > > > > > only the port would be used with the
Re: Proposed: Leader communication should listen on specified IP, not wildcard address
Hi Vishal, Each server uses a different configuration file, so config files for different servers can be different. If you were in a situation where you would _want_ to bind to the wildcard interface, then you would need different configuration files anyway. For example Assume Server 1 has two interfaces, one for each of 192.168.0.1 and 192.168.0.2. Server 1 config file - server.1=*:3181:4181 server.2=192.169.0.3:3181:4181 server.3=192.169.0.4:3181:4181 -- Server 2 config file - server.1=192.168.0.1:3181:4181 <--- note the ip server.2=192.169.0.3:3181:4181 server.3=192.169.0.4:3181:4181 -- Server 3 config file - server.1=192.168.0.2:3181:4181 <--- note the ip server.2=192.169.0.3:3181:4181 server.3=192.169.0.4:3181:4181 -- In this situation, where two servers are communicating with you through different addresses, server 1 needs to bind to *. However, in the normal case where everyone has the same config file, every server will communicate with server 1 on the same address, so server 1 has no need to bind to any other address. To me it seems like another config option would tell server 1 to bind to wildcard (or a host or whatever), but then the config files of the servers would still have to be different... Does that make sense? Jared On Fri, Aug 27, 2010 at 10:49 AM, Vishal K wrote: > Hi Jared, > > If we keep this optional, then the servers would not need to bind to any > specific interface. We will just use the current implementation > (InetSocketAddress(port) with wildcard IP). > You are right, keeping it optional does not buy us much. However, I did not > understand your solution of using "server.#:0.0.0.0[:port][:port]" in the > config file. > How will a peer know the IP address of other peers? > > -Vishal > > On Fri, Aug 27, 2010 at 10:20 AM, Jared Cantwell > wrote: > > > Thanks for the response Vishal. I agree that it should be used for both > > FLE > > and normal communication (both server ports). I believe the diff I sent > > incorporates that thinking, although I may not have been clear. I > > considered that it be optional, but cases where you would want it to bind > > on > > all interfaces seem rare to me, since it would require specializing the > > config files for each server so that two different servers communicated > > with > > a third server on different ports. Users who would want this to be > > optional > > could achieve the same effect by specifying their own > > "server.#=address[:port][:[port]" entry as > > "server.#:0.0.0.0[:port][:port]". > > > > I guess my proposal is that the address specified in the server.# should > be > > used. Advanced configurations can specify 0:0:0:0 to get the current > > behavior. This way there wouldn't need to be another option to say that > > (which would need customized anyway in such configurations). > > > > ~Jared > > > > On Fri, Aug 27, 2010 at 9:45 AM, Vishal K wrote: > > > > > Hi Jared, > > > > > > I like the idea of allowing communication with ZK on only the IP > > addresses > > > specified in the config file. This is useful when you are having a host > > > with > > > multiple IP address and/or when you want to have traffic isolation. In > > our > > > setup, we will use this feature for traffic isolation. We have multiple > > > interfaces on the server, and we want to route certain type of traffic > > > through certain interfaces. > > > > > > Why not have this feature for both FLE and leader-follower > communication? > > I > > > would suggest to keep this optional (add a property) since most users > > would > > > prefer to have the ability to reach the servers from all interfaces. > > > > > > -Vishal > > > > > > On Thu, Aug 26, 2010 at 1:34 PM, Jared Cantwell < > > jared.cantw...@gmail.com > > > >wrote: > > > > > > > Hello, > > > > > > > > My project currently has the need to specify the local address that > is > > > used > > > > for leader communication (and not use the default of listening on all > > > > interfaces). This is similar to the clientPortAddress parameter that > > was > > > > recently added. After reviewing the code, we can't think of a reason > > why > > > > only the port would be used with the wildcard interface, when servers > > are > > > > already connecting specifically to that interface anyway. Is binding > > to > > > > the > > > > wildcard interface for leader communication intentional? > > > > > > > > I believe the change would be straightforward-- one change for each > > > leader > > > > port used. Note: this doesn't account for all leader election > > > algorithms, > > > > only the default. > > > > > > > > Index: > > > > > src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java > > > > === > > > > --- > > > > > src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java > > > > (r
Re: Proposed: Leader communication should listen on specified IP, not wildcard address
Hi Jared, If we keep this optional, then the servers would not need to bind to any specific interface. We will just use the current implementation (InetSocketAddress(port) with wildcard IP). You are right, keeping it optional does not buy us much. However, I did not understand your solution of using "server.#:0.0.0.0[:port][:port]" in the config file. How will a peer know the IP address of other peers? -Vishal On Fri, Aug 27, 2010 at 10:20 AM, Jared Cantwell wrote: > Thanks for the response Vishal. I agree that it should be used for both > FLE > and normal communication (both server ports). I believe the diff I sent > incorporates that thinking, although I may not have been clear. I > considered that it be optional, but cases where you would want it to bind > on > all interfaces seem rare to me, since it would require specializing the > config files for each server so that two different servers communicated > with > a third server on different ports. Users who would want this to be > optional > could achieve the same effect by specifying their own > "server.#=address[:port][:[port]" entry as > "server.#:0.0.0.0[:port][:port]". > > I guess my proposal is that the address specified in the server.# should be > used. Advanced configurations can specify 0:0:0:0 to get the current > behavior. This way there wouldn't need to be another option to say that > (which would need customized anyway in such configurations). > > ~Jared > > On Fri, Aug 27, 2010 at 9:45 AM, Vishal K wrote: > > > Hi Jared, > > > > I like the idea of allowing communication with ZK on only the IP > addresses > > specified in the config file. This is useful when you are having a host > > with > > multiple IP address and/or when you want to have traffic isolation. In > our > > setup, we will use this feature for traffic isolation. We have multiple > > interfaces on the server, and we want to route certain type of traffic > > through certain interfaces. > > > > Why not have this feature for both FLE and leader-follower communication? > I > > would suggest to keep this optional (add a property) since most users > would > > prefer to have the ability to reach the servers from all interfaces. > > > > -Vishal > > > > On Thu, Aug 26, 2010 at 1:34 PM, Jared Cantwell < > jared.cantw...@gmail.com > > >wrote: > > > > > Hello, > > > > > > My project currently has the need to specify the local address that is > > used > > > for leader communication (and not use the default of listening on all > > > interfaces). This is similar to the clientPortAddress parameter that > was > > > recently added. After reviewing the code, we can't think of a reason > why > > > only the port would be used with the wildcard interface, when servers > are > > > already connecting specifically to that interface anyway. Is binding > to > > > the > > > wildcard interface for leader communication intentional? > > > > > > I believe the change would be straightforward-- one change for each > > leader > > > port used. Note: this doesn't account for all leader election > > algorithms, > > > only the default. > > > > > > Index: > > > src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java > > > === > > > --- > > > src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java > > > (revision 989805) > > > +++ > > > src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java > > > (working copy) > > > @@ -434,7 +434,7 @@ > > > ss = ServerSocketChannel.open(); > > > int port = > > > self.quorumPeers.get(self.getId()).electionAddr.getPort(); > > > ss.socket().setReuseAddress(true); > > > -InetSocketAddress addr = new > > InetSocketAddress(port); > > > +InetSocketAddress addr = > > > self.quorumPeers.get(self.getId()).electionAddr; > > > LOG.info("My election bind port: " + > > addr.toString()); > > > setName(addr.toString()); > > > ss.socket().bind(addr); > > > Index: src/java/main/org/apache/zookeeper/server/quorum/Leader.java > > > === > > > --- src/java/main/org/apache/zookeeper/server/quorum/Leader.java > > > (revision 989805) > > > +++ src/java/main/org/apache/zookeeper/server/quorum/Leader.java > > > (working > > > copy) > > > @@ -128,10 +128,11 @@ > > > Leader(QuorumPeer self,LeaderZooKeeperServer zk) throws IOException > { > > > this.self = self; > > > try { > > > -ss = new ServerSocket(self.getQuorumAddress().getPort()); > > > +ss = new ServerSocket(); > > > +ss.bind(self.getQuorumAddress()); > > > } catch (BindException e) { > > > -LOG.error("Couldn't bind to port " > > > -+ self.getQuorumAddress().getPort(), e); > > > +LOG.error("Couldn't bind to a
Re: Proposed: Leader communication should listen on specified IP, not wildcard address
Thanks for the response Vishal. I agree that it should be used for both FLE and normal communication (both server ports). I believe the diff I sent incorporates that thinking, although I may not have been clear. I considered that it be optional, but cases where you would want it to bind on all interfaces seem rare to me, since it would require specializing the config files for each server so that two different servers communicated with a third server on different ports. Users who would want this to be optional could achieve the same effect by specifying their own "server.#=address[:port][:[port]" entry as "server.#:0.0.0.0[:port][:port]". I guess my proposal is that the address specified in the server.# should be used. Advanced configurations can specify 0:0:0:0 to get the current behavior. This way there wouldn't need to be another option to say that (which would need customized anyway in such configurations). ~Jared On Fri, Aug 27, 2010 at 9:45 AM, Vishal K wrote: > Hi Jared, > > I like the idea of allowing communication with ZK on only the IP addresses > specified in the config file. This is useful when you are having a host > with > multiple IP address and/or when you want to have traffic isolation. In our > setup, we will use this feature for traffic isolation. We have multiple > interfaces on the server, and we want to route certain type of traffic > through certain interfaces. > > Why not have this feature for both FLE and leader-follower communication? I > would suggest to keep this optional (add a property) since most users would > prefer to have the ability to reach the servers from all interfaces. > > -Vishal > > On Thu, Aug 26, 2010 at 1:34 PM, Jared Cantwell >wrote: > > > Hello, > > > > My project currently has the need to specify the local address that is > used > > for leader communication (and not use the default of listening on all > > interfaces). This is similar to the clientPortAddress parameter that was > > recently added. After reviewing the code, we can't think of a reason why > > only the port would be used with the wildcard interface, when servers are > > already connecting specifically to that interface anyway. Is binding to > > the > > wildcard interface for leader communication intentional? > > > > I believe the change would be straightforward-- one change for each > leader > > port used. Note: this doesn't account for all leader election > algorithms, > > only the default. > > > > Index: > > src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java > > === > > --- > > src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java > > (revision 989805) > > +++ > > src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java > > (working copy) > > @@ -434,7 +434,7 @@ > > ss = ServerSocketChannel.open(); > > int port = > > self.quorumPeers.get(self.getId()).electionAddr.getPort(); > > ss.socket().setReuseAddress(true); > > -InetSocketAddress addr = new > InetSocketAddress(port); > > +InetSocketAddress addr = > > self.quorumPeers.get(self.getId()).electionAddr; > > LOG.info("My election bind port: " + > addr.toString()); > > setName(addr.toString()); > > ss.socket().bind(addr); > > Index: src/java/main/org/apache/zookeeper/server/quorum/Leader.java > > === > > --- src/java/main/org/apache/zookeeper/server/quorum/Leader.java > > (revision 989805) > > +++ src/java/main/org/apache/zookeeper/server/quorum/Leader.java > > (working > > copy) > > @@ -128,10 +128,11 @@ > > Leader(QuorumPeer self,LeaderZooKeeperServer zk) throws IOException { > > this.self = self; > > try { > > -ss = new ServerSocket(self.getQuorumAddress().getPort()); > > +ss = new ServerSocket(); > > +ss.bind(self.getQuorumAddress()); > > } catch (BindException e) { > > -LOG.error("Couldn't bind to port " > > -+ self.getQuorumAddress().getPort(), e); > > +LOG.error("Couldn't bind to address " > > ++ self.getQuorumAddress().getAddress() + ":" + > > self.getQuorumAddress().getPort(), e); > > throw e; > > } > > this.zk=zk; > > > > > > Does this seem like a reasonable change? Thoughts? > > > > ~Jared > > >
Re: Proposed: Leader communication should listen on specified IP, not wildcard address
Hi Jared, I like the idea of allowing communication with ZK on only the IP addresses specified in the config file. This is useful when you are having a host with multiple IP address and/or when you want to have traffic isolation. In our setup, we will use this feature for traffic isolation. We have multiple interfaces on the server, and we want to route certain type of traffic through certain interfaces. Why not have this feature for both FLE and leader-follower communication? I would suggest to keep this optional (add a property) since most users would prefer to have the ability to reach the servers from all interfaces. -Vishal On Thu, Aug 26, 2010 at 1:34 PM, Jared Cantwell wrote: > Hello, > > My project currently has the need to specify the local address that is used > for leader communication (and not use the default of listening on all > interfaces). This is similar to the clientPortAddress parameter that was > recently added. After reviewing the code, we can't think of a reason why > only the port would be used with the wildcard interface, when servers are > already connecting specifically to that interface anyway. Is binding to > the > wildcard interface for leader communication intentional? > > I believe the change would be straightforward-- one change for each leader > port used. Note: this doesn't account for all leader election algorithms, > only the default. > > Index: > src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java > === > --- > src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java > (revision 989805) > +++ > src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java > (working copy) > @@ -434,7 +434,7 @@ > ss = ServerSocketChannel.open(); > int port = > self.quorumPeers.get(self.getId()).electionAddr.getPort(); > ss.socket().setReuseAddress(true); > -InetSocketAddress addr = new InetSocketAddress(port); > +InetSocketAddress addr = > self.quorumPeers.get(self.getId()).electionAddr; > LOG.info("My election bind port: " + addr.toString()); > setName(addr.toString()); > ss.socket().bind(addr); > Index: src/java/main/org/apache/zookeeper/server/quorum/Leader.java > === > --- src/java/main/org/apache/zookeeper/server/quorum/Leader.java > (revision 989805) > +++ src/java/main/org/apache/zookeeper/server/quorum/Leader.java > (working > copy) > @@ -128,10 +128,11 @@ > Leader(QuorumPeer self,LeaderZooKeeperServer zk) throws IOException { > this.self = self; > try { > -ss = new ServerSocket(self.getQuorumAddress().getPort()); > +ss = new ServerSocket(); > +ss.bind(self.getQuorumAddress()); > } catch (BindException e) { > -LOG.error("Couldn't bind to port " > -+ self.getQuorumAddress().getPort(), e); > +LOG.error("Couldn't bind to address " > ++ self.getQuorumAddress().getAddress() + ":" + > self.getQuorumAddress().getPort(), e); > throw e; > } > this.zk=zk; > > > Does this seem like a reasonable change? Thoughts? > > ~Jared >
Proposed: Leader communication should listen on specified IP, not wildcard address
Hello, My project currently has the need to specify the local address that is used for leader communication (and not use the default of listening on all interfaces). This is similar to the clientPortAddress parameter that was recently added. After reviewing the code, we can't think of a reason why only the port would be used with the wildcard interface, when servers are already connecting specifically to that interface anyway. Is binding to the wildcard interface for leader communication intentional? I believe the change would be straightforward-- one change for each leader port used. Note: this doesn't account for all leader election algorithms, only the default. Index: src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java === --- src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java (revision 989805) +++ src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java (working copy) @@ -434,7 +434,7 @@ ss = ServerSocketChannel.open(); int port = self.quorumPeers.get(self.getId()).electionAddr.getPort(); ss.socket().setReuseAddress(true); -InetSocketAddress addr = new InetSocketAddress(port); +InetSocketAddress addr = self.quorumPeers.get(self.getId()).electionAddr; LOG.info("My election bind port: " + addr.toString()); setName(addr.toString()); ss.socket().bind(addr); Index: src/java/main/org/apache/zookeeper/server/quorum/Leader.java === --- src/java/main/org/apache/zookeeper/server/quorum/Leader.java (revision 989805) +++ src/java/main/org/apache/zookeeper/server/quorum/Leader.java(working copy) @@ -128,10 +128,11 @@ Leader(QuorumPeer self,LeaderZooKeeperServer zk) throws IOException { this.self = self; try { -ss = new ServerSocket(self.getQuorumAddress().getPort()); +ss = new ServerSocket(); +ss.bind(self.getQuorumAddress()); } catch (BindException e) { -LOG.error("Couldn't bind to port " -+ self.getQuorumAddress().getPort(), e); +LOG.error("Couldn't bind to address " ++ self.getQuorumAddress().getAddress() + ":" + self.getQuorumAddress().getPort(), e); throw e; } this.zk=zk; Does this seem like a reasonable change? Thoughts? ~Jared