Re: [infinispan-dev] Shipping logical address String in Hot Rod protocol v1.1?
On 29 Nov 2011, at 15:09, Bela Ban wrote: >> >> Looks dangerous from a JGroups perspective, why is it allowed? > > > I like to give users a lot of rope to hang themselves ... :-) ha ha How very C++ of you. :) -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Shipping logical address String in Hot Rod protocol v1.1?
On Nov 30, 2011, at 10:28 AM, Dan Berindei wrote: > On Wed, Nov 30, 2011 at 10:04 AM, Bela Ban wrote: >> >> >> On 11/30/11 8:59 AM, Dan Berindei wrote: >> Why don't you send the UUID as a (16 byte) string then ? >>> >>> Yeah, that would work. However, a UUID is not always a valid UTF-8 >>> string, so we should probably define it in the protocol as an array of >>> bytes (without any meaning). >> >> >> Yes. We did something similar in JGroups, take a look at >> ENCRYPT.byteArrayToHexString(). >> > > Since HotRod is a binary protocol I meant to send the UUID as it is > (no encoding whatsoever). > > I think sending the UUID as a hex-encoded string, a raw byte array, or > a 32-bit hash (since that's all we need on the client) are all valid > choices. Right, I think relying on the UUID underneath the JGroups address is a better choice than the logical address itself, so let's use that as base. My preference is for a 32-bit hash which is nice and compact. https://issues.jboss.org/browse/ISPN-1578 > >> -- >> Bela Ban >> Lead JGroups (http://www.jgroups.org) >> JBoss / Red Hat >> ___ >> infinispan-dev mailing list >> infinispan-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Shipping logical address String in Hot Rod protocol v1.1?
On Wed, Nov 30, 2011 at 10:04 AM, Bela Ban wrote: > > > On 11/30/11 8:59 AM, Dan Berindei wrote: > >>> Why don't you send the UUID as a (16 byte) string then ? >>> >> >> Yeah, that would work. However, a UUID is not always a valid UTF-8 >> string, so we should probably define it in the protocol as an array of >> bytes (without any meaning). > > > Yes. We did something similar in JGroups, take a look at > ENCRYPT.byteArrayToHexString(). > Since HotRod is a binary protocol I meant to send the UUID as it is (no encoding whatsoever). I think sending the UUID as a hex-encoded string, a raw byte array, or a 32-bit hash (since that's all we need on the client) are all valid choices. > -- > Bela Ban > Lead JGroups (http://www.jgroups.org) > JBoss / Red Hat > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Shipping logical address String in Hot Rod protocol v1.1?
On 11/30/11 8:59 AM, Dan Berindei wrote: >> Why don't you send the UUID as a (16 byte) string then ? >> > > Yeah, that would work. However, a UUID is not always a valid UTF-8 > string, so we should probably define it in the protocol as an array of > bytes (without any meaning). Yes. We did something similar in JGroups, take a look at ENCRYPT.byteArrayToHexString(). -- Bela Ban Lead JGroups (http://www.jgroups.org) JBoss / Red Hat ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Shipping logical address String in Hot Rod protocol v1.1?
On Wed, Nov 30, 2011 at 9:46 AM, Bela Ban wrote: > > > On 11/29/11 6:16 PM, Dan Berindei wrote: > >>> The underlying UUID would be printed (not a nice sight !)... >>> >> >> Hmm, I sometimes see UUIDs in the test suite logs. This makes me >> think, can we rely on the discovery protocol always giving us the >> logical names of the other cluster members during/after the >> viewAccepted() callback? > > > No, you can't. For example, if someone did not list *all* IP addresses > in TCPPING.initial_hosts, we would not get the logical names of some > members. > Ok, so using the logical names is definitely out. > > >>> Can't you grab the UUID which is 2 longs and simply send 2 longs across >>> the wire ? The UUID is guaranteed to be unique. >>> >> >> If we commit to sending a UUID to the HotRod clients it will be harder >> to send something else in the future. One of the reasons we chose to >> send a string was so that we could change the way we generate that >> string without breaking the clients. > > > Why don't you send the UUID as a (16 byte) string then ? > Yeah, that would work. However, a UUID is not always a valid UTF-8 string, so we should probably define it in the protocol as an array of bytes (without any meaning). > > > > -- > Bela Ban > Lead JGroups (http://www.jgroups.org) > JBoss / Red Hat > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Shipping logical address String in Hot Rod protocol v1.1?
On 11/29/11 6:16 PM, Dan Berindei wrote: >> The underlying UUID would be printed (not a nice sight !)... >> > > Hmm, I sometimes see UUIDs in the test suite logs. This makes me > think, can we rely on the discovery protocol always giving us the > logical names of the other cluster members during/after the > viewAccepted() callback? No, you can't. For example, if someone did not list *all* IP addresses in TCPPING.initial_hosts, we would not get the logical names of some members. >> Can't you grab the UUID which is 2 longs and simply send 2 longs across >> the wire ? The UUID is guaranteed to be unique. >> > > If we commit to sending a UUID to the HotRod clients it will be harder > to send something else in the future. One of the reasons we chose to > send a string was so that we could change the way we generate that > string without breaking the clients. Why don't you send the UUID as a (16 byte) string then ? -- Bela Ban Lead JGroups (http://www.jgroups.org) JBoss / Red Hat ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Shipping logical address String in Hot Rod protocol v1.1?
On Tue, Nov 29, 2011 at 5:09 PM, Bela Ban wrote: > > > On 11/29/11 3:50 PM, Galder Zamarreño wrote: >> >> On Nov 29, 2011, at 2:23 PM, Bela Ban wrote: >> >>> >>> >>> On 11/29/11 2:10 PM, Galder Zamarreño wrote: Hi, We've been having a discussion this morning with regards to the Hot Rod changes introduced in 5.1 with regards to hashing. When Hot Rod server is deployed in AS, in order to start correctly, it requires the Hot Rod server to start before any other (clustered) caches in AS. This is because any hashing can only be done once the hash has been calculated on the hot rod endpoint. Although this can be fixed in a hacky way (have all caches configured to start lazily and let the Hot Rod server start the topology cache and then all defined caches, ugh), we're considering a slight change to Hot Rod protocol v 1.1 (https://docs.jboss.org/author/display/ISPN/Hot+Rod+Protocol+-+Version+1.1) that would solve this problem. Instead of hashing on the hot rod endpoint address (host:port), we could hash on JGroups's Address UTF-8 toString representation, which is the logical address. The advantage here is any cache can calculate the hash on this from the start, no need to wait for Hot Rod server to start. The downside is that Hot Rod clients need to be aware of this UTF-8 string in order to hash to the same thing, so it'd mean shipping this back to the clients alongside the hot rod endpoint info. >>> >>> >>> I have a few concerns, maybe not an issue with the way HotRod uses this, >>> but I wanted to bring this up anyway: >> >> These are very valid concerns. >> >>> - What if a logical name is null ? A user can do JChannel.setName(null); >> >> Hmmm, what's the JGroups address toString() representation in that case? > > > The underlying UUID would be printed (not a nice sight !)... > Hmm, I sometimes see UUIDs in the test suite logs. This makes me think, can we rely on the discovery protocol always giving us the logical names of the other cluster members during/after the viewAccepted() callback? > > >>> - What if the logical name changes at runtime ? Certainly not >>> recomended, but the APi allows this, or I should rather say, doesn't >>> prevent it... :-( >> >> If the logical name changes, is a view change triggered? > > > No. Maybe I should disallow changing a logical name after connect() has > been called... > > >> How are other nodes gonna locate this node if it's logical address has >> changed? > > > Through the discovery process. Again, maybe I should just disallow this > use case, as it IMO doesn't make any sense. > > >> Looks dangerous from a JGroups perspective, why is it allowed? > > > I like to give users a lot of rope to hang themselves ... :-) ha ha > > OK, you convinced me: https://issues.jboss.org/browse/JGRP-1395, for > 3.0.2 and 3.1.0 > +1 > >>> - What if we have multiple members with the same logical name ? >> >> Can that happen? > > > Yes > It's not very likely with standalone Infinispan, as we also add a random number to the configured node name. We also have the option of enforcing a unique node name by just failing to start when we detect duplicates. Don't know how AS sets the node names, but I'm pretty sure they also ensure a unique node name. > >> How can JGroups work correctly if multiple members have the same logical >> name? How can unicasts be directed correctly in this case? > > > Because the logical name is just a nice way to hide a UUID. JGroups > never uses the logical address for anything but pretty-printing; instead > the underlying UUID is used for addressing purposes. > > >>> - At startup, we may not yet have the logical names of all members, as >>> this is done as part of the discovery protocol… >> >> Hmmm, don't think that's important. What matters is that when node A starts, >> after the channel is open, it's logical address is availble locally. > > > OK , that'll be the case > > >> That's the same right now. When Hot Rod starts, it maps the running node's >> JGroups address to its Hot Rod enpoint address. > > > OK > > >>> - Can't you hash on the address ? It'll be available as soon as >>> connect() returns. Is that too late ? >> >> That would mean hashing a Java object which is not portable, hence why we >> need a hash on something that's portable, i.e. a UTF-8 String. > > > Can't you grab the UUID which is 2 longs and simply send 2 longs across > the wire ? The UUID is guaranteed to be unique. > If we commit to sending a UUID to the HotRod clients it will be harder to send something else in the future. One of the reasons we chose to send a string was so that we could change the way we generate that string without breaking the clients. For example I'm very interested in starting nodes with a fixed hash seed, so that after a restart the local cache store is still relevant for this node. It would be also nice if the order of nodes
Re: [infinispan-dev] Shipping logical address String in Hot Rod protocol v1.1?
On 11/29/11 3:50 PM, Galder Zamarreño wrote: > > On Nov 29, 2011, at 2:23 PM, Bela Ban wrote: > >> >> >> On 11/29/11 2:10 PM, Galder Zamarreño wrote: >>> Hi, >>> >>> We've been having a discussion this morning with regards to the Hot Rod >>> changes introduced in 5.1 with regards to hashing. >>> >>> When Hot Rod server is deployed in AS, in order to start correctly, it >>> requires the Hot Rod server to start before any other (clustered) caches in >>> AS. This is because any hashing can only be done once the hash has been >>> calculated on the hot rod endpoint. >>> >>> Although this can be fixed in a hacky way (have all caches configured to >>> start lazily and let the Hot Rod server start the topology cache and then >>> all defined caches, ugh), we're considering a slight change to Hot Rod >>> protocol v 1.1 >>> (https://docs.jboss.org/author/display/ISPN/Hot+Rod+Protocol+-+Version+1.1) >>> that would solve this problem. >>> >>> Instead of hashing on the hot rod endpoint address (host:port), we could >>> hash on JGroups's Address UTF-8 toString representation, which is the >>> logical address. The advantage here is any cache can calculate the hash on >>> this from the start, no need to wait for Hot Rod server to start. The >>> downside is that Hot Rod clients need to be aware of this UTF-8 string in >>> order to hash to the same thing, so it'd mean shipping this back to the >>> clients alongside the hot rod endpoint info. >> >> >> I have a few concerns, maybe not an issue with the way HotRod uses this, >> but I wanted to bring this up anyway: > > These are very valid concerns. > >> - What if a logical name is null ? A user can do JChannel.setName(null); > > Hmmm, what's the JGroups address toString() representation in that case? The underlying UUID would be printed (not a nice sight !)... >> - What if the logical name changes at runtime ? Certainly not >> recomended, but the APi allows this, or I should rather say, doesn't >> prevent it... :-( > > If the logical name changes, is a view change triggered? No. Maybe I should disallow changing a logical name after connect() has been called... > How are other nodes gonna locate this node if it's logical address has > changed? Through the discovery process. Again, maybe I should just disallow this use case, as it IMO doesn't make any sense. > Looks dangerous from a JGroups perspective, why is it allowed? I like to give users a lot of rope to hang themselves ... :-) ha ha OK, you convinced me: https://issues.jboss.org/browse/JGRP-1395, for 3.0.2 and 3.1.0 >> - What if we have multiple members with the same logical name ? > > Can that happen? Yes > How can JGroups work correctly if multiple members have the same logical > name? How can unicasts be directed correctly in this case? Because the logical name is just a nice way to hide a UUID. JGroups never uses the logical address for anything but pretty-printing; instead the underlying UUID is used for addressing purposes. >> - At startup, we may not yet have the logical names of all members, as >> this is done as part of the discovery protocol… > > Hmmm, don't think that's important. What matters is that when node A starts, > after the channel is open, it's logical address is availble locally. OK , that'll be the case > That's the same right now. When Hot Rod starts, it maps the running node's > JGroups address to its Hot Rod enpoint address. OK >> - Can't you hash on the address ? It'll be available as soon as >> connect() returns. Is that too late ? > > That would mean hashing a Java object which is not portable, hence why we > need a hash on something that's portable, i.e. a UTF-8 String. Can't you grab the UUID which is 2 longs and simply send 2 longs across the wire ? The UUID is guaranteed to be unique. >> Note that you can also generate your own addresses (using >> AddressGenerator) if that helps… > > H, actually, this could be useful. What are the chances of generating two > identical addresses? :) That depends on the AddressGenerator you rpovide. The idea behind AddressGenerator is to use a UUID, but add some additional data to every address, such as machineId, rackId and siteId, as used in RELAY. -- Bela Ban Lead JGroups (http://www.jgroups.org) JBoss / Red Hat ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Shipping logical address String in Hot Rod protocol v1.1?
On Nov 29, 2011, at 2:23 PM, Bela Ban wrote: > > > On 11/29/11 2:10 PM, Galder Zamarreño wrote: >> Hi, >> >> We've been having a discussion this morning with regards to the Hot Rod >> changes introduced in 5.1 with regards to hashing. >> >> When Hot Rod server is deployed in AS, in order to start correctly, it >> requires the Hot Rod server to start before any other (clustered) caches in >> AS. This is because any hashing can only be done once the hash has been >> calculated on the hot rod endpoint. >> >> Although this can be fixed in a hacky way (have all caches configured to >> start lazily and let the Hot Rod server start the topology cache and then >> all defined caches, ugh), we're considering a slight change to Hot Rod >> protocol v 1.1 >> (https://docs.jboss.org/author/display/ISPN/Hot+Rod+Protocol+-+Version+1.1) >> that would solve this problem. >> >> Instead of hashing on the hot rod endpoint address (host:port), we could >> hash on JGroups's Address UTF-8 toString representation, which is the >> logical address. The advantage here is any cache can calculate the hash on >> this from the start, no need to wait for Hot Rod server to start. The >> downside is that Hot Rod clients need to be aware of this UTF-8 string in >> order to hash to the same thing, so it'd mean shipping this back to the >> clients alongside the hot rod endpoint info. > > > I have a few concerns, maybe not an issue with the way HotRod uses this, > but I wanted to bring this up anyway: These are very valid concerns. > - What if a logical name is null ? A user can do JChannel.setName(null); Hmmm, what's the JGroups address toString() representation in that case? > > - What if the logical name changes at runtime ? Certainly not > recomended, but the APi allows this, or I should rather say, doesn't > prevent it... :-( If the logical name changes, is a view change triggered? How are other nodes gonna locate this node if it's logical address has changed? Looks dangerous from a JGroups perspective, why is it allowed? > - What if we have multiple members with the same logical name ? Can that happen? How can JGroups work correctly if multiple members have the same logical name? How can unicasts be directed correctly in this case? > - At startup, we may not yet have the logical names of all members, as > this is done as part of the discovery protocol… Hmmm, don't think that's important. What matters is that when node A starts, after the channel is open, it's logical address is availble locally. That's the same right now. When Hot Rod starts, it maps the running node's JGroups address to its Hot Rod enpoint address. > - Can't you hash on the address ? It'll be available as soon as > connect() returns. Is that too late ? That would mean hashing a Java object which is not portable, hence why we need a hash on something that's portable, i.e. a UTF-8 String. > > Note that you can also generate your own addresses (using > AddressGenerator) if that helps… H, actually, this could be useful. What are the chances of generating two identical addresses? :) > > > > -- > Bela Ban > Lead JGroups (http://www.jgroups.org) > JBoss / Red Hat > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Shipping logical address String in Hot Rod protocol v1.1?
On 11/29/11 2:10 PM, Galder Zamarreño wrote: > Hi, > > We've been having a discussion this morning with regards to the Hot Rod > changes introduced in 5.1 with regards to hashing. > > When Hot Rod server is deployed in AS, in order to start correctly, it > requires the Hot Rod server to start before any other (clustered) caches in > AS. This is because any hashing can only be done once the hash has been > calculated on the hot rod endpoint. > > Although this can be fixed in a hacky way (have all caches configured to > start lazily and let the Hot Rod server start the topology cache and then all > defined caches, ugh), we're considering a slight change to Hot Rod protocol v > 1.1 > (https://docs.jboss.org/author/display/ISPN/Hot+Rod+Protocol+-+Version+1.1) > that would solve this problem. > > Instead of hashing on the hot rod endpoint address (host:port), we could hash > on JGroups's Address UTF-8 toString representation, which is the logical > address. The advantage here is any cache can calculate the hash on this from > the start, no need to wait for Hot Rod server to start. The downside is that > Hot Rod clients need to be aware of this UTF-8 string in order to hash to the > same thing, so it'd mean shipping this back to the clients alongside the hot > rod endpoint info. I have a few concerns, maybe not an issue with the way HotRod uses this, but I wanted to bring this up anyway: - What if a logical name is null ? A user can do JChannel.setName(null); - What if the logical name changes at runtime ? Certainly not recomended, but the APi allows this, or I should rather say, doesn't prevent it... :-( - What if we have multiple members with the same logical name ? - At startup, we may not yet have the logical names of all members, as this is done as part of the discovery protocol... - Can't you hash on the address ? It'll be available as soon as connect() returns. Is that too late ? Note that you can also generate your own addresses (using AddressGenerator) if that helps... -- Bela Ban Lead JGroups (http://www.jgroups.org) JBoss / Red Hat ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
[infinispan-dev] Shipping logical address String in Hot Rod protocol v1.1?
Hi, We've been having a discussion this morning with regards to the Hot Rod changes introduced in 5.1 with regards to hashing. When Hot Rod server is deployed in AS, in order to start correctly, it requires the Hot Rod server to start before any other (clustered) caches in AS. This is because any hashing can only be done once the hash has been calculated on the hot rod endpoint. Although this can be fixed in a hacky way (have all caches configured to start lazily and let the Hot Rod server start the topology cache and then all defined caches, ugh), we're considering a slight change to Hot Rod protocol v 1.1 (https://docs.jboss.org/author/display/ISPN/Hot+Rod+Protocol+-+Version+1.1) that would solve this problem. Instead of hashing on the hot rod endpoint address (host:port), we could hash on JGroups's Address UTF-8 toString representation, which is the logical address. The advantage here is any cache can calculate the hash on this from the start, no need to wait for Hot Rod server to start. The downside is that Hot Rod clients need to be aware of this UTF-8 string in order to hash to the same thing, so it'd mean shipping this back to the clients alongside the hot rod endpoint info. In spite of the added bytes, there's a benefit to this, and that is the fact that we're not tying a specific format to the String. That is, we're just telling the clients to take this UTF-8 string and hash on that. So, it's internal representation could evolve over time with no impact to the client. In the current v 1.1 protocol, clients and servers assume a String of format host:port to do the hashing. A third way to fix this would have been to have Hot Rod servers run with a different cache manager to the rest (but still same JGroups channel), but according to Dan, AS7 is not capable of doing this. So, what does people think of this protocol change in v1.1? This version was introduced in 5.1 and since we still have not released yet, this is the right time to consider protocol changes like this. Personally, I'm in favour. Cheers, -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
