On Mar 19, 2013, at 8:56 PM, Mircea Markus <[email protected]> wrote:
> > On 18 Mar 2013, at 12:21, Galder Zamarreño wrote: > >> Hi all, >> >> A heads up on what is going on with https://issues.jboss.org/browse/ISPN-2281 >> >> While discussing this, Tristan and I came to the conclusion that we could >> avoid the need to create some wrappers required to fulfill requirements in >> this JIRA, and as a side effect, reduce the memory consumption of Infinispan >> servers, if we could have internal data containers based on concurrent hash >> maps that took a custom function for equals/hashCode…etc. By doing that, you >> could effectively have **byte[] keys and values for maps**. >> >> By doing that, you avoid creating wrappers (yippee!) for keys (bye bye >> ByteArrayKey), and combined with a better way to pass metadata into >> Infinispan Caches (i.e. version) that is stored within the internal cache >> entries, you avoid wrapper values too! (bye bye CacheValue). > Do you plan to use Versioned*CacheValue for storing values? if so you'd still > create a version object to be aggregated in Versioned*CacheValue. ^ Yes, precisely! This was very easy to do actually. Once I had the API to feed in version (and other metadata down the line…), it was very easy to change the core to use internal cache entry classes to store. Hence, no extra wrapper value other than what Infinispan uses internally. >> >> Doing the latter was relatively simple (I have this stashed), but having a >> CHM that could take a byte[] as key wasn't that easy, since we can't change >> JDK CHM. > why's that? copyright? ^ Well, I cannot modify the JDK source code directly. I'd have to fork the JDK CHM, but I prefer to take the CHMv8 version since it gets rid of segments. >> >> This is why, I've created a new CHM, based on the CHMv8, called >> ComparingConcurrentHashMapv8 (thx Tristan for the name!). The work for this >> can be seen in: >> https://github.com/galderz/infinispan/commit/351e29d327d163ca8e941edf873f6d46b43cfae1 > this relies on sun.misc.Unsafe, so won't work with a non-oracle JVM. ^ Hmmm, possibly. So we might need a JDK CHM version for it. I've already transformed BoundedCHM to it, so woud be easy to achieve. > This class would be aggregated in the DataContainer in the case we use > infinispan in server mode, right? ^ Right now, I've added two configuration options to dataContainer element, called comparingKey and comparingValue, both of which are class names to implementations of Comparing. If either is defined, a Comparing*CHM would be used. So, Hot Rod servers would need to have ComparingByteArray defined in comparingKey and comparingValue in their configuration. I think this is something we can control in both the default Infinispan and JDG servers. Thanks for the excellent feedback! :) >> >> I'm sending it here so that I can get feedback early on. I've added some >> tests as well that verify that ComparingConcurrentHashMapv8, with byte[] >> keys and values, works as expected, and checks that the expectations are >> opposite with JDK CHM. It also tests new function-based methods. >> >> To make it easier to track changes as original CHMv8 evolves, I've marked >> all changes with a marker comment that should make it easy to apply same >> changes in new CHMv8 versions. Plus, with the tests I've added, it can >> easily be seen if it works as expected or not. >> >> Two important TODOs, which will be most likely separated into separate JIRAs: >> >> 1. Note that TreeBin has not been modified to use custom equals/hashCode >> functions. That is cos I need to implement a way to compare byte arrays, >> i.e. provide equivalent logic for Comparable.compare(). >> >> 2. Compare memory consumption of a CHMv8 with wrapper classes for byte >> arrays versus ComparingConcurrentHashMapv8<byte[], byte[]>. I'll do that >> once it's closer to CR stages. Right now fulfilling requirements in >> ISPN-2281 is more priority. >> >> Finally, I need to do the same thing with out BoundedConcurrentHashMap, iow, >> provide a way to do comparison based on custom equals/hashCode. That's gonna >> be my next task, before I get to transform Infinispan Servers to take a type >> directly, and avoid relying on ByteArrayKey or CacheValue wrappers. >> >> IOW, you'll be able to say: create an Infinsipan Server that has String as >> key and value of type X, where X is the actual data type, no metadata!! The >> metadata (version, encoding, whatever is requried to fulfill the >> compatibility reqs in ISPN-2281) will be passed as part of the >> put/replace…etc (I will email this around when in place). >> >> Cheers, >> -- >> Galder Zamarreño >> [email protected] >> twitter.com/galderz >> >> Project Lead, Escalante >> http://escalante.io >> >> Engineer, Infinispan >> http://infinispan.org >> >> >> _______________________________________________ >> infinispan-dev mailing list >> [email protected] >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > Cheers, > -- > Mircea Markus > Infinispan lead (www.infinispan.org) > > > > > > _______________________________________________ > infinispan-dev mailing list > [email protected] > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Galder Zamarreño [email protected] twitter.com/galderz Project Lead, Escalante http://escalante.io Engineer, Infinispan http://infinispan.org _______________________________________________ infinispan-dev mailing list [email protected] https://lists.jboss.org/mailman/listinfo/infinispan-dev
