Tombstones as well as external versioning - something Hibernate 2LC has needed for a while (and Max doesn't ever stop bugging me about!)
Re: the serialisability, how about this: why make Metadata in interface? Why not a concrete class, with a fixed set of attributes (lifespan, maxIdle, Version (interface), etc). Then we ship an externalizer for this Metadata class. - M On 8 Apr 2013, at 13:52, Sanne Grinovero <[email protected]> wrote: > Got it, thanks! > +1 especially as it helps bringing tombstones, an urgent feature IMHO. > > Sanne > > On 8 April 2013 13:11, Galder Zamarreño <[email protected]> wrote: >> >> On Apr 8, 2013, at 1:46 PM, Sanne Grinovero <[email protected]> wrote: >> >>> I fail to understand the purpose of the feature then. What prevents me >>> to use the existing code today just storing some extra fields in my >>> custom values? >> >> ^ Nothing, this is doable. >> >>> What do we get by adding this code? >> >> ^ You avoid the need of the wrapper since we already have a wrapper >> internally, which is ICE. ICEs can already keep versions around, why do I >> need a wrapper class that stores a version for Hot Rod server? >> >> Down the line, we could decide to leave the metadata around to better >> support use cases like this: >> https://issues.jboss.org/browse/ISPN-506 >> https://community.jboss.org/wiki/VersioningDesignDocument - The tombstones >> that Max refers to could potentially be tombstone ICEs with only metadata >> info. >> >> Cheers, >> >>> >>> Sanne >>> >>> On 8 April 2013 12:40, Galder Zamarreño <[email protected]> wrote: >>>> >>>> On Apr 8, 2013, at 1:26 PM, Sanne Grinovero <[email protected]> wrote: >>>> >>>>> On 8 April 2013 12:06, Galder Zamarreño <[email protected]> wrote: >>>>>> >>>>>> On Apr 8, 2013, at 12:56 PM, Sanne Grinovero <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 8 April 2013 11:44, Galder Zamarreño <[email protected]> wrote: >>>>>>> >>>>>>> On Apr 8, 2013, at 12:35 PM, Galder Zamarreño <[email protected]> wrote: >>>>>>> >>>>>>>> >>>>>>>> On Apr 8, 2013, at 11:17 AM, Manik Surtani <[email protected]> wrote: >>>>>>>> >>>>>>>>> All sounds very good. One important thing to consider is that the >>>>>>>>> reference to Metadata passed in by the client app will be tied to the >>>>>>>>> ICE for the entire lifespan of the ICE. You'll need to think about a >>>>>>>>> defensive copy or some other form of making the Metadata immutable >>>>>>>>> (by the user application, at least) the moment it is passed in. >>>>>>>> >>>>>>>> ^ Excellent point, it could be a nightmare if users could change the >>>>>>>> metadata reference by the ICE at will. I'll have a think on how to >>>>>>>> best achieve this. >>>>>>> >>>>>>> ^ The metadata is gonna have to be marshalled somehow to ship to other >>>>>>> nodes, so that could be a way to achieve it, by enforcing this somehow. >>>>>>> When the cache receives it, it can marshaller/unmarshall it to make a >>>>>>> copy >>>>>>> >>>>>>> One way would be to make Metadata extend Serializable, but not keen on >>>>>>> that. Another would be to somehow force the interface to define the >>>>>>> Externalizer to use (i.e. an interface method like getExternalizer()), >>>>>>> but that's akward when it comes to unmarshalling… what about forcing >>>>>>> the Metadata object to be provided with a @SerializeWith annotation? >>>>>>> >>>>>>> Why is getExternalizer() awkward for unmarshalling? >>>>>> >>>>>> ^ Because you don't have an instance yet, so what's the Externalizer for >>>>>> it? IOW, there's no much point to doing that, simply register it >>>>>> depending on your desire: >>>>>> https://docs.jboss.org/author/display/ISPN/Plugging+Infinispan+With+User+Defined+Externalizers >>>>> >>>>> That's what I would expect. >>>>> >>>>>> >>>>>>> I would expect you to have the marshaller already known during >>>>>>> deserialization. >>>>>> >>>>>> You would, as long as you follow the instructions in >>>>>> https://docs.jboss.org/author/display/ISPN/Plugging+Infinispan+With+User+Defined+Externalizers >>>>>> >>>>>>> Agreed that extensing Serializable is not a good idea. >>>>>>> >>>>>>> Are you thinking about the impact on CacheStore(s) and state transfer? >>>>>> >>>>>> ^ What about it in particular? >>>>>> >>>>>>> Eviction of no longer used metadata ? >>>>>> >>>>>> ^ Since the metadata is part of the entry, it'd initially go when the >>>>>> entry is evicted. We might wanna leave it around in some cases… but it'd >>>>>> be for other use cases. >>>>> >>>>> I thought the plan was to have entries refer to the metadata, but that >>>>> different entries sharing the same metadata would point to the same >>>>> instance. >>>> >>>> ^ Could be, but most likely not. >>>> >>>>> So this metadata needs to be stored separately in the CacheStore, >>>>> preloaded as appropriate, transferred during state transfer, >>>>> passivated when convenient and cleaned up when no longer referred to. >>>> >>>> ^ Well, it's part of the internal cache entry, so it'd be treated just >>>> like ICE. >>>> >>>>> Am I wrong? Seems you plan to store a copy of the metadata within each >>>>> ICE. >>>> >>>> ^ The idea is to store it alongside right now, but maybe at some point it >>>> might make sense to leave it around (i.e. for 2LC use case), but this >>>> won't be done yet. >>>> >>>>> >>>>> >>>>>> >>>>>> I'm also considering separating the serialization/marshalling concerns >>>>>> from the defensive copying concerns. IOW, add a copy() method to the >>>>>> Metadata interface, or have a separate interface for those externally >>>>>> provided objects that require to be defensive copied. IOW, do something >>>>>> like what Scala Case Classes do with their copy() method, but without >>>>>> the issues of clone… I need to investigate this further to come up with >>>>>> a nice solution. >>>>>> >>>>>> One positive side to splitting both concerns is speed. A Metadata >>>>>> implementation might have ways to make a copy of itself which are more >>>>>> efficient than marshalling/unmarshalling. >>>>>> >>>>>> Thoughts? >>>>>> >>>>>>> >>>>>>> Sanne >>>>>>> >>>>>>> >>>>>>> Any other ideas? >>>>>>> >>>>>>>> >>>>>>>> Cheers, >>>>>>>> >>>>>>>>> >>>>>>>>> On 8 Apr 2013, at 09:24, Galder Zamarreño <[email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> As mentioned in >>>>>>>>>> http://lists.jboss.org/pipermail/infinispan-dev/2013-March/012348.html, >>>>>>>>>> in paralell to the switch to Equivalent* collections, I was also >>>>>>>>>> working on being able to pass metadata into Infinispan caches. This >>>>>>>>>> is done to better support the ability to store custom metadata in >>>>>>>>>> Infinispan without the need of extra wrappers. So, the idea is that >>>>>>>>>> InternalCacheEntry instances will have a a reference to this >>>>>>>>>> Metadata. >>>>>>>>>> >>>>>>>>>> One of that metadata is version, which I've been using as test bed >>>>>>>>>> to see if clients could pass succesfully version information via >>>>>>>>>> metadata. As you already know, Hot Rod requires to store version >>>>>>>>>> information. Before, this was stored in a class called CacheValue >>>>>>>>>> alongside the value itself, but the work I've done in [1], this is >>>>>>>>>> passed via the new API I've added in [2]. >>>>>>>>>> >>>>>>>>>> So, I'd like to get some thoughts on this new API. I hope that with >>>>>>>>>> these new put/replace versions, we can get rid of the nightmare >>>>>>>>>> which is all the other put/replace calls taking lifespan and/or >>>>>>>>>> maxIdle information. In the end, I think there should be two basic >>>>>>>>>> puts: >>>>>>>>>> >>>>>>>>>> - put(K, V) >>>>>>>>>> - put(K, V, Metadata) >>>>>>>>>> >>>>>>>>>> And their equivalents. >>>>>>>>>> >>>>>>>>>> IMPORTANT NOTE 1: The implementation details are bound to change, >>>>>>>>>> because the entire Metadata needs to be stored in >>>>>>>>>> InternalCacheEntry, not just version, lifespan..etc. I'll further >>>>>>>>>> develop the implementation once I get into adding more metadata, >>>>>>>>>> i.e. when working on interoperability with REST. So, don't pay too >>>>>>>>>> much attention to the implementation itself, focus on the >>>>>>>>>> AdvancedCache API itself and let's refine that. >>>>>>>>>> >>>>>>>>>> IMPORTANT NOTE 2: The interoperability work in commit in [1] is WIP, >>>>>>>>>> so please let's avoid discussing it in this email thread. Once I >>>>>>>>>> have a more final version I'll send an email about it. >>>>>>>>>> >>>>>>>>>> Apart from working on enhancements to the API, I'm now carry on >>>>>>>>>> tackling the interoperability work with aim to have an initial >>>>>>>>>> version of the Embedded <-> Hot Rod interoperability as first step. >>>>>>>>>> Once that's in, it can be released to get early feedback while the >>>>>>>>>> rest of interoperability modes are developed. >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> >>>>>>>>>> [1] >>>>>>>>>> https://github.com/galderz/infinispan/commit/a35956fe291d2b2dc3b7fa7bf44d8965ffb1a54d >>>>>>>>>> [2] >>>>>>>>>> https://github.com/galderz/infinispan/commit/a35956fe291d2b2dc3b7fa7bf44d8965ffb1a54d#L10R313 >>>>>>>>>> -- >>>>>>>>>> Galder Zamarreño >>>>>>>>>> [email protected] >>>>>>>>>> twitter.com/galderz >>>>>>>>>> >>>>>>>>>> Project Lead, Escalante >>>>>>>>>> http://escalante.io >>>>>>>>>> >>>>>>>>>> Engineer, Infinispan >>>>>>>>>> http://infinispan.org >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> infinispan-dev mailing list >>>>>>>>>> [email protected] >>>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Manik Surtani >>>>>>>>> [email protected] >>>>>>>>> twitter.com/maniksurtani >>>>>>>>> >>>>>>>>> Platform Architect, JBoss Data Grid >>>>>>>>> http://red.ht/data-grid >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> infinispan-dev mailing list >>>>>>>>> [email protected] >>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Galder Zamarreño >>>>>>>> [email protected] >>>>>>>> twitter.com/galderz >>>>>>>> >>>>>>>> Project Lead, Escalante >>>>>>>> http://escalante.io >>>>>>>> >>>>>>>> Engineer, Infinispan >>>>>>>> http://infinispan.org >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Galder Zamarreño >>>>>>> [email protected] >>>>>>> twitter.com/galderz >>>>>>> >>>>>>> Project Lead, Escalante >>>>>>> http://escalante.io >>>>>>> >>>>>>> Engineer, Infinispan >>>>>>> http://infinispan.org >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> infinispan-dev mailing list >>>>>>> [email protected] >>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>> >>>>>>> _______________________________________________ >>>>>>> infinispan-dev mailing list >>>>>>> [email protected] >>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>> >>>>>> >>>>>> -- >>>>>> Galder Zamarreño >>>>>> [email protected] >>>>>> twitter.com/galderz >>>>>> >>>>>> Project Lead, Escalante >>>>>> http://escalante.io >>>>>> >>>>>> Engineer, Infinispan >>>>>> http://infinispan.org >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> infinispan-dev mailing list >>>>>> [email protected] >>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>> >>>>> _______________________________________________ >>>>> infinispan-dev mailing list >>>>> [email protected] >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> >>>> >>>> -- >>>> Galder Zamarreño >>>> [email protected] >>>> twitter.com/galderz >>>> >>>> Project Lead, Escalante >>>> http://escalante.io >>>> >>>> Engineer, Infinispan >>>> http://infinispan.org >>>> >>>> >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> [email protected] >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> [email protected] >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> -- >> Galder Zamarreño >> [email protected] >> twitter.com/galderz >> >> Project Lead, Escalante >> http://escalante.io >> >> Engineer, Infinispan >> http://infinispan.org >> >> >> _______________________________________________ >> infinispan-dev mailing list >> [email protected] >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > [email protected] > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani [email protected] twitter.com/maniksurtani Platform Architect, JBoss Data Grid http://red.ht/data-grid _______________________________________________ infinispan-dev mailing list [email protected] https://lists.jboss.org/mailman/listinfo/infinispan-dev
