Values in the CachedMap are serialized accordingly to your transaction type
:
CachedMap c = new CachedMap(s, _opts.localCacheSize);
MapState ms;
if(_type == StateType.NON_TRANSACTIONAL) {
ms = NonTransactionalMap.build(c);
} else if(_type==StateType.OPAQUE) {
ms = OpaqueMap.build(c);
} else if(_type==StateType.TRANSACTIONAL){
ms = TransactionalMap.build(c);
}
2014-06-14 2:07 GMT+09:00 Raphael Hsieh <[email protected]>:
> If we don't serialize the data when we store it in the cache, doesn't that
> defeat the purpose of having an OpaqueValue in order to keep transactional
> consistency and the processed exactly once semantics?
>
>
> On Thu, Jun 12, 2014 at 8:57 AM, Raphael Hsieh <[email protected]>
> wrote:
>
>> How come we only serialize data when storing it into the external
>> datastore and not the local cache then?
>>
>>
>> On Thu, Jun 12, 2014 at 7:56 AM, Romain Leroux <[email protected]>
>> wrote:
>>
>>>
>>> https://github.com/nathanmarz/trident-memcached/blob/master/src/jvm/trident/memcached/MemcachedState.java#L62
>>> localCacheSize sets the size of the LRU cache used locally, so no
>>> CachedMap data are not replicated among all host.
>>>
>>> The idea is that you use this on a grouped stream to store the result of
>>> your aggregation.
>>> So based on the hash (the groupBy fields) all related tuples always go
>>> to the same host.
>>> You want to avoid storing the same cached data in all hosts, otherwise
>>> scaling out is meaningless.
>>>
>>>
>>>
>>> 2014-06-12 6:58 GMT+09:00 Raphael Hsieh <[email protected]>:
>>>
>>> I am imitating the MemCachedState class found here:
>>>> https://github.com/nathanmarz/trident-memcached/blob/master/src/jvm/trident/memcached/MemcachedState.java
>>>> .
>>>>
>>>> I was wondering what the CachedMap is used for. I see that it creates a
>>>> cache layer between the final datastore in order to retrieve values much
>>>> quicker than accessing the datastore every time, however I am unsure about
>>>> a couple details.
>>>>
>>>> Is this cache replicated among all hosts? When I do a 'multiGet' I
>>>> expect to retrieve data I had previously stored. If the cache is specific
>>>> to each host, I wouldn't necessarily get the same data I had most recently
>>>> stored.
>>>>
>>>> Also, how does this work with Opaque Transactional consistency? It
>>>> seems that in the MemCachedState example we serialize the data once we
>>>> store it to the external datastore, however the data in the cache is not
>>>> serialized. why is this?
>>>> Shouldn't the local cache have the same data as the external datastore ?
>>>>
>>>> thanks
>>>> --
>>>> Raphael Hsieh
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>> --
>> Raphael Hsieh
>>
>>
>
>
> --
> Raphael Hsieh
> Amazon.com
> Software Development Engineer I
> (978) 764-9014
>
>
>
>
