Re: [infinispan-dev] State transfer is transferring the same data twice
On 19 Jul 2011, at 17:17, Mircea Markus wrote: > can't you make StateTransferManagerImpl.applyInMemoryState() to always write > to the cache store and modify > StateTransferManagerImpl.generatePersistentState to skip sending a key if it > is already in the memory/DataContainer? > That would require passing a filter to CacheStore.toStrem… …. and potentially changing a lot of the cache store impls… -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] State transfer is transferring the same data twice
On 15 Jul 2011, at 20:00, Dan Berindei wrote: > Hi guys > > While fixing https://issues.jboss.org/browse/ISPN-1243 I found that in > certain cases state transfer will copy the in-memory data twice: first > when copying the in-memory data and then again while copying the > persistent data. > > I was getting duplicate key exceptions in > StateTransferManagerImpl.applyPersistentState(), so I changed > StateTransferManagerImpl.applyInMemoryState() to skip writing to the > cache store if persistent state transfer is enabled and passivation is > disabled. can't you make StateTransferManagerImpl.applyInMemoryState() to always write to the cache store and modify StateTransferManagerImpl.generatePersistentState to skip sending a key if it is already in the memory/DataContainer? That would require passing a filter to CacheStore.toStrem... > That seems to work, but I'm wondering if there are other > cases that I missed. > > For 5.1, since we're going to use the rehashing code, we can check if > a key is in memory and not include it again in the persistent state, > but that seems too intrusive to include in 5.0. WDYT? > > Cheers > Dan > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] State transfer is transferring the same data twice
On 15 Jul 2011, at 20:00, Dan Berindei wrote: > Hi guys > > While fixing https://issues.jboss.org/browse/ISPN-1243 I found that in > certain cases state transfer will copy the in-memory data twice: first > when copying the in-memory data and then again while copying the > persistent data. > > I was getting duplicate key exceptions in > StateTransferManagerImpl.applyPersistentState(), so I changed > StateTransferManagerImpl.applyInMemoryState() to skip writing to the > cache store if persistent state transfer is enabled and passivation is > disabled. That seems to work, but I'm wondering if there are other > cases that I missed. > > For 5.1, since we're going to use the rehashing code, we can check if > a key is in memory and not include it again in the persistent state, > but that seems too intrusive to include in 5.0. WDYT? Agreed - too intrusive for 5.0 now. -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
[infinispan-dev] State transfer is transferring the same data twice
Hi guys While fixing https://issues.jboss.org/browse/ISPN-1243 I found that in certain cases state transfer will copy the in-memory data twice: first when copying the in-memory data and then again while copying the persistent data. I was getting duplicate key exceptions in StateTransferManagerImpl.applyPersistentState(), so I changed StateTransferManagerImpl.applyInMemoryState() to skip writing to the cache store if persistent state transfer is enabled and passivation is disabled. That seems to work, but I'm wondering if there are other cases that I missed. For 5.1, since we're going to use the rehashing code, we can check if a key is in memory and not include it again in the persistent state, but that seems too intrusive to include in 5.0. WDYT? Cheers Dan ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
