Semen Boikov created IGNITE-1275:
------------------------------------
Summary: 'Get' from marshaller cache hangs on topology change
Key: IGNITE-1275
URL: https://issues.apache.org/jira/browse/IGNITE-1275
Project: Ignite
Issue Type: Bug
Components: cache
Reporter: Semen Boikov
Assignee: Alexey Goncharuk
Priority: Critical
Fix For: ignite-1.4
Observer this hang on TC in test
IgniteCachePutRetryTransactionalSelfTest.testExplicitTransactionRetries:
{noformat}
[11:27:17] : [org.apache.ignite:ignite-core] Thread
[name="ignite-#91721%sys-dht.IgniteCachePutRetryTransactionalSelfTest2%",
id=115833, state=WAITING, blockCnt=0, waitCnt=3]
[11:27:17] : [org.apache.ignite:ignite-core] Lock
[object=o.a.i.i.processors.cache.distributed.dht.GridPartitionedGetFuture@4a555b67,
ownerName=null, ownerId=-1]
[11:27:17] : [org.apache.ignite:ignite-core] at
sun.misc.Unsafe.park(Native Method)
[11:27:17] : [org.apache.ignite:ignite-core] at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[11:27:17] : [org.apache.ignite:ignite-core] at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
[11:27:17] : [org.apache.ignite:ignite-core] at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
[11:27:17] : [org.apache.ignite:ignite-core] at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.i.util.future.GridFutureAdapter.get(GridFutureAdapter.java:115)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.i.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4404)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.i.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:1448)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.i.MarshallerContextImpl.className(MarshallerContextImpl.java:139)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.i.MarshallerContextAdapter.getClass(MarshallerContextAdapter.java:166)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.marshaller.optimized.OptimizedMarshallerUtils.classDescriptor(OptimizedMarshallerUtils.java:252)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:246)
[11:27:17] : [org.apache.ignite:ignite-core] at
java.io.ObjectInputStream.readObject(ObjectInputStream.java:364)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.lang.IgniteBiTuple.readExternal(IgniteBiTuple.java:268)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:451)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:746)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:252)
[11:27:17] : [org.apache.ignite:ignite-core] at
java.io.ObjectInputStream.readObject(ObjectInputStream.java:364)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.marshaller.optimized.OptimizedObjectInputStream.readLinkedList(OptimizedObjectInputStream.java:610)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:227)
[11:27:17] : [org.apache.ignite:ignite-core] at
java.io.ObjectInputStream.readObject(ObjectInputStream.java:364)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.marshaller.optimized.OptimizedMarshaller.unmarshal(OptimizedMarshaller.java:236)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.i.processors.cache.transactions.IgniteTxEntry.unmarshal(IgniteTxEntry.java:749)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.i.processors.cache.GridCacheMessage.unmarshalTx(GridCacheMessage.java:367)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.i.processors.cache.distributed.GridDistributedTxPrepareRequest.finishUnmarshal(GridDistributedTxPrepareRequest.java:300)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.i.processors.cache.distributed.dht.GridDhtTxPrepareRequest.finishUnmarshal(GridDhtTxPrepareRequest.java:293)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.i.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:973)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.i.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:229)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.i.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:158)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.i.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:48)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.i.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:127)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.i.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:764)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.i.managers.communication.GridIoManager.access$1500(GridIoManager.java:59)
[11:27:17] : [org.apache.ignite:ignite-core] at
o.a.i.i.managers.communication.GridIoManager$5.run(GridIoManager.java:727)
[11:27:17] : [org.apache.ignite:ignite-core] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[11:27:17] : [org.apache.ignite:ignite-core] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[11:27:17] : [org.apache.ignite:ignite-core] at
java.lang.Thread.run(Thread.java:722)
{noformat}
GridPartitionedGetFuture hangs when GridDistributedTxPrepareRequest is
unmarshalled. Looks like for marshaller cache we need implement special 'get'
mode which should not wait for topology futures (similar to 'tryPutIfAbsent'
used for put in marshaller cache).
Found one more issue debugging this failure: test uses only server nodes and
since marshaller cache uses FULL_SYNC mode, value in marshaller cache should be
available locally and there should not be necessary to create
GridPartitionedGetFuture. But it seems FULL_SYNC mode is broken for
'putIfAbsent' in atomic cache.
If two threads do 'putIfAbsent' concurrenly:
- thread1 on primary node executes 'innerUpdate', creates entry, sets value and
going to send messages to backups
- thread2 on primary executes 'innerUpdate', filter fails since value already
set by thread1, when filter fails backups are not updated, thread2 exits from
'putIfAbsent' but thread1 did not update backups yet
Also need to check case when GridDhtAtomicUpdateFuture is completed with error
during node stop, looks like error is ignored in
GridDhtAtomicUpdateFuture.onDone.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
