[ 
https://issues.apache.org/jira/browse/IGNITE-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16725803#comment-16725803
 ] 

Andrey Gura commented on IGNITE-3269:
-------------------------------------

[~oleg-ostanin] What is the intention of benchmarking cluster when we have lost 
partitions? IMHO, ignoring situations like this is bad idea.

> Benchmark driver stops working when one of servers left grid and backupcount 0
> ------------------------------------------------------------------------------
>
>                 Key: IGNITE-3269
>                 URL: https://issues.apache.org/jira/browse/IGNITE-3269
>             Project: Ignite
>          Issue Type: Bug
>          Components: yardstick
>    Affects Versions: 1.7
>            Reporter: Ksenia Rybakova
>            Assignee: Oleg Ostanin
>            Priority: Critical
>         Attachments: failed_client_log.zip
>
>
> Test config: 4 hosts with 4 clients and 1 server each, backup count = 0.
> During the test one of the servers was killed. After that one of the clients
> stopped working:
> {noformat}
> [15:31:45,248][WARN ][sys-#27%null%][TcpCommunicationSpi] Connect timed out 
> (consider increasing 'failureDetectionTimeout' configuration property) 
> [addr=fosters-216/10.20.0.216:47100, failureDetectionTimeout=10000]
> [15:31:45,256][WARN ][sys-#27%null%][TcpCommunicationSpi] Failed to connect 
> to a remote node (make sure that destination node is alive and operating 
> system firewall is disabled on local and remote hosts) 
> [addrs=[fosters-216/10.20.0.216:4
> [15:31:45,276][WARN ][disco-event-worker-#52%null%][GridDiscoveryManager] 
> Node FAILED: TcpDiscoveryNode [id=f5bcc4d3-aa56-4e51-8672-e799a42990a6, 
> addrs=[10.20.0.216, 127.0.0.1], sockAddrs=[fosters-216/10.20.0.216:47500, 
> /10.20.0.216:4750
> [15:31:45,324][INFO ][disco-event-worker-#52%null%][GridDiscoveryManager] 
> Topology snapshot [ver=22, servers=3, clients=16, CPUs=64, heap=150.0GB]
> ERROR: The benchmark of random operation failed.
> Type '--help' for usage.
> Finishing main test [ts=1465252305413, date=Mon Jun 06 15:31:45 PDT 2016]
> [15:31:45] (err) Failed to execute compound future reducer: 
> GridNearTxFinishFuture 
> [futId=9c7413d2551-758034b3-0248-4d62-b901-be86d119183c, tx=GridNearTxLocal 
> [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, colocatedLocallyMa
> Type '--help' for usage.
> [15:31:45] (err) Failed to execute compound future reducer: 
> GridNearTxFinishFuture 
> [futId=6f7413d2551-758034b3-0248-4d62-b901-be86d119183c, tx=GridNearTxLocal 
> [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, colocatedLocallyMa
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:700)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:644)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:607)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:135)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:133)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:624)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:322)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:246)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:83)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:205)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1219)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:847)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:105)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:810)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> [15:31:45] (err) Failed to execute compound future reducer: 
> GridNearTxFinishFuture 
> [futId=ff7413d2551-758034b3-0248-4d62-b901-be86d119183c, tx=GridNearTxLocal 
> [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, colocatedLocallyMa
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:700)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:644)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:607)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:135)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:133)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:624)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:322)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:246)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:83)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:205)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1219)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:847)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:105)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:810)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> org.apache.ignite.cache.CacheServerNotFoundException: Failed to map keys for 
> cache (all partition nodes left the grid) [topVer=AffinityTopologyVersion 
> [topVer=21, minorTopVer=0], cache=tx-offheap]
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1468)
>         at 
> org.apache.ignite.internal.processors.cache.IgniteCacheProxy.cacheException(IgniteCacheProxy.java:1972)
>         at 
> org.apache.ignite.internal.processors.cache.IgniteCacheProxy.get(IgniteCacheProxy.java:914)
>         at 
> org.apache.ignite.yardstick.cache.load.IgniteCacheRandomOperationBenchmark.doGet(IgniteCacheRandomOperationBenchmark.java:670)
>         at 
> org.apache.ignite.yardstick.cache.load.IgniteCacheRandomOperationBenchmark.executeRandomOperation(IgniteCacheRandomOperationBenchmark.java:518)
>         at 
> org.apache.ignite.yardstick.cache.load.IgniteCacheRandomOperationBenchmark.access$900(IgniteCacheRandomOperationBenchmark.java:83)
>         at 
> org.apache.ignite.yardstick.cache.load.IgniteCacheRandomOperationBenchmark$2.call(IgniteCacheRandomOperationBenchmark.java:628)
>         at 
> org.apache.ignite.yardstick.IgniteBenchmarkUtils.doInTransaction(IgniteBenchmarkUtils.java:52)
>         at 
> org.apache.ignite.yardstick.cache.load.IgniteCacheRandomOperationBenchmark.executeInTransaction(IgniteCacheRandomOperationBenchmark.java:619)
>         at 
> org.apache.ignite.yardstick.cache.load.IgniteCacheRandomOperationBenchmark.test(IgniteCacheRandomOperationBenchmark.java:153)
>         at 
> org.yardstickframework.impl.BenchmarkRunner$2.run(BenchmarkRunner.java:176)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: class 
> org.apache.ignite.internal.cluster.ClusterTopologyServerNotFoundException: 
> Failed to map keys for cache (all partition nodes left the grid) 
> [topVer=AffinityTopologyVersion [topVer=21, minorTopVer=0], cache=tx-offheap]
>         at 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.serverNotFoundError(GridPartitionedSingleGetFuture.java:660)
>         at 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.mapKeyToNode(GridPartitionedSingleGetFuture.java:343)
>         at 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.map(GridPartitionedSingleGetFuture.java:204)
>         at 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.access$300(GridPartitionedSingleGetFuture.java:66)
>         at 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture$4.run(GridPartitionedSingleGetFuture.java:723)
>         at 
> org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6515)
>         at 
> org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:803)
>         at 
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         ... 1 more
> class org.apache.ignite.IgniteCheckedException: Transaction has been already 
> completed.
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:700)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:644)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:607)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:135)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:133)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:624)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:322)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:246)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:83)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:205)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1219)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:847)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:105)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:810)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> [15:31:45] (err) Failed to execute compound future reducer: 
> GridNearTxFinishFuture 
> [futId=ff7413d2551-758034b3-0248-4d62-b901-be86d119183c, tx=GridNearTxLocal 
> [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, colocatedLocallyMa
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:700)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:644)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:607)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:135)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:133)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:624)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:322)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:246)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:83)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:205)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1219)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:847)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:105)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:810)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> class org.apache.ignite.IgniteCheckedException: Transaction has been already 
> completed.
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:700)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:644)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:607)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:135)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:133)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:624)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:322)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:246)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:83)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:205)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1219)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:847)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:105)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:810)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> [15:31:45] (err) Failed to execute compound future reducer: 
> GridNearTxFinishFuture 
> [futId=558413d2551-758034b3-0248-4d62-b901-be86d119183c, tx=GridNearTxLocal 
> [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, colocatedLocallyMa
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:700)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:644)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:607)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:135)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:133)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:624)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:322)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:246)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:83)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:205)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1219)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:847)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:105)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:810)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> [15:31:45] (err) Failed to execute compound future reducer: 
> GridNearTxFinishFuture 
> [futId=ff7413d2551-758034b3-0248-4d62-b901-be86d119183c, tx=GridNearTxLocal 
> [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, colocatedLocallyMa
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:700)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:644)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:607)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:135)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:133)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:624)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:322)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:246)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:83)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:205)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1219)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:847)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:105)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:810)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> class org.apache.ignite.IgniteCheckedException: Transaction has been already 
> completed.
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:700)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:644)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:607)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:135)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:133)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:624)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:322)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:246)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:83)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:205)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1219)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:847)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:105)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:810)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> [15:31:45] (err) Failed to execute compound future reducer: 
> GridNearTxFinishFuture 
> [futId=558413d2551-758034b3-0248-4d62-b901-be86d119183c, tx=GridNearTxLocal 
> [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, colocatedLocallyMa
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:700)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:644)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:607)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:135)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:133)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:624)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:322)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:246)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:83)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:205)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1219)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:847)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:105)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:810)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> [15:31:45] (err) Failed to execute compound future reducer: 
> GridNearTxFinishFuture 
> [futId=ff7413d2551-758034b3-0248-4d62-b901-be86d119183c, tx=GridNearTxLocal 
> [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, colocatedLocallyMa
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:700)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:644)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:607)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:135)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:133)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:624)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:322)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:246)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:83)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:205)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1219)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:847)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:105)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:810)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> class org.apache.ignite.IgniteCheckedException: Transaction has been already 
> completed.
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:700)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:644)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:607)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:135)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:133)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:624)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:322)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:246)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:83)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:205)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1219)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:847)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:105)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:810)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> [15:31:45] (err) Failed to execute compound future reducer: 
> GridNearTxFinishFuture 
> [futId=558413d2551-758034b3-0248-4d62-b901-be86d119183c, tx=GridNearTxLocal 
> [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, colocatedLocallyMa
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:700)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:644)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:607)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:135)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:133)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:624)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:322)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:246)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:83)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:205)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1219)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:847)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:105)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:810)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> [15:31:45] (err) Failed to execute compound future reducer: 
> GridNearTxFinishFuture 
> [futId=ff7413d2551-758034b3-0248-4d62-b901-be86d119183c, tx=GridNearTxLocal 
> [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, colocatedLocallyMa
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:700)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:644)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:607)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:135)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:133)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:624)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:322)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:246)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:83)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:205)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1219)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:847)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:105)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:810)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> class org.apache.ignite.IgniteCheckedException: Transaction has been already 
> completed.
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:700)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:644)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:607)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:135)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:133)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:624)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:322)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:246)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:83)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:205)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1219)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:847)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:105)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:810)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> [15:31:45] (err) Failed to execute compound future reducer: 
> GridNearTxFinishFuture 
> [futId=558413d2551-758034b3-0248-4d62-b901-be86d119183c, tx=GridNearTxLocal 
> [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, colocatedLocallyMa
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:700)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:644)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:607)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:135)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:133)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:624)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:322)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:246)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:83)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:205)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1219)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:847)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:105)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:810)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> [15:31:45] (err) Failed to execute compound future reducer: 
> GridNearTxFinishFuture 
> [futId=ff7413d2551-758034b3-0248-4d62-b901-be86d119183c, tx=GridNearTxLocal 
> [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, colocatedLocallyMa
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:700)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:644)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:607)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:135)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:133)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:624)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:322)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:246)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:83)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:205)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1219)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:847)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:105)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:810)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> class org.apache.ignite.IgniteCheckedException: Transaction has been already 
> completed.
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:700)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:644)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:607)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:135)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:133)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:624)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:322)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:246)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:83)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:205)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1219)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:847)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:105)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:810)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> [15:31:45] (err) Failed to execute compound future reducer: 
> GridNearTxFinishFuture 
> [futId=558413d2551-758034b3-0248-4d62-b901-be86d119183c, tx=GridNearTxLocal 
> [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, colocatedLocallyMa
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:700)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:644)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:607)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:135)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:133)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:624)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:322)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:246)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:83)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:205)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1219)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:847)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:105)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:810)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> [15:31:45] (err) Failed to execute compound future reducer: 
> GridNearTxFinishFuture 
> [futId=ff7413d2551-758034b3-0248-4d62-b901-be86d119183c, tx=GridNearTxLocal 
> [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, colocatedLocallyMa
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:700)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:644)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:607)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:135)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:133)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:624)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:322)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:246)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:83)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:205)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1219)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:847)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:105)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:810)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> class org.apache.ignite.IgniteCheckedException: Transaction has been already 
> completed.
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:700)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:644)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:607)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:135)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:133)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:624)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:322)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:246)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:83)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:205)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1219)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:847)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:105)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:810)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> [15:31:45] (err) Failed to execute compound future reducer: 
> GridNearTxFinishFuture 
> [futId=ff7413d2551-758034b3-0248-4d62-b901-be86d119183c, tx=GridNearTxLocal 
> [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, colocatedLocallyMa
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:700)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:644)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:607)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:135)
>         at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:133)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:624)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:322)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:246)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:83)
>         at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:205)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1219)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:847)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:105)
>         at 
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:810)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> ...
> {noformat}
> Complete log is attached.
> Expected behaviour: client prints some error message in log, but continues 
> performing the test.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to