Hi Alper, Stripe pool threads do message processing. In stacktrace, I see node can send message to other node due to network issue or node failure. Thread try to reconnect to the node during failure detection timeout.
What Ignite version do you use? Would you please share full logs? On Fri, Jul 14, 2017 at 1:24 PM, Alper Tekinalp <[email protected]> wrote: > Hi. > > What does following log means: > > [WARN ] 2017-07-12 23:00:50.786 [grid-timeout-worker-#71%cache-server%] G > - >>> Possible starvation in striped pool: sys-stripe-10-#11%cache-server% > [Message closure [msg=GridIoMessage [plc=2, topic=TOPIC_CACHE, topicOrd=8, > ordered=false, timeout=0, skipOnTimeout=false, msg=GridNearSingleGetRequest > .... > 43f6-946a-6593f8dcde7d, taskNameHash=0, createTtl=0, accessTtl=-1]]]] > deadlock: false > completed: 699284 > Thread [name="sys-stripe-10-#11%cache-server%", id=44, state=WAITING, > blockCnt=18197, waitCnt=642544] > Lock [object=o.a.i.spi.communication.tcp.TcpCommunicationSpi$ > ConnectFuture@604f22f, ownerName=null, ownerId=-1] > at sun.misc.Unsafe.park(Native Method) > at java.util.concurrent.locks.LockSupport.park(LockSupport. > java:186) > at java.util.concurrent.locks.AbstractQueuedSynchronizer. > parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834) > at java.util.concurrent.locks.AbstractQueuedSynchronizer. > doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994) > at java.util.concurrent.locks.AbstractQueuedSynchronizer. > acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) > at o.a.i.i.util.future.GridFutureAdapter.get0( > GridFutureAdapter.java:161) > at o.a.i.i.util.future.GridFutureAdapter.get( > GridFutureAdapter.java:119) > at o.a.i.spi.communication.tcp.TcpCommunicationSpi.reserveClient( > TcpCommunicationSpi.java:2515) > at o.a.i.spi.communication.tcp.TcpCommunicationSpi.sendMessage0( > TcpCommunicationSpi.java:2340) > at o.a.i.spi.communication.tcp.TcpCommunicationSpi.sendMessage( > TcpCommunicationSpi.java:2304) > at o.a.i.i.managers.communication.GridIoManager. > send(GridIoManager.java:1281) > at o.a.i.i.managers.communication.GridIoManager. > sendOrderedMessage(GridIoManager.java:1517) > at o.a.i.i.processors.continuous.GridContinuousProcessor. > sendWithRetries(GridContinuousProcessor.java:1335) > at o.a.i.i.processors.continuous.GridContinuousProcessor. > sendWithRetries(GridContinuousProcessor.java:1306) > at o.a.i.i.processors.continuous.GridContinuousProcessor. > sendWithRetries(GridContinuousProcessor.java:1288) > at o.a.i.i.processors.continuous.GridContinuousProcessor. > sendNotification(GridContinuousProcessor.java:949) > at o.a.i.i.processors.continuous.GridContinuousProcessor. > addNotification(GridContinuousProcessor.java:892) > at o.a.i.i.processors.cache.query.continuous. > CacheContinuousQueryHandler.onEntryUpdate(CacheContinuousQueryHandler. > java:814) > at o.a.i.i.processors.cache.query.continuous. > CacheContinuousQueryHandler.access$800(CacheContinuousQueryHandler. > java:91) > at o.a.i.i.processors.cache.query.continuous. > CacheContinuousQueryHandler$1$1.apply(CacheContinuousQueryHandler. > java:426) > at o.a.i.i.processors.cache.query.continuous. > CacheContinuousQueryHandler$1$1.apply(CacheContinuousQueryHandler. > java:421) > at o.a.i.i.processors.cache.distributed.dht.atomic. > GridDhtAtomicAbstractUpdateFuture.onDone(GridDhtAtomicAbstractUpdateFut > ure.java:450) > at o.a.i.i.processors.cache.distributed.dht.atomic. > GridDhtAtomicAbstractUpdateFuture.onDone(GridDhtAtomicAbstractUpdateFut > ure.java:56) > at o.a.i.i.util.future.GridFutureAdapter.onDone( > GridFutureAdapter.java:321) > at o.a.i.i.processors.cache.distributed.dht.atomic. > GridDhtAtomicAbstractUpdateFuture.registerResponse( > GridDhtAtomicAbstractUpdateFuture.java:343) > at o.a.i.i.processors.cache.distributed.dht.atomic. > GridDhtAtomicAbstractUpdateFuture.onResult(GridDhtAtomicAbstractUpdateFut > ure.java:399) > at o.a.i.i.processors.cache.distributed.dht.atomic. > GridDhtAtomicCache.processDhtAtomicDeferredUpdate > Response(GridDhtAtomicCache.java:3388) > at o.a.i.i.processors.cache.distributed.dht.atomic. > GridDhtAtomicCache.access$2000(GridDhtAtomicCache.java:126) > at o.a.i.i.processors.cache.distributed.dht.atomic. > GridDhtAtomicCache$10.apply(GridDhtAtomicCache.java:412) > at o.a.i.i.processors.cache.distributed.dht.atomic. > GridDhtAtomicCache$10.apply(GridDhtAtomicCache.java:407) > at o.a.i.i.processors.cache.GridCacheIoManager.processMessage( > GridCacheIoManager.java:827) > at o.a.i.i.processors.cache.GridCacheIoManager.onMessage0( > GridCacheIoManager.java:369) > at o.a.i.i.processors.cache.GridCacheIoManager.handleMessage( > GridCacheIoManager.java:293) > at o.a.i.i.processors.cache.GridCacheIoManager.access$000( > GridCacheIoManager.java:95) > at o.a.i.i.processors.cache.GridCacheIoManager$1. > onMessage(GridCacheIoManager.java:238) > at o.a.i.i.managers.communication.GridIoManager. > invokeListener(GridIoManager.java:1215) > at o.a.i.i.managers.communication.GridIoManager. > processRegularMessage0(GridIoManager.java:843) > at o.a.i.i.managers.communication.GridIoManager. > access$2100(GridIoManager.java:108) > at o.a.i.i.managers.communication.GridIoManager$6. > run(GridIoManager.java:783) > at o.a.i.i.util.StripedExecutor$Stripe.run(StripedExecutor. > java:428) > at java.lang.Thread.run(Thread.java:745) > > > We encounter that log time to time specially when loading big partitioned > caches across the cluster. What does stripes threads do? Is there a way to > avoid that? > > Regards. > > -- > Alper Tekinalp > > Software Developer > Evam Streaming Analytics > > Atatürk Mah. Turgut Özal Bulv. > Gardenya 5 Plaza K:6 Ataşehir > 34758 İSTANBUL > > Tel: +90 216 455 01 53 Fax: +90 216 455 01 54 > www.evam.com.tr > <http://www.evam.com> > -- Best regards, Andrey V. Mashenkov
