Hi,
We have observed one issue with IGNITE today is , the client thread has been
waiting forever when it is trying to create a cache, inserting the data and
trying to know the size of the cache [ REPLICATED caches, backups value for
these caches is 2147483647 ]. We are getting this issue few times only, but
daily one or 2 times it is coming in our environment. Following are the
logs, can any one please check and suggest us.
Java Version : 1.7.0_45 [ we can't upgrade to newer version.]
Ignite Version : 1.8
Ignite server nodes = 3
Unique Ignite Client services = 6
cache configuration:
DS - Client Service 1:
Thread stack trace from thread dump When first time it is occurred:
-
when we are trying to sync the data from DS service to ignite server, thread
has been waiting from client is forever. Following is the stack trace from
the thread dump of this client service.
"pool-160-thread-1" prio=10 tid=0x7f39800bc800 nid=0x404e waiting on
condition [0x7f396f9f8000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00078ac134c8> (a
org.apache.ignite.internal.ComputeTaskInternalFuture)
at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:160)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:118)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.size(GridCacheAdapter.java:3833)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.size(IgniteCacheProxy.java:997)
at
com.pg.casino.service.datasync.DataSyncManager.clearFeedCaches(DataSyncManager.java:446)
at
com.pg.casino.service.datasync.DataSyncCallable.call(DataSyncCallable.java:119)
at
com.pg.casino.service.datasync.DataSyncCallable.call(DataSyncCallable.java:21)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Thread stack trace from thread dump When 2nd time it is occurred:
-
When it is occurred first time, we have restarted this DS client service and
again it has started to do the same task, but at that time it took 20 mins
of time to insert the 30 rows of data into ignite. Following are the logs of
DS service. We spawn 2 threads to from this client service and insert the
data into different caches in ignite. There is no other service will insert
or modify the data in ignite server other than this DS service.
pool-4-thread-2:
09:38:08,632 - INFO - pool-4-thread-2 - IG_LOGGER DSIgniteCacheManager- got
cache mapping for-bt
09:38:07,283 - INFO - pool-4-thread-2 - DS_LOGGER DataSyncManager->syncBL()
is started
thread stack trace in thread dump:
"pool-4-thread-2" prio=10 tid=0x00651800 nid=0x6619 waiting on
condition [0x7fd25e232000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x000790c4df90> (a
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicUpdateFuture)
at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:160)
at
