Now, I tried pausing replace method at before said point for just 1sec and It throws below error.
So in production if this sequence occurs, node1->is executing replace and in between node2->executed remove, what be the outcome would replace method return 'false' because it did not succeed or throw exception? Jan 29, 2020 5:40:17 PM org.apache.ignite.logger.java.JavaLogger error SEVERE: Blocked system-critical thread has been detected. This can lead to cluster-wide undefined behaviour [threadName=grid-nio-worker-tcp-comm-2, blockedFor=44s] Jan 29, 2020 5:40:17 PM java.util.logging.LogManager$RootLogger log SEVERE: Critical system error detected. Will be handled accordingly to configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=grid-nio-worker-tcp-comm-2, igniteInstanceName=null, finished=false, heartbeatTs=1580299773918]]] class org.apache.ignite.IgniteException: GridWorker [name=grid-nio-worker-tcp-comm-2, igniteInstanceName=null, finished=false, heartbeatTs=1580299773918] at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826) at org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233) at org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297) at org.apache.ignite.spi.discovery.tcp.ServerImpl$TcpServer.body(ServerImpl.java:5871) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at org.apache.ignite.spi.discovery.tcp.ServerImpl$TcpServerThread.body(ServerImpl.java:5763) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) Jan 29, 2020 5:40:18 PM java.util.logging.LogManager$RootLogger log SEVERE: Critical system error detected. Will be handled accordingly to configured handler [hnd=StopNodeFailureHandler [super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=SEGMENTATION, err=null]] Jan 29, 2020 5:40:18 PM org.apache.ignite.logger.java.JavaLogger error SEVERE: Ignite node is in invalid state due to a critical failure. Jan 29, 2020 5:40:18 PM java.util.logging.LogManager$RootLogger log SEVERE: Stopping local node on Ignite failure: [failureCtx=FailureContext [type=SEGMENTATION, err=null]] On Wed, Jan 29, 2020 at 4:46 PM Stephen Darlington < stephen.darling...@gridgain.com> wrote: > That is an error you get if the JVM pauses for too long. Most often this > is a sign of poorly configured garbage collection but in this case is > almost certainly because you stopped the node in the debugger. This is not > caused by the use of replace or clear. > > On 29 Jan 2020, at 07:22, Narsi Reddy Nallamilli < > narsi.nallami...@gmail.com> wrote: > > Hi Stephen, > > Yes, you are correct. I meant replace and remove. > > I tried to see what happens when we remove the key when replace is in > progress and node resulted in SEVERE error and node is blocked. > > I stopped replace method in between after it determined that key exists at > line 613, GridDhtAtomicCache , ignite 2.7.6 and then executed removed and > then resumed replace method ti threw below error. > > I was expecting it threw some relevant exception. It is a bug? > > Jan 29, 2020 12:44:04 PM org.apache.ignite.logger.java.JavaLogger error > SEVERE: Blocked system-critical thread has been detected. This can lead to > cluster-wide undefined behaviour [threadName=partition-exchanger, > blockedFor=12s] > Jan 29, 2020 12:44:04 PM java.util.logging.LogManager$RootLogger log > SEVERE: Critical system error detected. Will be handled accordingly to > configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, > timeout=0, super=AbstractFailureHandler > [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED, > SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext > [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker > [name=partition-exchanger, igniteInstanceName=null, finished=false, > heartbeatTs=1580282031672]]] > class org.apache.ignite.IgniteException: GridWorker > [name=partition-exchanger, igniteInstanceName=null, finished=false, > heartbeatTs=1580282031672] > at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831) > at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826) > at > org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233) > at > org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297) > at > org.apache.ignite.internal.processors.timeout.GridTimeoutProcessor$TimeoutWorker.body(GridTimeoutProcessor.java:221) > at > org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) > at java.lang.Thread.run(Thread.java:748) > > Jan 29, 2020 12:44:04 PM org.apache.ignite.logger.java.JavaLogger error > SEVERE: Blocked system-critical thread has been detected. This can lead to > cluster-wide undefined behaviour [threadName=grid-nio-worker-tcp-comm-1, > blockedFor=12s] > Jan 29, 2020 12:44:04 PM java.util.logging.LogManager$RootLogger log > SEVERE: Critical system error detected. Will be handled accordingly to > configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, > timeout=0, super=AbstractFailureHandler > [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED, > SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext > [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker > [name=grid-nio-worker-tcp-comm-1, igniteInstanceName=null, finished=false, > heartbeatTs=1580282031672]]] > class org.apache.ignite.IgniteException: GridWorker > [name=grid-nio-worker-tcp-comm-1, igniteInstanceName=null, finished=false, > heartbeatTs=1580282031672] > at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831) > at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826) > at > org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233) > at > org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297) > at > org.apache.ignite.internal.processors.timeout.GridTimeoutProcessor$TimeoutWorker.body(GridTimeoutProcessor.java:221) > at > org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) > at java.lang.Thread.run(Thread.java:748) > > On Tue, Jan 28, 2020 at 6:49 PM Stephen Darlington < > stephen.darling...@gridgain.com> wrote: > >> First, I assume you mean remove rather than clear? Clear removes all >> entries in the cache (and takes no parameters). >> >> With that, yes, your sequence of events could happen. >> >> There are (at least) two alternative ways of doing it. >> >> “Put” doesn’t “check that a value already exists.” It simply puts the >> value in the cache, regardless of whether a value exists or not. If you >> want to replace an existing value, you should use the “replace” method. >> >> Alternatively, you could use transactions. >> >> Regards, >> Stephen >> >> > On 28 Jan 2020, at 13:01, Narsi Reddy Nallamilli < >> narsi.nallami...@gmail.com> wrote: >> > >> > Hello, >> > >> > Consider I have below cache on server node >> > >> > Cache >> > ----------- >> > key,value >> > ------------ >> > 1 | a >> > ------------ >> > >> > From client node1, I try to do put(1,a) and from client node2 I try to >> do clear(1). >> > >> > Are put and clear atomic, mean could this happen? >> > >> > 1. cliennode1 put operation identified that key with 1 exist and it >> decided to update the value. >> > 2. clientnode2 cleared the entry from cache >> > 3. clientnode1 put operation now tries to update the entry with key 1 >> > >> > >> >> >> > >
