Kamal, You see such *warnings* on client nodes because Cassandra throws such exceptions *NoHostAvailableException *or *ReadTimeoutException.* It happens when Cassandra node you are trying to communicate with is overloaded. In a such situation Ignite-Cassandra module just tries to retry the operation which is failed. If it fails several time again and again, it will close Cassandra connection and reconnect again. In summary it will try 20 attempts to succeed *read/write/delete* operation (with incrementally increasing random sleep timeout) until completely realize that it can do nothing and just throwing exception.
Thus the warnings you see is just an indicator that some of your Cassandra nodes is overloaded and can't handle all client requests and Ignite-Cassandra module trying to do its best to complete your *read/write/delete* request. Igor On Tue, Aug 16, 2016 at 1:54 AM, Kamal C <[email protected]> wrote: > Igor, > > I'm facing this issue in the cassandra nodes side. I've added a > distributed lock (Ignite BinarySemaphore) in > the client to avoid the error as you've mentioned when multiple clients > start simultaneously. > > Sometimes, In client nodes, I face the below warnings: > > WARN [2016-07-30T10:11:08,092] CassandraCacheStore: warning(): Cassandra > session refreshed > WARN [2016-07-30T10:11:08,093] CassandraCacheStore: warning(): Prepared > statement cluster error detected, refreshing Cassandra session > com.datastax.driver.core.exceptions.InvalidQueryException: Tried to > execute unknown prepared query : 0x92523c1ca5cb89ede8b4e16b79b31c63. You > may have used a PreparedStatement that was created with another Cluster > instance. > at > com.datastax.driver.core.SessionManager.makeRequestMessage(SessionManager.java:568) > ~[cassandra-driver-core-3.0.0.jar:?] > at > com.datastax.driver.core.SessionManager.executeAsync(SessionManager.java:131) > ~[cassandra-driver-core-3.0.0.jar:?] > at org.apache.ignite.cache.store.cassandra.session.CassandraSes > sionImpl.execute(CassandraSessionImpl.java:225) > [ignite-cassandra-1.6.0.jar:1.6.0] > at org.apache.ignite.cache.store.cassandra.CassandraCacheStore. > deleteAll(CassandraCacheStore.java:354) [ignite-cassandra-1.6.0.jar:1.6.0] > at org.apache.ignite.internal.processors.cache.store.GridCacheS > toreManagerAdapter.removeAll(GridCacheStoreManagerAdapter.java:725) > [ignite-core-1.6.0.jar:1.6.0 > ] > at org.apache.ignite.internal.processors.cache.distributed.dht. > atomic.GridDhtAtomicCache.updatePartialBatch(GridDhtAtomicCache.java:2482) > [ignite-core-1.6.0.ja > r:1.6.0] > at org.apache.ignite.internal.processors.cache.distributed.dht. > atomic.GridDhtAtomicCache.updateWithBatch(GridDhtAtomicCache.java:2087) > [ignite-core-1.6.0.jar:1 > .6.0] > at org.apache.ignite.internal.processors.cache.distributed.dht. > atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1624) > [ignite-core-1.6 > .0.jar:1.6.0] > at org.apache.ignite.internal.processors.cache.distributed.dht. > atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1484) > [ignite-core-1.6. > 0.jar:1.6.0] > at org.apache.ignite.internal.processors.cache.distributed.dht. > atomic.GridNearAtomicUpdateFuture.mapSingle(GridNearAtomicUpdateFuture.java:555) > [ignite-core-1.6.0.jar:1.6.0] > at org.apache.ignite.internal.processors.cache.distributed.dht. > atomic.GridNearAtomicUpdateFuture.map(GridNearAtomicUpdateFuture.java:749) > [ignite-core-1.6.0.jar:1.6.0] > at org.apache.ignite.internal.processors.cache.distributed.dht. > atomic.GridNearAtomicUpdateFuture.mapOnTopology(GridNearAtomicUpdateFuture.java:544) > [ignite-core-1.6.0.jar:1.6.0] > at org.apache.ignite.internal.processors.cache.distributed.dht. > atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:202) > [ignite-core-1.6.0.jar:1.6.0] > at org.apache.ignite.internal.processors.cache.distributed.dht. > atomic.GridDhtAtomicCache$25.apply(GridDhtAtomicCache.java:1242) > [ignite-core-1.6.0.jar:1.6.0] > at org.apache.ignite.internal.processors.cache.distributed.dht. > atomic.GridDhtAtomicCache$25.apply(GridDhtAtomicCache.java:1240) > [ignite-core-1.6.0.jar:1.6.0] > at org.apache.ignite.internal.processors.cache.distributed.dht. > atomic.GridDhtAtomicCache.asyncOp(GridDhtAtomicCache.java:703) > [ignite-core-1.6.0.jar:1.6.0] > at org.apache.ignite.internal.processors.cache.distributed.dht. > atomic.GridDhtAtomicCache.removeAllAsync0(GridDhtAtomicCache.java:1240) > [ignite-core-1.6.0.jar:1.6.0] > at org.apache.ignite.internal.processors.cache.distributed.dht. > atomic.GridDhtAtomicCache.removeAllAsync(GridDhtAtomicCache.java:620) > [ignite-core-1.6.0.jar:1.6.0] > at org.apache.ignite.internal.processors.cache.distributed.dht. > atomic.GridDhtAtomicCache.removeAll(GridDhtAtomicCache.java:613) > [ignite-core-1.6.0.jar:1.6.0] > at org.apache.ignite.internal.processors.cache.IgniteCacheProxy > .removeAll(IgniteCacheProxy.java:1409) [ignite-core-1.6.0.jar:1.6.0] > at org.apache.ignite.internal.processors.datastreamer.DataStrea > merCacheUpdaters.updateAll(DataStreamerCacheUpdaters.java:93) > [ignite-core-1.6.0.jar:1.6.0] > at org.apache.ignite.internal.processors.datastreamer.DataStrea > merCacheUpdaters$Batched.receive(DataStreamerCacheUpdaters.java:162) > [ignite-core-1.6.0.jar:1.6.0] > at org.apache.ignite.internal.processors.datastreamer.DataStrea > merUpdateJob.call(DataStreamerUpdateJob.java:140) > [ignite-core-1.6.0.jar:1.6.0] > at > org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6459) > [ignite-core-1.6.0.jar:1.6.0] > at org.apache.ignite.internal.processors.closure.GridClosurePro > cessor$2.body(GridClosureProcessor.java:944) [ignite-core-1.6.0.jar:1.6.0] > at > org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) > [ignite-core-1.6.0.jar:1.6.0] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [?:1.7.0_40] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [?:1.7.0_40] > at java.lang.Thread.run(Thread.java:724) [?:1.7.0_40] > > Val, > > I've posted the same issue in the Cassandra forums > http://www.mail-archive.com/[email protected]/msg48370.html > > Regards, > Kamal C > > > On Sat, Aug 13, 2016 at 2:39 AM, Igor Rudyak <[email protected]> wrote: > >> Hi Kamal, >> >> Do you have this problem on Ignite-Cassandra module side or on Cassandra >> nodes side? If you have it on Ignite-Cassandra module side that could be >> because of this: https://issues.apache.org/jira/browse/CASSANDRA-8387. >> It sometimes happen when you start from clean Cassandra cluster without any >> keyspaces/tables to store your Ignite caches. >> >> In this situation Ignite-Cassandra module tries to create new >> keyspaces/tables in Cassandra and executes CQL statements like "create >> table .... if not exists". According to the Cassandra ticket above, when >> statement like "create table" is executed simultaneously from multiple >> clients (Ignite nodes in our case) it could cause such an error in the log. >> >> >> >> Regards, >> Igor Rudyak >> >> On Fri, Aug 12, 2016 at 12:20 PM, vkulichenko < >> [email protected]> wrote: >> >>> Hi Kamal, >>> >>> I would ask on the Cassandra forum about this. Can you do this? The >>> exception looks very weird to me, because it complains about the >>> mismatch, >>> but the IDs are actually equal: >>> >>> org.apache.cassandra.exceptions.ConfigurationException: Column family ID >>> mismatch (found b2b47bf0-5f94-11e6-867a-cb7beeeeb92c; expected >>> b2b1bcd0-5f94-11e6-867a-cb7beeeeb92c) >>> >>> -Val >>> >>> >>> >>> -- >>> View this message in context: http://apache-ignite-users.705 >>> 18.x6.nabble.com/Cassandra-Ignite-Exception-tp7013p7024.html >>> Sent from the Apache Ignite Users mailing list archive at Nabble.com. >>> >> >> >
