Hello! Do you have persistence? If so, are you sure that all 3 of your nodes are in baseline topology?
Regards, -- Ilya Kasnacheev чт, 30 апр. 2020 г. в 16:09, Courtney Robinson <[email protected]>: > We're continuing migration from using the Java API to purley SQL and have > encountered a situation on our development cluster where even though ALL > tables are created with backups=2, as in > > template=partitioned,backups=2,affinity_key=instanceId,atomicity=ATOMIC,cache_name=<some > name here> > > In the logs, with 3 nodes in this test environment we have: > > 2020-04-29 22:55:50.083 INFO 9 --- [orker-#40%hypi%] >> o.apache.ignite.internal.exchange.time : Started exchange init >> [topVer=AffinityTopologyVersion [topVer=27, minorTopVer=1], crd=true, >> evt=DISCOVERY_CUSTOM_EVT, evtNode=e0b6889f-219b-4686-ab52-725bfe7848b2, >> customEvt=DynamicCacheChangeBatch >> [id=a81a0e7c171-3f0fbbc0-b996-448c-98f7-119d7e485f04, reqs=ArrayList >> [DynamicCacheChangeRequest [cacheName=hypi_whatsapp_Item, hasCfg=true, >> nodeId=e0b6889f-219b-4686-ab52-725bfe7848b2, clientStartOnly=false, >> stop=false, destroy=false, disabledAfterStartfalse]], >> exchangeActions=ExchangeActions [startCaches=[hypi_whatsapp_Item], >> stopCaches=null, startGrps=[hypi_whatsapp_Item], stopGrps=[], >> resetParts=null, stateChangeRequest=null], startCaches=false], >> allowMerge=false, exchangeFreeSwitch=false] >> 2020-04-29 22:55:50.280 INFO 9 --- [orker-#40%hypi%] >> o.a.i.i.p.cache.GridCacheProcessor : Started cache >> [name=hypi_whatsapp_Item, id=1391701259, dataRegionName=hypi, >> mode=PARTITIONED, atomicity=ATOMIC, backups=2, mvcc=false] >> 2020-04-29 22:55:50.289 INFO 9 --- [ sys-#648%hypi%] >> o.a.i.i.p.a.GridAffinityAssignmentCache : Local node affinity assignment >> distribution is not ideal [cache=hypi_whatsapp_Item, >> expectedPrimary=1024.00, actualPrimary=0, expectedBackups=2048.00, >> actualBackups=0, warningThreshold=50.00%] >> 2020-04-29 22:55:50.293 INFO 9 --- [orker-#40%hypi%] >> .c.d.d.p.GridDhtPartitionsExchangeFuture : Finished waiting for partition >> release future [topVer=AffinityTopologyVersion [topVer=27, minorTopVer=1], >> waitTime=0ms, futInfo=NA, mode=DISTRIBUTED] >> 2020-04-29 22:55:50.330 INFO 9 --- [orker-#40%hypi%] >> .c.d.d.p.GridDhtPartitionsExchangeFuture : Finished waiting for partitions >> release latch: ServerLatch [permits=0, pendingAcks=HashSet [], >> super=CompletableLatch [id=CompletableLatchUid [id=exchange, >> topVer=AffinityTopologyVersion [topVer=27, minorTopVer=1]]]] > > > You can see the line > > Local node affinity assignment distribution is not ideal > > > but it's clear they the backup = 2 is there. To verify, I stoped 2 of the > three nodes and sure enough I get the exception > > Failed to find data nodes for cache: InstanceMapping > > > Is there some additional configuration needed for partitioned SQL caches > to have the backups as configured? > Until now we used the Java API with put/get and didn't have an issue with > backups. > > Full exception below: > > org.apache.ignite.cache.CacheServerNotFoundException: Failed to find data > nodes for cache: InstanceMapping >> at >> org.apache.ignite.internal.processors.query.h2.twostep.ReducePartitionMapper.stableDataNodes(ReducePartitionMapper.java:197) >> at >> org.apache.ignite.internal.processors.query.h2.twostep.ReducePartitionMapper.nodesForPartitions(ReducePartitionMapper.java:119) >> at >> org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor.query(GridReduceQueryExecutor.java:466) >> at >> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$7.iterator(IgniteH2Indexing.java:1687) >> at >> org.apache.ignite.internal.processors.cache.QueryCursorImpl.iter(QueryCursorImpl.java:106) >> at >> org.apache.ignite.internal.processors.cache.query.RegisteredQueryCursor.iter(RegisteredQueryCursor.java:66) >> at >> org.apache.ignite.internal.processors.cache.QueryCursorImpl.iterator(QueryCursorImpl.java:96) >> at io.hypi.arc.os.ignite.IgniteRepo.findInstanceCtx(IgniteRepo.java:140) >> at io.hypi.arc.os.handlers.BaseHandler.evaluateQuery(BaseHandler.java:70) >> at io.hypi.arc.os.handlers.HttpHandler.runQuery(HttpHandler.java:141) >> at io.hypi.arc.os.handlers.HttpHandler.graphql(HttpHandler.java:135) >> at jdk.internal.reflect.GeneratedMethodAccessor106.invoke(Unknown Source) >> at >> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.base/java.lang.reflect.Method.invoke(Method.java:566) >> at >> org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190) >> at >> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138) >> at >> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104) >> at >> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:892) >> at >> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797) >> at >> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) >> at >> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1039) >> at >> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942) >> at >> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005) >> at >> org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:908) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:660) >> at >> org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) >> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) >> at >> io.hypi.arc.os.config.CorsConfiguration$1.doFilter(CorsConfiguration.java:60) >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) >> at >> org.springframework.boot.actuate.web.trace.servlet.HttpTraceFilter.doFilterInternal(HttpTraceFilter.java:88) >> at >> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:118) >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) >> at >> org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:114) >> at >> org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:104) >> at >> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:118) >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) >> at >> org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) >> at >> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:118) >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) >> at >> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) >> at >> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) >> at >> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:526) >> at >> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) >> at >> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) >> at >> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) >> at >> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) >> at >> org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408) >> at >> org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) >> at >> org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:860) >> at >> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1587) >> at >> org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) >> at >> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) >> at >> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) >> at >> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) >> at java.base/java.lang.Thread.run(Thread.java:834) >> >> > Regards, > Courtney Robinson > Founder and CEO, Hypi > Tel: ++44 208 123 2413 (GMT+0) <https://hypi.io> > https://hypi.io >
