Hi John, Please find the full stack trace below.
[Thread 69] 2019-12-02 08:35:53.323 UTC+0000 com.actuate.iserver.services.ServiceBase.execute() WARNING: org.apache.geode.cache.client.ServerOperationException: remote server on GBRPSM020006568(10772:loner):62563:aebd35c5: While performing a remote get at org.apache.geode.cache.client.internal.AbstractOp.processObjResponse(AbstractOp.java:285) at org.apache.geode.cache.client.internal.GetOp$GetOpImpl.processResponse(GetOp.java:144) at org.apache.geode.cache.client.internal.AbstractOp.attemptReadResponse(AbstractOp.java:171) at org.apache.geode.cache.client.internal.AbstractOp.attempt(AbstractOp.java:382) at org.apache.geode.cache.client.internal.ConnectionImpl.execute(ConnectionImpl.java:266) at org.apache.geode.cache.client.internal.pooling.PooledConnection.execute(PooledConnection.java:332) at org.apache.geode.cache.client.internal.OpExecutorImpl.executeWithPossibleReAuthentication(OpExecutorImpl.java:900) at org.apache.geode.cache.client.internal.OpExecutorImpl.execute(OpExecutorImpl.java:158) at org.apache.geode.cache.client.internal.OpExecutorImpl.execute(OpExecutorImpl.java:115) at org.apache.geode.cache.client.internal.PoolImpl.execute(PoolImpl.java:737) at org.apache.geode.cache.client.internal.GetOp.execute(GetOp.java:92) at org.apache.geode.cache.client.internal.ServerRegionProxy.get(ServerRegionProxy.java:116) at org.apache.geode.internal.cache.LocalRegion.findObjectInSystem(LocalRegion.java:2834) at org.apache.geode.internal.cache.LocalRegion.nonTxnFindObject(LocalRegion.java:1485) at org.apache.geode.internal.cache.LocalRegionDataView.findObject(LocalRegionDataView.java:175) at org.apache.geode.internal.cache.LocalRegion.get(LocalRegion.java:1369) at org.apache.geode.internal.cache.LocalRegion.get(LocalRegion.java:1302) at org.apache.geode.internal.cache.LocalRegion.get(LocalRegion.java:1288) at org.apache.geode.internal.cache.AbstractRegion.get(AbstractRegion.java:273) at com.actuate.iserver.utils.cache.AuthIdCacheClient.getCachedObject(AuthIdCacheClient.java:95) at com.actuate.iserver.utils.CachedAuthIdProperties.getCachedUserProperties(CachedAuthIdProperties.java:95) at com.actuate.iserver.services.util.AuthenticationUtil.verifyTokenMappingRecord(AuthenticationUtil.java:100) at com.actuate.iserver.services.util.AuthenticationUtil.validateAndReturnAuthId(AuthenticationUtil.java:59) at com.actuate.iserver.services.util.ServiceProcessor.readHeader(ServiceProcessor.java:28) at com.actuate.iserver.services.EncycServiceBase.readHeader(EncycServiceBase.java:197) at com.actuate.iserver.services.EncycServiceBase.preExecute(EncycServiceBase.java:44) at com.actuate.iserver.services.ServiceBase.execute(ServiceBase.java:70) at com.actuate.iserver.services.api.idapi.actuate11.ActuateSoapBindingImpl.getVolumeProperties(ActuateSoapBindingImpl.java:226) at com.actuate.schemas.actuate11.wsdl.ActuateAPIMessageReceiverInOut.invokeBusinessLogic(ActuateAPIMessageReceiverInOut.java:2085) at com.actuate.iserver.services.api.idapi.actuate11.ActuateAPIMessageReceiver.invokeBusinessLogic(ActuateAPIMessageReceiver.java:29) at org.apache.axis2.receivers.AbstractInOutMessageReceiver.invokeBusinessLogic(AbstractInOutMessageReceiver.java:40) at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:114) at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:173) at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:173) at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:144) at com.actuate.webserver.NimbleAxis2Servlet.doPostWrapper(NimbleAxis2Servlet.java:88) at com.actuate.ihub.filter.LocalFilter.processRequest(LocalFilter.java:315) at com.actuate.ihub.filter.RequestFilter.doFilter(RequestFilter.java:269) at com.actuate.ihub.filter.RequestFilterChain.processFilter(RequestFilterChain.java:52) at com.actuate.ihub.filter.RequestFilterManager.processFilters(RequestFilterManager.java:264) at com.actuate.ihub.web.FilterServlet.doRequest(FilterServlet.java:153) at com.actuate.ihub.web.FilterServlet.doPost(FilterServlet.java:82) at javax.servlet.http.HttpServlet.service(HttpServlet.java:648) at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:94) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1132) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1539) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1495) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.geode.cache.RegionDestroyedException: Server connection from [identity(10.43.164.241(10772:loner):62563:aebd35c5,connection=1; port=62563]: Region named /AuthIdRegion/AuthIdRegion was not found during get request at org.apache.geode.internal.cache.tier.sockets.BaseCommand.writeRegionDestroyedEx(BaseCommand.java:633) at org.apache.geode.internal.cache.tier.sockets.command.Get70.cmdExecute(Get70.java:126) at org.apache.geode.internal.cache.tier.sockets.BaseCommand.execute(BaseCommand.java:141) at org.apache.geode.internal.cache.tier.sockets.ServerConnection.doNormalMsg(ServerConnection.java:783) at org.apache.geode.internal.cache.tier.sockets.ServerConnection.doOneMessage(ServerConnection.java:914) at org.apache.geode.internal.cache.tier.sockets.ServerConnection.run(ServerConnection.java:1171) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.geode.internal.cache.tier.sockets.AcceptorImpl$1$1.run(AcceptorImpl.java:519) ... 1 more Region creation is done using java code on server using the following method. public Region createRegion() { File file = new File(geodeConfigHome); if (!file.exists()) { if (file.mkdir()) { AC_LOGGER.infoN("geodeConfigHome : created "+ geodeConfigHome); } }else{ AC_LOGGER.infoN("geodeConfigHome : already exists "+ geodeConfigHome); } //Create Cache Cache cache = new CacheFactory().create(); ResourceManager rm = cache.getResourceManager(); rm.setCriticalHeapPercentage(m_heapCriticalPercentage); rm.setEvictionHeapPercentage(m_heapEvictionPercentage); //Set Diskstore attributes and create it. int maxOpLogSize = Integer.parseInt( configureManager.getLogFileSizeInMB( )); createDiskStore(cache, geodeConfigHome, userDetailsDataStoreFileName, maxOpLogSize); //Set RegionFactory attributes and create it. RegionFactory rf = cache.createRegionFactory(RegionShortcut.REPLICATE_PERSISTENT_OVERFLOW); rf.addCacheListener(new IHubUserPropertiesRegionListener()); rf.setEntryTimeToLive(new ExpirationAttributes(getAuthIdLifeLimit(),ExpirationAction.DESTROY)); rf.setEvictionAttributes(EvictionAttributes.createLRUEntryAttributes(configureManager.getMaxUserProprtiesCacheEntries( ), EvictionAction.OVERFLOW_TO_DISK)); rf.setDiskStoreName( userDetailsDataStoreFileName ); Region cacheRegionInstance = rf.create(regionName); return cacheRegionInstance; } public void createDiskStore(Cache cache, String geodeConfigHome, String diskStoreName, long maxOpLogSize) { AC_LOGGER.infoN("geodeConfigHome :"+ geodeConfigHome); AC_LOGGER.infoN("diskStoreName :"+ diskStoreName); DiskStoreFactory diskStoreFactory = cache.createDiskStoreFactory(); diskStoreFactory.setAllowForceCompaction( m_allowForceCompaction); diskStoreFactory.setMaxOplogSize(maxOpLogSize); diskStoreFactory.setAutoCompact(m_autoCompact); diskStoreFactory.setCompactionThreshold( m_compactionThreshold); diskStoreFactory.setQueueSize(m_queueSize); diskStoreFactory.setTimeInterval(m_timeInterval); diskStoreFactory.setDiskUsageCriticalPercentage( m_diskUsageCriticalPercentage); diskStoreFactory.setDiskUsageWarningPercentage( m_diskUsageWarningPercentage); File file = new File(geodeConfigHome); diskStoreFactory.setDiskDirs(new File[]{file}); diskStoreFactory.create(diskStoreName); } Thanks, Rajesh On Tue, Dec 3, 2019 at 11:40 PM John Blum <jb...@pivotal.io> wrote: > This is an old bug (which was most certainly present in Apache Geode 1.1 > since I think I remember seeing this bug all the way up through 1.3). > Technically, IIRC, there is no Region on the server by the name indicated > in the misleading error message, /AuthIdRegion/AuthIdRegion, it was > simply a problem with the error message. Off the top of my head, I cannot > remember what ultimately caused the RegionDestroyedException in this > case, though. > > Rajesh - is it possible for you to upgrade to Apache Geode 1.10, which is > now the latest version? You might still have an error/configuration > problem, but at least the error message should be more insightful. Please > include the full stack trace and error message along with how you > configured the Region on the server. > > -j > > > > On Tue, Dec 3, 2019 at 9:53 AM Dan Smith <dsm...@pivotal.io> wrote: > >> Huh, that's weird. Is it possible that somewhere in your code you are >> calling createSubregion on the returned authIdCacheRegionInstance? The >> syntax /AuthIdRegion/AuthIdRegion indicates a region named AuthIdRegion >> with a subregion also named AuthIdRegion. >> >> -Dan >> >> On Mon, Dec 2, 2019 at 8:04 PM Rajesh Kumar <rajeshkumar.ii...@gmail.com> >> wrote: >> >>> we are using apache geode 1.1.0 version. sometimes randomly while trying >>> to get details from a region we are getting exceptions. On looking at the >>> logs it seems the region is appended to itself and a region not found >>> exception is thrown. >>> >>> >>> >>> Caused by: org.apache.geode.cache.RegionDestroyedException: Server >>> connection from >>> [identity(10.43.164.241(10772:loner):62563:aebd35c5,connection=1; >>> port=62563]: Region named /AuthIdRegion/AuthIdRegion was not found during >>> get request >>> >>> at >>> org.apache.geode.internal.cache.tier.sockets.BaseCommand.writeRegionDestroyedEx(BaseCommand.java:633) >>> >>> at >>> org.apache.geode.internal.cache.tier.sockets.command.Get70.cmdExecute(Get70.java:126) >>> >>> at >>> org.apache.geode.internal.cache.tier.sockets.BaseCommand.execute(BaseCommand.java:141) >>> >>> at >>> org.apache.geode.internal.cache.tier.sockets.ServerConnection.doNormalMsg(ServerConnection.java:783) >>> >>> at >>> org.apache.geode.internal.cache.tier.sockets.ServerConnection.doOneMessage(ServerConnection.java:914) >>> >>> at >>> org.apache.geode.internal.cache.tier.sockets.ServerConnection.run(ServerConnection.java:1171) >>> >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >>> >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >>> >>> at >>> org.apache.geode.internal.cache.tier.sockets.AcceptorImpl$1$1.run(AcceptorImpl.java:519) >>> >>> ... 1 more >>> >>> >>> >>> >>> >>> The name of my region is /AuthIdRegion but it tries to query using >>> "/AuthIdRegion/AuthIdRegion" >>> >>> >>> >>> we use client region factory to create a region instance >>> >>> "authIdCacheRegionInstance = GeodeCache.getCacheInstance( >>> ).createClientRegionFactory(ClientRegionShortcut.PROXY).create("AuthIdRegion");" >>> >>> >>> >>> It will be really helpful if someone can point what exactly is the issue. >>> >>> >>> >>> Thanks, >>> >>> Rajesh >>> >> > > -- > -John > john.blum10101 (skype) >