Bruce Schuchardt created GEODE-2155: ---------------------------------------
Summary: Auto-reconnect fails with NPE Key: GEODE-2155 URL: https://issues.apache.org/jira/browse/GEODE-2155 Project: Geode Issue Type: Bug Components: membership Reporter: Bruce Schuchardt The symptom of this bug is {noformat} [severe 2016/11/29 15:55:42.982 PST gemfire1_carlos_13646 <ReconnectThread> tid=0x4d] Unexpected exception while booting membership services java.lang.NullPointerException at org.apache.geode.distributed.internal.membership.gms.messenger.JGroupsMessenger.establishLocalAddress(JGroupsMessenger.java:446) at org.apache.geode.distributed.internal.membership.gms.messenger.JGroupsMessenger.start(JGroupsMessenger.java:342) at org.apache.geode.distributed.internal.membership.gms.Services.start(Services.java:153) at org.apache.geode.distributed.internal.membership.gms.GMSMemberFactory.newMembershipManager(GMSMemberFactory.java:102) at org.apache.geode.distributed.internal.membership.MemberFactory.newMembershipManager(MemberFactory.java:89) at org.apache.geode.distributed.internal.DistributionManager.<init>(DistributionManager.java:1112) at org.apache.geode.distributed.internal.DistributionManager.<init>(DistributionManager.java:1160) at org.apache.geode.distributed.internal.DistributionManager.create(DistributionManager.java:531) at org.apache.geode.distributed.internal.InternalDistributedSystem.initialize(InternalDistributedSystem.java:683) at org.apache.geode.distributed.internal.InternalDistributedSystem.newInstance(InternalDistributedSystem.java:297) at org.apache.geode.distributed.DistributedSystem.connect(DistributedSystem.java:202) at org.apache.geode.distributed.internal.InternalDistributedSystem.reconnect(InternalDistributedSystem.java:2688) at org.apache.geode.distributed.internal.InternalDistributedSystem.tryReconnect(InternalDistributedSystem.java:2521) at org.apache.geode.distributed.internal.InternalDistributedSystem.disconnect(InternalDistributedSystem.java:989) at org.apache.geode.distributed.internal.DistributionManager$MyListener.membershipFailure(DistributionManager.java:4299) at org.apache.geode.distributed.internal.membership.gms.mgr.GMSMembershipManager.uncleanShutdown(GMSMembershipManager.java:1530) at org.apache.geode.distributed.internal.membership.gms.mgr.GMSMembershipManager.lambda$forceDisconnect$18(GMSMembershipManager.java:2550) at java.lang.Thread.run(Thread.java:745) {noformat} This is caused by a previous attempt to recreate the cache in a "reconnect thread" encountered problems {noformat} [warning 2016/11/29 15:54:42.943 PST gemfire1_carlos_13646 <ReconnectThread> tid=0x4d] Exception occurred while trying to create the cache during reconnect org.apache.geode.cache.CacheXmlException: While reading Cache XML null. Class "tx.TxLoader" is not an instance of Declarable. at org.apache.geode.internal.cache.xmlcache.CacheXmlParser.createDeclarable(CacheXmlParser.java:1946) at org.apache.geode.internal.cache.xmlcache.CacheXmlParser.endCacheLoader(CacheXmlParser.java:1997) at org.apache.geode.internal.cache.xmlcache.CacheXmlParser.endElement(CacheXmlParser.java:2964) at org.apache.geode.internal.cache.xmlcache.CacheXmlParser$DefaultHandlerDelegate.endElement(CacheXmlParser.java:3379) at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609) at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.endElement(XMLSchemaValidator.java:857) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1783) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2970) at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606) at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:118) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777) at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141) at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213) at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643) at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:327) at javax.xml.parsers.SAXParser.parse(SAXParser.java:195) at org.apache.geode.internal.cache.xmlcache.CacheXmlParser.parse(CacheXmlParser.java:224) at org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4474) at org.apache.geode.internal.cache.GemFireCacheImpl.initializeDeclarativeCache(GemFireCacheImpl.java:1445) at org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1245) at org.apache.geode.internal.cache.GemFireCacheImpl.basicCreate(GemFireCacheImpl.java:796) at org.apache.geode.internal.cache.GemFireCacheImpl.create(GemFireCacheImpl.java:773) at org.apache.geode.distributed.internal.InternalDistributedSystem.reconnect(InternalDistributedSystem.java:2751) at org.apache.geode.distributed.internal.InternalDistributedSystem.tryReconnect(InternalDistributedSystem.java:2521) at org.apache.geode.distributed.internal.InternalDistributedSystem.disconnect(InternalDistributedSystem.java:989) at org.apache.geode.distributed.internal.DistributionManager$MyListener.membershipFailure(DistributionManager.java:4299) at org.apache.geode.distributed.internal.membership.gms.mgr.GMSMembershipManager.uncleanShutdown(GMSMembershipManager.java:1530) at org.apache.geode.distributed.internal.membership.gms.mgr.GMSMembershipManager.lambda$forceDisconnect$18(GMSMembershipManager.java:2550) at java.lang.Thread.run(Thread.java:745) {noformat} The cache closes the DistributedSystem when this happens. Probably the best solution to this is to have it cease trying to reconnect since the same problem will happen every time it tries to rebuild the cache. -- This message was sent by Atlassian JIRA (v6.3.4#6332)