[ https://issues.apache.org/jira/browse/GEODE-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bruce Schuchardt updated GEODE-3760: ------------------------------------ Description: This was hit in GemFire 8 but it applies to GEODE {noformat} [severe 2017/09/17 15:35:02.030 tz <Removing shunned GemFire node xyz tid=0x242a9] Uncaught exception in thread Thread[Removing shunned GemFire node xyz Reader Threads] com.gemstone.gemfire.distributed.DistributedSystemDisconnectedException: Channel closed, caused by com.gemstone.gemfire.ForcedDisconnectException: This member has been forced out of the distributed system by xyz. Please consult GemFire logs to find the reason. (GMS shun) at com.gemstone.gemfire.distributed.internal.membership.jgroup.JGroupMembershipManager.requestMemberRemoval(JGroupMembershipManager.java:2842) at com.gemstone.gemfire.distributed.internal.membership.jgroup.JGroupMembershipManager$1.run(JGroupMembershipManager.java:1938) Caused by: com.gemstone.gemfire.ForcedDisconnectException: This member has been forced out of the distributed system by xyz. Please consult GemFire logs to find the reason. (GMS shun) at com.gemstone.org.jgroups.protocols.pbcast.GMS.installView(GMS.java:1264) at com.gemstone.org.jgroups.protocols.pbcast.GMS.installView(GMS.java:1192) at com.gemstone.org.jgroups.protocols.pbcast.ParticipantGmsImpl.handleViewChange(ParticipantGmsImpl.java:261) at com.gemstone.org.jgroups.protocols.pbcast.GMS.up(GMS.java:1625) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.VIEW_SYNC.handleView(VIEW_SYNC.java:252) at com.gemstone.org.jgroups.protocols.VIEW_SYNC.up(VIEW_SYNC.java:177) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.AUTH.up(AUTH.java:189) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.GemFireTimeSync.up(GemFireTimeSync.java:282) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.FRAG2.up(FRAG2.java:259) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:274) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.UNICAST.up(UNICAST.java:318) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:629) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:215) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:569) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.FD.up(FD.java:447) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.Discovery.up(Discovery.java:265) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.TP.handleIncomingMessage(TP.java:1198) at com.gemstone.org.jgroups.protocols.TP.handleIncomingPacket(TP.java:1110) at com.gemstone.org.jgroups.protocols.TP.receive(TP.java:1009) at com.gemstone.org.jgroups.protocols.UDP$UcastReceiver.run(UDP.java:1403) at java.lang.Thread.run(Thread.java:745) {noformat} was: This was hit in GemFire 8 but it applies to GEODE {noformat} [severe 2017/09/17 15:35:02.030 HKT gemfire_APIM_ESB_Cluster.sdcgcggflla01pv.locator36001 <Removing shunned GemFire node 163.36.41.162(gemfire_APIM_ESB_Cluster.sdcgcggflla02pv.locator36001:16055:locator)<ec><v155>:36295> tid=0x242a9] Uncaught exception in thread Thread[Removing shunned GemFire node 163.36.41.162(gemfire_APIM_ESB_Cluster.sdcgcggflla02pv.locator36001:16055:locator)<ec><v155>:36295,10,P2P Reader Threads] com.gemstone.gemfire.distributed.DistributedSystemDisconnectedException: Channel closed, caused by com.gemstone.gemfire.ForcedDisconnectException: This member has been forced out of the distributed system by 169.195.111.181(gemfire_APIM_ESB_Cluster.sdcgcggfcla05pv.cacheServer36010:10258)<v158>:36606. Please consult GemFire logs to find the reason. (GMS shun) at com.gemstone.gemfire.distributed.internal.membership.jgroup.JGroupMembershipManager.requestMemberRemoval(JGroupMembershipManager.java:2842) at com.gemstone.gemfire.distributed.internal.membership.jgroup.JGroupMembershipManager$1.run(JGroupMembershipManager.java:1938) Caused by: com.gemstone.gemfire.ForcedDisconnectException: This member has been forced out of the distributed system by 169.195.111.181(gemfire_APIM_ESB_Cluster.sdcgcggfcla05pv.cacheServer36010:10258)<v158>:36606. Please consult GemFire logs to find the reason. (GMS shun) at com.gemstone.org.jgroups.protocols.pbcast.GMS.installView(GMS.java:1264) at com.gemstone.org.jgroups.protocols.pbcast.GMS.installView(GMS.java:1192) at com.gemstone.org.jgroups.protocols.pbcast.ParticipantGmsImpl.handleViewChange(ParticipantGmsImpl.java:261) at com.gemstone.org.jgroups.protocols.pbcast.GMS.up(GMS.java:1625) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.VIEW_SYNC.handleView(VIEW_SYNC.java:252) at com.gemstone.org.jgroups.protocols.VIEW_SYNC.up(VIEW_SYNC.java:177) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.AUTH.up(AUTH.java:189) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.GemFireTimeSync.up(GemFireTimeSync.java:282) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.FRAG2.up(FRAG2.java:259) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:274) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.UNICAST.up(UNICAST.java:318) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:629) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:215) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:569) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.FD.up(FD.java:447) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.Discovery.up(Discovery.java:265) at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) at com.gemstone.org.jgroups.protocols.TP.handleIncomingMessage(TP.java:1198) at com.gemstone.org.jgroups.protocols.TP.handleIncomingPacket(TP.java:1110) at com.gemstone.org.jgroups.protocols.TP.receive(TP.java:1009) at com.gemstone.org.jgroups.protocols.UDP$UcastReceiver.run(UDP.java:1403) at java.lang.Thread.run(Thread.java:745) {noformat} > uncaught exception in shunned member removal thread > --------------------------------------------------- > > Key: GEODE-3760 > URL: https://issues.apache.org/jira/browse/GEODE-3760 > Project: Geode > Issue Type: Bug > Components: membership > Reporter: Bruce Schuchardt > > This was hit in GemFire 8 but it applies to GEODE > {noformat} > [severe 2017/09/17 15:35:02.030 tz <Removing shunned GemFire node xyz > tid=0x242a9] Uncaught exception in thread Thread[Removing shunned GemFire > node xyz Reader Threads] > com.gemstone.gemfire.distributed.DistributedSystemDisconnectedException: > Channel closed, caused by com.gemstone.gemfire.ForcedDisconnectException: > This member has been forced out of the distributed system by xyz. Please > consult GemFire logs to find the reason. (GMS shun) > at > com.gemstone.gemfire.distributed.internal.membership.jgroup.JGroupMembershipManager.requestMemberRemoval(JGroupMembershipManager.java:2842) > at > com.gemstone.gemfire.distributed.internal.membership.jgroup.JGroupMembershipManager$1.run(JGroupMembershipManager.java:1938) > Caused by: com.gemstone.gemfire.ForcedDisconnectException: This member has > been forced out of the distributed system by xyz. Please consult GemFire > logs to find the reason. (GMS shun) > at > com.gemstone.org.jgroups.protocols.pbcast.GMS.installView(GMS.java:1264) > at > com.gemstone.org.jgroups.protocols.pbcast.GMS.installView(GMS.java:1192) > at > com.gemstone.org.jgroups.protocols.pbcast.ParticipantGmsImpl.handleViewChange(ParticipantGmsImpl.java:261) > at com.gemstone.org.jgroups.protocols.pbcast.GMS.up(GMS.java:1625) > at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) > at > com.gemstone.org.jgroups.protocols.VIEW_SYNC.handleView(VIEW_SYNC.java:252) > at com.gemstone.org.jgroups.protocols.VIEW_SYNC.up(VIEW_SYNC.java:177) > at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) > at com.gemstone.org.jgroups.protocols.AUTH.up(AUTH.java:189) > at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) > at > com.gemstone.org.jgroups.protocols.GemFireTimeSync.up(GemFireTimeSync.java:282) > at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) > at com.gemstone.org.jgroups.protocols.FRAG2.up(FRAG2.java:259) > at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) > at com.gemstone.org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:274) > at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) > at com.gemstone.org.jgroups.protocols.UNICAST.up(UNICAST.java:318) > at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) > at com.gemstone.org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:629) > at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) > at > com.gemstone.org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:215) > at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) > at com.gemstone.org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:569) > at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) > at com.gemstone.org.jgroups.protocols.FD.up(FD.java:447) > at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) > at com.gemstone.org.jgroups.protocols.Discovery.up(Discovery.java:265) > at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:768) > at > com.gemstone.org.jgroups.protocols.TP.handleIncomingMessage(TP.java:1198) > at > com.gemstone.org.jgroups.protocols.TP.handleIncomingPacket(TP.java:1110) > at com.gemstone.org.jgroups.protocols.TP.receive(TP.java:1009) > at > com.gemstone.org.jgroups.protocols.UDP$UcastReceiver.run(UDP.java:1403) > at java.lang.Thread.run(Thread.java:745) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)