[jira] [Commented] (QPID-8140) [Broker-J][BDB HA] Removal of non existing group member can end up in broker crash due to uncaught MemberNotFoundException
[ https://issues.apache.org/jira/browse/QPID-8140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419855#comment-16419855 ] Keith Wall commented on QPID-8140: -- Changes look reasonable to me. > [Broker-J][BDB HA] Removal of non existing group member can end up in broker > crash due to uncaught MemberNotFoundException > -- > > Key: QPID-8140 > URL: https://issues.apache.org/jira/browse/QPID-8140 > Project: Qpid > Issue Type: Bug > Components: Broker-J >Affects Versions: qpid-java-broker-7.0.2, qpid-java-broker-7.0.0, > qpid-java-broker-7.0.1 >Reporter: Alex Rudyy >Assignee: Alex Rudyy >Priority: Minor > Fix For: qpid-java-broker-7.1.0 > > Attachments: > TEST-org.apache.qpid.server.store.berkeleydb.replication.BDBHAVirtualHostNodeRestTest.testDeleteMasterNode.txt > > > When two concurrent requests are made to delete a group member using REST API > for remote replication node, one can successfully remove the node, whilst > other can end-up in un-handled {{MemberNotFoundException}} which crashes the > broker. > An example of remote node delete request > {code} > curl -v -k -u admin -X DELETE > http://localhost:8080/api/latest/remotereplicationnode/node2/node1 > {code} > For failed requests an exception like the one below is reported > {noformat} > ERROR [VirtualHostNode-node2-Config] o.a.q.s.u.ServerScopedRuntimeException > Exception on node removal from group > com.sleepycat.je.rep.MemberNotFoundException: (JE 7.4.5) Node: node1 is not > currently a member of the group: > BDBHAVirtualHostNodeRestTest.testDeleteMasterNode, it has been removed. > at > com.sleepycat.je.rep.util.ReplicationGroupAdmin.checkMember(ReplicationGroupAdmin.java:576) > at > com.sleepycat.je.rep.util.ReplicationGroupAdmin.removeMember(ReplicationGroupAdmin.java:314) > at > org.apache.qpid.server.store.berkeleydb.replication.ReplicatedEnvironmentFacade.removeNodeFromGroup(ReplicatedEnvironmentFacade.java:1210) > at > org.apache.qpid.server.virtualhostnode.berkeleydb.BDBHARemoteReplicationNodeImpl.onDelete(BDBHARemoteReplicationNodeImpl.java:139) > at > org.apache.qpid.server.model.AbstractConfiguredObject$21.onSuccess(AbstractConfiguredObject.java:2759) > at > org.apache.qpid.server.model.AbstractConfiguredObject$24$1.run(AbstractConfiguredObject.java:2894) > at > org.apache.qpid.server.model.AbstractConfiguredObject$24$1.run(AbstractConfiguredObject.java:2890) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.qpid.server.model.AbstractConfiguredObject$24.onSuccess(AbstractConfiguredObject.java:2889) > at > com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1237) > at > org.apache.qpid.server.configuration.updater.TaskExecutorImpl$ImmediateIfSameThreadExecutor.execute(TaskExecutorImpl.java:400) > at > org.apache.qpid.server.configuration.updater.TaskExecutorImpl.execute(TaskExecutorImpl.java:183) > at > com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:911) > at > com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:822) > at > com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:664) > at > org.apache.qpid.server.model.AbstractConfiguredObject$ChainedSettableFuture.set(AbstractConfiguredObject.java:2710) > at > org.apache.qpid.server.model.AbstractConfiguredObject$21$1.onSuccess(AbstractConfiguredObject.java:2765) > at > org.apache.qpid.server.model.AbstractConfiguredObject$24$1.run(AbstractConfiguredObject.java:2894) > at > org.apache.qpid.server.model.AbstractConfiguredObject$24$1.run(AbstractConfiguredObject.java:2890) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.qpid.server.model.AbstractConfiguredObject$24.onSuccess(AbstractConfiguredObject.java:2889) > at > com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1237) > at > org.apache.qpid.server.configuration.updater.TaskExecutorImpl$ImmediateIfSameThreadExecutor.execute(TaskExecutorImpl.java:400) > at > org.apache.qpid.server.configuration.updater.TaskExecutorImpl.execute(TaskExecutorImpl.java:183) > at > com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:911) > at > com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:645) > at >
[jira] [Commented] (QPID-8140) [Broker-J][BDB HA] Removal of non existing group member can end up in broker crash due to uncaught MemberNotFoundException
[ https://issues.apache.org/jira/browse/QPID-8140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419242#comment-16419242 ] ASF subversion and git services commented on QPID-8140: --- Commit 1d832c616859ed5a5648315da4eba696207c9f15 in qpid-broker-j's branch refs/heads/master from [~alex.rufous] [ https://git-wip-us.apache.org/repos/asf?p=qpid-broker-j.git;h=1d832c6 ] QPID-8140: [Broker-J][BDB HA] Handle MemberNotFoundException on removal of non-existing node > [Broker-J][BDB HA] Removal of non existing group member can end up in broker > crash due to uncaught MemberNotFoundException > -- > > Key: QPID-8140 > URL: https://issues.apache.org/jira/browse/QPID-8140 > Project: Qpid > Issue Type: Bug > Components: Broker-J >Affects Versions: qpid-java-broker-7.0.2, qpid-java-broker-7.0.0, > qpid-java-broker-7.0.1 >Reporter: Alex Rudyy >Assignee: Alex Rudyy >Priority: Major > Attachments: > TEST-org.apache.qpid.server.store.berkeleydb.replication.BDBHAVirtualHostNodeRestTest.testDeleteMasterNode.txt > > > When two concurrent requests are made to delete a group member using REST API > for remote replication node, one can successfully remove the node, whilst > other can end-up in un-handled {{MemberNotFoundException}} which crashes the > broker. > An example of remote node delete request > {code} > curl -v -k -u admin -X DELETE > http://localhost:8080/api/latest/remotereplicationnode/node2/node1 > {code} > For failed requests an exception like the one below is reported > {noformat} > ERROR [VirtualHostNode-node2-Config] o.a.q.s.u.ServerScopedRuntimeException > Exception on node removal from group > com.sleepycat.je.rep.MemberNotFoundException: (JE 7.4.5) Node: node1 is not > currently a member of the group: > BDBHAVirtualHostNodeRestTest.testDeleteMasterNode, it has been removed. > at > com.sleepycat.je.rep.util.ReplicationGroupAdmin.checkMember(ReplicationGroupAdmin.java:576) > at > com.sleepycat.je.rep.util.ReplicationGroupAdmin.removeMember(ReplicationGroupAdmin.java:314) > at > org.apache.qpid.server.store.berkeleydb.replication.ReplicatedEnvironmentFacade.removeNodeFromGroup(ReplicatedEnvironmentFacade.java:1210) > at > org.apache.qpid.server.virtualhostnode.berkeleydb.BDBHARemoteReplicationNodeImpl.onDelete(BDBHARemoteReplicationNodeImpl.java:139) > at > org.apache.qpid.server.model.AbstractConfiguredObject$21.onSuccess(AbstractConfiguredObject.java:2759) > at > org.apache.qpid.server.model.AbstractConfiguredObject$24$1.run(AbstractConfiguredObject.java:2894) > at > org.apache.qpid.server.model.AbstractConfiguredObject$24$1.run(AbstractConfiguredObject.java:2890) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.qpid.server.model.AbstractConfiguredObject$24.onSuccess(AbstractConfiguredObject.java:2889) > at > com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1237) > at > org.apache.qpid.server.configuration.updater.TaskExecutorImpl$ImmediateIfSameThreadExecutor.execute(TaskExecutorImpl.java:400) > at > org.apache.qpid.server.configuration.updater.TaskExecutorImpl.execute(TaskExecutorImpl.java:183) > at > com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:911) > at > com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:822) > at > com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:664) > at > org.apache.qpid.server.model.AbstractConfiguredObject$ChainedSettableFuture.set(AbstractConfiguredObject.java:2710) > at > org.apache.qpid.server.model.AbstractConfiguredObject$21$1.onSuccess(AbstractConfiguredObject.java:2765) > at > org.apache.qpid.server.model.AbstractConfiguredObject$24$1.run(AbstractConfiguredObject.java:2894) > at > org.apache.qpid.server.model.AbstractConfiguredObject$24$1.run(AbstractConfiguredObject.java:2890) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.qpid.server.model.AbstractConfiguredObject$24.onSuccess(AbstractConfiguredObject.java:2889) > at > com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1237) > at > org.apache.qpid.server.configuration.updater.TaskExecutorImpl$ImmediateIfSameThreadExecutor.execute(TaskExecutorImpl.java:400) > at > org.apache.qpid.server.configuration.updater.TaskExecutorImpl.execute(TaskExecutorImpl.java:183) > at >