[ https://issues.apache.org/jira/browse/GEODE-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Swapnil Bawaskar updated GEODE-3155: ------------------------------------ Fix Version/s: 1.3.0 > GMSJoinLeaveJUnitTest.testCoordinatorFindRequestSuccess failing in CI > --------------------------------------------------------------------- > > Key: GEODE-3155 > URL: https://issues.apache.org/jira/browse/GEODE-3155 > Project: Geode > Issue Type: Bug > Components: membership, messaging > Reporter: Galen O'Sullivan > Labels: flaky > Fix For: 1.3.0 > > > https://builds.apache.org/blue/organizations/jenkins/Geode-nightly/detail/Geode-nightly/880/tests > I don't know what the problem is, needs debugging. > Test output follows: > {code} > java.lang.AssertionError: Should be able to join > Stacktrace > java.lang.AssertionError: Should be able to join > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.assertTrue(Assert.java:41) > at > org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeaveJUnitTest.testCoordinatorFindRequestSuccess(GMSJoinLeaveJUnitTest.java:1218) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) > at org.junit.runners.ParentRunner.run(ParentRunner.java:363) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:114) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:57) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:66) > at > org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) > at > org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32) > at > org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93) > at com.sun.proxy.$Proxy2.processTestClass(Unknown Source) > at > org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:109) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) > at > org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:377) > at > org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:54) > at > org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:40) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:748) > Standard Output > [info 2017/06/28 11:05:53.704 UTC <Test worker> tid=0x13] received join > request from asf902(31916)<ec>:8888 > [info 2017/06/28 11:05:53.728 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8887, asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, > asf902(31916)<ec>:8891] > old view is: null > [info 2017/06/28 11:05:53.729 UTC <Test worker> tid=0x13] received leave > request from asf902(31916)<ec>:8888 for asf902(31916)<ec>:8888 > [info 2017/06/28 11:05:53.731 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:05:53.739 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v15>:8887 > [info 2017/06/28 11:05:53.740 UTC <Geode Membership View Creator> tid=0x19] > View Creator thread is starting > [info 2017/06/28 11:05:53.745 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8887] > old view is: null > [info 2017/06/28 11:05:53.746 UTC <Test worker> tid=0x13] Membership received > a request to remove asf902(31916)<ec>:8887 from asf902(31916)<ec>:8889 > reason=testing > [info 2017/06/28 11:05:53.752 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8887] > old view is: null > [info 2017/06/28 11:05:53.754 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:05:53.754 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v12>:8887 > [info 2017/06/28 11:05:53.755 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|2] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec><v12>:8887] > old view is: View[asf902(31916)<ec>:8888|1] members: > [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec><v12>:8887, > asf902(31916)<ec><v12>:8887] > [info 2017/06/28 11:05:53.755 UTC <Geode Membership View Creator> tid=0x1c] > View Creator thread is starting > [info 2017/06/28 11:05:53.757 UTC <Geode Membership View Creator> tid=0x1c] > preparing new view View[asf902(31916)<ec><v12>:8887|12] members: > [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec><v12>:8887, > asf902(31916)<ec><v12>:8887] > [info 2017/06/28 11:05:53.764 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8887] > old view is: null > [info 2017/06/28 11:05:53.765 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|2] members: [asf902(31916)<ec>:8887{lead}, > asf902(31916)<ec>:8887{lead}] > old view is: View[asf902(31916)<ec>:8888|1] members: > [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887, asf902(31916)<ec>:8887] > [info 2017/06/28 11:05:53.765 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:05:53.766 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v10>:8887 > [info 2017/06/28 11:05:53.767 UTC <Geode Membership View Creator> tid=0x1e] > View Creator thread is starting > [info 2017/06/28 11:05:53.777 UTC <Thread-10> tid=0x20] received new view: > View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec>:8887] > old view is: null > [info 2017/06/28 11:05:53.796 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:05:53.796 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}] > old view is: null > [info 2017/06/28 11:05:53.796 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v0>:8887 > [info 2017/06/28 11:05:53.797 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec><v0>:8887|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec><v0>:8887, > asf902(31916)<ec>:8891] > old view is: View[asf902(31916)<ec><v0>:8887|0] members: > [asf902(31916)<ec><v0>:8887{lead}] > [info 2017/06/28 11:05:53.797 UTC <Geode Membership View Creator> tid=0x22] > View Creator thread is starting > [info 2017/06/28 11:05:53.925 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:05:53.925 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}] > old view is: null > [info 2017/06/28 11:05:53.926 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v0>:8887 > [info 2017/06/28 11:05:53.927 UTC <Geode Membership View Creator> tid=0x27] > View Creator thread is starting > [info 2017/06/28 11:05:53.928 UTC <Geode Membership View Creator> tid=0x27] > no recipients for new view aside from myself > [info 2017/06/28 11:05:54.032 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8887] > old view is: null > [info 2017/06/28 11:05:54.033 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|2] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec>:8887] > old view is: View[asf902(31916)<ec>:8888|1] members: > [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887] > [info 2017/06/28 11:05:54.034 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|3] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8887] crashed: [asf902(31916)<ec>:8889, > asf902(31916)<ec>:8890] > old view is: View[asf902(31916)<ec>:8888|2] members: > [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8889, > asf902(31916)<ec>:8890, asf902(31916)<ec>:8887] > [warn 2017/06/28 11:05:54.034 UTC <Test worker> tid=0x13] total weight lost > in this view change is 40 of 65. Quorum has been lost! > [info 2017/06/28 11:05:54.040 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8887] > old view is: null > [info 2017/06/28 11:05:54.041 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|2] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec>:8887] > old view is: View[asf902(31916)<ec>:8888|1] members: > [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887] > [info 2017/06/28 11:05:54.041 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|3] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8887] crashed: [asf902(31916)<ec>:8889, > asf902(31916)<ec>:8890] > old view is: View[asf902(31916)<ec>:8888|2] members: > [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8889, > asf902(31916)<ec>:8890, asf902(31916)<ec>:8887] > [warn 2017/06/28 11:05:54.041 UTC <Test worker> tid=0x13] total weight lost > in this view change is 40 of 65. Quorum has been lost! > [info 2017/06/28 11:05:54.046 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:05:54.046 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}] > old view is: null > [info 2017/06/28 11:05:54.046 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v0>:8887 > [info 2017/06/28 11:05:54.047 UTC <Geode Membership View Creator> tid=0x2d] > View Creator thread is starting > [info 2017/06/28 11:05:54.049 UTC <Geode Membership View Creator> tid=0x2d] > no recipients for new view aside from myself > [info 2017/06/28 11:05:54.547 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec><v0>:8887|1] members: > [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916)<ec>:8889] > old view is: View[asf902(31916)<ec><v0>:8887|0] members: > [asf902(31916)<ec><v0>:8887{lead}] > Current viewid 1 > [info 2017/06/28 11:05:54.551 UTC <Test worker> tid=0x13] received join > request from asf902(31916)<ec>:8890 > [info 2017/06/28 11:05:54.853 UTC <Geode Membership View Creator> tid=0x2d] > View Creator is processing 1 requests for the next membership view > [info 2017/06/28 11:05:54.854 UTC <Geode Membership View Creator> tid=0x2d] > preparing new view View[asf902(31916)<ec><v0>:8887|2] members: > [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916)<ec>:8889, asf902(31916)<ec><v2>:8890] > Empty sleeps 1 stopping: false > Empty sleeps 2 stopping: false > Empty sleeps 3 stopping: false > Empty sleeps 4 stopping: false > Empty sleeps 5 stopping: false > [info 2017/06/28 11:05:59.856 UTC <Geode Membership View Creator> tid=0x2d] > finished waiting for responses to view preparation > [warn 2017/06/28 11:05:59.857 UTC <Geode Membership View Creator> tid=0x2d] > these members failed to respond to the view change: [asf902(31916)<ec>:8889, > asf902(31916)<ec>:8888, asf902(31916)<ec><v2>:8890] > [info 2017/06/28 11:05:59.862 UTC <Geode Membership View Creator> tid=0x2d] > adding these unresponsive members to the crash-set for the next view: > [asf902(31916)<ec>:8888] > [info 2017/06/28 11:05:59.863 UTC <Geode Membership View Creator> tid=0x2d] > asf902(31916)<ec>:8888 had a weight of 10 > [info 2017/06/28 11:05:59.863 UTC <Geode Membership View Creator> tid=0x2d] > preparing new view View[asf902(31916)<ec><v0>:8887|3] members: > [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8889, > asf902(31916)<ec><v2>:8890] crashed: [asf902(31916)<ec>:8888] > Empty sleeps 6 stopping: false > Empty sleeps 7 stopping: false > Empty sleeps 8 stopping: false > Empty sleeps 9 stopping: false > Empty sleeps 10 stopping: false > [info 2017/06/28 11:06:04.865 UTC <Geode Membership View Creator> tid=0x2d] > finished waiting for responses to view preparation > [warn 2017/06/28 11:06:04.865 UTC <Geode Membership View Creator> tid=0x2d] > these members failed to respond to the view change: [asf902(31916)<ec>:8889, > asf902(31916)<ec><v2>:8890] > [info 2017/06/28 11:06:04.905 UTC <Geode Membership View Creator> tid=0x2d] > received new view: View[asf902(31916)<ec><v0>:8887|3] members: > [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8889, > asf902(31916)<ec><v2>:8890] crashed: [asf902(31916)<ec>:8888] > old view is: View[asf902(31916)<ec><v0>:8887|1] members: > [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916)<ec>:8889] > [info 2017/06/28 11:06:04.906 UTC <Geode Membership View Creator> tid=0x2d] > asf902(31916)<ec>:8888 had a weight of 10 > [info 2017/06/28 11:06:04.906 UTC <Geode Membership View Creator> tid=0x2d] > sending new view View[asf902(31916)<ec><v0>:8887|3] members: > [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8889, > asf902(31916)<ec><v2>:8890] crashed: [asf902(31916)<ec>:8888] > [info 2017/06/28 11:06:04.907 UTC <Geode Membership View Creator> tid=0x2d] > View Creator is processing 1 requests for the next membership view > Empty sleeps 11 stopping: false > [info 2017/06/28 11:06:05.570 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec><v1>:8887|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec>:8891, > asf902(31916)<ec><v1>:8887] > old view is: null > [info 2017/06/28 11:06:05.571 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec><v1>:8887 > [info 2017/06/28 11:06:05.572 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v1>:8887 > [info 2017/06/28 11:06:05.574 UTC <Geode Membership View Creator> tid=0x35] > View Creator thread is starting > [info 2017/06/28 11:06:05.586 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8887] > old view is: null > [info 2017/06/28 11:06:05.590 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec>:8887, > asf902(31916)<ec>:8891] > old view is: null > [info 2017/06/28 11:06:05.591 UTC <Test worker> tid=0x13] received leave > request from asf902(31916)<ec>:8889 for asf902(31916)<ec>:8889 > [info 2017/06/28 11:06:05.591 UTC <Test worker> tid=0x13] received leave > request from asf902(31916)<ec>:8890 for asf902(31916)<ec>:8890 > [info 2017/06/28 11:06:05.591 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|3] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8887, asf902(31916)<ec>:8889, asf902(31916)<ec>:8891] > old view is: View[asf902(31916)<ec>:8888|1] members: > [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8889, > asf902(31916)<ec>:8890, asf902(31916)<ec>:8887, asf902(31916)<ec>:8891] > [info 2017/06/28 11:06:05.592 UTC <Test worker> tid=0x13] received leave > request from asf902(31916)<ec>:8888 for asf902(31916)<ec>:8888 > [info 2017/06/28 11:06:05.592 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:06:05.593 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v15>:8887 > [info 2017/06/28 11:06:05.593 UTC <Geode Membership View Creator> tid=0x39] > View Creator thread is starting > [info 2017/06/28 11:06:05.597 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec>:8887, > asf902(31916)<ec>:8891] > old view is: null > [info 2017/06/28 11:06:05.597 UTC <Test worker> tid=0x13] received leave > request from asf902(31916)<ec>:8889 for asf902(31916)<ec>:8889 > [info 2017/06/28 11:06:05.597 UTC <Test worker> tid=0x13] received leave > request from asf902(31916)<ec>:8890 for asf902(31916)<ec>:8890 > [info 2017/06/28 11:06:05.598 UTC <Test worker> tid=0x13] received leave > request from asf902(31916)<ec>:8888 for asf902(31916)<ec>:8888 > [info 2017/06/28 11:06:05.598 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:06:05.599 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v8>:8887 > [info 2017/06/28 11:06:05.599 UTC <Geode Membership View Creator> tid=0x3b] > View Creator thread is starting > [info 2017/06/28 11:06:05.606 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8887|1] members: [asf902(31916)<ec>:8887{lead}, > asf902(31916)<ec>:8888] > old view is: null > [info 2017/06/28 11:06:05.607 UTC <Test worker> tid=0x13] received leave > request from asf902(31916)<ec>:8889 for asf902(31916)<ec>:8887 > [info 2017/06/28 11:06:05.607 UTC <Test worker> tid=0x13] I am being told to > leave the distributed system by asf902(31916)<ec>:8889 > [info 2017/06/28 11:06:05.610 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:06:05.611 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}] > old view is: null > [info 2017/06/28 11:06:05.611 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v0>:8887 > [info 2017/06/28 11:06:05.612 UTC <Test worker> tid=0x13] received join > request from asf902(31916)<ec>:8888 > [info 2017/06/28 11:06:05.612 UTC <Geode Membership View Creator> tid=0x3e] > View Creator thread is starting > [info 2017/06/28 11:06:05.617 UTC <Geode Membership View Creator> tid=0x3e] > no recipients for new view aside from myself > [info 2017/06/28 11:06:05.918 UTC <Geode Membership View Creator> tid=0x3e] > View Creator is processing 1 requests for the next membership view > [info 2017/06/28 11:06:05.919 UTC <Geode Membership View Creator> tid=0x3e] > preparing new view View[asf902(31916)<ec><v0>:8887|1] members: > [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec><v1>:8888] > [info 2017/06/28 11:06:05.925 UTC <Geode Membership View Creator> tid=0x3e] > View Creator thread interrupted > [info 2017/06/28 11:06:05.931 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|3] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec>:8887, > asf902(31916)<ec>:8891] > old view is: null > [info 2017/06/28 11:06:05.934 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8887|1] members: [asf902(31916)<ec>:8887{lead}, > asf902(31916)<ec>:8888] > old view is: null > [info 2017/06/28 11:06:05.935 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:06:05.936 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v6>:8887 > [info 2017/06/28 11:06:05.936 UTC <Test worker> tid=0x13] Membership received > a request to remove asf902(31916)<ec>:8888 from asf902(31916)<ec>:8888 > reason=testing > [info 2017/06/28 11:06:05.937 UTC <Geode Membership View Creator> tid=0x43] > View Creator thread is starting > [info 2017/06/28 11:06:05.937 UTC <Test worker> tid=0x13] Membership received > a request to remove asf902(31916)<ec>:8888 from asf902(31916)<ec>:8888 > reason=testing > [info 2017/06/28 11:06:05.939 UTC <Geode Membership View Creator> tid=0x43] > preparing new view View[asf902(31916)<ec><v6>:8887|6] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916)<ec>:8889] > [info 2017/06/28 11:06:10.941 UTC <Geode Membership View Creator> tid=0x43] > finished waiting for responses to view preparation > [warn 2017/06/28 11:06:10.941 UTC <Geode Membership View Creator> tid=0x43] > these members failed to respond to the view change: [asf902(31916)<ec>:8889, > asf902(31916)<ec>:8888] > [info 2017/06/28 11:06:10.942 UTC <Geode View Creator verification thread 1> > tid=0x44] checking state of member asf902(31916)<ec>:8889 > [info 2017/06/28 11:06:10.943 UTC <Geode View Creator verification thread 1> > tid=0x44] member asf902(31916)<ec>:8889 failed availability check > [info 2017/06/28 11:06:15.917 UTC <Geode Membership View Creator> tid=0x43] > adding these unresponsive members to the crash-set for the next view: > [asf902(31916)<ec>:8889, asf902(31916)<ec>:8888] > [info 2017/06/28 11:06:15.918 UTC <Geode Membership View Creator> tid=0x43] > preparing new view View[asf902(31916)<ec><v6>:8887|7] members: > [asf902(31916)<ec><v6>:8887{lead}] crashed: [asf902(31916)<ec>:8889, > asf902(31916)<ec>:8888] > [info 2017/06/28 11:06:15.918 UTC <Geode Membership View Creator> tid=0x43] > finished waiting for responses to view preparation > [info 2017/06/28 11:06:15.918 UTC <Geode Membership View Creator> tid=0x43] > received new view: View[asf902(31916)<ec><v6>:8887|7] members: > [asf902(31916)<ec><v6>:8887{lead}] crashed: [asf902(31916)<ec>:8889, > asf902(31916)<ec>:8888] > old view is: View[asf902(31916)<ec><v6>:8887|1] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916)<ec>:8889] > [info 2017/06/28 11:06:15.918 UTC <Geode Membership View Creator> tid=0x43] > asf902(31916)<ec>:8889 had a weight of 10 > [info 2017/06/28 11:06:15.918 UTC <Geode Membership View Creator> tid=0x43] > asf902(31916)<ec>:8888 had a weight of 10 > [warn 2017/06/28 11:06:15.919 UTC <Geode Membership View Creator> tid=0x43] > total weight lost in this view change is 20 of 35. Quorum has been lost! > [info 2017/06/28 11:06:15.920 UTC <Geode Membership View Creator> tid=0x43] > sending new view View[asf902(31916)<ec><v6>:8887|7] members: > [asf902(31916)<ec><v6>:8887{lead}] crashed: [asf902(31916)<ec>:8889, > asf902(31916)<ec>:8888] > [info 2017/06/28 11:06:16.221 UTC <Geode Membership View Creator> tid=0x43] > View Creator is processing 1 requests for the next membership view > [info 2017/06/28 11:06:16.942 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:06:16.942 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}] > old view is: null > [info 2017/06/28 11:06:16.942 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v0>:8887 > [info 2017/06/28 11:06:16.942 UTC <Geode Membership View Creator> tid=0x46] > View Creator thread is starting > [info 2017/06/28 11:06:16.943 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec><v0>:8887|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec><v0>:8887, > asf902(31916)<ec>:8891] > old view is: View[asf902(31916)<ec><v0>:8887|0] members: > [asf902(31916)<ec><v0>:8887{lead}] > [info 2017/06/28 11:06:16.945 UTC <Geode Membership View Creator> tid=0x46] > no recipients for new view aside from myself > [info 2017/06/28 11:06:17.245 UTC <Geode Membership View Creator> tid=0x46] > View Creator is processing 3 requests for the next membership view > [info 2017/06/28 11:06:17.247 UTC <Geode Membership View Creator> tid=0x46] > asf902(31916)<ec>:8889 had a weight of 10 > [info 2017/06/28 11:06:17.248 UTC <Geode Membership View Creator> tid=0x46] > asf902(31916)<ec>:8891 had a weight of 10 > [info 2017/06/28 11:06:17.248 UTC <Geode Membership View Creator> tid=0x46] > asf902(31916)<ec>:8890 had a weight of 10 > [warn 2017/06/28 11:06:17.248 UTC <Geode Membership View Creator> tid=0x46] > total weight lost in this view change is 30 of 55. Quorum has been lost! > [info 2017/06/28 11:06:17.250 UTC <Geode Membership View Creator> tid=0x46] > View Creator thread interrupted > [info 2017/06/28 11:06:17.254 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8887] > old view is: null > [info 2017/06/28 11:06:17.260 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8887] > old view is: null > [info 2017/06/28 11:06:17.260 UTC <Test worker> tid=0x13] Membership ignoring > removal request for asf902(31916)<ec>:8889 from non-member > asf902(31916)<ec>:9000 > [info 2017/06/28 11:06:17.264 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:06:17.264 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}] > old view is: null > [info 2017/06/28 11:06:17.265 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v0>:8887 > [fatal 2017/06/28 11:06:17.265 UTC <Test worker> tid=0x13] This member is no > longer in the membership view. My ID is asf902(31916)<ec><v0>:8887 and the > new view is View[asf902(31916)<ec>:8888|2] members: > [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8889, > asf902(31916)<ec>:8890, asf902(31916)<ec>:8891] > [info 2017/06/28 11:06:17.265 UTC <Geode Membership View Creator> tid=0x4a] > View Creator thread is starting > [info 2017/06/28 11:06:18.267 UTC <Geode Membership View Creator> tid=0x4a] > no recipients for new view aside from myself > [info 2017/06/28 11:06:18.271 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8887] > old view is: null > [info 2017/06/28 11:06:18.275 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8889, asf902(31916)<ec>:8887] > old view is: null > [info 2017/06/28 11:06:18.277 UTC <Test worker> tid=0x13] received leave > request from asf902(31916)<ec>:8889 for asf902(31916)<ec>:8889 > [info 2017/06/28 11:06:18.277 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:06:18.278 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v11>:8887 > [info 2017/06/28 11:06:18.279 UTC <Geode Membership View Creator> tid=0x4d] > View Creator thread is starting > [info 2017/06/28 11:06:18.296 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:06:18.297 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}] > old view is: null > [info 2017/06/28 11:06:18.298 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v0>:8887 > [info 2017/06/28 11:06:18.299 UTC <Geode Membership View Creator> tid=0x4f] > View Creator thread is starting > [info 2017/06/28 11:06:19.300 UTC <Geode Membership View Creator> tid=0x4f] > no recipients for new view aside from myself > [info 2017/06/28 11:06:19.304 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:06:19.305 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}] > old view is: null > [info 2017/06/28 11:06:19.305 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v0>:8887 > [info 2017/06/28 11:06:19.306 UTC <Geode Membership View Creator> tid=0x51] > View Creator thread is starting > [info 2017/06/28 11:06:19.310 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:06:19.311 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}] > old view is: null > [info 2017/06/28 11:06:19.311 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v0>:8887 > [info 2017/06/28 11:06:19.311 UTC <Geode Membership View Creator> tid=0x53] > View Creator thread is starting > [info 2017/06/28 11:06:19.314 UTC <Geode Membership View Creator> tid=0x53] > no recipients for new view aside from myself > [info 2017/06/28 11:06:19.812 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec><v0>:8887|1] members: > [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916)<ec>:8889] > old view is: View[asf902(31916)<ec><v0>:8887|0] members: > [asf902(31916)<ec><v0>:8887{lead}] > Current viewid 1 > [info 2017/06/28 11:06:19.812 UTC <Test worker> tid=0x13] received join > request from asf902(31916)<ec>:8890 > [info 2017/06/28 11:06:20.113 UTC <Geode Membership View Creator> tid=0x53] > View Creator is processing 1 requests for the next membership view > [info 2017/06/28 11:06:20.114 UTC <Geode Membership View Creator> tid=0x53] > preparing new view View[asf902(31916)<ec><v0>:8887|2] members: > [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916)<ec>:8889, asf902(31916)<ec><v2>:8890] > Empty sleeps 1 stopping: false > Empty sleeps 2 stopping: false > Empty sleeps 3 stopping: false > Empty sleeps 4 stopping: false > Empty sleeps 5 stopping: false > [info 2017/06/28 11:06:25.116 UTC <Geode Membership View Creator> tid=0x53] > finished waiting for responses to view preparation > [warn 2017/06/28 11:06:25.117 UTC <Geode Membership View Creator> tid=0x53] > these members failed to respond to the view change: [asf902(31916)<ec>:8889, > asf902(31916)<ec>:8888, asf902(31916)<ec><v2>:8890] > [info 2017/06/28 11:06:25.118 UTC <Geode View Creator verification thread 2> > tid=0x55] received leave request from asf902(31916)<ec>:8888 for > asf902(31916)<ec>:8888 > [info 2017/06/28 11:06:25.121 UTC <Geode Membership View Creator> tid=0x53] > received new view: View[asf902(31916)<ec><v0>:8887|2] members: > [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916)<ec>:8889, asf902(31916)<ec><v2>:8890] > old view is: View[asf902(31916)<ec><v0>:8887|1] members: > [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916)<ec>:8889] > [info 2017/06/28 11:06:25.123 UTC <Geode Membership View Creator> tid=0x53] > sending new view View[asf902(31916)<ec><v0>:8887|2] members: > [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916)<ec>:8889, asf902(31916)<ec><v2>:8890] > [info 2017/06/28 11:06:25.123 UTC <Geode Membership View Creator> tid=0x53] > View Creator is processing 1 requests for the next membership view > [info 2017/06/28 11:06:25.124 UTC <Geode Membership View Creator> tid=0x53] > preparing new view View[asf902(31916)<ec><v0>:8887|3] members: > [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8889, > asf902(31916)<ec><v2>:8890] shutdown: [asf902(31916)<ec>:8888] > Empty sleeps 6 stopping: false > [info 2017/06/28 11:06:25.814 UTC <Geode Membership View Creator> tid=0x53] > View Creator thread interrupted > [info 2017/06/28 11:06:25.823 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8887|1] members: [asf902(31916)<ec>:8887{lead}, > asf902(31916)<ec>:8888] > old view is: null > [info 2017/06/28 11:06:25.825 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:06:25.826 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v6>:8887 > [info 2017/06/28 11:06:25.828 UTC <Geode Membership View Creator> tid=0x58] > View Creator thread is starting > [info 2017/06/28 11:06:25.830 UTC <Geode Membership View Creator> tid=0x58] > preparing new view View[asf902(31916)<ec><v6>:8887|6] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888] > [info 2017/06/28 11:06:25.930 UTC <Geode Membership View Creator> tid=0x58] > finished waiting for responses to view preparation > [warn 2017/06/28 11:06:25.931 UTC <Geode Membership View Creator> tid=0x58] > received a conflicting membership view from asf902(31916)<ec>:8888 during > preparation: View[asf902(31916:locator)<ec>:8892|11] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916:locator)<ec>:8892] > [info 2017/06/28 11:06:25.931 UTC <Geode Membership View Creator> tid=0x58] > View preparation interrupted - a locator is taking over as membership > coordinator in this view: View[asf902(31916:locator)<ec>:8892|11] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916:locator)<ec>:8892] > [info 2017/06/28 11:06:26.138 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:06:26.138 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}] > old view is: null > [info 2017/06/28 11:06:26.138 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v0>:8887 > [info 2017/06/28 11:06:26.139 UTC <Geode Membership View Creator> tid=0x60] > View Creator thread is starting > [info 2017/06/28 11:06:26.149 UTC <Geode Membership View Creator> tid=0x60] > no recipients for new view aside from myself > [info 2017/06/28 11:06:26.241 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec><v0>:8887|1] members: > [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8889, > asf902(31916)<ec>:8890] > old view is: View[asf902(31916)<ec><v0>:8887|0] members: > [asf902(31916)<ec><v0>:8887{lead}] > [info 2017/06/28 11:06:26.242 UTC <Test worker> tid=0x13] received leave > request from asf902(31916)<ec>:8889 for asf902(31916)<ec>:8889 > [info 2017/06/28 11:06:31.248 UTC <Geode Membership View Creator> tid=0x60] > View Creator is processing 1 requests for the next membership view > [info 2017/06/28 11:06:31.249 UTC <Geode Membership View Creator> tid=0x60] > preparing new view View[asf902(31916)<ec><v0>:8887|2] members: > [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8890] shutdown: > [asf902(31916)<ec>:8889] > [info 2017/06/28 11:06:36.251 UTC <Geode Membership View Creator> tid=0x60] > finished waiting for responses to view preparation > [warn 2017/06/28 11:06:36.251 UTC <Geode Membership View Creator> tid=0x60] > these members failed to respond to the view change: [asf902(31916)<ec>:8890] > [info 2017/06/28 11:06:36.251 UTC <Geode View Creator verification thread 1> > tid=0x65] checking state of member asf902(31916)<ec>:8890 > [info 2017/06/28 11:06:36.252 UTC <Geode View Creator verification thread 1> > tid=0x65] member asf902(31916)<ec>:8890 passed availability check > [info 2017/06/28 11:06:36.253 UTC <Geode Membership View Creator> tid=0x60] > received new view: View[asf902(31916)<ec><v0>:8887|2] members: > [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8890] shutdown: > [asf902(31916)<ec>:8889] > old view is: View[asf902(31916)<ec><v0>:8887|1] members: > [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8889, > asf902(31916)<ec>:8890] > [info 2017/06/28 11:06:36.254 UTC <Geode Membership View Creator> tid=0x60] > sending new view View[asf902(31916)<ec><v0>:8887|2] members: > [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8890] shutdown: > [asf902(31916)<ec>:8889] > [info 2017/06/28 11:06:36.280 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8887|1] members: [asf902(31916)<ec>:8887{lead}, > asf902(31916)<ec>:8888] > old view is: null > [info 2017/06/28 11:06:36.281 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:06:36.281 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v6>:8887 > [info 2017/06/28 11:06:36.281 UTC <Test worker> tid=0x13] received leave > request from asf902(31916)<ec>:8888 for asf902(31916)<ec>:8888 > [info 2017/06/28 11:06:36.281 UTC <Geode Membership View Creator> tid=0x67] > View Creator thread is starting > [info 2017/06/28 11:06:36.281 UTC <Test worker> tid=0x13] received leave > request from asf902(31916)<ec>:8888 for asf902(31916)<ec>:8888 > [info 2017/06/28 11:06:36.283 UTC <Geode Membership View Creator> tid=0x67] > preparing new view View[asf902(31916)<ec><v6>:8887|6] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888] > [info 2017/06/28 11:06:36.284 UTC <Geode Membership View Creator> tid=0x67] > finished waiting for responses to view preparation > [info 2017/06/28 11:06:36.285 UTC <Geode Membership View Creator> tid=0x67] > received new view: View[asf902(31916)<ec><v6>:8887|6] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888] > old view is: View[asf902(31916)<ec><v6>:8887|1] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888] > [info 2017/06/28 11:06:36.285 UTC <Geode Membership View Creator> tid=0x67] > sending new view View[asf902(31916)<ec><v6>:8887|6] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888] > [info 2017/06/28 11:06:36.586 UTC <Geode Membership View Creator> tid=0x67] > View Creator is processing 2 requests for the next membership view > [info 2017/06/28 11:06:36.586 UTC <Geode Membership View Creator> tid=0x67] > received new view: View[asf902(31916)<ec><v6>:8887|7] members: > [asf902(31916)<ec><v6>:8887{lead}] shutdown: [asf902(31916)<ec>:8888] > old view is: View[asf902(31916)<ec><v6>:8887|6] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888] > [info 2017/06/28 11:06:36.587 UTC <Geode Membership View Creator> tid=0x67] > no recipients for new view aside from myself > [info 2017/06/28 11:06:37.286 UTC <Test worker> tid=0x13] received join > request from asf902(31916)<ec>:8700(version:GFE 5.6) > [warn 2017/06/28 11:06:37.286 UTC <Test worker> tid=0x13] detected an attempt > to start a peer using an older version of the product > asf902(31916)<ec>:8700(version:GFE 5.6) > [info 2017/06/28 11:06:37.290 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8887] > old view is: null > [info 2017/06/28 11:06:37.290 UTC <Test worker> tid=0x13] received leave > request from asf902(31916)<ec><v0>:8889 for asf902(31916)<ec><v0>:8889 > [info 2017/06/28 11:06:37.293 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8887] > old view is: null > [info 2017/06/28 11:06:37.294 UTC <Test worker> tid=0x13] Membership received > a request to remove asf902(31916)<ec>:8888 from asf902(31916)<ec>:8888 > reason=testing > [info 2017/06/28 11:06:37.294 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:06:37.295 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v13>:8887 > [info 2017/06/28 11:06:37.295 UTC <Geode Membership View Creator> tid=0x6b] > View Creator thread is starting > [info 2017/06/28 11:06:37.297 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:06:37.298 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}] > old view is: null > [info 2017/06/28 11:06:37.298 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v0>:8887 > [info 2017/06/28 11:06:37.299 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec><v0>:8887|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec><v0>:8887, > asf902(31916)<ec>:8891] > old view is: View[asf902(31916)<ec><v0>:8887|0] members: > [asf902(31916)<ec><v0>:8887{lead}] > [info 2017/06/28 11:06:37.299 UTC <Geode Membership View Creator> tid=0x6d] > View Creator thread is starting > [info 2017/06/28 11:06:37.301 UTC <Test worker> tid=0x13] received join > request from asf902(31916)<ec>:8888 > [info 2017/06/28 11:06:37.304 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:06:37.304 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}] > old view is: null > [info 2017/06/28 11:06:37.304 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v0>:8887 > [info 2017/06/28 11:06:37.307 UTC <Geode Membership View Creator> tid=0x70] > View Creator thread is starting > [info 2017/06/28 11:06:37.309 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8887] > old view is: null > [info 2017/06/28 11:06:37.310 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|2] members: [asf902(31916)<ec>:8887{lead}, > asf902(31916)<ec>:8890] > old view is: View[asf902(31916)<ec>:8888|1] members: > [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887] > [info 2017/06/28 11:06:37.310 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:06:37.310 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v14>:8887 > [info 2017/06/28 11:06:37.311 UTC <Geode Membership View Creator> tid=0x72] > View Creator thread is starting > [info 2017/06/28 11:06:37.312 UTC <Geode Membership View Creator> tid=0x72] > preparing new view View[asf902(31916)<ec><v14>:8887|14] members: > [asf902(31916)<ec><v14>:8887{lead}, asf902(31916)<ec>:8890] > [info 2017/06/28 11:06:42.314 UTC <Geode Membership View Creator> tid=0x72] > finished waiting for responses to view preparation > [warn 2017/06/28 11:06:42.315 UTC <Geode Membership View Creator> tid=0x72] > these members failed to respond to the view change: [asf902(31916)<ec>:8890] > [info 2017/06/28 11:06:42.315 UTC <Geode View Creator verification thread 1> > tid=0x75] checking state of member asf902(31916)<ec>:8890 > [info 2017/06/28 11:06:42.316 UTC <Geode View Creator verification thread 1> > tid=0x75] member asf902(31916)<ec>:8890 failed availability check > [info 2017/06/28 11:06:42.329 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8887|1] members: [asf902(31916)<ec>:8887{lead}, > asf902(31916)<ec>:8888] > old view is: null > [info 2017/06/28 11:06:42.329 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:06:42.330 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v6>:8887 > [info 2017/06/28 11:06:42.330 UTC <Test worker> tid=0x13] received join > request from asf902(31916)<ec>:8890 > [info 2017/06/28 11:06:42.330 UTC <Geode Membership View Creator> tid=0x77] > View Creator thread is starting > [info 2017/06/28 11:06:42.331 UTC <Test worker> tid=0x13] received join > request from asf902(31916)<ec>:8890 > [info 2017/06/28 11:06:42.331 UTC <Geode Membership View Creator> tid=0x77] > preparing new view View[asf902(31916)<ec><v6>:8887|6] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916)<ec>:8889] > [info 2017/06/28 11:06:47.332 UTC <Geode Membership View Creator> tid=0x77] > finished waiting for responses to view preparation > [warn 2017/06/28 11:06:47.333 UTC <Geode Membership View Creator> tid=0x77] > these members failed to respond to the view change: [asf902(31916)<ec>:8889, > asf902(31916)<ec>:8888] > [info 2017/06/28 11:06:47.334 UTC <Geode View Creator verification thread 1> > tid=0x78] checking state of member asf902(31916)<ec>:8889 > [info 2017/06/28 11:06:47.334 UTC <Geode View Creator verification thread 2> > tid=0x79] checking state of member asf902(31916)<ec>:8888 > [info 2017/06/28 11:06:47.334 UTC <Geode View Creator verification thread 1> > tid=0x78] member asf902(31916)<ec>:8889 passed availability check > [info 2017/06/28 11:06:47.335 UTC <Geode View Creator verification thread 2> > tid=0x79] member asf902(31916)<ec>:8888 passed availability check > [info 2017/06/28 11:06:47.335 UTC <Geode Membership View Creator> tid=0x77] > received new view: View[asf902(31916)<ec><v6>:8887|6] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916)<ec>:8889] > old view is: View[asf902(31916)<ec><v6>:8887|1] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916)<ec>:8889] > [info 2017/06/28 11:06:47.336 UTC <Geode Membership View Creator> tid=0x77] > sending new view View[asf902(31916)<ec><v6>:8887|6] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916)<ec>:8889] > [info 2017/06/28 11:06:47.637 UTC <Geode Membership View Creator> tid=0x77] > View Creator is processing 2 requests for the next membership view > [info 2017/06/28 11:06:47.637 UTC <Geode Membership View Creator> tid=0x77] > preparing new view View[asf902(31916)<ec><v6>:8887|7] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916)<ec>:8889, asf902(31916)<ec><v7>:8890] > [info 2017/06/28 11:06:52.638 UTC <Geode Membership View Creator> tid=0x77] > finished waiting for responses to view preparation > [warn 2017/06/28 11:06:52.639 UTC <Geode Membership View Creator> tid=0x77] > these members failed to respond to the view change: [asf902(31916)<ec>:8889, > asf902(31916)<ec>:8888, asf902(31916)<ec><v7>:8890] > [info 2017/06/28 11:06:52.639 UTC <Geode View Creator verification thread 1> > tid=0x7a] checking state of member asf902(31916)<ec>:8889 > [info 2017/06/28 11:06:52.640 UTC <Geode View Creator verification thread 3> > tid=0x7c] checking state of member asf902(31916)<ec><v7>:8890 > [info 2017/06/28 11:06:52.640 UTC <Geode View Creator verification thread 2> > tid=0x7b] checking state of member asf902(31916)<ec>:8888 > [info 2017/06/28 11:06:52.640 UTC <Geode View Creator verification thread 1> > tid=0x7a] member asf902(31916)<ec>:8889 passed availability check > [info 2017/06/28 11:06:52.640 UTC <Geode View Creator verification thread 3> > tid=0x7c] member asf902(31916)<ec><v7>:8890 passed availability check > [info 2017/06/28 11:06:52.641 UTC <Geode View Creator verification thread 2> > tid=0x7b] member asf902(31916)<ec>:8888 passed availability check > [info 2017/06/28 11:06:52.642 UTC <Geode Membership View Creator> tid=0x77] > received new view: View[asf902(31916)<ec><v6>:8887|7] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916)<ec>:8889, asf902(31916)<ec><v7>:8890] > old view is: View[asf902(31916)<ec><v6>:8887|6] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916)<ec>:8889] > [info 2017/06/28 11:06:52.642 UTC <Geode Membership View Creator> tid=0x77] > sending new view View[asf902(31916)<ec><v6>:8887|7] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916)<ec>:8889, asf902(31916)<ec><v7>:8890] > [info 2017/06/28 11:06:53.336 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8887|1] members: [asf902(31916)<ec>:8887{lead}, > asf902(31916)<ec>:8888] > old view is: null > [info 2017/06/28 11:06:53.337 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:06:53.337 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v6>:8887 > [info 2017/06/28 11:06:53.338 UTC <Geode Membership View Creator> tid=0x7e] > View Creator thread is starting > [info 2017/06/28 11:06:53.339 UTC <Geode Membership View Creator> tid=0x7e] > preparing new view View[asf902(31916)<ec><v6>:8887|7] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916)<ec><v6>:8889] > [info 2017/06/28 11:06:55.338 UTC <Geode Membership View Creator> tid=0x7e] > finished waiting for responses to view preparation > [info 2017/06/28 11:06:55.338 UTC <Geode Membership View Creator> tid=0x7e] > received new view: View[asf902(31916)<ec><v6>:8887|7] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916)<ec><v6>:8889] > old view is: View[asf902(31916)<ec><v6>:8887|1] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888] > [info 2017/06/28 11:06:55.340 UTC <Geode Membership View Creator> tid=0x7e] > sending new view View[asf902(31916)<ec><v6>:8887|7] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916)<ec><v6>:8889] > new view is View[asf902(31916)<ec><v6>:8887|7] members: > [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, > asf902(31916)<ec><v6>:8889] > [info 2017/06/28 11:06:55.445 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8887] > old view is: null > [info 2017/06/28 11:06:55.445 UTC <Test worker> tid=0x13] received leave > request from asf902(31916)<ec>:8888 for asf902(31916)<ec>:8888 > [info 2017/06/28 11:06:55.445 UTC <Test worker> tid=0x13] This member is > becoming the membership coordinator with address asf902(31916)<ec>:8887 > [info 2017/06/28 11:06:55.446 UTC <Test worker> tid=0x13] ViewCreator > starting on:asf902(31916)<ec><v8>:8887 > [info 2017/06/28 11:06:55.446 UTC <Geode Membership View Creator> tid=0x82] > View Creator thread is starting > [info 2017/06/28 11:06:55.449 UTC <Test worker> tid=0x13] received new view: > View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, > asf902(31916)<ec>:8887] > old view is: null > [fatal 2017/06/28 11:06:55.449 UTC <Test worker> tid=0x13] This member is no > longer in the membership view. My ID is asf902(31916)<ec>:8887 and the new > view is View[asf902(31916)<ec>:8888|2] members: > [asf902(31916)<ec>:8889{lead}, asf902(31916)<ec>:8890, asf902(31916)<ec>:8891] > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)