[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on hudson.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Cabrera updated ZOOKEEPER-690: --- Attachment: nohup-AsyncHammerTest-201004301209.txt jstack-AsyncHammerTest-201004301209.txt nohup-QuorumPeerMainTest-201004301209.txt AsyncTestHammer test fails on hudson. - Key: ZOOKEEPER-690 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690 Project: Zookeeper Issue Type: Bug Reporter: Mahadev konar Assignee: Henry Robinson Priority: Blocker Fix For: 3.3.1, 3.4.0 Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, jstack-AsyncHammerTest-201004301209.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, nohup-AsyncHammerTest-201004301209.txt, nohup-QuorumPeerMainTest-201004301209.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on hudson.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flavio Paiva Junqueira updated ZOOKEEPER-690: - Fix Version/s: (was: 3.3.1) Priority: Major (was: Blocker) Removing the blocker flag and making it major for 3.4.0. AsyncTestHammer test fails on hudson. - Key: ZOOKEEPER-690 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690 Project: Zookeeper Issue Type: Bug Reporter: Mahadev konar Assignee: Henry Robinson Fix For: 3.4.0 Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, jstack-AsyncHammerTest-201004301209.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, nohup-AsyncHammerTest-201004301209.txt, nohup-QuorumPeerMainTest-201004301209.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on hudson.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flavio Paiva Junqueira updated ZOOKEEPER-690: - Status: Open (was: Patch Available) AsyncTestHammer test fails on hudson. - Key: ZOOKEEPER-690 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690 Project: Zookeeper Issue Type: Bug Reporter: Mahadev konar Assignee: Henry Robinson Fix For: 3.4.0 Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, jstack-AsyncHammerTest-201004301209.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, nohup-AsyncHammerTest-201004301209.txt, nohup-QuorumPeerMainTest-201004301209.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on hudson.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henry Robinson updated ZOOKEEPER-690: - Attachment: ZOOKEEPER-690.patch I have found what I hope is the problem. Because QuorumPeers duplicate their 'LearnerType' in two places there's the possibility that they may get out of sync. This is what was happening here - it was a test bug. Although the Observers knew that they were Observers, the other nodes did not. This affected the leader election protocol as other node did not know to reject an Observer. I feel like we should refactor the QuorumPeer.QuorumServer code so as not to duplicate information, but for the time being I think this patch will work. I have also taken the opportunity to standardise the naming of 'learnertype' throughout the code (in some places it was called 'peertype' adding to the confusion). Tests pass on my machine, but I can't guarantee that the problem is fixed as I could never recreate the error. Thanks to Flavio for catching the broken invariant! AsyncTestHammer test fails on hudson. - Key: ZOOKEEPER-690 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690 Project: Zookeeper Issue Type: Bug Reporter: Mahadev konar Assignee: Henry Robinson Priority: Blocker Fix For: 3.3.1, 3.4.0 Attachments: jstack-201004201053.txt, nohup-201004201053.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on hudson.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henry Robinson updated ZOOKEEPER-690: - Status: Patch Available (was: Open) AsyncTestHammer test fails on hudson. - Key: ZOOKEEPER-690 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690 Project: Zookeeper Issue Type: Bug Reporter: Mahadev konar Assignee: Henry Robinson Priority: Blocker Fix For: 3.3.1, 3.4.0 Attachments: jstack-201004201053.txt, nohup-201004201053.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on hudson.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Cabrera updated ZOOKEEPER-690: --- Attachment: TEST-org.apache.zookeeper.test.AsyncHammerTest.txt AsyncTestHammer test fails on hudson. - Key: ZOOKEEPER-690 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690 Project: Zookeeper Issue Type: Bug Reporter: Mahadev konar Assignee: Henry Robinson Priority: Blocker Fix For: 3.3.1, 3.4.0 Attachments: jstack-201004201053.txt, nohup-201004201053.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on hudson.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Cabrera updated ZOOKEEPER-690: --- Attachment: (was: TEST-org.apache.zookeeper.test.AsyncHammerTest.txt) AsyncTestHammer test fails on hudson. - Key: ZOOKEEPER-690 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690 Project: Zookeeper Issue Type: Bug Reporter: Mahadev konar Assignee: Henry Robinson Priority: Blocker Fix For: 3.3.1, 3.4.0 Attachments: jstack-201004201053.txt, nohup-201004201053.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on hudson.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Cabrera updated ZOOKEEPER-690: --- Attachment: nohup-201004291409.txt jstack-201004291409.txt Disregard the last comment. I have now cleaned and rebuilt and now it still locks up. AsyncTestHammer test fails on hudson. - Key: ZOOKEEPER-690 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690 Project: Zookeeper Issue Type: Bug Reporter: Mahadev konar Assignee: Henry Robinson Priority: Blocker Fix For: 3.3.1, 3.4.0 Attachments: jstack-201004201053.txt, jstack-201004291409.txt, nohup-201004201053.txt, nohup-201004291409.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on hudson.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henry Robinson updated ZOOKEEPER-690: - Attachment: ZOOKEEPER-690.patch Alan - would you mind trying this new patch? Thanks for your patience. I suspect that something might still be a bit flaky with these tests (not the code, but the tests), but I hope this will fix this particular problem. AsyncTestHammer test fails on hudson. - Key: ZOOKEEPER-690 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690 Project: Zookeeper Issue Type: Bug Reporter: Mahadev konar Assignee: Henry Robinson Priority: Blocker Fix For: 3.3.1, 3.4.0 Attachments: jstack-201004201053.txt, jstack-201004291409.txt, nohup-201004201053.txt, nohup-201004291409.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on hudson.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Hunt updated ZOOKEEPER-690: --- Fix Version/s: (was: 3.3.0) 3.3.1 not a blocker for 3.3.0, tracking in 3.3.1. Have not seen these since, ie not reproduceable but let's keep an eye open. AsyncTestHammer test fails on hudson. - Key: ZOOKEEPER-690 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690 Project: Zookeeper Issue Type: Bug Reporter: Mahadev konar Assignee: Patrick Hunt Priority: Critical Fix For: 3.3.1 the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on hudson.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Hunt updated ZOOKEEPER-690: --- Priority: Critical (was: Blocker) downgrading from blocker, haven't seen this again after many runs. we'll keep an eye but shouldn't block 3.3.0 (and may have been fixed) AsyncTestHammer test fails on hudson. - Key: ZOOKEEPER-690 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690 Project: Zookeeper Issue Type: Bug Reporter: Mahadev konar Assignee: Patrick Hunt Priority: Critical Fix For: 3.3.0 the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on hudson.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated ZOOKEEPER-690: Fix Version/s: (was: 3.4.0) 3.3.0 AsyncTestHammer test fails on hudson. - Key: ZOOKEEPER-690 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690 Project: Zookeeper Issue Type: Bug Reporter: Mahadev konar Assignee: Mahadev konar Priority: Blocker Fix For: 3.3.0 the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
