[jira] [Commented] (CASSANDRA-7713) CommitLogTest failure causes cascading unit test failures
[
https://issues.apache.org/jira/browse/CASSANDRA-7713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188930#comment-14188930
]
Michael Shuler commented on CASSANDRA-7713:
---
+1 - the last 4 cassandra-2.0 unit test jenkins runs have been bitten by this
problem, making the results useless
CommitLogTest failure causes cascading unit test failures
-
Key: CASSANDRA-7713
URL: https://issues.apache.org/jira/browse/CASSANDRA-7713
Project: Cassandra
Issue Type: Test
Reporter: Michael Shuler
Assignee: Bogdan Kanivets
Fix For: 2.0.12
Attachments: CommitLogTest.system.log.txt
When CommitLogTest.testCommitFailurePolicy_stop fails or times out,
{{commitDir.setWritable(true)}} is never reached, so the
build/test/cassandra/commitlog directory is left without write permissions,
causing cascading failure of all subsequent tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7713) CommitLogTest failure causes cascading unit test failures
[
https://issues.apache.org/jira/browse/CASSANDRA-7713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183104#comment-14183104
]
Joshua McKenzie commented on CASSANDRA-7713:
CASSANDRA-7927 is 2.1.2+ so we might want to backport the unit-test revision
logic into 2.0.12 (make handleCommitError public / VisibleForTesting, pass
exception to it rather than doing read-only directory approach).
CommitLogTest failure causes cascading unit test failures
-
Key: CASSANDRA-7713
URL: https://issues.apache.org/jira/browse/CASSANDRA-7713
Project: Cassandra
Issue Type: Test
Reporter: Michael Shuler
Assignee: Bogdan Kanivets
Fix For: 2.0.12
Attachments: CommitLogTest.system.log.txt
When CommitLogTest.testCommitFailurePolicy_stop fails or times out,
{{commitDir.setWritable(true)}} is never reached, so the
build/test/cassandra/commitlog directory is left without write permissions,
causing cascading failure of all subsequent tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7713) CommitLogTest failure causes cascading unit test failures
[
https://issues.apache.org/jira/browse/CASSANDRA-7713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14175097#comment-14175097
]
Joshua McKenzie commented on CASSANDRA-7713:
7927 removes the logic to set the directory read-only and instead creates the
exception and pushes it into the handler to let it flow through and shut things
down so this would no longer be an issue.
CommitLogTest failure causes cascading unit test failures
-
Key: CASSANDRA-7713
URL: https://issues.apache.org/jira/browse/CASSANDRA-7713
Project: Cassandra
Issue Type: Test
Reporter: Michael Shuler
Assignee: Bogdan Kanivets
Fix For: 2.0.12
Attachments: CommitLogTest.system.log.txt
When CommitLogTest.testCommitFailurePolicy_stop fails or times out,
{{commitDir.setWritable(true)}} is never reached, so the
build/test/cassandra/commitlog directory is left without write permissions,
causing cascading failure of all subsequent tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7713) CommitLogTest failure causes cascading unit test failures
[
https://issues.apache.org/jira/browse/CASSANDRA-7713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174345#comment-14174345
]
Michael Shuler commented on CASSANDRA-7713:
---
I tried adding an @After block to the test, which appears to not be run when
timeout occurs.. :(
{noformat}
diff --git a/test/unit/org/apache/cassandra/db/CommitLogTest.java
b/test/unit/org/apache/cassandra/db/CommitLogTest.java
index 1be29a6..f30d527 100644
--- a/test/unit/org/apache/cassandra/db/CommitLogTest.java
+++ b/test/unit/org/apache/cassandra/db/CommitLogTest.java
@@ -30,6 +30,7 @@ import java.util.zip.Checksum;
import com.google.common.util.concurrent.Uninterruptibles;
import org.junit.Assert;
import org.junit.Test;
+import org.junit.After;
import org.apache.cassandra.SchemaLoader;
import org.apache.cassandra.Util;
@@ -318,4 +319,12 @@ public class CommitLogTest extends SchemaLoader
row = command.getRow(notDurableKs);
Assert.assertEquals(null, row.cf);
}
+
+@After
+public void resetCommitLogDir()
+{
+File commitDir = new File(DatabaseDescriptor.getCommitLogLocation());
+commitDir.setWritable(true);
+System.out.println(reset commitlogdir: + commitDir);
+}
}
{noformat}
(System.out.println was just for debugging, and this is output during a
successful run)
CommitLogTest failure causes cascading unit test failures
-
Key: CASSANDRA-7713
URL: https://issues.apache.org/jira/browse/CASSANDRA-7713
Project: Cassandra
Issue Type: Test
Reporter: Michael Shuler
Assignee: Bogdan Kanivets
Fix For: 2.0.11
Attachments: CommitLogTest.system.log.txt
When CommitLogTest.testCommitFailurePolicy_stop fails or times out,
{{commitDir.setWritable(true)}} is never reached, so the
build/test/cassandra/commitlog directory is left without write permissions,
causing cascading failure of all subsequent tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7713) CommitLogTest failure causes cascading unit test failures
[
https://issues.apache.org/jira/browse/CASSANDRA-7713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174391#comment-14174391
]
Michael Shuler commented on CASSANDRA-7713:
---
An @AfterClass didn't help, either..
{noformat}
diff --git a/test/unit/org/apache/cassandra/db/CommitLogTest.java
b/test/unit/org/apache/cassandra/db/CommitLogTest.java
index 1be29a6..19faace 100644
--- a/test/unit/org/apache/cassandra/db/CommitLogTest.java
+++ b/test/unit/org/apache/cassandra/db/CommitLogTest.java
@@ -29,6 +29,7 @@ import java.util.zip.Checksum;
import com.google.common.util.concurrent.Uninterruptibles;
import org.junit.Assert;
+import org.junit.AfterClass;
import org.junit.Test;
import org.apache.cassandra.SchemaLoader;
@@ -48,6 +49,15 @@ import static
org.apache.cassandra.utils.ByteBufferUtil.bytes;
public class CommitLogTest extends SchemaLoader
{
+@AfterClass
+public static void resetCommitLogDir()
+{
+// junit timeout leaves commitDir non-writable - CASSANDRA-7713
+File commitDir = new File(DatabaseDescriptor.getCommitLogLocation());
+commitDir.setWritable(true);
+System.out.println(reset commitlogdir: + commitDir);
+}
+
@Test
public void testRecoveryWithEmptyLog() throws Exception
{
{noformat}
CommitLogTest failure causes cascading unit test failures
-
Key: CASSANDRA-7713
URL: https://issues.apache.org/jira/browse/CASSANDRA-7713
Project: Cassandra
Issue Type: Test
Reporter: Michael Shuler
Assignee: Bogdan Kanivets
Fix For: 2.0.11
Attachments: CommitLogTest.system.log.txt
When CommitLogTest.testCommitFailurePolicy_stop fails or times out,
{{commitDir.setWritable(true)}} is never reached, so the
build/test/cassandra/commitlog directory is left without write permissions,
causing cascading failure of all subsequent tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7713) CommitLogTest failure causes cascading unit test failures
[
https://issues.apache.org/jira/browse/CASSANDRA-7713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174392#comment-14174392
]
Michael Shuler commented on CASSANDRA-7713:
---
[~dankan] have you had any luck with this?
CommitLogTest failure causes cascading unit test failures
-
Key: CASSANDRA-7713
URL: https://issues.apache.org/jira/browse/CASSANDRA-7713
Project: Cassandra
Issue Type: Test
Reporter: Michael Shuler
Assignee: Bogdan Kanivets
Fix For: 2.0.11
Attachments: CommitLogTest.system.log.txt
When CommitLogTest.testCommitFailurePolicy_stop fails or times out,
{{commitDir.setWritable(true)}} is never reached, so the
build/test/cassandra/commitlog directory is left without write permissions,
causing cascading failure of all subsequent tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7713) CommitLogTest failure causes cascading unit test failures
[
https://issues.apache.org/jira/browse/CASSANDRA-7713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174435#comment-14174435
]
Michael Shuler commented on CASSANDRA-7713:
---
I'm half-way thinking that the way to handle this timeout problem is to simply
add a test that is run immediately after CommitLogTest that simply resets
{{commitDir.setWritable(true);}}
CommitLogTest failure causes cascading unit test failures
-
Key: CASSANDRA-7713
URL: https://issues.apache.org/jira/browse/CASSANDRA-7713
Project: Cassandra
Issue Type: Test
Reporter: Michael Shuler
Assignee: Bogdan Kanivets
Fix For: 2.0.11
Attachments: CommitLogTest.system.log.txt
When CommitLogTest.testCommitFailurePolicy_stop fails or times out,
{{commitDir.setWritable(true)}} is never reached, so the
build/test/cassandra/commitlog directory is left without write permissions,
causing cascading failure of all subsequent tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7713) CommitLogTest failure causes cascading unit test failures
[
https://issues.apache.org/jira/browse/CASSANDRA-7713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174571#comment-14174571
]
Bogdan Kanivets commented on CASSANDRA-7713:
Another way is to not use sleepUninterruptibly. It looks like this prevents
'finally' block to execute in time. That was my original solution, but then
I've noticed that there is more general problem with the test - when you remove
'logFile.setWritable(false)' it still passes.
I'll try to look into it this weekend to get a patch (have been busy lately),
but feel free to reassign
CommitLogTest failure causes cascading unit test failures
-
Key: CASSANDRA-7713
URL: https://issues.apache.org/jira/browse/CASSANDRA-7713
Project: Cassandra
Issue Type: Test
Reporter: Michael Shuler
Assignee: Bogdan Kanivets
Fix For: 2.0.11
Attachments: CommitLogTest.system.log.txt
When CommitLogTest.testCommitFailurePolicy_stop fails or times out,
{{commitDir.setWritable(true)}} is never reached, so the
build/test/cassandra/commitlog directory is left without write permissions,
causing cascading failure of all subsequent tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7713) CommitLogTest failure causes cascading unit test failures
[
https://issues.apache.org/jira/browse/CASSANDRA-7713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174588#comment-14174588
]
Joshua McKenzie commented on CASSANDRA-7713:
This came up while I was working on CASSANDRA-7927. The _stop test was always
returning true regardless of whether or not the stop case on
CommitLog.handleCommitError actually stopped anything or not, and having that
dangling directory w/out write permissions can cause problems (as evidenced
here).
I've updated that unit test in that ticket to actually pass the stop error in
rather than trying to simulate it to confirm the Gossiper is shutting down; if
the general public is happy with that solution we can close this as duplicate /
not a problem after that ticket.
CommitLogTest failure causes cascading unit test failures
-
Key: CASSANDRA-7713
URL: https://issues.apache.org/jira/browse/CASSANDRA-7713
Project: Cassandra
Issue Type: Test
Reporter: Michael Shuler
Assignee: Bogdan Kanivets
Fix For: 2.0.11
Attachments: CommitLogTest.system.log.txt
When CommitLogTest.testCommitFailurePolicy_stop fails or times out,
{{commitDir.setWritable(true)}} is never reached, so the
build/test/cassandra/commitlog directory is left without write permissions,
causing cascading failure of all subsequent tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7713) CommitLogTest failure causes cascading unit test failures
[
https://issues.apache.org/jira/browse/CASSANDRA-7713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174613#comment-14174613
]
Michael Shuler commented on CASSANDRA-7713:
---
Without any recommendation on CASSANDRA-7927, since I haven't had a good look
at it, we don't want to close this ticket without
fixing/replacing/removing/something CommitFailurePolicy_stop leaving this
non-writable dir behind - not only do following tests fail, things like `git
clean -df` don't work, if this test dorks. :)
I'd be happy if the solution was to pass in a code trigger of some sort than
actually setting the dir read-only - whatever tests functionality properly.
CommitLogTest failure causes cascading unit test failures
-
Key: CASSANDRA-7713
URL: https://issues.apache.org/jira/browse/CASSANDRA-7713
Project: Cassandra
Issue Type: Test
Reporter: Michael Shuler
Assignee: Bogdan Kanivets
Fix For: 2.0.11
Attachments: CommitLogTest.system.log.txt
When CommitLogTest.testCommitFailurePolicy_stop fails or times out,
{{commitDir.setWritable(true)}} is never reached, so the
build/test/cassandra/commitlog directory is left without write permissions,
causing cascading failure of all subsequent tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
