[jira] [Comment Edited] (CASSANDRA-13265) Expiration in OutboundTcpConnection can block the reader Thread

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894783#comment-15894783
 ] 

Jason Brown edited comment on CASSANDRA-13265 at 3/5/17 2:04 AM:
-

I'm +1 on the ticket, but [~aweisberg] should also weigh in.

As a note, we have {{ApproximateTime}} for an estimated time, but that works 
off of {{System.currentTimeMillis()}} rather than {{System.nanoTime()}}, so 
monotonicity.

UPDATE: Actually, I'd like to see a unit test implemented, like what I 
described earlier.


was (Author: jasobrown):
I'm +1 on the ticket, but [~aweisberg] should also weigh in.

As a note, we have {{ApproximateTime}} for an estimated time, but that works 
off of {{System.currentTimeMillis()}} rather than {{System.nanoTime()}}, so 
monotonicity.

> Expiration in OutboundTcpConnection can block the reader Thread
> ---
>
> Key: CASSANDRA-13265
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13265
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 3.0.9
> Java HotSpot(TM) 64-Bit Server VM version 25.112-b15 (Java version 
> 1.8.0_112-b15)
> Linux 3.16
>Reporter: Christian Esken
>Assignee: Christian Esken
> Attachments: cassandra.pb-cache4-dus.2017-02-17-19-36-26.chist.xz, 
> cassandra.pb-cache4-dus.2017-02-17-19-36-26.td.xz
>
>
> I observed that sometimes a single node in a Cassandra cluster fails to 
> communicate to the other nodes. This can happen at any time, during peak load 
> or low load. Restarting that single node from the cluster fixes the issue.
> Before going in to details, I want to state that I have analyzed the 
> situation and am already developing a possible fix. Here is the analysis so 
> far:
> - A Threaddump in this situation showed  324 Threads in the 
> OutboundTcpConnection class that want to lock the backlog queue for doing 
> expiration.
> - A class histogram shows 262508 instances of 
> OutboundTcpConnection$QueuedMessage.
> What is the effect of it? As soon as the Cassandra node has reached a certain 
> amount of queued messages, it starts thrashing itself to death. Each of the 
> Thread fully locks the Queue for reading and writing by calling 
> iterator.next(), making the situation worse and worse.
> - Writing: Only after 262508 locking operation it can progress with actually 
> writing to the Queue.
> - Reading: Is also blocked, as 324 Threads try to do iterator.next(), and 
> fully lock the Queue
> This means: Writing blocks the Queue for reading, and readers might even be 
> starved which makes the situation even worse.
> -
> The setup is:
>  - 3-node cluster
>  - replication factor 2
>  - Consistency LOCAL_ONE
>  - No remote DC's
>  - high write throughput (10 INSERT statements per second and more during 
> peak times).
>  



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13231) org.apache.cassandra.db.DirectoriesTest(testStandardDirs) unit test failing

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-13231:

   Resolution: Fixed
Fix Version/s: 4.0
   3.11.0
   3.0.12
   2.2.10
Reproduced In: 2.2.9, 4.x  (was: 4.x)
   Status: Resolved  (was: Patch Available)

committed as sha {{44fefeffa53cf95223c4c7b6f79afaafe9b93e2e}} to 2.2, 3.0, 
3.11, and trunk. Thanks!

> org.apache.cassandra.db.DirectoriesTest(testStandardDirs) unit test failing
> ---
>
> Key: CASSANDRA-13231
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13231
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Kjellman
>Assignee: Michael Kjellman
> Fix For: 2.2.10, 3.0.12, 3.11.0, 4.0
>
> Attachments: 674.diff
>
>
> The testStandardDirs(org.apache.cassandra.db.DirectoriesTest) unit test 
> always fails. This appears to be due to a commit by Yuki for CASSANDRA-10587 
> which switched the SSTable descriptor to use the canonical path.
> From one of Yuki's comments in CASSANDRA-10587:
> "I ended up fixing Descriptor object to always have canonical path as its 
> directory.
> This way we don't need to think about given directory is relative or absolute.
> In fact, right now Desctiptor (and corresponding SSTable) is not considered 
> equal between Descriptor's directory being relative and absolute. (Added 
> simple unit test to DescriptorTest)."
> The issue here is that canonical path will expand out differently than even 
> absolute path. In this case /var/folders -> /private/var/folders. The unit 
> test is looking for /var/folders/... but the Descriptor expands out to 
> /private/var/folders and the unit test fails.
> Descriptor#L88 seems to be the real root cause.
>[junit] Testcase: 
> testStandardDirs(org.apache.cassandra.db.DirectoriesTest):   FAILED
> [junit] 
> expected:
>  but 
> was:
> [junit] junit.framework.AssertionFailedError: 
> expected:
>  but 
> was:
> [junit]   at 
> org.apache.cassandra.db.DirectoriesTest.testStandardDirs(DirectoriesTest.java:159)
> [junit] 
> [junit] 
> [junit] Test org.apache.cassandra.db.DirectoriesTest FAILED
> I'm guessing given we went to canonicalPath() on purpose the "fix" here is to 
> call .getCanonicalFile() on both expected Files generated (snapshotDir and 
> backupsDir) for the junit assert.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13231) org.apache.cassandra.db.DirectoriesTest(testStandardDirs) unit test failing

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895952#comment-15895952
 ] 

Jason Brown commented on CASSANDRA-13231:
-

Note: I didn't update CHANGES.txt as this was only a fix for unit tests, and 
then only on one platform.

> org.apache.cassandra.db.DirectoriesTest(testStandardDirs) unit test failing
> ---
>
> Key: CASSANDRA-13231
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13231
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Kjellman
>Assignee: Michael Kjellman
> Fix For: 2.2.10, 3.0.12, 3.11.0, 4.0
>
> Attachments: 674.diff
>
>
> The testStandardDirs(org.apache.cassandra.db.DirectoriesTest) unit test 
> always fails. This appears to be due to a commit by Yuki for CASSANDRA-10587 
> which switched the SSTable descriptor to use the canonical path.
> From one of Yuki's comments in CASSANDRA-10587:
> "I ended up fixing Descriptor object to always have canonical path as its 
> directory.
> This way we don't need to think about given directory is relative or absolute.
> In fact, right now Desctiptor (and corresponding SSTable) is not considered 
> equal between Descriptor's directory being relative and absolute. (Added 
> simple unit test to DescriptorTest)."
> The issue here is that canonical path will expand out differently than even 
> absolute path. In this case /var/folders -> /private/var/folders. The unit 
> test is looking for /var/folders/... but the Descriptor expands out to 
> /private/var/folders and the unit test fails.
> Descriptor#L88 seems to be the real root cause.
>[junit] Testcase: 
> testStandardDirs(org.apache.cassandra.db.DirectoriesTest):   FAILED
> [junit] 
> expected:
>  but 
> was:
> [junit] junit.framework.AssertionFailedError: 
> expected:
>  but 
> was:
> [junit]   at 
> org.apache.cassandra.db.DirectoriesTest.testStandardDirs(DirectoriesTest.java:159)
> [junit] 
> [junit] 
> [junit] Test org.apache.cassandra.db.DirectoriesTest FAILED
> I'm guessing given we went to canonicalPath() on purpose the "fix" here is to 
> call .getCanonicalFile() on both expected Files generated (snapshotDir and 
> backupsDir) for the junit assert.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[07/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jasobrown]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/77d45ea5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/77d45ea5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/77d45ea5

Branch: refs/heads/trunk
Commit: 77d45ea53ebabd92c02e1da8c143a2826b0cbd5d
Parents: e3968cf 44fefef
Author: Jason Brown 
Authored: Sat Mar 4 15:37:38 2017 -0800
Committer: Jason Brown 
Committed: Sat Mar 4 15:38:20 2017 -0800

--
 test/unit/org/apache/cassandra/db/DirectoriesTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/77d45ea5/test/unit/org/apache/cassandra/db/DirectoriesTest.java
--
diff --cc test/unit/org/apache/cassandra/db/DirectoriesTest.java
index f8d01a8,bc5b7bf..7c4ff7a
--- a/test/unit/org/apache/cassandra/db/DirectoriesTest.java
+++ b/test/unit/org/apache/cassandra/db/DirectoriesTest.java
@@@ -152,12 -144,12 +152,12 @@@ public class DirectoriesTes
  Directories directories = new Directories(cfm);
  assertEquals(cfDir(cfm), 
directories.getDirectoryForNewSSTables());
  
 -Descriptor desc = new Descriptor(cfDir(cfm), KS, cfm.cfName, 1, 
Descriptor.Type.FINAL);
 +Descriptor desc = new Descriptor(cfDir(cfm), KS, cfm.cfName, 1);
  File snapshotDir = new File(cfDir(cfm),  File.separator + 
Directories.SNAPSHOT_SUBDIR + File.separator + "42");
- assertEquals(snapshotDir, Directories.getSnapshotDirectory(desc, 
"42"));
+ assertEquals(snapshotDir.getCanonicalFile(), 
Directories.getSnapshotDirectory(desc, "42"));
  
  File backupsDir = new File(cfDir(cfm),  File.separator + 
Directories.BACKUPS_SUBDIR);
- assertEquals(backupsDir, Directories.getBackupsDirectory(desc));
+ assertEquals(backupsDir.getCanonicalFile(), 
Directories.getBackupsDirectory(desc));
  }
  }
  

[02/10] cassandra git commit: org.apache.cassandra.db.DirectoriesTest(testStandardDirs) unit test failing

[jasobrown]

org.apache.cassandra.db.DirectoriesTest(testStandardDirs) unit test failing

patch by mkjellman, reviewed by jasobrown for CASSANDRA-13231


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/44fefeff
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/44fefeff
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/44fefeff

Branch: refs/heads/cassandra-3.0
Commit: 44fefeffa53cf95223c4c7b6f79afaafe9b93e2e
Parents: 288d243
Author: Michael Kjellman 
Authored: Sat Mar 4 04:57:34 2017 -0800
Committer: Jason Brown 
Committed: Sat Mar 4 15:37:19 2017 -0800

--
 test/unit/org/apache/cassandra/db/DirectoriesTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/44fefeff/test/unit/org/apache/cassandra/db/DirectoriesTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/DirectoriesTest.java 
b/test/unit/org/apache/cassandra/db/DirectoriesTest.java
index 008e343..bc5b7bf 100644
--- a/test/unit/org/apache/cassandra/db/DirectoriesTest.java
+++ b/test/unit/org/apache/cassandra/db/DirectoriesTest.java
@@ -137,7 +137,7 @@ public class DirectoriesTest
 }
 
 @Test
-public void testStandardDirs()
+public void testStandardDirs() throws IOException
 {
 for (CFMetaData cfm : CFM)
 {
@@ -146,10 +146,10 @@ public class DirectoriesTest
 
 Descriptor desc = new Descriptor(cfDir(cfm), KS, cfm.cfName, 1, 
Descriptor.Type.FINAL);
 File snapshotDir = new File(cfDir(cfm),  File.separator + 
Directories.SNAPSHOT_SUBDIR + File.separator + "42");
-assertEquals(snapshotDir, Directories.getSnapshotDirectory(desc, 
"42"));
+assertEquals(snapshotDir.getCanonicalFile(), 
Directories.getSnapshotDirectory(desc, "42"));
 
 File backupsDir = new File(cfDir(cfm),  File.separator + 
Directories.BACKUPS_SUBDIR);
-assertEquals(backupsDir, Directories.getBackupsDirectory(desc));
+assertEquals(backupsDir.getCanonicalFile(), 
Directories.getBackupsDirectory(desc));
 }
 }
 

[05/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jasobrown]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/77d45ea5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/77d45ea5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/77d45ea5

Branch: refs/heads/cassandra-3.11
Commit: 77d45ea53ebabd92c02e1da8c143a2826b0cbd5d
Parents: e3968cf 44fefef
Author: Jason Brown 
Authored: Sat Mar 4 15:37:38 2017 -0800
Committer: Jason Brown 
Committed: Sat Mar 4 15:38:20 2017 -0800

--
 test/unit/org/apache/cassandra/db/DirectoriesTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/77d45ea5/test/unit/org/apache/cassandra/db/DirectoriesTest.java
--
diff --cc test/unit/org/apache/cassandra/db/DirectoriesTest.java
index f8d01a8,bc5b7bf..7c4ff7a
--- a/test/unit/org/apache/cassandra/db/DirectoriesTest.java
+++ b/test/unit/org/apache/cassandra/db/DirectoriesTest.java
@@@ -152,12 -144,12 +152,12 @@@ public class DirectoriesTes
  Directories directories = new Directories(cfm);
  assertEquals(cfDir(cfm), 
directories.getDirectoryForNewSSTables());
  
 -Descriptor desc = new Descriptor(cfDir(cfm), KS, cfm.cfName, 1, 
Descriptor.Type.FINAL);
 +Descriptor desc = new Descriptor(cfDir(cfm), KS, cfm.cfName, 1);
  File snapshotDir = new File(cfDir(cfm),  File.separator + 
Directories.SNAPSHOT_SUBDIR + File.separator + "42");
- assertEquals(snapshotDir, Directories.getSnapshotDirectory(desc, 
"42"));
+ assertEquals(snapshotDir.getCanonicalFile(), 
Directories.getSnapshotDirectory(desc, "42"));
  
  File backupsDir = new File(cfDir(cfm),  File.separator + 
Directories.BACKUPS_SUBDIR);
- assertEquals(backupsDir, Directories.getBackupsDirectory(desc));
+ assertEquals(backupsDir.getCanonicalFile(), 
Directories.getBackupsDirectory(desc));
  }
  }
  

[01/10] cassandra git commit: org.apache.cassandra.db.DirectoriesTest(testStandardDirs) unit test failing

[jasobrown]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 288d243d8 -> 44fefeffa
  refs/heads/cassandra-3.0 e3968cfd1 -> 77d45ea53
  refs/heads/cassandra-3.11 92b7322bb -> 7f5dc696f
  refs/heads/trunk 979b19b7c -> 1b552658b


org.apache.cassandra.db.DirectoriesTest(testStandardDirs) unit test failing

patch by mkjellman, reviewed by jasobrown for CASSANDRA-13231


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/44fefeff
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/44fefeff
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/44fefeff

Branch: refs/heads/cassandra-2.2
Commit: 44fefeffa53cf95223c4c7b6f79afaafe9b93e2e
Parents: 288d243
Author: Michael Kjellman 
Authored: Sat Mar 4 04:57:34 2017 -0800
Committer: Jason Brown 
Committed: Sat Mar 4 15:37:19 2017 -0800

--
 test/unit/org/apache/cassandra/db/DirectoriesTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/44fefeff/test/unit/org/apache/cassandra/db/DirectoriesTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/DirectoriesTest.java 
b/test/unit/org/apache/cassandra/db/DirectoriesTest.java
index 008e343..bc5b7bf 100644
--- a/test/unit/org/apache/cassandra/db/DirectoriesTest.java
+++ b/test/unit/org/apache/cassandra/db/DirectoriesTest.java
@@ -137,7 +137,7 @@ public class DirectoriesTest
 }
 
 @Test
-public void testStandardDirs()
+public void testStandardDirs() throws IOException
 {
 for (CFMetaData cfm : CFM)
 {
@@ -146,10 +146,10 @@ public class DirectoriesTest
 
 Descriptor desc = new Descriptor(cfDir(cfm), KS, cfm.cfName, 1, 
Descriptor.Type.FINAL);
 File snapshotDir = new File(cfDir(cfm),  File.separator + 
Directories.SNAPSHOT_SUBDIR + File.separator + "42");
-assertEquals(snapshotDir, Directories.getSnapshotDirectory(desc, 
"42"));
+assertEquals(snapshotDir.getCanonicalFile(), 
Directories.getSnapshotDirectory(desc, "42"));
 
 File backupsDir = new File(cfDir(cfm),  File.separator + 
Directories.BACKUPS_SUBDIR);
-assertEquals(backupsDir, Directories.getBackupsDirectory(desc));
+assertEquals(backupsDir.getCanonicalFile(), 
Directories.getBackupsDirectory(desc));
 }
 }
 

[06/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jasobrown]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/77d45ea5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/77d45ea5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/77d45ea5

Branch: refs/heads/cassandra-3.0
Commit: 77d45ea53ebabd92c02e1da8c143a2826b0cbd5d
Parents: e3968cf 44fefef
Author: Jason Brown 
Authored: Sat Mar 4 15:37:38 2017 -0800
Committer: Jason Brown 
Committed: Sat Mar 4 15:38:20 2017 -0800

--
 test/unit/org/apache/cassandra/db/DirectoriesTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/77d45ea5/test/unit/org/apache/cassandra/db/DirectoriesTest.java
--
diff --cc test/unit/org/apache/cassandra/db/DirectoriesTest.java
index f8d01a8,bc5b7bf..7c4ff7a
--- a/test/unit/org/apache/cassandra/db/DirectoriesTest.java
+++ b/test/unit/org/apache/cassandra/db/DirectoriesTest.java
@@@ -152,12 -144,12 +152,12 @@@ public class DirectoriesTes
  Directories directories = new Directories(cfm);
  assertEquals(cfDir(cfm), 
directories.getDirectoryForNewSSTables());
  
 -Descriptor desc = new Descriptor(cfDir(cfm), KS, cfm.cfName, 1, 
Descriptor.Type.FINAL);
 +Descriptor desc = new Descriptor(cfDir(cfm), KS, cfm.cfName, 1);
  File snapshotDir = new File(cfDir(cfm),  File.separator + 
Directories.SNAPSHOT_SUBDIR + File.separator + "42");
- assertEquals(snapshotDir, Directories.getSnapshotDirectory(desc, 
"42"));
+ assertEquals(snapshotDir.getCanonicalFile(), 
Directories.getSnapshotDirectory(desc, "42"));
  
  File backupsDir = new File(cfDir(cfm),  File.separator + 
Directories.BACKUPS_SUBDIR);
- assertEquals(backupsDir, Directories.getBackupsDirectory(desc));
+ assertEquals(backupsDir.getCanonicalFile(), 
Directories.getBackupsDirectory(desc));
  }
  }
  

[03/10] cassandra git commit: org.apache.cassandra.db.DirectoriesTest(testStandardDirs) unit test failing

[jasobrown]

org.apache.cassandra.db.DirectoriesTest(testStandardDirs) unit test failing

patch by mkjellman, reviewed by jasobrown for CASSANDRA-13231


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/44fefeff
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/44fefeff
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/44fefeff

Branch: refs/heads/cassandra-3.11
Commit: 44fefeffa53cf95223c4c7b6f79afaafe9b93e2e
Parents: 288d243
Author: Michael Kjellman 
Authored: Sat Mar 4 04:57:34 2017 -0800
Committer: Jason Brown 
Committed: Sat Mar 4 15:37:19 2017 -0800

--
 test/unit/org/apache/cassandra/db/DirectoriesTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/44fefeff/test/unit/org/apache/cassandra/db/DirectoriesTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/DirectoriesTest.java 
b/test/unit/org/apache/cassandra/db/DirectoriesTest.java
index 008e343..bc5b7bf 100644
--- a/test/unit/org/apache/cassandra/db/DirectoriesTest.java
+++ b/test/unit/org/apache/cassandra/db/DirectoriesTest.java
@@ -137,7 +137,7 @@ public class DirectoriesTest
 }
 
 @Test
-public void testStandardDirs()
+public void testStandardDirs() throws IOException
 {
 for (CFMetaData cfm : CFM)
 {
@@ -146,10 +146,10 @@ public class DirectoriesTest
 
 Descriptor desc = new Descriptor(cfDir(cfm), KS, cfm.cfName, 1, 
Descriptor.Type.FINAL);
 File snapshotDir = new File(cfDir(cfm),  File.separator + 
Directories.SNAPSHOT_SUBDIR + File.separator + "42");
-assertEquals(snapshotDir, Directories.getSnapshotDirectory(desc, 
"42"));
+assertEquals(snapshotDir.getCanonicalFile(), 
Directories.getSnapshotDirectory(desc, "42"));
 
 File backupsDir = new File(cfDir(cfm),  File.separator + 
Directories.BACKUPS_SUBDIR);
-assertEquals(backupsDir, Directories.getBackupsDirectory(desc));
+assertEquals(backupsDir.getCanonicalFile(), 
Directories.getBackupsDirectory(desc));
 }
 }
 

[08/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[jasobrown]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7f5dc696
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7f5dc696
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7f5dc696

Branch: refs/heads/cassandra-3.11
Commit: 7f5dc696f6658d2e0a40c24276e858a1cf30ce85
Parents: 92b7322 77d45ea
Author: Jason Brown 
Authored: Sat Mar 4 15:38:39 2017 -0800
Committer: Jason Brown 
Committed: Sat Mar 4 15:39:11 2017 -0800

--
 test/unit/org/apache/cassandra/db/DirectoriesTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7f5dc696/test/unit/org/apache/cassandra/db/DirectoriesTest.java
--
diff --cc test/unit/org/apache/cassandra/db/DirectoriesTest.java
index 5841c36,7c4ff7a..7e800e6
--- a/test/unit/org/apache/cassandra/db/DirectoriesTest.java
+++ b/test/unit/org/apache/cassandra/db/DirectoriesTest.java
@@@ -156,12 -152,12 +156,12 @@@ public class DirectoriesTes
  Directories directories = new Directories(cfm);
  assertEquals(cfDir(cfm), 
directories.getDirectoryForNewSSTables());
  
 -Descriptor desc = new Descriptor(cfDir(cfm), KS, cfm.cfName, 1);
 +Descriptor desc = new Descriptor(cfDir(cfm), KS, cfm.cfName, 1, 
SSTableFormat.Type.BIG);
  File snapshotDir = new File(cfDir(cfm),  File.separator + 
Directories.SNAPSHOT_SUBDIR + File.separator + "42");
- assertEquals(snapshotDir, Directories.getSnapshotDirectory(desc, 
"42"));
+ assertEquals(snapshotDir.getCanonicalFile(), 
Directories.getSnapshotDirectory(desc, "42"));
  
  File backupsDir = new File(cfDir(cfm),  File.separator + 
Directories.BACKUPS_SUBDIR);
- assertEquals(backupsDir, Directories.getBackupsDirectory(desc));
+ assertEquals(backupsDir.getCanonicalFile(), 
Directories.getBackupsDirectory(desc));
  }
  }
  

[10/10] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[jasobrown]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1b552658
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1b552658
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1b552658

Branch: refs/heads/trunk
Commit: 1b552658bdf132b3f8921f2fa22d8d3fdcf2a2b2
Parents: 979b19b 7f5dc69
Author: Jason Brown 
Authored: Sat Mar 4 15:39:28 2017 -0800
Committer: Jason Brown 
Committed: Sat Mar 4 15:39:59 2017 -0800

--
 test/unit/org/apache/cassandra/db/DirectoriesTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1b552658/test/unit/org/apache/cassandra/db/DirectoriesTest.java
--
diff --cc test/unit/org/apache/cassandra/db/DirectoriesTest.java
index 904354a,7e800e6..ffd4d3e
--- a/test/unit/org/apache/cassandra/db/DirectoriesTest.java
+++ b/test/unit/org/apache/cassandra/db/DirectoriesTest.java
@@@ -147,19 -149,19 +147,19 @@@ public class DirectoriesTes
  }
  
  @Test
- public void testStandardDirs()
+ public void testStandardDirs() throws IOException
  {
 -for (CFMetaData cfm : CFM)
 +for (TableMetadata cfm : CFM)
  {
  Directories directories = new Directories(cfm);
  assertEquals(cfDir(cfm), 
directories.getDirectoryForNewSSTables());
  
 -Descriptor desc = new Descriptor(cfDir(cfm), KS, cfm.cfName, 1, 
SSTableFormat.Type.BIG);
 +Descriptor desc = new Descriptor(cfDir(cfm), KS, cfm.name, 1, 
SSTableFormat.Type.BIG);
  File snapshotDir = new File(cfDir(cfm),  File.separator + 
Directories.SNAPSHOT_SUBDIR + File.separator + "42");
- assertEquals(snapshotDir, Directories.getSnapshotDirectory(desc, 
"42"));
+ assertEquals(snapshotDir.getCanonicalFile(), 
Directories.getSnapshotDirectory(desc, "42"));
  
  File backupsDir = new File(cfDir(cfm),  File.separator + 
Directories.BACKUPS_SUBDIR);
- assertEquals(backupsDir, Directories.getBackupsDirectory(desc));
+ assertEquals(backupsDir.getCanonicalFile(), 
Directories.getBackupsDirectory(desc));
  }
  }
  

[04/10] cassandra git commit: org.apache.cassandra.db.DirectoriesTest(testStandardDirs) unit test failing

[jasobrown]

org.apache.cassandra.db.DirectoriesTest(testStandardDirs) unit test failing

patch by mkjellman, reviewed by jasobrown for CASSANDRA-13231


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/44fefeff
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/44fefeff
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/44fefeff

Branch: refs/heads/trunk
Commit: 44fefeffa53cf95223c4c7b6f79afaafe9b93e2e
Parents: 288d243
Author: Michael Kjellman 
Authored: Sat Mar 4 04:57:34 2017 -0800
Committer: Jason Brown 
Committed: Sat Mar 4 15:37:19 2017 -0800

--
 test/unit/org/apache/cassandra/db/DirectoriesTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/44fefeff/test/unit/org/apache/cassandra/db/DirectoriesTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/DirectoriesTest.java 
b/test/unit/org/apache/cassandra/db/DirectoriesTest.java
index 008e343..bc5b7bf 100644
--- a/test/unit/org/apache/cassandra/db/DirectoriesTest.java
+++ b/test/unit/org/apache/cassandra/db/DirectoriesTest.java
@@ -137,7 +137,7 @@ public class DirectoriesTest
 }
 
 @Test
-public void testStandardDirs()
+public void testStandardDirs() throws IOException
 {
 for (CFMetaData cfm : CFM)
 {
@@ -146,10 +146,10 @@ public class DirectoriesTest
 
 Descriptor desc = new Descriptor(cfDir(cfm), KS, cfm.cfName, 1, 
Descriptor.Type.FINAL);
 File snapshotDir = new File(cfDir(cfm),  File.separator + 
Directories.SNAPSHOT_SUBDIR + File.separator + "42");
-assertEquals(snapshotDir, Directories.getSnapshotDirectory(desc, 
"42"));
+assertEquals(snapshotDir.getCanonicalFile(), 
Directories.getSnapshotDirectory(desc, "42"));
 
 File backupsDir = new File(cfDir(cfm),  File.separator + 
Directories.BACKUPS_SUBDIR);
-assertEquals(backupsDir, Directories.getBackupsDirectory(desc));
+assertEquals(backupsDir.getCanonicalFile(), 
Directories.getBackupsDirectory(desc));
 }
 }
 

[09/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[jasobrown]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7f5dc696
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7f5dc696
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7f5dc696

Branch: refs/heads/trunk
Commit: 7f5dc696f6658d2e0a40c24276e858a1cf30ce85
Parents: 92b7322 77d45ea
Author: Jason Brown 
Authored: Sat Mar 4 15:38:39 2017 -0800
Committer: Jason Brown 
Committed: Sat Mar 4 15:39:11 2017 -0800

--
 test/unit/org/apache/cassandra/db/DirectoriesTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7f5dc696/test/unit/org/apache/cassandra/db/DirectoriesTest.java
--
diff --cc test/unit/org/apache/cassandra/db/DirectoriesTest.java
index 5841c36,7c4ff7a..7e800e6
--- a/test/unit/org/apache/cassandra/db/DirectoriesTest.java
+++ b/test/unit/org/apache/cassandra/db/DirectoriesTest.java
@@@ -156,12 -152,12 +156,12 @@@ public class DirectoriesTes
  Directories directories = new Directories(cfm);
  assertEquals(cfDir(cfm), 
directories.getDirectoryForNewSSTables());
  
 -Descriptor desc = new Descriptor(cfDir(cfm), KS, cfm.cfName, 1);
 +Descriptor desc = new Descriptor(cfDir(cfm), KS, cfm.cfName, 1, 
SSTableFormat.Type.BIG);
  File snapshotDir = new File(cfDir(cfm),  File.separator + 
Directories.SNAPSHOT_SUBDIR + File.separator + "42");
- assertEquals(snapshotDir, Directories.getSnapshotDirectory(desc, 
"42"));
+ assertEquals(snapshotDir.getCanonicalFile(), 
Directories.getSnapshotDirectory(desc, "42"));
  
  File backupsDir = new File(cfDir(cfm),  File.separator + 
Directories.BACKUPS_SUBDIR);
- assertEquals(backupsDir, Directories.getBackupsDirectory(desc));
+ assertEquals(backupsDir.getCanonicalFile(), 
Directories.getBackupsDirectory(desc));
  }
  }
  

[jira] [Updated] (CASSANDRA-12837) Add multi-threaded support to nodetool rebuild_index

[vincent royer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

vincent royer updated CASSANDRA-12837:
--
Fix Version/s: (was: 3.0.12)
   3.11.0

> Add multi-threaded support to nodetool rebuild_index
> 
>
> Key: CASSANDRA-12837
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12837
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: vincent royer
>Priority: Minor
>  Labels: patch, performance
> Fix For: 3.11.0, 4.x
>
> Attachments: 0001-CASSANDRA-12837-multi-threaded-rebuild_index.patch, 
> CASSANDRA-12837-2.2.9.txt
>
>
> Add multi-thread nodetool rebuild_index to improve performances.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13268) Allow to create custom secondary index on static columns

[vincent royer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

vincent royer updated CASSANDRA-13268:
--
Fix Version/s: (was: 3.0.x)
   4.x
   3.11.x

> Allow to create custom secondary index on static columns
> 
>
> Key: CASSANDRA-13268
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13268
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core, CQL
>Reporter: vincent royer
>Priority: Trivial
>  Labels: features
> Fix For: 3.11.x, 4.x
>
> Attachments: 0001-CASSANDRA-13268-custom-index-on-static-columns.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> Custom secondary index implementations (like elassandra) could gain avantage 
> to index static columns, even if not searchable with CQL. Here is a proposal 
> to allow index creation on static columns.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13231) org.apache.cassandra.db.DirectoriesTest(testStandardDirs) unit test failing

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895666#comment-15895666
 ] 

Jason Brown commented on CASSANDRA-13231:
-

I'll note that I was able to repo the error on macOS for cassandra 2.2 and up, 
but not on linux for the same branches.

> org.apache.cassandra.db.DirectoriesTest(testStandardDirs) unit test failing
> ---
>
> Key: CASSANDRA-13231
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13231
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Kjellman
>Assignee: Michael Kjellman
> Attachments: 674.diff
>
>
> The testStandardDirs(org.apache.cassandra.db.DirectoriesTest) unit test 
> always fails. This appears to be due to a commit by Yuki for CASSANDRA-10587 
> which switched the SSTable descriptor to use the canonical path.
> From one of Yuki's comments in CASSANDRA-10587:
> "I ended up fixing Descriptor object to always have canonical path as its 
> directory.
> This way we don't need to think about given directory is relative or absolute.
> In fact, right now Desctiptor (and corresponding SSTable) is not considered 
> equal between Descriptor's directory being relative and absolute. (Added 
> simple unit test to DescriptorTest)."
> The issue here is that canonical path will expand out differently than even 
> absolute path. In this case /var/folders -> /private/var/folders. The unit 
> test is looking for /var/folders/... but the Descriptor expands out to 
> /private/var/folders and the unit test fails.
> Descriptor#L88 seems to be the real root cause.
>[junit] Testcase: 
> testStandardDirs(org.apache.cassandra.db.DirectoriesTest):   FAILED
> [junit] 
> expected:
>  but 
> was:
> [junit] junit.framework.AssertionFailedError: 
> expected:
>  but 
> was:
> [junit]   at 
> org.apache.cassandra.db.DirectoriesTest.testStandardDirs(DirectoriesTest.java:159)
> [junit] 
> [junit] 
> [junit] Test org.apache.cassandra.db.DirectoriesTest FAILED
> I'm guessing given we went to canonicalPath() on purpose the "fix" here is to 
> call .getCanonicalFile() on both expected Files generated (snapshotDir and 
> backupsDir) for the junit assert.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13231) org.apache.cassandra.db.DirectoriesTest(testStandardDirs) unit test failing

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895664#comment-15895664
 ] 

Jason Brown commented on CASSANDRA-13231:
-

As this is a change only to a unit test, not running the dtests:

||2.2||3.0||3.11||trunk||
|[branch|https://github.com/jasobrown/cassandra/tree/13231-2.2]|[branch|https://github.com/jasobrown/cassandra/tree/13231-3.0]|[branch|https://github.com/jasobrown/cassandra/tree/13231-3.11]|[branch|https://github.com/jasobrown/cassandra/tree/13231-trunk]|
|[testall|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-13231-2.2-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-13231-3.0-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-13231-3.11-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-13231-trunk-testall/]|


> org.apache.cassandra.db.DirectoriesTest(testStandardDirs) unit test failing
> ---
>
> Key: CASSANDRA-13231
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13231
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Kjellman
>Assignee: Michael Kjellman
> Attachments: 674.diff
>
>
> The testStandardDirs(org.apache.cassandra.db.DirectoriesTest) unit test 
> always fails. This appears to be due to a commit by Yuki for CASSANDRA-10587 
> which switched the SSTable descriptor to use the canonical path.
> From one of Yuki's comments in CASSANDRA-10587:
> "I ended up fixing Descriptor object to always have canonical path as its 
> directory.
> This way we don't need to think about given directory is relative or absolute.
> In fact, right now Desctiptor (and corresponding SSTable) is not considered 
> equal between Descriptor's directory being relative and absolute. (Added 
> simple unit test to DescriptorTest)."
> The issue here is that canonical path will expand out differently than even 
> absolute path. In this case /var/folders -> /private/var/folders. The unit 
> test is looking for /var/folders/... but the Descriptor expands out to 
> /private/var/folders and the unit test fails.
> Descriptor#L88 seems to be the real root cause.
>[junit] Testcase: 
> testStandardDirs(org.apache.cassandra.db.DirectoriesTest):   FAILED
> [junit] 
> expected:
>  but 
> was:
> [junit] junit.framework.AssertionFailedError: 
> expected:
>  but 
> was:
> [junit]   at 
> org.apache.cassandra.db.DirectoriesTest.testStandardDirs(DirectoriesTest.java:159)
> [junit] 
> [junit] 
> [junit] Test org.apache.cassandra.db.DirectoriesTest FAILED
> I'm guessing given we went to canonicalPath() on purpose the "fix" here is to 
> call .getCanonicalFile() on both expected Files generated (snapshotDir and 
> backupsDir) for the junit assert.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13231) org.apache.cassandra.db.DirectoriesTest(testStandardDirs) unit test failing

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-13231:

Reviewer: Jason Brown

> org.apache.cassandra.db.DirectoriesTest(testStandardDirs) unit test failing
> ---
>
> Key: CASSANDRA-13231
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13231
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Kjellman
>Assignee: Michael Kjellman
> Attachments: 674.diff
>
>
> The testStandardDirs(org.apache.cassandra.db.DirectoriesTest) unit test 
> always fails. This appears to be due to a commit by Yuki for CASSANDRA-10587 
> which switched the SSTable descriptor to use the canonical path.
> From one of Yuki's comments in CASSANDRA-10587:
> "I ended up fixing Descriptor object to always have canonical path as its 
> directory.
> This way we don't need to think about given directory is relative or absolute.
> In fact, right now Desctiptor (and corresponding SSTable) is not considered 
> equal between Descriptor's directory being relative and absolute. (Added 
> simple unit test to DescriptorTest)."
> The issue here is that canonical path will expand out differently than even 
> absolute path. In this case /var/folders -> /private/var/folders. The unit 
> test is looking for /var/folders/... but the Descriptor expands out to 
> /private/var/folders and the unit test fails.
> Descriptor#L88 seems to be the real root cause.
>[junit] Testcase: 
> testStandardDirs(org.apache.cassandra.db.DirectoriesTest):   FAILED
> [junit] 
> expected:
>  but 
> was:
> [junit] junit.framework.AssertionFailedError: 
> expected:
>  but 
> was:
> [junit]   at 
> org.apache.cassandra.db.DirectoriesTest.testStandardDirs(DirectoriesTest.java:159)
> [junit] 
> [junit] 
> [junit] Test org.apache.cassandra.db.DirectoriesTest FAILED
> I'm guessing given we went to canonicalPath() on purpose the "fix" here is to 
> call .getCanonicalFile() on both expected Files generated (snapshotDir and 
> backupsDir) for the junit assert.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13042) The two cassandra nodes suddenly encounter hints each other and failed replaying.

[KimSeongJin (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895655#comment-15895655
 ] 

KimSeongJin commented on CASSANDRA-13042:
-

+1 Cassandra 2.2.8 version also shows the same issue.

> The two cassandra nodes suddenly encounter hints each other and failed 
> replaying.
> -
>
> Key: CASSANDRA-13042
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13042
> Project: Cassandra
>  Issue Type: Bug
>Reporter: YheonHo.Choi
>Priority: Critical
> Attachments: out_2.2.2.1.txt, out_2.2.2.2.txt
>
>
> Although there are no changes to cassandra, two node suddenly encounter hints 
> and failed replaying.
> Any commands like disablethrift, disablegossip can not solve the above 
> problem and the only way was restart.
> When we check the status of cluster, all nodes are looks UN but 
> describecluster show unreachable each other.
> Here's the state of the cassandra during the above problem occurred.
> IP addresses in report anonymized:
> cassandra version: 2.2.5
> node 1 = 1.1.1.1
> node 2 = 1.1.1.2
> others = x.x.x.x
> system.log
> {code}
> ## result of nodetool status on 1.1.1.1
> INFO  [HintedHandoff:1] 2016-11-24 06:15:07,969 HintedHandOffManager.java:367 
> - Started hinted handoff for host: 8caa54f3-7d67-40d6-b224-8c64a1d289be with 
> IP: /1.1.1.2
> INFO  [HintedHandoff:1] 2016-11-24 06:15:09,969 HintedHandOffManager.java:486 
> - Failed replaying hints to /1.1.1.2; aborting (0 delivered), error : 
> Operation timed out - received only 0 responses.
> INFO  [HintedHandoff:2] 2016-11-24 06:25:09,736 HintedHandOffManager.java:367 
> - Started hinted handoff for host: 8caa54f3-7d67-40d6-b224-8c64a1d289be with 
> IP: /1.1.1.2
> INFO  [HintedHandoff:2] 2016-11-24 06:25:11,738 HintedHandOffManager.java:486 
> - Failed replaying hints to /1.1.1.2; aborting (0 delivered), error : 
> Operation timed out - received only 0 responses.
> WARN  [MemtableFlushWriter:55270] 2016-11-24 06:25:12,625 
> BigTableWriter.java:184 - Writing large partition 
> system/hints:d640677d-f354-aa8c-be89-d2a1648c24b2 (109029803 bytes)
> WARN  [CompactionExecutor:37908] 2016-11-24 06:35:23,682 
> BigTableWriter.java:184 - Writing large partition 
> system/hints:8caa54f3-7d67-40d6-b224-8c64a1d289be (250651758 bytes)
> INFO  [HintedHandoff:1] 2016-11-24 06:35:23,727 HintedHandOffManager.java:367 
> - Started hinted handoff for host: 8caa54f3-7d67-40d6-b224-8c64a1d289be with 
> IP: /1.1.1.2
> INFO  [HintedHandoff:1] 2016-11-24 06:35:25,728 HintedHandOffManager.java:486 
> - Failed replaying hints to /1.1.1.2; aborting (0 delivered), error : 
> Operation timed out - received only 0 responses.
> WARN  [CompactionExecutor:37909] 2016-11-24 06:45:53,615 
> BigTableWriter.java:184 - Writing large partition 
> system/hints:8caa54f3-7d67-40d6-b224-8c64a1d289be (340801514 bytes)
> INFO  [HintedHandoff:2] 2016-11-24 06:45:53,718 HintedHandOffManager.java:367 
> - Started hinted handoff for host: 8caa54f3-7d67-40d6-b224-8c64a1d289be with 
> IP: /1.1.1.2
> INFO  [HintedHandoff:2] 2016-11-24 06:45:55,719 HintedHandOffManager.java:486 
> - Failed replaying hints to /1.1.1.2; aborting (0 delivered), error : 
> Operation timed out - received only 0 responses.
> WARN  [CompactionExecutor:37912] 2016-11-24 06:56:20,884 
> BigTableWriter.java:184 - Writing large partition 
> system/hints:8caa54f3-7d67-40d6-b224-8c64a1d289be (472465093 bytes)
> INFO  [HintedHandoff:1] 2016-11-24 06:56:20,966 HintedHandOffManager.java:367 
> - Started hinted handoff for host: 8caa54f3-7d67-40d6-b224-8c64a1d289be with 
> IP: /1.1.1.2
> INFO  [HintedHandoff:1] 2016-11-24 06:56:22,967 HintedHandOffManager.java:486 
> - Failed replaying hints to /1.1.1.2; aborting (0 delivered), error : 
> Operation timed out - received only 0 responses.
> WARN  [CompactionExecutor:37911] 2016-11-24 07:07:12,568 
> BigTableWriter.java:184 - Writing large partition 
> system/hints:8caa54f3-7d67-40d6-b224-8c64a1d289be (577392172 bytes)
> INFO  [HintedHandoff:2] 2016-11-24 07:07:12,643 HintedHandOffManager.java:367 
> - Started hinted handoff for host: 8caa54f3-7d67-40d6-b224-8c64a1d289be with 
> IP: /1.1.1.2
> INFO  [HintedHandoff:2] 2016-11-24 07:07:14,643 HintedHandOffManager.java:486 
> - Failed replaying hints to /1.1.1.2; aborting (0 delivered), error : 
> Operation timed out - received only 0 responses.
> INFO  [IndexSummaryManager:1] 2016-11-24 07:09:15,929 
> IndexSummaryRedistribution.java:74 - Redistributing index summaries
> ## result of nodetool status on 1.1.1.2
> INFO  [HintedHandoff:1] 2016-11-24 06:11:37,300 HintedHandOffManager.java:367 
> - Started hinted handoff for host: b79124e9-394c-4400-a8e7-a0c94aec6878 with 
> IP: /1.1.1.1
> INFO  [HintedHandoff:1] 2016-11-24 06:11:39,301 HintedHandOffManager.java:486 
> 

[jira] [Commented] (CASSANDRA-13294) Possible data loss on upgrade 2.1 - 3.0

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895581#comment-15895581
 ] 

Marcus Eriksson commented on CASSANDRA-13294:
-

It was not caused by the filename change, it was caused by CASSANDRA-7066 / 
CASSANDRA-10421

And it does not affect trunk since we don't support reading 2.1 files there

> Possible data loss on upgrade 2.1 - 3.0
> ---
>
> Key: CASSANDRA-13294
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13294
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Blocker
> Fix For: 3.0.x, 3.11.x
>
>
> After finishing a compaction we delete the compacted away files. This is done 
> [here|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/lifecycle/LogFile.java#L328-L337]
>  which uses 
> [this|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/lifecycle/LogRecord.java#L265-L271]
>  to get the files - we get all files starting with {{absoluteFilePath}}. 
> Absolute file path is generated 
> [here|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/sstable/Descriptor.java#L142-L153].
>  For 3.0 version files the filename looks like this: 
> {{/blabla/keyspace1/standard1-bdb031c0ff7b11e6940fdd0479dd8912/mc-1332-big}} 
> but for 2.1 version files, they look like this: 
> {{/blabla/keyspace1/standard1-bdb031c0ff7b11e6940fdd0479dd8912/keyspace1-standard1-ka-2}}.
> The problem is then that if we were to finish a compaction including the 
> legacy file, we would actually delete all legacy files having a generation 
> starting with '2'



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13294) Possible data loss on upgrade 2.1 - 3.0

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-13294:

Fix Version/s: 3.11.x

> Possible data loss on upgrade 2.1 - 3.0
> ---
>
> Key: CASSANDRA-13294
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13294
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Blocker
> Fix For: 3.0.x, 3.11.x
>
>
> After finishing a compaction we delete the compacted away files. This is done 
> [here|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/lifecycle/LogFile.java#L328-L337]
>  which uses 
> [this|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/lifecycle/LogRecord.java#L265-L271]
>  to get the files - we get all files starting with {{absoluteFilePath}}. 
> Absolute file path is generated 
> [here|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/sstable/Descriptor.java#L142-L153].
>  For 3.0 version files the filename looks like this: 
> {{/blabla/keyspace1/standard1-bdb031c0ff7b11e6940fdd0479dd8912/mc-1332-big}} 
> but for 2.1 version files, they look like this: 
> {{/blabla/keyspace1/standard1-bdb031c0ff7b11e6940fdd0479dd8912/keyspace1-standard1-ka-2}}.
> The problem is then that if we were to finish a compaction including the 
> legacy file, we would actually delete all legacy files having a generation 
> starting with '2'



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)