[jira] [Commented] (CASSANDRA-8838) Resumable bootstrap streaming
[
https://issues.apache.org/jira/browse/CASSANDRA-8838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14364788#comment-14364788
]
Sylvain Lebresne commented on CASSANDRA-8838:
-
bq. The first issue is that starting a node with {{no_wait=True}} fails to
detect the process pid and the test stops there, this applies to all four tests.
I've noticed that too and this is not specific to this issue: there is quite a
few dtests that uses this and are currently failing on trunk on my box for the
reason you mention. It appears that on current trunk, on my box, the Cassandra
process takes > 10 seconds to even get created (which is definitively something
relatively new). I haven't investigated what made that happen so it could be
nice to bisect it so we at least know, but as you said, removing the
{{no_wait}} flag at least fixes the tests.
> Resumable bootstrap streaming
> -
>
> Key: CASSANDRA-8838
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8838
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Minor
> Labels: dense-storage
> Fix For: 3.0
>
>
> This allows the bootstrapping node not to be streamed already received data.
> The bootstrapping node records received keyspace/ranges as one stream session
> completes. When some sessions with other nodes fail, bootstrapping fails
> also, though next time it re-bootstraps, already received keyspace/ranges are
> skipped to be streamed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8976) Remove BINARY from DROPPABLE_VERBS
[
https://issues.apache.org/jira/browse/CASSANDRA-8976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Brosius updated CASSANDRA-8976:
Attachment: binary.txt
> Remove BINARY from DROPPABLE_VERBS
> --
>
> Key: CASSANDRA-8976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8976
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Chris Lohfink
>Assignee: Dave Brosius
>Priority: Trivial
> Labels: lhf
> Fix For: 3.0
>
> Attachments: binary.txt
>
>
> The deprecated {{Verb.BINARY}} type has been listed in
> {{MessagingService.DROPPABLE_VERBS}} for awhile even though it has no value.
> This is pretty irrelevant but it shows up in results to {{nodetool tpstats}}
> which may seem to be related to the binary protocol.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8976) Remove BINARY from DROPPABLE_VERBS
[
https://issues.apache.org/jira/browse/CASSANDRA-8976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14364806#comment-14364806
]
Dave Brosius edited comment on CASSANDRA-8976 at 3/17/15 9:22 AM:
--
Verb.BINARY has not been used at all since 1.1. One assumes you are not allowed
to run mixed 1.1/3.0 cluster, and so it can be removed from DROPPABLE_VERBS but
it would seem the enum itself has to remain otherwise it will mess up 2.1 <->
3.0 serialization of Verbs, as serialization is done by ordinal.
so the patch removes it from DROPPABLE_VERBS.
was (Author: dbrosius):
Verb.BINARY has not been used at all since 1.1. One assumes you are not allowed
to run mixed 1.1/3.0 cluster, and so it can be removed from DROPPABLE_VERBS but
it would seem the enum itself has to remain otherwise it will mess up
serialization of 2.1 <-> 3.0 Verbs, as serialization is done by ordinal.
so the patch removes it from DROPPABLE_VERBS.
> Remove BINARY from DROPPABLE_VERBS
> --
>
> Key: CASSANDRA-8976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8976
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Chris Lohfink
>Assignee: Dave Brosius
>Priority: Trivial
> Labels: lhf
> Fix For: 3.0
>
> Attachments: binary.txt
>
>
> The deprecated {{Verb.BINARY}} type has been listed in
> {{MessagingService.DROPPABLE_VERBS}} for awhile even though it has no value.
> This is pretty irrelevant but it shows up in results to {{nodetool tpstats}}
> which may seem to be related to the binary protocol.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8976) Remove BINARY from DROPPABLE_VERBS
[
https://issues.apache.org/jira/browse/CASSANDRA-8976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14364835#comment-14364835
]
Aleksey Yeschenko commented on CASSANDRA-8976:
--
+1
> Remove BINARY from DROPPABLE_VERBS
> --
>
> Key: CASSANDRA-8976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8976
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Chris Lohfink
>Assignee: Dave Brosius
>Priority: Trivial
> Labels: lhf
> Fix For: 3.0
>
> Attachments: binary.txt
>
>
> The deprecated {{Verb.BINARY}} type has been listed in
> {{MessagingService.DROPPABLE_VERBS}} for awhile even though it has no value.
> This is pretty irrelevant but it shows up in results to {{nodetool tpstats}}
> which may seem to be related to the binary protocol.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8976) Remove BINARY from DROPPABLE_VERBS
[
https://issues.apache.org/jira/browse/CASSANDRA-8976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-8976:
-
Reviewer: Aleksey Yeschenko
> Remove BINARY from DROPPABLE_VERBS
> --
>
> Key: CASSANDRA-8976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8976
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Chris Lohfink
>Assignee: Dave Brosius
>Priority: Trivial
> Labels: lhf
> Fix For: 3.0
>
> Attachments: binary.txt
>
>
> The deprecated {{Verb.BINARY}} type has been listed in
> {{MessagingService.DROPPABLE_VERBS}} for awhile even though it has no value.
> This is pretty irrelevant but it shows up in results to {{nodetool tpstats}}
> which may seem to be related to the binary protocol.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8561) Tombstone log warning does not log partition key
[ https://issues.apache.org/jira/browse/CASSANDRA-8561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lyuben Todorov updated CASSANDRA-8561: -- Attachment: cassandra-2.1-8561.diff > Tombstone log warning does not log partition key > > > Key: CASSANDRA-8561 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8561 > Project: Cassandra > Issue Type: Improvement > Components: Core > Environment: Datastax DSE 4.5 >Reporter: Jens Rantil >Assignee: Lyuben Todorov > Labels: logging > Fix For: 2.1.4 > > Attachments: cassandra-2.1-8561.diff > > > AFAIK, the tombstone warning in system.log does not contain the primary key. > See: https://gist.github.com/JensRantil/44204676f4dbea79ea3a > Including it would help a lot in diagnosing why the (CQL) row has so many > tombstones. > Let me know if I have misunderstood something. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[2/2] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt src/java/org/apache/cassandra/db/compaction/LeveledManifest.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c0f159b8 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c0f159b8 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c0f159b8 Branch: refs/heads/cassandra-2.1 Commit: c0f159b82a9d805990818f60735623e2a7d8c517 Parents: 89f2c1d 0814737 Author: Marcus Eriksson Authored: Tue Mar 17 11:13:40 2015 +0100 Committer: Marcus Eriksson Committed: Tue Mar 17 11:13:40 2015 +0100 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/db/compaction/LeveledManifest.java | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0f159b8/CHANGES.txt -- diff --cc CHANGES.txt index a48787f,65f2da4..7b8e0ad --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,49 -1,5 +1,50 @@@ -2.0.14: +2.1.4 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841) + * Don't set clientMode in SSTableLoader (CASSANDRA-8238) + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535) + * Allow invalidating permissions and cache time (CASSANDRA-8722) + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0 + are executed (CASSANDRA-8418) + * Fix cassandra-stress so it respects the CL passed in user mode (CASSANDRA-8948) + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786) + * cassandra-stress reports per-operation statistics, plus misc (CASSANDRA-8769) + * Add SimpleDate (cql date) and Time (cql time) types (CASSANDRA-7523) + * Use long for key count in cfstats (CASSANDRA-8913) + * Make SSTableRewriter.abort() more robust to failure (CASSANDRA-8832) + * Remove cold_reads_to_omit from STCS (CASSANDRA-8860) + * Make EstimatedHistogram#percentile() use ceil instead of floor (CASSANDRA-8883) + * Fix top partitions reporting wrong cardinality (CASSANDRA-8834) + * Fix rare NPE in KeyCacheSerializer (CASSANDRA-8067) + * Pick sstables for validation as late as possible inc repairs (CASSANDRA-8366) + * Fix commitlog getPendingTasks to not increment (CASSANDRA-8856) + * Fix parallelism adjustment in range and secondary index queries + when the first fetch does not satisfy the limit (CASSANDRA-8856) + * Check if the filtered sstables is non-empty in STCS (CASSANDRA-8843) + * Upgrade java-driver used for cassandra-stress (CASSANDRA-8842) + * Fix CommitLog.forceRecycleAllSegments() memory access error (CASSANDRA-8812) + * Improve assertions in Memory (CASSANDRA-8792) + * Fix SSTableRewriter cleanup (CASSANDRA-8802) + * Introduce SafeMemory for CompressionMetadata.Writer (CASSANDRA-8758) + * 'nodetool info' prints exception against older node (CASSANDRA-8796) + * Ensure SSTableReader.last corresponds exactly with the file end (CASSANDRA-8750) + * Make SSTableWriter.openEarly more robust and obvious (CASSANDRA-8747) + * Enforce SSTableReader.first/last (CASSANDRA-8744) + * Cleanup SegmentedFile API (CASSANDRA-8749) + * Avoid overlap with early compaction replacement (CASSANDRA-8683) + * Safer Resource Management++ (CASSANDRA-8707) + * Write partition size estimates into a system table (CASSANDRA-7688) + * cqlsh: Fix keys() and full() collection indexes in DESCRIBE output + (CASSANDRA-8154) + * Show progress of streaming in nodetool netstats (CASSANDRA-8886) + * IndexSummaryBuilder utilises offheap memory, and shares data between + each IndexSummary opened from it (CASSANDRA-8757) + * markCompacting only succeeds if the exact SSTableReader instances being + marked are in the live set (CASSANDRA-8689) + * cassandra-stress support for varint (CASSANDRA-8882) + * Fix Adler32 digest for compressed sstables (CASSANDRA-8778) + * Add nodetool statushandoff/statusbackup (CASSANDRA-8912) +Merged from 2.0: + * Improve compaction estimated tasks estimation (CASSANDRA-8904) * Fix duplicate up/down messages sent to native clients (CASSANDRA-7816) * Expose commit log archive status via JMX (CASSANDRA-8734) * Provide better exceptions for invalid replication strategy parameters http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0f159b8/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java -- diff --cc src/java/org/apache/cassandra/db/compaction/LeveledManifest.java index f2af848,6554eb2..c076a64 --- a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java +++ b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java @@@ -672,8 -622,9 +672,
cassandra git commit: Match estimated tasks arithmetic to score in LCS
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 31a182876 -> 08147376a
Match estimated tasks arithmetic to score in LCS
Patch by carlyeks; reviewed by marcuse for CASSANDRA-8904
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/08147376
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/08147376
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/08147376
Branch: refs/heads/cassandra-2.0
Commit: 08147376aa16570c07a4522931cc8fa330519d3e
Parents: 31a1828
Author: Carl Yeksigian
Authored: Mon Mar 16 14:54:08 2015 +
Committer: Marcus Eriksson
Committed: Tue Mar 17 11:10:14 2015 +0100
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/db/compaction/LeveledManifest.java | 3 ++-
2 files changed, 3 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/08147376/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8843908..65f2da4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.14:
+ * Improve compaction estimated tasks estimation (CASSANDRA-8904)
* Fix duplicate up/down messages sent to native clients (CASSANDRA-7816)
* Expose commit log archive status via JMX (CASSANDRA-8734)
* Provide better exceptions for invalid replication strategy parameters
http://git-wip-us.apache.org/repos/asf/cassandra/blob/08147376/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
index aefd573..6554eb2 100644
--- a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
+++ b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
@@ -623,7 +623,8 @@ public class LeveledManifest
for (int i = generations.length - 1; i >= 0; i--)
{
List sstables = generations[i];
-estimated[i] = Math.max(0L, SSTableReader.getTotalBytes(sstables)
- maxBytesForLevel(i)) / maxSSTableSizeInBytes;
+ // If there is 1 byte over TBL - (MBL * 1.001), there is still a
task left, so we need to round up.
+ estimated[i] = (long)Math.ceil((double)Math.max(0L,
SSTableReader.getTotalBytes(sstables) - (long)(maxBytesForLevel(i) * 1.001)) /
(double)maxSSTableSizeInBytes);
tasks += estimated[i];
}
[1/2] cassandra git commit: Match estimated tasks arithmetic to score in LCS
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 89f2c1de6 -> c0f159b82
Match estimated tasks arithmetic to score in LCS
Patch by carlyeks; reviewed by marcuse for CASSANDRA-8904
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/08147376
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/08147376
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/08147376
Branch: refs/heads/cassandra-2.1
Commit: 08147376aa16570c07a4522931cc8fa330519d3e
Parents: 31a1828
Author: Carl Yeksigian
Authored: Mon Mar 16 14:54:08 2015 +
Committer: Marcus Eriksson
Committed: Tue Mar 17 11:10:14 2015 +0100
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/db/compaction/LeveledManifest.java | 3 ++-
2 files changed, 3 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/08147376/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8843908..65f2da4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.14:
+ * Improve compaction estimated tasks estimation (CASSANDRA-8904)
* Fix duplicate up/down messages sent to native clients (CASSANDRA-7816)
* Expose commit log archive status via JMX (CASSANDRA-8734)
* Provide better exceptions for invalid replication strategy parameters
http://git-wip-us.apache.org/repos/asf/cassandra/blob/08147376/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
index aefd573..6554eb2 100644
--- a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
+++ b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
@@ -623,7 +623,8 @@ public class LeveledManifest
for (int i = generations.length - 1; i >= 0; i--)
{
List sstables = generations[i];
-estimated[i] = Math.max(0L, SSTableReader.getTotalBytes(sstables)
- maxBytesForLevel(i)) / maxSSTableSizeInBytes;
+ // If there is 1 byte over TBL - (MBL * 1.001), there is still a
task left, so we need to round up.
+ estimated[i] = (long)Math.ceil((double)Math.max(0L,
SSTableReader.getTotalBytes(sstables) - (long)(maxBytesForLevel(i) * 1.001)) /
(double)maxSSTableSizeInBytes);
tasks += estimated[i];
}
[5/5] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0b4cce86 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0b4cce86 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0b4cce86 Branch: refs/heads/trunk Commit: 0b4cce86a222c4e9d9a0b32567ab16da519d87d3 Parents: d182fa1 c0f159b Author: Marcus Eriksson Authored: Tue Mar 17 11:14:53 2015 +0100 Committer: Marcus Eriksson Committed: Tue Mar 17 11:14:53 2015 +0100 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/db/compaction/LeveledManifest.java | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/0b4cce86/CHANGES.txt -- diff --cc CHANGES.txt index ef2a7c7,7b8e0ad..be57f1b --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -113,7 -44,7 +113,8 @@@ * Fix Adler32 digest for compressed sstables (CASSANDRA-8778) * Add nodetool statushandoff/statusbackup (CASSANDRA-8912) Merged from 2.0: +2.0.14: + * Improve compaction estimated tasks estimation (CASSANDRA-8904) * Fix duplicate up/down messages sent to native clients (CASSANDRA-7816) * Expose commit log archive status via JMX (CASSANDRA-8734) * Provide better exceptions for invalid replication strategy parameters http://git-wip-us.apache.org/repos/asf/cassandra/blob/0b4cce86/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java --
[1/5] cassandra git commit: bump versions
Repository: cassandra Updated Branches: refs/heads/trunk d182fa1b4 -> 0b4cce86a bump versions Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/31a18287 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/31a18287 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/31a18287 Branch: refs/heads/trunk Commit: 31a182876eb5ddf7eef2e40fbbe232c236413671 Parents: 2199a87 Author: T Jake Luciani Authored: Thu Mar 12 21:01:11 2015 -0400 Committer: T Jake Luciani Committed: Mon Mar 16 11:48:11 2015 -0400 -- build.xml| 2 +- debian/changelog | 6 ++ 2 files changed, 7 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/31a18287/build.xml -- diff --git a/build.xml b/build.xml index 78c6db3..bdbd10a 100644 --- a/build.xml +++ b/build.xml @@ -25,7 +25,7 @@ - + http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/> http://git-wip-us.apache.org/repos/asf/cassandra/blob/31a18287/debian/changelog -- diff --git a/debian/changelog b/debian/changelog index 53fa20f..2c80800 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +cassandra (2.0.13) unstable; urgency=medium + + * New release + + -- Jake Luciani Thu, 12 Mar 2015 21:00:06 -0400 + cassandra (2.0.12) unstable; urgency=medium * New release
[3/5] cassandra git commit: Match estimated tasks arithmetic to score in LCS
Match estimated tasks arithmetic to score in LCS
Patch by carlyeks; reviewed by marcuse for CASSANDRA-8904
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/08147376
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/08147376
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/08147376
Branch: refs/heads/trunk
Commit: 08147376aa16570c07a4522931cc8fa330519d3e
Parents: 31a1828
Author: Carl Yeksigian
Authored: Mon Mar 16 14:54:08 2015 +
Committer: Marcus Eriksson
Committed: Tue Mar 17 11:10:14 2015 +0100
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/db/compaction/LeveledManifest.java | 3 ++-
2 files changed, 3 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/08147376/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8843908..65f2da4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.14:
+ * Improve compaction estimated tasks estimation (CASSANDRA-8904)
* Fix duplicate up/down messages sent to native clients (CASSANDRA-7816)
* Expose commit log archive status via JMX (CASSANDRA-8734)
* Provide better exceptions for invalid replication strategy parameters
http://git-wip-us.apache.org/repos/asf/cassandra/blob/08147376/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
index aefd573..6554eb2 100644
--- a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
+++ b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
@@ -623,7 +623,8 @@ public class LeveledManifest
for (int i = generations.length - 1; i >= 0; i--)
{
List sstables = generations[i];
-estimated[i] = Math.max(0L, SSTableReader.getTotalBytes(sstables)
- maxBytesForLevel(i)) / maxSSTableSizeInBytes;
+ // If there is 1 byte over TBL - (MBL * 1.001), there is still a
task left, so we need to round up.
+ estimated[i] = (long)Math.ceil((double)Math.max(0L,
SSTableReader.getTotalBytes(sstables) - (long)(maxBytesForLevel(i) * 1.001)) /
(double)maxSSTableSizeInBytes);
tasks += estimated[i];
}
[2/5] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/89f2c1de Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/89f2c1de Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/89f2c1de Branch: refs/heads/trunk Commit: 89f2c1de63a75871367aee0f22985c8d2c4f9930 Parents: 1a77a69 31a1828 Author: T Jake Luciani Authored: Mon Mar 16 11:49:39 2015 -0400 Committer: T Jake Luciani Committed: Mon Mar 16 11:49:39 2015 -0400 -- --
[4/5] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt src/java/org/apache/cassandra/db/compaction/LeveledManifest.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c0f159b8 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c0f159b8 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c0f159b8 Branch: refs/heads/trunk Commit: c0f159b82a9d805990818f60735623e2a7d8c517 Parents: 89f2c1d 0814737 Author: Marcus Eriksson Authored: Tue Mar 17 11:13:40 2015 +0100 Committer: Marcus Eriksson Committed: Tue Mar 17 11:13:40 2015 +0100 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/db/compaction/LeveledManifest.java | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0f159b8/CHANGES.txt -- diff --cc CHANGES.txt index a48787f,65f2da4..7b8e0ad --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,49 -1,5 +1,50 @@@ -2.0.14: +2.1.4 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841) + * Don't set clientMode in SSTableLoader (CASSANDRA-8238) + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535) + * Allow invalidating permissions and cache time (CASSANDRA-8722) + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0 + are executed (CASSANDRA-8418) + * Fix cassandra-stress so it respects the CL passed in user mode (CASSANDRA-8948) + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786) + * cassandra-stress reports per-operation statistics, plus misc (CASSANDRA-8769) + * Add SimpleDate (cql date) and Time (cql time) types (CASSANDRA-7523) + * Use long for key count in cfstats (CASSANDRA-8913) + * Make SSTableRewriter.abort() more robust to failure (CASSANDRA-8832) + * Remove cold_reads_to_omit from STCS (CASSANDRA-8860) + * Make EstimatedHistogram#percentile() use ceil instead of floor (CASSANDRA-8883) + * Fix top partitions reporting wrong cardinality (CASSANDRA-8834) + * Fix rare NPE in KeyCacheSerializer (CASSANDRA-8067) + * Pick sstables for validation as late as possible inc repairs (CASSANDRA-8366) + * Fix commitlog getPendingTasks to not increment (CASSANDRA-8856) + * Fix parallelism adjustment in range and secondary index queries + when the first fetch does not satisfy the limit (CASSANDRA-8856) + * Check if the filtered sstables is non-empty in STCS (CASSANDRA-8843) + * Upgrade java-driver used for cassandra-stress (CASSANDRA-8842) + * Fix CommitLog.forceRecycleAllSegments() memory access error (CASSANDRA-8812) + * Improve assertions in Memory (CASSANDRA-8792) + * Fix SSTableRewriter cleanup (CASSANDRA-8802) + * Introduce SafeMemory for CompressionMetadata.Writer (CASSANDRA-8758) + * 'nodetool info' prints exception against older node (CASSANDRA-8796) + * Ensure SSTableReader.last corresponds exactly with the file end (CASSANDRA-8750) + * Make SSTableWriter.openEarly more robust and obvious (CASSANDRA-8747) + * Enforce SSTableReader.first/last (CASSANDRA-8744) + * Cleanup SegmentedFile API (CASSANDRA-8749) + * Avoid overlap with early compaction replacement (CASSANDRA-8683) + * Safer Resource Management++ (CASSANDRA-8707) + * Write partition size estimates into a system table (CASSANDRA-7688) + * cqlsh: Fix keys() and full() collection indexes in DESCRIBE output + (CASSANDRA-8154) + * Show progress of streaming in nodetool netstats (CASSANDRA-8886) + * IndexSummaryBuilder utilises offheap memory, and shares data between + each IndexSummary opened from it (CASSANDRA-8757) + * markCompacting only succeeds if the exact SSTableReader instances being + marked are in the live set (CASSANDRA-8689) + * cassandra-stress support for varint (CASSANDRA-8882) + * Fix Adler32 digest for compressed sstables (CASSANDRA-8778) + * Add nodetool statushandoff/statusbackup (CASSANDRA-8912) +Merged from 2.0: + * Improve compaction estimated tasks estimation (CASSANDRA-8904) * Fix duplicate up/down messages sent to native clients (CASSANDRA-7816) * Expose commit log archive status via JMX (CASSANDRA-8734) * Provide better exceptions for invalid replication strategy parameters http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0f159b8/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java -- diff --cc src/java/org/apache/cassandra/db/compaction/LeveledManifest.java index f2af848,6554eb2..c076a64 --- a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java +++ b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java @@@ -672,8 -622,9 +672,9 @@@ pu
[jira] [Commented] (CASSANDRA-5791) A nodetool command to validate all sstables in a node
[ https://issues.apache.org/jira/browse/CASSANDRA-5791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14364963#comment-14364963 ] Branimir Lambov commented on CASSANDRA-5791: +1 [~benedict], I think the patch is ready. Could you commit it? > A nodetool command to validate all sstables in a node > - > > Key: CASSANDRA-5791 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5791 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Jeff Jirsa >Priority: Minor > Attachments: cassandra-5791-patch-3.diff, cassandra-5791.patch-2 > > > CUrrently there is no nodetool command to validate all sstables on disk. The > only way to do this is to run a repair and see if it succeeds. But we cannot > repair the system keyspace. > Also we can run upgrade sstables but that re writes all the sstables. > This command should check the hash of all sstables and return whether all > data is readable all not. This should NOT care about consistency. > The compressed sstables do not have hash so not sure how it will work there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8966) SequentialWriter should be Ref counted
[ https://issues.apache.org/jira/browse/CASSANDRA-8966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365069#comment-14365069 ] Benedict commented on CASSANDRA-8966: - Just an FYI that I'm looking at this now also, and plan to commit a patch shortly including it, along with some final tidying up of the contracts in SSTableRewriter, SSTableWriter et al, to improve safety under failure > SequentialWriter should be Ref counted > -- > > Key: CASSANDRA-8966 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8966 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict >Priority: Minor > Fix For: 3.0 > > > A LHF to introduce some more resource safety -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8838) Resumable bootstrap streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-8838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365081#comment-14365081 ] Stefania commented on CASSANDRA-8838: - On my box it takes just under 4 seconds with both cassandra-2.1 and trunk. With cassandra-2.0 however, it takes just over 2 seconds. I'll try to spend more time investigating this. > Resumable bootstrap streaming > - > > Key: CASSANDRA-8838 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8838 > Project: Cassandra > Issue Type: Sub-task >Reporter: Yuki Morishita >Assignee: Yuki Morishita >Priority: Minor > Labels: dense-storage > Fix For: 3.0 > > > This allows the bootstrapping node not to be streamed already received data. > The bootstrapping node records received keyspace/ranges as one stream session > completes. When some sessions with other nodes fail, bootstrapping fails > also, though next time it re-bootstraps, already received keyspace/ranges are > skipped to be streamed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: remove BINARY from DROPPABLE_VERBS patch by dbrosius reviewed by ayeschenko for CASSANDRA-8976
Repository: cassandra Updated Branches: refs/heads/trunk 0b4cce86a -> c9a6d7ce0 remove BINARY from DROPPABLE_VERBS patch by dbrosius reviewed by ayeschenko for CASSANDRA-8976 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c9a6d7ce Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c9a6d7ce Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c9a6d7ce Branch: refs/heads/trunk Commit: c9a6d7ce05ff75412d5f40c740b33b33a7e813a0 Parents: 0b4cce8 Author: Dave Brosius Authored: Tue Mar 17 09:39:33 2015 -0400 Committer: Dave Brosius Committed: Tue Mar 17 09:40:54 2015 -0400 -- src/java/org/apache/cassandra/net/MessagingService.java | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c9a6d7ce/src/java/org/apache/cassandra/net/MessagingService.java -- diff --git a/src/java/org/apache/cassandra/net/MessagingService.java b/src/java/org/apache/cassandra/net/MessagingService.java index a5cbfa7..1f952a3 100644 --- a/src/java/org/apache/cassandra/net/MessagingService.java +++ b/src/java/org/apache/cassandra/net/MessagingService.java @@ -289,8 +289,7 @@ public final class MessagingService implements MessagingServiceMBean * all correspond to client requests or something triggered by them; we don't want to * drop internal messages like bootstrap or repair notifications. */ -public static final EnumSet DROPPABLE_VERBS = EnumSet.of(Verb.BINARY, - Verb._TRACE, +public static final EnumSet DROPPABLE_VERBS = EnumSet.of(Verb._TRACE, Verb.MUTATION, Verb.COUNTER_MUTATION, Verb.READ_REPAIR,
[jira] [Created] (CASSANDRA-8978) CQLSSTableWriter causes ArrayIndexOutOfBoundsException
Thomas Borg Salling created CASSANDRA-8978:
--
Summary: CQLSSTableWriter causes ArrayIndexOutOfBoundsException
Key: CASSANDRA-8978
URL: https://issues.apache.org/jira/browse/CASSANDRA-8978
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: 3.8.0-42-generic #62~precise1-Ubuntu SMP Wed Jun 4
22:04:18 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
java version "1.8.0_20"
Java(TM) SE Runtime Environment (build 1.8.0_20-b26)
Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode)
Reporter: Thomas Borg Salling
On long-running jobs with CQLSSTableWriter preparing sstables for later bulk
load via sstableloader, occassionally get this sporadic error.
I can run the exact same job again - and it will succeed or fail with the same
error at another location in the input stream. The error is appears to occur
"randomly" - with the same input it may occur never, early or late in the run
with no apparent logic or system.
I use five instances of CQLSSTableWriter in the application (to write
redundantly to five different tables). But these instances do not exist at the
same time; and thus never used concurrently.
{code}
09:26:33.582 [main] INFO d.dma.ais.store.FileSSTableConverter - Finished
processing directory, 369582175 packets was converted from /nas1/
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at dk.dma.commons.app.CliCommandList$1.execute(CliCommandList.java:50)
at dk.dma.commons.app.CliCommandList.invoke(CliCommandList.java:80)
at dk.dma.ais.store.Main.main(Main.java:34)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 297868
at
org.apache.cassandra.db.ArrayBackedSortedColumns.append(ArrayBackedSortedColumns.java:196)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.appendOrReconcile(ArrayBackedSortedColumns.java:191)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:176)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.maybeSortCells(ArrayBackedSortedColumns.java:125)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.access$1100(ArrayBackedSortedColumns.java:44)
at
org.apache.cassandra.db.ArrayBackedSortedColumns$CellCollection.iterator(ArrayBackedSortedColumns.java:622)
at org.apache.cassandra.db.ColumnFamily.iterator(ColumnFamily.java:476)
at
org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:129)
at
org.apache.cassandra.io.sstable.SSTableWriter.rawAppend(SSTableWriter.java:233)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:218)
at
org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter$DiskWriter.run(SSTableSimpleUnsortedWriter.java:215){code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8978) CQLSSTableWriter causes ArrayIndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/CASSANDRA-8978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Borg Salling updated CASSANDRA-8978:
---
Description:
On long-running jobs with CQLSSTableWriter preparing sstables for later bulk
load via sstableloader, occassionally I get the sporadic error shown below.
I can run the exact same job again - and it will succeed or fail with the same
error at another location in the input stream. The error is appears to occur
"randomly" - with the same input it may occur never, early or late in the run
with no apparent logic or system.
I use five instances of CQLSSTableWriter in the application (to write
redundantly to five different tables). But these instances do not exist at the
same time; and thus never used concurrently.
{code}
09:26:33.582 [main] INFO d.dma.ais.store.FileSSTableConverter - Finished
processing directory, 369582175 packets was converted from /nas1/
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at dk.dma.commons.app.CliCommandList$1.execute(CliCommandList.java:50)
at dk.dma.commons.app.CliCommandList.invoke(CliCommandList.java:80)
at dk.dma.ais.store.Main.main(Main.java:34)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 297868
at
org.apache.cassandra.db.ArrayBackedSortedColumns.append(ArrayBackedSortedColumns.java:196)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.appendOrReconcile(ArrayBackedSortedColumns.java:191)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:176)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.maybeSortCells(ArrayBackedSortedColumns.java:125)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.access$1100(ArrayBackedSortedColumns.java:44)
at
org.apache.cassandra.db.ArrayBackedSortedColumns$CellCollection.iterator(ArrayBackedSortedColumns.java:622)
at org.apache.cassandra.db.ColumnFamily.iterator(ColumnFamily.java:476)
at
org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:129)
at
org.apache.cassandra.io.sstable.SSTableWriter.rawAppend(SSTableWriter.java:233)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:218)
at
org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter$DiskWriter.run(SSTableSimpleUnsortedWriter.java:215){code}
was:
On long-running jobs with CQLSSTableWriter preparing sstables for later bulk
load via sstableloader, occassionally get this sporadic error.
I can run the exact same job again - and it will succeed or fail with the same
error at another location in the input stream. The error is appears to occur
"randomly" - with the same input it may occur never, early or late in the run
with no apparent logic or system.
I use five instances of CQLSSTableWriter in the application (to write
redundantly to five different tables). But these instances do not exist at the
same time; and thus never used concurrently.
{code}
09:26:33.582 [main] INFO d.dma.ais.store.FileSSTableConverter - Finished
processing directory, 369582175 packets was converted from /nas1/
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at dk.dma.commons.app.CliCommandList$1.execute(CliCommandList.java:50)
at dk.dma.commons.app.CliCommandList.invoke(CliCommandList.java:80)
at dk.dma.ais.store.Main.main(Main.java:34)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 297868
at
org.apache.cassandra.db.ArrayBackedSortedColumns.append(ArrayBackedSortedColumns.java:196)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.appendOrReconcile(ArrayBackedSortedColumns.java:191)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:176)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.maybeSortCells(ArrayBackedSortedColumns.java:125)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.access$1100(ArrayBackedSortedColumns.java:44)
at
org.apache.cassandra.db.ArrayBackedSortedColumns$CellCollection.iterator(ArrayBackedSortedColumns.java:622)
at org.apache.cassandra.db.ColumnFamily.iterator(ColumnFamily.java:476)
at
org.apache.cassa
[jira] [Updated] (CASSANDRA-8978) CQLSSTableWriter causes ArrayIndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/CASSANDRA-8978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Borg Salling updated CASSANDRA-8978:
---
Description:
On long-running jobs with CQLSSTableWriter preparing sstables for later bulk
load via sstableloader, occassionally I get the sporadic error shown below.
I can run the exact same job again - and it will succeed or fail with the same
error at another location in the input stream. The error is appears to occur
"randomly" - with the same input it may occur never, early or late in the run
with no apparent logic or system.
I use five instances of CQLSSTableWriter in the application (to write
redundantly to five different tables). But these instances do not exist at the
same time; and thus never used concurrently.
{code}
09:26:33.582 [main] INFO d.dma.ais.store.FileSSTableConverter - Finished
processing directory, 369582175 packets was converted from /nas1/
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at dk.dma.commons.app.CliCommandList$1.execute(CliCommandList.java:50)
at dk.dma.commons.app.CliCommandList.invoke(CliCommandList.java:80)
at dk.dma.ais.store.Main.main(Main.java:34)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 297868
at
org.apache.cassandra.db.ArrayBackedSortedColumns.append(ArrayBackedSortedColumns.java:196)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.appendOrReconcile(ArrayBackedSortedColumns.java:191)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:176)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.maybeSortCells(ArrayBackedSortedColumns.java:125)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.access$1100(ArrayBackedSortedColumns.java:44)
at
org.apache.cassandra.db.ArrayBackedSortedColumns$CellCollection.iterator(ArrayBackedSortedColumns.java:622)
at org.apache.cassandra.db.ColumnFamily.iterator(ColumnFamily.java:476)
at
org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:129)
at
org.apache.cassandra.io.sstable.SSTableWriter.rawAppend(SSTableWriter.java:233)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:218)
at
org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter$DiskWriter.run(SSTableSimpleUnsortedWriter.java:215){code}
So far I overcome this problem by simply retrying with another run of the
application in attempt to generate the sstables. But this is a rather time
consuming and shaky approach - and I feel a bit uneasy relying on the produced
sstables, though their contents appear to by correct with I sample them with
cqlsh 'select' after load into Cassandra.
was:
On long-running jobs with CQLSSTableWriter preparing sstables for later bulk
load via sstableloader, occassionally I get the sporadic error shown below.
I can run the exact same job again - and it will succeed or fail with the same
error at another location in the input stream. The error is appears to occur
"randomly" - with the same input it may occur never, early or late in the run
with no apparent logic or system.
I use five instances of CQLSSTableWriter in the application (to write
redundantly to five different tables). But these instances do not exist at the
same time; and thus never used concurrently.
{code}
09:26:33.582 [main] INFO d.dma.ais.store.FileSSTableConverter - Finished
processing directory, 369582175 packets was converted from /nas1/
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at dk.dma.commons.app.CliCommandList$1.execute(CliCommandList.java:50)
at dk.dma.commons.app.CliCommandList.invoke(CliCommandList.java:80)
at dk.dma.ais.store.Main.main(Main.java:34)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 297868
at
org.apache.cassandra.db.ArrayBackedSortedColumns.append(ArrayBackedSortedColumns.java:196)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.appendOrReconcile(ArrayBackedSortedColumns.java:191)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:176)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.maybeSortCells(Array
[jira] [Updated] (CASSANDRA-8978) CQLSSTableWriter causes ArrayIndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/CASSANDRA-8978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Borg Salling updated CASSANDRA-8978:
---
Description:
On long-running jobs with CQLSSTableWriter preparing sstables for later bulk
load via sstableloader, occassionally I get the sporadic error shown below.
I can run the exact same job again - and it will succeed or fail with the same
error at another location in the input stream. The error is appears to occur
"randomly" - with the same input it may occur never, early or late in the run
with no apparent logic or system.
I use five instances of CQLSSTableWriter in the application (to write
redundantly to five different tables). But these instances do not exist at the
same time; and thus never used concurrently.
{code}
09:26:33.582 [main] INFO d.dma.ais.store.FileSSTableConverter - Finished
processing directory, 369582175 packets was converted from /nas1/
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at dk.dma.commons.app.CliCommandList$1.execute(CliCommandList.java:50)
at dk.dma.commons.app.CliCommandList.invoke(CliCommandList.java:80)
at dk.dma.ais.store.Main.main(Main.java:34)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 297868
at
org.apache.cassandra.db.ArrayBackedSortedColumns.append(ArrayBackedSortedColumns.java:196)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.appendOrReconcile(ArrayBackedSortedColumns.java:191)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:176)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.maybeSortCells(ArrayBackedSortedColumns.java:125)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.access$1100(ArrayBackedSortedColumns.java:44)
at
org.apache.cassandra.db.ArrayBackedSortedColumns$CellCollection.iterator(ArrayBackedSortedColumns.java:622)
at org.apache.cassandra.db.ColumnFamily.iterator(ColumnFamily.java:476)
at
org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:129)
at
org.apache.cassandra.io.sstable.SSTableWriter.rawAppend(SSTableWriter.java:233)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:218)
at
org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter$DiskWriter.run(SSTableSimpleUnsortedWriter.java:215){code}
So far I overcome this problem by simply retrying with another run of the
application in attempt to generate the sstables. But this is a rather time
consuming and shaky approach - and I feel a bit uneasy relying on the produced
sstables, though their contents appear to be correct when I sample them with
cqlsh 'select' after load into Cassandra.
was:
On long-running jobs with CQLSSTableWriter preparing sstables for later bulk
load via sstableloader, occassionally I get the sporadic error shown below.
I can run the exact same job again - and it will succeed or fail with the same
error at another location in the input stream. The error is appears to occur
"randomly" - with the same input it may occur never, early or late in the run
with no apparent logic or system.
I use five instances of CQLSSTableWriter in the application (to write
redundantly to five different tables). But these instances do not exist at the
same time; and thus never used concurrently.
{code}
09:26:33.582 [main] INFO d.dma.ais.store.FileSSTableConverter - Finished
processing directory, 369582175 packets was converted from /nas1/
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at dk.dma.commons.app.CliCommandList$1.execute(CliCommandList.java:50)
at dk.dma.commons.app.CliCommandList.invoke(CliCommandList.java:80)
at dk.dma.ais.store.Main.main(Main.java:34)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 297868
at
org.apache.cassandra.db.ArrayBackedSortedColumns.append(ArrayBackedSortedColumns.java:196)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.appendOrReconcile(ArrayBackedSortedColumns.java:191)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:176)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.maybeSortCells(Array
[jira] [Commented] (CASSANDRA-8974) Need to update to latest dependencies
[ https://issues.apache.org/jira/browse/CASSANDRA-8974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365178#comment-14365178 ] Joe Fasano commented on CASSANDRA-8974: --- Beyond general dependency updating, here is the feedback I am getting For Jackon - There is no formal CVS yet, but this version of the Jackson does have a vulnerability. See http://markmail.org/message/7t76h5svb6snsqck?q=+list:org%2Ecodehaus%2Ejackson%2Eannounce For Jodatime - joda-time 1.6 is considered to be EOL under company policy. Anything that was released over 6 years ago is likely to be considered.It's generally against company policy to use EOL third party software. > Need to update to latest dependencies > - > > Key: CASSANDRA-8974 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8974 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Reporter: Joe Fasano > Fix For: 3.0 > > > Open C* 3.0 to deal with upgrading all the dependencies. > This is a general issue to update all dependencies. > Specifically for example, I have been told by my team that some of the > cassandra dependencies have some security vulnerabilities and should be > upgraded. > > Joda Time 1.6 should be upgraded to 2.7 > > Jackson 1.9.2 should be upgraded to 1.9.13 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7409) Allow multiple overlapping sstables in L1
[ https://issues.apache.org/jira/browse/CASSANDRA-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365250#comment-14365250 ] Alan Boudreault commented on CASSANDRA-7409: Devs, I've written a first version of the test plan for the compaction strategies. See: https://github.com/riptano/cassandra-test-plans/wiki/Compaction-performance-test-plan All comments/additions are welcome. > Allow multiple overlapping sstables in L1 > - > > Key: CASSANDRA-7409 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7409 > Project: Cassandra > Issue Type: Improvement >Reporter: Carl Yeksigian >Assignee: Carl Yeksigian > Labels: compaction > Fix For: 3.0 > > > Currently, when a normal L0 compaction takes place (not STCS), we take up to > MAX_COMPACTING_L0 L0 sstables and all of the overlapping L1 sstables and > compact them together. If we didn't have to deal with the overlapping L1 > tables, we could compact a higher number of L0 sstables together into a set > of non-overlapping L1 sstables. > This could be done by delaying the invariant that L1 has no overlapping > sstables. Going from L1 to L2, we would be compacting fewer sstables together > which overlap. > When reading, we will not have the same one sstable per level (except L0) > guarantee, but this can be bounded (once we have too many sets of sstables, > either compact them back into the same level, or compact them up to the next > level). > This could be generalized to allow any level to be the maximum for this > overlapping strategy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8977) Potential DOS Exposure from Netty
[ https://issues.apache.org/jira/browse/CASSANDRA-8977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-8977: -- Assignee: Brandon Williams > Potential DOS Exposure from Netty > - > > Key: CASSANDRA-8977 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8977 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Benjamin Coverston >Assignee: Brandon Williams > > CVE-2014-3488 : The SslHandler in Netty before 3.9.2 allows remote attackers > to cause a denial of service (infinite loop and CPU consumption) via a > crafted SSLv2Hello message. > 2.0 currently uses Netty 3.6.6 > Upgrading to 3.9.2+ will fix this issue, and patch a few other known > vulnerabilities that we do not currently expose. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8568) Impose new API on data tracker modifications that makes correct usage obvious and imposes safety
[ https://issues.apache.org/jira/browse/CASSANDRA-8568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365266#comment-14365266 ] Jonathan Ellis commented on CASSANDRA-8568: --- SGTM. > Impose new API on data tracker modifications that makes correct usage obvious > and imposes safety > > > Key: CASSANDRA-8568 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8568 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Assignee: Benedict > > DataTracker has become a bit of a quagmire, and not at all obvious to > interface with, with many subtly different modifiers. I suspect it is still > subtly broken, especially around error recovery. > I propose piggy-backing on CASSANDRA-7705 to offer RAII (and GC-enforced, for > those situations where a try/finally block isn't possible) objects that have > transactional behaviour, and with few simple declarative methods that can be > composed simply to provide all of the functionality we currently need. > See CASSANDRA-8399 for context -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8977) Potential DOS Exposure from Netty
[ https://issues.apache.org/jira/browse/CASSANDRA-8977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365311#comment-14365311 ] Brandon Williams commented on CASSANDRA-8977: - I really don't like changing deps in a minor, but I guess for a vulnerability we should consider it (but really if you expose the ports to an attacker, you're an idiot.) [~enigmacurry] can your team upgrade and test please? > Potential DOS Exposure from Netty > - > > Key: CASSANDRA-8977 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8977 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Benjamin Coverston >Assignee: Brandon Williams > > CVE-2014-3488 : The SslHandler in Netty before 3.9.2 allows remote attackers > to cause a denial of service (infinite loop and CPU consumption) via a > crafted SSLv2Hello message. > 2.0 currently uses Netty 3.6.6 > Upgrading to 3.9.2+ will fix this issue, and patch a few other known > vulnerabilities that we do not currently expose. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: ninja-fix UFTest/CQLTester
Repository: cassandra
Updated Branches:
refs/heads/trunk c9a6d7ce0 -> ac3062188
ninja-fix UFTest/CQLTester
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ac306218
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ac306218
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ac306218
Branch: refs/heads/trunk
Commit: ac306218848f62be71db8cea5ce90c4835c626c6
Parents: c9a6d7c
Author: Robert Stupp
Authored: Tue Mar 17 16:24:57 2015 +0100
Committer: Robert Stupp
Committed: Tue Mar 17 16:24:57 2015 +0100
--
.../org/apache/cassandra/cql3/CQLTester.java| 27
test/unit/org/apache/cassandra/cql3/UFTest.java | 20 +++
2 files changed, 32 insertions(+), 15 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ac306218/test/unit/org/apache/cassandra/cql3/CQLTester.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/CQLTester.java
b/test/unit/org/apache/cassandra/cql3/CQLTester.java
index 831a8d7..d8914a9 100644
--- a/test/unit/org/apache/cassandra/cql3/CQLTester.java
+++ b/test/unit/org/apache/cassandra/cql3/CQLTester.java
@@ -75,11 +75,28 @@ public abstract class CQLTester
private static org.apache.cassandra.transport.Server server;
private static final int nativePort;
private static final InetAddress nativeAddr;
-private static final Cluster cluster[] = new
Cluster[Server.CURRENT_VERSION];
-private static final Session session[] = new
Session[Server.CURRENT_VERSION];
+private static final Cluster[] cluster;
+private static final Session[] session;
+
+static int maxProtocolVersion;
+static {
+int version;
+for (version = 1; version <= Server.CURRENT_VERSION; version++)
+{
+try
+{
+ProtocolVersion.fromInt(version);
+}
+catch (IllegalArgumentException e)
+{
+version--;
+break;
+}
+}
+maxProtocolVersion = version;
+cluster = new Cluster[maxProtocolVersion];
+session = new Session[maxProtocolVersion];
-static
-{
// Once per-JVM is enough
SchemaLoader.prepareServer();
@@ -210,7 +227,7 @@ public abstract class CQLTester
server = new org.apache.cassandra.transport.Server(nativeAddr,
nativePort);
server.start();
-for (int version = 1; version <= Server.CURRENT_VERSION; version++)
+for (int version = 1; version <= maxProtocolVersion; version++)
{
if (cluster[version-1] != null)
continue;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ac306218/test/unit/org/apache/cassandra/cql3/UFTest.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/UFTest.java
b/test/unit/org/apache/cassandra/cql3/UFTest.java
index 84a7dd9..b3cc545 100644
--- a/test/unit/org/apache/cassandra/cql3/UFTest.java
+++ b/test/unit/org/apache/cassandra/cql3/UFTest.java
@@ -670,7 +670,7 @@ public class UFTest extends CQLTester
row(list, set, map));
// same test - but via native protocol
-for (int version = Server.VERSION_2; version <=
Server.CURRENT_VERSION; version++)
+for (int version = Server.VERSION_2; version <= maxProtocolVersion;
version++)
assertRowsNet(version,
executeNet(version, "SELECT " + fList + "(lst), " +
fSet + "(st), " + fMap + "(mp) FROM %s WHERE key = 1"),
row(list, set, map));
@@ -751,7 +751,7 @@ public class UFTest extends CQLTester
Assert.assertNull(row.getBytes("t"));
Assert.assertNull(row.getBytes("u"));
-for (int version = Server.VERSION_2; version <=
Server.CURRENT_VERSION; version++)
+for (int version = Server.VERSION_2; version <= maxProtocolVersion;
version++)
{
Row r = executeNet(version, "SELECT " +
fList + "(lst) as l, " +
@@ -867,7 +867,7 @@ public class UFTest extends CQLTester
DataType.set(DataType.text()),
DataType.map(DataType.cint(),
DataType.cboolean()));
TupleValue tup = tType.newValue(1d, list, set, map);
-for (int version = Server.VERSION_2; version <=
Server.CURRENT_VERSION; version++)
+for (int version = Server.VERSION_2; version <= maxProtocolVersion;
version++)
{
assertRowsNet(version,
executeNet(version, "SELECT " + fTup0 + "(tup) FROM
%s WHERE key = 1"),
@@ -894,7 +894,7 @@ public class
[jira] [Commented] (CASSANDRA-8977) Potential DOS Exposure from Netty
[ https://issues.apache.org/jira/browse/CASSANDRA-8977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365317#comment-14365317 ] Ryan McGuire commented on CASSANDRA-8977: - Sure. > Potential DOS Exposure from Netty > - > > Key: CASSANDRA-8977 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8977 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Benjamin Coverston >Assignee: Brandon Williams > > CVE-2014-3488 : The SslHandler in Netty before 3.9.2 allows remote attackers > to cause a denial of service (infinite loop and CPU consumption) via a > crafted SSLv2Hello message. > 2.0 currently uses Netty 3.6.6 > Upgrading to 3.9.2+ will fix this issue, and patch a few other known > vulnerabilities that we do not currently expose. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8851) Uncaught exception on thread Thread[SharedPool-Worker-16,5,main] after upgrade to 2.1.3
[
https://issues.apache.org/jira/browse/CASSANDRA-8851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365360#comment-14365360
]
Benedict commented on CASSANDRA-8851:
-
Hi [~Hachmann]. Thanks, could you please provide the schema and your full
system log (at least as far as a couple of hours prior to the very first
occurrence of the exception)?
> Uncaught exception on thread Thread[SharedPool-Worker-16,5,main] after
> upgrade to 2.1.3
> ---
>
> Key: CASSANDRA-8851
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8851
> Project: Cassandra
> Issue Type: Bug
> Environment: ubuntu
>Reporter: Tobias Schlottke
>Assignee: Benedict
>Priority: Critical
> Fix For: 2.1.4
>
>
> Hi there,
> after upgrading to 2.1.3 we've got the following error every few seconds:
> {code}
> WARN [SharedPool-Worker-16] 2015-02-23 10:20:36,392
> AbstractTracingAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[SharedPool-Worker-16,5,main]: {}
> java.lang.AssertionError: null
> at org.apache.cassandra.io.util.Memory.size(Memory.java:307)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.utils.obs.OffHeapBitSet.capacity(OffHeapBitSet.java:61)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at org.apache.cassandra.utils.BloomFilter.indexes(BloomFilter.java:74)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.utils.BloomFilter.isPresent(BloomFilter.java:98)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:1366)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:1350)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.db.columniterator.SSTableSliceIterator.(SSTableSliceIterator.java:41)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:185)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:62)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:273)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:62)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1915)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1748)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:342)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:57)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> ~[na:1.7.0_45]
> at
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [apache-cassandra-2.1.3.jar:2.1.3]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
> {code}
> This seems to crash the compactions and pushes up server load and piles up
> compactions.
> Any idea / possible workaround?
> Best,
> Tobias
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8977) Potential DOS Exposure from Netty
[ https://issues.apache.org/jira/browse/CASSANDRA-8977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan McGuire updated CASSANDRA-8977: Tester: Michael Shuler > Potential DOS Exposure from Netty > - > > Key: CASSANDRA-8977 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8977 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Benjamin Coverston >Assignee: Brandon Williams > > CVE-2014-3488 : The SslHandler in Netty before 3.9.2 allows remote attackers > to cause a denial of service (infinite loop and CPU consumption) via a > crafted SSLv2Hello message. > 2.0 currently uses Netty 3.6.6 > Upgrading to 3.9.2+ will fix this issue, and patch a few other known > vulnerabilities that we do not currently expose. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8979) MerkleTree mismatch for deleted and non-existing rows
Stefan Podkowinski created CASSANDRA-8979: - Summary: MerkleTree mismatch for deleted and non-existing rows Key: CASSANDRA-8979 URL: https://issues.apache.org/jira/browse/CASSANDRA-8979 Project: Cassandra Issue Type: Bug Components: Core Reporter: Stefan Podkowinski Validation compaction will currently create different hashes for rows that have been deleted compared to nodes that have not seen the rows at all or have already compacted them away. In case this sounds familiar to you, see CASSANDRA-4905 which was supposed to prevent hashing of expired tombstones. This still seems to be in place, but does not address the issue completely. Or there was a change in 2.0 that rendered the patch ineffective. The problem is that rowHash() in the Validator will return a new hash in any case, whether the PrecompactedRow did actually update the digest or not. This will lead to the case that a purged, PrecompactedRow will not change the digest, but we end up with a different tree compared to not having rowHash called at all (such as in case the row already doesn't exist). As an implication, repair jobs will constantly detect mismatches between older sstables containing purgable rows and nodes that have already compacted these rows. After transfering the reported ranges, the newly created sstables will immediately get deleted again during the following compaction. This will happen for each repair run over again until the sstable with the purgable row finally gets compacted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-8979) MerkleTree mismatch for deleted and non-existing rows
[ https://issues.apache.org/jira/browse/CASSANDRA-8979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-8979: - Assignee: Yuki Morishita > MerkleTree mismatch for deleted and non-existing rows > - > > Key: CASSANDRA-8979 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8979 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Stefan Podkowinski >Assignee: Yuki Morishita > > Validation compaction will currently create different hashes for rows that > have been deleted compared to nodes that have not seen the rows at all or > have already compacted them away. > In case this sounds familiar to you, see CASSANDRA-4905 which was supposed to > prevent hashing of expired tombstones. This still seems to be in place, but > does not address the issue completely. Or there was a change in 2.0 that > rendered the patch ineffective. > The problem is that rowHash() in the Validator will return a new hash in any > case, whether the PrecompactedRow did actually update the digest or not. This > will lead to the case that a purged, PrecompactedRow will not change the > digest, but we end up with a different tree compared to not having rowHash > called at all (such as in case the row already doesn't exist). > As an implication, repair jobs will constantly detect mismatches between > older sstables containing purgable rows and nodes that have already compacted > these rows. After transfering the reported ranges, the newly created sstables > will immediately get deleted again during the following compaction. This will > happen for each repair run over again until the sstable with the purgable row > finally gets compacted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8979) MerkleTree mismatch for deleted and non-existing rows
[ https://issues.apache.org/jira/browse/CASSANDRA-8979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Podkowinski updated CASSANDRA-8979: -- Attachment: cassandra-2.0-8979-validator_patch.txt cassandra-2.0-8979-test.txt > MerkleTree mismatch for deleted and non-existing rows > - > > Key: CASSANDRA-8979 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8979 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Stefan Podkowinski >Assignee: Yuki Morishita > Attachments: cassandra-2.0-8979-test.txt, > cassandra-2.0-8979-validator_patch.txt > > > Validation compaction will currently create different hashes for rows that > have been deleted compared to nodes that have not seen the rows at all or > have already compacted them away. > In case this sounds familiar to you, see CASSANDRA-4905 which was supposed to > prevent hashing of expired tombstones. This still seems to be in place, but > does not address the issue completely. Or there was a change in 2.0 that > rendered the patch ineffective. > The problem is that rowHash() in the Validator will return a new hash in any > case, whether the PrecompactedRow did actually update the digest or not. This > will lead to the case that a purged, PrecompactedRow will not change the > digest, but we end up with a different tree compared to not having rowHash > called at all (such as in case the row already doesn't exist). > As an implication, repair jobs will constantly detect mismatches between > older sstables containing purgable rows and nodes that have already compacted > these rows. After transfering the reported ranges, the newly created sstables > will immediately get deleted again during the following compaction. This will > happen for each repair run over again until the sstable with the purgable row > finally gets compacted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8839) DatabaseDescriptor throws NPE when rpc_interface is used
[
https://issues.apache.org/jira/browse/CASSANDRA-8839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365447#comment-14365447
]
Ariel Weisberg commented on CASSANDRA-8839:
---
-Djava.net.preferIPv4Stack is all or nothing and I think we should provide
mechanism not policy. When I just tested it matters whether you select IPv4 or
IPv6 addresses for an interface and there is no transparent automatic routing
of incoming connections between them.
I just don't want to end up in a scenario where people go to deploy in
networking environments they don't have complete control over and they can't
make it work without unnecessary platform specific friction. It's not a lot of
code to implement preferring the ipv6 vs ipv4 stack for rpc and listen
interface.
I am still open to not adding the config it's just not my preference. The
change in behavior would then be that when we are provided with an interface we
bind to the first addressed provided by the JVM in the enumeration and ignore
the others.
> DatabaseDescriptor throws NPE when rpc_interface is used
>
>
> Key: CASSANDRA-8839
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8839
> Project: Cassandra
> Issue Type: Bug
> Components: Config
> Environment: 2.1.3
>Reporter: Jan Kesten
>Assignee: Ariel Weisberg
> Fix For: 2.1.4
>
>
> Copy from mail to dev mailinglist.
> When using
> - listen_interface instead of listen_address
> - rpc_interface instead of rpc_address
> starting 2.1.3 throws an NPE:
> {code}
> ERROR [main] 2015-02-20 07:50:09,661 DatabaseDescriptor.java:144 - Fatal
> error during configuration loading
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:411)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:133)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:110)
> [apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:465)
> [apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:554)
> [apache-cassandra-2.1.3.jar:2.1.3]
> {code}
> Occurs on debian package as well as in tar.gz distribution.
> {code}
> /* Local IP, hostname or interface to bind RPC server to */
> if(conf.rpc_address !=null&& conf.rpc_interface !=null)
> {
> throw newConfigurationException("Set rpc_address OR rpc_interface, not
> both");
> }
> else if(conf.rpc_address !=null)
> {
> try
> {
> rpcAddress = InetAddress.getByName(conf.rpc_address);
> }
> catch(UnknownHostException e)
> {
> throw newConfigurationException("Unknown host in rpc_address "+
> conf.rpc_address);
> }
> }
> else if(conf.rpc_interface !=null)
> {
> listenAddress =
> getNetworkInterfaceAddress(conf.rpc_interface,"rpc_interface");
> }
> else
> {
> rpcAddress = FBUtilities.getLocalAddress();
> }
> {code}
> I think that listenAddress in the second else block is an error. In my case
> rpc_interface is eth0, so listenAddress gets set, and rpcAddress remains
> unset. The result is NPE in line 411:
> {code}
> if(rpcAddress.isAnyLocalAddress())
> {code}
> After changing rpc_interface to rpc_address everything works as expected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8980) UDF shouldn't log errors at ERROR
[ https://issues.apache.org/jira/browse/CASSANDRA-8980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365467#comment-14365467 ] Sylvain Lebresne commented on CASSANDRA-8980: - [~snazy] provided there is no disagreement on this on principle, you have my blessing for committing the fix directly. > UDF shouldn't log errors at ERROR > - > > Key: CASSANDRA-8980 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8980 > Project: Cassandra > Issue Type: Improvement >Reporter: Sylvain Lebresne >Assignee: Robert Stupp >Priority: Trivial > Fix For: 3.0 > > > If a UDT throws an exception, the code currently log it at ERROR. We > shouldn't do that as that makes it very easy for someone to flood the log > file. Besides, we send back an exception to the user in that case so logging > is kind of unnecessary (we can of course keep it at debug though). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8980) UDF shouldn't log errors at ERROR
Sylvain Lebresne created CASSANDRA-8980: --- Summary: UDF shouldn't log errors at ERROR Key: CASSANDRA-8980 URL: https://issues.apache.org/jira/browse/CASSANDRA-8980 Project: Cassandra Issue Type: Improvement Reporter: Sylvain Lebresne Assignee: Robert Stupp Priority: Trivial Fix For: 3.0 If a UDT throws an exception, the code currently log it at ERROR. We shouldn't do that as that makes it very easy for someone to flood the log file. Besides, we send back an exception to the user in that case so logging is kind of unnecessary (we can of course keep it at debug though). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: UDF shouldn't log errors at ERROR
Repository: cassandra
Updated Branches:
refs/heads/trunk ac3062188 -> 287366752
UDF shouldn't log errors at ERROR
Patch by Robert Stupp for CASSANDRA-8980
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/28736675
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/28736675
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/28736675
Branch: refs/heads/trunk
Commit: 2873667527b555221622579a849cd294ec3ea42c
Parents: ac30621
Author: Robert Stupp
Authored: Tue Mar 17 18:03:41 2015 +0100
Committer: Robert Stupp
Committed: Tue Mar 17 18:03:41 2015 +0100
--
.../apache/cassandra/cql3/functions/JavaSourceUDFFactory.java| 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/28736675/src/java/org/apache/cassandra/cql3/functions/JavaSourceUDFFactory.java
--
diff --git
a/src/java/org/apache/cassandra/cql3/functions/JavaSourceUDFFactory.java
b/src/java/org/apache/cassandra/cql3/functions/JavaSourceUDFFactory.java
index 2fb855a..3033d98 100644
--- a/src/java/org/apache/cassandra/cql3/functions/JavaSourceUDFFactory.java
+++ b/src/java/org/apache/cassandra/cql3/functions/JavaSourceUDFFactory.java
@@ -202,7 +202,7 @@ public final class JavaSourceUDFFactory
* }
* catch (Throwable t)
* {
- * logger.error("Invocation of function '{}' failed", this, t);
+ * logger.info("Invocation of function '{}' failed", this, t);
* if (t instanceof VirtualMachineError)
* throw (VirtualMachineError)t;
* throw
org.apache.cassandra.exceptions.FunctionExecutionException.build(this, t);
@@ -244,7 +244,7 @@ public final class JavaSourceUDFFactory
" }\n" +
" catch (Throwable t)\n" +
" {\n" +
-"logger.error(\"Invocation of function '{}' failed\",
this, t);\n" +
+"logger.info(\"Invocation of function '{}' failed\",
this, t);\n" +
// handle OutOfMemoryError and other fatals not here!
"if (t instanceof VirtualMachineError)\n" +
" throw (VirtualMachineError)t;\n" +
[jira] [Resolved] (CASSANDRA-8980) UDF shouldn't log errors at ERROR
[ https://issues.apache.org/jira/browse/CASSANDRA-8980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp resolved CASSANDRA-8980. - Resolution: Fixed OK :) Committed as 2873667 Was a one-line fix for Java-source UDFs. Scripted UDFs didn't log at ERROR level. > UDF shouldn't log errors at ERROR > - > > Key: CASSANDRA-8980 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8980 > Project: Cassandra > Issue Type: Improvement >Reporter: Sylvain Lebresne >Assignee: Robert Stupp >Priority: Trivial > Fix For: 3.0 > > > If a UDT throws an exception, the code currently log it at ERROR. We > shouldn't do that as that makes it very easy for someone to flood the log > file. Besides, we send back an exception to the user in that case so logging > is kind of unnecessary (we can of course keep it at debug though). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8981) IndexSummaryManagerTest.testCompactionsRace failing on trunk
Joshua McKenzie created CASSANDRA-8981:
--
Summary: IndexSummaryManagerTest.testCompactionsRace failing on
trunk
Key: CASSANDRA-8981
URL: https://issues.apache.org/jira/browse/CASSANDRA-8981
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Joshua McKenzie
Priority: Minor
Fix For: 3.0
Keep running it repeatedly w/showoutput="yes" in build.xml on junit and you'll
see it time out with:
{noformat}
[junit] WARN 17:02:56 Unable to cancel in-progress compactions for
StandardRace. Perhaps there is an unusually large row in progress somewhere,
or the system is simply overloaded.
[junit] WARN 17:02:56 Unable to cancel in-progress compactions for
StandardRace. Perhaps there is an unusually large row in progress somewhere,
or the system is simply overloaded.
[junit] WARN 17:02:57 Unable to cancel in-progress compactions for
StandardRace. Perhaps there is an unusually large row in progress somewhere,
or the system is simply overloaded.
[junit] WARN 17:02:57 Unable to cancel in-progress compactions for
StandardRace. Perhaps there is an unusually large row in progress somewhere,
or the system is simply overloaded.
{noformat}
I originally thought this was a Windows specific problem (CASSANDRA-8962) but
can reproduce on linux by just repeatedly running the test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8962) Fix SSTableRewriterTest on Windows
[ https://issues.apache.org/jira/browse/CASSANDRA-8962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-8962: --- Description: Platform specific failures: org.apache.cassandra.io.sstable.SSTableRewriterTest.testNumberOfFiles_truncate org.apache.cassandra.io.sstable.SSTableRewriterTest.testSmallFiles org.apache.cassandra.io.sstable.SSTableRewriterTest.testNumberOfFiles_dont_clean_readers org.apache.cassandra.io.sstable.SSTableRewriterTest.testNumberOfFiles_finish_empty_new_writer was: Platform specific failures: org.apache.cassandra.io.sstable.IndexSummaryManagerTest.testCompactionRace org.apache.cassandra.io.sstable.SSTableRewriterTest.testNumberOfFiles_truncate org.apache.cassandra.io.sstable.SSTableRewriterTest.testSmallFiles org.apache.cassandra.io.sstable.SSTableRewriterTest.testNumberOfFiles_dont_clean_readers org.apache.cassandra.io.sstable.SSTableRewriterTest.testNumberOfFiles_finish_empty_new_writer > Fix SSTableRewriterTest on Windows > -- > > Key: CASSANDRA-8962 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8962 > Project: Cassandra > Issue Type: Bug >Reporter: Joshua McKenzie >Assignee: Joshua McKenzie >Priority: Trivial > Labels: Windows > Fix For: 3.0 > > > Platform specific failures: > org.apache.cassandra.io.sstable.SSTableRewriterTest.testNumberOfFiles_truncate > org.apache.cassandra.io.sstable.SSTableRewriterTest.testSmallFiles > org.apache.cassandra.io.sstable.SSTableRewriterTest.testNumberOfFiles_dont_clean_readers > org.apache.cassandra.io.sstable.SSTableRewriterTest.testNumberOfFiles_finish_empty_new_writer -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8981) IndexSummaryManagerTest.testCompactionsRace failing on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-8981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365545#comment-14365545
]
Joshua McKenzie commented on CASSANDRA-8981:
[~benedict]: Looks like you wrote this test and you've been in this space quite
a bit recently - would you be willing to take a look at this?
> IndexSummaryManagerTest.testCompactionsRace failing on trunk
>
>
> Key: CASSANDRA-8981
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8981
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Joshua McKenzie
>Priority: Minor
> Fix For: 3.0
>
>
> Keep running it repeatedly w/showoutput="yes" in build.xml on junit and
> you'll see it time out with:
> {noformat}
> [junit] WARN 17:02:56 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> [junit] WARN 17:02:56 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> [junit] WARN 17:02:57 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> [junit] WARN 17:02:57 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> {noformat}
> I originally thought this was a Windows specific problem (CASSANDRA-8962) but
> can reproduce on linux by just repeatedly running the test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8962) Fix SSTableRewriterTest on Windows
[ https://issues.apache.org/jira/browse/CASSANDRA-8962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-8962: --- Summary: Fix SSTableRewriterTest on Windows (was: Fix IndexSummaryManagerTest and SSTableRewriterTest on Windows) > Fix SSTableRewriterTest on Windows > -- > > Key: CASSANDRA-8962 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8962 > Project: Cassandra > Issue Type: Bug >Reporter: Joshua McKenzie >Assignee: Joshua McKenzie >Priority: Trivial > Labels: Windows > Fix For: 3.0 > > > Platform specific failures: > org.apache.cassandra.io.sstable.IndexSummaryManagerTest.testCompactionRace > org.apache.cassandra.io.sstable.SSTableRewriterTest.testNumberOfFiles_truncate > org.apache.cassandra.io.sstable.SSTableRewriterTest.testSmallFiles > org.apache.cassandra.io.sstable.SSTableRewriterTest.testNumberOfFiles_dont_clean_readers > org.apache.cassandra.io.sstable.SSTableRewriterTest.testNumberOfFiles_finish_empty_new_writer -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8839) DatabaseDescriptor throws NPE when rpc_interface is used
[
https://issues.apache.org/jira/browse/CASSANDRA-8839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365533#comment-14365533
]
Carl Yeksigian commented on CASSANDRA-8839:
---
Yeah, it doesn't work quite the way I thought it would when testing the
sockets. Specifying "::" is the only way to get a dual-stack ServerSocket.
+1 to the latest branches.
> DatabaseDescriptor throws NPE when rpc_interface is used
>
>
> Key: CASSANDRA-8839
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8839
> Project: Cassandra
> Issue Type: Bug
> Components: Config
> Environment: 2.1.3
>Reporter: Jan Kesten
>Assignee: Ariel Weisberg
> Fix For: 2.1.4
>
>
> Copy from mail to dev mailinglist.
> When using
> - listen_interface instead of listen_address
> - rpc_interface instead of rpc_address
> starting 2.1.3 throws an NPE:
> {code}
> ERROR [main] 2015-02-20 07:50:09,661 DatabaseDescriptor.java:144 - Fatal
> error during configuration loading
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:411)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:133)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:110)
> [apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:465)
> [apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:554)
> [apache-cassandra-2.1.3.jar:2.1.3]
> {code}
> Occurs on debian package as well as in tar.gz distribution.
> {code}
> /* Local IP, hostname or interface to bind RPC server to */
> if(conf.rpc_address !=null&& conf.rpc_interface !=null)
> {
> throw newConfigurationException("Set rpc_address OR rpc_interface, not
> both");
> }
> else if(conf.rpc_address !=null)
> {
> try
> {
> rpcAddress = InetAddress.getByName(conf.rpc_address);
> }
> catch(UnknownHostException e)
> {
> throw newConfigurationException("Unknown host in rpc_address "+
> conf.rpc_address);
> }
> }
> else if(conf.rpc_interface !=null)
> {
> listenAddress =
> getNetworkInterfaceAddress(conf.rpc_interface,"rpc_interface");
> }
> else
> {
> rpcAddress = FBUtilities.getLocalAddress();
> }
> {code}
> I think that listenAddress in the second else block is an error. In my case
> rpc_interface is eth0, so listenAddress gets set, and rpcAddress remains
> unset. The result is NPE in line 411:
> {code}
> if(rpcAddress.isAnyLocalAddress())
> {code}
> After changing rpc_interface to rpc_address everything works as expected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8981) IndexSummaryManagerTest.testCompactionsRace failing on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-8981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365557#comment-14365557
]
Benedict commented on CASSANDRA-8981:
-
Jake wrote this test, but sure. I'll take a look, perhaps tomorrow
> IndexSummaryManagerTest.testCompactionsRace failing on trunk
>
>
> Key: CASSANDRA-8981
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8981
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Joshua McKenzie
>Priority: Minor
> Fix For: 3.0
>
>
> Keep running it repeatedly w/showoutput="yes" in build.xml on junit and
> you'll see it time out with:
> {noformat}
> [junit] WARN 17:02:56 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> [junit] WARN 17:02:56 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> [junit] WARN 17:02:57 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> [junit] WARN 17:02:57 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> {noformat}
> I originally thought this was a Windows specific problem (CASSANDRA-8962) but
> can reproduce on linux by just repeatedly running the test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
cassandra git commit: UDF shouldn't log errors at ERROR (change to DEBUG)
Repository: cassandra
Updated Branches:
refs/heads/trunk 287366752 -> ac9ccfdf7
UDF shouldn't log errors at ERROR (change to DEBUG)
Patch by Robert Stupp for CASSANDRA-8980
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ac9ccfdf
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ac9ccfdf
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ac9ccfdf
Branch: refs/heads/trunk
Commit: ac9ccfdf7c1d06cf01eeb84561a7c514674cc3b8
Parents: 2873667
Author: Robert Stupp
Authored: Tue Mar 17 18:15:52 2015 +0100
Committer: Robert Stupp
Committed: Tue Mar 17 18:15:52 2015 +0100
--
.../apache/cassandra/cql3/functions/JavaSourceUDFFactory.java| 4 ++--
src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ac9ccfdf/src/java/org/apache/cassandra/cql3/functions/JavaSourceUDFFactory.java
--
diff --git
a/src/java/org/apache/cassandra/cql3/functions/JavaSourceUDFFactory.java
b/src/java/org/apache/cassandra/cql3/functions/JavaSourceUDFFactory.java
index 3033d98..1abb40b 100644
--- a/src/java/org/apache/cassandra/cql3/functions/JavaSourceUDFFactory.java
+++ b/src/java/org/apache/cassandra/cql3/functions/JavaSourceUDFFactory.java
@@ -202,7 +202,7 @@ public final class JavaSourceUDFFactory
* }
* catch (Throwable t)
* {
- * logger.info("Invocation of function '{}' failed", this, t);
+ * logger.debug("Invocation of function '{}' failed", this, t);
* if (t instanceof VirtualMachineError)
* throw (VirtualMachineError)t;
* throw
org.apache.cassandra.exceptions.FunctionExecutionException.build(this, t);
@@ -244,7 +244,7 @@ public final class JavaSourceUDFFactory
" }\n" +
" catch (Throwable t)\n" +
" {\n" +
-"logger.info(\"Invocation of function '{}' failed\",
this, t);\n" +
+"logger.debug(\"Invocation of function '{}' failed\",
this, t);\n" +
// handle OutOfMemoryError and other fatals not here!
"if (t instanceof VirtualMachineError)\n" +
" throw (VirtualMachineError)t;\n" +
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ac9ccfdf/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
--
diff --git a/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
b/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
index 06452e6..4fe6ac9 100644
--- a/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
+++ b/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
@@ -139,7 +139,7 @@ public class ScriptBasedUDF extends UDFunction
}
catch (RuntimeException | ScriptException e)
{
-logger.info("Execution of UDF '{}' failed", name, e);
+logger.debug("Execution of UDF '{}' failed", name, e);
throw FunctionExecutionException.create(this, e);
}
}
[jira] [Commented] (CASSANDRA-8394) Cassandra 3.0 Auth changes
[ https://issues.apache.org/jira/browse/CASSANDRA-8394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365648#comment-14365648 ] Jake Farrell commented on CASSANDRA-8394: - Really like the direction this is heading, would love to also see auth be auto replicated to all nodes without having to update the keyspace and run repair on all nodes like the current implementation > Cassandra 3.0 Auth changes > -- > > Key: CASSANDRA-8394 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8394 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Sam Tunnicliffe > Labels: docs-impacting > Fix For: 3.0 > > > This will be the task that will group all the 3.0 Auth changes, of which we > need quite some: > 1. Implement Roles (CASSANDRA-7653) > 2. Merge ISaslAwareAuthenticator and IAuthenticator, making IAuthenticator > SASL-only, and support full SASL capabilities (incl. proxy authentication - > see CASSANDRA-7686, CASSANDRA-8068) > 3. Remove the current simplistic permissions cache with > implementation-specific user/credentials/permissions caches, at least in the > implementation we ship with C* (CASSANDRA-7715, CASSANDRA-8194) > 4. Consider adding a way to split superuser-ness and the rights to manage > users (CASSANDRA-7216, CASSANDRA-8650) > 5. Add permissions for types and functions (CASSANDRA-7557) > 6. Consider re-introducing the TRUNCATE permission (CASSANDRA-8082) > 7. Re-introduce the DESCRIBE permission (CASSANDRA-8163) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7712) temporary files need to be cleaned by unit tests
[
https://issues.apache.org/jira/browse/CASSANDRA-7712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365690#comment-14365690
]
Michael Shuler commented on CASSANDRA-7712:
---
A couple suggestions were attempted as workarounds, but all the unit test data
was still written to /tmp/
{noformat}
(trunk)mshuler@hana:~/git/cassandra$ export TMPDIR=`pwd`/testdata
(trunk)mshuler@hana:~/git/cassandra$ ant test -Djava.io.tmpdir=`pwd`/testdata
...
{noformat}
> temporary files need to be cleaned by unit tests
>
>
> Key: CASSANDRA-7712
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7712
> Project: Cassandra
> Issue Type: Test
> Components: Tests
>Reporter: Michael Shuler
>Priority: Minor
> Labels: bootcamp, lhf
> Fix For: 2.0.14
>
> Attachments: 7712-hung-CliTest_system.log.gz, 7712-v2.txt,
> 7712-v3.txt, CASSANDRA-7712_apache_cassandra_2.0.txt
>
>
> There are many unit test temporary files left behind after test runs. In the
> case of CI servers, I have seen >70,000 files accumulate in /tmp over a
> period of time. Each unit test should make an effort to remove its temporary
> files when the test is completed.
> My current unit test cleanup block:
> {noformat}
> # clean up after unit tests..
> rm -rf /tmp/140*-0 /tmp/CFWith* /tmp/Counter1* /tmp/DescriptorTest*
> /tmp/Keyspace1* \
> /tmp/KeyStreamingTransferTestSpace* /tmp/SSTableExportTest*
> /tmp/SSTableImportTest* \
> /tmp/Standard1* /tmp/Statistics.db* /tmp/StreamingTransferTest*
> /tmp/ValuesWithQuotes* \
> /tmp/cassandra* /tmp/jna-* /tmp/ks-cf-ib-1-* /tmp/lengthtest*
> /tmp/liblz4-java*.so /tmp/readtest* \
> /tmp/set_length_during_read_mode* /tmp/set_negative_length*
> /tmp/snappy-*.so
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[1/3] cassandra git commit: DatabaseDescriptor throws NPE when rpc_interface is used
Repository: cassandra Updated Branches: refs/heads/cassandra-2.1 c0f159b82 -> 89cdfd8e0 refs/heads/trunk ac9ccfdf7 -> 8d570fa79 DatabaseDescriptor throws NPE when rpc_interface is used patch by Ariel Weisberg; reviewed by Carl Yeksigian for CASSANDRA-8839 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/89cdfd8e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/89cdfd8e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/89cdfd8e Branch: refs/heads/cassandra-2.1 Commit: 89cdfd8e075d8883d776d7f881735f1c25e3cb54 Parents: c0f159b Author: Ariel Weisberg Authored: Tue Mar 17 19:27:54 2015 +0100 Committer: Robert Stupp Committed: Tue Mar 17 19:27:54 2015 +0100 -- CHANGES.txt | 1 + conf/cassandra.yaml | 12 ++ .../org/apache/cassandra/config/Config.java | 6 +- .../cassandra/config/DatabaseDescriptor.java| 205 +++ .../config/YamlConfigurationLoader.java | 16 +- .../config/DatabaseDescriptorTest.java | 139 - 6 files changed, 277 insertions(+), 102 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/89cdfd8e/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 7b8e0ad..30bf698 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.1.4 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839) * Don't check if an sstable is live for offline compactions (CASSANDRA-8841) * Don't set clientMode in SSTableLoader (CASSANDRA-8238) * Fix SSTableRewriter with disabled early open (CASSANDRA-8535) http://git-wip-us.apache.org/repos/asf/cassandra/blob/89cdfd8e/conf/cassandra.yaml -- diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml index cea12b3..2b43ba7 100644 --- a/conf/cassandra.yaml +++ b/conf/cassandra.yaml @@ -370,8 +370,14 @@ ssl_storage_port: 7001 # address associated with the hostname (it might not be). # # Setting listen_address to 0.0.0.0 is always wrong. +# +# If you choose to specify the interface by name and the interface has an ipv4 and an ipv6 address +# you can specify which should be chosen using listen_interface_prefer_ipv6. If false the first ipv4 +# address will be used. If true the first ipv6 address will be used. Defaults to false preferring +# ipv4. If there is only one address it will be selected regardless of ipv4/ipv6. listen_address: localhost # listen_interface: eth0 +# listen_interface_prefer_ipv6: false # Address to broadcast to other Cassandra nodes # Leaving this blank will set it to the same value as listen_address @@ -422,8 +428,14 @@ start_rpc: true # set broadcast_rpc_address to a value other than 0.0.0.0. # # For security reasons, you should not expose this port to the internet. Firewall it if needed. +# +# If you choose to specify the interface by name and the interface has an ipv4 and an ipv6 address +# you can specify which should be chosen using rpc_interface_prefer_ipv6. If false the first ipv4 +# address will be used. If true the first ipv6 address will be used. Defaults to false preferring +# ipv4. If there is only one address it will be selected regardless of ipv4/ipv6. rpc_address: localhost # rpc_interface: eth1 +# rpc_interface_prefer_ipv6: false # port for Thrift to listen for clients on rpc_port: 9160 http://git-wip-us.apache.org/repos/asf/cassandra/blob/89cdfd8e/src/java/org/apache/cassandra/config/Config.java -- diff --git a/src/java/org/apache/cassandra/config/Config.java b/src/java/org/apache/cassandra/config/Config.java index ccd4467..fbbd1dd 100644 --- a/src/java/org/apache/cassandra/config/Config.java +++ b/src/java/org/apache/cassandra/config/Config.java @@ -34,7 +34,7 @@ import org.apache.cassandra.utils.FBUtilities; /** * A class that contains configuration properties for the cassandra node it runs within. - * + * * Properties declared as volatile can be mutated via JMX. */ public class Config @@ -101,12 +101,14 @@ public class Config public Integer ssl_storage_port = 7001; public String listen_address; public String listen_interface; +public Boolean listen_interface_prefer_ipv6 = false; public String broadcast_address; public String internode_authenticator; public Boolean start_rpc = true; public String rpc_address; public String rpc_interface; +public Boolean rpc_interface_prefer_ipv6 = false; public String broadcast_rpc_address; public Integer rpc_port = 9160; public Integer rpc_listen_backlog = 50; @@ -155,7 +157,7 @@ public cla
[jira] [Updated] (CASSANDRA-8839) DatabaseDescriptor throws NPE when rpc_interface is used
[
https://issues.apache.org/jira/browse/CASSANDRA-8839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-8839:
Attachment: 8893-2.1.txt
8839.txt
Committed (attached rebased patches)
> DatabaseDescriptor throws NPE when rpc_interface is used
>
>
> Key: CASSANDRA-8839
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8839
> Project: Cassandra
> Issue Type: Bug
> Components: Config
> Environment: 2.1.3
>Reporter: Jan Kesten
>Assignee: Ariel Weisberg
> Fix For: 2.1.4
>
> Attachments: 8839.txt, 8893-2.1.txt
>
>
> Copy from mail to dev mailinglist.
> When using
> - listen_interface instead of listen_address
> - rpc_interface instead of rpc_address
> starting 2.1.3 throws an NPE:
> {code}
> ERROR [main] 2015-02-20 07:50:09,661 DatabaseDescriptor.java:144 - Fatal
> error during configuration loading
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:411)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:133)
> ~[apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:110)
> [apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:465)
> [apache-cassandra-2.1.3.jar:2.1.3]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:554)
> [apache-cassandra-2.1.3.jar:2.1.3]
> {code}
> Occurs on debian package as well as in tar.gz distribution.
> {code}
> /* Local IP, hostname or interface to bind RPC server to */
> if(conf.rpc_address !=null&& conf.rpc_interface !=null)
> {
> throw newConfigurationException("Set rpc_address OR rpc_interface, not
> both");
> }
> else if(conf.rpc_address !=null)
> {
> try
> {
> rpcAddress = InetAddress.getByName(conf.rpc_address);
> }
> catch(UnknownHostException e)
> {
> throw newConfigurationException("Unknown host in rpc_address "+
> conf.rpc_address);
> }
> }
> else if(conf.rpc_interface !=null)
> {
> listenAddress =
> getNetworkInterfaceAddress(conf.rpc_interface,"rpc_interface");
> }
> else
> {
> rpcAddress = FBUtilities.getLocalAddress();
> }
> {code}
> I think that listenAddress in the second else block is an error. In my case
> rpc_interface is eth0, so listenAddress gets set, and rpcAddress remains
> unset. The result is NPE in line 411:
> {code}
> if(rpcAddress.isAnyLocalAddress())
> {code}
> After changing rpc_interface to rpc_address everything works as expected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8d570fa7 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8d570fa7 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8d570fa7 Branch: refs/heads/trunk Commit: 8d570fa7992f8eaae55e8c13c57e6e84491062ee Parents: ac9ccfd 89cdfd8 Author: Ariel Weisberg Authored: Tue Mar 17 19:31:54 2015 +0100 Committer: Robert Stupp Committed: Tue Mar 17 19:31:54 2015 +0100 -- CHANGES.txt | 1 + conf/cassandra.yaml | 12 ++ .../org/apache/cassandra/config/Config.java | 6 +- .../cassandra/config/DatabaseDescriptor.java| 202 ++- .../config/YamlConfigurationLoader.java | 16 +- .../config/DatabaseDescriptorTest.java | 139 - 6 files changed, 274 insertions(+), 102 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/8d570fa7/CHANGES.txt -- diff --cc CHANGES.txt index be57f1b,30bf698..f813496 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,76 -1,5 +1,77 @@@ +3.0 + * Add WriteFailureException to native protocol, notify coordinator of + write failures (CASSANDRA-8592) + * Convert SequentialWriter to nio (CASSANDRA-8709) + * Add role based access control (CASSANDRA-7653, 8650, 7216, 8760, 8849, 8761, 8850) + * Record client ip address in tracing sessions (CASSANDRA-8162) + * Indicate partition key columns in response metadata for prepared + statements (CASSANDRA-7660) + * Merge UUIDType and TimeUUIDType parse logic (CASSANDRA-8759) + * Avoid memory allocation when searching index summary (CASSANDRA-8793) + * Optimise (Time)?UUIDType Comparisons (CASSANDRA-8730) + * Make CRC32Ex into a separate maven dependency (CASSANDRA-8836) + * Use preloaded jemalloc w/ Unsafe (CASSANDRA-8714) + * Avoid accessing partitioner through StorageProxy (CASSANDRA-8244, 8268) + * Upgrade Metrics library and remove depricated metrics (CASSANDRA-5657) + * Serializing Row cache alternative, fully off heap (CASSANDRA-7438) + * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707) + * Make CassandraException unchecked, extend RuntimeException (CASSANDRA-8560) + * Support direct buffer decompression for reads (CASSANDRA-8464) + * DirectByteBuffer compatible LZ4 methods (CASSANDRA-7039) + * Group sstables for anticompaction correctly (CASSANDRA-8578) + * Add ReadFailureException to native protocol, respond + immediately when replicas encounter errors while handling + a read request (CASSANDRA-7886) + * Switch CommitLogSegment from RandomAccessFile to nio (CASSANDRA-8308) + * Allow mixing token and partition key restrictions (CASSANDRA-7016) + * Support index key/value entries on map collections (CASSANDRA-8473) + * Modernize schema tables (CASSANDRA-8261) + * Support for user-defined aggregation functions (CASSANDRA-8053) + * Fix NPE in SelectStatement with empty IN values (CASSANDRA-8419) + * Refactor SelectStatement, return IN results in natural order instead + of IN value list order and ignore duplicate values in partition key IN restrictions (CASSANDRA-7981) + * Support UDTs, tuples, and collections in user-defined + functions (CASSANDRA-7563) + * Fix aggregate fn results on empty selection, result column name, + and cqlsh parsing (CASSANDRA-8229) + * Mark sstables as repaired after full repair (CASSANDRA-7586) + * Extend Descriptor to include a format value and refactor reader/writer + APIs (CASSANDRA-7443) + * Integrate JMH for microbenchmarks (CASSANDRA-8151) + * Keep sstable levels when bootstrapping (CASSANDRA-7460) + * Add Sigar library and perform basic OS settings check on startup (CASSANDRA-7838) + * Support for aggregation functions (CASSANDRA-4914) + * Remove cassandra-cli (CASSANDRA-7920) + * Accept dollar quoted strings in CQL (CASSANDRA-7769) + * Make assassinate a first class command (CASSANDRA-7935) + * Support IN clause on any partition key column (CASSANDRA-7855) + * Support IN clause on any clustering column (CASSANDRA-4762) + * Improve compaction logging (CASSANDRA-7818) + * Remove YamlFileNetworkTopologySnitch (CASSANDRA-7917) + * Do anticompaction in groups (CASSANDRA-6851) + * Support user-defined functions (CASSANDRA-7395, 7526, 7562, 7740, 7781, 7929, + 7924, 7812, 8063, 7813, 7708) + * Permit configurable timestamps with cassandra-stress (CASSANDRA-7416) + * Move sstable RandomAccessReader to nio2, which allows using the + FILE_SHARE_DELETE flag on Windows (CASSANDRA-4050) + * Remove CQL2 (CASSANDRA-5918) + * Add Thrift get_multi_slice call (CASSANDRA-6757) + * Optimize fetching multiple cells by name (
[2/3] cassandra git commit: DatabaseDescriptor throws NPE when rpc_interface is used
DatabaseDescriptor throws NPE when rpc_interface is used patch by Ariel Weisberg; reviewed by Carl Yeksigian for CASSANDRA-8839 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/89cdfd8e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/89cdfd8e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/89cdfd8e Branch: refs/heads/trunk Commit: 89cdfd8e075d8883d776d7f881735f1c25e3cb54 Parents: c0f159b Author: Ariel Weisberg Authored: Tue Mar 17 19:27:54 2015 +0100 Committer: Robert Stupp Committed: Tue Mar 17 19:27:54 2015 +0100 -- CHANGES.txt | 1 + conf/cassandra.yaml | 12 ++ .../org/apache/cassandra/config/Config.java | 6 +- .../cassandra/config/DatabaseDescriptor.java| 205 +++ .../config/YamlConfigurationLoader.java | 16 +- .../config/DatabaseDescriptorTest.java | 139 - 6 files changed, 277 insertions(+), 102 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/89cdfd8e/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 7b8e0ad..30bf698 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.1.4 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839) * Don't check if an sstable is live for offline compactions (CASSANDRA-8841) * Don't set clientMode in SSTableLoader (CASSANDRA-8238) * Fix SSTableRewriter with disabled early open (CASSANDRA-8535) http://git-wip-us.apache.org/repos/asf/cassandra/blob/89cdfd8e/conf/cassandra.yaml -- diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml index cea12b3..2b43ba7 100644 --- a/conf/cassandra.yaml +++ b/conf/cassandra.yaml @@ -370,8 +370,14 @@ ssl_storage_port: 7001 # address associated with the hostname (it might not be). # # Setting listen_address to 0.0.0.0 is always wrong. +# +# If you choose to specify the interface by name and the interface has an ipv4 and an ipv6 address +# you can specify which should be chosen using listen_interface_prefer_ipv6. If false the first ipv4 +# address will be used. If true the first ipv6 address will be used. Defaults to false preferring +# ipv4. If there is only one address it will be selected regardless of ipv4/ipv6. listen_address: localhost # listen_interface: eth0 +# listen_interface_prefer_ipv6: false # Address to broadcast to other Cassandra nodes # Leaving this blank will set it to the same value as listen_address @@ -422,8 +428,14 @@ start_rpc: true # set broadcast_rpc_address to a value other than 0.0.0.0. # # For security reasons, you should not expose this port to the internet. Firewall it if needed. +# +# If you choose to specify the interface by name and the interface has an ipv4 and an ipv6 address +# you can specify which should be chosen using rpc_interface_prefer_ipv6. If false the first ipv4 +# address will be used. If true the first ipv6 address will be used. Defaults to false preferring +# ipv4. If there is only one address it will be selected regardless of ipv4/ipv6. rpc_address: localhost # rpc_interface: eth1 +# rpc_interface_prefer_ipv6: false # port for Thrift to listen for clients on rpc_port: 9160 http://git-wip-us.apache.org/repos/asf/cassandra/blob/89cdfd8e/src/java/org/apache/cassandra/config/Config.java -- diff --git a/src/java/org/apache/cassandra/config/Config.java b/src/java/org/apache/cassandra/config/Config.java index ccd4467..fbbd1dd 100644 --- a/src/java/org/apache/cassandra/config/Config.java +++ b/src/java/org/apache/cassandra/config/Config.java @@ -34,7 +34,7 @@ import org.apache.cassandra.utils.FBUtilities; /** * A class that contains configuration properties for the cassandra node it runs within. - * + * * Properties declared as volatile can be mutated via JMX. */ public class Config @@ -101,12 +101,14 @@ public class Config public Integer ssl_storage_port = 7001; public String listen_address; public String listen_interface; +public Boolean listen_interface_prefer_ipv6 = false; public String broadcast_address; public String internode_authenticator; public Boolean start_rpc = true; public String rpc_address; public String rpc_interface; +public Boolean rpc_interface_prefer_ipv6 = false; public String broadcast_rpc_address; public Integer rpc_port = 9160; public Integer rpc_listen_backlog = 50; @@ -155,7 +157,7 @@ public class Config public Double commitlog_sync_batch_window_in_ms; public Integer commitlog_sync_period_in_ms; public int commitlog_seg
[jira] [Created] (CASSANDRA-8982) sstableloader outputs progress information and summary stats to stderr instead of stdout
Tom Alexander created CASSANDRA-8982:
Summary: sstableloader outputs progress information and summary
stats to stderr instead of stdout
Key: CASSANDRA-8982
URL: https://issues.apache.org/jira/browse/CASSANDRA-8982
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Ubuntu 12.04.3 LTS
Cassandra 2.1.3
Reporter: Tom Alexander
On Cassandra 2.1.0+, the progress information from sstableloader is output into
the stderr stream instead of stdout.
This can be reproduced by running an sstableloader and redirecting stderr to a
file:
{code}
sstableloader -d 127.0.0.1
/home/automaton/cassandra-src/data/data/keyspace1/standard1-abfab2e0ccb811e49d4417363885fa00
2> error.log
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8982) sstableloader outputs progress information and summary stats to stderr instead of stdout
[
https://issues.apache.org/jira/browse/CASSANDRA-8982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Thompson updated CASSANDRA-8982:
---
Fix Version/s: 2.1.4
> sstableloader outputs progress information and summary stats to stderr
> instead of stdout
>
>
> Key: CASSANDRA-8982
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8982
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Ubuntu 12.04.3 LTS
> Cassandra 2.1.3
>Reporter: Tom Alexander
> Fix For: 2.1.4
>
>
> On Cassandra 2.1.0+, the progress information from sstableloader is output
> into the stderr stream instead of stdout.
> This can be reproduced by running an sstableloader and redirecting stderr to
> a file:
> {code}
> sstableloader -d 127.0.0.1
> /home/automaton/cassandra-src/data/data/keyspace1/standard1-abfab2e0ccb811e49d4417363885fa00
> 2> error.log
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8982) sstableloader outputs progress information and summary stats to stderr instead of stdout
[
https://issues.apache.org/jira/browse/CASSANDRA-8982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Thompson updated CASSANDRA-8982:
---
Since Version: 2.1.0
> sstableloader outputs progress information and summary stats to stderr
> instead of stdout
>
>
> Key: CASSANDRA-8982
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8982
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Ubuntu 12.04.3 LTS
> Cassandra 2.1.3
>Reporter: Tom Alexander
> Fix For: 2.1.4
>
>
> On Cassandra 2.1.0+, the progress information from sstableloader is output
> into the stderr stream instead of stdout.
> This can be reproduced by running an sstableloader and redirecting stderr to
> a file:
> {code}
> sstableloader -d 127.0.0.1
> /home/automaton/cassandra-src/data/data/keyspace1/standard1-abfab2e0ccb811e49d4417363885fa00
> 2> error.log
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8981) IndexSummaryManagerTest.testCompactionsRace failing on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-8981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365742#comment-14365742
]
Joshua McKenzie commented on CASSANDRA-8981:
That's what I get for trusting the output of git blame. ;)
> IndexSummaryManagerTest.testCompactionsRace failing on trunk
>
>
> Key: CASSANDRA-8981
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8981
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Joshua McKenzie
>Priority: Minor
> Fix For: 3.0
>
>
> Keep running it repeatedly w/showoutput="yes" in build.xml on junit and
> you'll see it time out with:
> {noformat}
> [junit] WARN 17:02:56 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> [junit] WARN 17:02:56 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> [junit] WARN 17:02:57 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> [junit] WARN 17:02:57 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> {noformat}
> I originally thought this was a Windows specific problem (CASSANDRA-8962) but
> can reproduce on linux by just repeatedly running the test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8981) IndexSummaryManagerTest.testCompactionsRace failing on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-8981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365749#comment-14365749
]
Benedict commented on CASSANDRA-8981:
-
Well, I did commit it, so you can't blame git blame for blaming me :)
> IndexSummaryManagerTest.testCompactionsRace failing on trunk
>
>
> Key: CASSANDRA-8981
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8981
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Joshua McKenzie
>Priority: Minor
> Fix For: 3.0
>
>
> Keep running it repeatedly w/showoutput="yes" in build.xml on junit and
> you'll see it time out with:
> {noformat}
> [junit] WARN 17:02:56 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> [junit] WARN 17:02:56 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> [junit] WARN 17:02:57 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> [junit] WARN 17:02:57 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> {noformat}
> I originally thought this was a Windows specific problem (CASSANDRA-8962) but
> can reproduce on linux by just repeatedly running the test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8981) IndexSummaryManagerTest.testCompactionsRace failing on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-8981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365769#comment-14365769
]
T Jake Luciani commented on CASSANDRA-8981:
---
Looks like it's just a timeout. The test is purposely trying to cause a race
condition. Perhaps this should be a long-unit test
> IndexSummaryManagerTest.testCompactionsRace failing on trunk
>
>
> Key: CASSANDRA-8981
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8981
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Joshua McKenzie
>Priority: Minor
> Fix For: 3.0
>
>
> Keep running it repeatedly w/showoutput="yes" in build.xml on junit and
> you'll see it time out with:
> {noformat}
> [junit] WARN 17:02:56 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> [junit] WARN 17:02:56 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> [junit] WARN 17:02:57 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> [junit] WARN 17:02:57 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> {noformat}
> I originally thought this was a Windows specific problem (CASSANDRA-8962) but
> can reproduce on linux by just repeatedly running the test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8981) IndexSummaryManagerTest.testCompactionsRace intermittently timing out on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-8981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-8981:
---
Summary: IndexSummaryManagerTest.testCompactionsRace intermittently timing
out on trunk (was: IndexSummaryManagerTest.testCompactionsRace failing on
trunk)
> IndexSummaryManagerTest.testCompactionsRace intermittently timing out on trunk
> --
>
> Key: CASSANDRA-8981
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8981
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Joshua McKenzie
>Priority: Minor
> Fix For: 3.0
>
>
> Keep running it repeatedly w/showoutput="yes" in build.xml on junit and
> you'll see it time out with:
> {noformat}
> [junit] WARN 17:02:56 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> [junit] WARN 17:02:56 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> [junit] WARN 17:02:57 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> [junit] WARN 17:02:57 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> {noformat}
> I originally thought this was a Windows specific problem (CASSANDRA-8962) but
> can reproduce on linux by just repeatedly running the test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8983) Move Hadoop code out of tree
Jeremiah Jordan created CASSANDRA-8983: -- Summary: Move Hadoop code out of tree Key: CASSANDRA-8983 URL: https://issues.apache.org/jira/browse/CASSANDRA-8983 Project: Cassandra Issue Type: Improvement Components: Hadoop Reporter: Jeremiah Jordan Priority: Minor After CASSANDRA-8358 all the Hadoop stuff should be going through the Java driver. Since there shouldn't be any internal dependancies after that, we should break out the Hadoop client driver code into its own repo, like we did for all the other client drivers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-8982) sstableloader outputs progress information and summary stats to stderr instead of stdout
[
https://issues.apache.org/jira/browse/CASSANDRA-8982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuki Morishita reassigned CASSANDRA-8982:
-
Assignee: Yuki Morishita
> sstableloader outputs progress information and summary stats to stderr
> instead of stdout
>
>
> Key: CASSANDRA-8982
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8982
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Ubuntu 12.04.3 LTS
> Cassandra 2.1.3
>Reporter: Tom Alexander
>Assignee: Yuki Morishita
> Fix For: 2.1.4
>
> Attachments: 0001-use-stdout-for-normal-output.patch
>
>
> On Cassandra 2.1.0+, the progress information from sstableloader is output
> into the stderr stream instead of stdout.
> This can be reproduced by running an sstableloader and redirecting stderr to
> a file:
> {code}
> sstableloader -d 127.0.0.1
> /home/automaton/cassandra-src/data/data/keyspace1/standard1-abfab2e0ccb811e49d4417363885fa00
> 2> error.log
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8982) sstableloader outputs progress information and summary stats to stderr instead of stdout
[
https://issues.apache.org/jira/browse/CASSANDRA-8982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuki Morishita updated CASSANDRA-8982:
--
Attachment: 0001-use-stdout-for-normal-output.patch
Patch attached to use stdout for progress and stats reporting.
> sstableloader outputs progress information and summary stats to stderr
> instead of stdout
>
>
> Key: CASSANDRA-8982
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8982
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Ubuntu 12.04.3 LTS
> Cassandra 2.1.3
>Reporter: Tom Alexander
> Fix For: 2.1.4
>
> Attachments: 0001-use-stdout-for-normal-output.patch
>
>
> On Cassandra 2.1.0+, the progress information from sstableloader is output
> into the stderr stream instead of stdout.
> This can be reproduced by running an sstableloader and redirecting stderr to
> a file:
> {code}
> sstableloader -d 127.0.0.1
> /home/automaton/cassandra-src/data/data/keyspace1/standard1-abfab2e0ccb811e49d4417363885fa00
> 2> error.log
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8982) sstableloader outputs progress information and summary stats to stderr instead of stdout
[
https://issues.apache.org/jira/browse/CASSANDRA-8982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-8982:
Reviewer: Brandon Williams (was: Joshua McKenzie)
+1
> sstableloader outputs progress information and summary stats to stderr
> instead of stdout
>
>
> Key: CASSANDRA-8982
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8982
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Ubuntu 12.04.3 LTS
> Cassandra 2.1.3
>Reporter: Tom Alexander
>Assignee: Yuki Morishita
> Fix For: 2.1.4
>
> Attachments: 0001-use-stdout-for-normal-output.patch
>
>
> On Cassandra 2.1.0+, the progress information from sstableloader is output
> into the stderr stream instead of stdout.
> This can be reproduced by running an sstableloader and redirecting stderr to
> a file:
> {code}
> sstableloader -d 127.0.0.1
> /home/automaton/cassandra-src/data/data/keyspace1/standard1-abfab2e0ccb811e49d4417363885fa00
> 2> error.log
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7712) temporary files need to be cleaned by unit tests
[
https://issues.apache.org/jira/browse/CASSANDRA-7712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler updated CASSANDRA-7712:
--
Attachment: 7712_workaround.txt
Attached 7712_workaround.txt patch to set a {{tmp.dir}} property and use that
as a {{jvmarg}} in testlist. Tested this with and without passing in the
option with no problems - the jvm also created my custom tmp.dir for me. This
workaround provides us the ability to set the temporary data dir within our git
workspace, and since we are using {{git clean -xdf}} at the beginning of every
unit test run, this data will no longer accumulate.
> temporary files need to be cleaned by unit tests
>
>
> Key: CASSANDRA-7712
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7712
> Project: Cassandra
> Issue Type: Test
> Components: Tests
>Reporter: Michael Shuler
>Priority: Minor
> Labels: bootcamp, lhf
> Fix For: 2.0.14
>
> Attachments: 7712-hung-CliTest_system.log.gz, 7712-v2.txt,
> 7712-v3.txt, 7712_workaround.txt, CASSANDRA-7712_apache_cassandra_2.0.txt
>
>
> There are many unit test temporary files left behind after test runs. In the
> case of CI servers, I have seen >70,000 files accumulate in /tmp over a
> period of time. Each unit test should make an effort to remove its temporary
> files when the test is completed.
> My current unit test cleanup block:
> {noformat}
> # clean up after unit tests..
> rm -rf /tmp/140*-0 /tmp/CFWith* /tmp/Counter1* /tmp/DescriptorTest*
> /tmp/Keyspace1* \
> /tmp/KeyStreamingTransferTestSpace* /tmp/SSTableExportTest*
> /tmp/SSTableImportTest* \
> /tmp/Standard1* /tmp/Statistics.db* /tmp/StreamingTransferTest*
> /tmp/ValuesWithQuotes* \
> /tmp/cassandra* /tmp/jna-* /tmp/ks-cf-ib-1-* /tmp/lengthtest*
> /tmp/liblz4-java*.so /tmp/readtest* \
> /tmp/set_length_during_read_mode* /tmp/set_negative_length*
> /tmp/snappy-*.so
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-7712) temporary files need to be cleaned by unit tests
[
https://issues.apache.org/jira/browse/CASSANDRA-7712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler reassigned CASSANDRA-7712:
-
Assignee: Michael Shuler
> temporary files need to be cleaned by unit tests
>
>
> Key: CASSANDRA-7712
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7712
> Project: Cassandra
> Issue Type: Test
> Components: Tests
>Reporter: Michael Shuler
>Assignee: Michael Shuler
>Priority: Minor
> Labels: bootcamp, lhf
> Fix For: 2.0.14
>
> Attachments: 7712-hung-CliTest_system.log.gz, 7712-v2.txt,
> 7712-v3.txt, 7712_workaround.txt, CASSANDRA-7712_apache_cassandra_2.0.txt
>
>
> There are many unit test temporary files left behind after test runs. In the
> case of CI servers, I have seen >70,000 files accumulate in /tmp over a
> period of time. Each unit test should make an effort to remove its temporary
> files when the test is completed.
> My current unit test cleanup block:
> {noformat}
> # clean up after unit tests..
> rm -rf /tmp/140*-0 /tmp/CFWith* /tmp/Counter1* /tmp/DescriptorTest*
> /tmp/Keyspace1* \
> /tmp/KeyStreamingTransferTestSpace* /tmp/SSTableExportTest*
> /tmp/SSTableImportTest* \
> /tmp/Standard1* /tmp/Statistics.db* /tmp/StreamingTransferTest*
> /tmp/ValuesWithQuotes* \
> /tmp/cassandra* /tmp/jna-* /tmp/ks-cf-ib-1-* /tmp/lengthtest*
> /tmp/liblz4-java*.so /tmp/readtest* \
> /tmp/set_length_during_read_mode* /tmp/set_negative_length*
> /tmp/snappy-*.so
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8838) Resumable bootstrap streaming
[
https://issues.apache.org/jira/browse/CASSANDRA-8838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365869#comment-14365869
]
Yuki Morishita commented on CASSANDRA-8838:
---
Alright, I force pushed updates to dtests. This version is at least reliable on
my machine.
https://github.com/yukim/cassandra-dtest/tree/CASSANDRA-8838
bq. With no_wait=True it waits for 2 seconds
That's not what I expected, I removed them all.
bq. I encountered another issue, in that the streaming completes before we kill
node 1
{{node#start()}} sometimes goes too far before we want to kill the node. This
version spawns a thread to watch log and kill node 1 so it reacts as expected.
I also fixed my new tests in {{replace_address_test.py}} that does not properly
replacing.
> Resumable bootstrap streaming
> -
>
> Key: CASSANDRA-8838
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8838
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Minor
> Labels: dense-storage
> Fix For: 3.0
>
>
> This allows the bootstrapping node not to be streamed already received data.
> The bootstrapping node records received keyspace/ranges as one stream session
> completes. When some sessions with other nodes fail, bootstrapping fails
> also, though next time it re-bootstraps, already received keyspace/ranges are
> skipped to be streamed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6809) Compressed Commit Log
[ https://issues.apache.org/jira/browse/CASSANDRA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365867#comment-14365867 ] Ariel Weisberg commented on CASSANDRA-6809: --- I think what Benedict is describing where we only deallocate if the queue has more than one available (and > 3) is good idea. If the knob is set too low it might give better behavior at steady state? I would still be +1 since I think it won't matter once we multi-thread compression (and I intend to do that for 3.1). > Compressed Commit Log > - > > Key: CASSANDRA-6809 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6809 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Assignee: Branimir Lambov >Priority: Minor > Labels: docs-impacting, performance > Fix For: 3.0 > > Attachments: ComitLogStress.java, logtest.txt > > > It seems an unnecessary oversight that we don't compress the commit log. > Doing so should improve throughput, but some care will need to be taken to > ensure we use as much of a segment as possible. I propose decoupling the > writing of the records from the segments. Basically write into a (queue of) > DirectByteBuffer, and have the sync thread compress, say, ~64K chunks every X > MB written to the CL (where X is ordinarily CLS size), and then pack as many > of the compressed chunks into a CLS as possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[5/6] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/77d58f90 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/77d58f90 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/77d58f90 Branch: refs/heads/cassandra-2.1 Commit: 77d58f900efac62c7c8dcc63c3de5ecb0fe4e343 Parents: 89cdfd8 b2aa67e Author: Brandon Williams Authored: Tue Mar 17 14:34:26 2015 -0500 Committer: Brandon Williams Committed: Tue Mar 17 14:34:26 2015 -0500 -- CHANGES.txt | 1 + build.xml | 2 ++ 2 files changed, 3 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/77d58f90/CHANGES.txt -- diff --cc CHANGES.txt index 30bf698,f10a89a..6b8618b --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,50 -1,5 +1,51 @@@ -2.0.14: +2.1.4 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839) + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841) + * Don't set clientMode in SSTableLoader (CASSANDRA-8238) + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535) + * Allow invalidating permissions and cache time (CASSANDRA-8722) + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0 + are executed (CASSANDRA-8418) + * Fix cassandra-stress so it respects the CL passed in user mode (CASSANDRA-8948) + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786) + * cassandra-stress reports per-operation statistics, plus misc (CASSANDRA-8769) + * Add SimpleDate (cql date) and Time (cql time) types (CASSANDRA-7523) + * Use long for key count in cfstats (CASSANDRA-8913) + * Make SSTableRewriter.abort() more robust to failure (CASSANDRA-8832) + * Remove cold_reads_to_omit from STCS (CASSANDRA-8860) + * Make EstimatedHistogram#percentile() use ceil instead of floor (CASSANDRA-8883) + * Fix top partitions reporting wrong cardinality (CASSANDRA-8834) + * Fix rare NPE in KeyCacheSerializer (CASSANDRA-8067) + * Pick sstables for validation as late as possible inc repairs (CASSANDRA-8366) + * Fix commitlog getPendingTasks to not increment (CASSANDRA-8856) + * Fix parallelism adjustment in range and secondary index queries + when the first fetch does not satisfy the limit (CASSANDRA-8856) + * Check if the filtered sstables is non-empty in STCS (CASSANDRA-8843) + * Upgrade java-driver used for cassandra-stress (CASSANDRA-8842) + * Fix CommitLog.forceRecycleAllSegments() memory access error (CASSANDRA-8812) + * Improve assertions in Memory (CASSANDRA-8792) + * Fix SSTableRewriter cleanup (CASSANDRA-8802) + * Introduce SafeMemory for CompressionMetadata.Writer (CASSANDRA-8758) + * 'nodetool info' prints exception against older node (CASSANDRA-8796) + * Ensure SSTableReader.last corresponds exactly with the file end (CASSANDRA-8750) + * Make SSTableWriter.openEarly more robust and obvious (CASSANDRA-8747) + * Enforce SSTableReader.first/last (CASSANDRA-8744) + * Cleanup SegmentedFile API (CASSANDRA-8749) + * Avoid overlap with early compaction replacement (CASSANDRA-8683) + * Safer Resource Management++ (CASSANDRA-8707) + * Write partition size estimates into a system table (CASSANDRA-7688) + * cqlsh: Fix keys() and full() collection indexes in DESCRIBE output + (CASSANDRA-8154) + * Show progress of streaming in nodetool netstats (CASSANDRA-8886) + * IndexSummaryBuilder utilises offheap memory, and shares data between + each IndexSummary opened from it (CASSANDRA-8757) + * markCompacting only succeeds if the exact SSTableReader instances being + marked are in the live set (CASSANDRA-8689) + * cassandra-stress support for varint (CASSANDRA-8882) + * Fix Adler32 digest for compressed sstables (CASSANDRA-8778) + * Add nodetool statushandoff/statusbackup (CASSANDRA-8912) +Merged from 2.0: + * Allow specifying the tmp dir (CASSANDRA-7712) * Improve compaction estimated tasks estimation (CASSANDRA-8904) * Fix duplicate up/down messages sent to native clients (CASSANDRA-7816) * Expose commit log archive status via JMX (CASSANDRA-8734) http://git-wip-us.apache.org/repos/asf/cassandra/blob/77d58f90/build.xml -- diff --cc build.xml index 4ced4e2,f5e14ba..bdb3646 --- a/build.xml +++ b/build.xml @@@ -1182,21 -1099,8 +1183,22 @@@ + + + + + + + + + + + + + + +
[2/6] cassandra git commit: Allow specifying the tmp dir
Allow specifying the tmp dir Patch by mshuler, reviewed by brandonwilliams for CASSANDRA-7712 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b2aa67ea Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b2aa67ea Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b2aa67ea Branch: refs/heads/cassandra-2.1 Commit: b2aa67ea298044f4c4ad905e54963f0c03643b68 Parents: 0814737 Author: Brandon Williams Authored: Tue Mar 17 14:33:27 2015 -0500 Committer: Brandon Williams Committed: Tue Mar 17 14:33:27 2015 -0500 -- CHANGES.txt | 1 + build.xml | 2 ++ 2 files changed, 3 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2aa67ea/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 65f2da4..f10a89a 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.0.14: + * Allow specifying the tmp dir (CASSANDRA-7712) * Improve compaction estimated tasks estimation (CASSANDRA-8904) * Fix duplicate up/down messages sent to native clients (CASSANDRA-7816) * Expose commit log archive status via JMX (CASSANDRA-8734) http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2aa67ea/build.xml -- diff --git a/build.xml b/build.xml index bdbd10a..f5e14ba 100644 --- a/build.xml +++ b/build.xml @@ -60,6 +60,7 @@ + @@ -1098,6 +1099,7 @@ +
[6/6] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/db1cf3e0 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/db1cf3e0 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/db1cf3e0 Branch: refs/heads/trunk Commit: db1cf3e084c2b36d66610e78ef89351df5d5d76a Parents: 8d570fa 77d58f9 Author: Brandon Williams Authored: Tue Mar 17 14:34:59 2015 -0500 Committer: Brandon Williams Committed: Tue Mar 17 14:34:59 2015 -0500 -- CHANGES.txt | 2 +- build.xml | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/db1cf3e0/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/db1cf3e0/build.xml -- diff --cc build.xml index 3fb4240,bdb3646..c99e8f8 --- a/build.xml +++ b/build.xml @@@ -62,9 -60,9 +62,10 @@@ + +
[3/6] cassandra git commit: Allow specifying the tmp dir
Allow specifying the tmp dir Patch by mshuler, reviewed by brandonwilliams for CASSANDRA-7712 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b2aa67ea Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b2aa67ea Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b2aa67ea Branch: refs/heads/trunk Commit: b2aa67ea298044f4c4ad905e54963f0c03643b68 Parents: 0814737 Author: Brandon Williams Authored: Tue Mar 17 14:33:27 2015 -0500 Committer: Brandon Williams Committed: Tue Mar 17 14:33:27 2015 -0500 -- CHANGES.txt | 1 + build.xml | 2 ++ 2 files changed, 3 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2aa67ea/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 65f2da4..f10a89a 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.0.14: + * Allow specifying the tmp dir (CASSANDRA-7712) * Improve compaction estimated tasks estimation (CASSANDRA-8904) * Fix duplicate up/down messages sent to native clients (CASSANDRA-7816) * Expose commit log archive status via JMX (CASSANDRA-8734) http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2aa67ea/build.xml -- diff --git a/build.xml b/build.xml index bdbd10a..f5e14ba 100644 --- a/build.xml +++ b/build.xml @@ -60,6 +60,7 @@ + @@ -1098,6 +1099,7 @@ +
[4/6] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/77d58f90 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/77d58f90 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/77d58f90 Branch: refs/heads/trunk Commit: 77d58f900efac62c7c8dcc63c3de5ecb0fe4e343 Parents: 89cdfd8 b2aa67e Author: Brandon Williams Authored: Tue Mar 17 14:34:26 2015 -0500 Committer: Brandon Williams Committed: Tue Mar 17 14:34:26 2015 -0500 -- CHANGES.txt | 1 + build.xml | 2 ++ 2 files changed, 3 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/77d58f90/CHANGES.txt -- diff --cc CHANGES.txt index 30bf698,f10a89a..6b8618b --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,50 -1,5 +1,51 @@@ -2.0.14: +2.1.4 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839) + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841) + * Don't set clientMode in SSTableLoader (CASSANDRA-8238) + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535) + * Allow invalidating permissions and cache time (CASSANDRA-8722) + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0 + are executed (CASSANDRA-8418) + * Fix cassandra-stress so it respects the CL passed in user mode (CASSANDRA-8948) + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786) + * cassandra-stress reports per-operation statistics, plus misc (CASSANDRA-8769) + * Add SimpleDate (cql date) and Time (cql time) types (CASSANDRA-7523) + * Use long for key count in cfstats (CASSANDRA-8913) + * Make SSTableRewriter.abort() more robust to failure (CASSANDRA-8832) + * Remove cold_reads_to_omit from STCS (CASSANDRA-8860) + * Make EstimatedHistogram#percentile() use ceil instead of floor (CASSANDRA-8883) + * Fix top partitions reporting wrong cardinality (CASSANDRA-8834) + * Fix rare NPE in KeyCacheSerializer (CASSANDRA-8067) + * Pick sstables for validation as late as possible inc repairs (CASSANDRA-8366) + * Fix commitlog getPendingTasks to not increment (CASSANDRA-8856) + * Fix parallelism adjustment in range and secondary index queries + when the first fetch does not satisfy the limit (CASSANDRA-8856) + * Check if the filtered sstables is non-empty in STCS (CASSANDRA-8843) + * Upgrade java-driver used for cassandra-stress (CASSANDRA-8842) + * Fix CommitLog.forceRecycleAllSegments() memory access error (CASSANDRA-8812) + * Improve assertions in Memory (CASSANDRA-8792) + * Fix SSTableRewriter cleanup (CASSANDRA-8802) + * Introduce SafeMemory for CompressionMetadata.Writer (CASSANDRA-8758) + * 'nodetool info' prints exception against older node (CASSANDRA-8796) + * Ensure SSTableReader.last corresponds exactly with the file end (CASSANDRA-8750) + * Make SSTableWriter.openEarly more robust and obvious (CASSANDRA-8747) + * Enforce SSTableReader.first/last (CASSANDRA-8744) + * Cleanup SegmentedFile API (CASSANDRA-8749) + * Avoid overlap with early compaction replacement (CASSANDRA-8683) + * Safer Resource Management++ (CASSANDRA-8707) + * Write partition size estimates into a system table (CASSANDRA-7688) + * cqlsh: Fix keys() and full() collection indexes in DESCRIBE output + (CASSANDRA-8154) + * Show progress of streaming in nodetool netstats (CASSANDRA-8886) + * IndexSummaryBuilder utilises offheap memory, and shares data between + each IndexSummary opened from it (CASSANDRA-8757) + * markCompacting only succeeds if the exact SSTableReader instances being + marked are in the live set (CASSANDRA-8689) + * cassandra-stress support for varint (CASSANDRA-8882) + * Fix Adler32 digest for compressed sstables (CASSANDRA-8778) + * Add nodetool statushandoff/statusbackup (CASSANDRA-8912) +Merged from 2.0: + * Allow specifying the tmp dir (CASSANDRA-7712) * Improve compaction estimated tasks estimation (CASSANDRA-8904) * Fix duplicate up/down messages sent to native clients (CASSANDRA-7816) * Expose commit log archive status via JMX (CASSANDRA-8734) http://git-wip-us.apache.org/repos/asf/cassandra/blob/77d58f90/build.xml -- diff --cc build.xml index 4ced4e2,f5e14ba..bdb3646 --- a/build.xml +++ b/build.xml @@@ -1182,21 -1099,8 +1183,22 @@@ + + + + + + + + + + + + + + +
[1/6] cassandra git commit: Allow specifying the tmp dir
Repository: cassandra Updated Branches: refs/heads/cassandra-2.0 08147376a -> b2aa67ea2 refs/heads/cassandra-2.1 89cdfd8e0 -> 77d58f900 refs/heads/trunk 8d570fa79 -> db1cf3e08 Allow specifying the tmp dir Patch by mshuler, reviewed by brandonwilliams for CASSANDRA-7712 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b2aa67ea Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b2aa67ea Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b2aa67ea Branch: refs/heads/cassandra-2.0 Commit: b2aa67ea298044f4c4ad905e54963f0c03643b68 Parents: 0814737 Author: Brandon Williams Authored: Tue Mar 17 14:33:27 2015 -0500 Committer: Brandon Williams Committed: Tue Mar 17 14:33:27 2015 -0500 -- CHANGES.txt | 1 + build.xml | 2 ++ 2 files changed, 3 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2aa67ea/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 65f2da4..f10a89a 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.0.14: + * Allow specifying the tmp dir (CASSANDRA-7712) * Improve compaction estimated tasks estimation (CASSANDRA-8904) * Fix duplicate up/down messages sent to native clients (CASSANDRA-7816) * Expose commit log archive status via JMX (CASSANDRA-8734) http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2aa67ea/build.xml -- diff --git a/build.xml b/build.xml index bdbd10a..f5e14ba 100644 --- a/build.xml +++ b/build.xml @@ -60,6 +60,7 @@ + @@ -1098,6 +1099,7 @@ +
[1/3] cassandra git commit: Use stdout for progress and stats in sstableloader
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 77d58f900 -> db1a741ea
refs/heads/trunk db1cf3e08 -> 50ea0182e
Use stdout for progress and stats in sstableloader
patch by yukim; reviewed by Brandon Williams for CASSANDRA-8982
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/db1a741e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/db1a741e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/db1a741e
Branch: refs/heads/cassandra-2.1
Commit: db1a741ea2f30e992345bd06deab843313718775
Parents: 77d58f9
Author: Yuki Morishita
Authored: Tue Mar 17 14:00:54 2015 -0500
Committer: Yuki Morishita
Committed: Tue Mar 17 14:37:55 2015 -0500
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/tools/BulkLoader.java | 4 ++--
2 files changed, 3 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/db1a741e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6b8618b..56a7164 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -44,6 +44,7 @@
* cassandra-stress support for varint (CASSANDRA-8882)
* Fix Adler32 digest for compressed sstables (CASSANDRA-8778)
* Add nodetool statushandoff/statusbackup (CASSANDRA-8912)
+ * Use stdout for progress and stats in sstableloader (CASSANDRA-8982)
Merged from 2.0:
* Allow specifying the tmp dir (CASSANDRA-7712)
* Improve compaction estimated tasks estimation (CASSANDRA-8904)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/db1a741e/src/java/org/apache/cassandra/tools/BulkLoader.java
--
diff --git a/src/java/org/apache/cassandra/tools/BulkLoader.java
b/src/java/org/apache/cassandra/tools/BulkLoader.java
index 73f4ec5..88a4404 100644
--- a/src/java/org/apache/cassandra/tools/BulkLoader.java
+++ b/src/java/org/apache/cassandra/tools/BulkLoader.java
@@ -227,7 +227,7 @@ public class BulkLoader
peak = average;
sb.append("(avg: ").append(average).append(" MB/s)");
-System.err.print(sb.toString());
+System.out.print(sb.toString());
}
}
@@ -250,7 +250,7 @@ public class BulkLoader
sb.append(String.format(" %-30s: %-10d%n", "Total duration (ms):
", durationMS));
sb.append(String.format(" %-30s: %-10d%n", "Average transfer
rate (MB/s): ", + average));
sb.append(String.format(" %-30s: %-10d%n", "Peak transfer rate
(MB/s): ", + peak));
-System.err.println(sb.toString());
+System.out.println(sb.toString());
}
}
[2/3] cassandra git commit: Use stdout for progress and stats in sstableloader
Use stdout for progress and stats in sstableloader
patch by yukim; reviewed by Brandon Williams for CASSANDRA-8982
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/db1a741e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/db1a741e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/db1a741e
Branch: refs/heads/trunk
Commit: db1a741ea2f30e992345bd06deab843313718775
Parents: 77d58f9
Author: Yuki Morishita
Authored: Tue Mar 17 14:00:54 2015 -0500
Committer: Yuki Morishita
Committed: Tue Mar 17 14:37:55 2015 -0500
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/tools/BulkLoader.java | 4 ++--
2 files changed, 3 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/db1a741e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6b8618b..56a7164 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -44,6 +44,7 @@
* cassandra-stress support for varint (CASSANDRA-8882)
* Fix Adler32 digest for compressed sstables (CASSANDRA-8778)
* Add nodetool statushandoff/statusbackup (CASSANDRA-8912)
+ * Use stdout for progress and stats in sstableloader (CASSANDRA-8982)
Merged from 2.0:
* Allow specifying the tmp dir (CASSANDRA-7712)
* Improve compaction estimated tasks estimation (CASSANDRA-8904)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/db1a741e/src/java/org/apache/cassandra/tools/BulkLoader.java
--
diff --git a/src/java/org/apache/cassandra/tools/BulkLoader.java
b/src/java/org/apache/cassandra/tools/BulkLoader.java
index 73f4ec5..88a4404 100644
--- a/src/java/org/apache/cassandra/tools/BulkLoader.java
+++ b/src/java/org/apache/cassandra/tools/BulkLoader.java
@@ -227,7 +227,7 @@ public class BulkLoader
peak = average;
sb.append("(avg: ").append(average).append(" MB/s)");
-System.err.print(sb.toString());
+System.out.print(sb.toString());
}
}
@@ -250,7 +250,7 @@ public class BulkLoader
sb.append(String.format(" %-30s: %-10d%n", "Total duration (ms):
", durationMS));
sb.append(String.format(" %-30s: %-10d%n", "Average transfer
rate (MB/s): ", + average));
sb.append(String.format(" %-30s: %-10d%n", "Peak transfer rate
(MB/s): ", + peak));
-System.err.println(sb.toString());
+System.out.println(sb.toString());
}
}
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365884#comment-14365884 ] Alex Liu commented on CASSANDRA-7410: - Waiting for CASSANDRA-8358 > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.14 > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/50ea0182 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/50ea0182 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/50ea0182 Branch: refs/heads/trunk Commit: 50ea0182e784413e2fba89a97bf7d0c6b8d4570b Parents: db1cf3e db1a741 Author: Yuki Morishita Authored: Tue Mar 17 14:38:33 2015 -0500 Committer: Yuki Morishita Committed: Tue Mar 17 14:38:33 2015 -0500 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/tools/BulkLoader.java | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/50ea0182/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/50ea0182/src/java/org/apache/cassandra/tools/BulkLoader.java --
[jira] [Commented] (CASSANDRA-8358) Bundled tools shouldn't be using Thrift API
[ https://issues.apache.org/jira/browse/CASSANDRA-8358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365885#comment-14365885 ] Alex Liu commented on CASSANDRA-8358: - When can this ticket committed to 2.1 branch? > Bundled tools shouldn't be using Thrift API > --- > > Key: CASSANDRA-8358 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8358 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Philip Thompson > Fix For: 3.0 > > > In 2.1, we switched cqlsh to the python-driver. > In 3.0, we got rid of cassandra-cli. > Yet there is still code that's using legacy Thrift API. We want to convert it > all to use the java-driver instead. > 1. BulkLoader uses Thrift to query the schema tables. It should be using > java-driver metadata APIs directly instead. > 2. o.a.c.hadoop.cql3.CqlRecordWriter is using Thrift > 3. o.a.c.hadoop.ColumnFamilyRecordReader is using Thrift > 4. o.a.c.hadoop.AbstractCassandraStorage is using Thrift > 5. o.a.c.hadoop.pig.CqlStorage is using Thrift > Some of the things listed above use Thrift to get the list of partition key > columns or clustering columns. Those should be converted to use the Metadata > API of the java-driver. > Somewhat related to that, we also have badly ported code from Thrift in > o.a.c.hadoop.cql3.CqlRecordReader (see fetchKeys()) that manually fetches > columns from schema tables instead of properly using the driver's Metadata > API. > We need all of it fixed. One exception, for now, is > o.a.c.hadoop.AbstractColumnFamilyInputFormat - it's using Thrift for its > describe_splits_ex() call that cannot be currently replaced by any > java-driver call (?). > Once this is done, we can stop starting Thrift RPC port by default in > cassandra.yaml. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8358) Bundled tools shouldn't be using Thrift API
[ https://issues.apache.org/jira/browse/CASSANDRA-8358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365890#comment-14365890 ] Aleksey Yeschenko commented on CASSANDRA-8358: -- This ticket is headed into 3.0, not 2.1. Will be committed as soon as I review. > Bundled tools shouldn't be using Thrift API > --- > > Key: CASSANDRA-8358 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8358 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Philip Thompson > Fix For: 3.0 > > > In 2.1, we switched cqlsh to the python-driver. > In 3.0, we got rid of cassandra-cli. > Yet there is still code that's using legacy Thrift API. We want to convert it > all to use the java-driver instead. > 1. BulkLoader uses Thrift to query the schema tables. It should be using > java-driver metadata APIs directly instead. > 2. o.a.c.hadoop.cql3.CqlRecordWriter is using Thrift > 3. o.a.c.hadoop.ColumnFamilyRecordReader is using Thrift > 4. o.a.c.hadoop.AbstractCassandraStorage is using Thrift > 5. o.a.c.hadoop.pig.CqlStorage is using Thrift > Some of the things listed above use Thrift to get the list of partition key > columns or clustering columns. Those should be converted to use the Metadata > API of the java-driver. > Somewhat related to that, we also have badly ported code from Thrift in > o.a.c.hadoop.cql3.CqlRecordReader (see fetchKeys()) that manually fetches > columns from schema tables instead of properly using the driver's Metadata > API. > We need all of it fixed. One exception, for now, is > o.a.c.hadoop.AbstractColumnFamilyInputFormat - it's using Thrift for its > describe_splits_ex() call that cannot be currently replaced by any > java-driver call (?). > Once this is done, we can stop starting Thrift RPC port by default in > cassandra.yaml. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated CASSANDRA-7410: Comment: was deleted (was: Waiting for CASSANDRA-8358) > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.14 > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-7712) temporary files need to be cleaned by unit tests
[
https://issues.apache.org/jira/browse/CASSANDRA-7712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365859#comment-14365859
]
Michael Shuler edited comment on CASSANDRA-7712 at 3/17/15 7:53 PM:
Attached 7712_workaround.txt patch to set a {{tmp.dir}} property and use that
as a {{jvmarg}} in testlist. Tested this with and without passing in the
option with no problems - -the jvm also created my custom tmp.dir for me- (I
must have done a run with my tmp.dir already created - we do need to mkdir it).
This workaround provides us the ability to set the temporary data dir within
our git workspace, and since we are using {{git clean -xdf}} at the beginning
of every unit test run, this data will no longer accumulate.
was (Author: mshuler):
Attached 7712_workaround.txt patch to set a {{tmp.dir}} property and use that
as a {{jvmarg}} in testlist. Tested this with and without passing in the
option with no problems - the jvm also created my custom tmp.dir for me. This
workaround provides us the ability to set the temporary data dir within our git
workspace, and since we are using {{git clean -xdf}} at the beginning of every
unit test run, this data will no longer accumulate.
> temporary files need to be cleaned by unit tests
>
>
> Key: CASSANDRA-7712
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7712
> Project: Cassandra
> Issue Type: Test
> Components: Tests
>Reporter: Michael Shuler
>Assignee: Michael Shuler
>Priority: Minor
> Labels: bootcamp, lhf
> Fix For: 2.1.4, 2.0.14
>
> Attachments: 7712-hung-CliTest_system.log.gz, 7712-v2.txt,
> 7712-v3.txt, 7712_workaround.txt, CASSANDRA-7712_apache_cassandra_2.0.txt
>
>
> There are many unit test temporary files left behind after test runs. In the
> case of CI servers, I have seen >70,000 files accumulate in /tmp over a
> period of time. Each unit test should make an effort to remove its temporary
> files when the test is completed.
> My current unit test cleanup block:
> {noformat}
> # clean up after unit tests..
> rm -rf /tmp/140*-0 /tmp/CFWith* /tmp/Counter1* /tmp/DescriptorTest*
> /tmp/Keyspace1* \
> /tmp/KeyStreamingTransferTestSpace* /tmp/SSTableExportTest*
> /tmp/SSTableImportTest* \
> /tmp/Standard1* /tmp/Statistics.db* /tmp/StreamingTransferTest*
> /tmp/ValuesWithQuotes* \
> /tmp/cassandra* /tmp/jna-* /tmp/ks-cf-ib-1-* /tmp/lengthtest*
> /tmp/liblz4-java*.so /tmp/readtest* \
> /tmp/set_length_during_read_mode* /tmp/set_negative_length*
> /tmp/snappy-*.so
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365910#comment-14365910 ] Alex Liu commented on CASSANDRA-7410: - [~brandon.williams] Do you have time to review it? > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.14 > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8936) Remove IRequestSink, clean up IMessageSink use
[ https://issues.apache.org/jira/browse/CASSANDRA-8936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365912#comment-14365912 ] Jeremiah Jordan commented on CASSANDRA-8936: LGTM +1 > Remove IRequestSink, clean up IMessageSink use > -- > > Key: CASSANDRA-8936 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8936 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Trivial > Fix For: 3.0 > > Attachments: 8936-v2.txt, 8936.txt > > > Nothing's using IRequestSink anymore, and IMessageSink interface does not > ideally represent its current uses. > A tirival cleanup patch that deals with both issues attached. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365925#comment-14365925 ] Brandon Williams commented on CASSANDRA-7410: - Seems like waiting on CASSANDRA-8358 is the right move, at least for trunk, so we don't compound a problem. > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.14 > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: Clean up I*Sink testing hooks
Repository: cassandra
Updated Branches:
refs/heads/trunk 50ea0182e -> 642546aba
Clean up I*Sink testing hooks
patch by Aleksey Yeschenko; reviewed by Jeremiah Jordan for
CASSANDRA-8936
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/642546ab
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/642546ab
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/642546ab
Branch: refs/heads/trunk
Commit: 642546abac7527b640e228cdeb1993c53fc71582
Parents: 50ea018
Author: Aleksey Yeschenko
Authored: Tue Mar 17 13:10:21 2015 -0700
Committer: Aleksey Yeschenko
Committed: Tue Mar 17 13:10:21 2015 -0700
--
.../org/apache/cassandra/net/IMessageSink.java | 37 +++
.../apache/cassandra/net/MessagingService.java | 54 +-
.../apache/cassandra/service/StorageProxy.java | 32 ++
.../org/apache/cassandra/sink/IMessageSink.java | 42
.../org/apache/cassandra/sink/IRequestSink.java | 32 --
.../org/apache/cassandra/sink/SinkManager.java | 104 ---
.../apache/cassandra/repair/ValidatorTest.java | 31 +++---
.../apache/cassandra/service/RemoveTest.java| 3 +-
8 files changed, 88 insertions(+), 247 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/642546ab/src/java/org/apache/cassandra/net/IMessageSink.java
--
diff --git a/src/java/org/apache/cassandra/net/IMessageSink.java
b/src/java/org/apache/cassandra/net/IMessageSink.java
new file mode 100644
index 000..5150901
--- /dev/null
+++ b/src/java/org/apache/cassandra/net/IMessageSink.java
@@ -0,0 +1,37 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.net;
+
+import java.net.InetAddress;
+
+public interface IMessageSink
+{
+/**
+ * Allow or drop an outgoing message
+ *
+ * @return true if the message is allowed, false if it should be dropped
+ */
+boolean allowOutgoingMessage(MessageOut message, int id, InetAddress to);
+
+/**
+ * Allow or drop an incoming message
+ *
+ * @return true if the message is allowed, false if it should be dropped
+ */
+boolean allowIncomingMessage(MessageIn message, int id);
+}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/642546ab/src/java/org/apache/cassandra/net/MessagingService.java
--
diff --git a/src/java/org/apache/cassandra/net/MessagingService.java
b/src/java/org/apache/cassandra/net/MessagingService.java
index 1f952a3..fb699e4 100644
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@ -25,6 +25,7 @@ import java.nio.channels.ClosedChannelException;
import java.nio.channels.ServerSocketChannel;
import java.util.*;
import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.CopyOnWriteArraySet;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
@@ -60,7 +61,6 @@ import org.apache.cassandra.io.util.FileUtils;
import org.apache.cassandra.locator.ILatencySubscriber;
import org.apache.cassandra.metrics.ConnectionMetrics;
import org.apache.cassandra.metrics.DroppedMessageMetrics;
-import org.apache.cassandra.sink.SinkManager;
import org.apache.cassandra.repair.messages.RepairMessage;
import org.apache.cassandra.security.SSLFactory;
import org.apache.cassandra.service.*;
@@ -308,10 +308,24 @@ public final class MessagingService implements
MessagingServiceMBean
// protocol versions of the other nodes in the cluster
private final ConcurrentMap versions = new
NonBlockingHashMap();
+// message sinks are a testing hook
+private final Set messageSinks = new CopyOnWriteArraySet<>();
+
+public void addMessageSink(IMessageSink sink)
+{
+messageSinks.add(sink);
+}
+
+public void clearMessageSinks()
+{
+messageSinks.clear();
+}
+
private static class MSHandle
[jira] [Commented] (CASSANDRA-8670) Large columns + NIO memory pooling causes excessive direct memory usage
[
https://issues.apache.org/jira/browse/CASSANDRA-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366114#comment-14366114
]
Ariel Weisberg commented on CASSANDRA-8670:
---
I can confirm that compression determines whether you see an issue with memory
used by intracluster messaging.
The memory used by Netty shouldn't scale with cluster size the same way.
Still figuring out how to test for the issue when Netty is dominating direct
bytebuffer usage. I don't see a way to get Netty to report on it's memory usage
nor a way to get the allocations done by NIO tracked.
Monkey patching where art thou?
> Large columns + NIO memory pooling causes excessive direct memory usage
> ---
>
> Key: CASSANDRA-8670
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8670
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 3.0
>
>
> If you provide a large byte array to NIO and ask it to populate the byte
> array from a socket it will allocate a thread local byte buffer that is the
> size of the requested read no matter how large it is. Old IO wraps new IO for
> sockets (but not files) so old IO is effected as well.
> Even If you are using Buffered{Input | Output}Stream you can end up passing a
> large byte array to NIO. The byte array read method will pass the array to
> NIO directly if it is larger than the internal buffer.
> Passing large cells between nodes as part of intra-cluster messaging can
> cause the NIO pooled buffers to quickly reach a high watermark and stay
> there. This ends up costing 2x the largest cell size because there is a
> buffer for input and output since they are different threads. This is further
> multiplied by the number of nodes in the cluster - 1 since each has a
> dedicated thread pair with separate thread locals.
> Anecdotally it appears that the cost is doubled beyond that although it isn't
> clear why. Possibly the control connections or possibly there is some way in
> which multiple
> Need a workload in CI that tests the advertised limits of cells on a cluster.
> It would be reasonable to ratchet down the max direct memory for the test to
> trigger failures if a memory pooling issue is introduced. I don't think we
> need to test concurrently pulling in a lot of them, but it should at least
> work serially.
> The obvious fix to address this issue would be to read in smaller chunks when
> dealing with large values. I think small should still be relatively large (4
> megabytes) so that code that is reading from a disk can amortize the cost of
> a seek. It can be hard to tell what the underlying thing being read from is
> going to be in some of the contexts where we might choose to implement
> switching to reading chunks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6178) Consider allowing timestamp at the protocol level ... and deprecating server side timestamps
[ https://issues.apache.org/jira/browse/CASSANDRA-6178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366136#comment-14366136 ] Jon Haddad commented on CASSANDRA-6178: --- I would argue there's a valid use case for being able to disable client side timestamps and only use server side ones. > Consider allowing timestamp at the protocol level ... and deprecating server > side timestamps > > > Key: CASSANDRA-6178 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6178 > Project: Cassandra > Issue Type: Improvement >Reporter: Sylvain Lebresne >Assignee: Sylvain Lebresne > > Generating timestamps server side by default for CQL has been done for > convenience, so that end-user don't have to provide one with every query. > However, doing it server side has the downside that updates made sequentially > by one single client (thread) are no guaranteed to have sequentially > increasing timestamps. Unless a client thread is always pinned to one > specific server connection that is, but no good client driver out (that is, > including thrit driver) there does that because that's contradictory to > abstracting fault tolerance to the driver user (and goes again most sane load > balancing strategy). > Very concretely, this means that if you write a very trivial test program > that sequentially insert a value and then erase it (or overwrite it), then, > if you let CQL pick timestamp server side, the deletion might not erase the > just inserted value (because the delete might reach a different coordinator > than the insert and thus get a lower timestamp). From the user point of view, > this is a very confusing behavior, and understandably so: if timestamps are > optional, you'd hope that they are least respect the sequentiality of > operation from a single client thread. > Of course we do support client-side assigned timestamps so it's not like the > test above is not fixable. And you could argue that's it's not a bug per-se. > Still, it's a very confusing "default" behavior for something very simple, > which suggest it's not the best default. > You could also argue that inserting a value and deleting/overwriting right > away (in the same thread) is not something real program often do. And indeed, > it's likely that in practice server-side timestamps work fine for most real > application. Still, it's too easy to get counter-intuitive behavior with > server-side timestamps and I think we should consider moving away from them. > So what I'd suggest is that we push back the job of providing timestamp > client side. But to make it easy for the driver to generate it (rather than > the end user), we should allow providing said timestamp at the protocol level. > As a side note, letting the client provide the timestamp would also have the > advantage of making it easy for the driver to retry failed operations with > their initial timestamp, so that retries are truly idempotent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-6178) Consider allowing timestamp at the protocol level ... and deprecating server side timestamps
[ https://issues.apache.org/jira/browse/CASSANDRA-6178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366136#comment-14366136 ] Jon Haddad edited comment on CASSANDRA-6178 at 3/17/15 9:39 PM: I would argue there's a valid use case for being able to disable client side timestamps and only use server side ones, which would put me in the camp of being very against this issue. was (Author: rustyrazorblade): I would argue there's a valid use case for being able to disable client side timestamps and only use server side ones. > Consider allowing timestamp at the protocol level ... and deprecating server > side timestamps > > > Key: CASSANDRA-6178 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6178 > Project: Cassandra > Issue Type: Improvement >Reporter: Sylvain Lebresne >Assignee: Sylvain Lebresne > > Generating timestamps server side by default for CQL has been done for > convenience, so that end-user don't have to provide one with every query. > However, doing it server side has the downside that updates made sequentially > by one single client (thread) are no guaranteed to have sequentially > increasing timestamps. Unless a client thread is always pinned to one > specific server connection that is, but no good client driver out (that is, > including thrit driver) there does that because that's contradictory to > abstracting fault tolerance to the driver user (and goes again most sane load > balancing strategy). > Very concretely, this means that if you write a very trivial test program > that sequentially insert a value and then erase it (or overwrite it), then, > if you let CQL pick timestamp server side, the deletion might not erase the > just inserted value (because the delete might reach a different coordinator > than the insert and thus get a lower timestamp). From the user point of view, > this is a very confusing behavior, and understandably so: if timestamps are > optional, you'd hope that they are least respect the sequentiality of > operation from a single client thread. > Of course we do support client-side assigned timestamps so it's not like the > test above is not fixable. And you could argue that's it's not a bug per-se. > Still, it's a very confusing "default" behavior for something very simple, > which suggest it's not the best default. > You could also argue that inserting a value and deleting/overwriting right > away (in the same thread) is not something real program often do. And indeed, > it's likely that in practice server-side timestamps work fine for most real > application. Still, it's too easy to get counter-intuitive behavior with > server-side timestamps and I think we should consider moving away from them. > So what I'd suggest is that we push back the job of providing timestamp > client side. But to make it easy for the driver to generate it (rather than > the end user), we should allow providing said timestamp at the protocol level. > As a side note, letting the client provide the timestamp would also have the > advantage of making it easy for the driver to retry failed operations with > their initial timestamp, so that retries are truly idempotent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8940) Inconsistent select count and select distinct
[
https://issues.apache.org/jira/browse/CASSANDRA-8940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Frens Jan Rumph updated CASSANDRA-8940:
---
Environment: 2.1.2
Fix Version/s: (was: 2.1.2)
> Inconsistent select count and select distinct
> -
>
> Key: CASSANDRA-8940
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8940
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: 2.1.2
>Reporter: Frens Jan Rumph
>
> When performing {{select count( * ) from ...}} I expect the results to be
> consistent over multiple query executions if the table at hand is not written
> to / deleted from in the mean time. However, in my set-up it is not. The
> counts returned vary considerable (several percent). The same holds for
> {{select distinct partition-key-columns from ...}}.
> I have a table in a keyspace with replication_factor = 1 which is something
> like:
> {code}
> CREATE TABLE tbl (
> id frozen,
> bucket bigint,
> offset int,
> value double,
> PRIMARY KEY ((id, bucket), offset)
> )
> {code}
> The frozen udt is:
> {code}
> CREATE TYPE id_type (
> tags map
> );
> {code}
> The table contains around 35k rows (I'm not trying to be funny here ...). The
> consistency level for the queries was ONE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-6809) Compressed Commit Log
[ https://issues.apache.org/jira/browse/CASSANDRA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Boudreault updated CASSANDRA-6809: --- Tester: Alan Boudreault > Compressed Commit Log > - > > Key: CASSANDRA-6809 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6809 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Assignee: Branimir Lambov >Priority: Minor > Labels: docs-impacting, performance > Fix For: 3.0 > > Attachments: ComitLogStress.java, logtest.txt > > > It seems an unnecessary oversight that we don't compress the commit log. > Doing so should improve throughput, but some care will need to be taken to > ensure we use as much of a segment as possible. I propose decoupling the > writing of the records from the segments. Basically write into a (queue of) > DirectByteBuffer, and have the sync thread compress, say, ~64K chunks every X > MB written to the CL (where X is ordinarily CLS size), and then pack as many > of the compressed chunks into a CLS as possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8984) Introduce Transactional API for behaviours that can corrupt system state
Benedict created CASSANDRA-8984: --- Summary: Introduce Transactional API for behaviours that can corrupt system state Key: CASSANDRA-8984 URL: https://issues.apache.org/jira/browse/CASSANDRA-8984 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Benedict Fix For: 2.1.4 As a penultimate (and probably final for 2.1, if we agree to introduce it there) round of changes to the internals managing sstable writing, I've introduced a new API called "Transactional" that I hope will make it much easier to write correct behaviour. As things stand we conflate a lot of behaviours into methods like "close" - the recent changes unpicked some of these, but didn't go far enough. My proposal here introduces an interface designed to support four actions (on top of their normal function): * prepareToCommit * commit * abort * cleanup In normal operation, once we have finished constructing a state change we call prepareToCommit; once all such state changes are prepared, we call commit. If at any point everything fails, abort is called. In _either_ case, cleanup is called at the very last. These transactional objects are all AutoCloseable, with the behaviour being to rollback any changes unless commit has completed successfully. The changes are actually less invasive than it might sound, since we did recently introduce abort in some places, as well as have commit like methods. This simply formalises the behaviour, and makes it consistent between all objects that interact in this way. Much of the code change is boilerplate, such as moving an object into a try-declaration, although the change is still non-trivial. What it _does_ do is eliminate a _lot_ of special casing that we have had since 2.1 was released. The data tracker API changes and compaction leftover cleanups should finish the job with making this much easier to reason about, but this change I think is worthwhile considering for 2.1, since we've just overhauled this entire area (and not released these changes), and this change is essentially just the finishing touches, so the risk is minimal and the potential gains reasonably significant. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8966) SequentialWriter should be Ref counted
[ https://issues.apache.org/jira/browse/CASSANDRA-8966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366253#comment-14366253 ] Benedict commented on CASSANDRA-8966: - Another FYI, that I've left out the headline aspect of this ticket from the change I'm filing, so if you're still interested in making this addition you're welcome to have a crack at it! CASSANDRA-8984 refactors the APIs to make them safer, but does not introduce Ref counting. > SequentialWriter should be Ref counted > -- > > Key: CASSANDRA-8966 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8966 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict >Priority: Minor > Fix For: 3.0 > > > A LHF to introduce some more resource safety -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-8939) Stack overflow when reading data ingested through SSTableLoader
[
https://issues.apache.org/jira/browse/CASSANDRA-8939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benedict resolved CASSANDRA-8939.
-
Resolution: Duplicate
Fix Version/s: 2.1.4
> Stack overflow when reading data ingested through SSTableLoader
> ---
>
> Key: CASSANDRA-8939
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8939
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Single C* node
> Linux Mint 17.1, kernel 3.16.0-30-generic
> Oracle Java 7u75.
>Reporter: Piotr Kołaczkowski
>Assignee: Benedict
> Fix For: 2.1.4
>
> Attachments: 8939.txt
>
>
> I created an empty table:
> {noformat}
> CREATE TABLE test.kv (
> key int PRIMARY KEY,
> value text
> ) WITH bloom_filter_fp_chance = 0.01
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'min_threshold': '4', 'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32'}
> AND compression = {'sstable_compression':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> {noformat}
> Then I loaded some rows into it using CqlSSTableWriter and SSTableLoader
> (programmatically, doing it the same way as BulkLoader is doing it). The
> streaming finished with no errors.
> I can even read all the data back with cqlsh:
> {noformat}
> cqlsh> SELECT key, value FROM test.kv;
>
> 3405 | foo3405
> 5504 | foo5504
> 3476 | foo3476
> 2542 | foo2542
> 6931 | foo6931
> ---MORE---
> (1 rows)
> {noformat}
> However, filtering by token fails:
> {noformat}
> cqlsh> SELECT key, value FROM test.kv WHERE token(key) > 854443789258213092;
> OperationTimedOut: errors={}, last_host=127.0.0.1
> cqlsh>
> {noformat}
> Server log repors a StackOverflowException:
> {noformat}
> WARN 15:10:05 Uncaught exception on thread
> Thread[SharedPool-Worker-2,5,main]: {}
> java.lang.StackOverflowError: null
> at
> java.nio.charset.CharsetDecoder.implReplaceWith(CharsetDecoder.java:302)
> ~[na:1.7.0_75]
> at java.nio.charset.CharsetDecoder.replaceWith(CharsetDecoder.java:288)
> ~[na:1.7.0_75]
> at java.nio.charset.CharsetDecoder.(CharsetDecoder.java:203)
> ~[na:1.7.0_75]
> at java.nio.charset.CharsetDecoder.(CharsetDecoder.java:226)
> ~[na:1.7.0_75]
> at sun.nio.cs.UTF_8$Decoder.(UTF_8.java:84) ~[na:1.7.0_75]
> at sun.nio.cs.UTF_8$Decoder.(UTF_8.java:81) ~[na:1.7.0_75]
> at sun.nio.cs.UTF_8.newDecoder(UTF_8.java:68) ~[na:1.7.0_75]
> at
> org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:152)
> ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
> at
> org.apache.cassandra.serializers.AbstractTextSerializer.deserialize(AbstractTextSerializer.java:39)
> ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
> at
> org.apache.cassandra.serializers.AbstractTextSerializer.deserialize(AbstractTextSerializer.java:26)
> ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
> at
> org.apache.cassandra.db.marshal.AbstractType.getString(AbstractType.java:82)
> ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
> at
> org.apache.cassandra.cql3.ColumnIdentifier.(ColumnIdentifier.java:54)
> ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
> at
> org.apache.cassandra.db.composites.CompoundSparseCellNameType.idFor(CompoundSparseCellNameType.java:169)
> ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
> at
> org.apache.cassandra.db.composites.CompoundSparseCellNameType.makeWith(CompoundSparseCellNameType.java:177)
> ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
> at
> org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNameType.java:106)
> ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:397)
> ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:381)
> ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
> at
> org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)
> ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
> at
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52)
> ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
> at
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46)
> ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
> at
> com.go
[jira] [Commented] (CASSANDRA-8838) Resumable bootstrap streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-8838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366406#comment-14366406 ] Stefania commented on CASSANDRA-8838: - +1 on everything now, tests look great and pass without problems on my box. I cannot commit myself so I did not resolve the ticket just yet but it can be resolved and committed at any time. > Resumable bootstrap streaming > - > > Key: CASSANDRA-8838 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8838 > Project: Cassandra > Issue Type: Sub-task >Reporter: Yuki Morishita >Assignee: Yuki Morishita >Priority: Minor > Labels: dense-storage > Fix For: 3.0 > > > This allows the bootstrapping node not to be streamed already received data. > The bootstrapping node records received keyspace/ranges as one stream session > completes. When some sessions with other nodes fail, bootstrapping fails > also, though next time it re-bootstraps, already received keyspace/ranges are > skipped to be streamed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: use parameterized logging
Repository: cassandra
Updated Branches:
refs/heads/trunk 642546aba -> 4a4f83334
use parameterized logging
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4a4f8333
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4a4f8333
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4a4f8333
Branch: refs/heads/trunk
Commit: 4a4f83334265b1c2200e818b3439c146e77d46be
Parents: 642546a
Author: Dave Brosius
Authored: Tue Mar 17 21:10:23 2015 -0400
Committer: Dave Brosius
Committed: Tue Mar 17 21:10:23 2015 -0400
--
.../org/apache/cassandra/io/sstable/format/big/BigTableWriter.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4a4f8333/src/java/org/apache/cassandra/io/sstable/format/big/BigTableWriter.java
--
diff --git
a/src/java/org/apache/cassandra/io/sstable/format/big/BigTableWriter.java
b/src/java/org/apache/cassandra/io/sstable/format/big/BigTableWriter.java
index ea2549d..ec5c165 100644
--- a/src/java/org/apache/cassandra/io/sstable/format/big/BigTableWriter.java
+++ b/src/java/org/apache/cassandra/io/sstable/format/big/BigTableWriter.java
@@ -124,7 +124,7 @@ public class BigTableWriter extends SSTableWriter
first = lastWrittenKey;
if (logger.isTraceEnabled())
-logger.trace("wrote " + decoratedKey + " at " + dataEnd);
+logger.trace("wrote {} at {}", decoratedKey, dataEnd);
iwriter.append(decoratedKey, index, dataEnd);
dbuilder.addPotentialBoundary(dataEnd);
}
cassandra git commit: use long math for long results
Repository: cassandra
Updated Branches:
refs/heads/trunk 4a4f83334 -> 5a09483df
use long math for long results
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5a09483d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5a09483d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5a09483d
Branch: refs/heads/trunk
Commit: 5a09483df399b9e498e30624d0e3227fb60d7fa1
Parents: 4a4f833
Author: Dave Brosius
Authored: Tue Mar 17 21:16:49 2015 -0400
Committer: Dave Brosius
Committed: Tue Mar 17 21:16:49 2015 -0400
--
.../org/apache/cassandra/io/compress/CompressionMetadata.java| 4 ++--
src/java/org/apache/cassandra/utils/StreamingHistogram.java | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a09483d/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java
--
diff --git a/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java
b/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java
index 182cdd2..928541a 100644
--- a/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java
+++ b/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java
@@ -188,7 +188,7 @@ public class CompressionMetadata
{
try
{
-offsets.setLong(i * 8, input.readLong());
+offsets.setLong(i * 8L, input.readLong());
}
catch (EOFException e)
{
@@ -290,7 +290,7 @@ public class CompressionMetadata
{
if (count == maxCount)
{
-SafeMemory newOffsets = offsets.copy((maxCount *= 2L) * 8);
+SafeMemory newOffsets = offsets.copy((maxCount *= 2L) * 8L);
offsets.close();
offsets = newOffsets;
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a09483d/src/java/org/apache/cassandra/utils/StreamingHistogram.java
--
diff --git a/src/java/org/apache/cassandra/utils/StreamingHistogram.java
b/src/java/org/apache/cassandra/utils/StreamingHistogram.java
index 3f5a715..eb884be 100644
--- a/src/java/org/apache/cassandra/utils/StreamingHistogram.java
+++ b/src/java/org/apache/cassandra/utils/StreamingHistogram.java
@@ -201,7 +201,7 @@ public class StreamingHistogram
Map entries = histogram.getAsMap();
size += typeSizes.sizeof(entries.size());
// size of entries = size * (8(double) + 8(long))
-size += entries.size() * (8 + 8);
+size += entries.size() * (8L + 8L);
return size;
}
}
[jira] [Commented] (CASSANDRA-8984) Introduce Transactional API for behaviours that can corrupt system state
[ https://issues.apache.org/jira/browse/CASSANDRA-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366454#comment-14366454 ] Benedict commented on CASSANDRA-8984: - Patch available [here|https://github.com/belliottsmith/cassandra/commits/transactional-writers] [~krummas] [~JoshuaMcKenzie] WDYT? You've seen the ugliness of these codepaths more than anyone else: does this go some way towards sanitising them? The final cleanup will have to wait until CASSANDRA-8568 and CASSANDRA-7066, but I think this gets us close to the finish line. The main annoying thing about this patch is that we've always used close() to complete a change, and now this will rollback a change without a preceding commit, which means it's possible I've missed the introduction of a commit() somewhere if it isn't covered by a unit test (I've tried to search exhaustively for occurrences, but this is hard to be certain about). Hopefully that would become apparent very quickly, though, and I don't see a good alternative for safe rollback. > Introduce Transactional API for behaviours that can corrupt system state > > > Key: CASSANDRA-8984 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8984 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict >Assignee: Benedict > Fix For: 2.1.4 > > > As a penultimate (and probably final for 2.1, if we agree to introduce it > there) round of changes to the internals managing sstable writing, I've > introduced a new API called "Transactional" that I hope will make it much > easier to write correct behaviour. As things stand we conflate a lot of > behaviours into methods like "close" - the recent changes unpicked some of > these, but didn't go far enough. My proposal here introduces an interface > designed to support four actions (on top of their normal function): > * prepareToCommit > * commit > * abort > * cleanup > In normal operation, once we have finished constructing a state change we > call prepareToCommit; once all such state changes are prepared, we call > commit. If at any point everything fails, abort is called. In _either_ case, > cleanup is called at the very last. > These transactional objects are all AutoCloseable, with the behaviour being > to rollback any changes unless commit has completed successfully. > The changes are actually less invasive than it might sound, since we did > recently introduce abort in some places, as well as have commit like methods. > This simply formalises the behaviour, and makes it consistent between all > objects that interact in this way. Much of the code change is boilerplate, > such as moving an object into a try-declaration, although the change is still > non-trivial. What it _does_ do is eliminate a _lot_ of special casing that we > have had since 2.1 was released. The data tracker API changes and compaction > leftover cleanups should finish the job with making this much easier to > reason about, but this change I think is worthwhile considering for 2.1, > since we've just overhauled this entire area (and not released these > changes), and this change is essentially just the finishing touches, so the > risk is minimal and the potential gains reasonably significant. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-8981) IndexSummaryManagerTest.testCompactionsRace intermittently timing out on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-8981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benedict reassigned CASSANDRA-8981:
---
Assignee: Benedict
> IndexSummaryManagerTest.testCompactionsRace intermittently timing out on trunk
> --
>
> Key: CASSANDRA-8981
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8981
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Joshua McKenzie
>Assignee: Benedict
>Priority: Minor
> Fix For: 3.0
>
>
> Keep running it repeatedly w/showoutput="yes" in build.xml on junit and
> you'll see it time out with:
> {noformat}
> [junit] WARN 17:02:56 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> [junit] WARN 17:02:56 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> [junit] WARN 17:02:57 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> [junit] WARN 17:02:57 Unable to cancel in-progress compactions for
> StandardRace. Perhaps there is an unusually large row in progress somewhere,
> or the system is simply overloaded.
> {noformat}
> I originally thought this was a Windows specific problem (CASSANDRA-8962) but
> can reproduce on linux by just repeatedly running the test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8568) Impose new API on data tracker modifications that makes correct usage obvious and imposes safety
[ https://issues.apache.org/jira/browse/CASSANDRA-8568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-8568: Fix Version/s: 3.0 > Impose new API on data tracker modifications that makes correct usage obvious > and imposes safety > > > Key: CASSANDRA-8568 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8568 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Assignee: Benedict > Fix For: 3.0 > > > DataTracker has become a bit of a quagmire, and not at all obvious to > interface with, with many subtly different modifiers. I suspect it is still > subtly broken, especially around error recovery. > I propose piggy-backing on CASSANDRA-7705 to offer RAII (and GC-enforced, for > those situations where a try/finally block isn't possible) objects that have > transactional behaviour, and with few simple declarative methods that can be > composed simply to provide all of the functionality we currently need. > See CASSANDRA-8399 for context -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8568) Impose new API on data tracker modifications that makes correct usage obvious and imposes safety
[ https://issues.apache.org/jira/browse/CASSANDRA-8568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366487#comment-14366487 ] Benedict commented on CASSANDRA-8568: - Improvements to Byteman testing can be a follow up ticket, which I'll get to in a week or two. It's ready for review as stands, although I would prefer CASSANDRA-8984 take priority. > Impose new API on data tracker modifications that makes correct usage obvious > and imposes safety > > > Key: CASSANDRA-8568 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8568 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Assignee: Benedict > Fix For: 3.0 > > > DataTracker has become a bit of a quagmire, and not at all obvious to > interface with, with many subtly different modifiers. I suspect it is still > subtly broken, especially around error recovery. > I propose piggy-backing on CASSANDRA-7705 to offer RAII (and GC-enforced, for > those situations where a try/finally block isn't possible) objects that have > transactional behaviour, and with few simple declarative methods that can be > composed simply to provide all of the functionality we currently need. > See CASSANDRA-8399 for context -- This message was sent by Atlassian JIRA (v6.3.4#6332)
