[jira] [Commented] (CASSANDRA-13104) Use StandardCharset instead of Charset.forName()
[
https://issues.apache.org/jira/browse/CASSANDRA-13104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803725#comment-15803725
]
Stefania commented on CASSANDRA-13104:
--
LGTM, thanks for the patch!
> Use StandardCharset instead of Charset.forName()
>
>
> Key: CASSANDRA-13104
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13104
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 4.0
>
>
> Trivial change replacing {{Charset.forName()}} with {{StandardCharset}}
> instances.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-13104) Use StandardCharset instead of Charset.forName()
[
https://issues.apache.org/jira/browse/CASSANDRA-13104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefania updated CASSANDRA-13104:
-
Status: Ready to Commit (was: Patch Available)
> Use StandardCharset instead of Charset.forName()
>
>
> Key: CASSANDRA-13104
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13104
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 4.0
>
>
> Trivial change replacing {{Charset.forName()}} with {{StandardCharset}}
> instances.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-13104) Use StandardCharset instead of Charset.forName()
[
https://issues.apache.org/jira/browse/CASSANDRA-13104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefania updated CASSANDRA-13104:
-
Reviewer: Stefania (was: Stefania Alborghetti)
> Use StandardCharset instead of Charset.forName()
>
>
> Key: CASSANDRA-13104
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13104
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 4.0
>
>
> Trivial change replacing {{Charset.forName()}} with {{StandardCharset}}
> instances.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13090) Coalescing strategy sleep too much
[
https://issues.apache.org/jira/browse/CASSANDRA-13090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803318#comment-15803318
]
Nate McCall commented on CASSANDRA-13090:
-
Since we are dropping some settings into {{cassandra.yaml}} for coalescing, it
would be great if we can add some additional details in the comments about
common variations (when you would do what and why). [~aweisberg] A couple of
tidbits from your initial blog post would be cool.
> Coalescing strategy sleep too much
> --
>
> Key: CASSANDRA-13090
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13090
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
>Reporter: Corentin Chary
> Fix For: 3.x
>
> Attachments: 0001-Fix-wait-time-coalescing-CASSANDRA-13090.patch
>
>
> With the current code maybeSleep is called even if we managed to take
> maxItems out of the backlog. In this case we should really avoid sleeping
> because it means that backlog is building up.
> I'll send a patch shortly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-13105) Multi-index query incorrectly returns 0 rows
[
https://issues.apache.org/jira/browse/CASSANDRA-13105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nate McCall updated CASSANDRA-13105:
Component/s: sasi
> Multi-index query incorrectly returns 0 rows
>
>
> Key: CASSANDRA-13105
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13105
> Project: Cassandra
> Issue Type: Bug
> Components: sasi
> Environment: 3.9.0 on linux & osx
>Reporter: Voytek Jarnot
>
> Setup:
> {code}
> create table test1(id1 text PRIMARY KEY, val1 text, val2 text);
> create custom index test1_idx_val1 on test1(val1) using
> 'org.apache.cassandra.index.sasi.SASIIndex';
> create custom index test1_idx_val2 on test1(val2) using
> 'org.apache.cassandra.index.sasi.SASIIndex';
> insert into test1(id1, val1, val2) values ('1', '1val1', '1val2');
> insert into test1(id1, val1, val2) values ('2', '~~', '2val2');
> {code}
> Queries:
> {code}
> (1) select * from test1 where val1 = '~~';
> (2) select * from test1 where val1 < '~~' allow filtering;
> (3) select * from test1 where val2 = '1val2';
> (4) select * from test1 where val1 < '~~' and val2 = '1val2' allow filtering;
> {code}
> 1, 2, and 3 all work correctly. 4 does not work.
> 2, 3, and 4 should return the same row (id1='1'); 2 and 3 do, 4 returns 0
> rows.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
[
https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803137#comment-15803137
]
Jon Haddad commented on CASSANDRA-12979:
Updated NEWS.txt and CHANGES.txt on all 3 patches. Changed the log error to
warn. Not sure about the spacing nit though - there's 1 line above the single
line comment, which is consistent with the other comments in the file. Maybe
the patch had some weird formatting?
> checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread
> scope
> ---
>
> Key: CASSANDRA-12979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12979
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jon Haddad
>Assignee: Jon Haddad
> Labels: compaction
> Fix For: 2.2.9, 3.0.11, 4.0, 3.x
>
> Attachments: 2.2.patch, 3.0.patch, trunk-12979.patch
>
>
> If a compaction occurs that looks like it'll take up more space than
> remaining disk available, the compaction manager attempts to reduce the scope
> of the compaction by calling {{reduceScopeForLimitedSpace()}} repeatedly.
> Unfortunately, the while loop passes the {{estimatedWriteSize}} calculated
> from the original call to {{hasAvailableDiskSpace}}, so the comparisons that
> are done will always be against the size of the original compaction, rather
> than the reduced scope one.
> Full method below:
> {code}
> protected void checkAvailableDiskSpace(long estimatedSSTables, long
> expectedWriteSize)
> {
> if(!cfs.isCompactionDiskSpaceCheckEnabled() && compactionType ==
> OperationType.COMPACTION)
> {
> logger.info("Compaction space check is disabled");
> return;
> }
> while (!getDirectories().hasAvailableDiskSpace(estimatedSSTables,
> expectedWriteSize))
> {
> if (!reduceScopeForLimitedSpace())
> throw new RuntimeException(String.format("Not enough space
> for compaction, estimated sstables = %d, expected write size = %d",
> estimatedSSTables, expectedWriteSize));
>
> }
> }
> {code}
> I'm proposing to recalculate the {{estimatedSSTables}} and
> {{expectedWriteSize}} after each iteration of {{reduceScopeForLimitedSpace}}.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
[
https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-12979:
---
Attachment: (was: 3.0.patch)
> checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread
> scope
> ---
>
> Key: CASSANDRA-12979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12979
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jon Haddad
>Assignee: Jon Haddad
> Labels: compaction
> Fix For: 2.2.9, 3.0.11, 4.0, 3.x
>
> Attachments: 2.2.patch, 3.0.patch, trunk-12979.patch
>
>
> If a compaction occurs that looks like it'll take up more space than
> remaining disk available, the compaction manager attempts to reduce the scope
> of the compaction by calling {{reduceScopeForLimitedSpace()}} repeatedly.
> Unfortunately, the while loop passes the {{estimatedWriteSize}} calculated
> from the original call to {{hasAvailableDiskSpace}}, so the comparisons that
> are done will always be against the size of the original compaction, rather
> than the reduced scope one.
> Full method below:
> {code}
> protected void checkAvailableDiskSpace(long estimatedSSTables, long
> expectedWriteSize)
> {
> if(!cfs.isCompactionDiskSpaceCheckEnabled() && compactionType ==
> OperationType.COMPACTION)
> {
> logger.info("Compaction space check is disabled");
> return;
> }
> while (!getDirectories().hasAvailableDiskSpace(estimatedSSTables,
> expectedWriteSize))
> {
> if (!reduceScopeForLimitedSpace())
> throw new RuntimeException(String.format("Not enough space
> for compaction, estimated sstables = %d, expected write size = %d",
> estimatedSSTables, expectedWriteSize));
>
> }
> }
> {code}
> I'm proposing to recalculate the {{estimatedSSTables}} and
> {{expectedWriteSize}} after each iteration of {{reduceScopeForLimitedSpace}}.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
[
https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-12979:
---
Attachment: 3.0.patch
> checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread
> scope
> ---
>
> Key: CASSANDRA-12979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12979
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jon Haddad
>Assignee: Jon Haddad
> Labels: compaction
> Fix For: 2.2.9, 3.0.11, 4.0, 3.x
>
> Attachments: 2.2.patch, 3.0.patch, trunk-12979.patch
>
>
> If a compaction occurs that looks like it'll take up more space than
> remaining disk available, the compaction manager attempts to reduce the scope
> of the compaction by calling {{reduceScopeForLimitedSpace()}} repeatedly.
> Unfortunately, the while loop passes the {{estimatedWriteSize}} calculated
> from the original call to {{hasAvailableDiskSpace}}, so the comparisons that
> are done will always be against the size of the original compaction, rather
> than the reduced scope one.
> Full method below:
> {code}
> protected void checkAvailableDiskSpace(long estimatedSSTables, long
> expectedWriteSize)
> {
> if(!cfs.isCompactionDiskSpaceCheckEnabled() && compactionType ==
> OperationType.COMPACTION)
> {
> logger.info("Compaction space check is disabled");
> return;
> }
> while (!getDirectories().hasAvailableDiskSpace(estimatedSSTables,
> expectedWriteSize))
> {
> if (!reduceScopeForLimitedSpace())
> throw new RuntimeException(String.format("Not enough space
> for compaction, estimated sstables = %d, expected write size = %d",
> estimatedSSTables, expectedWriteSize));
>
> }
> }
> {code}
> I'm proposing to recalculate the {{estimatedSSTables}} and
> {{expectedWriteSize}} after each iteration of {{reduceScopeForLimitedSpace}}.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
[
https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-12979:
---
Attachment: (was:
30-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch)
> checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread
> scope
> ---
>
> Key: CASSANDRA-12979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12979
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jon Haddad
>Assignee: Jon Haddad
> Labels: compaction
> Fix For: 2.2.9, 3.0.11, 4.0, 3.x
>
> Attachments: 2.2.patch, 3.0.patch, trunk-12979.patch
>
>
> If a compaction occurs that looks like it'll take up more space than
> remaining disk available, the compaction manager attempts to reduce the scope
> of the compaction by calling {{reduceScopeForLimitedSpace()}} repeatedly.
> Unfortunately, the while loop passes the {{estimatedWriteSize}} calculated
> from the original call to {{hasAvailableDiskSpace}}, so the comparisons that
> are done will always be against the size of the original compaction, rather
> than the reduced scope one.
> Full method below:
> {code}
> protected void checkAvailableDiskSpace(long estimatedSSTables, long
> expectedWriteSize)
> {
> if(!cfs.isCompactionDiskSpaceCheckEnabled() && compactionType ==
> OperationType.COMPACTION)
> {
> logger.info("Compaction space check is disabled");
> return;
> }
> while (!getDirectories().hasAvailableDiskSpace(estimatedSSTables,
> expectedWriteSize))
> {
> if (!reduceScopeForLimitedSpace())
> throw new RuntimeException(String.format("Not enough space
> for compaction, estimated sstables = %d, expected write size = %d",
> estimatedSSTables, expectedWriteSize));
>
> }
> }
> {code}
> I'm proposing to recalculate the {{estimatedSSTables}} and
> {{expectedWriteSize}} after each iteration of {{reduceScopeForLimitedSpace}}.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
[
https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-12979:
---
Attachment: (was:
2.2-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch)
> checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread
> scope
> ---
>
> Key: CASSANDRA-12979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12979
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jon Haddad
>Assignee: Jon Haddad
> Labels: compaction
> Fix For: 2.2.9, 3.0.11, 4.0, 3.x
>
> Attachments: 2.2.patch, 3.0.patch, trunk-12979.patch
>
>
> If a compaction occurs that looks like it'll take up more space than
> remaining disk available, the compaction manager attempts to reduce the scope
> of the compaction by calling {{reduceScopeForLimitedSpace()}} repeatedly.
> Unfortunately, the while loop passes the {{estimatedWriteSize}} calculated
> from the original call to {{hasAvailableDiskSpace}}, so the comparisons that
> are done will always be against the size of the original compaction, rather
> than the reduced scope one.
> Full method below:
> {code}
> protected void checkAvailableDiskSpace(long estimatedSSTables, long
> expectedWriteSize)
> {
> if(!cfs.isCompactionDiskSpaceCheckEnabled() && compactionType ==
> OperationType.COMPACTION)
> {
> logger.info("Compaction space check is disabled");
> return;
> }
> while (!getDirectories().hasAvailableDiskSpace(estimatedSSTables,
> expectedWriteSize))
> {
> if (!reduceScopeForLimitedSpace())
> throw new RuntimeException(String.format("Not enough space
> for compaction, estimated sstables = %d, expected write size = %d",
> estimatedSSTables, expectedWriteSize));
>
> }
> }
> {code}
> I'm proposing to recalculate the {{estimatedSSTables}} and
> {{expectedWriteSize}} after each iteration of {{reduceScopeForLimitedSpace}}.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
[
https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-12979:
---
Attachment: 2.2.patch
> checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread
> scope
> ---
>
> Key: CASSANDRA-12979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12979
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jon Haddad
>Assignee: Jon Haddad
> Labels: compaction
> Fix For: 2.2.9, 3.0.11, 4.0, 3.x
>
> Attachments: 2.2.patch, 3.0.patch, trunk-12979.patch
>
>
> If a compaction occurs that looks like it'll take up more space than
> remaining disk available, the compaction manager attempts to reduce the scope
> of the compaction by calling {{reduceScopeForLimitedSpace()}} repeatedly.
> Unfortunately, the while loop passes the {{estimatedWriteSize}} calculated
> from the original call to {{hasAvailableDiskSpace}}, so the comparisons that
> are done will always be against the size of the original compaction, rather
> than the reduced scope one.
> Full method below:
> {code}
> protected void checkAvailableDiskSpace(long estimatedSSTables, long
> expectedWriteSize)
> {
> if(!cfs.isCompactionDiskSpaceCheckEnabled() && compactionType ==
> OperationType.COMPACTION)
> {
> logger.info("Compaction space check is disabled");
> return;
> }
> while (!getDirectories().hasAvailableDiskSpace(estimatedSSTables,
> expectedWriteSize))
> {
> if (!reduceScopeForLimitedSpace())
> throw new RuntimeException(String.format("Not enough space
> for compaction, estimated sstables = %d, expected write size = %d",
> estimatedSSTables, expectedWriteSize));
>
> }
> }
> {code}
> I'm proposing to recalculate the {{estimatedSSTables}} and
> {{expectedWriteSize}} after each iteration of {{reduceScopeForLimitedSpace}}.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
[
https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-12979:
---
Attachment: (was:
trunk-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch)
> checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread
> scope
> ---
>
> Key: CASSANDRA-12979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12979
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jon Haddad
>Assignee: Jon Haddad
> Labels: compaction
> Fix For: 2.2.9, 3.0.11, 4.0, 3.x
>
> Attachments:
> 2.2-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch, 3.0.patch,
> 30-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
> trunk-12979.patch
>
>
> If a compaction occurs that looks like it'll take up more space than
> remaining disk available, the compaction manager attempts to reduce the scope
> of the compaction by calling {{reduceScopeForLimitedSpace()}} repeatedly.
> Unfortunately, the while loop passes the {{estimatedWriteSize}} calculated
> from the original call to {{hasAvailableDiskSpace}}, so the comparisons that
> are done will always be against the size of the original compaction, rather
> than the reduced scope one.
> Full method below:
> {code}
> protected void checkAvailableDiskSpace(long estimatedSSTables, long
> expectedWriteSize)
> {
> if(!cfs.isCompactionDiskSpaceCheckEnabled() && compactionType ==
> OperationType.COMPACTION)
> {
> logger.info("Compaction space check is disabled");
> return;
> }
> while (!getDirectories().hasAvailableDiskSpace(estimatedSSTables,
> expectedWriteSize))
> {
> if (!reduceScopeForLimitedSpace())
> throw new RuntimeException(String.format("Not enough space
> for compaction, estimated sstables = %d, expected write size = %d",
> estimatedSSTables, expectedWriteSize));
>
> }
> }
> {code}
> I'm proposing to recalculate the {{estimatedSSTables}} and
> {{expectedWriteSize}} after each iteration of {{reduceScopeForLimitedSpace}}.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
[
https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-12979:
---
Attachment: 3.0.patch
> checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread
> scope
> ---
>
> Key: CASSANDRA-12979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12979
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jon Haddad
>Assignee: Jon Haddad
> Labels: compaction
> Fix For: 2.2.9, 3.0.11, 4.0, 3.x
>
> Attachments:
> 2.2-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch, 3.0.patch,
> 30-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
> trunk-12979.patch,
> trunk-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch
>
>
> If a compaction occurs that looks like it'll take up more space than
> remaining disk available, the compaction manager attempts to reduce the scope
> of the compaction by calling {{reduceScopeForLimitedSpace()}} repeatedly.
> Unfortunately, the while loop passes the {{estimatedWriteSize}} calculated
> from the original call to {{hasAvailableDiskSpace}}, so the comparisons that
> are done will always be against the size of the original compaction, rather
> than the reduced scope one.
> Full method below:
> {code}
> protected void checkAvailableDiskSpace(long estimatedSSTables, long
> expectedWriteSize)
> {
> if(!cfs.isCompactionDiskSpaceCheckEnabled() && compactionType ==
> OperationType.COMPACTION)
> {
> logger.info("Compaction space check is disabled");
> return;
> }
> while (!getDirectories().hasAvailableDiskSpace(estimatedSSTables,
> expectedWriteSize))
> {
> if (!reduceScopeForLimitedSpace())
> throw new RuntimeException(String.format("Not enough space
> for compaction, estimated sstables = %d, expected write size = %d",
> estimatedSSTables, expectedWriteSize));
>
> }
> }
> {code}
> I'm proposing to recalculate the {{estimatedSSTables}} and
> {{expectedWriteSize}} after each iteration of {{reduceScopeForLimitedSpace}}.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13106) Unnecessary assertion
[
https://issues.apache.org/jira/browse/CASSANDRA-13106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803106#comment-15803106
]
Simon Zhou commented on CASSANDRA-13106:
Ah, got your point! Just want to fix the issue by opening this ticket.
> Unnecessary assertion
> -
>
> Key: CASSANDRA-13106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13106
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Simon Zhou
>Assignee: Simon Zhou
>Priority: Minor
> Attachments: CASSANDRA-13106.patch
>
>
> We had over 70 thousand sstables and it's slow to bootstrap new node, even
> though the CPU utilization for main thread of Cassandra was nearly 100%. So
> we took a few stack traces and found that the main thread were busy running
> this line in Tracker.java:
> {code}
> assert Iterables.all(removed, remove);
> {code}
> Not exactly sure whether this line causes CPU utilization/bootstrapping
> issue, but this line is redundant because the Predict we pass in is
> Predicates.alwaysTrue(), which means the assertion always
> returns true. So I propose to remove that line.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-13106) Unnecessary assertion
[
https://issues.apache.org/jira/browse/CASSANDRA-13106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803078#comment-15803078
]
Dave Brosius edited comment on CASSANDRA-13106 at 1/6/17 1:06 AM:
--
if you remove the -ea flag, this code will not be executed, and thus this issue
will disappear.
was (Author: dbrosius):
if you remove the -ea flag, this issue will disappear.
> Unnecessary assertion
> -
>
> Key: CASSANDRA-13106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13106
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Simon Zhou
>Assignee: Simon Zhou
>Priority: Minor
> Attachments: CASSANDRA-13106.patch
>
>
> We had over 70 thousand sstables and it's slow to bootstrap new node, even
> though the CPU utilization for main thread of Cassandra was nearly 100%. So
> we took a few stack traces and found that the main thread were busy running
> this line in Tracker.java:
> {code}
> assert Iterables.all(removed, remove);
> {code}
> Not exactly sure whether this line causes CPU utilization/bootstrapping
> issue, but this line is redundant because the Predict we pass in is
> Predicates.alwaysTrue(), which means the assertion always
> returns true. So I propose to remove that line.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13106) Unnecessary assertion
[
https://issues.apache.org/jira/browse/CASSANDRA-13106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803078#comment-15803078
]
Dave Brosius commented on CASSANDRA-13106:
--
if you remove the -ea flag, this issue will disappear.
> Unnecessary assertion
> -
>
> Key: CASSANDRA-13106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13106
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Simon Zhou
>Assignee: Simon Zhou
>Priority: Minor
> Attachments: CASSANDRA-13106.patch
>
>
> We had over 70 thousand sstables and it's slow to bootstrap new node, even
> though the CPU utilization for main thread of Cassandra was nearly 100%. So
> we took a few stack traces and found that the main thread were busy running
> this line in Tracker.java:
> {code}
> assert Iterables.all(removed, remove);
> {code}
> Not exactly sure whether this line causes CPU utilization/bootstrapping
> issue, but this line is redundant because the Predict we pass in is
> Predicates.alwaysTrue(), which means the assertion always
> returns true. So I propose to remove that line.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
[
https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-12979:
---
Attachment: trunk-12979.patch
> checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread
> scope
> ---
>
> Key: CASSANDRA-12979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12979
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jon Haddad
>Assignee: Jon Haddad
> Labels: compaction
> Fix For: 2.2.9, 3.0.11, 4.0, 3.x
>
> Attachments:
> 2.2-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
> 30-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
> trunk-12979.patch,
> trunk-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch
>
>
> If a compaction occurs that looks like it'll take up more space than
> remaining disk available, the compaction manager attempts to reduce the scope
> of the compaction by calling {{reduceScopeForLimitedSpace()}} repeatedly.
> Unfortunately, the while loop passes the {{estimatedWriteSize}} calculated
> from the original call to {{hasAvailableDiskSpace}}, so the comparisons that
> are done will always be against the size of the original compaction, rather
> than the reduced scope one.
> Full method below:
> {code}
> protected void checkAvailableDiskSpace(long estimatedSSTables, long
> expectedWriteSize)
> {
> if(!cfs.isCompactionDiskSpaceCheckEnabled() && compactionType ==
> OperationType.COMPACTION)
> {
> logger.info("Compaction space check is disabled");
> return;
> }
> while (!getDirectories().hasAvailableDiskSpace(estimatedSSTables,
> expectedWriteSize))
> {
> if (!reduceScopeForLimitedSpace())
> throw new RuntimeException(String.format("Not enough space
> for compaction, estimated sstables = %d, expected write size = %d",
> estimatedSSTables, expectedWriteSize));
>
> }
> }
> {code}
> I'm proposing to recalculate the {{estimatedSSTables}} and
> {{expectedWriteSize}} after each iteration of {{reduceScopeForLimitedSpace}}.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
[
https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-12979:
---
Attachment: (was: trunk-12979.patch)
> checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread
> scope
> ---
>
> Key: CASSANDRA-12979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12979
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jon Haddad
>Assignee: Jon Haddad
> Labels: compaction
> Fix For: 2.2.9, 3.0.11, 4.0, 3.x
>
> Attachments:
> 2.2-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
> 30-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
> trunk-12979.patch,
> trunk-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch
>
>
> If a compaction occurs that looks like it'll take up more space than
> remaining disk available, the compaction manager attempts to reduce the scope
> of the compaction by calling {{reduceScopeForLimitedSpace()}} repeatedly.
> Unfortunately, the while loop passes the {{estimatedWriteSize}} calculated
> from the original call to {{hasAvailableDiskSpace}}, so the comparisons that
> are done will always be against the size of the original compaction, rather
> than the reduced scope one.
> Full method below:
> {code}
> protected void checkAvailableDiskSpace(long estimatedSSTables, long
> expectedWriteSize)
> {
> if(!cfs.isCompactionDiskSpaceCheckEnabled() && compactionType ==
> OperationType.COMPACTION)
> {
> logger.info("Compaction space check is disabled");
> return;
> }
> while (!getDirectories().hasAvailableDiskSpace(estimatedSSTables,
> expectedWriteSize))
> {
> if (!reduceScopeForLimitedSpace())
> throw new RuntimeException(String.format("Not enough space
> for compaction, estimated sstables = %d, expected write size = %d",
> estimatedSSTables, expectedWriteSize));
>
> }
> }
> {code}
> I'm proposing to recalculate the {{estimatedSSTables}} and
> {{expectedWriteSize}} after each iteration of {{reduceScopeForLimitedSpace}}.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13106) Unnecessary assertion
[
https://issues.apache.org/jira/browse/CASSANDRA-13106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803059#comment-15803059
]
Simon Zhou commented on CASSANDRA-13106:
Thanks [~dbrosius]. Yes, I do have -ea flag but doesn't that only cause ~5%
performance degradation?
> Unnecessary assertion
> -
>
> Key: CASSANDRA-13106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13106
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Simon Zhou
>Assignee: Simon Zhou
>Priority: Minor
> Attachments: CASSANDRA-13106.patch
>
>
> We had over 70 thousand sstables and it's slow to bootstrap new node, even
> though the CPU utilization for main thread of Cassandra was nearly 100%. So
> we took a few stack traces and found that the main thread were busy running
> this line in Tracker.java:
> {code}
> assert Iterables.all(removed, remove);
> {code}
> Not exactly sure whether this line causes CPU utilization/bootstrapping
> issue, but this line is redundant because the Predict we pass in is
> Predicates.alwaysTrue(), which means the assertion always
> returns true. So I propose to remove that line.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13106) Unnecessary assertion
[
https://issues.apache.org/jira/browse/CASSANDRA-13106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803045#comment-15803045
]
Dave Brosius commented on CASSANDRA-13106:
--
are you passing the -ea flag to the java process when running cassandra? just
remove it.
> Unnecessary assertion
> -
>
> Key: CASSANDRA-13106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13106
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Simon Zhou
>Assignee: Simon Zhou
>Priority: Minor
> Attachments: CASSANDRA-13106.patch
>
>
> We had over 70 thousand sstables and it's slow to bootstrap new node, even
> though the CPU utilization for main thread of Cassandra was nearly 100%. So
> we took a few stack traces and found that the main thread were busy running
> this line in Tracker.java:
> {code}
> assert Iterables.all(removed, remove);
> {code}
> Not exactly sure whether this line causes CPU utilization/bootstrapping
> issue, but this line is redundant because the Predict we pass in is
> Predicates.alwaysTrue(), which means the assertion always
> returns true. So I propose to remove that line.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
[
https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-12979:
---
Attachment: trunk-12979.patch
> checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread
> scope
> ---
>
> Key: CASSANDRA-12979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12979
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jon Haddad
>Assignee: Jon Haddad
> Labels: compaction
> Fix For: 2.2.9, 3.0.11, 4.0, 3.x
>
> Attachments:
> 2.2-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
> 30-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
> trunk-12979.patch,
> trunk-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch
>
>
> If a compaction occurs that looks like it'll take up more space than
> remaining disk available, the compaction manager attempts to reduce the scope
> of the compaction by calling {{reduceScopeForLimitedSpace()}} repeatedly.
> Unfortunately, the while loop passes the {{estimatedWriteSize}} calculated
> from the original call to {{hasAvailableDiskSpace}}, so the comparisons that
> are done will always be against the size of the original compaction, rather
> than the reduced scope one.
> Full method below:
> {code}
> protected void checkAvailableDiskSpace(long estimatedSSTables, long
> expectedWriteSize)
> {
> if(!cfs.isCompactionDiskSpaceCheckEnabled() && compactionType ==
> OperationType.COMPACTION)
> {
> logger.info("Compaction space check is disabled");
> return;
> }
> while (!getDirectories().hasAvailableDiskSpace(estimatedSSTables,
> expectedWriteSize))
> {
> if (!reduceScopeForLimitedSpace())
> throw new RuntimeException(String.format("Not enough space
> for compaction, estimated sstables = %d, expected write size = %d",
> estimatedSSTables, expectedWriteSize));
>
> }
> }
> {code}
> I'm proposing to recalculate the {{estimatedSSTables}} and
> {{expectedWriteSize}} after each iteration of {{reduceScopeForLimitedSpace}}.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
[
https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803036#comment-15803036
]
Nate McCall commented on CASSANDRA-12979:
-
[~rustyrazorblade] Also, let's add a line to NEWS.txt since this is different
(better, but different) runtime behavior.
> checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread
> scope
> ---
>
> Key: CASSANDRA-12979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12979
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jon Haddad
>Assignee: Jon Haddad
> Labels: compaction
> Fix For: 2.2.9, 3.0.11, 4.0, 3.x
>
> Attachments:
> 2.2-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
> 30-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
> trunk-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch
>
>
> If a compaction occurs that looks like it'll take up more space than
> remaining disk available, the compaction manager attempts to reduce the scope
> of the compaction by calling {{reduceScopeForLimitedSpace()}} repeatedly.
> Unfortunately, the while loop passes the {{estimatedWriteSize}} calculated
> from the original call to {{hasAvailableDiskSpace}}, so the comparisons that
> are done will always be against the size of the original compaction, rather
> than the reduced scope one.
> Full method below:
> {code}
> protected void checkAvailableDiskSpace(long estimatedSSTables, long
> expectedWriteSize)
> {
> if(!cfs.isCompactionDiskSpaceCheckEnabled() && compactionType ==
> OperationType.COMPACTION)
> {
> logger.info("Compaction space check is disabled");
> return;
> }
> while (!getDirectories().hasAvailableDiskSpace(estimatedSSTables,
> expectedWriteSize))
> {
> if (!reduceScopeForLimitedSpace())
> throw new RuntimeException(String.format("Not enough space
> for compaction, estimated sstables = %d, expected write size = %d",
> estimatedSSTables, expectedWriteSize));
>
> }
> }
> {code}
> I'm proposing to recalculate the {{estimatedSSTables}} and
> {{expectedWriteSize}} after each iteration of {{reduceScopeForLimitedSpace}}.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13106) Unnecessary assertion
[
https://issues.apache.org/jira/browse/CASSANDRA-13106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802990#comment-15802990
]
Simon Zhou commented on CASSANDRA-13106:
[~benedict], could you take a look at this one line patch?
> Unnecessary assertion
> -
>
> Key: CASSANDRA-13106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13106
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Simon Zhou
>Assignee: Simon Zhou
>Priority: Minor
> Attachments: CASSANDRA-13106.patch
>
>
> We had over 70 thousand sstables and it's slow to bootstrap new node, even
> though the CPU utilization for main thread of Cassandra was nearly 100%. So
> we took a few stack traces and found that the main thread were busy running
> this line in Tracker.java:
> {code}
> assert Iterables.all(removed, remove);
> {code}
> Not exactly sure whether this line causes CPU utilization/bootstrapping
> issue, but this line is redundant because the Predict we pass in is
> Predicates.alwaysTrue(), which means the assertion always
> returns true. So I propose to remove that line.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-13106) Unnecessary assertion
[
https://issues.apache.org/jira/browse/CASSANDRA-13106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Simon Zhou updated CASSANDRA-13106:
---
Attachment: CASSANDRA-13106.patch
> Unnecessary assertion
> -
>
> Key: CASSANDRA-13106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13106
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Simon Zhou
>Assignee: Simon Zhou
>Priority: Minor
> Attachments: CASSANDRA-13106.patch
>
>
> We had over 70 thousand sstables and it's slow to bootstrap new node, even
> though the CPU utilization for main thread of Cassandra was nearly 100%. So
> we took a few stack traces and found that the main thread were busy running
> this line in Tracker.java:
> {code}
> assert Iterables.all(removed, remove);
> {code}
> Not exactly sure whether this line causes CPU utilization/bootstrapping
> issue, but this line is redundant because the Predict we pass in is
> Predicates.alwaysTrue(), which means the assertion always
> returns true. So I propose to remove that line.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-13106) Unnecessary assertion
[
https://issues.apache.org/jira/browse/CASSANDRA-13106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Simon Zhou updated CASSANDRA-13106:
---
Attachment: (was: 0001-Remove-unnecessary-assertion.patch)
> Unnecessary assertion
> -
>
> Key: CASSANDRA-13106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13106
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Simon Zhou
>Assignee: Simon Zhou
>Priority: Minor
> Attachments: CASSANDRA-13106.patch
>
>
> We had over 70 thousand sstables and it's slow to bootstrap new node, even
> though the CPU utilization for main thread of Cassandra was nearly 100%. So
> we took a few stack traces and found that the main thread were busy running
> this line in Tracker.java:
> {code}
> assert Iterables.all(removed, remove);
> {code}
> Not exactly sure whether this line causes CPU utilization/bootstrapping
> issue, but this line is redundant because the Predict we pass in is
> Predicates.alwaysTrue(), which means the assertion always
> returns true. So I propose to remove that line.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-13106) Unnecessary assert
Simon Zhou created CASSANDRA-13106:
--
Summary: Unnecessary assert
Key: CASSANDRA-13106
URL: https://issues.apache.org/jira/browse/CASSANDRA-13106
Project: Cassandra
Issue Type: Improvement
Reporter: Simon Zhou
Assignee: Simon Zhou
Priority: Minor
Attachments: 0001-Remove-unnecessary-assertion.patch
We had over 70 thousand sstables and it's slow to bootstrap new node, even
though the CPU utilization for main thread of Cassandra was nearly 100%. So we
took a few stack traces and found that the main thread were busy running this
line in Tracker.java:
{code}
assert Iterables.all(removed, remove);
{code}
Not exactly sure whether this line causes CPU utilization/bootstrapping issue,
but this line is redundant because the Predict we pass in is
Predicates.alwaysTrue(), which means the assertion always
returns true. So I propose to remove that line.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-13106) Unnecessary assertion
[
https://issues.apache.org/jira/browse/CASSANDRA-13106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Simon Zhou updated CASSANDRA-13106:
---
Attachment: 0001-Remove-unnecessary-assertion.patch
> Unnecessary assertion
> -
>
> Key: CASSANDRA-13106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13106
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Simon Zhou
>Assignee: Simon Zhou
>Priority: Minor
> Attachments: 0001-Remove-unnecessary-assertion.patch
>
>
> We had over 70 thousand sstables and it's slow to bootstrap new node, even
> though the CPU utilization for main thread of Cassandra was nearly 100%. So
> we took a few stack traces and found that the main thread were busy running
> this line in Tracker.java:
> {code}
> assert Iterables.all(removed, remove);
> {code}
> Not exactly sure whether this line causes CPU utilization/bootstrapping
> issue, but this line is redundant because the Predict we pass in is
> Predicates.alwaysTrue(), which means the assertion always
> returns true. So I propose to remove that line.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-13106) Unnecessary assertion
[
https://issues.apache.org/jira/browse/CASSANDRA-13106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Simon Zhou updated CASSANDRA-13106:
---
Summary: Unnecessary assertion (was: Unnecessary assert)
> Unnecessary assertion
> -
>
> Key: CASSANDRA-13106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13106
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Simon Zhou
>Assignee: Simon Zhou
>Priority: Minor
> Attachments: 0001-Remove-unnecessary-assertion.patch
>
>
> We had over 70 thousand sstables and it's slow to bootstrap new node, even
> though the CPU utilization for main thread of Cassandra was nearly 100%. So
> we took a few stack traces and found that the main thread were busy running
> this line in Tracker.java:
> {code}
> assert Iterables.all(removed, remove);
> {code}
> Not exactly sure whether this line causes CPU utilization/bootstrapping
> issue, but this line is redundant because the Predict we pass in is
> Predicates.alwaysTrue(), which means the assertion always
> returns true. So I propose to remove that line.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
[
https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802971#comment-15802971
]
Nate McCall commented on CASSANDRA-12979:
-
Some nits:
- (super minor) Looks like an extra line break before this comment {{// Note
that we have removed files that are still marked as compacting.}} on the trunk
patch
- let's do WARN for the message about no space remaining so it's on the same
output as the initial warning message
Otherwise, LGTM (particularly since [~s_delima] is running it in production
already :)
> checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread
> scope
> ---
>
> Key: CASSANDRA-12979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12979
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jon Haddad
>Assignee: Jon Haddad
> Labels: compaction
> Fix For: 2.2.9, 3.0.11, 4.0, 3.x
>
> Attachments:
> 2.2-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
> 30-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
> trunk-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch
>
>
> If a compaction occurs that looks like it'll take up more space than
> remaining disk available, the compaction manager attempts to reduce the scope
> of the compaction by calling {{reduceScopeForLimitedSpace()}} repeatedly.
> Unfortunately, the while loop passes the {{estimatedWriteSize}} calculated
> from the original call to {{hasAvailableDiskSpace}}, so the comparisons that
> are done will always be against the size of the original compaction, rather
> than the reduced scope one.
> Full method below:
> {code}
> protected void checkAvailableDiskSpace(long estimatedSSTables, long
> expectedWriteSize)
> {
> if(!cfs.isCompactionDiskSpaceCheckEnabled() && compactionType ==
> OperationType.COMPACTION)
> {
> logger.info("Compaction space check is disabled");
> return;
> }
> while (!getDirectories().hasAvailableDiskSpace(estimatedSSTables,
> expectedWriteSize))
> {
> if (!reduceScopeForLimitedSpace())
> throw new RuntimeException(String.format("Not enough space
> for compaction, estimated sstables = %d, expected write size = %d",
> estimatedSSTables, expectedWriteSize));
>
> }
> }
> {code}
> I'm proposing to recalculate the {{estimatedSSTables}} and
> {{expectedWriteSize}} after each iteration of {{reduceScopeForLimitedSpace}}.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-13105) Multi-index query incorrectly returns 0 rows
Voytek Jarnot created CASSANDRA-13105:
-
Summary: Multi-index query incorrectly returns 0 rows
Key: CASSANDRA-13105
URL: https://issues.apache.org/jira/browse/CASSANDRA-13105
Project: Cassandra
Issue Type: Bug
Environment: 3.9.0 on linux & osx
Reporter: Voytek Jarnot
Setup:
{code}
create table test1(id1 text PRIMARY KEY, val1 text, val2 text);
create custom index test1_idx_val1 on test1(val1) using
'org.apache.cassandra.index.sasi.SASIIndex';
create custom index test1_idx_val2 on test1(val2) using
'org.apache.cassandra.index.sasi.SASIIndex';
insert into test1(id1, val1, val2) values ('1', '1val1', '1val2');
insert into test1(id1, val1, val2) values ('2', '~~', '2val2');
{code}
Queries:
{code}
(1) select * from test1 where val1 = '~~';
(2) select * from test1 where val1 < '~~' allow filtering;
(3) select * from test1 where val2 = '1val2';
(4) select * from test1 where val1 < '~~' and val2 = '1val2' allow filtering;
{code}
1, 2, and 3 all work correctly. 4 does not work.
2, 3, and 4 should return the same row (id1='1'); 2 and 3 do, 4 returns 0 rows.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
[
https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802896#comment-15802896
]
Jon Haddad commented on CASSANDRA-12979:
Ready for review.
> checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread
> scope
> ---
>
> Key: CASSANDRA-12979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12979
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jon Haddad
>Assignee: Jon Haddad
> Labels: compaction
> Fix For: 2.2.9, 3.0.11, 4.0, 3.x
>
> Attachments:
> 2.2-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
> 30-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
> trunk-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch
>
>
> If a compaction occurs that looks like it'll take up more space than
> remaining disk available, the compaction manager attempts to reduce the scope
> of the compaction by calling {{reduceScopeForLimitedSpace()}} repeatedly.
> Unfortunately, the while loop passes the {{estimatedWriteSize}} calculated
> from the original call to {{hasAvailableDiskSpace}}, so the comparisons that
> are done will always be against the size of the original compaction, rather
> than the reduced scope one.
> Full method below:
> {code}
> protected void checkAvailableDiskSpace(long estimatedSSTables, long
> expectedWriteSize)
> {
> if(!cfs.isCompactionDiskSpaceCheckEnabled() && compactionType ==
> OperationType.COMPACTION)
> {
> logger.info("Compaction space check is disabled");
> return;
> }
> while (!getDirectories().hasAvailableDiskSpace(estimatedSSTables,
> expectedWriteSize))
> {
> if (!reduceScopeForLimitedSpace())
> throw new RuntimeException(String.format("Not enough space
> for compaction, estimated sstables = %d, expected write size = %d",
> estimatedSSTables, expectedWriteSize));
>
> }
> }
> {code}
> I'm proposing to recalculate the {{estimatedSSTables}} and
> {{expectedWriteSize}} after each iteration of {{reduceScopeForLimitedSpace}}.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
[
https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-12979:
---
Attachment: 2.2-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch
> checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread
> scope
> ---
>
> Key: CASSANDRA-12979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12979
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jon Haddad
>Assignee: Jon Haddad
> Labels: compaction
> Fix For: 2.2.9, 3.0.11, 4.0, 3.x
>
> Attachments:
> 2.2-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
> 30-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
> trunk-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch
>
>
> If a compaction occurs that looks like it'll take up more space than
> remaining disk available, the compaction manager attempts to reduce the scope
> of the compaction by calling {{reduceScopeForLimitedSpace()}} repeatedly.
> Unfortunately, the while loop passes the {{estimatedWriteSize}} calculated
> from the original call to {{hasAvailableDiskSpace}}, so the comparisons that
> are done will always be against the size of the original compaction, rather
> than the reduced scope one.
> Full method below:
> {code}
> protected void checkAvailableDiskSpace(long estimatedSSTables, long
> expectedWriteSize)
> {
> if(!cfs.isCompactionDiskSpaceCheckEnabled() && compactionType ==
> OperationType.COMPACTION)
> {
> logger.info("Compaction space check is disabled");
> return;
> }
> while (!getDirectories().hasAvailableDiskSpace(estimatedSSTables,
> expectedWriteSize))
> {
> if (!reduceScopeForLimitedSpace())
> throw new RuntimeException(String.format("Not enough space
> for compaction, estimated sstables = %d, expected write size = %d",
> estimatedSSTables, expectedWriteSize));
>
> }
> }
> {code}
> I'm proposing to recalculate the {{estimatedSSTables}} and
> {{expectedWriteSize}} after each iteration of {{reduceScopeForLimitedSpace}}.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
[
https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-12979:
---
Attachment: (was:
2.2-0001-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch)
> checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread
> scope
> ---
>
> Key: CASSANDRA-12979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12979
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jon Haddad
>Assignee: Jon Haddad
> Labels: compaction
> Fix For: 2.2.9, 3.0.11, 4.0, 3.x
>
> Attachments:
> 2.2-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
> 30-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
> trunk-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch
>
>
> If a compaction occurs that looks like it'll take up more space than
> remaining disk available, the compaction manager attempts to reduce the scope
> of the compaction by calling {{reduceScopeForLimitedSpace()}} repeatedly.
> Unfortunately, the while loop passes the {{estimatedWriteSize}} calculated
> from the original call to {{hasAvailableDiskSpace}}, so the comparisons that
> are done will always be against the size of the original compaction, rather
> than the reduced scope one.
> Full method below:
> {code}
> protected void checkAvailableDiskSpace(long estimatedSSTables, long
> expectedWriteSize)
> {
> if(!cfs.isCompactionDiskSpaceCheckEnabled() && compactionType ==
> OperationType.COMPACTION)
> {
> logger.info("Compaction space check is disabled");
> return;
> }
> while (!getDirectories().hasAvailableDiskSpace(estimatedSSTables,
> expectedWriteSize))
> {
> if (!reduceScopeForLimitedSpace())
> throw new RuntimeException(String.format("Not enough space
> for compaction, estimated sstables = %d, expected write size = %d",
> estimatedSSTables, expectedWriteSize));
>
> }
> }
> {code}
> I'm proposing to recalculate the {{estimatedSSTables}} and
> {{expectedWriteSize}} after each iteration of {{reduceScopeForLimitedSpace}}.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
[
https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802816#comment-15802816
]
Jon Haddad commented on CASSANDRA-12979:
Trunk functions correctly:
{code}
DEBUG [MemtableFlushWriter:1] 2017-01-05 15:07:58,036
ColumnFamilyStore.java:1194 - Flushed to
[BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-411-big-Data.db')]
(1 sstables, 964.442KiB), biggest 964.442KiB, smallest 964.442KiB
DEBUG [CompactionExecutor:2] 2017-01-05 15:07:59,051 CompactionTask.java:255 -
Compacted (c4056ba0-d39b-11e6-b610-ef0dd3682585) 32 sstables to
[/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-339-big,]
to level=0. 27.586MiB to 28.058MiB (~101% of original) in 14,704ms. Read
Throughput = 1.876MiB/s, Write Throughput = 1.908MiB/s, Row Throughput =
~8,580/s. 128,704 total partitions merged to 128,704. Partition merge counts
were {1:128704, }
DEBUG [CompactionExecutor:1] 2017-01-05 15:08:01,743 CompactionTask.java:255 -
Compacted (c64bb220-d39b-11e6-b610-ef0dd3682585) 29 sstables to
[/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-360-big,]
to level=0. 63.751MiB to 64.450MiB (~101% of original) in 13,580ms. Read
Throughput = 4.694MiB/s, Write Throughput = 4.746MiB/s, Row Throughput =
~20,991/s. 293,883 total partitions merged to 293,883. Partition merge counts
were {1:293883, }
WARN [CompactionExecutor:2] 2017-01-05 15:08:01,743 CompactionTask.java:94 -
insufficient space to compact all requested files. 27.529812MB required,
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-406-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-385-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-387-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-389-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-381-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-370-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-372-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-375-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-393-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-391-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-401-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-411-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-409-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-403-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-384-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-407-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-363-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-382-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-365-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-361-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-388-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-378-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-369-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-380-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-371-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-374-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-394-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-396-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-400-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b6
[jira] [Updated] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
[
https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-12979:
---
Attachment: trunk-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch
> checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread
> scope
> ---
>
> Key: CASSANDRA-12979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12979
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jon Haddad
>Assignee: Jon Haddad
> Labels: compaction
> Fix For: 2.2.9, 3.0.11, 4.0, 3.x
>
> Attachments:
> 2.2-0001-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
> 30-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
> trunk-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch
>
>
> If a compaction occurs that looks like it'll take up more space than
> remaining disk available, the compaction manager attempts to reduce the scope
> of the compaction by calling {{reduceScopeForLimitedSpace()}} repeatedly.
> Unfortunately, the while loop passes the {{estimatedWriteSize}} calculated
> from the original call to {{hasAvailableDiskSpace}}, so the comparisons that
> are done will always be against the size of the original compaction, rather
> than the reduced scope one.
> Full method below:
> {code}
> protected void checkAvailableDiskSpace(long estimatedSSTables, long
> expectedWriteSize)
> {
> if(!cfs.isCompactionDiskSpaceCheckEnabled() && compactionType ==
> OperationType.COMPACTION)
> {
> logger.info("Compaction space check is disabled");
> return;
> }
> while (!getDirectories().hasAvailableDiskSpace(estimatedSSTables,
> expectedWriteSize))
> {
> if (!reduceScopeForLimitedSpace())
> throw new RuntimeException(String.format("Not enough space
> for compaction, estimated sstables = %d, expected write size = %d",
> estimatedSSTables, expectedWriteSize));
>
> }
> }
> {code}
> I'm proposing to recalculate the {{estimatedSSTables}} and
> {{expectedWriteSize}} after each iteration of {{reduceScopeForLimitedSpace}}.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
[
https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-12979:
---
Labels: compaction (was: )
> checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread
> scope
> ---
>
> Key: CASSANDRA-12979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12979
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jon Haddad
>Assignee: Jon Haddad
> Labels: compaction
> Fix For: 2.2.9, 3.0.11, 4.0, 3.x
>
> Attachments:
> 2.2-0001-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
> 30-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch
>
>
> If a compaction occurs that looks like it'll take up more space than
> remaining disk available, the compaction manager attempts to reduce the scope
> of the compaction by calling {{reduceScopeForLimitedSpace()}} repeatedly.
> Unfortunately, the while loop passes the {{estimatedWriteSize}} calculated
> from the original call to {{hasAvailableDiskSpace}}, so the comparisons that
> are done will always be against the size of the original compaction, rather
> than the reduced scope one.
> Full method below:
> {code}
> protected void checkAvailableDiskSpace(long estimatedSSTables, long
> expectedWriteSize)
> {
> if(!cfs.isCompactionDiskSpaceCheckEnabled() && compactionType ==
> OperationType.COMPACTION)
> {
> logger.info("Compaction space check is disabled");
> return;
> }
> while (!getDirectories().hasAvailableDiskSpace(estimatedSSTables,
> expectedWriteSize))
> {
> if (!reduceScopeForLimitedSpace())
> throw new RuntimeException(String.format("Not enough space
> for compaction, estimated sstables = %d, expected write size = %d",
> estimatedSSTables, expectedWriteSize));
>
> }
> }
> {code}
> I'm proposing to recalculate the {{estimatedSSTables}} and
> {{expectedWriteSize}} after each iteration of {{reduceScopeForLimitedSpace}}.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
[
https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-12979:
---
Attachment: 30-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch
> checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread
> scope
> ---
>
> Key: CASSANDRA-12979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12979
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jon Haddad
>Assignee: Jon Haddad
> Fix For: 2.2.9, 3.0.11, 4.0, 3.x
>
> Attachments:
> 2.2-0001-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
> 30-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch
>
>
> If a compaction occurs that looks like it'll take up more space than
> remaining disk available, the compaction manager attempts to reduce the scope
> of the compaction by calling {{reduceScopeForLimitedSpace()}} repeatedly.
> Unfortunately, the while loop passes the {{estimatedWriteSize}} calculated
> from the original call to {{hasAvailableDiskSpace}}, so the comparisons that
> are done will always be against the size of the original compaction, rather
> than the reduced scope one.
> Full method below:
> {code}
> protected void checkAvailableDiskSpace(long estimatedSSTables, long
> expectedWriteSize)
> {
> if(!cfs.isCompactionDiskSpaceCheckEnabled() && compactionType ==
> OperationType.COMPACTION)
> {
> logger.info("Compaction space check is disabled");
> return;
> }
> while (!getDirectories().hasAvailableDiskSpace(estimatedSSTables,
> expectedWriteSize))
> {
> if (!reduceScopeForLimitedSpace())
> throw new RuntimeException(String.format("Not enough space
> for compaction, estimated sstables = %d, expected write size = %d",
> estimatedSSTables, expectedWriteSize));
>
> }
> }
> {code}
> I'm proposing to recalculate the {{estimatedSSTables}} and
> {{expectedWriteSize}} after each iteration of {{reduceScopeForLimitedSpace}}.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
[
https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802783#comment-15802783
]
Jon Haddad commented on CASSANDRA-12979:
Similar output for 3.0:
{code}DEBUG [SlabPoolCleaner] 2017-01-05 14:42:49,576
ColumnFamilyStore.java:854 - Enqueuing flush of standard1: 3393696 (32%)
on-heap, 0 (0%) off-heap
DEBUG [MemtableFlushWriter:3] 2017-01-05 14:42:49,577 Memtable.java:364 -
Writing Memtable-standard1@1306908314(1.065MiB serialized bytes, 4002 ops,
32%/0% of on/off-heap limit)
DEBUG [MemtableFlushWriter:3] 2017-01-05 14:42:49,628 Memtable.java:397 -
Completed flushing
/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-325-big-Data.db
(878.008KiB) for commitlog position ReplayPosition(segmentId=1483655869016,
position=16570861)
DEBUG [CompactionExecutor:4] 2017-01-05 14:42:52,205 CompactionTask.java:235 -
Compacted (48323b00-d398-11e6-a3e2-7b0b736bccfe) 32 sstables to
[/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-319-big,]
to level=0. 28,817,720 bytes to 29,327,595 (~101% of original) in 4,091ms =
6.836708MB/s. 128,262 total partitions merged to 128,262. Partition merge
counts were {1:128262, }
DEBUG [CompactionExecutor:3] 2017-01-05 14:42:52,264 CompactionTask.java:235 -
Compacted (4347af80-d398-11e6-a3e2-7b0b736bccfe) 26 sstables to
[/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-284-big,]
to level=0. 44,583,630 bytes to 45,115,065 (~101% of original) in 12,399ms =
3.470044MB/s. 196,716 total partitions merged to 196,716. Partition merge
counts were {1:196716, }
WARN [CompactionExecutor:4] 2017-01-05 14:42:52,264 CompactionTask.java:98 -
insufficient space to compact all requested files. 112.973495MB required,
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-284-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-318-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-319-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-320-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-323-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-293-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-325-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-324-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-322-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-300-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-321-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-258-big-Data.db')
WARN [CompactionExecutor:4] 2017-01-05 14:42:52,265 CompactionTask.java:319 -
Not enough space for compaction, 112.973495MB estimated. Reducing scope.
DEBUG [CompactionExecutor:4] 2017-01-05 14:42:52,265 CompactionTask.java:158 -
Compacting (4aabc681-d398-11e6-a3e2-7b0b736bccfe)
[/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-318-big-Data.db:level=0,
/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-319-big-Data.db:level=0,
/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-320-big-Data.db:level=0,
/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-323-big-Data.db:level=0,
/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-293-big-Data.db:level=0,
/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-325-big-Data.db:level=0,
/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-324-big-Data.db:level=0,
/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-322-big-Data.db:level=0,
/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-300-big-Data.db:level=0,
/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-321-big-Data.db:level=0,
/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-258-big-Data.db:level=0,
]
DEBUG [CompactionExecutor:4] 2017-01-05 14:42:56,395 CompactionTask.java:235 -
Compacted (4aabc681-d398-11e6-a3e2-7b0b736bccfe) 11 sstables to
[/Volumes/tiny/keyspace1/standard1-07c5ebc0d39811e6a3e27b0b736bccfe/mc-326-big,]
to level=0. 73,346,235 bytes to 73,709,675 (~100% of original) in 4,129ms =
17.024709MB/s. 320,961 total partitions merged to 320,961. Partition merge
counts were {1:320961, }
WARN [CompactionExecutor:4
[jira] [Updated] (CASSANDRA-12883) Remove support for non-JavaScript UDFs
[
https://issues.apache.org/jira/browse/CASSANDRA-12883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-12883:
-
Resolution: Fixed
Fix Version/s: (was: 2.2.x)
3.12
3.0.11
2.2.9
Status: Resolved (was: Patch Available)
Thanks!
Committed as
[6f360b6d239a7b3d3b625e1bcefcaeb2f6b2be1d|https://github.com/apache/cassandra/commit/6f360b6d239a7b3d3b625e1bcefcaeb2f6b2be1d]
to [cassandra-2.2|https://github.com/apache/cassandra/tree/cassandra-2.2] and
merged all the way up to trunk.
> Remove support for non-JavaScript UDFs
> --
>
> Key: CASSANDRA-12883
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12883
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 2.2.9, 3.0.11, 3.12
>
>
> As recently reported in the user mailing list, JSR-223 languages other than
> JavaScript no longer work since version 3.0.
> The reason is that the sandbox implemented in CASSANDRA-9402 restricts the
> use of "evil" packages, classes and functions. Unfortunately, even "non-evil"
> packages from JSR-223 providers are blocked.
> In order to get a JSR-223 provider working fine, we need to allow JSR-223
> provider specific packages and also allow specific runtime permissions.
> The fact that "arbitrary" JSR-223 providers no longer work since 3.0 has just
> been reported recently, means that this functionality (i.e. non-JavaSCript
> JSR-223 UDFs) is obviously not used.
> Therefore I propose to remove support for UDFs that do not use Java or
> JavaScript in 4.0. This will also allow to specialize scripted UDFs on
> Nashorn and allow to use its security features, although these are limited,
> more extensively. (Clarification: this ticket is just about to remove that
> support)
> Also want to point out that we never "officially" supported UDFs that are not
> Java or JavaScript.
> Sample error message:
> {code}
> Traceback (most recent call last):
> File "/usr/bin/cqlsh.py", line 1264, in perform_simple_statement
> result = future.result()
> File
> "/usr/share/cassandra/lib/cassandra-driver-internal-only-3.5.0.post0-d8d0456.zip/cassandra-driver-3.5.0.post0-d8d0456/cassandra/cluster.py",
> line 3650, in result
> raise self._final_exception
> FunctionFailure: Error from server: code=1400 [User Defined Function failure]
> message="execution of 'e.test123[bigint]' failed:
> java.security.AccessControlException: access denied:
> ("java.lang.RuntimePermission"
> "accessClassInPackage.org.python.jline.console")
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13103) incorrect jvm metric names
[
https://issues.apache.org/jira/browse/CASSANDRA-13103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802580#comment-15802580
]
Nate McCall commented on CASSANDRA-13103:
-
>From "weird jvm metrics" thread on user ML, from OP this appeared somewhere
>between 3.5 and 3.9. ML Thread:
https://lists.apache.org/thread.html/558bd7c1f9eff0be60939bae620e13dbbef3a819846e2c4735aa2cfa@%3Cuser.cassandra.apache.org%3E
> incorrect jvm metric names
> --
>
> Key: CASSANDRA-13103
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13103
> Project: Cassandra
> Issue Type: Bug
> Components: Observability
>Reporter: Alain Rastoul
>Priority: Minor
> Labels: lhf
>
> Some jvm metrics have a double dot in name like:
> jvm.memory..total.max , jvm.memory..total.init (etc).
> it seems that an extra dot is added at the end of the name in
> CassandraDaemon.java, around line 367 (in 3.0.10):
> ...
> // enable metrics provided by metrics-jvm.jar
> CassandraMetricsRegistry.Metrics.register("jvm.buffers.", new
> BufferPoolMetricSet(ManagementFactory.getPlatformMBeanServer()));
> CassandraMetricsRegistry.Metrics.register("jvm.gc.", new
> GarbageCollectorMetricSet());
> CassandraMetricsRegistry.Metrics.register("jvm.memory.", new
> MemoryUsageGaugeSet());
> and also added in append method of MetricRegistry.
> Call stack is:
> MetricRegistry>>registerAll(String prefix, MetricSet metrics)
> MetricRegistry>>static String name(String name, String... names)
> MetricRegistry>>static void append(StringBuilder builder, String part)
> and in append the dot is also added:
> ...
> if(builder.length() > 0) {
> builder.append('.');
> }
> builder.append(part);
> ...
> The codahale MetricRegistry class seems to have no recent modification of
> name or append methods, so it look like a small bug.
> May be the fix could be to simply not to add the final dot in the metric
> name, ie "jvm.buffers" instead of "jvm.buffers."
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[10/10] cassandra git commit: Merge branch 'cassandra-3.X' into trunk
Merge branch 'cassandra-3.X' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2cabf9c4 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2cabf9c4 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2cabf9c4 Branch: refs/heads/trunk Commit: 2cabf9c49d0061a9325181a2b66711faecfec31d Parents: d384e78 eca51a0 Author: Robert Stupp Authored: Thu Jan 5 22:28:04 2017 +0100 Committer: Robert Stupp Committed: Thu Jan 5 22:28:04 2017 +0100 -- CHANGES.txt | 2 + NEWS.txt| 3 ++ doc/cql3/CQL.textile| 2 +- lib/jsr223/clojure/README.txt | 8 --- lib/jsr223/groovy/README.txt| 35 - lib/jsr223/jaskell/README.txt | 5 -- lib/jsr223/jruby/README.txt | 54 lib/jsr223/jython/README.txt| 33 lib/jsr223/scala/README.txt | 37 -- .../cql3/functions/ScriptBasedUDFunction.java | 35 + .../cql3/validation/entities/UFScriptTest.java | 3 +- .../validation/entities/UFSecurityTest.java | 12 - 12 files changed, 31 insertions(+), 198 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/2cabf9c4/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/2cabf9c4/NEWS.txt --
[07/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6e716c6d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6e716c6d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6e716c6d Branch: refs/heads/trunk Commit: 6e716c6da41900950e32a5549b3bb1e858ecad18 Parents: c0765ed 6f360b6 Author: Robert Stupp Authored: Thu Jan 5 22:20:31 2017 +0100 Committer: Robert Stupp Committed: Thu Jan 5 22:20:31 2017 +0100 -- CHANGES.txt | 1 + NEWS.txt| 3 ++ doc/cql3/CQL.textile| 2 +- lib/jsr223/clojure/README.txt | 8 --- lib/jsr223/groovy/README.txt| 35 - lib/jsr223/jaskell/README.txt | 5 -- lib/jsr223/jruby/README.txt | 54 lib/jsr223/jython/README.txt| 33 lib/jsr223/scala/README.txt | 37 -- .../cql3/functions/ScriptBasedUDFunction.java | 30 +-- .../cql3/validation/entities/UFScriptTest.java | 3 +- .../validation/entities/UFSecurityTest.java | 12 - 12 files changed, 29 insertions(+), 194 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6e716c6d/CHANGES.txt -- diff --cc CHANGES.txt index 666a771,b41313d..77a310d --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,23 -1,5 +1,24 @@@ -2.2.9 +3.0.11 + * Fixed flacky SSTableRewriterTest: check file counts before calling validateCFS (CASSANDRA-12348) + * Fix deserialization of 2.x DeletedCells (CASSANDRA-12620) + * Add parent repair session id to anticompaction log message (CASSANDRA-12186) + * Improve contention handling on failure to acquire MV lock for streaming and hints (CASSANDRA-12905) + * Fix DELETE and UPDATE queries with empty IN restrictions (CASSANDRA-12829) + * Mark MVs as built after successful bootstrap (CASSANDRA-12984) + * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME (CASSANDRA-13040) + * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021) + * Thread local pools never cleaned up (CASSANDRA-13033) + * Set RPC_READY to false when draining or if a node is marked as shutdown (CASSANDRA-12781) + * CQL often queries static columns unnecessarily (CASSANDRA-12768) + * Make sure sstables only get committed when it's safe to discard commit log records (CASSANDRA-12956) + * Reject default_time_to_live option when creating or altering MVs (CASSANDRA-12868) + * Nodetool should use a more sane max heap size (CASSANDRA-12739) + * LocalToken ensures token values are cloned on heap (CASSANDRA-12651) + * AnticompactionRequestSerializer serializedSize is incorrect (CASSANDRA-12934) + * Prevent reloading of logback.xml from UDF sandbox (CASSANDRA-12535) + * Reenable HeapPool (CASSANDRA-12900) +Merged from 2.2: + * Remove support for non-JavaScript UDFs (CASSANDRA-12883) * Fix DynamicEndpointSnitch noop in multi-datacenter situations (CASSANDRA-13074) * cqlsh copy-from: encode column names to avoid primary key parsing errors (CASSANDRA-12909) * Temporarily fix bug that creates commit log when running offline tools (CASSANDRA-8616) http://git-wip-us.apache.org/repos/asf/cassandra/blob/6e716c6d/NEWS.txt -- diff --cc NEWS.txt index 32b5084,37949a1..b4e6551 --- a/NEWS.txt +++ b/NEWS.txt @@@ -13,77 -13,20 +13,80 @@@ restore snapshots created with the prev 'sstableloader' tool. You can upgrade the file format of your snapshots using the provided 'sstableupgrade' tool. -2.2.9 +3.0.11 = +Upgrading +- + - Nothing specific to this release, but please see previous versions upgrading section, + especially if you are upgrading from 2.2. + - Specifying the default_time_to_live option when creating or altering a + materialized view was erroneously accepted (and ignored). It is now + properly rejected. ++ - Only Java and JavaScript are now supported UDF languages. ++ The sandbox in 3.0 already prevented the use of script languages except Java ++ and JavaScript. + +3.0.10 += + +Upgrading +- + - memtable_allocation_type: offheap_buffers is no longer allowed to be specified in the 3.0 series. + This was an oversight that can cause segfaults. Offheap was re-introduced in 3.4 see CASSANDRA-11039 + and CASSANDRA-9472 for details. + +3.0.9 += + +Upgrading +- + - The ReversedType behaviour has been corrected for clustering columns of + BYTES type containing empty value.
[09/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.X
Merge branch 'cassandra-3.0' into cassandra-3.X Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/eca51a03 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/eca51a03 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/eca51a03 Branch: refs/heads/trunk Commit: eca51a0348141f45ea9bb31b76571ed81284c32c Parents: 5f3b359 6e716c6 Author: Robert Stupp Authored: Thu Jan 5 22:26:48 2017 +0100 Committer: Robert Stupp Committed: Thu Jan 5 22:26:48 2017 +0100 -- CHANGES.txt | 2 + NEWS.txt| 3 ++ doc/cql3/CQL.textile| 2 +- lib/jsr223/clojure/README.txt | 8 --- lib/jsr223/groovy/README.txt| 35 - lib/jsr223/jaskell/README.txt | 5 -- lib/jsr223/jruby/README.txt | 54 lib/jsr223/jython/README.txt| 33 lib/jsr223/scala/README.txt | 37 -- .../cql3/functions/ScriptBasedUDFunction.java | 35 + .../cql3/validation/entities/UFScriptTest.java | 3 +- .../validation/entities/UFSecurityTest.java | 12 - 12 files changed, 31 insertions(+), 198 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/eca51a03/CHANGES.txt -- diff --cc CHANGES.txt index cacb7cd,77a310d..66c2ee6 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,136 -1,5 +1,138 @@@ -3.0.11 - * Fixed flacky SSTableRewriterTest: check file counts before calling validateCFS (CASSANDRA-12348) +3.12 + * Use new token allocation for non bootstrap case as well (CASSANDRA-13080) + * Avoid byte-array copy when key cache is disabled (CASSANDRA-13084) + * More fixes to the TokenAllocator (CASSANDRA-12990) + * Require forceful decommission if number of nodes is less than replication factor (CASSANDRA-12510) + * Allow IN restrictions on column families with collections (CASSANDRA-12654) + * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034) + * nodetool stopdaemon errors out (CASSANDRA-13030) + * Log message size in trace message in OutboundTcpConnection (CASSANDRA-13028) + * Add timeUnit Days for cassandra-stress (CASSANDRA-13029) + * Add mutation size and batch metrics (CASSANDRA-12649) + * Add method to get size of endpoints to TokenMetadata (CASSANDRA-12999) + * Fix primary index calculation for SASI (CASSANDRA-12910) + * Expose time spent waiting in thread pool queue (CASSANDRA-8398) + * Conditionally update index built status to avoid unnecessary flushes (CASSANDRA-12969) + * NoReplicationTokenAllocator should work with zero replication factor (CASSANDRA-12983) + * cqlsh auto completion: refactor definition of compaction strategy options (CASSANDRA-12946) + * Add support for arithmetic operators (CASSANDRA-11935) + * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954) ++Merged from 3.0: ++ * Remove support for non-JavaScript UDFs (CASSANDRA-12883) + + +3.10 + * Fixed query monitoring for range queries (CASSANDRA-13050) + * Remove outboundBindAny configuration property (CASSANDRA-12673) + * Use correct bounds for all-data range when filtering (CASSANDRA-12666) + * Remove timing window in test case (CASSANDRA-12875) + * Resolve unit testing without JCE security libraries installed (CASSANDRA-12945) + * Fix inconsistencies in cassandra-stress load balancing policy (CASSANDRA-12919) + * Fix validation of non-frozen UDT cells (CASSANDRA-12916) + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903) + * Fix Murmur3PartitionerTest (CASSANDRA-12858) + * Move cqlsh syntax rules into separate module and allow easier customization (CASSANDRA-12897) + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283) + * Fix cassandra-stress truncate option (CASSANDRA-12695) + * Fix crossNode value when receiving messages (CASSANDRA-12791) + * Don't load MX4J beans twice (CASSANDRA-12869) + * Extend native protocol request flags, add versions to SUPPORTED, and introduce ProtocolVersion enum (CASSANDRA-12838) + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836) + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845) + * Properly format IPv6 addresses when logging JMX service URL (CASSANDRA-12454) + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777) + * Use non-token restrictions for bounds when token restrictions are overridden (CASSANDRA-12419) + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803) + * Use different build directories for Eclipse and Ant (CASSANDRA-12466) + * Avoid potential Attribu
[03/10] cassandra git commit: Remove support for non-JavaScript UDFs
Remove support for non-JavaScript UDFs
patch by Robert Stupp; reviewed by Tyler Hobbs for CASSANDRA-12883
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6f360b6d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6f360b6d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6f360b6d
Branch: refs/heads/cassandra-3.X
Commit: 6f360b6d239a7b3d3b625e1bcefcaeb2f6b2be1d
Parents: 0cf0f67
Author: Robert Stupp
Authored: Thu Jan 5 22:18:37 2017 +0100
Committer: Robert Stupp
Committed: Thu Jan 5 22:19:50 2017 +0100
--
CHANGES.txt | 1 +
NEWS.txt| 16
.../cassandra/cql3/functions/ScriptBasedUDF.java| 5 +
3 files changed, 22 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6f360b6d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d31ffc8..b41313d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.9
+ * Remove support for non-JavaScript UDFs (CASSANDRA-12883)
* Fix DynamicEndpointSnitch noop in multi-datacenter situations
(CASSANDRA-13074)
* cqlsh copy-from: encode column names to avoid primary key parsing errors
(CASSANDRA-12909)
* Temporarily fix bug that creates commit log when running offline tools
(CASSANDRA-8616)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6f360b6d/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 0a3ab36..37949a1 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -13,6 +13,19 @@ restore snapshots created with the previous major version
using the
'sstableloader' tool. You can upgrade the file format of your snapshots
using the provided 'sstableupgrade' tool.
+2.2.9
+=
+
+Deprecation
+---
+
+(See note about the new feature User-Defined-Functions in 2.2.0.)
+
+Since the security manager added in 3.0 only allows Java and JavaScript
+UDFs to be run, UDFs for other languages are deprecated and support for
+non-Java and non-JavaScript UDFs is deprecated in 2.2 and has been removed
+in version 3.0.11.
+
2.2.8
=
@@ -230,6 +243,9 @@ New features
3.0. This will inherently be backwards-incompatible with any 2.2
UDF that perform insecure operations such as opening a socket or
writing to the filesystem.
+
+ Per the previous note about adding a security manager in 3.0, this
security manager
+ means that non JavaScipt UDF's won't run, there for their use is
deprecated.
- Row-cache is now fully off-heap.
- jemalloc is now automatically preloaded and used on Linux and OS-X if
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6f360b6d/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
--
diff --git a/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
b/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
index e55d450..2d46934 100644
--- a/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
+++ b/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
@@ -73,6 +73,11 @@ public class ScriptBasedUDF extends UDFunction
{
super(name, argNames, argTypes, returnType, calledOnNullInput,
language, body);
+if (!"JavaScript".equalsIgnoreCase(language))
+logger.warn("Support for UDFs using '" + language + "' has been
deprecated and removed in 3.0. If '" +
+language + "' actually is JavaScript, change the
language used in CREATE/ALTER FUNCTION to " +
+"'javascript'.");
+
Compilable scriptEngine = scriptEngines.get(language);
if (scriptEngine == null)
throw new InvalidRequestException(String.format("Invalid language
'%s' for function '%s'", language, name));
[02/10] cassandra git commit: Remove support for non-JavaScript UDFs
Remove support for non-JavaScript UDFs
patch by Robert Stupp; reviewed by Tyler Hobbs for CASSANDRA-12883
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6f360b6d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6f360b6d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6f360b6d
Branch: refs/heads/cassandra-3.0
Commit: 6f360b6d239a7b3d3b625e1bcefcaeb2f6b2be1d
Parents: 0cf0f67
Author: Robert Stupp
Authored: Thu Jan 5 22:18:37 2017 +0100
Committer: Robert Stupp
Committed: Thu Jan 5 22:19:50 2017 +0100
--
CHANGES.txt | 1 +
NEWS.txt| 16
.../cassandra/cql3/functions/ScriptBasedUDF.java| 5 +
3 files changed, 22 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6f360b6d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d31ffc8..b41313d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.9
+ * Remove support for non-JavaScript UDFs (CASSANDRA-12883)
* Fix DynamicEndpointSnitch noop in multi-datacenter situations
(CASSANDRA-13074)
* cqlsh copy-from: encode column names to avoid primary key parsing errors
(CASSANDRA-12909)
* Temporarily fix bug that creates commit log when running offline tools
(CASSANDRA-8616)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6f360b6d/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 0a3ab36..37949a1 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -13,6 +13,19 @@ restore snapshots created with the previous major version
using the
'sstableloader' tool. You can upgrade the file format of your snapshots
using the provided 'sstableupgrade' tool.
+2.2.9
+=
+
+Deprecation
+---
+
+(See note about the new feature User-Defined-Functions in 2.2.0.)
+
+Since the security manager added in 3.0 only allows Java and JavaScript
+UDFs to be run, UDFs for other languages are deprecated and support for
+non-Java and non-JavaScript UDFs is deprecated in 2.2 and has been removed
+in version 3.0.11.
+
2.2.8
=
@@ -230,6 +243,9 @@ New features
3.0. This will inherently be backwards-incompatible with any 2.2
UDF that perform insecure operations such as opening a socket or
writing to the filesystem.
+
+ Per the previous note about adding a security manager in 3.0, this
security manager
+ means that non JavaScipt UDF's won't run, there for their use is
deprecated.
- Row-cache is now fully off-heap.
- jemalloc is now automatically preloaded and used on Linux and OS-X if
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6f360b6d/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
--
diff --git a/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
b/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
index e55d450..2d46934 100644
--- a/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
+++ b/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
@@ -73,6 +73,11 @@ public class ScriptBasedUDF extends UDFunction
{
super(name, argNames, argTypes, returnType, calledOnNullInput,
language, body);
+if (!"JavaScript".equalsIgnoreCase(language))
+logger.warn("Support for UDFs using '" + language + "' has been
deprecated and removed in 3.0. If '" +
+language + "' actually is JavaScript, change the
language used in CREATE/ALTER FUNCTION to " +
+"'javascript'.");
+
Compilable scriptEngine = scriptEngines.get(language);
if (scriptEngine == null)
throw new InvalidRequestException(String.format("Invalid language
'%s' for function '%s'", language, name));
[04/10] cassandra git commit: Remove support for non-JavaScript UDFs
Remove support for non-JavaScript UDFs
patch by Robert Stupp; reviewed by Tyler Hobbs for CASSANDRA-12883
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6f360b6d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6f360b6d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6f360b6d
Branch: refs/heads/trunk
Commit: 6f360b6d239a7b3d3b625e1bcefcaeb2f6b2be1d
Parents: 0cf0f67
Author: Robert Stupp
Authored: Thu Jan 5 22:18:37 2017 +0100
Committer: Robert Stupp
Committed: Thu Jan 5 22:19:50 2017 +0100
--
CHANGES.txt | 1 +
NEWS.txt| 16
.../cassandra/cql3/functions/ScriptBasedUDF.java| 5 +
3 files changed, 22 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6f360b6d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d31ffc8..b41313d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.9
+ * Remove support for non-JavaScript UDFs (CASSANDRA-12883)
* Fix DynamicEndpointSnitch noop in multi-datacenter situations
(CASSANDRA-13074)
* cqlsh copy-from: encode column names to avoid primary key parsing errors
(CASSANDRA-12909)
* Temporarily fix bug that creates commit log when running offline tools
(CASSANDRA-8616)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6f360b6d/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 0a3ab36..37949a1 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -13,6 +13,19 @@ restore snapshots created with the previous major version
using the
'sstableloader' tool. You can upgrade the file format of your snapshots
using the provided 'sstableupgrade' tool.
+2.2.9
+=
+
+Deprecation
+---
+
+(See note about the new feature User-Defined-Functions in 2.2.0.)
+
+Since the security manager added in 3.0 only allows Java and JavaScript
+UDFs to be run, UDFs for other languages are deprecated and support for
+non-Java and non-JavaScript UDFs is deprecated in 2.2 and has been removed
+in version 3.0.11.
+
2.2.8
=
@@ -230,6 +243,9 @@ New features
3.0. This will inherently be backwards-incompatible with any 2.2
UDF that perform insecure operations such as opening a socket or
writing to the filesystem.
+
+ Per the previous note about adding a security manager in 3.0, this
security manager
+ means that non JavaScipt UDF's won't run, there for their use is
deprecated.
- Row-cache is now fully off-heap.
- jemalloc is now automatically preloaded and used on Linux and OS-X if
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6f360b6d/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
--
diff --git a/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
b/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
index e55d450..2d46934 100644
--- a/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
+++ b/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
@@ -73,6 +73,11 @@ public class ScriptBasedUDF extends UDFunction
{
super(name, argNames, argTypes, returnType, calledOnNullInput,
language, body);
+if (!"JavaScript".equalsIgnoreCase(language))
+logger.warn("Support for UDFs using '" + language + "' has been
deprecated and removed in 3.0. If '" +
+language + "' actually is JavaScript, change the
language used in CREATE/ALTER FUNCTION to " +
+"'javascript'.");
+
Compilable scriptEngine = scriptEngines.get(language);
if (scriptEngine == null)
throw new InvalidRequestException(String.format("Invalid language
'%s' for function '%s'", language, name));
[05/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6e716c6d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6e716c6d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6e716c6d Branch: refs/heads/cassandra-3.X Commit: 6e716c6da41900950e32a5549b3bb1e858ecad18 Parents: c0765ed 6f360b6 Author: Robert Stupp Authored: Thu Jan 5 22:20:31 2017 +0100 Committer: Robert Stupp Committed: Thu Jan 5 22:20:31 2017 +0100 -- CHANGES.txt | 1 + NEWS.txt| 3 ++ doc/cql3/CQL.textile| 2 +- lib/jsr223/clojure/README.txt | 8 --- lib/jsr223/groovy/README.txt| 35 - lib/jsr223/jaskell/README.txt | 5 -- lib/jsr223/jruby/README.txt | 54 lib/jsr223/jython/README.txt| 33 lib/jsr223/scala/README.txt | 37 -- .../cql3/functions/ScriptBasedUDFunction.java | 30 +-- .../cql3/validation/entities/UFScriptTest.java | 3 +- .../validation/entities/UFSecurityTest.java | 12 - 12 files changed, 29 insertions(+), 194 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6e716c6d/CHANGES.txt -- diff --cc CHANGES.txt index 666a771,b41313d..77a310d --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,23 -1,5 +1,24 @@@ -2.2.9 +3.0.11 + * Fixed flacky SSTableRewriterTest: check file counts before calling validateCFS (CASSANDRA-12348) + * Fix deserialization of 2.x DeletedCells (CASSANDRA-12620) + * Add parent repair session id to anticompaction log message (CASSANDRA-12186) + * Improve contention handling on failure to acquire MV lock for streaming and hints (CASSANDRA-12905) + * Fix DELETE and UPDATE queries with empty IN restrictions (CASSANDRA-12829) + * Mark MVs as built after successful bootstrap (CASSANDRA-12984) + * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME (CASSANDRA-13040) + * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021) + * Thread local pools never cleaned up (CASSANDRA-13033) + * Set RPC_READY to false when draining or if a node is marked as shutdown (CASSANDRA-12781) + * CQL often queries static columns unnecessarily (CASSANDRA-12768) + * Make sure sstables only get committed when it's safe to discard commit log records (CASSANDRA-12956) + * Reject default_time_to_live option when creating or altering MVs (CASSANDRA-12868) + * Nodetool should use a more sane max heap size (CASSANDRA-12739) + * LocalToken ensures token values are cloned on heap (CASSANDRA-12651) + * AnticompactionRequestSerializer serializedSize is incorrect (CASSANDRA-12934) + * Prevent reloading of logback.xml from UDF sandbox (CASSANDRA-12535) + * Reenable HeapPool (CASSANDRA-12900) +Merged from 2.2: + * Remove support for non-JavaScript UDFs (CASSANDRA-12883) * Fix DynamicEndpointSnitch noop in multi-datacenter situations (CASSANDRA-13074) * cqlsh copy-from: encode column names to avoid primary key parsing errors (CASSANDRA-12909) * Temporarily fix bug that creates commit log when running offline tools (CASSANDRA-8616) http://git-wip-us.apache.org/repos/asf/cassandra/blob/6e716c6d/NEWS.txt -- diff --cc NEWS.txt index 32b5084,37949a1..b4e6551 --- a/NEWS.txt +++ b/NEWS.txt @@@ -13,77 -13,20 +13,80 @@@ restore snapshots created with the prev 'sstableloader' tool. You can upgrade the file format of your snapshots using the provided 'sstableupgrade' tool. -2.2.9 +3.0.11 = +Upgrading +- + - Nothing specific to this release, but please see previous versions upgrading section, + especially if you are upgrading from 2.2. + - Specifying the default_time_to_live option when creating or altering a + materialized view was erroneously accepted (and ignored). It is now + properly rejected. ++ - Only Java and JavaScript are now supported UDF languages. ++ The sandbox in 3.0 already prevented the use of script languages except Java ++ and JavaScript. + +3.0.10 += + +Upgrading +- + - memtable_allocation_type: offheap_buffers is no longer allowed to be specified in the 3.0 series. + This was an oversight that can cause segfaults. Offheap was re-introduced in 3.4 see CASSANDRA-11039 + and CASSANDRA-9472 for details. + +3.0.9 += + +Upgrading +- + - The ReversedType behaviour has been corrected for clustering columns of + BYTES type containing empty
[06/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6e716c6d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6e716c6d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6e716c6d Branch: refs/heads/cassandra-3.0 Commit: 6e716c6da41900950e32a5549b3bb1e858ecad18 Parents: c0765ed 6f360b6 Author: Robert Stupp Authored: Thu Jan 5 22:20:31 2017 +0100 Committer: Robert Stupp Committed: Thu Jan 5 22:20:31 2017 +0100 -- CHANGES.txt | 1 + NEWS.txt| 3 ++ doc/cql3/CQL.textile| 2 +- lib/jsr223/clojure/README.txt | 8 --- lib/jsr223/groovy/README.txt| 35 - lib/jsr223/jaskell/README.txt | 5 -- lib/jsr223/jruby/README.txt | 54 lib/jsr223/jython/README.txt| 33 lib/jsr223/scala/README.txt | 37 -- .../cql3/functions/ScriptBasedUDFunction.java | 30 +-- .../cql3/validation/entities/UFScriptTest.java | 3 +- .../validation/entities/UFSecurityTest.java | 12 - 12 files changed, 29 insertions(+), 194 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6e716c6d/CHANGES.txt -- diff --cc CHANGES.txt index 666a771,b41313d..77a310d --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,23 -1,5 +1,24 @@@ -2.2.9 +3.0.11 + * Fixed flacky SSTableRewriterTest: check file counts before calling validateCFS (CASSANDRA-12348) + * Fix deserialization of 2.x DeletedCells (CASSANDRA-12620) + * Add parent repair session id to anticompaction log message (CASSANDRA-12186) + * Improve contention handling on failure to acquire MV lock for streaming and hints (CASSANDRA-12905) + * Fix DELETE and UPDATE queries with empty IN restrictions (CASSANDRA-12829) + * Mark MVs as built after successful bootstrap (CASSANDRA-12984) + * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME (CASSANDRA-13040) + * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021) + * Thread local pools never cleaned up (CASSANDRA-13033) + * Set RPC_READY to false when draining or if a node is marked as shutdown (CASSANDRA-12781) + * CQL often queries static columns unnecessarily (CASSANDRA-12768) + * Make sure sstables only get committed when it's safe to discard commit log records (CASSANDRA-12956) + * Reject default_time_to_live option when creating or altering MVs (CASSANDRA-12868) + * Nodetool should use a more sane max heap size (CASSANDRA-12739) + * LocalToken ensures token values are cloned on heap (CASSANDRA-12651) + * AnticompactionRequestSerializer serializedSize is incorrect (CASSANDRA-12934) + * Prevent reloading of logback.xml from UDF sandbox (CASSANDRA-12535) + * Reenable HeapPool (CASSANDRA-12900) +Merged from 2.2: + * Remove support for non-JavaScript UDFs (CASSANDRA-12883) * Fix DynamicEndpointSnitch noop in multi-datacenter situations (CASSANDRA-13074) * cqlsh copy-from: encode column names to avoid primary key parsing errors (CASSANDRA-12909) * Temporarily fix bug that creates commit log when running offline tools (CASSANDRA-8616) http://git-wip-us.apache.org/repos/asf/cassandra/blob/6e716c6d/NEWS.txt -- diff --cc NEWS.txt index 32b5084,37949a1..b4e6551 --- a/NEWS.txt +++ b/NEWS.txt @@@ -13,77 -13,20 +13,80 @@@ restore snapshots created with the prev 'sstableloader' tool. You can upgrade the file format of your snapshots using the provided 'sstableupgrade' tool. -2.2.9 +3.0.11 = +Upgrading +- + - Nothing specific to this release, but please see previous versions upgrading section, + especially if you are upgrading from 2.2. + - Specifying the default_time_to_live option when creating or altering a + materialized view was erroneously accepted (and ignored). It is now + properly rejected. ++ - Only Java and JavaScript are now supported UDF languages. ++ The sandbox in 3.0 already prevented the use of script languages except Java ++ and JavaScript. + +3.0.10 += + +Upgrading +- + - memtable_allocation_type: offheap_buffers is no longer allowed to be specified in the 3.0 series. + This was an oversight that can cause segfaults. Offheap was re-introduced in 3.4 see CASSANDRA-11039 + and CASSANDRA-9472 for details. + +3.0.9 += + +Upgrading +- + - The ReversedType behaviour has been corrected for clustering columns of + BYTES type containing empty
[08/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.X
Merge branch 'cassandra-3.0' into cassandra-3.X Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/eca51a03 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/eca51a03 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/eca51a03 Branch: refs/heads/cassandra-3.X Commit: eca51a0348141f45ea9bb31b76571ed81284c32c Parents: 5f3b359 6e716c6 Author: Robert Stupp Authored: Thu Jan 5 22:26:48 2017 +0100 Committer: Robert Stupp Committed: Thu Jan 5 22:26:48 2017 +0100 -- CHANGES.txt | 2 + NEWS.txt| 3 ++ doc/cql3/CQL.textile| 2 +- lib/jsr223/clojure/README.txt | 8 --- lib/jsr223/groovy/README.txt| 35 - lib/jsr223/jaskell/README.txt | 5 -- lib/jsr223/jruby/README.txt | 54 lib/jsr223/jython/README.txt| 33 lib/jsr223/scala/README.txt | 37 -- .../cql3/functions/ScriptBasedUDFunction.java | 35 + .../cql3/validation/entities/UFScriptTest.java | 3 +- .../validation/entities/UFSecurityTest.java | 12 - 12 files changed, 31 insertions(+), 198 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/eca51a03/CHANGES.txt -- diff --cc CHANGES.txt index cacb7cd,77a310d..66c2ee6 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,136 -1,5 +1,138 @@@ -3.0.11 - * Fixed flacky SSTableRewriterTest: check file counts before calling validateCFS (CASSANDRA-12348) +3.12 + * Use new token allocation for non bootstrap case as well (CASSANDRA-13080) + * Avoid byte-array copy when key cache is disabled (CASSANDRA-13084) + * More fixes to the TokenAllocator (CASSANDRA-12990) + * Require forceful decommission if number of nodes is less than replication factor (CASSANDRA-12510) + * Allow IN restrictions on column families with collections (CASSANDRA-12654) + * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034) + * nodetool stopdaemon errors out (CASSANDRA-13030) + * Log message size in trace message in OutboundTcpConnection (CASSANDRA-13028) + * Add timeUnit Days for cassandra-stress (CASSANDRA-13029) + * Add mutation size and batch metrics (CASSANDRA-12649) + * Add method to get size of endpoints to TokenMetadata (CASSANDRA-12999) + * Fix primary index calculation for SASI (CASSANDRA-12910) + * Expose time spent waiting in thread pool queue (CASSANDRA-8398) + * Conditionally update index built status to avoid unnecessary flushes (CASSANDRA-12969) + * NoReplicationTokenAllocator should work with zero replication factor (CASSANDRA-12983) + * cqlsh auto completion: refactor definition of compaction strategy options (CASSANDRA-12946) + * Add support for arithmetic operators (CASSANDRA-11935) + * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954) ++Merged from 3.0: ++ * Remove support for non-JavaScript UDFs (CASSANDRA-12883) + + +3.10 + * Fixed query monitoring for range queries (CASSANDRA-13050) + * Remove outboundBindAny configuration property (CASSANDRA-12673) + * Use correct bounds for all-data range when filtering (CASSANDRA-12666) + * Remove timing window in test case (CASSANDRA-12875) + * Resolve unit testing without JCE security libraries installed (CASSANDRA-12945) + * Fix inconsistencies in cassandra-stress load balancing policy (CASSANDRA-12919) + * Fix validation of non-frozen UDT cells (CASSANDRA-12916) + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903) + * Fix Murmur3PartitionerTest (CASSANDRA-12858) + * Move cqlsh syntax rules into separate module and allow easier customization (CASSANDRA-12897) + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283) + * Fix cassandra-stress truncate option (CASSANDRA-12695) + * Fix crossNode value when receiving messages (CASSANDRA-12791) + * Don't load MX4J beans twice (CASSANDRA-12869) + * Extend native protocol request flags, add versions to SUPPORTED, and introduce ProtocolVersion enum (CASSANDRA-12838) + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836) + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845) + * Properly format IPv6 addresses when logging JMX service URL (CASSANDRA-12454) + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777) + * Use non-token restrictions for bounds when token restrictions are overridden (CASSANDRA-12419) + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803) + * Use different build directories for Eclipse and Ant (CASSANDRA-12466) + * Avoid potential
[01/10] cassandra git commit: Remove support for non-JavaScript UDFs
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.2 0cf0f67df -> 6f360b6d2
refs/heads/cassandra-3.0 c0765ed82 -> 6e716c6da
refs/heads/cassandra-3.X 5f3b35981 -> eca51a034
refs/heads/trunk d384e781d -> 2cabf9c49
Remove support for non-JavaScript UDFs
patch by Robert Stupp; reviewed by Tyler Hobbs for CASSANDRA-12883
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6f360b6d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6f360b6d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6f360b6d
Branch: refs/heads/cassandra-2.2
Commit: 6f360b6d239a7b3d3b625e1bcefcaeb2f6b2be1d
Parents: 0cf0f67
Author: Robert Stupp
Authored: Thu Jan 5 22:18:37 2017 +0100
Committer: Robert Stupp
Committed: Thu Jan 5 22:19:50 2017 +0100
--
CHANGES.txt | 1 +
NEWS.txt| 16
.../cassandra/cql3/functions/ScriptBasedUDF.java| 5 +
3 files changed, 22 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6f360b6d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d31ffc8..b41313d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.9
+ * Remove support for non-JavaScript UDFs (CASSANDRA-12883)
* Fix DynamicEndpointSnitch noop in multi-datacenter situations
(CASSANDRA-13074)
* cqlsh copy-from: encode column names to avoid primary key parsing errors
(CASSANDRA-12909)
* Temporarily fix bug that creates commit log when running offline tools
(CASSANDRA-8616)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6f360b6d/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 0a3ab36..37949a1 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -13,6 +13,19 @@ restore snapshots created with the previous major version
using the
'sstableloader' tool. You can upgrade the file format of your snapshots
using the provided 'sstableupgrade' tool.
+2.2.9
+=
+
+Deprecation
+---
+
+(See note about the new feature User-Defined-Functions in 2.2.0.)
+
+Since the security manager added in 3.0 only allows Java and JavaScript
+UDFs to be run, UDFs for other languages are deprecated and support for
+non-Java and non-JavaScript UDFs is deprecated in 2.2 and has been removed
+in version 3.0.11.
+
2.2.8
=
@@ -230,6 +243,9 @@ New features
3.0. This will inherently be backwards-incompatible with any 2.2
UDF that perform insecure operations such as opening a socket or
writing to the filesystem.
+
+ Per the previous note about adding a security manager in 3.0, this
security manager
+ means that non JavaScipt UDF's won't run, there for their use is
deprecated.
- Row-cache is now fully off-heap.
- jemalloc is now automatically preloaded and used on Linux and OS-X if
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6f360b6d/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
--
diff --git a/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
b/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
index e55d450..2d46934 100644
--- a/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
+++ b/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
@@ -73,6 +73,11 @@ public class ScriptBasedUDF extends UDFunction
{
super(name, argNames, argTypes, returnType, calledOnNullInput,
language, body);
+if (!"JavaScript".equalsIgnoreCase(language))
+logger.warn("Support for UDFs using '" + language + "' has been
deprecated and removed in 3.0. If '" +
+language + "' actually is JavaScript, change the
language used in CREATE/ALTER FUNCTION to " +
+"'javascript'.");
+
Compilable scriptEngine = scriptEngines.get(language);
if (scriptEngine == null)
throw new InvalidRequestException(String.format("Invalid language
'%s' for function '%s'", language, name));
[jira] [Updated] (CASSANDRA-13103) incorrect jvm metric names
[
https://issues.apache.org/jira/browse/CASSANDRA-13103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nate McCall updated CASSANDRA-13103:
Labels: lhf (was: )
> incorrect jvm metric names
> --
>
> Key: CASSANDRA-13103
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13103
> Project: Cassandra
> Issue Type: Bug
> Components: Observability
>Reporter: Alain Rastoul
>Priority: Minor
> Labels: lhf
>
> Some jvm metrics have a double dot in name like:
> jvm.memory..total.max , jvm.memory..total.init (etc).
> it seems that an extra dot is added at the end of the name in
> CassandraDaemon.java, around line 367 (in 3.0.10):
> ...
> // enable metrics provided by metrics-jvm.jar
> CassandraMetricsRegistry.Metrics.register("jvm.buffers.", new
> BufferPoolMetricSet(ManagementFactory.getPlatformMBeanServer()));
> CassandraMetricsRegistry.Metrics.register("jvm.gc.", new
> GarbageCollectorMetricSet());
> CassandraMetricsRegistry.Metrics.register("jvm.memory.", new
> MemoryUsageGaugeSet());
> and also added in append method of MetricRegistry.
> Call stack is:
> MetricRegistry>>registerAll(String prefix, MetricSet metrics)
> MetricRegistry>>static String name(String name, String... names)
> MetricRegistry>>static void append(StringBuilder builder, String part)
> and in append the dot is also added:
> ...
> if(builder.length() > 0) {
> builder.append('.');
> }
> builder.append(part);
> ...
> The codahale MetricRegistry class seems to have no recent modification of
> name or append methods, so it look like a small bug.
> May be the fix could be to simply not to add the final dot in the metric
> name, ie "jvm.buffers" instead of "jvm.buffers."
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-13104) Use StandardCharset instead of Charset.forName()
[
https://issues.apache.org/jira/browse/CASSANDRA-13104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-13104:
-
Status: Patch Available (was: Open)
Branch and CI
||trunk|[branch|https://github.com/apache/cassandra/compare/trunk...snazy:13104-std-charset-trunk]|[testall|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-13104-std-charset-trunk-testall/lastSuccessfulBuild/]|[dtest|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-13104-std-charset-trunk-dtest/lastSuccessfulBuild/]
> Use StandardCharset instead of Charset.forName()
>
>
> Key: CASSANDRA-13104
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13104
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 4.0
>
>
> Trivial change replacing {{Charset.forName()}} with {{StandardCharset}}
> instances.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-13104) Use StandardCharset instead of Charset.forName()
Robert Stupp created CASSANDRA-13104:
Summary: Use StandardCharset instead of Charset.forName()
Key: CASSANDRA-13104
URL: https://issues.apache.org/jira/browse/CASSANDRA-13104
Project: Cassandra
Issue Type: Improvement
Reporter: Robert Stupp
Assignee: Robert Stupp
Priority: Minor
Fix For: 4.0
Trivial change replacing {{Charset.forName()}} with {{StandardCharset}}
instances.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12883) Remove support for non-JavaScript UDFs
[
https://issues.apache.org/jira/browse/CASSANDRA-12883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802437#comment-15802437
]
Tyler Hobbs commented on CASSANDRA-12883:
-
+1
> Remove support for non-JavaScript UDFs
> --
>
> Key: CASSANDRA-12883
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12883
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 2.2.x
>
>
> As recently reported in the user mailing list, JSR-223 languages other than
> JavaScript no longer work since version 3.0.
> The reason is that the sandbox implemented in CASSANDRA-9402 restricts the
> use of "evil" packages, classes and functions. Unfortunately, even "non-evil"
> packages from JSR-223 providers are blocked.
> In order to get a JSR-223 provider working fine, we need to allow JSR-223
> provider specific packages and also allow specific runtime permissions.
> The fact that "arbitrary" JSR-223 providers no longer work since 3.0 has just
> been reported recently, means that this functionality (i.e. non-JavaSCript
> JSR-223 UDFs) is obviously not used.
> Therefore I propose to remove support for UDFs that do not use Java or
> JavaScript in 4.0. This will also allow to specialize scripted UDFs on
> Nashorn and allow to use its security features, although these are limited,
> more extensively. (Clarification: this ticket is just about to remove that
> support)
> Also want to point out that we never "officially" supported UDFs that are not
> Java or JavaScript.
> Sample error message:
> {code}
> Traceback (most recent call last):
> File "/usr/bin/cqlsh.py", line 1264, in perform_simple_statement
> result = future.result()
> File
> "/usr/share/cassandra/lib/cassandra-driver-internal-only-3.5.0.post0-d8d0456.zip/cassandra-driver-3.5.0.post0-d8d0456/cassandra/cluster.py",
> line 3650, in result
> raise self._final_exception
> FunctionFailure: Error from server: code=1400 [User Defined Function failure]
> message="execution of 'e.test123[bigint]' failed:
> java.security.AccessControlException: access denied:
> ("java.lang.RuntimePermission"
> "accessClassInPackage.org.python.jline.console")
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-13080) Use new token allocation for non bootstrap case as well.
[
https://issues.apache.org/jira/browse/CASSANDRA-13080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dikang Gu updated CASSANDRA-13080:
--
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
committed as {{5f3b35981d28179606c52599d502de7a08b9b690}}
> Use new token allocation for non bootstrap case as well.
>
>
> Key: CASSANDRA-13080
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13080
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Dikang Gu
>Assignee: Dikang Gu
> Fix For: 3.12
>
>
> There are couple reasons I think we should use the new token allocation for
> non bootstrap case as well.
> 1. In some cases, We want to bring up nodes, but do not need to stream any
> data to the new nodes. So we want to allocate correct tokens, and skip the
> JOIN state, which should avoid triggering pending range calculation on other
> nodes as well. On use case is when we bring up a new DC.
> 2. We can unify the token allocation code path for both bootstrap and
> non-bootstrap use case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[3/3] cassandra git commit: Merge branch 'cassandra-3.X' into trunk
Merge branch 'cassandra-3.X' into trunk * cassandra-3.X: Use new token allocation for non bootstrap case as well. Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d384e781 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d384e781 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d384e781 Branch: refs/heads/trunk Commit: d384e781d6f7c028dbe88cfe9dd3e966e72cd046 Parents: 3d90bb0 5f3b359 Author: Dikang Gu Authored: Thu Jan 5 12:11:58 2017 -0800 Committer: Dikang Gu Committed: Thu Jan 5 12:13:29 2017 -0800 -- CHANGES.txt | 1 + .../org/apache/cassandra/dht/BootStrapper.java | 18 +++-- .../dht/tokenallocator/TokenAllocation.java | 3 - .../cassandra/service/StorageService.java | 69 +--- .../apache/cassandra/dht/BootStrapperTest.java | 2 +- 5 files changed, 48 insertions(+), 45 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d384e781/CHANGES.txt -- diff --cc CHANGES.txt index 08eed35,cacb7cd..8f625db --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,15 -1,5 +1,16 @@@ +4.0 + * Use Caffeine (W-TinyLFU) for on-heap caches (CASSANDRA-10855) + * Thrift removal (CASSANDRA-5) + * Remove pre-3.0 compatibility code for 4.0 (CASSANDRA-12716) + * Add column definition kind to dropped columns in schema (CASSANDRA-12705) + * Add (automate) Nodetool Documentation (CASSANDRA-12672) + * Update bundled cqlsh python driver to 3.7.0 (CASSANDRA-12736) + * Reject invalid replication settings when creating or altering a keyspace (CASSANDRA-12681) + * Clean up the SSTableReader#getScanner API wrt removal of RateLimiter (CASSANDRA-12422) + + 3.12 + * Use new token allocation for non bootstrap case as well (CASSANDRA-13080) * Avoid byte-array copy when key cache is disabled (CASSANDRA-13084) * More fixes to the TokenAllocator (CASSANDRA-12990) * Require forceful decommission if number of nodes is less than replication factor (CASSANDRA-12510) http://git-wip-us.apache.org/repos/asf/cassandra/blob/d384e781/src/java/org/apache/cassandra/service/StorageService.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d384e781/test/unit/org/apache/cassandra/dht/BootStrapperTest.java --
[1/3] cassandra git commit: Use new token allocation for non bootstrap case as well.
Repository: cassandra
Updated Branches:
refs/heads/cassandra-3.X aa7882e54 -> 5f3b35981
refs/heads/trunk 3d90bb0cc -> d384e781d
Use new token allocation for non bootstrap case as well.
patch by Dikang Gu; reviewed by Branimir Lambov for CASSANSRA-13080
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5f3b3598
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5f3b3598
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5f3b3598
Branch: refs/heads/cassandra-3.X
Commit: 5f3b35981d28179606c52599d502de7a08b9b690
Parents: aa7882e
Author: Dikang Gu
Authored: Tue Dec 27 11:55:13 2016 -0800
Committer: Dikang Gu
Committed: Thu Jan 5 12:11:14 2017 -0800
--
CHANGES.txt | 3 +-
.../org/apache/cassandra/dht/BootStrapper.java | 18 +++--
.../dht/tokenallocator/TokenAllocation.java | 3 -
.../cassandra/service/StorageService.java | 69 +---
.../apache/cassandra/dht/BootStrapperTest.java | 2 +-
5 files changed, 49 insertions(+), 46 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5f3b3598/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9ed66fd..cacb7cd 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
3.12
- * Avoid byte-array copy when key cache is disabled (CASANDRA-13084)
+ * Use new token allocation for non bootstrap case as well (CASSANDRA-13080)
+ * Avoid byte-array copy when key cache is disabled (CASSANDRA-13084)
* More fixes to the TokenAllocator (CASSANDRA-12990)
* Require forceful decommission if number of nodes is less than replication
factor (CASSANDRA-12510)
* Allow IN restrictions on column families with collections (CASSANDRA-12654)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5f3b3598/src/java/org/apache/cassandra/dht/BootStrapper.java
--
diff --git a/src/java/org/apache/cassandra/dht/BootStrapper.java
b/src/java/org/apache/cassandra/dht/BootStrapper.java
index 392dbf2..1e00f48 100644
--- a/src/java/org/apache/cassandra/dht/BootStrapper.java
+++ b/src/java/org/apache/cassandra/dht/BootStrapper.java
@@ -33,12 +33,15 @@ import org.apache.cassandra.db.TypeSizes;
import org.apache.cassandra.dht.tokenallocator.TokenAllocation;
import org.apache.cassandra.exceptions.ConfigurationException;
import org.apache.cassandra.gms.FailureDetector;
+import org.apache.cassandra.gms.Gossiper;
import org.apache.cassandra.io.IVersionedSerializer;
import org.apache.cassandra.io.util.DataInputPlus;
import org.apache.cassandra.io.util.DataOutputPlus;
import org.apache.cassandra.locator.AbstractReplicationStrategy;
import org.apache.cassandra.locator.TokenMetadata;
+import org.apache.cassandra.service.StorageService;
import org.apache.cassandra.streaming.*;
+import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.progress.ProgressEvent;
import org.apache.cassandra.utils.progress.ProgressEventNotifierSupport;
import org.apache.cassandra.utils.progress.ProgressEventType;
@@ -155,7 +158,7 @@ public class BootStrapper extends
ProgressEventNotifierSupport
* otherwise, if allocationKeyspace is specified use the token allocation
algorithm to generate suitable tokens
* else choose num_tokens tokens at random
*/
-public static Collection getBootstrapTokens(final TokenMetadata
metadata, InetAddress address) throws ConfigurationException
+public static Collection getBootstrapTokens(final TokenMetadata
metadata, InetAddress address, int schemaWaitDelay) throws
ConfigurationException
{
String allocationKeyspace =
DatabaseDescriptor.getAllocateTokensForKeyspace();
Collection initialTokens =
DatabaseDescriptor.getInitialTokens();
@@ -171,7 +174,7 @@ public class BootStrapper extends
ProgressEventNotifierSupport
throw new ConfigurationException("num_tokens must be >= 1");
if (allocationKeyspace != null)
-return allocateTokens(metadata, address, allocationKeyspace,
numTokens);
+return allocateTokens(metadata, address, allocationKeyspace,
numTokens, schemaWaitDelay);
if (numTokens == 1)
logger.warn("Picking random token for a single vnode. You should
probably add more vnodes and/or use the automatic token allocation mechanism.");
@@ -182,7 +185,7 @@ public class BootStrapper extends
ProgressEventNotifierSupport
private static Collection getSpecifiedTokens(final TokenMetadata
metadata,
Collection
initialTokens)
{
-logger.trace("tokens manually specified as {}", initialTokens);
+logger.info("tok
[2/3] cassandra git commit: Use new token allocation for non bootstrap case as well.
Use new token allocation for non bootstrap case as well.
patch by Dikang Gu; reviewed by Branimir Lambov for CASSANSRA-13080
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5f3b3598
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5f3b3598
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5f3b3598
Branch: refs/heads/trunk
Commit: 5f3b35981d28179606c52599d502de7a08b9b690
Parents: aa7882e
Author: Dikang Gu
Authored: Tue Dec 27 11:55:13 2016 -0800
Committer: Dikang Gu
Committed: Thu Jan 5 12:11:14 2017 -0800
--
CHANGES.txt | 3 +-
.../org/apache/cassandra/dht/BootStrapper.java | 18 +++--
.../dht/tokenallocator/TokenAllocation.java | 3 -
.../cassandra/service/StorageService.java | 69 +---
.../apache/cassandra/dht/BootStrapperTest.java | 2 +-
5 files changed, 49 insertions(+), 46 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5f3b3598/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9ed66fd..cacb7cd 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
3.12
- * Avoid byte-array copy when key cache is disabled (CASANDRA-13084)
+ * Use new token allocation for non bootstrap case as well (CASSANDRA-13080)
+ * Avoid byte-array copy when key cache is disabled (CASSANDRA-13084)
* More fixes to the TokenAllocator (CASSANDRA-12990)
* Require forceful decommission if number of nodes is less than replication
factor (CASSANDRA-12510)
* Allow IN restrictions on column families with collections (CASSANDRA-12654)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5f3b3598/src/java/org/apache/cassandra/dht/BootStrapper.java
--
diff --git a/src/java/org/apache/cassandra/dht/BootStrapper.java
b/src/java/org/apache/cassandra/dht/BootStrapper.java
index 392dbf2..1e00f48 100644
--- a/src/java/org/apache/cassandra/dht/BootStrapper.java
+++ b/src/java/org/apache/cassandra/dht/BootStrapper.java
@@ -33,12 +33,15 @@ import org.apache.cassandra.db.TypeSizes;
import org.apache.cassandra.dht.tokenallocator.TokenAllocation;
import org.apache.cassandra.exceptions.ConfigurationException;
import org.apache.cassandra.gms.FailureDetector;
+import org.apache.cassandra.gms.Gossiper;
import org.apache.cassandra.io.IVersionedSerializer;
import org.apache.cassandra.io.util.DataInputPlus;
import org.apache.cassandra.io.util.DataOutputPlus;
import org.apache.cassandra.locator.AbstractReplicationStrategy;
import org.apache.cassandra.locator.TokenMetadata;
+import org.apache.cassandra.service.StorageService;
import org.apache.cassandra.streaming.*;
+import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.progress.ProgressEvent;
import org.apache.cassandra.utils.progress.ProgressEventNotifierSupport;
import org.apache.cassandra.utils.progress.ProgressEventType;
@@ -155,7 +158,7 @@ public class BootStrapper extends
ProgressEventNotifierSupport
* otherwise, if allocationKeyspace is specified use the token allocation
algorithm to generate suitable tokens
* else choose num_tokens tokens at random
*/
-public static Collection getBootstrapTokens(final TokenMetadata
metadata, InetAddress address) throws ConfigurationException
+public static Collection getBootstrapTokens(final TokenMetadata
metadata, InetAddress address, int schemaWaitDelay) throws
ConfigurationException
{
String allocationKeyspace =
DatabaseDescriptor.getAllocateTokensForKeyspace();
Collection initialTokens =
DatabaseDescriptor.getInitialTokens();
@@ -171,7 +174,7 @@ public class BootStrapper extends
ProgressEventNotifierSupport
throw new ConfigurationException("num_tokens must be >= 1");
if (allocationKeyspace != null)
-return allocateTokens(metadata, address, allocationKeyspace,
numTokens);
+return allocateTokens(metadata, address, allocationKeyspace,
numTokens, schemaWaitDelay);
if (numTokens == 1)
logger.warn("Picking random token for a single vnode. You should
probably add more vnodes and/or use the automatic token allocation mechanism.");
@@ -182,7 +185,7 @@ public class BootStrapper extends
ProgressEventNotifierSupport
private static Collection getSpecifiedTokens(final TokenMetadata
metadata,
Collection
initialTokens)
{
-logger.trace("tokens manually specified as {}", initialTokens);
+logger.info("tokens manually specified as {}", initialTokens);
List tokens = new ArrayList<>(initialTokens.size());
for (String tokenString
[jira] [Updated] (CASSANDRA-13103) incorrect jvm metric names
[
https://issues.apache.org/jira/browse/CASSANDRA-13103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alain Rastoul updated CASSANDRA-13103:
--
Description:
Some jvm metrics have a double dot in name like:
jvm.memory..total.max , jvm.memory..total.init (etc).
it seems that an extra dot is added at the end of the name in
CassandraDaemon.java, around line 367 (in 3.0.10):
...
// enable metrics provided by metrics-jvm.jar
CassandraMetricsRegistry.Metrics.register("jvm.buffers.", new
BufferPoolMetricSet(ManagementFactory.getPlatformMBeanServer()));
CassandraMetricsRegistry.Metrics.register("jvm.gc.", new
GarbageCollectorMetricSet());
CassandraMetricsRegistry.Metrics.register("jvm.memory.", new
MemoryUsageGaugeSet());
and also added in append method of MetricRegistry.
Call stack is:
MetricRegistry>>registerAll(String prefix, MetricSet metrics)
MetricRegistry>>static String name(String name, String... names)
MetricRegistry>>static void append(StringBuilder builder, String part)
and in append the dot is also added:
...
if(builder.length() > 0) {
builder.append('.');
}
builder.append(part);
...
The codahale MetricRegistry class seems to have no recent modification of name
or append methods, so it look like a small bug.
May be the fix could be to simply not to add the final dot in the metric name,
ie "jvm.buffers" instead of "jvm.buffers."
was:
Some jvm metrics have a double dot in name like:
jvm.memory..total.max , jvm.memory..total.init (etc).
it seems that an extra dot is added at the end of the name in
CassandraDaemon.java, around line 367 (in 3.0.10):
...
// enable metrics provided by metrics-jvm.jar
CassandraMetricsRegistry.Metrics.register("jvm.buffers.", new
BufferPoolMetricSet(ManagementFactory.getPlatformMBeanServer()));
CassandraMetricsRegistry.Metrics.register("jvm.gc.", new
GarbageCollectorMetricSet());
CassandraMetricsRegistry.Metrics.register("jvm.memory.", new
MemoryUsageGaugeSet());
and also added in append method of MetricRegistry.
Call stack is:
MetricRegistry>>registerAll(String prefix, MetricSet metrics)
MetricRegistry>>static String name(String name, String... names)
MetricRegistry>>static void append(StringBuilder builder, String part)
and in append the dot is also added:
...
if(builder.length() > 0) {
builder.append('.');
}
builder.append(part);
...
The codahale MetricRegistry class seems to have no recent modification of name
or append methods, so it look like a small bug.
May be the fix could be to simply remove the final dot ?
> incorrect jvm metric names
> --
>
> Key: CASSANDRA-13103
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13103
> Project: Cassandra
> Issue Type: Bug
> Components: Observability
>Reporter: Alain Rastoul
>Priority: Minor
>
> Some jvm metrics have a double dot in name like:
> jvm.memory..total.max , jvm.memory..total.init (etc).
> it seems that an extra dot is added at the end of the name in
> CassandraDaemon.java, around line 367 (in 3.0.10):
> ...
> // enable metrics provided by metrics-jvm.jar
> CassandraMetricsRegistry.Metrics.register("jvm.buffers.", new
> BufferPoolMetricSet(ManagementFactory.getPlatformMBeanServer()));
> CassandraMetricsRegistry.Metrics.register("jvm.gc.", new
> GarbageCollectorMetricSet());
> CassandraMetricsRegistry.Metrics.register("jvm.memory.", new
> MemoryUsageGaugeSet());
> and also added in append method of MetricRegistry.
> Call stack is:
> MetricRegistry>>registerAll(String prefix, MetricSet metrics)
> MetricRegistry>>static String name(String name, String... names)
> MetricRegistry>>static void append(StringBuilder builder, String part)
> and in append the dot is also added:
> ...
> if(builder.length() > 0) {
> builder.append('.');
> }
> builder.append(part);
> ...
> The codahale MetricRegistry class seems to have no recent modification of
> name or append methods, so it look like a small bug.
> May be the fix could be to simply not to add the final dot in the metric
> name, ie "jvm.buffers" instead of "jvm.buffers."
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-13103) incorrect jvm metric names
Alain Rastoul created CASSANDRA-13103:
-
Summary: incorrect jvm metric names
Key: CASSANDRA-13103
URL: https://issues.apache.org/jira/browse/CASSANDRA-13103
Project: Cassandra
Issue Type: Bug
Components: Observability
Reporter: Alain Rastoul
Priority: Minor
Some jvm metrics have a double dot in name like:
jvm.memory..total.max , jvm.memory..total.init (etc).
it seems that an extra dot is added at the end of the name in
CassandraDaemon.java, around line 367 (in 3.0.10):
...
// enable metrics provided by metrics-jvm.jar
CassandraMetricsRegistry.Metrics.register("jvm.buffers.", new
BufferPoolMetricSet(ManagementFactory.getPlatformMBeanServer()));
CassandraMetricsRegistry.Metrics.register("jvm.gc.", new
GarbageCollectorMetricSet());
CassandraMetricsRegistry.Metrics.register("jvm.memory.", new
MemoryUsageGaugeSet());
and also added in append method of MetricRegistry.
Call stack is:
MetricRegistry>>registerAll(String prefix, MetricSet metrics)
MetricRegistry>>static String name(String name, String... names)
MetricRegistry>>static void append(StringBuilder builder, String part)
and in append the dot is also added:
...
if(builder.length() > 0) {
builder.append('.');
}
builder.append(part);
...
The codahale MetricRegistry class seems to have no recent modification of name
or append methods, so it look like a small bug.
May be the fix could be to simply remove the final dot ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-12172) Fail to bootstrap new node.
[
https://issues.apache.org/jira/browse/CASSANDRA-12172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jay Zhuang resolved CASSANDRA-12172.
Resolution: Duplicate
Resolving as duplicate to CASSANDRA-12281
> Fail to bootstrap new node.
> ---
>
> Key: CASSANDRA-12172
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12172
> Project: Cassandra
> Issue Type: Bug
>Reporter: Dikang Gu
>
> When I try to bootstrap new node in the cluster, sometimes it failed because
> of following exceptions.
> {code}
> 2016-07-12_05:14:55.58509 INFO 05:14:55 [main]: JOINING: Starting to
> bootstrap...
> 2016-07-12_05:14:56.07491 INFO 05:14:56 [GossipTasks:1]: InetAddress
> /2401:db00:2011:50c7:face:0:9:0 is now DOWN
> 2016-07-12_05:14:56.32219 Exception (java.lang.RuntimeException) encountered
> during startup: A node required to move the data consistently is down
> (/2401:db00:2011:50c7:face:0:9:0). If you wish to move the data from a
> potentially inconsis
> tent replica, restart the node with -Dcassandra.consistent.rangemovement=false
> 2016-07-12_05:14:56.32582 ERROR 05:14:56 [main]: Exception encountered during
> startup
> 2016-07-12_05:14:56.32583 java.lang.RuntimeException: A node required to move
> the data consistently is down (/2401:db00:2011:50c7:face:0:9:0). If you wish
> to move the data from a potentially inconsistent replica, restart the node
> with -Dc
> assandra.consistent.rangemovement=false
> 2016-07-12_05:14:56.32584 at
> org.apache.cassandra.dht.RangeStreamer.getAllRangesWithStrictSourcesFor(RangeStreamer.java:264)
> ~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
> 2016-07-12_05:14:56.32584 at
> org.apache.cassandra.dht.RangeStreamer.addRanges(RangeStreamer.java:147)
> ~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
> 2016-07-12_05:14:56.32584 at
> org.apache.cassandra.dht.BootStrapper.bootstrap(BootStrapper.java:82)
> ~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
> 2016-07-12_05:14:56.32584 at
> org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1230)
> ~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
> 2016-07-12_05:14:56.32584 at
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:924)
> ~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
> 2016-07-12_05:14:56.32585 at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:709)
> ~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
> 2016-07-12_05:14:56.32585 at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:585)
> ~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
> 2016-07-12_05:14:56.32585 at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:300)
> [apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
> 2016-07-12_05:14:56.32586 at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:516)
> [apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
> 2016-07-12_05:14:56.32586 at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:625)
> [apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
> 2016-07-12_05:14:56.32730 WARN 05:14:56 [StorageServiceShutdownHook]: No
> local state or state is in silent shutdown, not announcing shutdown
> {code}
> Here are more logs:
> https://gist.github.com/DikangGu/c6a83eafdbc091250eade4a3bddcc40b
> I'm pretty sure there are no DOWN nodes or restarted nodes in the cluster,
> but I still see a lot of nodes UP and DOWN in the gossip log, which failed
> the bootstrap at the end, is this a known bug?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13095) Timeouts between nodes
[
https://issues.apache.org/jira/browse/CASSANDRA-13095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802087#comment-15802087
]
Jay Zhuang commented on CASSANDRA-13095:
Nice finding. [~aweisberg] could you please review?
I'm not familiar with that. Tried the fix locally, looks good to me. Minor
comment: can we add check to make sure {{sleep}} is larger than 0?
https://github.com/apache/cassandra/blob/a8a43dd32eb92406d7d8b105e08c68b3d5c7df49/src/java/org/apache/cassandra/utils/CoalescingStrategies.java#L109
> Timeouts between nodes
> --
>
> Key: CASSANDRA-13095
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13095
> Project: Cassandra
> Issue Type: Bug
>Reporter: Danil Smirnov
>Priority: Minor
> Attachments: 13095-2.1.patch
>
>
> Recently I've run into a problem with heavily loaded cluster when sometimes
> messages between certain nodes become blocked with no reason.
> It looks like the same situation that described here
> https://issues.apache.org/jira/browse/CASSANDRA-12676?focusedCommentId=15736166&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15736166
> Thread dump showed infinite loop here:
> https://github.com/apache/cassandra/blob/a8a43dd32eb92406d7d8b105e08c68b3d5c7df49/src/java/org/apache/cassandra/utils/CoalescingStrategies.java#L109
> Apparently the problem is in the initial value of epoch filed in
> TimeHorizonMovingAverageCoalescingStrategy class. When it's value is not
> evenly divisible by BUCKET_INTERVAL, ix(epoch-1) does not point to the
> correct bucket. As a result, sum gradually increases and, upon reaching
> MEASURED_INTERVAL, averageGap becomes 0 and thread blocks.
> It's hard to reproduce because it takes a long time for sum to grow and when
> no messages are send for some time, sum becomes 0
> https://github.com/apache/cassandra/blob/a8a43dd32eb92406d7d8b105e08c68b3d5c7df49/src/java/org/apache/cassandra/utils/CoalescingStrategies.java#L301
> and bug is no longer reproducible (until connection between nodes is
> re-created).
> I've added a patch which should fix the problem. Don't know if it would be of
> any help since CASSANDRA-12676 will apparently disable this behaviour. One
> note about performance regressions though. There is a small chance it being
> result of the bug described here, so it might be worth testing performance
> after fixes and/or tuning the algorithm.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-13083) Fix dtest for CASSANDRA-13080
[ https://issues.apache.org/jira/browse/CASSANDRA-13083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dikang Gu resolved CASSANDRA-13083. --- Resolution: Fixed the pull request is merged. > Fix dtest for CASSANDRA-13080 > - > > Key: CASSANDRA-13083 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13083 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Dikang Gu >Assignee: Dikang Gu > Fix For: 3.12 > > > Fix the dtest failures. > https://cassci.datastax.com/view/Dev/view/DikangGu/job/DikangGu-CASSANDRA-13080-trunk-dtest/5/testReport/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12794) COPY FROM with NULL='' fails when inserting empty row in primary key
[
https://issues.apache.org/jira/browse/CASSANDRA-12794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tyler Hobbs updated CASSANDRA-12794:
Status: Ready to Commit (was: Patch Available)
> COPY FROM with NULL='' fails when inserting empty row in primary key
> -
>
> Key: CASSANDRA-12794
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12794
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: Tested using C* 2.1.15
>Reporter: Sucwinder Bassi
>Assignee: Stefania
> Fix For: 3.0.x, 3.x
>
>
> Using this table:
> {noformat}
> CREATE TABLE testtab ( a_id text, b_id text, c_id text, d_id text,
> order_id uuid, acc_id bigint, bucket bigint, r_id text, ts bigint,
> PRIMARY KEY ((a_id, b_id, c_id, d_id), order_id));
> {noformat}
> insert one row:
> {noformat}
> INSERT INTO testtab (a_id, b_id , c_id , d_id , order_id, r_id ) VALUES ( '',
> '', '', 'a1', 645e7d3c-aef7-4e3c-b834-24b792cf2e55, 'r1');
> {noformat}
> Use COPY to dump the row to temp.csv:
> {noformat}
> copy testtab TO 'temp.csv';
> {noformat}
> Which creates this file:
> {noformat}
> $ cat temp.csv
> ,,,a1,645e7d3c-aef7-4e3c-b834-24b792cf2e55,,,r1,
> {noformat}
> Truncate the testtab table and then use copy from with NULL='' to insert the
> row:
> {noformat}
> cqlsh:sbkeyspace> COPY testtab FROM 'temp.csv' with NULL='';
> Using 1 child processes
> Starting copy of sbkeyspace.testtab with columns ['a_id', 'b_id', 'c_id',
> 'd_id', 'order_id', 'acc_id', 'bucket', 'r_id', 'ts'].
> Failed to import 1 rows: ParseError - Cannot insert null value for primary
> key column 'a_id'. If you want to insert empty strings, consider using the
> WITH NULL= option for COPY., given up without retries
> Failed to process 1 rows; failed rows written to import_sbkeyspace_testtab.err
> Processed: 1 rows; Rate: 2 rows/s; Avg. rate: 3 rows/s
> 1 rows imported from 1 files in 0.398 seconds (0 skipped).
> {noformat}
> It shows 1 rows inserted, but the table is empty:
> {noformat}
> select * from testtab ;
> a_id | b_id | c_id | d_id | order_id | acc_id | bucket | r_id | ts
> --+--+--+--+--+++--+
> (0 rows)
> {noformat}
> The same error is returned even without the with NULL=''. Is it actually
> possible for copy from to insert an empty row into the primary key? The
> insert command shown above inserts the empty row for the primary key without
> any problems.
> Is this related to https://issues.apache.org/jira/browse/CASSANDRA-7792?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12794) COPY FROM with NULL='' fails when inserting empty row in primary key
[
https://issues.apache.org/jira/browse/CASSANDRA-12794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15801956#comment-15801956
]
Tyler Hobbs commented on CASSANDRA-12794:
-
I think this is a good solution. +1 on committing, the patch and tests look
good.
> COPY FROM with NULL='' fails when inserting empty row in primary key
> -
>
> Key: CASSANDRA-12794
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12794
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: Tested using C* 2.1.15
>Reporter: Sucwinder Bassi
>Assignee: Stefania
> Fix For: 3.0.x, 3.x
>
>
> Using this table:
> {noformat}
> CREATE TABLE testtab ( a_id text, b_id text, c_id text, d_id text,
> order_id uuid, acc_id bigint, bucket bigint, r_id text, ts bigint,
> PRIMARY KEY ((a_id, b_id, c_id, d_id), order_id));
> {noformat}
> insert one row:
> {noformat}
> INSERT INTO testtab (a_id, b_id , c_id , d_id , order_id, r_id ) VALUES ( '',
> '', '', 'a1', 645e7d3c-aef7-4e3c-b834-24b792cf2e55, 'r1');
> {noformat}
> Use COPY to dump the row to temp.csv:
> {noformat}
> copy testtab TO 'temp.csv';
> {noformat}
> Which creates this file:
> {noformat}
> $ cat temp.csv
> ,,,a1,645e7d3c-aef7-4e3c-b834-24b792cf2e55,,,r1,
> {noformat}
> Truncate the testtab table and then use copy from with NULL='' to insert the
> row:
> {noformat}
> cqlsh:sbkeyspace> COPY testtab FROM 'temp.csv' with NULL='';
> Using 1 child processes
> Starting copy of sbkeyspace.testtab with columns ['a_id', 'b_id', 'c_id',
> 'd_id', 'order_id', 'acc_id', 'bucket', 'r_id', 'ts'].
> Failed to import 1 rows: ParseError - Cannot insert null value for primary
> key column 'a_id'. If you want to insert empty strings, consider using the
> WITH NULL= option for COPY., given up without retries
> Failed to process 1 rows; failed rows written to import_sbkeyspace_testtab.err
> Processed: 1 rows; Rate: 2 rows/s; Avg. rate: 3 rows/s
> 1 rows imported from 1 files in 0.398 seconds (0 skipped).
> {noformat}
> It shows 1 rows inserted, but the table is empty:
> {noformat}
> select * from testtab ;
> a_id | b_id | c_id | d_id | order_id | acc_id | bucket | r_id | ts
> --+--+--+--+--+++--+
> (0 rows)
> {noformat}
> The same error is returned even without the with NULL=''. Is it actually
> possible for copy from to insert an empty row into the primary key? The
> insert command shown above inserts the empty row for the primary key without
> any problems.
> Is this related to https://issues.apache.org/jira/browse/CASSANDRA-7792?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12794) COPY FROM with NULL='' fails when inserting empty row in primary key
[
https://issues.apache.org/jira/browse/CASSANDRA-12794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tyler Hobbs updated CASSANDRA-12794:
Description:
Using this table:
{noformat}
CREATE TABLE testtab ( a_id text, b_id text, c_id text, d_id text,
order_id uuid, acc_id bigint, bucket bigint, r_id text, ts bigint, PRIMARY
KEY ((a_id, b_id, c_id, d_id), order_id));
{noformat}
insert one row:
{noformat}
INSERT INTO testtab (a_id, b_id , c_id , d_id , order_id, r_id ) VALUES ( '',
'', '', 'a1', 645e7d3c-aef7-4e3c-b834-24b792cf2e55, 'r1');
{noformat}
Use COPY to dump the row to temp.csv:
{noformat}
copy testtab TO 'temp.csv';
{noformat}
Which creates this file:
{noformat}
$ cat temp.csv
,,,a1,645e7d3c-aef7-4e3c-b834-24b792cf2e55,,,r1,
{noformat}
Truncate the testtab table and then use copy from with NULL='' to insert the
row:
{noformat}
cqlsh:sbkeyspace> COPY testtab FROM 'temp.csv' with NULL='';
Using 1 child processes
Starting copy of sbkeyspace.testtab with columns ['a_id', 'b_id', 'c_id',
'd_id', 'order_id', 'acc_id', 'bucket', 'r_id', 'ts'].
Failed to import 1 rows: ParseError - Cannot insert null value for primary key
column 'a_id'. If you want to insert empty strings, consider using the WITH
NULL= option for COPY., given up without retries
Failed to process 1 rows; failed rows written to import_sbkeyspace_testtab.err
Processed: 1 rows; Rate: 2 rows/s; Avg. rate: 3 rows/s
1 rows imported from 1 files in 0.398 seconds (0 skipped).
{noformat}
It shows 1 rows inserted, but the table is empty:
{noformat}
select * from testtab ;
a_id | b_id | c_id | d_id | order_id | acc_id | bucket | r_id | ts
--+--+--+--+--+++--+
(0 rows)
{noformat}
The same error is returned even without the with NULL=''. Is it actually
possible for copy from to insert an empty row into the primary key? The insert
command shown above inserts the empty row for the primary key without any
problems.
Is this related to https://issues.apache.org/jira/browse/CASSANDRA-7792?
was:
Using this table:
CREATE TABLE testtab ( a_id text, b_id text, c_id text, d_id text,
order_id uuid, acc_id bigint, bucket bigint, r_id text, ts bigint, PRIMARY
KEY ((a_id, b_id, c_id, d_id), order_id));
insert one row:
INSERT INTO testtab (a_id, b_id , c_id , d_id , order_id, r_id ) VALUES ( '',
'', '', 'a1', 645e7d3c-aef7-4e3c-b834-24b792cf2e55, 'r1');
Use COPY to dump the row to temp.csv:
copy testtab TO 'temp.csv';
Which creates this file:
$ cat temp.csv
,,,a1,645e7d3c-aef7-4e3c-b834-24b792cf2e55,,,r1,
Truncate the testtab table and then use copy from with NULL='' to insert the
row:
cqlsh:sbkeyspace> COPY testtab FROM 'temp.csv' with NULL='';
Using 1 child processes
Starting copy of sbkeyspace.testtab with columns ['a_id', 'b_id', 'c_id',
'd_id', 'order_id', 'acc_id', 'bucket', 'r_id', 'ts'].
Failed to import 1 rows: ParseError - Cannot insert null value for primary key
column 'a_id'. If you want to insert empty strings, consider using the WITH
NULL= option for COPY., given up without retries
Failed to process 1 rows; failed rows written to import_sbkeyspace_testtab.err
Processed: 1 rows; Rate: 2 rows/s; Avg. rate: 3 rows/s
1 rows imported from 1 files in 0.398 seconds (0 skipped).
It shows 1 rows inserted, but the table is empty:
select * from testtab ;
a_id | b_id | c_id | d_id | order_id | acc_id | bucket | r_id | ts
--+--+--+--+--+++--+
(0 rows)
The same error is returned even without the with NULL=''. Is it actually
possible for copy from to insert an empty row into the primary key? The insert
command shown above inserts the empty row for the primary key without any
problems.
Is this related to https://issues.apache.org/jira/browse/CASSANDRA-7792?
> COPY FROM with NULL='' fails when inserting empty row in primary key
> -
>
> Key: CASSANDRA-12794
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12794
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: Tested using C* 2.1.15
>Reporter: Sucwinder Bassi
>Assignee: Stefania
> Fix For: 3.0.x, 3.x
>
>
> Using this table:
> {noformat}
> CREATE TABLE testtab ( a_id text, b_id text, c_id text, d_id text,
> order_id uuid, acc_id bigint, bucket bigint, r_id text, ts bigint,
> PRIMARY KEY ((a_id, b_id, c_id, d_id), order_id));
> {noformat}
> insert one row:
> {noformat}
> INSERT INTO testtab (a_id, b_id , c_id , d_id , order_id, r_id ) VALUES ( '',
> '', '', 'a1', 645e7d3c-aef7-4e3c-b834-24b792cf2e55, 'r1');
> {noformat}
> Use COPY to dump the row to temp.csv:
> {noformat}
> copy testtab TO 'temp.csv';
> {noformat}
> Which creates this file:
>
[jira] [Created] (CASSANDRA-13102) test failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_bulk_round_trip_blogposts
Michael Shuler created CASSANDRA-13102:
--
Summary: test failure in
cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_bulk_round_trip_blogposts
Key: CASSANDRA-13102
URL: https://issues.apache.org/jira/browse/CASSANDRA-13102
Project: Cassandra
Issue Type: Bug
Reporter: Michael Shuler
example failure:
http://cassci.datastax.com/job/cassandra-3.0_dtest/875/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_bulk_round_trip_blogposts
{noformat}
Error Message
('Unable to connect to any servers', {'127.0.0.1': error(111, "Tried connecting
to [('127.0.0.1', 9042)]. Last error: Connection refused")})
>> begin captured logging <<
dtest: DEBUG: cluster ccm directory: /tmp/dtest-1c8hQM
dtest: DEBUG: Done setting configuration options:
{ 'initial_token': None,
'num_tokens': '32',
'phi_convict_threshold': 5,
'range_request_timeout_in_ms': 1,
'read_request_timeout_in_ms': 1,
'request_timeout_in_ms': 1,
'truncate_request_timeout_in_ms': 1,
'write_request_timeout_in_ms': 1}
dtest: DEBUG: removing ccm cluster test at: /tmp/dtest-1c8hQM
dtest: DEBUG: clearing ssl stores from [/tmp/dtest-1c8hQM] directory
dtest: DEBUG: cluster ccm directory: /tmp/dtest-C2OXmx
dtest: DEBUG: Done setting configuration options:
{ 'initial_token': None,
'num_tokens': '32',
'phi_convict_threshold': 5,
'range_request_timeout_in_ms': 1,
'read_request_timeout_in_ms': 1,
'request_timeout_in_ms': 1,
'truncate_request_timeout_in_ms': 1,
'write_request_timeout_in_ms': 1}
- >> end captured logging << -
Stacktrace
File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
File "/home/automaton/cassandra-dtest/dtest.py", line 1079, in wrapped
f(obj)
File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", line
2553, in test_bulk_round_trip_blogposts
stress_table='stresscql.blogposts')
File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", line
2456, in _test_bulk_round_trip
self.prepare(nodes=nodes, partitioner=partitioner,
configuration_options=configuration_options)
File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", line
129, in prepare
self.session = self.patient_cql_connection(self.node1)
File "/home/automaton/cassandra-dtest/dtest.py", line 508, in
patient_cql_connection
bypassed_exception=NoHostAvailable
File "/home/automaton/cassandra-dtest/dtest.py", line 201, in
retry_till_success
return fun(*args, **kwargs)
File "/home/automaton/cassandra-dtest/dtest.py", line 441, in cql_connection
protocol_version, port=port, ssl_opts=ssl_opts)
File "/home/automaton/cassandra-dtest/dtest.py", line 469, in _create_session
session = cluster.connect(wait_for_all_pools=True)
File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line 1180,
in connect
self.control_connection.connect()
File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line 2597,
in connect
self._set_new_connection(self._reconnect_internal())
File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line 2634,
in _reconnect_internal
raise NoHostAvailable("Unable to connect to any servers", errors)
'(\'Unable to connect to any servers\', {\'127.0.0.1\': error(111, "Tried
connecting to [(\'127.0.0.1\', 9042)]. Last error: Connection
refused")})\n >> begin captured logging <<
\ndtest: DEBUG: cluster ccm directory:
/tmp/dtest-1c8hQM\ndtest: DEBUG: Done setting configuration options:\n{
\'initial_token\': None,\n\'num_tokens\': \'32\',\n
\'phi_convict_threshold\': 5,\n\'range_request_timeout_in_ms\': 1,\n
\'read_request_timeout_in_ms\': 1,\n\'request_timeout_in_ms\': 1,\n
\'truncate_request_timeout_in_ms\': 1,\n
\'write_request_timeout_in_ms\': 1}\ndtest: DEBUG: removing ccm cluster
test at: /tmp/dtest-1c8hQM\ndtest: DEBUG: clearing ssl stores from
[/tmp/dtest-1c8hQM] directory\ndtest: DEBUG: cluster ccm directory:
/tmp/dtest-C2OXmx\ndtest: DEBUG: Done setting configuration options:\n{
\'initial_token\': None,\n\'num_tokens\': \'32\',\n
\'phi_convict_threshold\': 5,\n\'range_request_timeout_in_ms\': 1,\n
\'read_request_timeout_in_ms\': 1,\n\'request_timeout_in_ms\': 1,\n
\'truncate_request_timeout_in_ms\': 1,\n
\'write_request_timeout_in_ms\': 1}\n- >> end captured
logging << -'
{noformat}
(No node log file from test)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12890) testall failure in org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetadata
[
https://issues.apache.org/jira/browse/CASSANDRA-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuki Morishita updated CASSANDRA-12890:
---
Status: Ready to Commit (was: Patch Available)
> testall failure in
> org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetadata
> ---
>
> Key: CASSANDRA-12890
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12890
> Project: Cassandra
> Issue Type: Test
>Reporter: Michael Shuler
>Assignee: Sylvain Lebresne
> Labels: test-failure
> Fix For: 2.2.x
>
>
> This failed in both 'test' and 'test-compression' targets.
> example failure:
> http://cassci.datastax.com/job/cassandra-2.2_testall/602/testReport/org.apache.cassandra.db/ColumnFamilyStoreTest/testSliceByNamesCommandOldMetadata
> http://cassci.datastax.com/job/cassandra-2.2_testall/602/testReport/org.apache.cassandra.db/ColumnFamilyStoreTest/testSliceByNamesCommandOldMetadata_compression/
> {noformat}
> Stacktrace
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:171)
> at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:166)
> at
> org.apache.cassandra.io.sstable.format.SSTableWriter.rename(SSTableWriter.java:266)
> at
> org.apache.cassandra.db.ColumnFamilyStore.loadNewSSTables(ColumnFamilyStore.java:791)
> at
> org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetadata(ColumnFamilyStoreTest.java:1158)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12890) testall failure in org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetadata
[
https://issues.apache.org/jira/browse/CASSANDRA-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15801840#comment-15801840
]
Yuki Morishita commented on CASSANDRA-12890:
+1
> testall failure in
> org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetadata
> ---
>
> Key: CASSANDRA-12890
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12890
> Project: Cassandra
> Issue Type: Test
>Reporter: Michael Shuler
>Assignee: Sylvain Lebresne
> Labels: test-failure
> Fix For: 2.2.x
>
>
> This failed in both 'test' and 'test-compression' targets.
> example failure:
> http://cassci.datastax.com/job/cassandra-2.2_testall/602/testReport/org.apache.cassandra.db/ColumnFamilyStoreTest/testSliceByNamesCommandOldMetadata
> http://cassci.datastax.com/job/cassandra-2.2_testall/602/testReport/org.apache.cassandra.db/ColumnFamilyStoreTest/testSliceByNamesCommandOldMetadata_compression/
> {noformat}
> Stacktrace
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:171)
> at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:166)
> at
> org.apache.cassandra.io.sstable.format.SSTableWriter.rename(SSTableWriter.java:266)
> at
> org.apache.cassandra.db.ColumnFamilyStore.loadNewSSTables(ColumnFamilyStore.java:791)
> at
> org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetadata(ColumnFamilyStoreTest.java:1158)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-13052) Repair process is violating the start/end token limits for small ranges
[
https://issues.apache.org/jira/browse/CASSANDRA-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Podkowinski updated CASSANDRA-13052:
---
Status: Patch Available (was: In Progress)
> Repair process is violating the start/end token limits for small ranges
> ---
>
> Key: CASSANDRA-13052
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13052
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
> Environment: We tried this in 2.0.14 and 3.9, same bug.
>Reporter: Cristian P
>Assignee: Stefan Podkowinski
> Attachments: 13052-2.1.patch, ccm_reproduce-13052.txt,
> system-dev-debug-13052.log
>
>
> We tried to do a single token repair by providing 2 consecutive token values
> for a large column family. We soon notice heavy streaming and according to
> the logs the number of ranges streamed was in thousands.
> After investigation we found a bug in the two partitioner classes we use
> (RandomPartitioner and Murmur3Partitioner).
> The midpoint method used by MerkleTree.differenceHelper method to find ranges
> with differences for streaming returns abnormal values (way out of the
> initial range requested for repair) if the repair requested range is small (I
> expect smaller than 2^15).
> Here is the simple code to reproduce the bug for Murmur3Partitioner:
> Token left = new Murmur3Partitioner.LongToken(123456789L);
> Token right = new Murmur3Partitioner.LongToken(123456789L);
> IPartitioner partitioner = new Murmur3Partitioner();
> Token midpoint = partitioner.midpoint(left, right);
> System.out.println("Murmur3: [ " + left.getToken() + " : " +
> midpoint.getToken() + " : " + right.getToken() + " ]");
> The output is:
> Murmur3: [ 123456789 : -9223372036731319019 : 123456789 ]
> Note that the midpoint token is nowhere near the suggested repair range. This
> will happen if during the parsing of the tree (in
> MerkleTree.differenceHelper) in search for differences there isn't enough
> tokens for the split and the subrange becomes 0 (left.token=right.token) as
> in the above test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13052) Repair process is violating the start/end token limits for small ranges
[
https://issues.apache.org/jira/browse/CASSANDRA-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15801665#comment-15801665
]
Stefan Podkowinski commented on CASSANDRA-13052:
I've started tests for my attached patch.
||2.1||2.2||3.0||3.x||trunk||
|[branch|https://github.com/spodkowinski/cassandra/tree/CASSANDRA-13052-2.1]|[branch|https://github.com/spodkowinski/cassandra/tree/CASSANDRA-13052-2.2]|[branch|https://github.com/spodkowinski/cassandra/tree/CASSANDRA-13052-3.0]|[branch|https://github.com/spodkowinski/cassandra/tree/CASSANDRA-13052-3.x]|[branch|https://github.com/spodkowinski/cassandra/tree/CASSANDRA-13052-trunk]|
|[dtest|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-13052-2.1-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-13052-2.2-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-13052-3.0-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-13052-3.x-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-13052-trunk-dtest/]|
|[testall|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-13052-2.1-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-13052-2.2-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-13052-3.0-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-13052-3.x-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-13052-trunk-testall/]|
Happy to contribute a dtest as well if someone would step up reviewing the
patch.
> Repair process is violating the start/end token limits for small ranges
> ---
>
> Key: CASSANDRA-13052
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13052
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
> Environment: We tried this in 2.0.14 and 3.9, same bug.
>Reporter: Cristian P
>Assignee: Stefan Podkowinski
> Attachments: 13052-2.1.patch, ccm_reproduce-13052.txt,
> system-dev-debug-13052.log
>
>
> We tried to do a single token repair by providing 2 consecutive token values
> for a large column family. We soon notice heavy streaming and according to
> the logs the number of ranges streamed was in thousands.
> After investigation we found a bug in the two partitioner classes we use
> (RandomPartitioner and Murmur3Partitioner).
> The midpoint method used by MerkleTree.differenceHelper method to find ranges
> with differences for streaming returns abnormal values (way out of the
> initial range requested for repair) if the repair requested range is small (I
> expect smaller than 2^15).
> Here is the simple code to reproduce the bug for Murmur3Partitioner:
> Token left = new Murmur3Partitioner.LongToken(123456789L);
> Token right = new Murmur3Partitioner.LongToken(123456789L);
> IPartitioner partitioner = new Murmur3Partitioner();
> Token midpoint = partitioner.midpoint(left, right);
> System.out.println("Murmur3: [ " + left.getToken() + " : " +
> midpoint.getToken() + " : " + right.getToken() + " ]");
> The output is:
> Murmur3: [ 123456789 : -9223372036731319019 : 123456789 ]
> Note that the midpoint token is nowhere near the suggested repair range. This
> will happen if during the parsing of the tree (in
> MerkleTree.differenceHelper) in search for differences there isn't enough
> tokens for the split and the subrange becomes 0 (left.token=right.token) as
> in the above test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12203) AssertionError on compaction after upgrade (2.1.9 -> 3.7)
[
https://issues.apache.org/jira/browse/CASSANDRA-12203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuki Morishita updated CASSANDRA-12203:
---
Assignee: Yuki Morishita
Reproduced In: 3.7, 3.0.10 (was: 3.0.10, 3.7)
Status: Patch Available (was: Open)
> AssertionError on compaction after upgrade (2.1.9 -> 3.7)
> -
>
> Key: CASSANDRA-12203
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12203
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 3.7 (upgrade from 2.1.9)
> Java version "1.8.0_91"
> Ubuntu 14.04.4 LTS (GNU/Linux 3.13.0-83-generic x86_64)
>Reporter: Roman S. Borschel
>Assignee: Yuki Morishita
>Priority: Critical
> Fix For: 3.0.x, 3.x
>
>
> After upgrading a Cassandra cluster from 2.1.9 to 3.7, one column family
> (using SizeTieredCompaction) repeatedly and continuously failed compaction
> (and thus also repair) across the cluster, with all nodes producing the
> following errors in the logs:
> {noformat}
> 016-07-14T09:29:47.96855 |srv=cassandra|ERROR: Exception in thread
> Thread[CompactionExecutor:3,1,main]
> 2016-07-14T09:29:47.96858 |srv=cassandra|java.lang.AssertionError: null
> 2016-07-14T09:29:47.96859 |srv=cassandra| at
> org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$TombstoneTracker.openNew(UnfilteredDeserializer.java:650)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96860 |srv=cassandra| at
> org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$UnfilteredIterator.hasNext(UnfilteredDeserializer.java:423)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96860 |srv=cassandra| at
> org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer.hasNext(UnfilteredDeserializer.java:298)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96860 |srv=cassandra| at
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$OldFormatIterator.readStaticRow(SSTableSimpleIterator.java:133)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96861 |srv=cassandra| at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.(SSTableIdentityIterator.java:57)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96861 |srv=cassandra| at
> org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator$1.initializeIterator(BigTableScanner.java:334)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96862 |srv=cassandra| at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.maybeInit(LazilyInitializedUnfilteredRowIterator.java:48)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96862 |srv=cassandra| at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.isReverseOrder(LazilyInitializedUnfilteredRowIterator.java:70)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96863 |srv=cassandra| at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$1.reduce(UnfilteredPartitionIterators.java:109)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96863 |srv=cassandra| at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$1.reduce(UnfilteredPartitionIterators.java:100)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96864 |srv=cassandra| at
> org.apache.cassandra.utils.MergeIterator$Candidate.consume(MergeIterator.java:408)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96864 |srv=cassandra| at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:203)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96865 |srv=cassandra| at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96865 |srv=cassandra| at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96866 |srv=cassandra| at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$2.hasNext(UnfilteredPartitionIterators.java:150)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96866 |srv=cassandra| at
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:72)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96867 |srv=cassandra| at
> org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:226)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96867 |srv=cassandra| at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:182)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96867 |srv=cassandra| at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[
[jira] [Commented] (CASSANDRA-12203) AssertionError on compaction after upgrade (2.1.9 -> 3.7)
[
https://issues.apache.org/jira/browse/CASSANDRA-12203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15801620#comment-15801620
]
Yuki Morishita commented on CASSANDRA-12203:
Here is a trial patch.
||branch||testall||dtest||
|[12203-3.0|https://github.com/yukim/cassandra/tree/12203-3.0]|[testall|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-12203-3.0-testall/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-12203-3.0-dtest/lastCompletedBuild/testReport/]|
I also added test to dtest's upgrade test:
https://github.com/yukim/cassandra-dtest/commit/6a86d2f70d2c4bed03ed24eeef3161f5e21f44ef
> AssertionError on compaction after upgrade (2.1.9 -> 3.7)
> -
>
> Key: CASSANDRA-12203
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12203
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 3.7 (upgrade from 2.1.9)
> Java version "1.8.0_91"
> Ubuntu 14.04.4 LTS (GNU/Linux 3.13.0-83-generic x86_64)
>Reporter: Roman S. Borschel
>Priority: Critical
> Fix For: 3.0.x, 3.x
>
>
> After upgrading a Cassandra cluster from 2.1.9 to 3.7, one column family
> (using SizeTieredCompaction) repeatedly and continuously failed compaction
> (and thus also repair) across the cluster, with all nodes producing the
> following errors in the logs:
> {noformat}
> 016-07-14T09:29:47.96855 |srv=cassandra|ERROR: Exception in thread
> Thread[CompactionExecutor:3,1,main]
> 2016-07-14T09:29:47.96858 |srv=cassandra|java.lang.AssertionError: null
> 2016-07-14T09:29:47.96859 |srv=cassandra| at
> org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$TombstoneTracker.openNew(UnfilteredDeserializer.java:650)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96860 |srv=cassandra| at
> org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$UnfilteredIterator.hasNext(UnfilteredDeserializer.java:423)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96860 |srv=cassandra| at
> org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer.hasNext(UnfilteredDeserializer.java:298)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96860 |srv=cassandra| at
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$OldFormatIterator.readStaticRow(SSTableSimpleIterator.java:133)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96861 |srv=cassandra| at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.(SSTableIdentityIterator.java:57)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96861 |srv=cassandra| at
> org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator$1.initializeIterator(BigTableScanner.java:334)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96862 |srv=cassandra| at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.maybeInit(LazilyInitializedUnfilteredRowIterator.java:48)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96862 |srv=cassandra| at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.isReverseOrder(LazilyInitializedUnfilteredRowIterator.java:70)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96863 |srv=cassandra| at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$1.reduce(UnfilteredPartitionIterators.java:109)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96863 |srv=cassandra| at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$1.reduce(UnfilteredPartitionIterators.java:100)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96864 |srv=cassandra| at
> org.apache.cassandra.utils.MergeIterator$Candidate.consume(MergeIterator.java:408)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96864 |srv=cassandra| at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:203)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96865 |srv=cassandra| at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96865 |srv=cassandra| at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96866 |srv=cassandra| at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$2.hasNext(UnfilteredPartitionIterators.java:150)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96866 |srv=cassandra| at
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:72)
> ~[apache-cassandra-3.7.jar:3.7]
> 2016-07-14T09:29:47.96867 |srv=cassandra| at
> org.apache.cassandra.db.compaction.CompactionIterator.hasNext(Com
[jira] [Updated] (CASSANDRA-13052) Repair process is violating the start/end token limits for small ranges
[
https://issues.apache.org/jira/browse/CASSANDRA-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Podkowinski updated CASSANDRA-13052:
---
Attachment: 13052-2.1.patch
> Repair process is violating the start/end token limits for small ranges
> ---
>
> Key: CASSANDRA-13052
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13052
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
> Environment: We tried this in 2.0.14 and 3.9, same bug.
>Reporter: Cristian P
>Assignee: Stefan Podkowinski
> Attachments: 13052-2.1.patch, ccm_reproduce-13052.txt,
> system-dev-debug-13052.log
>
>
> We tried to do a single token repair by providing 2 consecutive token values
> for a large column family. We soon notice heavy streaming and according to
> the logs the number of ranges streamed was in thousands.
> After investigation we found a bug in the two partitioner classes we use
> (RandomPartitioner and Murmur3Partitioner).
> The midpoint method used by MerkleTree.differenceHelper method to find ranges
> with differences for streaming returns abnormal values (way out of the
> initial range requested for repair) if the repair requested range is small (I
> expect smaller than 2^15).
> Here is the simple code to reproduce the bug for Murmur3Partitioner:
> Token left = new Murmur3Partitioner.LongToken(123456789L);
> Token right = new Murmur3Partitioner.LongToken(123456789L);
> IPartitioner partitioner = new Murmur3Partitioner();
> Token midpoint = partitioner.midpoint(left, right);
> System.out.println("Murmur3: [ " + left.getToken() + " : " +
> midpoint.getToken() + " : " + right.getToken() + " ]");
> The output is:
> Murmur3: [ 123456789 : -9223372036731319019 : 123456789 ]
> Note that the midpoint token is nowhere near the suggested repair range. This
> will happen if during the parsing of the tree (in
> MerkleTree.differenceHelper) in search for differences there isn't enough
> tokens for the split and the subrange becomes 0 (left.token=right.token) as
> in the above test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12890) testall failure in org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetadata
[
https://issues.apache.org/jira/browse/CASSANDRA-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-12890:
Reviewer: Yuki Morishita
> testall failure in
> org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetadata
> ---
>
> Key: CASSANDRA-12890
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12890
> Project: Cassandra
> Issue Type: Test
>Reporter: Michael Shuler
>Assignee: Sylvain Lebresne
> Labels: test-failure
> Fix For: 2.2.x
>
>
> This failed in both 'test' and 'test-compression' targets.
> example failure:
> http://cassci.datastax.com/job/cassandra-2.2_testall/602/testReport/org.apache.cassandra.db/ColumnFamilyStoreTest/testSliceByNamesCommandOldMetadata
> http://cassci.datastax.com/job/cassandra-2.2_testall/602/testReport/org.apache.cassandra.db/ColumnFamilyStoreTest/testSliceByNamesCommandOldMetadata_compression/
> {noformat}
> Stacktrace
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:171)
> at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:166)
> at
> org.apache.cassandra.io.sstable.format.SSTableWriter.rename(SSTableWriter.java:266)
> at
> org.apache.cassandra.db.ColumnFamilyStore.loadNewSSTables(ColumnFamilyStore.java:791)
> at
> org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetadata(ColumnFamilyStoreTest.java:1158)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-13069) Local batchlog for MV may not be correctly written on node movements
[
https://issues.apache.org/jira/browse/CASSANDRA-13069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-13069:
Reviewer: Sylvain Lebresne
> Local batchlog for MV may not be correctly written on node movements
>
>
> Key: CASSANDRA-13069
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13069
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Paulo Motta
>
> Unless I'm really reading this wrong, I think the code
> [here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageProxy.java#L829-L843],
> which comes from CASSANDRA-10674, isn't working properly.
> More precisely, I believe we can have both paired and unpaired mutations, so
> that both {{if}} can be taken, but if that's the case, the 2nd write to the
> batchlog will basically overwrite (remove) the batchlog write of the 1st
> {{if}} and I don't think that's the intention. In practice, this means
> "paired" mutation won't be in the batchlog, which mean they won't be replayed
> at all if they fail.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-13006) Disable automatic heap dumps on OOM error
[ https://issues.apache.org/jira/browse/CASSANDRA-13006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-13006: Status: Open (was: Patch Available) > Disable automatic heap dumps on OOM error > - > > Key: CASSANDRA-13006 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13006 > Project: Cassandra > Issue Type: Bug > Components: Configuration >Reporter: anmols >Assignee: Benjamin Lerer >Priority: Minor > Fix For: 3.0.9 > > Attachments: 13006-3.0.9.txt > > > With CASSANDRA-9861, a change was added to enable collecting heap dumps by > default if the process encountered an OOM error. These heap dumps are stored > in the Apache Cassandra home directory unless configured otherwise (see > [Cassandra Support > Document|https://support.datastax.com/hc/en-us/articles/204225959-Generating-and-Analyzing-Heap-Dumps] > for this feature). > > The creation and storage of heap dumps aides debugging and investigative > workflows, but is not be desirable for a production environment where these > heap dumps may occupy a large amount of disk space and require manual > intervention for cleanups. > > Managing heap dumps on out of memory errors and configuring the paths for > these heap dumps are available as JVM options in JVM. The current behavior > conflicts with the Boolean JVM flag HeapDumpOnOutOfMemoryError. > > A patch can be proposed here that would make the heap dump on OOM error honor > the HeapDumpOnOutOfMemoryError flag. Users who would want to still generate > heap dumps on OOM errors can set the -XX:+HeapDumpOnOutOfMemoryError JVM > option. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12997) dtest failure in org.apache.cassandra.cql3.validation.operations.AlterTest.testDropListAndAddListWithSameName
[
https://issues.apache.org/jira/browse/CASSANDRA-12997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-12997:
Reviewer: Aleksey Yeschenko
> dtest failure in
> org.apache.cassandra.cql3.validation.operations.AlterTest.testDropListAndAddListWithSameName
> -
>
> Key: CASSANDRA-12997
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12997
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: Sylvain Lebresne
> Labels: test-failure, testall
>
> example failure:
> http://cassci.datastax.com/job/trunk_testall/1298/testReport/org.apache.cassandra.cql3.validation.operations/AlterTest/testDropListAndAddListWithSameName
> {code}
> Error Message
> Invalid value for row 0 column 2 (mycollection of type list), expected
> but got <[first element]>
> {code}{code}Stacktrace
> junit.framework.AssertionFailedError: Invalid value for row 0 column 2
> (mycollection of type list), expected but got <[first element]>
> at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:908)
> at
> org.apache.cassandra.cql3.validation.operations.AlterTest.testDropListAndAddListWithSameName(AlterTest.java:87)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8552) Large compactions run out of off-heap RAM
[
https://issues.apache.org/jira/browse/CASSANDRA-8552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15801545#comment-15801545
]
Leonard Siu commented on CASSANDRA-8552:
I had a similar problem and was able to resolve my particular flavor of the
problem.
I had the same issue using C* 2.2.4.
Sometimes it crash just C*, but sometimes the entire VM.
The memory and swap usage (I know swap is not recommended) gradually climb to
100% and crash.
I also notice it is always the same table.
I see there is a 20G file in the data directory.
I used sstablesplit on the sstable of that file and restarted the node.
This resolved the issue for me and autocompcation was able to run properly.
> Large compactions run out of off-heap RAM
> -
>
> Key: CASSANDRA-8552
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8552
> Project: Cassandra
> Issue Type: Bug
> Environment: Ubuntu 14.4
> AWS EC2
> 12 m1.xlarge nodes [4 cores, 16GB RAM, 1TB storage (251GB Used)]
> Java build 1.7.0_55-b13 and build 1.8.0_25-b17
>Reporter: Brent Haines
>Assignee: Benedict
>Priority: Blocker
> Attachments: Screen Shot 2015-01-02 at 9.36.11 PM.png, data.cql,
> fhandles.log, freelog.log, lsof.txt, meminfo.txt, sysctl.txt, system.log
>
>
> We have a large table of storing, effectively event logs and a pair of
> denormalized tables for indexing.
> When updating from 2.0 to 2.1 we saw performance improvements, but some
> random and silent crashes during nightly repairs. We lost a node (totally
> corrupted) and replaced it. That node has never stabilized -- it simply can't
> finish the compactions.
> Smaller compactions finish. Larger compactions, like these two never finish -
> {code}
> pending tasks: 48
>compaction type keyspace table completed total
> unit progress
> Compaction data stories 16532973358 75977993784
> bytes 21.76%
> Compaction data stories_by_text 10593780658 38555048812
> bytes 27.48%
> Active compaction remaining time : 0h10m51s
> {code}
> We are not getting exceptions and are not running out of heap space. The
> Ubuntu OOM killer is reaping the process after all of the memory is consumed.
> We watch memory in the opscenter console and it will grow. If we turn off the
> OOM killer for the process, it will run until everything else is killed
> instead and then the kernel panics.
> We have the following settings configured:
> 2G Heap
> 512M New
> {code}
> memtable_heap_space_in_mb: 1024
> memtable_offheap_space_in_mb: 1024
> memtable_allocation_type: heap_buffers
> commitlog_total_space_in_mb: 2048
> concurrent_compactors: 1
> compaction_throughput_mb_per_sec: 128
> {code}
> The compaction strategy is leveled (these are read-intensive tables that are
> rarely updated)
> I have tried every setting, every option and I have the system where the MTBF
> is about an hour now, but we never finish compacting because there are some
> large compactions pending. None of the GC tools or settings help because it
> is not a GC problem. It is an off-heap memory problem.
> We are getting these messages in our syslog
> {code}
> Jan 2 07:06:00 ip-10-0-2-226 kernel: [49801151.219527] BUG: Bad page map in
> process java pte:0320 pmd:2d6fa5067
> Jan 2 07:06:00 ip-10-0-2-226 kernel: [49801151.219545] addr:7fb820be3000
> vm_flags:0870 anon_vma: (null) mapping: (null)
> index:7fb820be3
> Jan 2 07:06:00 ip-10-0-2-226 kernel: [49801151.219556] CPU: 3 PID: 27344
> Comm: java Tainted: GB3.13.0-24-generic #47-Ubuntu
> Jan 2 07:06:00 ip-10-0-2-226 kernel: [49801151.219559] 880028510e40
> 88020d43da98 81715ac4 7fb820be3000
> Jan 2 07:06:00 ip-10-0-2-226 kernel: [49801151.219565] 88020d43dae0
> 81174183 0320 0007fb820be3
> Jan 2 07:06:00 ip-10-0-2-226 kernel: [49801151.219568] 8802d6fa5f18
> 0320 7fb820be3000 7fb820be4000
> Jan 2 07:06:00 ip-10-0-2-226 kernel: [49801151.219572] Call Trace:
> Jan 2 07:06:00 ip-10-0-2-226 kernel: [49801151.219584] []
> dump_stack+0x45/0x56
> Jan 2 07:06:00 ip-10-0-2-226 kernel: [49801151.219591] []
> print_bad_pte+0x1a3/0x250
> Jan 2 07:06:00 ip-10-0-2-226 kernel: [49801151.219594] []
> vm_normal_page+0x69/0x80
> Jan 2 07:06:00 ip-10-0-2-226 kernel: [49801151.219598] []
> unmap_page_range+0x3bb/0x7f0
> Jan 2 07:06:00 ip-10-0-2-226 kernel: [49801151.219602] []
> unmap_single_vma+0x81/0xf0
> Jan 2 07:06:00 ip-10-0-2-226 kernel: [49801151.219605] []
> unmap_vmas+0x49/0x90
> Jan 2 07:06:00 ip-10-0-2-226 kernel: [49801151.219610] []
> exit_mmap+0x9c/0x170
> Jan 2 07:06:00 ip-10-0-2-226 kernel: [49801151.219617] []
> ? __delayacct
[jira] [Updated] (CASSANDRA-12856) dtest failure in replication_test.SnitchConfigurationUpdateTest.test_cannot_restart_with_different_rack
[
https://issues.apache.org/jira/browse/CASSANDRA-12856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-12856:
Reviewer: Joel Knighton
> dtest failure in
> replication_test.SnitchConfigurationUpdateTest.test_cannot_restart_with_different_rack
> ---
>
> Key: CASSANDRA-12856
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12856
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: Stefania
> Labels: dtest, test-failure
> Attachments: node1.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_novnode_dtest/280/testReport/replication_test/SnitchConfigurationUpdateTest/test_cannot_restart_with_different_rack
> {code}
> Error Message
> Problem stopping node node1
> {code}{code}
> Stacktrace
> File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
> File "/home/automaton/cassandra-dtest/replication_test.py", line 630, in
> test_cannot_restart_with_different_rack
> node1.stop(wait_other_notice=True)
> File "/usr/local/lib/python2.7/dist-packages/ccmlib/node.py", line 727, in
> stop
> raise NodeError("Problem stopping node %s" % self.name)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12794) COPY FROM with NULL='' fails when inserting empty row in primary key
[ https://issues.apache.org/jira/browse/CASSANDRA-12794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-12794: Reproduced In: 2.1.15, 2.1.14 (was: 2.1.14, 2.1.15) Reviewer: Tyler Hobbs > COPY FROM with NULL='' fails when inserting empty row in primary key > - > > Key: CASSANDRA-12794 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12794 > Project: Cassandra > Issue Type: Bug > Components: Tools > Environment: Tested using C* 2.1.15 >Reporter: Sucwinder Bassi >Assignee: Stefania > Fix For: 3.0.x, 3.x > > > Using this table: > CREATE TABLE testtab ( a_id text, b_id text, c_id text, d_id text, > order_id uuid, acc_id bigint, bucket bigint, r_id text, ts bigint, > PRIMARY KEY ((a_id, b_id, c_id, d_id), order_id)); > insert one row: > INSERT INTO testtab (a_id, b_id , c_id , d_id , order_id, r_id ) VALUES ( '', > '', '', 'a1', 645e7d3c-aef7-4e3c-b834-24b792cf2e55, 'r1'); > Use COPY to dump the row to temp.csv: > copy testtab TO 'temp.csv'; > Which creates this file: > $ cat temp.csv > ,,,a1,645e7d3c-aef7-4e3c-b834-24b792cf2e55,,,r1, > Truncate the testtab table and then use copy from with NULL='' to insert the > row: > cqlsh:sbkeyspace> COPY testtab FROM 'temp.csv' with NULL=''; > Using 1 child processes > Starting copy of sbkeyspace.testtab with columns ['a_id', 'b_id', 'c_id', > 'd_id', 'order_id', 'acc_id', 'bucket', 'r_id', 'ts']. > Failed to import 1 rows: ParseError - Cannot insert null value for primary > key column 'a_id'. If you want to insert empty strings, consider using the > WITH NULL= option for COPY., given up without retries > Failed to process 1 rows; failed rows written to import_sbkeyspace_testtab.err > Processed: 1 rows; Rate: 2 rows/s; Avg. rate: 3 rows/s > 1 rows imported from 1 files in 0.398 seconds (0 skipped). > It shows 1 rows inserted, but the table is empty: > select * from testtab ; > a_id | b_id | c_id | d_id | order_id | acc_id | bucket | r_id | ts > --+--+--+--+--+++--+ > (0 rows) > The same error is returned even without the with NULL=''. Is it actually > possible for copy from to insert an empty row into the primary key? The > insert command shown above inserts the empty row for the primary key without > any problems. > Is this related to https://issues.apache.org/jira/browse/CASSANDRA-7792? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13096) Snapshots slow down jmx scrapping
[
https://issues.apache.org/jira/browse/CASSANDRA-13096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15801479#comment-15801479
]
Maxime Fouilleul commented on CASSANDRA-13096:
--
It is better with the whitelist indeed:
{code}
whitelistObjectNames:
["java.lang:*","org.apache.cassandra.metrics:*","org.apache.cassandra.net:type=FailureDetector"]
{code}
We have divided by two the duration on a node (from ~7sec to ~3sec).
But anyway, it does not explain why scrapping is so long on certain node and
drop to "instantaneous" when cleaning snapshots.
Thanks for your help.
> Snapshots slow down jmx scrapping
> -
>
> Key: CASSANDRA-13096
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13096
> Project: Cassandra
> Issue Type: Bug
>Reporter: Maxime Fouilleul
> Attachments: CPU Load.png, Clear Snapshots.png, JMX Scrape
> Duration.png
>
>
> Hello,
> We are scraping the jmx metrics through a prometheus exporter and we noticed
> that some nodes became really long to answer (more than 20 seconds). After
> some investigations we do not find any hardware problem or overload issues on
> there "slow" nodes. It happens on different clusters, some with only few giga
> bytes of dataset and it does not seams to be related to a specific version
> neither as it happens on 2.1, 2.2 and 3.0 nodes.
> After some unsuccessful actions, one of our ideas was to clean the snapshots
> staying on one problematic node:
> {code}
> nodetool clearsnapshot
> {code}
> And the magic happens... as you can see in the attached diagrams, the second
> we cleared the snapshots, the CPU activity dropped immediatly and the
> duration to scrape the jmx metrics goes from +20 secs to instantaneous...
> Can you enlighten us on this issue? Once again, it appears on our three 2.1,
> 2.2 and 3.0 versions, on different volumetry and it is not systematically
> linked to the snapshots as we have some nodes with the same snapshots volume
> which are going pretty well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13095) Timeouts between nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-13095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15801369#comment-15801369 ] Danil Smirnov commented on CASSANDRA-13095: --- [~jay.zhuang] Could you please take a look at this issue, I believe this is the cause of the problem you describe at https://issues.apache.org/jira/browse/CASSANDRA-12676?focusedCommentId=15736166&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15736166 , if it is still haven't been solved. > Timeouts between nodes > -- > > Key: CASSANDRA-13095 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13095 > Project: Cassandra > Issue Type: Bug >Reporter: Danil Smirnov >Priority: Minor > Attachments: 13095-2.1.patch > > > Recently I've run into a problem with heavily loaded cluster when sometimes > messages between certain nodes become blocked with no reason. > It looks like the same situation that described here > https://issues.apache.org/jira/browse/CASSANDRA-12676?focusedCommentId=15736166&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15736166 > Thread dump showed infinite loop here: > https://github.com/apache/cassandra/blob/a8a43dd32eb92406d7d8b105e08c68b3d5c7df49/src/java/org/apache/cassandra/utils/CoalescingStrategies.java#L109 > Apparently the problem is in the initial value of epoch filed in > TimeHorizonMovingAverageCoalescingStrategy class. When it's value is not > evenly divisible by BUCKET_INTERVAL, ix(epoch-1) does not point to the > correct bucket. As a result, sum gradually increases and, upon reaching > MEASURED_INTERVAL, averageGap becomes 0 and thread blocks. > It's hard to reproduce because it takes a long time for sum to grow and when > no messages are send for some time, sum becomes 0 > https://github.com/apache/cassandra/blob/a8a43dd32eb92406d7d8b105e08c68b3d5c7df49/src/java/org/apache/cassandra/utils/CoalescingStrategies.java#L301 > and bug is no longer reproducible (until connection between nodes is > re-created). > I've added a patch which should fix the problem. Don't know if it would be of > any help since CASSANDRA-12676 will apparently disable this behaviour. One > note about performance regressions though. There is a small chance it being > result of the bug described here, so it might be worth testing performance > after fixes and/or tuning the algorithm. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10825) OverloadedException is untested
[ https://issues.apache.org/jira/browse/CASSANDRA-10825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15801294#comment-15801294 ] Aleksey Yeschenko commented on CASSANDRA-10825: --- bq. This feature never worked as intended. With this patch applied it will start doing something. My concern is a user a user could suddenly start seeing OverloadedException and without the metrics they will not have visibility into it. That's a valid concern. Maybe we should just target trunk only here, to avoid surprising anyone with new (old) behaviour? Would be indeed rude to drop it into 2.1/2.2/3.0 minor release, with or without metrics. For trunk this is fine. Maybe even 3.X. wdyt? > OverloadedException is untested > --- > > Key: CASSANDRA-10825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10825 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths >Reporter: Ariel Weisberg >Assignee: Edward Capriolo > Attachments: jmx-hint.png > > > If you grep test/src and cassandra-dtest you will find that the string > OverloadedException doesn't appear anywhere. > In CASSANDRA-10477 it was found that there were cases where Paxos should > back-pressure and throw OverloadedException but didn't. > If OverloadedException is used for functional purposes then we should test > that it is thrown under expected conditions. If there are behaviors driven by > catching or tracking OverloadedException we should test those as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-13052) Repair process is violating the start/end token limits for small ranges
[
https://issues.apache.org/jira/browse/CASSANDRA-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Podkowinski updated CASSANDRA-13052:
---
Attachment: (was: system-dev-debug-13052.log)
> Repair process is violating the start/end token limits for small ranges
> ---
>
> Key: CASSANDRA-13052
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13052
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
> Environment: We tried this in 2.0.14 and 3.9, same bug.
>Reporter: Cristian P
>Assignee: Stefan Podkowinski
> Attachments: ccm_reproduce-13052.txt, system-dev-debug-13052.log
>
>
> We tried to do a single token repair by providing 2 consecutive token values
> for a large column family. We soon notice heavy streaming and according to
> the logs the number of ranges streamed was in thousands.
> After investigation we found a bug in the two partitioner classes we use
> (RandomPartitioner and Murmur3Partitioner).
> The midpoint method used by MerkleTree.differenceHelper method to find ranges
> with differences for streaming returns abnormal values (way out of the
> initial range requested for repair) if the repair requested range is small (I
> expect smaller than 2^15).
> Here is the simple code to reproduce the bug for Murmur3Partitioner:
> Token left = new Murmur3Partitioner.LongToken(123456789L);
> Token right = new Murmur3Partitioner.LongToken(123456789L);
> IPartitioner partitioner = new Murmur3Partitioner();
> Token midpoint = partitioner.midpoint(left, right);
> System.out.println("Murmur3: [ " + left.getToken() + " : " +
> midpoint.getToken() + " : " + right.getToken() + " ]");
> The output is:
> Murmur3: [ 123456789 : -9223372036731319019 : 123456789 ]
> Note that the midpoint token is nowhere near the suggested repair range. This
> will happen if during the parsing of the tree (in
> MerkleTree.differenceHelper) in search for differences there isn't enough
> tokens for the split and the subrange becomes 0 (left.token=right.token) as
> in the above test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-13052) Repair process is violating the start/end token limits for small ranges
[
https://issues.apache.org/jira/browse/CASSANDRA-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Podkowinski updated CASSANDRA-13052:
---
Attachment: system-dev-debug-13052.log
> Repair process is violating the start/end token limits for small ranges
> ---
>
> Key: CASSANDRA-13052
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13052
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
> Environment: We tried this in 2.0.14 and 3.9, same bug.
>Reporter: Cristian P
>Assignee: Stefan Podkowinski
> Attachments: ccm_reproduce-13052.txt, system-dev-debug-13052.log,
> system-dev-debug-13052.log
>
>
> We tried to do a single token repair by providing 2 consecutive token values
> for a large column family. We soon notice heavy streaming and according to
> the logs the number of ranges streamed was in thousands.
> After investigation we found a bug in the two partitioner classes we use
> (RandomPartitioner and Murmur3Partitioner).
> The midpoint method used by MerkleTree.differenceHelper method to find ranges
> with differences for streaming returns abnormal values (way out of the
> initial range requested for repair) if the repair requested range is small (I
> expect smaller than 2^15).
> Here is the simple code to reproduce the bug for Murmur3Partitioner:
> Token left = new Murmur3Partitioner.LongToken(123456789L);
> Token right = new Murmur3Partitioner.LongToken(123456789L);
> IPartitioner partitioner = new Murmur3Partitioner();
> Token midpoint = partitioner.midpoint(left, right);
> System.out.println("Murmur3: [ " + left.getToken() + " : " +
> midpoint.getToken() + " : " + right.getToken() + " ]");
> The output is:
> Murmur3: [ 123456789 : -9223372036731319019 : 123456789 ]
> Note that the midpoint token is nowhere near the suggested repair range. This
> will happen if during the parsing of the tree (in
> MerkleTree.differenceHelper) in search for differences there isn't enough
> tokens for the split and the subrange becomes 0 (left.token=right.token) as
> in the above test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13052) Repair process is violating the start/end token limits for small ranges
[
https://issues.apache.org/jira/browse/CASSANDRA-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15801192#comment-15801192
]
Stefan Podkowinski commented on CASSANDRA-13052:
Which repairs are affected? The tree depth will effectively calculated as
{{min(log(partitions), 20)}}, ie. the depth will increase logarithmic until
capped at 20. In the worst case scenario, the described bug will happen when
the repaired token range will be less than 2^20. Using the Murmur3Partitioner,
this is still less than individual ranges created by using vnodes in a large
cluster {{(2^64 / (vnodes * nodes))}}. As vnodes are created based on random
values, there's still a chance that two vnodes are close enough together to get
affected by this bug regulary. Although this requires a fairly large amount of
partitions and I'm wondering if it would be practically possible to run repairs
in this setup anyways. More likely this could be an issue for tools that would
work by calculating their own (smaller) ranges instead of running repairs based
on (v)nodes.
> Repair process is violating the start/end token limits for small ranges
> ---
>
> Key: CASSANDRA-13052
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13052
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
> Environment: We tried this in 2.0.14 and 3.9, same bug.
>Reporter: Cristian P
>Assignee: Stefan Podkowinski
> Attachments: ccm_reproduce-13052.txt, system-dev-debug-13052.log
>
>
> We tried to do a single token repair by providing 2 consecutive token values
> for a large column family. We soon notice heavy streaming and according to
> the logs the number of ranges streamed was in thousands.
> After investigation we found a bug in the two partitioner classes we use
> (RandomPartitioner and Murmur3Partitioner).
> The midpoint method used by MerkleTree.differenceHelper method to find ranges
> with differences for streaming returns abnormal values (way out of the
> initial range requested for repair) if the repair requested range is small (I
> expect smaller than 2^15).
> Here is the simple code to reproduce the bug for Murmur3Partitioner:
> Token left = new Murmur3Partitioner.LongToken(123456789L);
> Token right = new Murmur3Partitioner.LongToken(123456789L);
> IPartitioner partitioner = new Murmur3Partitioner();
> Token midpoint = partitioner.midpoint(left, right);
> System.out.println("Murmur3: [ " + left.getToken() + " : " +
> midpoint.getToken() + " : " + right.getToken() + " ]");
> The output is:
> Murmur3: [ 123456789 : -9223372036731319019 : 123456789 ]
> Note that the midpoint token is nowhere near the suggested repair range. This
> will happen if during the parsing of the tree (in
> MerkleTree.differenceHelper) in search for differences there isn't enough
> tokens for the split and the subrange becomes 0 (left.token=right.token) as
> in the above test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13052) Repair process is violating the start/end token limits for small ranges
[
https://issues.apache.org/jira/browse/CASSANDRA-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15801182#comment-15801182
]
Stefan Podkowinski commented on CASSANDRA-13052:
The issue is not that ranges will be added multiple times, but the fact that
the added range has an equal start and endpoint and is thus invalid by
definition. Things starts to go awry in the normalize method, which will handle
the range as a wrap-around value and turns the specified range into another
range of (MINIMUM, MINIMUM). This range is handled special in some occassions
and seems to cause {{DataTracker.sstablesInBounds()}} return basically all
sstables. Based on those tables, the SSTable section calculation will return
all content from each SSTable as well and so it looks to me that this will
cause streaming of all data.
> Repair process is violating the start/end token limits for small ranges
> ---
>
> Key: CASSANDRA-13052
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13052
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
> Environment: We tried this in 2.0.14 and 3.9, same bug.
>Reporter: Cristian P
>Assignee: Stefan Podkowinski
> Attachments: ccm_reproduce-13052.txt, system-dev-debug-13052.log
>
>
> We tried to do a single token repair by providing 2 consecutive token values
> for a large column family. We soon notice heavy streaming and according to
> the logs the number of ranges streamed was in thousands.
> After investigation we found a bug in the two partitioner classes we use
> (RandomPartitioner and Murmur3Partitioner).
> The midpoint method used by MerkleTree.differenceHelper method to find ranges
> with differences for streaming returns abnormal values (way out of the
> initial range requested for repair) if the repair requested range is small (I
> expect smaller than 2^15).
> Here is the simple code to reproduce the bug for Murmur3Partitioner:
> Token left = new Murmur3Partitioner.LongToken(123456789L);
> Token right = new Murmur3Partitioner.LongToken(123456789L);
> IPartitioner partitioner = new Murmur3Partitioner();
> Token midpoint = partitioner.midpoint(left, right);
> System.out.println("Murmur3: [ " + left.getToken() + " : " +
> midpoint.getToken() + " : " + right.getToken() + " ]");
> The output is:
> Murmur3: [ 123456789 : -9223372036731319019 : 123456789 ]
> Note that the midpoint token is nowhere near the suggested repair range. This
> will happen if during the parsing of the tree (in
> MerkleTree.differenceHelper) in search for differences there isn't enough
> tokens for the split and the subrange becomes 0 (left.token=right.token) as
> in the above test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13096) Snapshots slow down jmx scrapping
[
https://issues.apache.org/jira/browse/CASSANDRA-13096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15800953#comment-15800953
]
Stefan Podkowinski commented on CASSANDRA-13096:
I think you should at least limit scraping to metric mbeans by having a
whitelist as in [this|https://github.com/prometheus/jmx_exporter#configuration]
example. Constantly polling every attribute of all mbeans sounds like trouble
to me, as even reading non-metric mbeans may have side effects.
> Snapshots slow down jmx scrapping
> -
>
> Key: CASSANDRA-13096
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13096
> Project: Cassandra
> Issue Type: Bug
>Reporter: Maxime Fouilleul
> Attachments: CPU Load.png, Clear Snapshots.png, JMX Scrape
> Duration.png
>
>
> Hello,
> We are scraping the jmx metrics through a prometheus exporter and we noticed
> that some nodes became really long to answer (more than 20 seconds). After
> some investigations we do not find any hardware problem or overload issues on
> there "slow" nodes. It happens on different clusters, some with only few giga
> bytes of dataset and it does not seams to be related to a specific version
> neither as it happens on 2.1, 2.2 and 3.0 nodes.
> After some unsuccessful actions, one of our ideas was to clean the snapshots
> staying on one problematic node:
> {code}
> nodetool clearsnapshot
> {code}
> And the magic happens... as you can see in the attached diagrams, the second
> we cleared the snapshots, the CPU activity dropped immediatly and the
> duration to scrape the jmx metrics goes from +20 secs to instantaneous...
> Can you enlighten us on this issue? Once again, it appears on our three 2.1,
> 2.2 and 3.0 versions, on different volumetry and it is not systematically
> linked to the snapshots as we have some nodes with the same snapshots volume
> which are going pretty well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13096) Snapshots slow down jmx scrapping
[
https://issues.apache.org/jira/browse/CASSANDRA-13096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15800887#comment-15800887
]
Maxime Fouilleul commented on CASSANDRA-13096:
--
We use the jmx prometheus exporter
(https://github.com/prometheus/jmx_exporter).
>From the README:
{quote}
Note that the scraper always processes all mBeans, even if they're not exported.
{quote}
We use the default use in examples
(https://github.com/prometheus/jmx_exporter/blob/master/example_configs/cassandra.yml):
{code}
---
lowercaseOutputLabelNames: true
lowercaseOutputName: true
rules:
- pattern: org.apache.cassandra.metrics<>(Count|Value)
name: cassandra_$1_$3
labels:
address: "$2"
- pattern: org.apache.cassandra.metrics<>(Count|Value)
name: cassandra_$1_$5
labels:
"$1": "$4"
"$2": "$3"
{code}
We hit a bug recently (https://issues.apache.org/jira/browse/CASSANDRA-11594),
it looks like the same kind of issue, maybe it could help...
Thanks for your help.
> Snapshots slow down jmx scrapping
> -
>
> Key: CASSANDRA-13096
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13096
> Project: Cassandra
> Issue Type: Bug
>Reporter: Maxime Fouilleul
> Attachments: CPU Load.png, Clear Snapshots.png, JMX Scrape
> Duration.png
>
>
> Hello,
> We are scraping the jmx metrics through a prometheus exporter and we noticed
> that some nodes became really long to answer (more than 20 seconds). After
> some investigations we do not find any hardware problem or overload issues on
> there "slow" nodes. It happens on different clusters, some with only few giga
> bytes of dataset and it does not seams to be related to a specific version
> neither as it happens on 2.1, 2.2 and 3.0 nodes.
> After some unsuccessful actions, one of our ideas was to clean the snapshots
> staying on one problematic node:
> {code}
> nodetool clearsnapshot
> {code}
> And the magic happens... as you can see in the attached diagrams, the second
> we cleared the snapshots, the CPU activity dropped immediatly and the
> duration to scrape the jmx metrics goes from +20 secs to instantaneous...
> Can you enlighten us on this issue? Once again, it appears on our three 2.1,
> 2.2 and 3.0 versions, on different volumetry and it is not systematically
> linked to the snapshots as we have some nodes with the same snapshots volume
> which are going pretty well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13096) Snapshots slow down jmx scrapping
[
https://issues.apache.org/jira/browse/CASSANDRA-13096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15800761#comment-15800761
]
Stefan Podkowinski commented on CASSANDRA-13096:
Can you provide more details on how the prometheus integration is configured
(metrics-reporter-config push gateway, -servlet, or simple jmx scraping)? Did
you try to limit the number of read metrics to pin down a specific one as the
actual cause? What metrics are you reading?
> Snapshots slow down jmx scrapping
> -
>
> Key: CASSANDRA-13096
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13096
> Project: Cassandra
> Issue Type: Bug
>Reporter: Maxime Fouilleul
> Attachments: CPU Load.png, Clear Snapshots.png, JMX Scrape
> Duration.png
>
>
> Hello,
> We are scraping the jmx metrics through a prometheus exporter and we noticed
> that some nodes became really long to answer (more than 20 seconds). After
> some investigations we do not find any hardware problem or overload issues on
> there "slow" nodes. It happens on different clusters, some with only few giga
> bytes of dataset and it does not seams to be related to a specific version
> neither as it happens on 2.1, 2.2 and 3.0 nodes.
> After some unsuccessful actions, one of our ideas was to clean the snapshots
> staying on one problematic node:
> {code}
> nodetool clearsnapshot
> {code}
> And the magic happens... as you can see in the attached diagrams, the second
> we cleared the snapshots, the CPU activity dropped immediatly and the
> duration to scrape the jmx metrics goes from +20 secs to instantaneous...
> Can you enlighten us on this issue? Once again, it appears on our three 2.1,
> 2.2 and 3.0 versions, on different volumetry and it is not systematically
> linked to the snapshots as we have some nodes with the same snapshots volume
> which are going pretty well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-13090) Coalescing strategy sleep too much
[ https://issues.apache.org/jira/browse/CASSANDRA-13090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Corentin Chary updated CASSANDRA-13090: --- Attachment: 0001-Fix-wait-time-coalescing-CASSANDRA-13090.patch New version of the patch with discussed settings and unit tests. > Coalescing strategy sleep too much > -- > > Key: CASSANDRA-13090 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13090 > Project: Cassandra > Issue Type: Bug > Components: Streaming and Messaging >Reporter: Corentin Chary > Fix For: 3.x > > Attachments: 0001-Fix-wait-time-coalescing-CASSANDRA-13090.patch > > > With the current code maybeSleep is called even if we managed to take > maxItems out of the backlog. In this case we should really avoid sleeping > because it means that backlog is building up. > I'll send a patch shortly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-13090) Coalescing strategy sleep too much
[ https://issues.apache.org/jira/browse/CASSANDRA-13090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Corentin Chary updated CASSANDRA-13090: --- Attachment: (was: 0001-Fix-wait-time-coalescing-CASSANDRA-13090.patch) > Coalescing strategy sleep too much > -- > > Key: CASSANDRA-13090 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13090 > Project: Cassandra > Issue Type: Bug > Components: Streaming and Messaging >Reporter: Corentin Chary > Fix For: 3.x > > Attachments: 0001-Fix-wait-time-coalescing-CASSANDRA-13090.patch > > > With the current code maybeSleep is called even if we managed to take > maxItems out of the backlog. In this case we should really avoid sleeping > because it means that backlog is building up. > I'll send a patch shortly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
