[jira] [Created] (CASSANDRA-15005) Configurable whilelist for UDFs
A. Soroka created CASSANDRA-15005:
-
Summary: Configurable whilelist for UDFs
Key: CASSANDRA-15005
URL: https://issues.apache.org/jira/browse/CASSANDRA-15005
Project: Cassandra
Issue Type: Improvement
Components: CQL/Interpreter
Reporter: A. Soroka
I would like to use the UDF system to distribute some simple calculations on
values. For some use cases, this would require access only to some Java API
classes that aren't on the (hardcoded) whitelist (e.g.
{{java.security.MessageDigest}}). In other cases, it would require access to a
little non-C* library code, pre-distributed to nodes by out-of-band means.
As I understand the situation now, the whitelist for types UDFs can use is
hardcoded in java in
[UDFunction|[https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/cql3/functions/UDFunction.java#L99].]
This ticket, then, is a request for a facility that would allow that list to be
extended via some kind of deployment-time configuration. I realize that serious
security concerns immediately arise for this kind of functionality, but I hope
that by restricting it (only used during startup, no exposing the whitelist for
introspection, etc.) it could be quite practical.
I'd like very much to assist with this ticket if it is accepted. (I believe I
have sufficient Java skill to do that, but no real familiarity with C*'s
codebase, yet. :) )
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/02: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository. mshuler pushed a commit to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 91fb24eb37d2305ebb1a41a53910a888cffb6a88 Merge: 62d66a4 af91658 Author: Michael Shuler AuthorDate: Fri Feb 1 12:36:34 2019 -0600 Merge branch 'cassandra-2.2' into cassandra-3.0 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 02/02: Update version to 2.2.14
This is an automated email from the ASF dual-hosted git repository. mshuler pushed a commit to branch cassandra-2.2 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit af91658353ba601fc8cd08627e8d36bac62e936a Author: Michael Shuler AuthorDate: Fri Feb 1 12:36:13 2019 -0600 Update version to 2.2.14 --- NEWS.txt | 8 build.xml| 2 +- debian/changelog | 6 ++ 3 files changed, 15 insertions(+), 1 deletion(-) diff --git a/NEWS.txt b/NEWS.txt index decb3ff..730a48a 100644 --- a/NEWS.txt +++ b/NEWS.txt @@ -42,6 +42,14 @@ restore snapshots created with the previous major version using the 'sstableloader' tool. You can upgrade the file format of your snapshots using the provided 'sstableupgrade' tool. +2.2.14 +== + +Upgrading +- +- Nothing specific to this release, but please see 2.2 if you are upgrading + from a previous version. + 2.2.13 == diff --git a/build.xml b/build.xml index 8b524f7..b5682c8 100644 --- a/build.xml +++ b/build.xml @@ -25,7 +25,7 @@ - + http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/> diff --git a/debian/changelog b/debian/changelog index 2d00393..1abcca1 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +cassandra (2.2.14) unstable; urgency=medium + + * New release + + -- Michael Shuler Fri, 01 Feb 2019 12:34:51 -0600 + cassandra (2.2.13) unstable; urgency=medium * New release - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.1 updated: Update version to 2.1.21
This is an automated email from the ASF dual-hosted git repository. mshuler pushed a commit to branch cassandra-2.1 in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/cassandra-2.1 by this push: new 9bb7535 Update version to 2.1.21 9bb7535 is described below commit 9bb75358dfdf1b9824f9a454e70ee2c02bc64a45 Author: Michael Shuler AuthorDate: Fri Feb 1 12:33:15 2019 -0600 Update version to 2.1.21 --- NEWS.txt | 8 build.xml| 2 +- debian/changelog | 6 ++ 3 files changed, 15 insertions(+), 1 deletion(-) diff --git a/NEWS.txt b/NEWS.txt index 232f3cd..f81dc53 100644 --- a/NEWS.txt +++ b/NEWS.txt @@ -42,6 +42,14 @@ restore snapshots created with the previous major version using the 'sstableloader' tool. You can upgrade the file format of your snapshots using the provided 'sstableupgrade' tool. +2.1.21 +== + +Upgrading +- +- Nothing specific to this release, but please see 2.1 if you are upgrading + from a previous version. + 2.1.20 == diff --git a/build.xml b/build.xml index ce47717..f6fc1f6 100644 --- a/build.xml +++ b/build.xml @@ -25,7 +25,7 @@ - + http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/> diff --git a/debian/changelog b/debian/changelog index 5b22031..0c3a00f 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +cassandra (2.1.21) unstable; urgency=medium + + * New release + + -- Michael Shuler Fri, 01 Feb 2019 12:28:58 -0600 + cassandra (2.1.20) unstable; urgency=medium * New release - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (5ea3394 -> 7a89e31)
This is an automated email from the ASF dual-hosted git repository. mshuler pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 5ea3394 Merge branch 'cassandra-3.11' into trunk new 9bb7535 Update version to 2.1.21 new e791a7f Merge branch 'cassandra-2.1' into cassandra-2.2 new af91658 Update version to 2.2.14 new 91fb24e Merge branch 'cassandra-2.2' into cassandra-3.0 new edd52ce Update version to 3.0.18 new cf9b346 Merge branch 'cassandra-3.0' into cassandra-3.11 new fd47391 Update version to 3.11.4 new 7a89e31 Merge branch 'cassandra-3.11' into trunk The 8 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.2 updated (fdb1735 -> af91658)
This is an automated email from the ASF dual-hosted git repository. mshuler pushed a change to branch cassandra-2.2 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from fdb1735 In JVM dtests need to clean up after instance shutdown new 9bb7535 Update version to 2.1.21 new e791a7f Merge branch 'cassandra-2.1' into cassandra-2.2 new af91658 Update version to 2.2.14 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: NEWS.txt | 8 build.xml| 2 +- debian/changelog | 6 ++ 3 files changed, 15 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/02: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository. mshuler pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit cf9b346593130a9501f2d5f871aced18f7270c1c Merge: 737a566 edd52ce Author: Michael Shuler AuthorDate: Fri Feb 1 12:38:46 2019 -0600 Merge branch 'cassandra-3.0' into cassandra-3.11 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 02/02: Update version to 3.11.4
This is an automated email from the ASF dual-hosted git repository. mshuler pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit fd47391aae13bcf4ee995abcde1b0e180372d193 Author: Michael Shuler AuthorDate: Fri Feb 1 12:40:38 2019 -0600 Update version to 3.11.4 --- build.xml| 2 +- debian/changelog | 6 ++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/build.xml b/build.xml index 00f1783..94b07f3 100644 --- a/build.xml +++ b/build.xml @@ -25,7 +25,7 @@ - + http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/> diff --git a/debian/changelog b/debian/changelog index b4b788c..ba5efef 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +cassandra (3.11.4) unstable; urgency=medium + + * New release + + -- Michael Shuler Fri, 01 Feb 2019 12:39:21 -0600 + cassandra (3.11.3) unstable; urgency=medium * New release - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (737a566 -> fd47391)
This is an automated email from the ASF dual-hosted git repository. mshuler pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 737a566 Merge branch 'cassandra-3.0' into cassandra-3.11 new 9bb7535 Update version to 2.1.21 new e791a7f Merge branch 'cassandra-2.1' into cassandra-2.2 new af91658 Update version to 2.2.14 new 91fb24e Merge branch 'cassandra-2.2' into cassandra-3.0 new edd52ce Update version to 3.0.18 new cf9b346 Merge branch 'cassandra-3.0' into cassandra-3.11 new fd47391 Update version to 3.11.4 The 7 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: build.xml| 2 +- debian/changelog | 6 ++ 2 files changed, 7 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/02: Merge branch 'cassandra-2.1' into cassandra-2.2
This is an automated email from the ASF dual-hosted git repository. mshuler pushed a commit to branch cassandra-2.2 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit e791a7f94f0f55302075daedf15788a819e30207 Merge: fdb1735 9bb7535 Author: Michael Shuler AuthorDate: Fri Feb 1 12:33:35 2019 -0600 Merge branch 'cassandra-2.1' into cassandra-2.2 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (62d66a4 -> edd52ce)
This is an automated email from the ASF dual-hosted git repository. mshuler pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 62d66a4 Fix CHANGES.txt new 9bb7535 Update version to 2.1.21 new e791a7f Merge branch 'cassandra-2.1' into cassandra-2.2 new af91658 Update version to 2.2.14 new 91fb24e Merge branch 'cassandra-2.2' into cassandra-3.0 new edd52ce Update version to 3.0.18 The 5 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: build.xml| 2 +- debian/changelog | 6 ++ 2 files changed, 7 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 02/02: Update version to 3.0.18
This is an automated email from the ASF dual-hosted git repository. mshuler pushed a commit to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit edd52cef50a6242609a20d0d84c8eb74c580035e Author: Michael Shuler AuthorDate: Fri Feb 1 12:38:25 2019 -0600 Update version to 3.0.18 --- build.xml| 2 +- debian/changelog | 6 ++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/build.xml b/build.xml index 1d2c375..610176b 100644 --- a/build.xml +++ b/build.xml @@ -25,7 +25,7 @@ - + http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/> diff --git a/debian/changelog b/debian/changelog index 8ce02e3..4a1df4c 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +cassandra (3.0.18) unstable; urgency=medium + + * New release + + -- Michael Shuler Fri, 01 Feb 2019 12:37:12 -0600 + cassandra (3.0.17) unstable; urgency=medium * New release - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. mshuler pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 7a89e31d94f6e09500db9d5c0285f7cb27f8aff9 Merge: 5ea3394 fd47391 Author: Michael Shuler AuthorDate: Fri Feb 1 12:40:51 2019 -0600 Merge branch 'cassandra-3.11' into trunk - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Resolved] (CASSANDRA-14871) Severe concurrency issues in STCS,DTCS,TWCS,TMD.Topology,TypeParser
[
https://issues.apache.org/jira/browse/CASSANDRA-14871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp resolved CASSANDRA-14871.
--
Resolution: Fixed
Fix Version/s: (was: 3.11.x)
(was: 3.0.x)
3.11.4
3.0.18
Thanks for the review!
Committed as
[bc18b4dd4e33020d0d58c3701077d0af5c39bce6|https://github.com/apache/cassandra/commit/bc18b4dd4e33020d0d58c3701077d0af5c39bce6]
to [cassandra-3.0|https://github.com/apache/cassandra/tree/cassandra-3.0],
[merged|https://github.com/apache/cassandra/commit/16c96c20dadfbda98d4b5daf7f6c169b691459b9]
to [cassandra-3.11|https://github.com/apache/cassandra/tree/cassandra-3.11],
[merged|https://github.com/apache/cassandra/commit/ef6c5f8ba9dc18e7a5bacfd5d8461ae5d9f12df4]
to [trunk|https://github.com/apache/cassandra/tree/trunk].
> Severe concurrency issues in STCS,DTCS,TWCS,TMD.Topology,TypeParser
> ---
>
> Key: CASSANDRA-14871
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14871
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Critical
> Fix For: 3.0.18, 3.11.4, 4.0
>
>
> There are a couple of places in the code base that do not respect that
> j.u.HashMap + related classes are not thread safe and some parts rely on
> internals of the implementation of HM, which can change.
> We have observed failures like {{NullPointerException}} and
> {{ConcurrentModificationException}} as well as wrong behavior.
> Affected areas in the code base:
> * {{SizeTieredCompactionStrategy}}
> * {{DateTieredCompactionStrategy}}
> * {{TimeWindowCompactionStrategy}}
> * {{TokenMetadata.Topology}}
> * {{TypeParser}}
> * streaming / concurrent access to {{LifecycleTransaction}} (handled in
> CASSANDRA-14554)
> While the patches for the compaction strategies + {{TypeParser}} are pretty
> straight forward, the patch for {{TokenMetadata.Topology}} requires it to be
> made immutable.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (ef6c5f8 -> 5ea3394)
This is an automated email from the ASF dual-hosted git repository. snazy pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from ef6c5f8 Merge branch cassandra-3.11 into trunk new 62d66a4 Fix CHANGES.txt new 737a566 Merge branch 'cassandra-3.0' into cassandra-3.11 new 5ea3394 Merge branch 'cassandra-3.11' into trunk The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 1 + 1 file changed, 1 insertion(+) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (16c96c2 -> 737a566)
This is an automated email from the ASF dual-hosted git repository. snazy pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 16c96c2 Merge branch cassandra-3.0 into cassandra-3.11 new 62d66a4 Fix CHANGES.txt new 737a566 Merge branch 'cassandra-3.0' into cassandra-3.11 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 1 + 1 file changed, 1 insertion(+) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. snazy pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 5ea3394e9561591322356b19f41ef450fe1972ab Merge: ef6c5f8 737a566 Author: Robert Stupp AuthorDate: Fri Feb 1 12:34:56 2019 +0100 Merge branch 'cassandra-3.11' into trunk CHANGES.txt | 1 + 1 file changed, 1 insertion(+) diff --cc CHANGES.txt index a905ad3,76cd7d7..4f52cca --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -348,10 -2,13 +348,11 @@@ * Make stop-server.bat wait for Cassandra to terminate (CASSANDRA-14829) * Correct sstable sorting for garbagecollect and levelled compaction (CASSANDRA-14870) Merged from 3.0: + * Severe concurrency issues in STCS,DTCS,TWCS,TMD.Topology,TypeParser * Add a script to make running the cqlsh tests in cassandra repo easier (CASSANDRA-14951) * If SizeEstimatesRecorder misses a 'onDropTable' notification, the size_estimates table will never be cleared for that table. (CASSANDRA-14905) - * Counters fail to increment in 2.1/2.2 to 3.X mixed version clusters (CASSANDRA-14958) * Streaming needs to synchronise access to LifecycleTransaction (CASSANDRA-14554) * Fix cassandra-stress write hang with default options (CASSANDRA-14616) - * Differentiate between slices and RTs when decoding legacy bounds (CASSANDRA-14919) * Netty epoll IOExceptions caused by unclean client disconnects being logged at INFO (CASSANDRA-14909) * Unfiltered.isEmpty conflicts with Row extends AbstractCollection.isEmpty (CASSANDRA-14588) * RangeTombstoneList doesn't properly clean up mergeable or superseded rts in some cases (CASSANDRA-14894) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated: Fix CHANGES.txt
This is an automated email from the ASF dual-hosted git repository. snazy pushed a commit to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/cassandra-3.0 by this push: new 62d66a4 Fix CHANGES.txt 62d66a4 is described below commit 62d66a43c67042c46eb67e272c4dec5cb328c9d7 Author: Robert Stupp AuthorDate: Fri Feb 1 12:34:12 2019 +0100 Fix CHANGES.txt --- CHANGES.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGES.txt b/CHANGES.txt index 812af0a..e3b23ee 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.0.18 + * Severe concurrency issues in STCS,DTCS,TWCS,TMD.Topology,TypeParser * Add a script to make running the cqlsh tests in cassandra repo easier (CASSANDRA-14951) * If SizeEstimatesRecorder misses a 'onDropTable' notification, the size_estimates table will never be cleared for that table. (CASSANDRA-14905) * Counters fail to increment in 2.1/2.2 to 3.X mixed version clusters (CASSANDRA-14958) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository. snazy pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 737a5666b021caea7e74c33f12e30865a52e4b99 Merge: 16c96c2 62d66a4 Author: Robert Stupp AuthorDate: Fri Feb 1 12:34:43 2019 +0100 Merge branch 'cassandra-3.0' into cassandra-3.11 CHANGES.txt | 1 + 1 file changed, 1 insertion(+) diff --cc CHANGES.txt index d85a4af,e3b23ee..76cd7d7 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,7 -1,5 +1,8 @@@ -3.0.18 +3.11.4 + * Make stop-server.bat wait for Cassandra to terminate (CASSANDRA-14829) + * Correct sstable sorting for garbagecollect and levelled compaction (CASSANDRA-14870) +Merged from 3.0: + * Severe concurrency issues in STCS,DTCS,TWCS,TMD.Topology,TypeParser * Add a script to make running the cqlsh tests in cassandra repo easier (CASSANDRA-14951) * If SizeEstimatesRecorder misses a 'onDropTable' notification, the size_estimates table will never be cleared for that table. (CASSANDRA-14905) * Counters fail to increment in 2.1/2.2 to 3.X mixed version clusters (CASSANDRA-14958) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch cassandra-3.0 into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
snazy pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 16c96c20dadfbda98d4b5daf7f6c169b691459b9
Merge: 2d05bff bc18b4d
Author: Robert Stupp
AuthorDate: Fri Feb 1 12:26:43 2019 +0100
Merge branch cassandra-3.0 into cassandra-3.11
.../org/apache/cassandra/db/ColumnFamilyStore.java | 1 -
.../db/compaction/AbstractCompactionStrategy.java | 14 +-
.../compaction/DateTieredCompactionStrategy.java | 20 +-
.../compaction/SizeTieredCompactionStrategy.java | 6 +-
.../compaction/TimeWindowCompactionStrategy.java | 4 +-
.../apache/cassandra/db/marshal/TypeParser.java| 30 +-
.../cassandra/locator/NetworkTopologyStrategy.java | 3 +-
.../apache/cassandra/locator/TokenMetadata.java| 364 -
.../locator/NetworkTopologyStrategyTest.java | 3 +-
.../cassandra/locator/TokenMetadataTest.java | 23 +-
10 files changed, 280 insertions(+), 188 deletions(-)
diff --cc
src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
index bb9f4b9,7c38fa8..83d77a9
---
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
@@@ -105,10 -89,10 +105,14 @@@ public class DateTieredCompactionStrate
*/
private synchronized List getNextBackgroundSSTables(final
int gcBefore)
{
- if (sstables.isEmpty())
-if (Iterables.isEmpty(cfs.getSSTables(SSTableSet.LIVE)))
--return Collections.emptyList();
++Set uncompacting;
++synchronized (sstables)
++{
++if (sstables.isEmpty())
++return Collections.emptyList();
--Set uncompacting =
ImmutableSet.copyOf(filter(cfs.getUncompactingSSTables(), sstables::contains));
++uncompacting =
ImmutableSet.copyOf(filter(cfs.getUncompactingSSTables(), sstables::contains));
++}
Set expired = Collections.emptySet();
// we only check for expired sstables every 10 minutes (by default)
due to it being an expensive operation
diff --cc src/java/org/apache/cassandra/db/marshal/TypeParser.java
index 78af800,590eea3..7416d49
--- a/src/java/org/apache/cassandra/db/marshal/TypeParser.java
+++ b/src/java/org/apache/cassandra/db/marshal/TypeParser.java
@@@ -23,7 -23,9 +23,10 @@@ import java.lang.reflect.Method
import java.nio.ByteBuffer;
import java.util.*;
+ import com.google.common.base.Verify;
+ import com.google.common.collect.ImmutableMap;
+
+import org.apache.cassandra.cql3.FieldIdentifier;
import org.apache.cassandra.exceptions.*;
import org.apache.cassandra.utils.ByteBufferUtil;
import org.apache.cassandra.utils.FBUtilities;
diff --cc src/java/org/apache/cassandra/locator/NetworkTopologyStrategy.java
index 756b689,82183bb..d48dec3
--- a/src/java/org/apache/cassandra/locator/NetworkTopologyStrategy.java
+++ b/src/java/org/apache/cassandra/locator/NetworkTopologyStrategy.java
@@@ -28,8 -28,8 +28,9 @@@ import org.apache.cassandra.exceptions.
import org.apache.cassandra.dht.Token;
import org.apache.cassandra.locator.TokenMetadata.Topology;
import org.apache.cassandra.utils.FBUtilities;
+import org.apache.cassandra.utils.Pair;
+ import com.google.common.collect.ImmutableMultimap;
import com.google.common.collect.Multimap;
/**
@@@ -151,29 -92,22 +152,29 @@@ public class NetworkTopologyStrategy ex
// all endpoints in each DC, so we can check when we have exhausted
all the members of a DC
Multimap allEndpoints =
topology.getDatacenterEndpoints();
// all racks in a DC so we can check when we have exhausted all racks
in a DC
- Map> racks =
topology.getDatacenterRacks();
+ Map> racks =
topology.getDatacenterRacks();
assert !allEndpoints.isEmpty() && !racks.isEmpty() : "not aware of
any cluster members";
-// tracks the racks we have already placed replicas in
-Map> seenRacks = new
HashMap<>(datacenters.size());
-for (Map.Entry dc : datacenters.entrySet())
-seenRacks.put(dc.getKey(), new HashSet());
+int dcsToFill = 0;
+Map dcs = new
HashMap<>(datacenters.size() * 2);
-// tracks the endpoints that we skipped over while looking for unique
racks
-// when we relax the rack uniqueness we can append this to the
current result so we don't have to wind back the iterator
-Map> skippedDcEndpoints = new
HashMap<>(datacenters.size());
-for (Map.Entry dc : datacenters.entrySet())
-skippedDcEndpoints.put(dc.getKey(), new
LinkedHashSet());
+// Create a DatacenterEndpoints object for each non-empty DC.
+for (Map.Entry en : datacenters.entrySet())
+{
+String dc = en.getKey();
+int rf = en.getValue();
+
[cassandra] branch cassandra-3.11 updated (2d05bff -> 16c96c2)
This is an automated email from the ASF dual-hosted git repository. snazy pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 2d05bff Merge branch 'cassandra-3.0' into cassandra-3.11 new bc18b4d Severe concurrency issues in STCS,DTCS,TWCS,TMD.Topology,TypeParser new 16c96c2 Merge branch cassandra-3.0 into cassandra-3.11 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../org/apache/cassandra/db/ColumnFamilyStore.java | 1 - .../db/compaction/AbstractCompactionStrategy.java | 14 +- .../compaction/DateTieredCompactionStrategy.java | 20 +- .../compaction/SizeTieredCompactionStrategy.java | 6 +- .../compaction/TimeWindowCompactionStrategy.java | 4 +- .../apache/cassandra/db/marshal/TypeParser.java| 30 +- .../cassandra/locator/NetworkTopologyStrategy.java | 3 +- .../apache/cassandra/locator/TokenMetadata.java| 364 - .../locator/NetworkTopologyStrategyTest.java | 3 +- .../cassandra/locator/TokenMetadataTest.java | 23 +- 10 files changed, 280 insertions(+), 188 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated: Severe concurrency issues in STCS, DTCS, TWCS, TMD.Topology, TypeParser
This is an automated email from the ASF dual-hosted git repository.
snazy pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-3.0 by this push:
new bc18b4d Severe concurrency issues in
STCS,DTCS,TWCS,TMD.Topology,TypeParser
bc18b4d is described below
commit bc18b4dd4e33020d0d58c3701077d0af5c39bce6
Author: Robert Stupp
AuthorDate: Wed Oct 31 12:48:19 2018 +0100
Severe concurrency issues in STCS,DTCS,TWCS,TMD.Topology,TypeParser
patch by Robert Stupp; reviewed by Blake Eggleston for CASSANDRA-14781
---
.../org/apache/cassandra/db/ColumnFamilyStore.java | 1 -
.../db/compaction/AbstractCompactionStrategy.java | 14 +-
.../compaction/DateTieredCompactionStrategy.java | 12 +-
.../compaction/SizeTieredCompactionStrategy.java | 8 +-
.../compaction/TimeWindowCompactionStrategy.java | 4 +-
.../apache/cassandra/db/marshal/TypeParser.java| 30 +-
.../cassandra/locator/NetworkTopologyStrategy.java | 3 +-
.../apache/cassandra/locator/TokenMetadata.java| 354 -
.../cassandra/locator/TokenMetadataTest.java | 23 +-
9 files changed, 269 insertions(+), 180 deletions(-)
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 3482909..355d710 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -2138,7 +2138,6 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
assert data.getCompacting().isEmpty() : data.getCompacting();
Iterable sstables = getLiveSSTables();
sstables =
AbstractCompactionStrategy.filterSuspectSSTables(sstables);
-sstables = ImmutableList.copyOf(sstables);
LifecycleTransaction modifier = data.tryModify(sstables,
operationType);
assert modifier != null: "something marked things compacting
while compactions are disabled";
return modifier;
diff --git
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
index 9f07691..2348d19 100644
---
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
@@ -252,15 +252,15 @@ public abstract class AbstractCompactionStrategy
* @param originalCandidates The collection to check for blacklisted
SSTables
* @return list of the SSTables with blacklisted ones filtered out
*/
-public static Iterable
filterSuspectSSTables(Iterable originalCandidates)
+public static List
filterSuspectSSTables(Iterable originalCandidates)
{
-return Iterables.filter(originalCandidates, new
Predicate()
+List filtered = new ArrayList<>();
+for (SSTableReader sstable : originalCandidates)
{
-public boolean apply(SSTableReader sstable)
-{
-return !sstable.isMarkedSuspect();
-}
-});
+if (!sstable.isMarkedSuspect())
+filtered.add(sstable);
+}
+return filtered;
}
diff --git
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
index 3e6ae61..7c38fa8 100644
---
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
@@ -87,7 +87,7 @@ public class DateTieredCompactionStrategy extends
AbstractCompactionStrategy
* @param gcBefore
* @return
*/
-private List getNextBackgroundSSTables(final int gcBefore)
+private synchronized List getNextBackgroundSSTables(final
int gcBefore)
{
if (Iterables.isEmpty(cfs.getSSTables(SSTableSet.LIVE)))
return Collections.emptyList();
@@ -193,11 +193,6 @@ public class DateTieredCompactionStrategy extends
AbstractCompactionStrategy
});
}
-/**
- *
- * @param sstables
- * @return
- */
public static List>
createSSTableAndMinTimestampPairs(Iterable sstables)
{
List> sstableMinTimestampPairs =
Lists.newArrayListWithCapacity(Iterables.size(sstables));
@@ -205,14 +200,15 @@ public class DateTieredCompactionStrategy extends
AbstractCompactionStrategy
sstableMinTimestampPairs.add(Pair.create(sstable,
sstable.getMinTimestamp()));
return sstableMinTimestampPairs;
}
+
@Override
-public void addSSTable(SSTableReader sstable)
+public synchronized void addSSTable(SSTableReader sstable)
{
sstables.add(sstable);
}
@Override
-public void remov
[cassandra] branch trunk updated (7f634fe -> ef6c5f8)
This is an automated email from the ASF dual-hosted git repository. snazy pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 7f634fe Avoid leaking threads when remote nodes fail anticompaction and rate limit anticompactions new bc18b4d Severe concurrency issues in STCS,DTCS,TWCS,TMD.Topology,TypeParser new 16c96c2 Merge branch cassandra-3.0 into cassandra-3.11 new ef6c5f8 Merge branch cassandra-3.11 into trunk The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../org/apache/cassandra/db/ColumnFamilyStore.java | 1 - .../db/compaction/AbstractCompactionStrategy.java | 14 +- .../compaction/DateTieredCompactionStrategy.java | 20 +- .../compaction/SizeTieredCompactionStrategy.java | 6 +- .../compaction/TimeWindowCompactionStrategy.java | 4 +- .../apache/cassandra/db/marshal/TypeParser.java| 30 +- .../cassandra/locator/NetworkTopologyStrategy.java | 3 +- .../apache/cassandra/locator/TokenMetadata.java| 414 - .../locator/NetworkTopologyStrategyTest.java | 3 +- .../cassandra/locator/TokenMetadataTest.java | 23 +- 10 files changed, 312 insertions(+), 206 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch cassandra-3.11 into trunk
This is an automated email from the ASF dual-hosted git repository.
snazy pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit ef6c5f8ba9dc18e7a5bacfd5d8461ae5d9f12df4
Merge: 7f634fe 16c96c2
Author: Robert Stupp
AuthorDate: Fri Feb 1 12:27:49 2019 +0100
Merge branch cassandra-3.11 into trunk
.../org/apache/cassandra/db/ColumnFamilyStore.java | 1 -
.../db/compaction/AbstractCompactionStrategy.java | 14 +-
.../compaction/DateTieredCompactionStrategy.java | 20 +-
.../compaction/SizeTieredCompactionStrategy.java | 6 +-
.../compaction/TimeWindowCompactionStrategy.java | 4 +-
.../apache/cassandra/db/marshal/TypeParser.java| 30 +-
.../cassandra/locator/NetworkTopologyStrategy.java | 3 +-
.../apache/cassandra/locator/TokenMetadata.java| 414 -
.../locator/NetworkTopologyStrategyTest.java | 3 +-
.../cassandra/locator/TokenMetadataTest.java | 23 +-
10 files changed, 312 insertions(+), 206 deletions(-)
diff --cc src/java/org/apache/cassandra/locator/NetworkTopologyStrategy.java
index c9f24e0,d48dec3..be63ea1
--- a/src/java/org/apache/cassandra/locator/NetworkTopologyStrategy.java
+++ b/src/java/org/apache/cassandra/locator/NetworkTopologyStrategy.java
@@@ -175,9 -150,9 +176,9 @@@ public class NetworkTopologyStrategy ex
Topology topology = tokenMetadata.getTopology();
// all endpoints in each DC, so we can check when we have exhausted
all the members of a DC
-Multimap allEndpoints =
topology.getDatacenterEndpoints();
+Multimap allEndpoints =
topology.getDatacenterEndpoints();
// all racks in a DC so we can check when we have exhausted all racks
in a DC
- Map> racks =
topology.getDatacenterRacks();
-Map> racks =
topology.getDatacenterRacks();
++Map> racks =
topology.getDatacenterRacks();
assert !allEndpoints.isEmpty() && !racks.isEmpty() : "not aware of
any cluster members";
int dcsToFill = 0;
diff --cc src/java/org/apache/cassandra/locator/TokenMetadata.java
index f95ee0c,8a1d9d0..cb189c8
--- a/src/java/org/apache/cassandra/locator/TokenMetadata.java
+++ b/src/java/org/apache/cassandra/locator/TokenMetadata.java
@@@ -96,19 -95,28 +96,20 @@@ public class TokenMetadat
/* Use this lock for manipulating the token map */
private final ReadWriteLock lock = new ReentrantReadWriteLock(true);
- private volatile ArrayList sortedTokens;
+ private volatile ArrayList sortedTokens; // safe to be read
without a lock, as it's never mutated
+
+ private volatile Topology topology;
- private final Topology topology;
public final IPartitioner partitioner;
-private static final Comparator inetaddressCmp = new
Comparator()
-{
-public int compare(InetAddress o1, InetAddress o2)
-{
-return
ByteBuffer.wrap(o1.getAddress()).compareTo(ByteBuffer.wrap(o2.getAddress()));
-}
-};
-
// signals replication strategies that nodes have joined or left the ring
and they need to recompute ownership
private volatile long ringVersion = 0;
public TokenMetadata()
{
-this(SortedBiMultiValMap.create(null,
inetaddressCmp),
- HashBiMap.create(),
+this(SortedBiMultiValMap.create(),
+ HashBiMap.create(),
- new Topology(),
+ Topology.empty(),
DatabaseDescriptor.getPartitioner());
}
@@@ -186,7 -194,8 +187,8 @@@
try
{
boolean shouldSortTokens = false;
+ Topology.Builder topologyBuilder = topology.unbuild();
-for (InetAddress endpoint : endpointTokens.keySet())
+for (InetAddressAndPort endpoint : endpointTokens.keySet())
{
Collection tokens = endpointTokens.get(endpoint);
@@@ -372,14 -382,30 +375,30 @@@
}
}
-public Optional getReplacementNode(InetAddress endpoint)
+public Optional getReplacementNode(InetAddressAndPort
endpoint)
{
- return
Optional.ofNullable(replacementToOriginal.inverse().get(endpoint));
+ lock.readLock().lock();
+ try
+ {
+ return
Optional.ofNullable(replacementToOriginal.inverse().get(endpoint));
+ }
+ finally
+ {
+ lock.readLock().unlock();
+ }
}
-public Optional getReplacingNode(InetAddress endpoint)
+public Optional getReplacingNode(InetAddressAndPort
endpoint)
{
- return Optional.ofNullable((replacementToOriginal.get(endpoint)));
+ lock.readLock().lock();
+ try
+ {
+ return Optional.ofNullable((replacementToOriginal.get(endpoint)));
+ }
+ finally
+ {
+ lock.readLock().unlock();
+ }
}
public void removeBootstrapTokens(Collection tokens)
[jira] [Comment Edited] (CASSANDRA-15004) Anti-compaction briefly removes sstables from the read path
[ https://issues.apache.org/jira/browse/CASSANDRA-15004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16758123#comment-16758123 ] Benedict edited comment on CASSANDRA-15004 at 2/1/19 9:59 AM: -- Nice catch, and it looks like a good fix to me. (+1) was (Author: benedict): Nice catch, and it looks like a good fix to me. > Anti-compaction briefly removes sstables from the read path > --- > > Key: CASSANDRA-15004 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15004 > Project: Cassandra > Issue Type: Bug >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Major > Fix For: 4.0, 3.0.x, 3.11.x > > > Since we use multiple sstable rewriters in anticompaction, the first call to > prepareToCommit will remove the original sstables from the tracker view > before the other rewriters add their sstables. This creates a brief window > where reads can miss data. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14647) Reading cardinality from Statistics.db failed
[
https://issues.apache.org/jira/browse/CASSANDRA-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-14647:
Reproduced In: 3.0.17, 3.0.15 (was: 3.0.15, 3.0.17)
Reviewer: Marcus Eriksson
> Reading cardinality from Statistics.db failed
> -
>
> Key: CASSANDRA-14647
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14647
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
> Environment: Clients are doing only writes with Local One, cluster
> consist of 3 regions with RF3.
> Storage is configured wth jbod/XFS on 10 x 1Tb disks
> IOPS limit for each disk 500 (total 5000 iops)
> Bandwith for each disk 60mb/s (600 total)
> OS is Debian linux.
>Reporter: Vitali Djatsuk
>Assignee: Venkata Harikrishna Nukala
>Priority: Minor
> Fix For: 3.0.x
>
> Attachments: 14647-trunk-1.patch,
> cassandra_compaction_pending_tasks_7days.png
>
>
> There is some issue with sstable metadata which is visible in system.log, the
> messages says:
> {noformat}
> WARN [Thread-6] 2018-07-25 07:12:47,928 SSTableReader.java:249 - Reading
> cardinality from Statistics.db failed for
> /opt/data/disk5/data/keyspace/table/mc-big-Data.db.{noformat}
> Although there is no such file.
> The message has appeared after i've changed the compaction strategy from
> SizeTiered to Leveled. Compaction strategy has been changed region by region
> (total 3 regions) and it has coincided with the double client write traffic
> increase.
> I have tried to run nodetool scrub to rebuilt the sstable, but that does not
> fix the issue.
> So very hard to define the steps to reproduce, probably it will be:
> # run stress tool with write traffic
> # under load change compaction strategy from SireTiered to Leveled for the
> bunch of hosts
> # add more write traffic
> Reading the code it is said that if this metadata is broken, then "estimating
> the keys will be done using index summary".
>
> [https://github.com/apache/cassandra/blob/cassandra-3.0.17/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java#L247]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14647) Reading cardinality from Statistics.db failed
[
https://issues.apache.org/jira/browse/CASSANDRA-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson reassigned CASSANDRA-14647:
---
Assignee: Venkata Harikrishna Nukala (was: Marcus Eriksson)
> Reading cardinality from Statistics.db failed
> -
>
> Key: CASSANDRA-14647
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14647
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
> Environment: Clients are doing only writes with Local One, cluster
> consist of 3 regions with RF3.
> Storage is configured wth jbod/XFS on 10 x 1Tb disks
> IOPS limit for each disk 500 (total 5000 iops)
> Bandwith for each disk 60mb/s (600 total)
> OS is Debian linux.
>Reporter: Vitali Djatsuk
>Assignee: Venkata Harikrishna Nukala
>Priority: Minor
> Fix For: 3.0.x
>
> Attachments: 14647-trunk-1.patch,
> cassandra_compaction_pending_tasks_7days.png
>
>
> There is some issue with sstable metadata which is visible in system.log, the
> messages says:
> {noformat}
> WARN [Thread-6] 2018-07-25 07:12:47,928 SSTableReader.java:249 - Reading
> cardinality from Statistics.db failed for
> /opt/data/disk5/data/keyspace/table/mc-big-Data.db.{noformat}
> Although there is no such file.
> The message has appeared after i've changed the compaction strategy from
> SizeTiered to Leveled. Compaction strategy has been changed region by region
> (total 3 regions) and it has coincided with the double client write traffic
> increase.
> I have tried to run nodetool scrub to rebuilt the sstable, but that does not
> fix the issue.
> So very hard to define the steps to reproduce, probably it will be:
> # run stress tool with write traffic
> # under load change compaction strategy from SireTiered to Leveled for the
> bunch of hosts
> # add more write traffic
> Reading the code it is said that if this metadata is broken, then "estimating
> the keys will be done using index summary".
>
> [https://github.com/apache/cassandra/blob/cassandra-3.0.17/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java#L247]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14647) Reading cardinality from Statistics.db failed
[
https://issues.apache.org/jira/browse/CASSANDRA-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16758140#comment-16758140
]
Venkata Harikrishna Nukala edited comment on CASSANDRA-14647 at 2/1/19 9:35 AM:
[~krummas] I have created a patch for this and uploaded to this ticket
[^14647-trunk-1.patch]
was (Author: n.v.harikrishna):
[~krummas] I have created a patch for this and upload to this ticket
[^14647-trunk-1.patch]
> Reading cardinality from Statistics.db failed
> -
>
> Key: CASSANDRA-14647
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14647
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
> Environment: Clients are doing only writes with Local One, cluster
> consist of 3 regions with RF3.
> Storage is configured wth jbod/XFS on 10 x 1Tb disks
> IOPS limit for each disk 500 (total 5000 iops)
> Bandwith for each disk 60mb/s (600 total)
> OS is Debian linux.
>Reporter: Vitali Djatsuk
>Assignee: Marcus Eriksson
>Priority: Minor
> Fix For: 3.0.x
>
> Attachments: 14647-trunk-1.patch,
> cassandra_compaction_pending_tasks_7days.png
>
>
> There is some issue with sstable metadata which is visible in system.log, the
> messages says:
> {noformat}
> WARN [Thread-6] 2018-07-25 07:12:47,928 SSTableReader.java:249 - Reading
> cardinality from Statistics.db failed for
> /opt/data/disk5/data/keyspace/table/mc-big-Data.db.{noformat}
> Although there is no such file.
> The message has appeared after i've changed the compaction strategy from
> SizeTiered to Leveled. Compaction strategy has been changed region by region
> (total 3 regions) and it has coincided with the double client write traffic
> increase.
> I have tried to run nodetool scrub to rebuilt the sstable, but that does not
> fix the issue.
> So very hard to define the steps to reproduce, probably it will be:
> # run stress tool with write traffic
> # under load change compaction strategy from SireTiered to Leveled for the
> bunch of hosts
> # add more write traffic
> Reading the code it is said that if this metadata is broken, then "estimating
> the keys will be done using index summary".
>
> [https://github.com/apache/cassandra/blob/cassandra-3.0.17/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java#L247]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14647) Reading cardinality from Statistics.db failed
[
https://issues.apache.org/jira/browse/CASSANDRA-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16758140#comment-16758140
]
Venkata Harikrishna Nukala commented on CASSANDRA-14647:
[~krummas] I have created a patch for this and upload to this ticket
[^14647-trunk-1.patch]
> Reading cardinality from Statistics.db failed
> -
>
> Key: CASSANDRA-14647
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14647
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
> Environment: Clients are doing only writes with Local One, cluster
> consist of 3 regions with RF3.
> Storage is configured wth jbod/XFS on 10 x 1Tb disks
> IOPS limit for each disk 500 (total 5000 iops)
> Bandwith for each disk 60mb/s (600 total)
> OS is Debian linux.
>Reporter: Vitali Djatsuk
>Assignee: Marcus Eriksson
>Priority: Minor
> Fix For: 3.0.x
>
> Attachments: 14647-trunk-1.patch,
> cassandra_compaction_pending_tasks_7days.png
>
>
> There is some issue with sstable metadata which is visible in system.log, the
> messages says:
> {noformat}
> WARN [Thread-6] 2018-07-25 07:12:47,928 SSTableReader.java:249 - Reading
> cardinality from Statistics.db failed for
> /opt/data/disk5/data/keyspace/table/mc-big-Data.db.{noformat}
> Although there is no such file.
> The message has appeared after i've changed the compaction strategy from
> SizeTiered to Leveled. Compaction strategy has been changed region by region
> (total 3 regions) and it has coincided with the double client write traffic
> increase.
> I have tried to run nodetool scrub to rebuilt the sstable, but that does not
> fix the issue.
> So very hard to define the steps to reproduce, probably it will be:
> # run stress tool with write traffic
> # under load change compaction strategy from SireTiered to Leveled for the
> bunch of hosts
> # add more write traffic
> Reading the code it is said that if this metadata is broken, then "estimating
> the keys will be done using index summary".
>
> [https://github.com/apache/cassandra/blob/cassandra-3.0.17/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java#L247]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14647) Reading cardinality from Statistics.db failed
[
https://issues.apache.org/jira/browse/CASSANDRA-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venkata Harikrishna Nukala updated CASSANDRA-14647:
---
Attachment: 14647-trunk-1.patch
> Reading cardinality from Statistics.db failed
> -
>
> Key: CASSANDRA-14647
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14647
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
> Environment: Clients are doing only writes with Local One, cluster
> consist of 3 regions with RF3.
> Storage is configured wth jbod/XFS on 10 x 1Tb disks
> IOPS limit for each disk 500 (total 5000 iops)
> Bandwith for each disk 60mb/s (600 total)
> OS is Debian linux.
>Reporter: Vitali Djatsuk
>Assignee: Marcus Eriksson
>Priority: Minor
> Fix For: 3.0.x
>
> Attachments: 14647-trunk-1.patch,
> cassandra_compaction_pending_tasks_7days.png
>
>
> There is some issue with sstable metadata which is visible in system.log, the
> messages says:
> {noformat}
> WARN [Thread-6] 2018-07-25 07:12:47,928 SSTableReader.java:249 - Reading
> cardinality from Statistics.db failed for
> /opt/data/disk5/data/keyspace/table/mc-big-Data.db.{noformat}
> Although there is no such file.
> The message has appeared after i've changed the compaction strategy from
> SizeTiered to Leveled. Compaction strategy has been changed region by region
> (total 3 regions) and it has coincided with the double client write traffic
> increase.
> I have tried to run nodetool scrub to rebuilt the sstable, but that does not
> fix the issue.
> So very hard to define the steps to reproduce, probably it will be:
> # run stress tool with write traffic
> # under load change compaction strategy from SireTiered to Leveled for the
> bunch of hosts
> # add more write traffic
> Reading the code it is said that if this metadata is broken, then "estimating
> the keys will be done using index summary".
>
> [https://github.com/apache/cassandra/blob/cassandra-3.0.17/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java#L247]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15004) Anti-compaction briefly removes sstables from the read path
[ https://issues.apache.org/jira/browse/CASSANDRA-15004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16758121#comment-16758121 ] Marcus Eriksson edited comment on CASSANDRA-15004 at 2/1/19 9:09 AM: - +1 pushed a unit test: [3.0|https://github.com/krummas/cassandra/commits/blake/15004-3.0] [3.11|https://github.com/krummas/cassandra/commits/blake/15004-3.11] [trunk|https://github.com/krummas/cassandra/commits/blake/15004-trunk] edit: not sure what is going on with the dtests though, probably need a restart was (Author: krummas): +1 pushed a unit test: [3.0|https://github.com/krummas/cassandra/commits/blake/15004-3.0] [3.11|https://github.com/krummas/cassandra/commits/blake/15004-3.11] [trunk|https://github.com/krummas/cassandra/commits/blake/15004-trunk] > Anti-compaction briefly removes sstables from the read path > --- > > Key: CASSANDRA-15004 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15004 > Project: Cassandra > Issue Type: Bug >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Major > Fix For: 4.0, 3.0.x, 3.11.x > > > Since we use multiple sstable rewriters in anticompaction, the first call to > prepareToCommit will remove the original sstables from the tracker view > before the other rewriters add their sstables. This creates a brief window > where reads can miss data. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15004) Anti-compaction briefly removes sstables from the read path
[ https://issues.apache.org/jira/browse/CASSANDRA-15004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16758123#comment-16758123 ] Benedict commented on CASSANDRA-15004: -- Nice catch, and it looks like a good fix to me. > Anti-compaction briefly removes sstables from the read path > --- > > Key: CASSANDRA-15004 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15004 > Project: Cassandra > Issue Type: Bug >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Major > Fix For: 4.0, 3.0.x, 3.11.x > > > Since we use multiple sstable rewriters in anticompaction, the first call to > prepareToCommit will remove the original sstables from the tracker view > before the other rewriters add their sstables. This creates a brief window > where reads can miss data. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15004) Anti-compaction briefly removes sstables from the read path
[ https://issues.apache.org/jira/browse/CASSANDRA-15004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16758121#comment-16758121 ] Marcus Eriksson commented on CASSANDRA-15004: - +1 pushed a unit test: [3.0|https://github.com/krummas/cassandra/commits/blake/15004-3.0] [3.11|https://github.com/krummas/cassandra/commits/blake/15004-3.11] [trunk|https://github.com/krummas/cassandra/commits/blake/15004-trunk] > Anti-compaction briefly removes sstables from the read path > --- > > Key: CASSANDRA-15004 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15004 > Project: Cassandra > Issue Type: Bug >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Major > Fix For: 4.0, 3.0.x, 3.11.x > > > Since we use multiple sstable rewriters in anticompaction, the first call to > prepareToCommit will remove the original sstables from the tracker view > before the other rewriters add their sstables. This creates a brief window > where reads can miss data. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
