[jira] [Created] (CASSANDRA-18835) CassandraTableRepairManager should re-interrupt thread after catching InterruptedException
dan jatnieks created CASSANDRA-18835:
Summary: CassandraTableRepairManager should re-interrupt thread
after catching InterruptedException
Key: CASSANDRA-18835
URL: https://issues.apache.org/jira/browse/CASSANDRA-18835
Project: Cassandra
Issue Type: Bug
Reporter: dan jatnieks
Assignee: dan jatnieks
SonarCloud reports a bug on 4.0 branch in
{{CassandraTableRepairManager.snapshot() }} because {{catch}} could include an
{{InterruptedException}} that should either re-throw or re-interrupt.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18366) Test failure: org.apache.cassandra.distributed.test.FailingRepairTest - testFailingMessage[VALIDATION_REQ/parallel/true]
[
https://issues.apache.org/jira/browse/CASSANDRA-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761408#comment-17761408
]
dan jatnieks commented on CASSANDRA-18366:
--
{quote}the 4.1 version of FailingRepairTest, I see a similar change there:
[https://github.com/apache/cassandra/blob/20125c5053586ab46fa341ac71cc525b64932873/test/distributed/org/apache/cassandra/distributed/test/FailingRepairTest.java#L169-L172]
Maybe it's better to port that change back to 4.0 to keep things consistent?
{quote}
Well, just porting that one change to {{FailingRepairTest}} isn't going to
work. It was part of CASSANDRA-17116 which also added code to have the
{{InstanceKiller}} shutdown the instance when {{killCurrentJVM}} is called.
[https://github.com/apache/cassandra/blob/20125c5053586ab46fa341ac71cc525b64932873/test/distributed/org/apache/cassandra/distributed/impl/Instance.java#L683]
Since that's not in 4.0, the instance is not shutdown, and porting just that
change to {{FailingRepairTest}} from 4.1 doesn't fix the test.
So, unless we want to port all of CASSANDRA-17116 back to 4.0, the original
change I proposed that just re-starts gossip should be okay for 4.0.
> Test failure: org.apache.cassandra.distributed.test.FailingRepairTest -
> testFailingMessage[VALIDATION_REQ/parallel/true]
>
>
> Key: CASSANDRA-18366
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18366
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Brandon Williams
>Assignee: dan jatnieks
>Priority: Normal
> Fix For: 4.0.x
>
>
> First seen
> [here|https://app.circleci.com/pipelines/github/driftx/cassandra/928/workflows/f4e93a72-d4aa-47a2-996f-aa3fb018d848/jobs/16206]
> this test times out for me consistently on both j8 and j11 where 4.1 and
> trunk do not.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18366) Test failure: org.apache.cassandra.distributed.test.FailingRepairTest - testFailingMessage[VALIDATION_REQ/parallel/true]
[ https://issues.apache.org/jira/browse/CASSANDRA-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761389#comment-17761389 ] dan jatnieks commented on CASSANDRA-18366: -- Ok, I'll do that and submit a new PR. > Test failure: org.apache.cassandra.distributed.test.FailingRepairTest - > testFailingMessage[VALIDATION_REQ/parallel/true] > > > Key: CASSANDRA-18366 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18366 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/java >Reporter: Brandon Williams >Assignee: dan jatnieks >Priority: Normal > Fix For: 4.0.x > > > First seen > [here|https://app.circleci.com/pipelines/github/driftx/cassandra/928/workflows/f4e93a72-d4aa-47a2-996f-aa3fb018d848/jobs/16206] > this test times out for me consistently on both j8 and j11 where 4.1 and > trunk do not. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18366) Test failure: org.apache.cassandra.distributed.test.FailingRepairTest - testFailingMessage[VALIDATION_REQ/parallel/true]
[
https://issues.apache.org/jira/browse/CASSANDRA-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761384#comment-17761384
]
dan jatnieks commented on CASSANDRA-18366:
--
> Do you know why this wasn't a problem on 4.1 or later?
tbh, I hadn't checked, but now that I take a look at the 4.1 version of
{{FailingRepairTest}}, I see a similar change there:
https://github.com/apache/cassandra/blob/20125c5053586ab46fa341ac71cc525b64932873/test/distributed/org/apache/cassandra/distributed/test/FailingRepairTest.java#L169-L172
Maybe it's better to port that change back to 4.0 to keep things consistent?
> Test failure: org.apache.cassandra.distributed.test.FailingRepairTest -
> testFailingMessage[VALIDATION_REQ/parallel/true]
>
>
> Key: CASSANDRA-18366
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18366
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Brandon Williams
>Assignee: dan jatnieks
>Priority: Normal
> Fix For: 4.0.x
>
>
> First seen
> [here|https://app.circleci.com/pipelines/github/driftx/cassandra/928/workflows/f4e93a72-d4aa-47a2-996f-aa3fb018d848/jobs/16206]
> this test times out for me consistently on both j8 and j11 where 4.1 and
> trunk do not.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18366) Test failure: org.apache.cassandra.distributed.test.FailingRepairTest - testFailingMessage[VALIDATION_REQ/parallel/true]
[ https://issues.apache.org/jira/browse/CASSANDRA-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dan jatnieks updated CASSANDRA-18366: - Test and Documentation Plan: Started CircleCI pre-commit tests here (w/limited resources): https://app.circleci.com/pipelines/github/djatnieks/cassandra?branch=CASSANDRA-18366 Status: Patch Available (was: Open) > Test failure: org.apache.cassandra.distributed.test.FailingRepairTest - > testFailingMessage[VALIDATION_REQ/parallel/true] > > > Key: CASSANDRA-18366 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18366 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/java >Reporter: Brandon Williams >Assignee: dan jatnieks >Priority: Normal > Fix For: 4.0.x > > > First seen > [here|https://app.circleci.com/pipelines/github/driftx/cassandra/928/workflows/f4e93a72-d4aa-47a2-996f-aa3fb018d848/jobs/16206] > this test times out for me consistently on both j8 and j11 where 4.1 and > trunk do not. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18366) Test failure: org.apache.cassandra.distributed.test.FailingRepairTest - testFailingMessage[VALIDATION_REQ/parallel/true]
[ https://issues.apache.org/jira/browse/CASSANDRA-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dan jatnieks reassigned CASSANDRA-18366: Assignee: dan jatnieks > Test failure: org.apache.cassandra.distributed.test.FailingRepairTest - > testFailingMessage[VALIDATION_REQ/parallel/true] > > > Key: CASSANDRA-18366 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18366 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/java >Reporter: Brandon Williams >Assignee: dan jatnieks >Priority: Normal > Fix For: 4.0.x > > > First seen > [here|https://app.circleci.com/pipelines/github/driftx/cassandra/928/workflows/f4e93a72-d4aa-47a2-996f-aa3fb018d848/jobs/16206] > this test times out for me consistently on both j8 and j11 where 4.1 and > trunk do not. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18366) Test failure: org.apache.cassandra.distributed.test.FailingRepairTest - testFailingMessage[VALIDATION_REQ/parallel/true]
[
https://issues.apache.org/jira/browse/CASSANDRA-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761371#comment-17761371
]
dan jatnieks commented on CASSANDRA-18366:
--
I ran into this issue and found that when {{DefaultFSErrorHandler}} handles
{{die}} it stops transports, including gossip. When the next test starts,
gossip isn't running and the test hangs. The solution I found is to make sure
gossip is started before starting each test.
Here is a [4.0 PR |https://github.com/apache/cassandra/pull/2659]
> Test failure: org.apache.cassandra.distributed.test.FailingRepairTest -
> testFailingMessage[VALIDATION_REQ/parallel/true]
>
>
> Key: CASSANDRA-18366
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18366
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Brandon Williams
>Priority: Normal
> Fix For: 4.0.x
>
>
> First seen
> [here|https://app.circleci.com/pipelines/github/driftx/cassandra/928/workflows/f4e93a72-d4aa-47a2-996f-aa3fb018d848/jobs/16206]
> this test times out for me consistently on both j8 and j11 where 4.1 and
> trunk do not.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18582) BulkLoader withCipherSuites option is ignored
[
https://issues.apache.org/jira/browse/CASSANDRA-18582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17740855#comment-17740855
]
dan jatnieks commented on CASSANDRA-18582:
--
[~e.dimitrova] Thanks for starting the CI runs and for letting me know about
the tests failing in {{SSTableLoaderEncryptionOptionsTest}} ... that's actually
one of the tests I was working on in another ticket, so I should have noticed
:(
I fixed the test, rebased, and pushed updates to each branch.
I ended up squashing the changes though - so for clarity, I'll just describe
the problem with the test here.
The {{EncryptionOptions.cipherSuitesArray}} method was returning an empty array
when {{cipher_suites}} is {{null}}. However, something downstream of
{{RemoteEndpointAwareJdkSSLOptions}} doesn't like an empty array - I'm not sure
what, but just returning {{null}} solves it.
I also added an explicit {{--ssl-ciphers}} option to
{{SSTableLoaderEncryptionOptionsTest.bulkLoaderSuccessfullyStreamsOverSsl}} to
cover the case where the cipher suite option is being set.
And, I left
{{bulkLoaderSuccessfullyStreamsOverSslWithDeprecatedSslStoragePort}} without
specifying a {{cipher-suite}} option to cover the default (null) case.
Thanks again for the help!
> BulkLoader withCipherSuites option is ignored
> -
>
> Key: CASSANDRA-18582
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18582
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/bulk load
>Reporter: dan jatnieks
>Assignee: dan jatnieks
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.x
>
>
> The {{withCipherSuites}} option of {{BulkLoader}} is being ignored. It seems
> that since CASSANDRA-16362 the {{BulkLoader.buildSSLOptions}} method no
> longer applies the cipher suite options provided by
> {{clientEncryptionOptions}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18617) Disable the deprecated keyspace/table thresholds and convert them to Guardrails
[ https://issues.apache.org/jira/browse/CASSANDRA-18617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17740382#comment-17740382 ] dan jatnieks edited comment on CASSANDRA-18617 at 7/6/23 3:17 AM: -- I agree ... let's try the static table names and see how that goes. I pushed an update and opened a PR to comment on details: [https://github.com/apache/cassandra/pull/2467] was (Author: djatnieks): I agree ... let's try the static table names and see how that goes. I pushed an update and opened a PR to comment on details: [https://github.com/apache/cassandra/pull/2467] > Disable the deprecated keyspace/table thresholds and convert them to > Guardrails > --- > > Key: CASSANDRA-18617 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18617 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Guardrails >Reporter: dan jatnieks >Assignee: dan jatnieks >Priority: Normal > Fix For: 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > The non-guardrail thresholds 'keyspace_count_warn_threshold' and > 'table_count_warn_threshold' configuration settings were first added with > CASSANDRA-16309 in 4.0-beta4 and have subsequently been deprecated since > 4.1-alpha in CASSANDRA-17195 when they were replaced/migrated to guardrails > as part of CEP-3 (Guardrails). > These thresholds should now be removed from cassandra.yaml, while still > allowed in existing yaml files. > The old thresholds will be disabled by removing their default values from > Config.java, and any existing values for these thresholds will be converted > to the new guardrails using the '@Replaces' tag on the corresponding > guardrail values. > Since the old thresholds considered the number of system keyspace/tables in > their values, the '@Replaces' conversion will subtract the current number of > system tables from the old value and log a descriptive message. > See dev list discussion: > https://lists.apache.org/thread/0zjg08hrd6xv7lhvo96frz456b2rvr8b -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18617) Disable the deprecated keyspace/table thresholds and convert them to Guardrails
[ https://issues.apache.org/jira/browse/CASSANDRA-18617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dan jatnieks updated CASSANDRA-18617: - Test and Documentation Plan: Unit tests added and submitted free tier CI pre-commit pipeline: [https://app.circleci.com/pipelines/github/djatnieks/cassandra?branch=CASSANDRA-18617] Status: Patch Available (was: In Progress) > Disable the deprecated keyspace/table thresholds and convert them to > Guardrails > --- > > Key: CASSANDRA-18617 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18617 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Guardrails >Reporter: dan jatnieks >Assignee: dan jatnieks >Priority: Normal > Fix For: 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > The non-guardrail thresholds 'keyspace_count_warn_threshold' and > 'table_count_warn_threshold' configuration settings were first added with > CASSANDRA-16309 in 4.0-beta4 and have subsequently been deprecated since > 4.1-alpha in CASSANDRA-17195 when they were replaced/migrated to guardrails > as part of CEP-3 (Guardrails). > These thresholds should now be removed from cassandra.yaml, while still > allowed in existing yaml files. > The old thresholds will be disabled by removing their default values from > Config.java, and any existing values for these thresholds will be converted > to the new guardrails using the '@Replaces' tag on the corresponding > guardrail values. > Since the old thresholds considered the number of system keyspace/tables in > their values, the '@Replaces' conversion will subtract the current number of > system tables from the old value and log a descriptive message. > See dev list discussion: > https://lists.apache.org/thread/0zjg08hrd6xv7lhvo96frz456b2rvr8b -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18617) Disable the deprecated keyspace/table thresholds and convert them to Guardrails
[ https://issues.apache.org/jira/browse/CASSANDRA-18617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17740382#comment-17740382 ] dan jatnieks commented on CASSANDRA-18617: -- I agree ... let's try the static table names and see how that goes. I pushed an update and opened a PR to comment on details: [https://github.com/apache/cassandra/pull/2467] > Disable the deprecated keyspace/table thresholds and convert them to > Guardrails > --- > > Key: CASSANDRA-18617 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18617 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Guardrails >Reporter: dan jatnieks >Assignee: dan jatnieks >Priority: Normal > Fix For: 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > The non-guardrail thresholds 'keyspace_count_warn_threshold' and > 'table_count_warn_threshold' configuration settings were first added with > CASSANDRA-16309 in 4.0-beta4 and have subsequently been deprecated since > 4.1-alpha in CASSANDRA-17195 when they were replaced/migrated to guardrails > as part of CEP-3 (Guardrails). > These thresholds should now be removed from cassandra.yaml, while still > allowed in existing yaml files. > The old thresholds will be disabled by removing their default values from > Config.java, and any existing values for these thresholds will be converted > to the new guardrails using the '@Replaces' tag on the corresponding > guardrail values. > Since the old thresholds considered the number of system keyspace/tables in > their values, the '@Replaces' conversion will subtract the current number of > system tables from the old value and log a descriptive message. > See dev list discussion: > https://lists.apache.org/thread/0zjg08hrd6xv7lhvo96frz456b2rvr8b -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18617) Disable the deprecated keyspace/table thresholds and convert them to Guardrails
[
https://issues.apache.org/jira/browse/CASSANDRA-18617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17738306#comment-17738306
]
dan jatnieks commented on CASSANDRA-18617:
--
{quote}keeping a static int on SchemaConstants storing the number of system
tables.
{quote}
Hmm, I think I'd prefer the list of table names explicitly myself; a pain, but
more clear imo. With a list of names, the unit test could flag the specific
table name(s) that are missing; using a number it could only flag that the
count is wrong.
Here's another idea - use {{@Replaces}} tag in {{Config}} only to rename
{{table_count_warn_threshold}} to something else that is then only used in
{{GuardrailsOptions}} to perform the conversion. Something like this:
{noformat}
@Replaces(oldName = "table_count_warn_threshold", converter =
Converters.IDENTITY, deprecated = true)
public volatile int convert_deprecated_table_count_warn_threshold_to_guardrail
= -1;
{noformat}
This follows the normal {{@Replaces}} mechanism, and hopefully, with a good
enough replacement name and comment, it will be reasonably clear that the
replacement is used as an intermediate value for use after the system is fully
initialized.
> Disable the deprecated keyspace/table thresholds and convert them to
> Guardrails
> ---
>
> Key: CASSANDRA-18617
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18617
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Guardrails
>Reporter: dan jatnieks
>Assignee: dan jatnieks
>Priority: Normal
> Fix For: 5.x
>
>
> The non-guardrail thresholds 'keyspace_count_warn_threshold' and
> 'table_count_warn_threshold' configuration settings were first added with
> CASSANDRA-16309 in 4.0-beta4 and have subsequently been deprecated since
> 4.1-alpha in CASSANDRA-17195 when they were replaced/migrated to guardrails
> as part of CEP-3 (Guardrails).
> These thresholds should now be removed from cassandra.yaml, while still
> allowed in existing yaml files.
> The old thresholds will be disabled by removing their default values from
> Config.java, and any existing values for these thresholds will be converted
> to the new guardrails using the '@Replaces' tag on the corresponding
> guardrail values.
> Since the old thresholds considered the number of system keyspace/tables in
> their values, the '@Replaces' conversion will subtract the current number of
> system tables from the old value and log a descriptive message.
> See dev list discussion:
> https://lists.apache.org/thread/0zjg08hrd6xv7lhvo96frz456b2rvr8b
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18617) Disable the deprecated keyspace/table thresholds and convert them to Guardrails
[
https://issues.apache.org/jira/browse/CASSANDRA-18617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17737922#comment-17737922
]
dan jatnieks commented on CASSANDRA-18617:
--
I think that's a good idea to check for the deprecated threshold value in
{{{}GuardrailsOptions{}}}.
My only worry would be if others might consider a bit confusing to still have
{{table_count_warn_threshold}} in {{Config.java}}?
I have a WIP branch here:
https://github.com/djatnieks/cassandra/tree/CASSANDRA-18617
> Disable the deprecated keyspace/table thresholds and convert them to
> Guardrails
> ---
>
> Key: CASSANDRA-18617
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18617
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Guardrails
>Reporter: dan jatnieks
>Assignee: dan jatnieks
>Priority: Normal
> Fix For: 5.x
>
>
> The non-guardrail thresholds 'keyspace_count_warn_threshold' and
> 'table_count_warn_threshold' configuration settings were first added with
> CASSANDRA-16309 in 4.0-beta4 and have subsequently been deprecated since
> 4.1-alpha in CASSANDRA-17195 when they were replaced/migrated to guardrails
> as part of CEP-3 (Guardrails).
> These thresholds should now be removed from cassandra.yaml, while still
> allowed in existing yaml files.
> The old thresholds will be disabled by removing their default values from
> Config.java, and any existing values for these thresholds will be converted
> to the new guardrails using the '@Replaces' tag on the corresponding
> guardrail values.
> Since the old thresholds considered the number of system keyspace/tables in
> their values, the '@Replaces' conversion will subtract the current number of
> system tables from the old value and log a descriptive message.
> See dev list discussion:
> https://lists.apache.org/thread/0zjg08hrd6xv7lhvo96frz456b2rvr8b
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18582) BulkLoader withCipherSuites option is ignored
[
https://issues.apache.org/jira/browse/CASSANDRA-18582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks updated CASSANDRA-18582:
-
Test and Documentation Plan:
Run pre-commit tests via CircleCI - free tier, and lack of resources may be
responsible to many errors?
https://app.circleci.com/pipelines/github/djatnieks/cassandra?branch=CASSANDRA-18582-4.0
https://app.circleci.com/pipelines/github/djatnieks/cassandra?branch=CASSANDRA-18582-4.1
https://app.circleci.com/pipelines/github/djatnieks/cassandra?branch=CASSANDRA-18582-trunk
Status: Patch Available (was: In Progress)
> BulkLoader withCipherSuites option is ignored
> -
>
> Key: CASSANDRA-18582
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18582
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/bulk load
>Reporter: dan jatnieks
>Assignee: dan jatnieks
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.x
>
>
> The {{withCipherSuites}} option of {{BulkLoader}} is being ignored. It seems
> that since CASSANDRA-16362 the {{BulkLoader.buildSSLOptions}} method no
> longer applies the cipher suite options provided by
> {{clientEncryptionOptions}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18617) Disable the deprecated keyspace/table thresholds and convert them to Guardrails
[
https://issues.apache.org/jira/browse/CASSANDRA-18617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17737404#comment-17737404
]
dan jatnieks commented on CASSANDRA-18617:
--
Part of this change is to add converters that will take the old threshold
value, including the number of system keyspaces/tables, and convert that to a
guardrail value that excludes system keyspaces/tables.
To determine the number of system keyspaces, there is an existing
{{SchemaConstants.getSystemKeyspaces}} method that seems reasonable to use.
However, I'm finding it harder to find an existing way to get the number of
system tables.
Some test code uses {{metadata}} method in system keyspace classes to get
metadata including the tables, e.g. {{{}SystemKeyspace.metadata{}}}. However,
aside from maybe doing more work than is needed, this doesn't work when called
from converter code because just accessing {{SystemKeyspace}} triggers static
initialization, and at some point that leads to needing some value from
{{{}DatabaseDescriptor{}}}. But the initialization of {{DatabaseDescriptor}} is
what triggered the loading of the config in the first place (so it ends up with
an NPE trying to access an uninitialized field).
So I think something is needed similar to {{getSystemKeyspaces}} also located
in {{SchemaConstants}} where static initialization problems are less likely. A
downside to this approach is that it depends on being updated whenever a new
system table is added, but I supposed it's no worse than the existing situation
with {{{}getSystemKeyspaces{}}}.
> Disable the deprecated keyspace/table thresholds and convert them to
> Guardrails
> ---
>
> Key: CASSANDRA-18617
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18617
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Guardrails
>Reporter: dan jatnieks
>Assignee: dan jatnieks
>Priority: Normal
> Fix For: 5.x
>
>
> The non-guardrail thresholds 'keyspace_count_warn_threshold' and
> 'table_count_warn_threshold' configuration settings were first added with
> CASSANDRA-16309 in 4.0-beta4 and have subsequently been deprecated since
> 4.1-alpha in CASSANDRA-17195 when they were replaced/migrated to guardrails
> as part of CEP-3 (Guardrails).
> These thresholds should now be removed from cassandra.yaml, while still
> allowed in existing yaml files.
> The old thresholds will be disabled by removing their default values from
> Config.java, and any existing values for these thresholds will be converted
> to the new guardrails using the '@Replaces' tag on the corresponding
> guardrail values.
> Since the old thresholds considered the number of system keyspace/tables in
> their values, the '@Replaces' conversion will subtract the current number of
> system tables from the old value and log a descriptive message.
> See dev list discussion:
> https://lists.apache.org/thread/0zjg08hrd6xv7lhvo96frz456b2rvr8b
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18617) Disable the deprecated keyspace/table thresholds and convert them to Guardrails
[ https://issues.apache.org/jira/browse/CASSANDRA-18617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dan jatnieks updated CASSANDRA-18617: - Change Category: Operability Complexity: Normal Component/s: Feature/Guardrails Status: Open (was: Triage Needed) > Disable the deprecated keyspace/table thresholds and convert them to > Guardrails > --- > > Key: CASSANDRA-18617 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18617 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Guardrails >Reporter: dan jatnieks >Assignee: dan jatnieks >Priority: Normal > Fix For: 5.x > > > The non-guardrail thresholds 'keyspace_count_warn_threshold' and > 'table_count_warn_threshold' configuration settings were first added with > CASSANDRA-16309 in 4.0-beta4 and have subsequently been deprecated since > 4.1-alpha in CASSANDRA-17195 when they were replaced/migrated to guardrails > as part of CEP-3 (Guardrails). > These thresholds should now be removed from cassandra.yaml, while still > allowed in existing yaml files. > The old thresholds will be disabled by removing their default values from > Config.java, and any existing values for these thresholds will be converted > to the new guardrails using the '@Replaces' tag on the corresponding > guardrail values. > Since the old thresholds considered the number of system keyspace/tables in > their values, the '@Replaces' conversion will subtract the current number of > system tables from the old value and log a descriptive message. > See dev list discussion: > https://lists.apache.org/thread/0zjg08hrd6xv7lhvo96frz456b2rvr8b -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18617) Disable the deprecated keyspace/table thresholds and convert them to Guardrails
dan jatnieks created CASSANDRA-18617: Summary: Disable the deprecated keyspace/table thresholds and convert them to Guardrails Key: CASSANDRA-18617 URL: https://issues.apache.org/jira/browse/CASSANDRA-18617 Project: Cassandra Issue Type: Improvement Reporter: dan jatnieks Assignee: dan jatnieks The non-guardrail thresholds 'keyspace_count_warn_threshold' and 'table_count_warn_threshold' configuration settings were first added with CASSANDRA-16309 in 4.0-beta4 and have subsequently been deprecated since 4.1-alpha in CASSANDRA-17195 when they were replaced/migrated to guardrails as part of CEP-3 (Guardrails). These thresholds should now be removed from cassandra.yaml, while still allowed in existing yaml files. The old thresholds will be disabled by removing their default values from Config.java, and any existing values for these thresholds will be converted to the new guardrails using the '@Replaces' tag on the corresponding guardrail values. Since the old thresholds considered the number of system keyspace/tables in their values, the '@Replaces' conversion will subtract the current number of system tables from the old value and log a descriptive message. See dev list discussion: https://lists.apache.org/thread/0zjg08hrd6xv7lhvo96frz456b2rvr8b -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18540) negotiatedProtocolMustBeAcceptedProtocolTest tests fail with "TLSv1.1 failed to negotiate" on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732183#comment-17732183
]
dan jatnieks commented on CASSANDRA-18540:
--
Makes sense to me - I've pushed a new commit to each branch below that adds the
check for TLSv1.2 to the existing tests. The tests now accept both TLSv1.1 and
TLSv1.2.
||name||branch||commit||
|trunk| [trunk
branch|https://github.com/djatnieks/cassandra/tree/CASSANDRA-18540-trunk]|[trunk
commit|https://github.com/djatnieks/cassandra/commit/93feaa06d2e7fb974ff4e8094b2cdf0d3a9db2b6]|
|4.0|[4.0
branch|https://github.com/djatnieks/cassandra/tree/CASSANDRA-18540-4.0]|[4.0
commit|https://github.com/djatnieks/cassandra/commit/8bdfb503f6042c8af8560aac9bbca837eb9670a5]|
|4.1|[4.1
branch|https://github.com/djatnieks/cassandra/tree/CASSANDRA-18540-4.1]|[4.1
commit|https://github.com/apache/cassandra/commit/ebe4bcfc0feefa6727b7d5f0015fb6ff7215b179]|
This patch should work with jdk8 and jdk11 on 4.0, 4.1 and trunk.
And once CASSANDRA-18180 is complete it should also work with jdk17 - so I
think this ticket does not need to be blocked by CASSANDRA-18180. wdyt
[~e.dimitrova]?
> negotiatedProtocolMustBeAcceptedProtocolTest tests fail with "TLSv1.1 failed
> to negotiate" on JDK17
> ---
>
> Key: CASSANDRA-18540
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18540
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: dan jatnieks
>Assignee: dan jatnieks
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Note: This depends on having a fix for CASSANDRA-18180, otherwise most/all
> tests in {{NativeTransportEncryptionOptionsTest}} and
> {{InternodeEncryptionOptionsTest}} are failing due to that issue.
> Using the patch for CASSANDRA-18180, the
> {{negotiatedProtocolMustBeAcceptedProtocolTest}} test in both
> {{NativeTransportEncryptionOptionsTest}} and
> {{InternodeEncryptionOptionsTest}} fails with "TLSv1.1 failed to negotiate"
> on JDK17.
> From what I can see, the {{negotiatedProtocolMustBeAcceptedProtocolTest}} is
> failing because in JDK11 and JDK17 the "TLSv1.1" protocol is disabled.
> Since TLSv1.1 is disabled in JDK11 and 17, one possibility is to change the
> test to use TLSv1.2 instead of TLSv1.1. That should work directly with JDK11
> and 17, since TLSv1.2 is one of the defaults, and it won't be an issue for
> JDK8 as that will be dropped.
> Also, I think the point of the
> {{negotiatedProtocolMustBeAcceptedProtocolTest}} is to test that the
> {{accepted_protocols}} option is working correctly rather than the choice of
> _which_ protocol is used. Meaning, I don’t think the intent was to test
> TLSv1.1 specifically, rather that the mechanism of accepted protocols works
> and choosing TLSv1.1 was at the time convenient - but I could be wrong.
> It also seems to me like bit of a coincidence that these tests are currently
> working on JDK11, at least on CI. Indeed, running locally with JDK11, these
> fail for me:
> {noformat}
> $ pwd
> /Users/dan.jatnieks/apache/cassandra-4.0
> $ java -version
> openjdk version "11.0.11" 2021-04-20
> OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9 (build 11.0.11+9)
> OpenJDK 64-Bit Server VM AdoptOpenJDK-11.0.11+9 (build 11.0.11+9, mixed mode)
> $ ant test-jvm-dtest-some
> -Dtest.name=org.apache.cassandra.distributed.test.NativeTransportEncryptionOptionsTest
> -Duse.jdk11=true
> ...
> [junit-timeout] Testcase:
> negotiatedProtocolMustBeAcceptedProtocolTest(org.apache.cassandra.distributed.test.NativeTransportEncryptionOptionsTest):
>FAILED
> [junit-timeout] Should be possible to establish a TLSv1.1 connection
> expected: but was:
> [junit-timeout] junit.framework.AssertionFailedError: Should be possible to
> establish a TLSv1.1 connection expected: but
> was:
> [junit-timeout] at
> org.apache.cassandra.distributed.test.NativeTransportEncryptionOptionsTest.negotiatedProtocolMustBeAcceptedProtocolTest(NativeTransportEncryptionOptionsTest.java:160)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {noformat}
> I believe these work on CI because of CASSANDRA-16848 - in that ticket, after
> 2021-Apr JDK8 dropped TLSv1.1 which led to a fix in
> [cassandra-build|https://github.com/apache/cassandra-builds/commit/d1a3a0c59b3c5c17697d6a6656cd5d4f3a1cdbe9]
> docker code to make sure TLSv1.1 is accepted.
> I say
[jira] [Commented] (CASSANDRA-18582) BulkLoader withCipherSuites option is ignored
[
https://issues.apache.org/jira/browse/CASSANDRA-18582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17731377#comment-17731377
]
dan jatnieks commented on CASSANDRA-18582:
--
I've made the following patch:
||name||branch||commit||
|4.0|[4.0
branch|https://github.com/djatnieks/cassandra/tree/CASSANDRA-18582-4.0]|[4.0
commit|https://github.com/djatnieks/cassandra/commit/39caf4f91dc052bc2309ec8ced8c33bf10a8d37a]|
|4.1|[4.1
branch|https://github.com/djatnieks/cassandra/tree/CASSANDRA-18582-4.1]|[4.1
commit|https://github.com/apache/cassandra/commit/9b3caaf5a8a5f03d3c1438d0c998f51a1cc630bf]|
|trunk| [trunk
branch|https://github.com/djatnieks/cassandra/tree/CASSANDRA-18582-trunk]|[trunk
commit|https://github.com/djatnieks/cassandra/commit/3b27496aa57278b3d4854345d447cc625a784402]|
I haven't found a good way to add a test for this, at least partly because
{{RemoteEndpointAwareJdkSSLOptions}} from the driver doesn't expose the cipher
suites once they get set.
> BulkLoader withCipherSuites option is ignored
> -
>
> Key: CASSANDRA-18582
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18582
> Project: Cassandra
> Issue Type: Bug
>Reporter: dan jatnieks
>Assignee: dan jatnieks
>Priority: Normal
>
> The {{withCipherSuites}} option of {{BulkLoader}} is being ignored. It seems
> that since CASSANDRA-16362 the {{BulkLoader.buildSSLOptions}} method no
> longer applies the cipher suite options provided by
> {{clientEncryptionOptions}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18582) BulkLoader withCipherSuites option is ignored
dan jatnieks created CASSANDRA-18582:
Summary: BulkLoader withCipherSuites option is ignored
Key: CASSANDRA-18582
URL: https://issues.apache.org/jira/browse/CASSANDRA-18582
Project: Cassandra
Issue Type: Bug
Reporter: dan jatnieks
Assignee: dan jatnieks
The {{withCipherSuites}} option of {{BulkLoader}} is being ignored. It seems
that since CASSANDRA-16362 the {{BulkLoader.buildSSLOptions}} method no longer
applies the cipher suite options provided by {{clientEncryptionOptions}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18180) bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17731019#comment-17731019
]
dan jatnieks commented on CASSANDRA-18180:
--
bq. It seems there is a build failure for j8
That's expected as this patch works only for jdk11 and jdk17 and merging will
be blocked until jdk8 is dropped.
I added a link to CASSANDRA-18255 to indicate this.
> bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
> ---
>
> Key: CASSANDRA-18180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18180
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: dan jatnieks
>Priority: Normal
> Fix For: 5.x
>
> Attachments: cassandra-18180-directbufferref.diff
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> While working on CASSANDRA-17992 we hit:
> {code:java}
> java.lang.ClassCastException: class
> org.apache.cassandra.utils.memory.BufferPool$Chunk cannot be cast to class
> sun.nio.ch.DirectBuffer (org.apache.cassandra.utils.memory.BufferPool$Chunk
> is in unnamed module of loader 'app'; sun.nio.ch.DirectBuffer is in module
> java.base of loader 'bootstrap')\n\tat
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMEngine.overlapDetection(GaloisCounterMode.java:865)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMDecrypt.doFinal(GaloisCounterMode.java:1502)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode.engineDoFinal(GaloisCounterMode.java:447)\n\tat
>
> {code}
> -The issue is exposed with JDK 17, trunk; if interested, ping- [~e.dimitrova]
> -for current branch as there is no feature branch at the moment- we can
> build and start from trunk with JDK17 already. Circle CI can be run for JDK17
> too. For more information how to do that - .circleci/readme.md
> CC [~benedict]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18180) bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730700#comment-17730700
]
dan jatnieks edited comment on CASSANDRA-18180 at 6/8/23 8:15 PM:
--
I added a commit for {{MerkleTree}} - here is the latest branch and comparison:
[CASSANDRA-18180-trunk|https://github.com/djatnieks/cassandra/tree/CASSANDRA-18180-trunk]
[compare with
trunk|https://github.com/apache/cassandra/compare/trunk...djatnieks:cassandra:CASSANDRA-18180-trunk]
Btw, the reason I put {{DirectBufferRef}} in {{Ref.java}} is not to change the
package protected variable {{referent}}. And having {{DirectBufferRef}} makes
it simple to replace any use of {{Ref}} as a {{ByteBuffer}} attachment, such as
in {{MerkleTree}}.
was (Author: djatnieks):
I added a commit for {{MerkleTree}} - here is the latest branch and comparison:
[CASSANDRA-18180-trunk|https://github.com/djatnieks/cassandra/tree/CASSANDRA-18180-trunk]
[compare with
trunk|https://github.com/apache/cassandra/compare/trunk...djatnieks:cassandra:CASSANDRA-18180-trunk]
> bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
> ---
>
> Key: CASSANDRA-18180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18180
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: dan jatnieks
>Priority: Normal
> Fix For: 5.x
>
> Attachments: cassandra-18180-directbufferref.diff
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> While working on CASSANDRA-17992 we hit:
> {code:java}
> java.lang.ClassCastException: class
> org.apache.cassandra.utils.memory.BufferPool$Chunk cannot be cast to class
> sun.nio.ch.DirectBuffer (org.apache.cassandra.utils.memory.BufferPool$Chunk
> is in unnamed module of loader 'app'; sun.nio.ch.DirectBuffer is in module
> java.base of loader 'bootstrap')\n\tat
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMEngine.overlapDetection(GaloisCounterMode.java:865)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMDecrypt.doFinal(GaloisCounterMode.java:1502)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode.engineDoFinal(GaloisCounterMode.java:447)\n\tat
>
> {code}
> -The issue is exposed with JDK 17, trunk; if interested, ping- [~e.dimitrova]
> -for current branch as there is no feature branch at the moment- we can
> build and start from trunk with JDK17 already. Circle CI can be run for JDK17
> too. For more information how to do that - .circleci/readme.md
> CC [~benedict]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18180) bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730700#comment-17730700
]
dan jatnieks commented on CASSANDRA-18180:
--
I added a commit for {{MerkleTree}} - here is the latest branch and comparison:
[CASSANDRA-18180-trunk|https://github.com/djatnieks/cassandra/tree/CASSANDRA-18180-trunk]
[compare with
trunk|https://github.com/apache/cassandra/compare/trunk...djatnieks:cassandra:CASSANDRA-18180-trunk]
> bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
> ---
>
> Key: CASSANDRA-18180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18180
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: dan jatnieks
>Priority: Normal
> Fix For: 5.x
>
> Attachments: cassandra-18180-directbufferref.diff
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> While working on CASSANDRA-17992 we hit:
> {code:java}
> java.lang.ClassCastException: class
> org.apache.cassandra.utils.memory.BufferPool$Chunk cannot be cast to class
> sun.nio.ch.DirectBuffer (org.apache.cassandra.utils.memory.BufferPool$Chunk
> is in unnamed module of loader 'app'; sun.nio.ch.DirectBuffer is in module
> java.base of loader 'bootstrap')\n\tat
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMEngine.overlapDetection(GaloisCounterMode.java:865)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMDecrypt.doFinal(GaloisCounterMode.java:1502)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode.engineDoFinal(GaloisCounterMode.java:447)\n\tat
>
> {code}
> -The issue is exposed with JDK 17, trunk; if interested, ping- [~e.dimitrova]
> -for current branch as there is no feature branch at the moment- we can
> build and start from trunk with JDK17 already. Circle CI can be run for JDK17
> too. For more information how to do that - .circleci/readme.md
> CC [~benedict]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18540) negotiatedProtocolMustBeAcceptedProtocolTest tests fail with "TLSv1.1 failed to negotiate" on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730687#comment-17730687
]
dan jatnieks commented on CASSANDRA-18540:
--
Okay, I have made new patches that adds a comment to the test methods:
{noformat}
/**
* Tests that the negotiated protocol is the highest common protocol
between the client and server.
*
* Note: This test uses TLSV1.1, which is disabled by default in JDK 8 and
higher. If the test fails with
* FAILED_TO_NEGOTIATE, it may be necessary to check the java.security file
in your JDK installation and remove
* TLSv1.1 from the jdk.tls.disabledAlgorithms.
* @see https://issues.apache.org/jira/browse/CASSANDRA-18540;>CASSANDRA-18540
* @see https://senthilnayagan.medium.com/tlsv1-and-tlsv1-1-protocols-disabled-by-default-in-javas-latest-patch-released-on-april-20-2021-52c309f6b16d;>
* TLSv1 and TLSv1.1 Protocols are Disabled in Java!
*/
{noformat}
||name||branch||commit||
|trunk| [trunk
branch|https://github.com/djatnieks/cassandra/tree/CASSANDRA-18540-trunk]|[trunk
commit|https://github.com/djatnieks/cassandra/commit/441346d4be05da5819c19c2086c522b5a1b2fe30]|
|4.0|[4.0
branch|https://github.com/djatnieks/cassandra/tree/CASSANDRA-18540-4.0]|[4.0
commit|https://github.com/djatnieks/cassandra/commit/65406161ae56c570fd47a42e61921b841ab6f62d]|
|4.1|[4.1
branch|https://github.com/djatnieks/cassandra/tree/CASSANDRA-18540-4.1]|[4.1
commit|https://github.com/apache/cassandra/commit/937683e5d5bf374131e06fad8447d70fd99cec7f]|
> negotiatedProtocolMustBeAcceptedProtocolTest tests fail with "TLSv1.1 failed
> to negotiate" on JDK17
> ---
>
> Key: CASSANDRA-18540
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18540
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: dan jatnieks
>Assignee: dan jatnieks
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Note: This depends on having a fix for CASSANDRA-18180, otherwise most/all
> tests in {{NativeTransportEncryptionOptionsTest}} and
> {{InternodeEncryptionOptionsTest}} are failing due to that issue.
> Using the patch for CASSANDRA-18180, the
> {{negotiatedProtocolMustBeAcceptedProtocolTest}} test in both
> {{NativeTransportEncryptionOptionsTest}} and
> {{InternodeEncryptionOptionsTest}} fails with "TLSv1.1 failed to negotiate"
> on JDK17.
> From what I can see, the {{negotiatedProtocolMustBeAcceptedProtocolTest}} is
> failing because in JDK11 and JDK17 the "TLSv1.1" protocol is disabled.
> Since TLSv1.1 is disabled in JDK11 and 17, one possibility is to change the
> test to use TLSv1.2 instead of TLSv1.1. That should work directly with JDK11
> and 17, since TLSv1.2 is one of the defaults, and it won't be an issue for
> JDK8 as that will be dropped.
> Also, I think the point of the
> {{negotiatedProtocolMustBeAcceptedProtocolTest}} is to test that the
> {{accepted_protocols}} option is working correctly rather than the choice of
> _which_ protocol is used. Meaning, I don’t think the intent was to test
> TLSv1.1 specifically, rather that the mechanism of accepted protocols works
> and choosing TLSv1.1 was at the time convenient - but I could be wrong.
> It also seems to me like bit of a coincidence that these tests are currently
> working on JDK11, at least on CI. Indeed, running locally with JDK11, these
> fail for me:
> {noformat}
> $ pwd
> /Users/dan.jatnieks/apache/cassandra-4.0
> $ java -version
> openjdk version "11.0.11" 2021-04-20
> OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9 (build 11.0.11+9)
> OpenJDK 64-Bit Server VM AdoptOpenJDK-11.0.11+9 (build 11.0.11+9, mixed mode)
> $ ant test-jvm-dtest-some
> -Dtest.name=org.apache.cassandra.distributed.test.NativeTransportEncryptionOptionsTest
> -Duse.jdk11=true
> ...
> [junit-timeout] Testcase:
> negotiatedProtocolMustBeAcceptedProtocolTest(org.apache.cassandra.distributed.test.NativeTransportEncryptionOptionsTest):
>FAILED
> [junit-timeout] Should be possible to establish a TLSv1.1 connection
> expected: but was:
> [junit-timeout] junit.framework.AssertionFailedError: Should be possible to
> establish a TLSv1.1 connection expected: but
> was:
> [junit-timeout] at
> org.apache.cassandra.distributed.test.NativeTransportEncryptionOptionsTest.negotiatedProtocolMustBeAcceptedProtocolTest(NativeTransportEncryptionOptionsTest.java:160)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
>
[jira] [Comment Edited] (CASSANDRA-18180) bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730281#comment-17730281
]
dan jatnieks edited comment on CASSANDRA-18180 at 6/7/23 9:20 PM:
--
I think that's a good suggestion. So the idea I have for that is to add a class
like:
{noformat}
DirectBufferRef extends Ref implements DirectBuffer
{noformat}
This can use used in {{BufferPool.setAttachment}} , like so:
{noformat}
MemoryUtil.setAttachment(buffer, new DirectBufferRef<>(this, null));
{noformat}
Here's a patch with that approach; let me know if it matches the suggestion or
I could try something else: [^cassandra-18180-directbufferref.diff]
I also noticed that {{MerkleTree}} has some similar code that sets a {{Ref}} as
a {{ByteBuffer}} attachment, and in fact, I think any caller of
{{MemoryUtil.setAttachment}} probably needs to ensure that the attachment
object implements {{DirectBuffer}} in case it gets used with encryption enabled.
The {{MerkleTree}} case is a bit easier than {{BufferPool}} because it only
uses a {{null}} referent, so it's enough just to use something like the
{{DirectBufferRef}} mentioned above (no need to implement {{DirectBuffer}} in
{{MerkleTree}} afaict). [This is not included yet in the patch]
I thought that perhaps it would be a good idea to even enforce it in
{{{}MemoryUtil{}}}, something like:
{noformat}
public static void setAttachment(ByteBuffer
instance, T next)
{noformat}
However, then I found that {{SyncUtil.force}} uses {{MemoryUtil.getAttachment}}
and expects that the attachment may be {{{}Runnable{}}}. I'm not sure yet how
that gets set though - I didn't any other {{setAttachment}} calls with
{{Runnable}} except in a unit test. Right now I'm not sure what to do about
this one - and I'm concerned that when encryption is being used, this could
again lead to a {{{}ClassCastException{}}}.
was (Author: djatnieks):
I think that's a good suggestion. So the idea I have for that is to add a class
like:
{noformat}
DirectBufferRef extends Ref implements DirectBuffer
{noformat}
This can use used in {{BufferPool.setAttachment}} , like so:
{noformat}
MemoryUtil.setAttachment(buffer, new DirectBufferRef<>(this, null));
{noformat}
Here's a patch with that approach; let me know if it matches the suggestion or
I could try something else: [^cassandra-18180-directbufferref.diff]
I also noticed that {{MerkleTree}} has some similar code that sets a {{Ref}} as
a {{ByteBuffer}} attachment, and in fact, I think any caller of
{{MemoryUtil.setAttachment}} probably needs to ensure that the attachment
object implements {{DirectBuffer}} in case it gets used with encryption enabled.
The {{MerkleTree}} case is a bit easier than {{BufferPool}} because it only
uses a {{null}} referent, so it's enough just to use something like the
{{DirectBufferRef}} mentioned above (no need to implement {{DirectBuffer}} in
{{MerkleTree}} afaict).
I thought that perhaps it would be a good idea to even enforce it in
{{{}MemoryUtil{}}}, something like:
{noformat}
public static void setAttachment(ByteBuffer
instance, T next)
{noformat}
However, then I found that {{SyncUtil.force}} uses {{MemoryUtil.getAttachment}}
and expects that the attachment may be {{{}Runnable{}}}. I'm not sure yet how
that gets set though - I didn't any other {{setAttachment}} calls with
{{Runnable}} except in a unit test. Right now I'm not sure what to do about
this one - and I'm concerned that when encryption is being used, this could
again lead to a {{{}ClassCastException{}}}.
> bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
> ---
>
> Key: CASSANDRA-18180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18180
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: dan jatnieks
>Priority: Normal
> Fix For: 5.x
>
> Attachments: cassandra-18180-directbufferref.diff
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> While working on CASSANDRA-17992 we hit:
> {code:java}
> java.lang.ClassCastException: class
> org.apache.cassandra.utils.memory.BufferPool$Chunk cannot be cast to class
> sun.nio.ch.DirectBuffer (org.apache.cassandra.utils.memory.BufferPool$Chunk
> is in unnamed module of loader 'app'; sun.nio.ch.DirectBuffer is in module
> java.base of loader 'bootstrap')\n\tat
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMEngine.overlapDetection(GaloisCounterMode.java:865)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMDecrypt.doFinal(GaloisCounterMode.java:1502)\n\tat
>
>
[jira] [Commented] (CASSANDRA-18180) bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730281#comment-17730281
]
dan jatnieks commented on CASSANDRA-18180:
--
I think that's a good suggestion. So the idea I have for that is to add a class
like:
{noformat}
DirectBufferRef extends Ref implements DirectBuffer
{noformat}
This can use used in {{BufferPool.setAttachment}} , like so:
{noformat}
MemoryUtil.setAttachment(buffer, new DirectBufferRef<>(this, null));
{noformat}
Here's a patch with that approach; let me know if it matches the suggestion or
I could try something else: [^cassandra-18180-directbufferref.diff]
I also noticed that {{MerkleTree}} has some similar code that sets a {{Ref}} as
a {{ByteBuffer}} attachment, and in fact, I think any caller of
{{MemoryUtil.setAttachment}} probably needs to ensure that the attachment
object implements {{DirectBuffer}} in case it gets used with encryption enabled.
The {{MerkleTree}} case is a bit easier than {{BufferPool}} because it only
uses a {{null}} referent, so it's enough just to use something like the
{{DirectBufferRef}} mentioned above (no need to implement {{DirectBuffer}} in
{{MerkleTree}} afaict).
I thought that perhaps it would be a good idea to even enforce it in
{{{}MemoryUtil{}}}, something like:
{noformat}
public static void setAttachment(ByteBuffer
instance, T next)
{noformat}
However, then I found that {{SyncUtil.force}} uses {{MemoryUtil.getAttachment}}
and expects that the attachment may be {{{}Runnable{}}}. I'm not sure yet how
that gets set though - I didn't any other {{setAttachment}} calls with
{{Runnable}} except in a unit test. Right now I'm not sure what to do about
this one - and I'm concerned that when encryption is being used, this could
again lead to a {{{}ClassCastException{}}}.
> bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
> ---
>
> Key: CASSANDRA-18180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18180
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: dan jatnieks
>Priority: Normal
> Fix For: 5.x
>
> Attachments: cassandra-18180-directbufferref.diff
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> While working on CASSANDRA-17992 we hit:
> {code:java}
> java.lang.ClassCastException: class
> org.apache.cassandra.utils.memory.BufferPool$Chunk cannot be cast to class
> sun.nio.ch.DirectBuffer (org.apache.cassandra.utils.memory.BufferPool$Chunk
> is in unnamed module of loader 'app'; sun.nio.ch.DirectBuffer is in module
> java.base of loader 'bootstrap')\n\tat
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMEngine.overlapDetection(GaloisCounterMode.java:865)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMDecrypt.doFinal(GaloisCounterMode.java:1502)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode.engineDoFinal(GaloisCounterMode.java:447)\n\tat
>
> {code}
> -The issue is exposed with JDK 17, trunk; if interested, ping- [~e.dimitrova]
> -for current branch as there is no feature branch at the moment- we can
> build and start from trunk with JDK17 already. Circle CI can be run for JDK17
> too. For more information how to do that - .circleci/readme.md
> CC [~benedict]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18180) bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks updated CASSANDRA-18180:
-
Attachment: cassandra-18180-directbufferref.diff
> bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
> ---
>
> Key: CASSANDRA-18180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18180
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: dan jatnieks
>Priority: Normal
> Fix For: 5.x
>
> Attachments: cassandra-18180-directbufferref.diff
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> While working on CASSANDRA-17992 we hit:
> {code:java}
> java.lang.ClassCastException: class
> org.apache.cassandra.utils.memory.BufferPool$Chunk cannot be cast to class
> sun.nio.ch.DirectBuffer (org.apache.cassandra.utils.memory.BufferPool$Chunk
> is in unnamed module of loader 'app'; sun.nio.ch.DirectBuffer is in module
> java.base of loader 'bootstrap')\n\tat
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMEngine.overlapDetection(GaloisCounterMode.java:865)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMDecrypt.doFinal(GaloisCounterMode.java:1502)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode.engineDoFinal(GaloisCounterMode.java:447)\n\tat
>
> {code}
> -The issue is exposed with JDK 17, trunk; if interested, ping- [~e.dimitrova]
> -for current branch as there is no feature branch at the moment- we can
> build and start from trunk with JDK17 already. Circle CI can be run for JDK17
> too. For more information how to do that - .circleci/readme.md
> CC [~benedict]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17262) sstableloader ignores the native port option
[
https://issues.apache.org/jira/browse/CASSANDRA-17262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17725586#comment-17725586
]
dan jatnieks commented on CASSANDRA-17262:
--
Seems like a duplicate of CASSANDRA-17210?
> sstableloader ignores the native port option
>
>
> Key: CASSANDRA-17262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17262
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Tools
>Reporter: Oleg Zhovtanyuk
>Priority: Normal
>
> sstableloader silently ignores the native port option and connects to default
> port.
> E.g.
> {code:java}
> sstableloader -v -d 192.168.1.24 -p 32181 -sp 32182 -u USER -pw PASSWORD
> backups/test-workspace/test-table-9150b730742911ec9d011fb0ef4f5206 {code}
> fails with
> {code:java}
> All host(s) tried for query failed (tried: /192.168.1.24:9042
> (com.datastax.driver.core.exceptions.TransportException: [/192.168.1.24:9042]
> Cannot connect))
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: /192.168.1.24:9042
> (com.datastax.driver.core.exceptions.TransportException: [/192.168.1.24:9042]
> Cannot connect))
> at
> com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:270)
> at
> com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:109)
> at
> com.datastax.driver.core.Cluster$Manager.negotiateProtocolVersionAndConnect(Cluster.java:1813)
> at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1726)
> at com.datastax.driver.core.Cluster.init(Cluster.java:214)
> at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:387)
> at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:366)
> at com.datastax.driver.core.Cluster.connect(Cluster.java:311)
> at
> org.apache.cassandra.utils.NativeSSTableLoaderClient.init(NativeSSTableLoaderClient.java:75)
> at
> org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:183)
> at org.apache.cassandra.tools.BulkLoader.load(BulkLoader.java:83)
> at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:51)
> Exception in thread "main" org.apache.cassandra.tools.BulkLoadException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: /192.168.1.24:9042
> (com.datastax.driver.core.exceptions.TransportException: [/192.168.1.24:9042]
> Cannot connect))
> at org.apache.cassandra.tools.BulkLoader.load(BulkLoader.java:96)
> at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:51)
> Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All
> host(s) tried for query failed (tried: /192.168.1.24:9042
> (com.datastax.driver.core.exceptions.TransportException: [/192.168.1.24:9042]
> Cannot connect))
> at
> com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:270)
> at
> com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:109)
> at
> com.datastax.driver.core.Cluster$Manager.negotiateProtocolVersionAndConnect(Cluster.java:1813)
> at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1726)
> at com.datastax.driver.core.Cluster.init(Cluster.java:214)
> at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:387)
> at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:366)
> at com.datastax.driver.core.Cluster.connect(Cluster.java:311)
> at
> org.apache.cassandra.utils.NativeSSTableLoaderClient.init(NativeSSTableLoaderClient.java:75)
> at
> org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:183)
> at org.apache.cassandra.tools.BulkLoader.load(BulkLoader.java:83)
> ... 1 more {code}
> The solution is to pass the endpoint as '-d HOST:PORT', but it requires
> digging in the source code and it not documented.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18540) negotiatedProtocolMustBeAcceptedProtocolTest tests fail with "TLSv1.1 failed to negotiate" on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks updated CASSANDRA-18540:
-
Test and Documentation Plan:
Patch: [https://github.com/djatnieks/cassandra/tree/CASSANDRA-18540-trunk]
PR: https://github.com/apache/cassandra/pull/2359
CI: TBD because has dependency on CASSANDRA-18180
Status: Patch Available (was: In Progress)
> negotiatedProtocolMustBeAcceptedProtocolTest tests fail with "TLSv1.1 failed
> to negotiate" on JDK17
> ---
>
> Key: CASSANDRA-18540
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18540
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: dan jatnieks
>Assignee: dan jatnieks
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Note: This depends on having a fix for CASSANDRA-18180, otherwise most/all
> tests in {{NativeTransportEncryptionOptionsTest}} and
> {{InternodeEncryptionOptionsTest}} are failing due to that issue.
> Using the patch for CASSANDRA-18180, the
> {{negotiatedProtocolMustBeAcceptedProtocolTest}} test in both
> {{NativeTransportEncryptionOptionsTest}} and
> {{InternodeEncryptionOptionsTest}} fails with "TLSv1.1 failed to negotiate"
> on JDK17.
> From what I can see, the {{negotiatedProtocolMustBeAcceptedProtocolTest}} is
> failing because in JDK11 and JDK17 the "TLSv1.1" protocol is disabled.
> Since TLSv1.1 is disabled in JDK11 and 17, one possibility is to change the
> test to use TLSv1.2 instead of TLSv1.1. That should work directly with JDK11
> and 17, since TLSv1.2 is one of the defaults, and it won't be an issue for
> JDK8 as that will be dropped.
> Also, I think the point of the
> {{negotiatedProtocolMustBeAcceptedProtocolTest}} is to test that the
> {{accepted_protocols}} option is working correctly rather than the choice of
> _which_ protocol is used. Meaning, I don’t think the intent was to test
> TLSv1.1 specifically, rather that the mechanism of accepted protocols works
> and choosing TLSv1.1 was at the time convenient - but I could be wrong.
> It also seems to me like bit of a coincidence that these tests are currently
> working on JDK11, at least on CI. Indeed, running locally with JDK11, these
> fail for me:
> {noformat}
> $ pwd
> /Users/dan.jatnieks/apache/cassandra-4.0
> $ java -version
> openjdk version "11.0.11" 2021-04-20
> OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9 (build 11.0.11+9)
> OpenJDK 64-Bit Server VM AdoptOpenJDK-11.0.11+9 (build 11.0.11+9, mixed mode)
> $ ant test-jvm-dtest-some
> -Dtest.name=org.apache.cassandra.distributed.test.NativeTransportEncryptionOptionsTest
> -Duse.jdk11=true
> ...
> [junit-timeout] Testcase:
> negotiatedProtocolMustBeAcceptedProtocolTest(org.apache.cassandra.distributed.test.NativeTransportEncryptionOptionsTest):
>FAILED
> [junit-timeout] Should be possible to establish a TLSv1.1 connection
> expected: but was:
> [junit-timeout] junit.framework.AssertionFailedError: Should be possible to
> establish a TLSv1.1 connection expected: but
> was:
> [junit-timeout] at
> org.apache.cassandra.distributed.test.NativeTransportEncryptionOptionsTest.negotiatedProtocolMustBeAcceptedProtocolTest(NativeTransportEncryptionOptionsTest.java:160)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {noformat}
> I believe these work on CI because of CASSANDRA-16848 - in that ticket, after
> 2021-Apr JDK8 dropped TLSv1.1 which led to a fix in
> [cassandra-build|https://github.com/apache/cassandra-builds/commit/d1a3a0c59b3c5c17697d6a6656cd5d4f3a1cdbe9]
> docker code to make sure TLSv1.1 is accepted.
> I say coincidence because this change also makes it work for JDK11 and JDK17,
> and I've been able to verify that making a change locally to the JDK
> {{java.security}} file. I’m not sure that at the time of CASSANDRA-16848 it
> was intended for any JDK versions.
> The point of mentioning this is that if
> {{negotiatedProtocolMustBeAcceptedProtocolTest}} is changed to use TLSv1.2,
> and support for JDK8 is dropped, then the changes made in CASSANDRA-16848
> could also be reverted.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18180) bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks updated CASSANDRA-18180:
-
Test and Documentation Plan:
Patch:
[https://github.com/djatnieks/cassandra/tree/CASSANDRA-18180]
PR:
[https://github.com/apache/cassandra/pull/2358]
Tested locally with jdk11 and jdk17:
{code:java}
ant test-jvm-dtest-some
-Dtest.name=org.apache.cassandra.distributed.test.SSTableLoaderEncryptionOptionsTest{code}
was:
Patch:
[https://github.com/djatnieks/cassandra/tree/CASSANDRA-18180]
Tested locally with jdk11 and jdk17:
{code:java}
ant test-jvm-dtest-some
-Dtest.name=org.apache.cassandra.distributed.test.SSTableLoaderEncryptionOptionsTest{code}
> bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
> ---
>
> Key: CASSANDRA-18180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18180
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: dan jatnieks
>Priority: Normal
> Time Spent: 10m
> Remaining Estimate: 0h
>
> While working on CASSANDRA-17992 we hit:
> {code:java}
> java.lang.ClassCastException: class
> org.apache.cassandra.utils.memory.BufferPool$Chunk cannot be cast to class
> sun.nio.ch.DirectBuffer (org.apache.cassandra.utils.memory.BufferPool$Chunk
> is in unnamed module of loader 'app'; sun.nio.ch.DirectBuffer is in module
> java.base of loader 'bootstrap')\n\tat
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMEngine.overlapDetection(GaloisCounterMode.java:865)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMDecrypt.doFinal(GaloisCounterMode.java:1502)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode.engineDoFinal(GaloisCounterMode.java:447)\n\tat
>
> {code}
> -The issue is exposed with JDK 17, trunk; if interested, ping- [~e.dimitrova]
> -for current branch as there is no feature branch at the moment- we can
> build and start from trunk with JDK17 already. Circle CI can be run for JDK17
> too. For more information how to do that - .circleci/readme.md
> CC [~benedict]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18180) bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17725576#comment-17725576
]
dan jatnieks commented on CASSANDRA-18180:
--
{quote}I experimented with using a map for this, but ended up learning that
{{{}ByteBuffer{}}}'s are not suited to being used as map keys
{quote}
Just a note that it was pointed out to me that {{IdentityHashMap}} could work
for this, however it would need to be made thread safe and performance would
need to be verified.
> bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
> ---
>
> Key: CASSANDRA-18180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18180
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: dan jatnieks
>Priority: Normal
>
> While working on CASSANDRA-17992 we hit:
> {code:java}
> java.lang.ClassCastException: class
> org.apache.cassandra.utils.memory.BufferPool$Chunk cannot be cast to class
> sun.nio.ch.DirectBuffer (org.apache.cassandra.utils.memory.BufferPool$Chunk
> is in unnamed module of loader 'app'; sun.nio.ch.DirectBuffer is in module
> java.base of loader 'bootstrap')\n\tat
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMEngine.overlapDetection(GaloisCounterMode.java:865)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMDecrypt.doFinal(GaloisCounterMode.java:1502)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode.engineDoFinal(GaloisCounterMode.java:447)\n\tat
>
> {code}
> -The issue is exposed with JDK 17, trunk; if interested, ping- [~e.dimitrova]
> -for current branch as there is no feature branch at the moment- we can
> build and start from trunk with JDK17 already. Circle CI can be run for JDK17
> too. For more information how to do that - .circleci/readme.md
> CC [~benedict]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18180) bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723776#comment-17723776
]
dan jatnieks edited comment on CASSANDRA-18180 at 5/23/23 9:07 PM:
---
Recap: The {{attachment}} field of {{ByteBuffer}} is being used to store the
related {{BufferPool}} {{Chunk}} object that is associated with the buffer.
Since -JDK11- JDK16 some crypto overlap detection code in {{GaloisCounterMode}}
expects that any object attached to a {{ByteBuffer}} implements
{{{}DirectBuffer{}}}, and if it does not, will cause a
{{{}ClassCastException{}}}.
Since we see this {{ClassCastException}} in tests using encryption, it seems
it's triggered by a supported TLS setting rather than some unintended default.
The patch I provided uses the suggestion made by [~benedict] in this comment in
CASSANDRA-17992, which is to have {{Chunk}} (and also {{{}Ref{}}}) implement
{{{}DirectBuffer{}}}.
The main downside to this is that two additional {{--add-exports}} are required
to be able to access JDK internal class {{{}DirectBuffer{}}}.
Access to this internal class also has a secondary impact on
{{TestNameCheckTask}} as it uses reflection and tries to load all related
classes of tests being checked. This led to replacing the
{{checktestnameshelper}} ant task in {{build.xml}} with a java target so that
it is possible to pass the needed jvm args to {{{}TestNameCheckTask{}}}.
An alternative approach that avoids accessing jdk internals would, I think,
still need to associate {{ByteBuffer}} objects to {{Chunk}} objects. I
experimented with using a map for this, but ended up learning that
{{{}ByteBuffer{}}}'s are not suited to being used as map keys, as the docs
state:
{noformat}
because buffer hash codes are content-dependent, it is inadvisable to use
buffers as keys in hash maps or similar data structures unless it is known that
their contents will not change.{noformat}
So, you can put a {{ByteBuffer}} into a map, but if the contents change you
won't be able to retrieve/find it again.
I'm not sure what other workaround solution to propose that doesn't negatively
affect complexity and/or performance, but am happy to look into other ideas.
-There is another aspect of this that I don't fully understand that's somewhat
bothersome - this test only fails with JDK17 even though the same
{{GaloisCounterMode}} overlap detection code is present in JDK11; for some
reason the same code path is not executed and the tests pass with JDK11. I
wonder why that is?-
-I did check the enabled cipher suites for internode messaging and native
transport that I found looking at test run output - and those are the same with
11 and 17. So I guess it's a result of some change in the JDK? If I could be
due to something else, I don't know what that might be.-
My mistake - {{GaloisCounterMode}} overlap detection was only added in JDK16
with [JDK-8253821|https://bugs.openjdk.org/browse/JDK-8253821], so it does make
sense that this issue happens with JDK17 and not with JDK11.
was (Author: djatnieks):
Recap: The {{attachment}} field of {{ByteBuffer}} is being used to store the
related {{BufferPool}} {{Chunk}} object that is associated with the buffer.
Since JDK11 some crypto overlap detection code in {{GaloisCounterMode}} expects
that any object attached to a {{ByteBuffer}} implements {{{}DirectBuffer{}}},
and if it does not, will cause a {{{}ClassCastException{}}}.
Since we see this {{ClassCastException}} in tests using encryption, it seems
it's triggered by a supported TLS setting rather than some unintended default.
The patch I provided uses the suggestion made by [~benedict] in this comment in
CASSANDRA-17992, which is to have {{Chunk}} (and also {{{}Ref{}}}) implement
{{{}DirectBuffer{}}}.
The main downside to this is that two additional {{--add-exports}} are required
to be able to access JDK internal class {{{}DirectBuffer{}}}.
Access to this internal class also has a secondary impact on
{{TestNameCheckTask}} as it uses reflection and tries to load all related
classes of tests being checked. This led to replacing the
{{checktestnameshelper}} ant task in {{build.xml}} with a java target so that
it is possible to pass the needed jvm args to {{{}TestNameCheckTask{}}}.
An alternative approach that avoids accessing jdk internals would, I think,
still need to associate {{ByteBuffer}} objects to {{Chunk}} objects. I
experimented with using a map for this, but ended up learning that
{{{}ByteBuffer{}}}'s are not suited to being used as map keys, as the docs
state:
{noformat}
because buffer hash codes are content-dependent, it is inadvisable to use
buffers as keys in hash maps or similar data structures unless it is known that
their contents will not change.{noformat}
So, you can put a {{ByteBuffer}} into a map, but if the contents change you
won't be able to retrieve/find it
[jira] [Updated] (CASSANDRA-18540) negotiatedProtocolMustBeAcceptedProtocolTest tests fail with "TLSv1.1 failed to negotiate" on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks updated CASSANDRA-18540:
-
Description:
Note: This depends on having a fix for CASSANDRA-18180, otherwise most/all
tests in {{NativeTransportEncryptionOptionsTest}} and
{{InternodeEncryptionOptionsTest}} are failing due to that issue.
Using the patch for CASSANDRA-18180, the
{{negotiatedProtocolMustBeAcceptedProtocolTest}} test in both
{{NativeTransportEncryptionOptionsTest}} and {{InternodeEncryptionOptionsTest}}
fails with "TLSv1.1 failed to negotiate" on JDK17.
>From what I can see, the {{negotiatedProtocolMustBeAcceptedProtocolTest}} is
>failing because in JDK11 and JDK17 the "TLSv1.1" protocol is disabled.
Since TLSv1.1 is disabled in JDK11 and 17, one possibility is to change the
test to use TLSv1.2 instead of TLSv1.1. That should work directly with JDK11
and 17, since TLSv1.2 is one of the defaults, and it won't be an issue for JDK8
as that will be dropped.
Also, I think the point of the {{negotiatedProtocolMustBeAcceptedProtocolTest}}
is to test that the {{accepted_protocols}} option is working correctly rather
than the choice of _which_ protocol is used. Meaning, I don’t think the intent
was to test TLSv1.1 specifically, rather that the mechanism of accepted
protocols works and choosing TLSv1.1 was at the time convenient - but I could
be wrong.
It also seems to me like bit of a coincidence that these tests are currently
working on JDK11, at least on CI. Indeed, running locally with JDK11, these
fail for me:
{noformat}
$ pwd
/Users/dan.jatnieks/apache/cassandra-4.0
$ java -version
openjdk version "11.0.11" 2021-04-20
OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9 (build 11.0.11+9)
OpenJDK 64-Bit Server VM AdoptOpenJDK-11.0.11+9 (build 11.0.11+9, mixed mode)
$ ant test-jvm-dtest-some
-Dtest.name=org.apache.cassandra.distributed.test.NativeTransportEncryptionOptionsTest
-Duse.jdk11=true
...
[junit-timeout] Testcase:
negotiatedProtocolMustBeAcceptedProtocolTest(org.apache.cassandra.distributed.test.NativeTransportEncryptionOptionsTest):
FAILED
[junit-timeout] Should be possible to establish a TLSv1.1 connection
expected: but was:
[junit-timeout] junit.framework.AssertionFailedError: Should be possible to
establish a TLSv1.1 connection expected: but
was:
[junit-timeout] at
org.apache.cassandra.distributed.test.NativeTransportEncryptionOptionsTest.negotiatedProtocolMustBeAcceptedProtocolTest(NativeTransportEncryptionOptionsTest.java:160)
[junit-timeout] at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit-timeout] at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit-timeout] at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
{noformat}
I believe these work on CI because of CASSANDRA-16848 - in that ticket, after
2021-Apr JDK8 dropped TLSv1.1 which led to a fix in
[cassandra-build|https://github.com/apache/cassandra-builds/commit/d1a3a0c59b3c5c17697d6a6656cd5d4f3a1cdbe9]
docker code to make sure TLSv1.1 is accepted.
I say coincidence because this change also makes it work for JDK11 and JDK17,
and I've been able to verify that making a change locally to the JDK
{{java.security}} file. I’m not sure that at the time of CASSANDRA-16848 it was
intended for any JDK versions.
The point of mentioning this is that if
{{negotiatedProtocolMustBeAcceptedProtocolTest}} is changed to use TLSv1.2, and
support for JDK8 is dropped, then the changes made in CASSANDRA-16848 could
also be reverted.
was:
Note: This depends on having a fix for CASSANDRA-18180, otherwise most/all
tests in {{NativeTransportEncryptionOptionsTest}} and
{{InternodeEncryptionOptionsTest}} are failing due to that issue.
Using the patch for CASSANDRA-18180, the
{{negotiatedProtocolMustBeAcceptedProtocolTest}} test in both
{{NativeTransportEncryptionOptionsTest}} and {{InternodeEncryptionOptionsTest}}
fails with "TLSv1.1 failed to negotiate" on JDK17.
>From what I can see, the {{negotiatedProtocolMustBeAcceptedProtocolTest}} is
>failing because in JDK11 and JDK17 the "TLSv1.1" protocol is disabled.
Since TLSv1.1 is disabled in JDK11 and 17, one possibility is to change the
test to use TLSv1.2 instead of TLSv1.1. That should work directly with JDK11
and 17, since TLSv1.2 is one of the defaults, and it won't be an issue for JDK8
as that will be dropped.
Also, I think the point of the {{negotiatedProtocolMustBeAcceptedProtocolTest}}
is to test that the {{accepted_protocols}} option is working correctly rather
than the choice of _which_ protocol is used. Meaning, I don’t think the intent
was to test TLSv1.1 specifically, rather that the mechanism of accepted
protocols works and choosing TLSv1.1 was at the time
[jira] [Assigned] (CASSANDRA-18540) negotiatedProtocolMustBeAcceptedProtocolTest tests fail with "TLSv1.1 failed to negotiate" on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks reassigned CASSANDRA-18540:
Assignee: dan jatnieks
> negotiatedProtocolMustBeAcceptedProtocolTest tests fail with "TLSv1.1 failed
> to negotiate" on JDK17
> ---
>
> Key: CASSANDRA-18540
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18540
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: dan jatnieks
>Assignee: dan jatnieks
>Priority: Normal
> Fix For: 5.x
>
>
> Note: This depends on having a fix for CASSANDRA-18180, otherwise most/all
> tests in {{NativeTransportEncryptionOptionsTest}} and
> {{InternodeEncryptionOptionsTest}} are failing due to that issue.
> Using the patch for CASSANDRA-18180, the
> {{negotiatedProtocolMustBeAcceptedProtocolTest}} test in both
> {{NativeTransportEncryptionOptionsTest}} and
> {{InternodeEncryptionOptionsTest}} fails with "TLSv1.1 failed to negotiate"
> on JDK17.
> From what I can see, the {{negotiatedProtocolMustBeAcceptedProtocolTest}} is
> failing because in JDK11 and JDK17 the "TLSv1.1" protocol is disabled.
> Since TLSv1.1 is disabled in JDK11 and 17, one possibility is to change the
> test to use TLSv1.2 instead of TLSv1.1. That should work directly with JDK11
> and 17, since TLSv1.2 is one of the defaults, and it won't be an issue for
> JDK8 as that will be dropped.
> Also, I think the point of the
> {{negotiatedProtocolMustBeAcceptedProtocolTest}} is to test that the
> {{accepted_protocols}} option is working correctly rather than the choice of
> _which_ protocol is used. Meaning, I don’t think the intent was to test
> TLSv1.1 specifically, rather that the mechanism of accepted protocols works
> and choosing TLSv1.1 was at the time convenient - but I could be wrong.
> It also seems to me like bit of a coincidence that these tests are currently
> working on JDK11, at least on CI. Indeed, running locally with JDK11, these
> fail for me:
> {noformat}
> $ pwd
> /Users/dan.jatnieks/apache/cassandra-4.0
> $ java -version
> openjdk version "11.0.11" 2021-04-20
> OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9 (build 11.0.11+9)
> OpenJDK 64-Bit Server VM AdoptOpenJDK-11.0.11+9 (build 11.0.11+9, mixed mode)
> $ ant test-jvm-dtest-some
> -Dtest.name=org.apache.cassandra.distributed.test.NativeTransportEncryptionOptionsTest
> -Duse.jdk11=true
> ...
> [junit-timeout] Testcase:
> negotiatedProtocolMustBeAcceptedProtocolTest(org.apache.cassandra.distributed.test.NativeTransportEncryptionOptionsTest):
>FAILED
> [junit-timeout] Should be possible to establish a TLSv1.1 connection
> expected: but was:
> [junit-timeout] junit.framework.AssertionFailedError: Should be possible to
> establish a TLSv1.1 connection expected: but
> was:
> [junit-timeout] at
> org.apache.cassandra.distributed.test.NativeTransportEncryptionOptionsTest.negotiatedProtocolMustBeAcceptedProtocolTest(NativeTransportEncryptionOptionsTest.java:160)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {noformat}
> I believe these work on CI because of CASSANDRA-16848 - in that ticket, a
> specific JDK8 build (accidentally?) dropped TLSv1.1 (later added again) which
> led to adding some docker code to make sure TLSv1.1 is accepted.
> I say coincidence because this change also makes it work for JDK11 and JDK17,
> and I've been able to verify that making a change locally to the JDK
> {{java.security}} file. I’m not sure that at the time of CASSANDRA-16848 it
> was intended for any JDK versions.
> The point of mentioning this is that if
> {{negotiatedProtocolMustBeAcceptedProtocolTest}} is changed to use TLSv1.2,
> and support for JDK8 is dropped, then the changes made in CASSANDRA-16848
> could also be reverted.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18540) negotiatedProtocolMustBeAcceptedProtocolTest tests fail with "TLSv1.1 failed to negotiate" on JDK17
dan jatnieks created CASSANDRA-18540:
Summary: negotiatedProtocolMustBeAcceptedProtocolTest tests fail
with "TLSv1.1 failed to negotiate" on JDK17
Key: CASSANDRA-18540
URL: https://issues.apache.org/jira/browse/CASSANDRA-18540
Project: Cassandra
Issue Type: Bug
Components: CI
Reporter: dan jatnieks
Note: This depends on having a fix for CASSANDRA-18180, otherwise most/all
tests in {{NativeTransportEncryptionOptionsTest}} and
{{InternodeEncryptionOptionsTest}} are failing due to that issue.
Using the patch for CASSANDRA-18180, the
{{negotiatedProtocolMustBeAcceptedProtocolTest}} test in both
{{NativeTransportEncryptionOptionsTest}} and {{InternodeEncryptionOptionsTest}}
fails with "TLSv1.1 failed to negotiate" on JDK17.
>From what I can see, the {{negotiatedProtocolMustBeAcceptedProtocolTest}} is
>failing because in JDK11 and JDK17 the "TLSv1.1" protocol is disabled.
Since TLSv1.1 is disabled in JDK11 and 17, one possibility is to change the
test to use TLSv1.2 instead of TLSv1.1. That should work directly with JDK11
and 17, since TLSv1.2 is one of the defaults, and it won't be an issue for JDK8
as that will be dropped.
Also, I think the point of the {{negotiatedProtocolMustBeAcceptedProtocolTest}}
is to test that the {{accepted_protocols}} option is working correctly rather
than the choice of _which_ protocol is used. Meaning, I don’t think the intent
was to test TLSv1.1 specifically, rather that the mechanism of accepted
protocols works and choosing TLSv1.1 was at the time convenient - but I could
be wrong.
It also seems to me like bit of a coincidence that these tests are currently
working on JDK11, at least on CI. Indeed, running locally with JDK11, these
fail for me:
{noformat}
$ pwd
/Users/dan.jatnieks/apache/cassandra-4.0
$ java -version
openjdk version "11.0.11" 2021-04-20
OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9 (build 11.0.11+9)
OpenJDK 64-Bit Server VM AdoptOpenJDK-11.0.11+9 (build 11.0.11+9, mixed mode)
$ ant test-jvm-dtest-some
-Dtest.name=org.apache.cassandra.distributed.test.NativeTransportEncryptionOptionsTest
-Duse.jdk11=true
...
[junit-timeout] Testcase:
negotiatedProtocolMustBeAcceptedProtocolTest(org.apache.cassandra.distributed.test.NativeTransportEncryptionOptionsTest):
FAILED
[junit-timeout] Should be possible to establish a TLSv1.1 connection
expected: but was:
[junit-timeout] junit.framework.AssertionFailedError: Should be possible to
establish a TLSv1.1 connection expected: but
was:
[junit-timeout] at
org.apache.cassandra.distributed.test.NativeTransportEncryptionOptionsTest.negotiatedProtocolMustBeAcceptedProtocolTest(NativeTransportEncryptionOptionsTest.java:160)
[junit-timeout] at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit-timeout] at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit-timeout] at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
{noformat}
I believe these work on CI because of CASSANDRA-16848 - in that ticket, a
specific JDK8 build (accidentally?) dropped TLSv1.1 (later added again) which
led to adding some docker code to make sure TLSv1.1 is accepted.
I say coincidence because this change also makes it work for JDK11 and JDK17,
and I've been able to verify that making a change locally to the JDK
{{java.security}} file. I’m not sure that at the time of CASSANDRA-16848 it was
intended for any JDK versions.
The point of mentioning this is that if
{{negotiatedProtocolMustBeAcceptedProtocolTest}} is changed to use TLSv1.2, and
support for JDK8 is dropped, then the changes made in CASSANDRA-16848 could
also be reverted.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18180) bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723776#comment-17723776
]
dan jatnieks edited comment on CASSANDRA-18180 at 5/18/23 3:34 PM:
---
Recap: The {{attachment}} field of {{ByteBuffer}} is being used to store the
related {{BufferPool}} {{Chunk}} object that is associated with the buffer.
Since JDK11 some crypto overlap detection code in {{GaloisCounterMode}} expects
that any object attached to a {{ByteBuffer}} implements {{{}DirectBuffer{}}},
and if it does not, will cause a {{{}ClassCastException{}}}.
Since we see this {{ClassCastException}} in tests using encryption, it seems
it's triggered by a supported TLS setting rather than some unintended default.
The patch I provided uses the suggestion made by [~benedict] in this comment in
CASSANDRA-17992, which is to have {{Chunk}} (and also {{{}Ref{}}}) implement
{{{}DirectBuffer{}}}.
The main downside to this is that two additional {{--add-exports}} are required
to be able to access JDK internal class {{{}DirectBuffer{}}}.
Access to this internal class also has a secondary impact on
{{TestNameCheckTask}} as it uses reflection and tries to load all related
classes of tests being checked. This led to replacing the
{{checktestnameshelper}} ant task in {{build.xml}} with a java target so that
it is possible to pass the needed jvm args to {{{}TestNameCheckTask{}}}.
An alternative approach that avoids accessing jdk internals would, I think,
still need to associate {{ByteBuffer}} objects to {{Chunk}} objects. I
experimented with using a map for this, but ended up learning that
{{{}ByteBuffer{}}}'s are not suited to being used as map keys, as the docs
state:
{noformat}
because buffer hash codes are content-dependent, it is inadvisable to use
buffers as keys in hash maps or similar data structures unless it is known that
their contents will not change.{noformat}
So, you can put a {{ByteBuffer}} into a map, but if the contents change you
won't be able to retrieve/find it again.
I'm not sure what other workaround solution to propose that doesn't negatively
affect complexity and/or performance, but am happy to look into other ideas.
There is another aspect of this that I don't fully understand that's somewhat
bothersome - this test only fails with JDK17 even though the same
{{GaloisCounterMode}} overlap detection code is present in JDK11; for some
reason the same code path is not executed and the tests pass with JDK11. I
wonder why that is?
I did check the enabled cipher suites for internode messaging and native
transport that I found looking at test run output - and those are the same with
11 and 17. So I guess it's a result of some change in the JDK? If I could be
due to something else, I don't know what that might be.
was (Author: djatnieks):
Recap: The {{attachment}} field of {{ByteBuffer}} is being used to store the
related {{BufferPool}} {{Chunk}} object that is associated with the buffer.
Since JDK11 some crypto overlap detection code in {{GaloisCounterMode}} expects
that any object attached to a {{ByteBuffer}} implements {{{}DirectBuffer{}}},
and if it does not, will cause a {{{}ClassCastException{}}}.
Since we see this {{ClassCastException}} in tests using encryption, it seems
it's triggered by a supported TLS setting rather than some unintended default.
The patch I provided uses the suggestion made by [~benedict] in this comment in
CASSANDRA-17992, which is to have {{Chunk}} (and also {{{}Ref{}}}) implement
{{{}DirectBuffer{}}}.
The main downside to this is that two additional {{--add-exports}} are required
to be able to access JDK internal class {{{}DirectBuffer{}}}.
Access to this internal class also has a secondary impact on
{{TestNameCheckTask}} as it uses reflection and tries to load all related
classes of tests being checked. This led to replacing the
{{checktestnameshelper}} ant task in {{build.xml}} with a java target so that
it is possible to pass the needed jvm args to {{{}TestNameCheckTask{}}}.
An alternative approach that avoids accessing jdk internals would, I think,
still need to associate {{ByteBuffer}} objects to {{Chunk}} objects. I
experimented with using a map for this, but ended up learning that
{{{}ByteBuffer{}}}'s are not suited to being used as map keys, as the docs
state:
{noformat}
because buffer hash codes are content-dependent, it is inadvisable to use
buffers as keys in hash maps or similar data structures unless it is known that
their contents will not change.{noformat}
So, you can put a {{ByteBuffer}} into a map, but if the contents change you
won't be able to retrieve/find it again.
I'm not sure what other workaround solution to propose that doesn't negatively
affect complexity and/or performance, but am happy to look into other ideas.
There is another aspect of this that I don't fully understand
[jira] [Commented] (CASSANDRA-18180) bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723776#comment-17723776
]
dan jatnieks commented on CASSANDRA-18180:
--
Recap: The {{attachment}} field of {{ByteBuffer}} is being used to store the
related {{BufferPool}} {{Chunk}} object that is associated with the buffer.
Since JDK11 some crypto overlap detection code in {{GaloisCounterMode}} expects
that any object attached to a {{ByteBuffer}} implements {{{}DirectBuffer{}}},
and if it does not, will cause a {{{}ClassCastException{}}}.
Since we see this {{ClassCastException}} in tests using encryption, it seems
it's triggered by a supported TLS setting rather than some unintended default.
The patch I provided uses the suggestion made by [~benedict] in this comment in
CASSANDRA-17992, which is to have {{Chunk}} (and also {{{}Ref{}}}) implement
{{{}DirectBuffer{}}}.
The main downside to this is that two additional {{--add-exports}} are required
to be able to access JDK internal class {{{}DirectBuffer{}}}.
Access to this internal class also has a secondary impact on
{{TestNameCheckTask}} as it uses reflection and tries to load all related
classes of tests being checked. This led to replacing the
{{checktestnameshelper}} ant task in {{build.xml}} with a java target so that
it is possible to pass the needed jvm args to {{{}TestNameCheckTask{}}}.
An alternative approach that avoids accessing jdk internals would, I think,
still need to associate {{ByteBuffer}} objects to {{Chunk}} objects. I
experimented with using a map for this, but ended up learning that
{{{}ByteBuffer{}}}'s are not suited to being used as map keys, as the docs
state:
{noformat}
because buffer hash codes are content-dependent, it is inadvisable to use
buffers as keys in hash maps or similar data structures unless it is known that
their contents will not change.{noformat}
So, you can put a {{ByteBuffer}} into a map, but if the contents change you
won't be able to retrieve/find it again.
I'm not sure what other workaround solution to propose that doesn't negatively
affect complexity and/or performance, but am happy to look into other ideas.
There is another aspect of this that I don't fully understand that's somewhat
bothersome - while this test fails with JDK17 and the same
{{GaloisCounterMode}} overlap detection code is present in JDK11, only that
code path is not executed and the tests pass with JDK11. So I wonder why that
is?
I did check the enabled cipher suites for internode messaging and native
transport that I found looking at test run output - and those are the same with
11 and 17. So I guess it's a result of some change in the JDK? If I could be
due to something else, I don't know what that might be.
> bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
> ---
>
> Key: CASSANDRA-18180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18180
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: dan jatnieks
>Priority: Normal
>
> While working on CASSANDRA-17992 we hit:
> {code:java}
> java.lang.ClassCastException: class
> org.apache.cassandra.utils.memory.BufferPool$Chunk cannot be cast to class
> sun.nio.ch.DirectBuffer (org.apache.cassandra.utils.memory.BufferPool$Chunk
> is in unnamed module of loader 'app'; sun.nio.ch.DirectBuffer is in module
> java.base of loader 'bootstrap')\n\tat
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMEngine.overlapDetection(GaloisCounterMode.java:865)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMDecrypt.doFinal(GaloisCounterMode.java:1502)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode.engineDoFinal(GaloisCounterMode.java:447)\n\tat
>
> {code}
> -The issue is exposed with JDK 17, trunk; if interested, ping- [~e.dimitrova]
> -for current branch as there is no feature branch at the moment- we can
> build and start from trunk with JDK17 already. Circle CI can be run for JDK17
> too. For more information how to do that - .circleci/readme.md
> CC [~benedict]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18436) Unit tests in org.apache.cassandra.cql3.EmptyValuesTest class occasionally failing with JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17721509#comment-17721509
]
dan jatnieks commented on CASSANDRA-18436:
--
It looks like these pipeline builds timed out due to lack of resources in the
free tier...
{noformat}
Try out a larger resource class or running tests in parallel to speed up job
execution. Upgrade your pricing plan to take advantage of longer max job
runtimes.
context deadline exceeded{noformat}
> Unit tests in org.apache.cassandra.cql3.EmptyValuesTest class occasionally
> failing with JDK17
> -
>
> Key: CASSANDRA-18436
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18436
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: dan jatnieks
>Priority: Normal
> Fix For: 5.x
>
>
>
> All of them failed with the below stack trace for the same assertion failing:
> {code:java}
> junit.framework.AssertionFailedError: at
> org.apache.cassandra.cql3.EmptyValuesTest.verify(EmptyValuesTest.java:90) at
> org.apache.cassandra.cql3.EmptyValuesTest.verifyJsonInsert(EmptyValuesTest.java:112)
> at
> org.apache.cassandra.cql3.EmptyValuesTest.testEmptyDecimal(EmptyValuesTest.java:192)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
>
> Unfortunately I do not have a link to the CI run as this was seen last in
> private infra and not in CircleCI. Maybe we want to check with the
> multiplexer for flakiness.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18436) Unit tests in org.apache.cassandra.cql3.EmptyValuesTest class occasionally failing with JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17721058#comment-17721058
]
dan jatnieks commented on CASSANDRA-18436:
--
Thanks [~e.dimitrova] for the pointer that (of course) the top level "start"
must be approved as well :)
> Unit tests in org.apache.cassandra.cql3.EmptyValuesTest class occasionally
> failing with JDK17
> -
>
> Key: CASSANDRA-18436
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18436
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: dan jatnieks
>Priority: Normal
> Fix For: 5.x
>
>
>
> All of them failed with the below stack trace for the same assertion failing:
> {code:java}
> junit.framework.AssertionFailedError: at
> org.apache.cassandra.cql3.EmptyValuesTest.verify(EmptyValuesTest.java:90) at
> org.apache.cassandra.cql3.EmptyValuesTest.verifyJsonInsert(EmptyValuesTest.java:112)
> at
> org.apache.cassandra.cql3.EmptyValuesTest.testEmptyDecimal(EmptyValuesTest.java:192)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
>
> Unfortunately I do not have a link to the CI run as this was seen last in
> private infra and not in CircleCI. Maybe we want to check with the
> multiplexer for flakiness.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18436) Unit tests in org.apache.cassandra.cql3.EmptyValuesTest class occasionally failing with JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17721046#comment-17721046
]
dan jatnieks commented on CASSANDRA-18436:
--
I ran
{code}
./generate_11_and_17.sh -f
Generating new config.yml file for free tier from config_template_11_and_17.yml
Detecting new or modified tests with git diff --diff-filter=AMR trunk...HEAD:
org.apache.cassandra.cql3.EmptyValuesTest
Setting environment variables:
REPEATED_UTESTS: org.apache.cassandra.cql3.EmptyValuesTest
{code}
and after pushing it created this [CI for
java17_separate_tests|https://app.circleci.com/pipelines/github/djatnieks/cassandra/6/workflows/8f11795b-1665-4c82-8a35-3d82c7ff7413]
- I approved "start_j17_unit_tests_repeat" but it doesn't seem to start. Am I
doing it wrong?
Also the [java_11_separate_tests
workflow|https://app.circleci.com/pipelines/github/djatnieks/cassandra/6/workflows/31f3aa2f-4806-420e-83d1-22e2bebcc670]
has both "start_j11_unit_tests_repeat" and "start_j17_unit_tests_repeat" for
some reason. I approved them both, but again they don't start and I don't know
why?
> Unit tests in org.apache.cassandra.cql3.EmptyValuesTest class occasionally
> failing with JDK17
> -
>
> Key: CASSANDRA-18436
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18436
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: dan jatnieks
>Priority: Normal
> Fix For: 5.x
>
>
>
> All of them failed with the below stack trace for the same assertion failing:
> {code:java}
> junit.framework.AssertionFailedError: at
> org.apache.cassandra.cql3.EmptyValuesTest.verify(EmptyValuesTest.java:90) at
> org.apache.cassandra.cql3.EmptyValuesTest.verifyJsonInsert(EmptyValuesTest.java:112)
> at
> org.apache.cassandra.cql3.EmptyValuesTest.testEmptyDecimal(EmptyValuesTest.java:192)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
>
> Unfortunately I do not have a link to the CI run as this was seen last in
> private infra and not in CircleCI. Maybe we want to check with the
> multiplexer for flakiness.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16895) Build with Java 17
[ https://issues.apache.org/jira/browse/CASSANDRA-16895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dan jatnieks updated CASSANDRA-16895: - Description: This ticket is intended to group all issues found to support Java 17 in the future. Upgrade steps: * [Dependencies |https://mvnrepository.com/artifact/org.apache.cassandra/cassandra-all/4.0.1]to be updated (not all but at least those that require an update in order to work with Java 17) * More encapsulated JDK internal APIs. Some of the issues might be solved with the dependencies updates * Currently trunk compiles if we remove the Nashorn dependency (ant script tag, used for the test environment; UDFs) . The oracle recommendation to use Nashorn-core won't work for the project as it is under GPL 2.0. Most probably we will opt in for graal-sdk licensed under UPL * All tests to be cleaned * CI environment to be setup *NOTE:* GC tuning, performance testing were never agreed to be part of this ticket. Below is a snapshot of current CI failures with JDK17, it will be updated on a regular basis with a date of update *April 7th 2023* || ||Failing Test Classes||Ticket Numbers|| | |_Python DTests_| | |1|test_sjk|CASSANDRA-18343| | |_Java Ditributed Tests_| | |1-6|org.apache.cassandra.distributed.test.ReprepareOldBehaviourTest - all tests, org.apache.cassandra.distributed.test.PrepareBatchStatementsTest - all tests, org.apache.cassandra.distributed.test.IPMembershipTest - both tests, org.apache.cassandra.distributed.test.MixedModeFuzzTest, org.apache.cassandra.distributed.test.ReprepareFuzzTest, org.apache.cassandra.distributed.test.ReprepareNewBehaviourTest|CASSANDRA-16304| |7,8|org.apache.cassandra.distributed.test.NativeTransportEncryptionOptionsTest - all tests org.apache.cassandra.distributed.test.InternodeEncryptionOptionsTest - all tests|Both tests suffer from CASSANDRA-18180 fwiw, using the CASSANDRA-18180 branch, only the negotiatedProtocolMustBeAcceptedProtocolTest fails in both these tests. | |9|org.apache.cassandra.distributed.test.SSTableLoaderEncryptionOptionsTest - 2 tests|CASSANDRA-18180| | |_Unit Tests_| | |1|org.apache.cassandra.repair.RepairJobTest - 1 test|CASSANDRA-17884| |2|org.apache.cassandra.security.SSLFactoryTest - all tests|CASSANDRA-17992| |3,4|org.apache.cassandra.db.memtable.MemtableSizeOffheapBuffersTest, org.apache.cassandra.utils.concurrent.RefCountedTest|CASSANDRA-18329| |5,6|org.apache.cassandra.cql3.validation.entities.UFJavaTest, org.apache.cassandra.cql3.validation.entities.UFSecurityTest|CASSANDRA-18190| |7|org.apache.cassandra.cql3.EmptyValuesTest|CASSANDRA-18436| |8|org.apache.cassandra.transport.MessagePayloadTest-.jdk17|CASSANDRA-18437| was: This ticket is intended to group all issues found to support Java 17 in the future. Upgrade steps: * [Dependencies |https://mvnrepository.com/artifact/org.apache.cassandra/cassandra-all/4.0.1]to be updated (not all but at least those that require an update in order to work with Java 17) * More encapsulated JDK internal APIs. Some of the issues might be solved with the dependencies updates * Currently trunk compiles if we remove the Nashorn dependency (ant script tag, used for the test environment; UDFs) . The oracle recommendation to use Nashorn-core won't work for the project as it is under GPL 2.0. Most probably we will opt in for graal-sdk licensed under UPL * All tests to be cleaned * CI environment to be setup *NOTE:* GC tuning, performance testing were never agreed to be part of this ticket. Below is a snapshot of current CI failures with JDK17, it will be updated on a regular basis with a date of update *April 7th 2023* || ||Failing Test Classes||Ticket Numbers|| | |_Python DTests_| | |1|test_sjk|CASSANDRA-18343| | |_Java Ditributed Tests_| | |1-6|org.apache.cassandra.distributed.test.ReprepareOldBehaviourTest - all tests, org.apache.cassandra.distributed.test.PrepareBatchStatementsTest - all tests, org.apache.cassandra.distributed.test.IPMembershipTest - both tests, org.apache.cassandra.distributed.test.MixedModeFuzzTest, org.apache.cassandra.distributed.test.ReprepareFuzzTest, org.apache.cassandra.distributed.test.ReprepareNewBehaviourTest|CASSANDRA-16304| |7,8|org.apache.cassandra.distributed.test.NativeTransportEncryptionOptionsTest - all tests org.apache.cassandra.distributed.test.InternodeEncryptionOptionsTest - all tests|Both tests suffer from CASSANDRA-18180| |9|org.apache.cassandra.distributed.test.SSTableLoaderEncryptionOptionsTest - 2 tests|CASSANDRA-18180| | |_Unit Tests_| | |1|org.apache.cassandra.repair.RepairJobTest - 1 test|CASSANDRA-17884| |2|org.apache.cassandra.security.SSLFactoryTest - all tests|CASSANDRA-17992| |3,4|org.apache.cassandra.db.memtable.MemtableSizeOffheapBuffersTest, org.apache.cassandra.utils.concurrent.RefCountedTest|CASSANDRA-18329| |5,6|org.apache.cassandra.cql3.validation.entities.UFJavaTest,
[jira] [Assigned] (CASSANDRA-18436) Unit tests in org.apache.cassandra.cql3.EmptyValuesTest class occasionally failing with JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks reassigned CASSANDRA-18436:
Assignee: dan jatnieks
> Unit tests in org.apache.cassandra.cql3.EmptyValuesTest class occasionally
> failing with JDK17
> -
>
> Key: CASSANDRA-18436
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18436
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: dan jatnieks
>Priority: Normal
> Fix For: 5.x
>
>
>
> All of them failed with the below stack trace for the same assertion failing:
> {code:java}
> junit.framework.AssertionFailedError: at
> org.apache.cassandra.cql3.EmptyValuesTest.verify(EmptyValuesTest.java:90) at
> org.apache.cassandra.cql3.EmptyValuesTest.verifyJsonInsert(EmptyValuesTest.java:112)
> at
> org.apache.cassandra.cql3.EmptyValuesTest.testEmptyDecimal(EmptyValuesTest.java:192)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
>
> Unfortunately I do not have a link to the CI run as this was seen last in
> private infra and not in CircleCI. Maybe we want to check with the
> multiplexer for flakiness.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18180) bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks updated CASSANDRA-18180:
-
Test and Documentation Plan:
Patch:
[https://github.com/djatnieks/cassandra/tree/CASSANDRA-18180]
Tested locally with jdk11 and jdk17:
{code:java}
ant test-jvm-dtest-some
-Dtest.name=org.apache.cassandra.distributed.test.SSTableLoaderEncryptionOptionsTest{code}
Status: Patch Available (was: In Progress)
> bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
> ---
>
> Key: CASSANDRA-18180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18180
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: dan jatnieks
>Priority: Normal
>
> While working on CASSANDRA-17992 we hit:
> {code:java}
> java.lang.ClassCastException: class
> org.apache.cassandra.utils.memory.BufferPool$Chunk cannot be cast to class
> sun.nio.ch.DirectBuffer (org.apache.cassandra.utils.memory.BufferPool$Chunk
> is in unnamed module of loader 'app'; sun.nio.ch.DirectBuffer is in module
> java.base of loader 'bootstrap')\n\tat
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMEngine.overlapDetection(GaloisCounterMode.java:865)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMDecrypt.doFinal(GaloisCounterMode.java:1502)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode.engineDoFinal(GaloisCounterMode.java:447)\n\tat
>
> {code}
> -The issue is exposed with JDK 17, trunk; if interested, ping- [~e.dimitrova]
> -for current branch as there is no feature branch at the moment- we can
> build and start from trunk with JDK17 already. Circle CI can be run for JDK17
> too. For more information how to do that - .circleci/readme.md
> CC [~benedict]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18180) bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719564#comment-17719564
]
dan jatnieks commented on CASSANDRA-18180:
--
I created a patch that implements {{sun.nio.ch.DirectBuffer}} in
{{BufferPool$Chunk}} and {{Ref}} - doing this resolves the
{{ClassCastException}} coming from {{GaloisCounterMode.overlapDetection}}.
Since {{DirectBuffer}} is not publicly exposed, to make compilation work for
JDK11/17 it's necessary to add exports for {{java.base/jdk.internal.ref}} and
{{java.base/sun.nio.ch}}.
There is also a change to how {{TestNameCheckTask}} is invoked by
{{build.xml}}. Since {{TestNameCheckTask}} uses {{Reflections}} to get the
methods annotated with {{@Test}}, and that involves loading classes with the
ClassLoader it also needs to pass jvm args for jdk11/17 to avoid
{{IllegalAccessError}} to {{sun.nio.ch.DirectBuffer}} (not exported by
{{java.base}}). To accomplish this, the {{checktestnameshelper}} ant task in
{{build.xml}} is replaced with a java task, passing the necessary jvm args.
Note: these changes are not compatible with JDK8.
||Branch||CI||
|[trunk|https://github.com/djatnieks/cassandra/tree/CASSANDRA-18180]|[java
11|https://app.circleci.com/pipelines/github/djatnieks/cassandra/3/workflows/7ba8f619-835e-4464-b5a0-6c828352e3d0]
[java
17|https://app.circleci.com/pipelines/github/djatnieks/cassandra/3/workflows/86cc2e85-e718-4d09-aa55-fbec0c399733]|
> bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
> ---
>
> Key: CASSANDRA-18180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18180
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: dan jatnieks
>Priority: Normal
>
> While working on CASSANDRA-17992 we hit:
> {code:java}
> java.lang.ClassCastException: class
> org.apache.cassandra.utils.memory.BufferPool$Chunk cannot be cast to class
> sun.nio.ch.DirectBuffer (org.apache.cassandra.utils.memory.BufferPool$Chunk
> is in unnamed module of loader 'app'; sun.nio.ch.DirectBuffer is in module
> java.base of loader 'bootstrap')\n\tat
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMEngine.overlapDetection(GaloisCounterMode.java:865)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMDecrypt.doFinal(GaloisCounterMode.java:1502)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode.engineDoFinal(GaloisCounterMode.java:447)\n\tat
>
> {code}
> -The issue is exposed with JDK 17, trunk; if interested, ping- [~e.dimitrova]
> -for current branch as there is no feature branch at the moment- we can
> build and start from trunk with JDK17 already. Circle CI can be run for JDK17
> too. For more information how to do that - .circleci/readme.md
> CC [~benedict]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18436) Unit tests in org.apache.cassandra.cql3.EmptyValuesTest class occasionally failing with JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719555#comment-17719555
]
dan jatnieks commented on CASSANDRA-18436:
--
I can reproduce this locally with IntelliJ IDE, but not on the command line. I
added some error checking to the {{Process}} result and was able to understand
that while my IDE is using JDK17, the default java version on my system is
still java 8. Example of test result in IntelliJ
{code:java}
java.lang.AssertionError: Error: A JNI error has occurred, please check your
installation and try again
Exception in thread "main" java.lang.UnsupportedClassVersionError:
org/apache/cassandra/tools/SSTableExport has been compiled by a more recent
version of the Java Runtime (class file version 61.0), this version of the Java
Runtime only recognizes class file versions up to 52.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:601)
Expected :0
Actual :1 {code}
Maybe something like this would be useful to add?
||Branch||CI||
|[trunk|https://github.com/djatnieks/cassandra/tree/CASSANDRA-18436-trunk]|[java17|https://app.circleci.com/pipelines/github/djatnieks/cassandra/5/workflows/7678e370-43d6-4a94-a50b-ed30c72257f0]|
> Unit tests in org.apache.cassandra.cql3.EmptyValuesTest class occasionally
> failing with JDK17
> -
>
> Key: CASSANDRA-18436
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18436
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 5.x
>
>
>
> All of them failed with the below stack trace for the same assertion failing:
> {code:java}
> junit.framework.AssertionFailedError: at
> org.apache.cassandra.cql3.EmptyValuesTest.verify(EmptyValuesTest.java:90) at
> org.apache.cassandra.cql3.EmptyValuesTest.verifyJsonInsert(EmptyValuesTest.java:112)
> at
> org.apache.cassandra.cql3.EmptyValuesTest.testEmptyDecimal(EmptyValuesTest.java:192)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
>
> Unfortunately I do not have a link to the CI run as this was seen last in
> private infra and not in CircleCI. Maybe we want to check with the
> multiplexer for flakiness.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18426) Test failure: pdtest: dtest-upgrade.upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_current_2_1_x_To_indev_3_11_x.test_clustering_order_and_functions
[
https://issues.apache.org/jira/browse/CASSANDRA-18426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17714765#comment-17714765
]
dan jatnieks commented on CASSANDRA-18426:
--
Reviewing butler today confirms that these tests are fixed on the 3.11 branch
by the previously mentioned commit.
[Cassandra-3.11/438
TestCQLNodes2RF1_Upgrade_current_2_1_x_To_indev_3_11_x|https://ci-cassandra.apache.org/job/Cassandra-3.11/438/testReport/dtest-upgrade.upgrade_tests.cql_tests/TestCQLNodes2RF1_Upgrade_current_2_1_x_To_indev_3_11_x/]
*
TestCQLNodes2RF1_Upgrade_current_2_1_x_To_indev_3_11_x.test_clustering_order_and_functions
(upgrade)
*
TestCQLNodes2RF1_Upgrade_current_2_1_x_To_indev_3_11_x.test_function_with_null
(upgrade)
* TestCQLNodes2RF1_Upgrade_current_2_1_x_To_indev_3_11_x.test_timeuuid (upgrade)
[Cassandra-3.11/438
TestCQLNodes3RF3_Upgrade_current_2_1_x_To_indev_3_11_x|https://ci-cassandra.apache.org/job/Cassandra-3.11/438/testReport/dtest-upgrade.upgrade_tests.cql_tests/TestCQLNodes3RF3_Upgrade_current_2_1_x_To_indev_3_11_x/]
*
TestCQLNodes3RF3_Upgrade_current_2_1_x_To_indev_3_11_x.test_clustering_order_and_functions
(upgrade)
*
TestCQLNodes3RF3_Upgrade_current_2_1_x_To_indev_3_11_x.test_function_with_null
(upgrade)
* TestCQLNodes3RF3_Upgrade_current_2_1_x_To_indev_3_11_x.test_timeuuid (upgrade)
Expecting that they will also pass in the next 3.0 build, and then this issue
can be resolved.
> Test failure: pdtest:
> dtest-upgrade.upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_current_2_1_x_To_indev_3_11_x.test_clustering_order_and_functions
> --
>
> Key: CASSANDRA-18426
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18426
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Josh McKenzie
>Priority: Normal
>
> Same failure on:
> {{dtest-upgrade.upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_current_2_1_x_To_indev_3_11_x.test_timeuuid}},
> also on 3.0 branch
> {{Failed 3 times in the last 30 runs. Flakiness: 3%, Stability: 90%}}
> {code}
> Error Message
> cassandra.InvalidRequest: Error from server: code=2200 [Invalid query]
> message="Unknown function 'totimestamp'"
> {code}
> Maybe related to merge of CASSANDRA-18328?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18180) bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17713274#comment-17713274
]
dan jatnieks commented on CASSANDRA-18180:
--
{quote}I think you meant the comment on this ticket from 18/Jan/2023, not on
CASSANDRA-17992.
{quote}
Yes, that was my intent - I've update the comment to try and make it more clear
:)
{quote}It was made to provide additional context. The problem you managed to
reproduce is about what we do in regards to JDK and we hit in JDK17, it is not
about Netty.
{quote}
Ok, thanks, this helps clarify it for me - I will try and address the
{{ClassCastException}} here with this ticket then.
> bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
> ---
>
> Key: CASSANDRA-18180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18180
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: dan jatnieks
>Priority: Normal
>
> While working on CASSANDRA-17992 we hit:
> {code:java}
> java.lang.ClassCastException: class
> org.apache.cassandra.utils.memory.BufferPool$Chunk cannot be cast to class
> sun.nio.ch.DirectBuffer (org.apache.cassandra.utils.memory.BufferPool$Chunk
> is in unnamed module of loader 'app'; sun.nio.ch.DirectBuffer is in module
> java.base of loader 'bootstrap')\n\tat
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMEngine.overlapDetection(GaloisCounterMode.java:865)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMDecrypt.doFinal(GaloisCounterMode.java:1502)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode.engineDoFinal(GaloisCounterMode.java:447)\n\tat
>
> {code}
> -The issue is exposed with JDK 17, trunk; if interested, ping- [~e.dimitrova]
> -for current branch as there is no feature branch at the moment- we can
> build and start from trunk with JDK17 already. Circle CI can be run for JDK17
> too. For more information how to do that - .circleci/readme.md
> CC [~benedict]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18180) bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17712874#comment-17712874
]
dan jatnieks edited comment on CASSANDRA-18180 at 4/17/23 8:09 PM:
---
Running {{SSTableLoaderEncryptionOptionsTest}} locally, I am seeing the same
result as the first error mentioned in the previous comment, specifically:
{code:java}
Caused by: java.lang.ClassCastException: class
org.apache.cassandra.utils.memory.BufferPool$Chunk cannot be cast to class
sun.nio.ch.DirectBuffer (org.apache.cassandra.utils.memory.BufferPool$Chunk is
in unnamed module of loader
org.apache.cassandra.distributed.shared.InstanceClassLoader @4b34fff9;
sun.nio.ch.DirectBuffer is in module java.base of loader 'bootstrap')
at
java.base/com.sun.crypto.provider.GaloisCounterMode$GCMEngine.overlapDetection(GaloisCounterMode.java:865)
at
java.base/com.sun.crypto.provider.GaloisCounterMode$GCMDecrypt.doFinal(GaloisCounterMode.java:1502)
at
java.base/com.sun.crypto.provider.GaloisCounterMode.engineDoFinal(GaloisCounterMode.java:447)
at java.base/javax.crypto.Cipher.doFinal(Cipher.java:2500)
at
java.base/sun.security.ssl.SSLCipher$T12GcmReadCipherGenerator$GcmReadCipher.decrypt(SSLCipher.java:1659)
at
java.base/sun.security.ssl.SSLEngineInputRecord.decodeInputRecord(SSLEngineInputRecord.java:239)
at
java.base/sun.security.ssl.SSLEngineInputRecord.decode(SSLEngineInputRecord.java:196)
at
java.base/sun.security.ssl.SSLEngineInputRecord.decode(SSLEngineInputRecord.java:159)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:111)
at java.base/sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:736)
at
java.base/sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:691)
at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:506)
... 25 common frames omitted
{code}
Considering the comment made here on this ticket from 18/Jan/2023 about
{{GaloisCounterMode}} and {{DirectBuffer}} attachments, do we consider this
within the scope of CASSANDRA-17992, or as something separate?
was (Author: djatnieks):
Running `SSTableLoaderEncryptionOptionsTest` locally, I am seeing the same
result as the first error mentioned in the previous comment, specifically:
{code:java}
Caused by: java.lang.ClassCastException: class
org.apache.cassandra.utils.memory.BufferPool$Chunk cannot be cast to class
sun.nio.ch.DirectBuffer (org.apache.cassandra.utils.memory.BufferPool$Chunk is
in unnamed module of loader
org.apache.cassandra.distributed.shared.InstanceClassLoader @4b34fff9;
sun.nio.ch.DirectBuffer is in module java.base of loader 'bootstrap')
at
java.base/com.sun.crypto.provider.GaloisCounterMode$GCMEngine.overlapDetection(GaloisCounterMode.java:865)
at
java.base/com.sun.crypto.provider.GaloisCounterMode$GCMDecrypt.doFinal(GaloisCounterMode.java:1502)
at
java.base/com.sun.crypto.provider.GaloisCounterMode.engineDoFinal(GaloisCounterMode.java:447)
at java.base/javax.crypto.Cipher.doFinal(Cipher.java:2500)
at
java.base/sun.security.ssl.SSLCipher$T12GcmReadCipherGenerator$GcmReadCipher.decrypt(SSLCipher.java:1659)
at
java.base/sun.security.ssl.SSLEngineInputRecord.decodeInputRecord(SSLEngineInputRecord.java:239)
at
java.base/sun.security.ssl.SSLEngineInputRecord.decode(SSLEngineInputRecord.java:196)
at
java.base/sun.security.ssl.SSLEngineInputRecord.decode(SSLEngineInputRecord.java:159)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:111)
at java.base/sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:736)
at
java.base/sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:691)
at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:506)
... 25 common frames omitted
{code}
Considering the comment made 18/Jan/2023 about {{GaloisCounterMode}} and
{{DirectBuffer}} attachments, do we consider this within the scope of
CASSANDRA-17992, or as something separate?
> bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
> ---
>
> Key: CASSANDRA-18180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18180
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: dan jatnieks
>Priority: Normal
>
> While working on CASSANDRA-17992 we hit:
> {code:java}
> java.lang.ClassCastException: class
> org.apache.cassandra.utils.memory.BufferPool$Chunk cannot be cast to class
> sun.nio.ch.DirectBuffer (org.apache.cassandra.utils.memory.BufferPool$Chunk
> is in unnamed module of loader 'app'; sun.nio.ch.DirectBuffer is in module
> java.base of loader 'bootstrap')\n\tat
>
[jira] [Commented] (CASSANDRA-18180) bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17712874#comment-17712874
]
dan jatnieks commented on CASSANDRA-18180:
--
Running `SSTableLoaderEncryptionOptionsTest` locally, I am seeing the same
result as the first error mentioned in the previous comment, specifically:
{code:java}
Caused by: java.lang.ClassCastException: class
org.apache.cassandra.utils.memory.BufferPool$Chunk cannot be cast to class
sun.nio.ch.DirectBuffer (org.apache.cassandra.utils.memory.BufferPool$Chunk is
in unnamed module of loader
org.apache.cassandra.distributed.shared.InstanceClassLoader @4b34fff9;
sun.nio.ch.DirectBuffer is in module java.base of loader 'bootstrap')
at
java.base/com.sun.crypto.provider.GaloisCounterMode$GCMEngine.overlapDetection(GaloisCounterMode.java:865)
at
java.base/com.sun.crypto.provider.GaloisCounterMode$GCMDecrypt.doFinal(GaloisCounterMode.java:1502)
at
java.base/com.sun.crypto.provider.GaloisCounterMode.engineDoFinal(GaloisCounterMode.java:447)
at java.base/javax.crypto.Cipher.doFinal(Cipher.java:2500)
at
java.base/sun.security.ssl.SSLCipher$T12GcmReadCipherGenerator$GcmReadCipher.decrypt(SSLCipher.java:1659)
at
java.base/sun.security.ssl.SSLEngineInputRecord.decodeInputRecord(SSLEngineInputRecord.java:239)
at
java.base/sun.security.ssl.SSLEngineInputRecord.decode(SSLEngineInputRecord.java:196)
at
java.base/sun.security.ssl.SSLEngineInputRecord.decode(SSLEngineInputRecord.java:159)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:111)
at java.base/sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:736)
at
java.base/sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:691)
at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:506)
... 25 common frames omitted
{code}
Considering the comment made 18/Jan/2023 about {{GaloisCounterMode}} and
{{DirectBuffer}} attachments, do we consider this within the scope of
CASSANDRA-17992, or as something separate?
> bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
> ---
>
> Key: CASSANDRA-18180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18180
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: dan jatnieks
>Priority: Normal
>
> While working on CASSANDRA-17992 we hit:
> {code:java}
> java.lang.ClassCastException: class
> org.apache.cassandra.utils.memory.BufferPool$Chunk cannot be cast to class
> sun.nio.ch.DirectBuffer (org.apache.cassandra.utils.memory.BufferPool$Chunk
> is in unnamed module of loader 'app'; sun.nio.ch.DirectBuffer is in module
> java.base of loader 'bootstrap')\n\tat
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMEngine.overlapDetection(GaloisCounterMode.java:865)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMDecrypt.doFinal(GaloisCounterMode.java:1502)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode.engineDoFinal(GaloisCounterMode.java:447)\n\tat
>
> {code}
> -The issue is exposed with JDK 17, trunk; if interested, ping- [~e.dimitrova]
> -for current branch as there is no feature branch at the moment- we can
> build and start from trunk with JDK17 already. Circle CI can be run for JDK17
> too. For more information how to do that - .circleci/readme.md
> CC [~benedict]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18180) bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks reassigned CASSANDRA-18180:
Assignee: dan jatnieks
> bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
> ---
>
> Key: CASSANDRA-18180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18180
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: dan jatnieks
>Priority: Normal
>
> While working on CASSANDRA-17992 we hit:
> {code:java}
> java.lang.ClassCastException: class
> org.apache.cassandra.utils.memory.BufferPool$Chunk cannot be cast to class
> sun.nio.ch.DirectBuffer (org.apache.cassandra.utils.memory.BufferPool$Chunk
> is in unnamed module of loader 'app'; sun.nio.ch.DirectBuffer is in module
> java.base of loader 'bootstrap')\n\tat
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMEngine.overlapDetection(GaloisCounterMode.java:865)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMDecrypt.doFinal(GaloisCounterMode.java:1502)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode.engineDoFinal(GaloisCounterMode.java:447)\n\tat
>
> {code}
> The issue is exposed with JDK 17, trunk; if interested, ping [~e.dimitrova]
> for current branch as there is no feature branch at the moment
>
> CC [~benedict]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17608) Fix testMetricsWithRebuildAndStreamingToTwoNodes
[
https://issues.apache.org/jira/browse/CASSANDRA-17608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks updated CASSANDRA-17608:
-
Resolution: (was: Cannot Reproduce)
Status: Open (was: Resolved)
{{testMetricsWithRebuildAndStreamingToTwoNodes}} failed on {{trunk}} in [build
1445|https://butler.cassandra.apache.org/#/ci/upstream/workflow/Cassandra-trunk/failure/org.apache.cassandra.distributed.test.metrics/StreamingMetricsTest/testMetricsWithRebuildAndStreamingToTwoNodes]
> Fix testMetricsWithRebuildAndStreamingToTwoNodes
>
>
> Key: CASSANDRA-17608
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17608
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.1-beta1
>
>
> [https://ci-cassandra.apache.org/job/Cassandra-4.1/4/testReport/org.apache.cassandra.distributed.test.metrics/StreamingMetricsTest/testMetricsWithRebuildAndStreamingToTwoNodes/]
>
> h3.
> {code:java}
> Error Message
> [The amount of files received by node2 from node3 is not the expected one.
> [expected: 5, actual: 6]] expected:<[5]L> but was:<[6]L>
> Stacktrace
> junit.framework.AssertionFailedError: [The amount of files received by node2
> from node3 is not the expected one. [expected: 5, actual: 6]] expected:<[5]L>
> but was:<[6]L> at
> org.apache.cassandra.distributed.test.metrics.StreamingMetricsTest.lambda$checkDataReceived$e7abd6ea$1(StreamingMetricsTest.java:368)
> at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:81) at
> org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:47) at
> org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:57) at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-10747) CQL.textile syntax incorrectly includes optional keyspace for aggregate SFUNC and FINALFUNC
dan jatnieks created CASSANDRA-10747:
Summary: CQL.textile syntax incorrectly includes optional keyspace
for aggregate SFUNC and FINALFUNC
Key: CASSANDRA-10747
URL: https://issues.apache.org/jira/browse/CASSANDRA-10747
Project: Cassandra
Issue Type: Bug
Components: Documentation and Website
Reporter: dan jatnieks
Priority: Trivial
[CQL.textile|https://github.com/apache/cassandra/blob/trunk/doc/cql3/CQL.textile]
incorrectly includes an optional keyspace for the {{SFUNC}} and {{FINALFUNC}}
parts of a {{CREATE AGGREGATE}} statement.
The grammar for 2.2 and 3.0 does not allow a keyspace to be specified here.
>From the CQL.textile:
{noformat}
::= CREATE ( OR REPLACE )?
AGGREGATE ( IF NOT EXISTS )?
( '.' )?
'(' ( ',' )* ')'
SFUNC ( '.' )?
STYPE
( FINALFUNC ( '.' )?
)?
( INITCOND )?
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10741) Unable to create a function with argument of type Inet
dan jatnieks created CASSANDRA-10741:
Summary: Unable to create a function with argument of type Inet
Key: CASSANDRA-10741
URL: https://issues.apache.org/jira/browse/CASSANDRA-10741
Project: Cassandra
Issue Type: Bug
Components: CQL
Reporter: dan jatnieks
We are unable to create a function with an argument of type {{inet}} using 3.0.
This works in 2.2, but fails in 3.0
{noformat}
CREATE OR REPLACE FUNCTION test.f2 (p1 inet)
CALLED ON NULL INPUT RETURNS int LANGUAGE java AS 'return 2;';
{noformat}
>From cqlsh:
{noformat}
05:14 PM:~/projects/cassandra-3.0$ ./bin/cqlsh
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 3.0.0-SNAPSHOT | CQL spec 3.3.1 | Native protocol v4]
Use HELP for help.
cqlsh> CREATE OR REPLACE FUNCTION test.f2 (p1 inet)
... CALLED ON NULL INPUT RETURNS int LANGUAGE java AS 'return 2;';
InvalidRequest: code=2200 [Invalid query] message="Could not compile function
'test.f2' from Java source:
org.apache.cassandra.exceptions.InvalidRequestException: Java source
compilation failed:
GENERATED SOURCE ERROR: line 20 (in generated source): java.net.InetAddress
cannot be resolved to a type
GENERATED SOURCE ERROR: line 25 (in generated source): java.net.InetAddress
cannot be resolved to a type
generated source:
package org.apache.cassandra.cql3.udf.gen.ptest2ef2_4746343_7;
import java.nio.ByteBuffer;
import java.util.List;
import org.apache.cassandra.cql3.functions.JavaUDF;
import com.datastax.driver.core.DataType;
public final class Ctest2ef2_12216880_8 extends JavaUDF
{
public Ctest2ef2_12216880_8(DataType returnDataType, DataType[]
argDataTypes)
{
super(returnDataType, argDataTypes);
}
protected ByteBuffer executeImpl(int protocolVersion, List
params)
{
Integer result = xtest2ef2_16165915_9(
(java.net.InetAddress) super.compose(protocolVersion, 0,
params.get(0))
);
return super.decompose(protocolVersion, result);
}
private Integer xtest2ef2_16165915_9(java.net.InetAddress p1)
{
return 2;
}
}
"
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9771) Revert CASSANDRA-9542 (allow native functions in UDA)
[
https://issues.apache.org/jira/browse/CASSANDRA-9771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680926#comment-14680926
]
dan jatnieks commented on CASSANDRA-9771:
-
[~snazy]
We noticed that {{CreateAggregateStatement.validateFunctionKeyspace}} from
commit
[fc202a7|https://github.com/apache/cassandra/blob/fc202a756e666aff36811d995bb95956a1daeff8/src/java/org/apache/cassandra/cql3/statements/CreateAggregateStatement.java]
still allows the {{system}} keyspace if it is explicitly specified; this seems
to be contrary to the intention given in the [comment from
CASSANDRA-9542|https://issues.apache.org/jira/browse/CASSANDRA-9542?focusedCommentId=14620414page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14620414]
that would disallow any usage of functions outside the UDA keyspace.
Revert CASSANDRA-9542 (allow native functions in UDA)
-
Key: CASSANDRA-9771
URL: https://issues.apache.org/jira/browse/CASSANDRA-9771
Project: Cassandra
Issue Type: Task
Reporter: Robert Stupp
Assignee: Robert Stupp
Priority: Blocker
Fix For: 2.2.0, 3.0.0 rc1
As [noted in this
comment|https://issues.apache.org/jira/browse/CASSANDRA-9542?focusedCommentId=14620414page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14620414]
of CASSANDRA-9542, we should revert it.
Setting priority to blocker, since once 9542 gets into 2.2.0, we cannot
revert it.
Will provide a patch soon.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-9559) IndexOutOfBoundsException inserting into TupleType
dan jatnieks created CASSANDRA-9559:
---
Summary: IndexOutOfBoundsException inserting into TupleType
Key: CASSANDRA-9559
URL: https://issues.apache.org/jira/browse/CASSANDRA-9559
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: dan jatnieks
Trying to run this query on the 2.2 branch resulted in
IndexOutOfBoundsException:
{noformat}
INSERT INTO datatypes.alltypes (pk,c_tuple) VALUES (1,'01:02:03.456789012')
{noformat}
The column {{c_tuple}} is of type {{tupleint}}.
The cause seems to be that {{TupleType.fromString}} is splitting the source
string on {{:}} (meant for UDT only?) but does not consider that the number of
those fields may exceed the number of types defined in the Tuple.
A simple fix would be to bound the for loop with {{types.size()}} instead of
{{fieldStrings.size()}} but still some error should be raised if
{{fieldStrings.size() types.size()}}.
{noformat}
java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
at java.util.ArrayList.rangeCheck(ArrayList.java:635) ~[na:1.7.0_51]
at java.util.ArrayList.get(ArrayList.java:411) ~[na:1.7.0_51]
at org.apache.cassandra.db.marshal.TupleType.type(TupleType.java:60)
~[main/:na]
at
org.apache.cassandra.db.marshal.TupleType.fromString(TupleType.java:224)
~[main/:na]
at
org.apache.cassandra.cql3.Constants$Literal.parsedValue(Constants.java:152)
~[main/:na]
at
org.apache.cassandra.cql3.Constants$Literal.prepare(Constants.java:138)
~[main/:na]
at
org.apache.cassandra.cql3.Constants$Literal.prepare(Constants.java:91)
~[main/:na]
at
org.apache.cassandra.cql3.Operation$SetValue.prepare(Operation.java:167)
~[main/:na]
at
org.apache.cassandra.cql3.statements.UpdateStatement$ParsedInsert.prepareInternal(UpdateStatement.java:213)
~[main/:na]
at
org.apache.cassandra.cql3.statements.ModificationStatement$Parsed.prepare(ModificationStatement.java:710)
~[main/:na]
at
org.apache.cassandra.cql3.statements.ModificationStatement$Parsed.prepare(ModificationStatement.java:698)
~[main/:na]
at
org.apache.cassandra.cql3.QueryProcessor.getStatement(QueryProcessor.java:493)
~[main/:na]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:247)
~[main/:na]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:241)
~[main/:na]
at
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:119)
~[main/:na]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
[main/:na]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
[main/:na]
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[na:1.7.0_51]
at
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
[main/:na]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
[main/:na]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9559) IndexOutOfBoundsException inserting into TupleType
[
https://issues.apache.org/jira/browse/CASSANDRA-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14575409#comment-14575409
]
dan jatnieks commented on CASSANDRA-9559:
-
Just checked 2.1.3 and got the same exception.
IndexOutOfBoundsException inserting into TupleType
--
Key: CASSANDRA-9559
URL: https://issues.apache.org/jira/browse/CASSANDRA-9559
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: dan jatnieks
Trying to run this query on the 2.2 branch resulted in
IndexOutOfBoundsException:
{noformat}
INSERT INTO datatypes.alltypes (pk,c_tuple) VALUES (1,'01:02:03.456789012')
{noformat}
The column {{c_tuple}} is of type {{tupleint}}.
The cause seems to be that {{TupleType.fromString}} is splitting the source
string on {{:}} (meant for UDT only?) but does not consider that the number
of those fields may exceed the number of types defined in the Tuple.
A simple fix would be to bound the for loop with {{types.size()}} instead of
{{fieldStrings.size()}} but still some error should be raised if
{{fieldStrings.size() types.size()}}.
{noformat}
java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
at java.util.ArrayList.rangeCheck(ArrayList.java:635) ~[na:1.7.0_51]
at java.util.ArrayList.get(ArrayList.java:411) ~[na:1.7.0_51]
at org.apache.cassandra.db.marshal.TupleType.type(TupleType.java:60)
~[main/:na]
at
org.apache.cassandra.db.marshal.TupleType.fromString(TupleType.java:224)
~[main/:na]
at
org.apache.cassandra.cql3.Constants$Literal.parsedValue(Constants.java:152)
~[main/:na]
at
org.apache.cassandra.cql3.Constants$Literal.prepare(Constants.java:138)
~[main/:na]
at
org.apache.cassandra.cql3.Constants$Literal.prepare(Constants.java:91)
~[main/:na]
at
org.apache.cassandra.cql3.Operation$SetValue.prepare(Operation.java:167)
~[main/:na]
at
org.apache.cassandra.cql3.statements.UpdateStatement$ParsedInsert.prepareInternal(UpdateStatement.java:213)
~[main/:na]
at
org.apache.cassandra.cql3.statements.ModificationStatement$Parsed.prepare(ModificationStatement.java:710)
~[main/:na]
at
org.apache.cassandra.cql3.statements.ModificationStatement$Parsed.prepare(ModificationStatement.java:698)
~[main/:na]
at
org.apache.cassandra.cql3.QueryProcessor.getStatement(QueryProcessor.java:493)
~[main/:na]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:247)
~[main/:na]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:241)
~[main/:na]
at
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:119)
~[main/:na]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
[main/:na]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
[main/:na]
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[na:1.7.0_51]
at
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
[main/:na]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
[main/:na]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9559) IndexOutOfBoundsException inserting into TupleType
[
https://issues.apache.org/jira/browse/CASSANDRA-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks updated CASSANDRA-9559:
Since Version: 2.1.3
IndexOutOfBoundsException inserting into TupleType
--
Key: CASSANDRA-9559
URL: https://issues.apache.org/jira/browse/CASSANDRA-9559
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: dan jatnieks
Trying to run this query on the 2.2 branch resulted in
IndexOutOfBoundsException:
{noformat}
INSERT INTO datatypes.alltypes (pk,c_tuple) VALUES (1,'01:02:03.456789012')
{noformat}
The column {{c_tuple}} is of type {{tupleint}}.
The cause seems to be that {{TupleType.fromString}} is splitting the source
string on {{:}} (meant for UDT only?) but does not consider that the number
of those fields may exceed the number of types defined in the Tuple.
A simple fix would be to bound the for loop with {{types.size()}} instead of
{{fieldStrings.size()}} but still some error should be raised if
{{fieldStrings.size() types.size()}}.
{noformat}
java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
at java.util.ArrayList.rangeCheck(ArrayList.java:635) ~[na:1.7.0_51]
at java.util.ArrayList.get(ArrayList.java:411) ~[na:1.7.0_51]
at org.apache.cassandra.db.marshal.TupleType.type(TupleType.java:60)
~[main/:na]
at
org.apache.cassandra.db.marshal.TupleType.fromString(TupleType.java:224)
~[main/:na]
at
org.apache.cassandra.cql3.Constants$Literal.parsedValue(Constants.java:152)
~[main/:na]
at
org.apache.cassandra.cql3.Constants$Literal.prepare(Constants.java:138)
~[main/:na]
at
org.apache.cassandra.cql3.Constants$Literal.prepare(Constants.java:91)
~[main/:na]
at
org.apache.cassandra.cql3.Operation$SetValue.prepare(Operation.java:167)
~[main/:na]
at
org.apache.cassandra.cql3.statements.UpdateStatement$ParsedInsert.prepareInternal(UpdateStatement.java:213)
~[main/:na]
at
org.apache.cassandra.cql3.statements.ModificationStatement$Parsed.prepare(ModificationStatement.java:710)
~[main/:na]
at
org.apache.cassandra.cql3.statements.ModificationStatement$Parsed.prepare(ModificationStatement.java:698)
~[main/:na]
at
org.apache.cassandra.cql3.QueryProcessor.getStatement(QueryProcessor.java:493)
~[main/:na]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:247)
~[main/:na]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:241)
~[main/:na]
at
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:119)
~[main/:na]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
[main/:na]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
[main/:na]
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[na:1.7.0_51]
at
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
[main/:na]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
[main/:na]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-9412) Cleanup cqlsh help text for Grant/Revoke
dan jatnieks created CASSANDRA-9412:
---
Summary: Cleanup cqlsh help text for Grant/Revoke
Key: CASSANDRA-9412
URL: https://issues.apache.org/jira/browse/CASSANDRA-9412
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: dan jatnieks
Assignee: Sam Tunnicliffe
Priority: Trivial
Following up earlier discussion with Sam. The help text for Grant and Revoke in
cqlsh displays the target as
{noformat}
TO [ROLE rolename | USER username]
{noformat}
but should be just
{noformat}
TO roleOrUsername
{noformat}
There are also out of date comments in Cql.g on grantRoleStatement and
revokeRoleStatement that indicate a {{ROLE}} keyword is used in these
statements when that is no longer the case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9077) NPE
[
https://issues.apache.org/jira/browse/CASSANDRA-9077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks updated CASSANDRA-9077:
Attachment: 9077-stacktrace.log
Attaching stacktrace from system.log.
NPE
---
Key: CASSANDRA-9077
URL: https://issues.apache.org/jira/browse/CASSANDRA-9077
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: dan jatnieks
Priority: Minor
Attachments: 9077-stacktrace.log
I am seeing an NPE on the latest 2.1 branch with this sequence of deletes
from a list - first delete the entire list, then attempt to delete one
element.
I expected to see {{List index 0 out of bound, list has size 0}} but instead
got an NPE.
{noformat}
./bin/cqlsh
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 2.1.3-SNAPSHOT | CQL spec 3.2.0 | Native protocol v3]
Use HELP for help.
cqlsh use frozen_collections ;
cqlsh:frozen_collections DROP TABLE IF EXISTS t;
cqlsh:frozen_collections CREATE TABLE t (id text PRIMARY KEY, l listtext,
s settext);
cqlsh:frozen_collections INSERT INTO t (id, l, s) VALUES ('user', ['1'],
{'1'});
cqlsh:frozen_collections
cqlsh:frozen_collections DELETE l FROM t WHERE id ='user';
cqlsh:frozen_collections //INSERT INTO t (id, l) VALUES ('user', ['1']);
cqlsh:frozen_collections DELETE l[0] FROM t WHERE id = 'user';
ServerError: ErrorMessage code= [Server error]
message=java.lang.NullPointerException
cqlsh:frozen_collections
cqlsh:frozen_collections DELETE s FROM t WHERE id ='user';
cqlsh:frozen_collections DELETE s['1'] FROM t WHERE id = 'user';{noformat}
It appears the {{DELETE emails...}} directly followed by {{DELETE
emails[0]...}} is the offending sequence. Either one alone works fine, as
does adding an intervening insert/update.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-9077) NPE
dan jatnieks created CASSANDRA-9077:
---
Summary: NPE
Key: CASSANDRA-9077
URL: https://issues.apache.org/jira/browse/CASSANDRA-9077
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: dan jatnieks
Priority: Minor
I am seeing an NPE on the latest 2.1 branch with this sequence of deletes from
a list - first delete the entire list, then attempt to delete one element.
I expected to see {{List index 0 out of bound, list has size 0}} but instead
got an NPE.
{noformat}
./bin/cqlsh
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 2.1.3-SNAPSHOT | CQL spec 3.2.0 | Native protocol v3]
Use HELP for help.
cqlsh use frozen_collections ;
cqlsh:frozen_collections DROP TABLE IF EXISTS t;
cqlsh:frozen_collections CREATE TABLE t (id text PRIMARY KEY, l listtext, s
settext);
cqlsh:frozen_collections INSERT INTO t (id, l, s) VALUES ('user', ['1'],
{'1'});
cqlsh:frozen_collections
cqlsh:frozen_collections DELETE l FROM t WHERE id ='user';
cqlsh:frozen_collections //INSERT INTO t (id, l) VALUES ('user', ['1']);
cqlsh:frozen_collections DELETE l[0] FROM t WHERE id = 'user';
ServerError: ErrorMessage code= [Server error]
message=java.lang.NullPointerException
cqlsh:frozen_collections
cqlsh:frozen_collections DELETE s FROM t WHERE id ='user';
cqlsh:frozen_collections DELETE s['1'] FROM t WHERE id = 'user';{noformat}
It appears the {{DELETE emails...}} directly followed by {{DELETE
emails[0]...}} is the offending sequence. Either one alone works fine, as does
adding an intervening insert/update.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9077) NPE
[
https://issues.apache.org/jira/browse/CASSANDRA-9077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks updated CASSANDRA-9077:
Description:
I am seeing an NPE on the latest 2.1 branch with this sequence of deletes from
a list - first delete the entire list, then attempt to delete one element.
I expected to see {{List index 0 out of bound, list has size 0}} but instead
got an NPE.
{noformat}
./bin/cqlsh
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 2.1.3-SNAPSHOT | CQL spec 3.2.0 | Native protocol v3]
Use HELP for help.
cqlsh use frozen_collections ;
cqlsh:frozen_collections DROP TABLE IF EXISTS t;
cqlsh:frozen_collections CREATE TABLE t (id text PRIMARY KEY, l listtext, s
settext);
cqlsh:frozen_collections INSERT INTO t (id, l, s) VALUES ('user', ['1'],
{'1'});
cqlsh:frozen_collections
cqlsh:frozen_collections DELETE l FROM t WHERE id ='user';
cqlsh:frozen_collections //INSERT INTO t (id, l) VALUES ('user', ['1']);
cqlsh:frozen_collections DELETE l[0] FROM t WHERE id = 'user';
ServerError: ErrorMessage code= [Server error]
message=java.lang.NullPointerException
cqlsh:frozen_collections
cqlsh:frozen_collections DELETE s FROM t WHERE id ='user';
cqlsh:frozen_collections DELETE s['1'] FROM t WHERE id = 'user';
{noformat}
It appears the {{DELETE emails...}} directly followed by {{DELETE
emails[0]...}} is the offending sequence. Either one alone works fine, as does
adding an intervening insert/update.
The same sequence performed on a Set rather than List works (as shown above).
was:
I am seeing an NPE on the latest 2.1 branch with this sequence of deletes from
a list - first delete the entire list, then attempt to delete one element.
I expected to see {{List index 0 out of bound, list has size 0}} but instead
got an NPE.
{noformat}
./bin/cqlsh
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 2.1.3-SNAPSHOT | CQL spec 3.2.0 | Native protocol v3]
Use HELP for help.
cqlsh use frozen_collections ;
cqlsh:frozen_collections DROP TABLE IF EXISTS t;
cqlsh:frozen_collections CREATE TABLE t (id text PRIMARY KEY, l listtext, s
settext);
cqlsh:frozen_collections INSERT INTO t (id, l, s) VALUES ('user', ['1'],
{'1'});
cqlsh:frozen_collections
cqlsh:frozen_collections DELETE l FROM t WHERE id ='user';
cqlsh:frozen_collections //INSERT INTO t (id, l) VALUES ('user', ['1']);
cqlsh:frozen_collections DELETE l[0] FROM t WHERE id = 'user';
ServerError: ErrorMessage code= [Server error]
message=java.lang.NullPointerException
cqlsh:frozen_collections
cqlsh:frozen_collections DELETE s FROM t WHERE id ='user';
cqlsh:frozen_collections DELETE s['1'] FROM t WHERE id = 'user';{noformat}
It appears the {{DELETE emails...}} directly followed by {{DELETE
emails[0]...}} is the offending sequence. Either one alone works fine, as does
adding an intervening insert/update.
NPE
---
Key: CASSANDRA-9077
URL: https://issues.apache.org/jira/browse/CASSANDRA-9077
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: dan jatnieks
Priority: Minor
Attachments: 9077-stacktrace.log
I am seeing an NPE on the latest 2.1 branch with this sequence of deletes
from a list - first delete the entire list, then attempt to delete one
element.
I expected to see {{List index 0 out of bound, list has size 0}} but instead
got an NPE.
{noformat}
./bin/cqlsh
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 2.1.3-SNAPSHOT | CQL spec 3.2.0 | Native protocol v3]
Use HELP for help.
cqlsh use frozen_collections ;
cqlsh:frozen_collections DROP TABLE IF EXISTS t;
cqlsh:frozen_collections CREATE TABLE t (id text PRIMARY KEY, l listtext,
s settext);
cqlsh:frozen_collections INSERT INTO t (id, l, s) VALUES ('user', ['1'],
{'1'});
cqlsh:frozen_collections
cqlsh:frozen_collections DELETE l FROM t WHERE id ='user';
cqlsh:frozen_collections //INSERT INTO t (id, l) VALUES ('user', ['1']);
cqlsh:frozen_collections DELETE l[0] FROM t WHERE id = 'user';
ServerError: ErrorMessage code= [Server error]
message=java.lang.NullPointerException
cqlsh:frozen_collections
cqlsh:frozen_collections DELETE s FROM t WHERE id ='user';
cqlsh:frozen_collections DELETE s['1'] FROM t WHERE id = 'user';
{noformat}
It appears the {{DELETE emails...}} directly followed by {{DELETE
emails[0]...}} is the offending sequence. Either one alone works fine, as
does adding an intervening insert/update.
The same sequence performed on a Set rather than List works (as shown above).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-7634) cqlsh error tracing CAS
dan jatnieks created CASSANDRA-7634:
---
Summary: cqlsh error tracing CAS
Key: CASSANDRA-7634
URL: https://issues.apache.org/jira/browse/CASSANDRA-7634
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: dan jatnieks
Priority: Minor
On branch cassandra-2.1.0
Getting message {{'NoneType' object has no attribute 'microseconds'}} from
cqlsh while tracing a CAS statement.
{noformat}
Connected to devc-large at 146.148.39.53:9042.
[cqlsh 5.0.1 | Cassandra 2.1.0-rc4-SNAPSHOT | CQL spec 3.2.0 | Native protocol
v3]
Use HELP for help.
cqlsh use test2;
cqlsh:test2 update cas set c2 = 2 where c1 = 1 if c3 = 1;
applied
-
True
cqlsh:test2 tracing on;
Now tracing requests.
cqlsh:test2 update cas set c2 = 2 where c1 = 1 if c3 = 1;
applied
-
True
'NoneType' object has no attribute 'microseconds'
cqlsh:test2
{noformat}
Tracing {{select *}} from the same table works as expected, but tracing the
conditional update results in the error.
More details:
{noformat}
cqlsh:test2 desc keyspace
CREATE KEYSPACE test2 WITH replication = {'class': 'SimpleStrategy',
'replication_factor': '3'} AND durable_writes = true;
CREATE TABLE test2.cas (
c1 int PRIMARY KEY,
c2 int,
c3 int
) WITH bloom_filter_fp_chance = 0.01
AND caching = '{keys:ALL, rows_per_partition:NONE}'
AND comment = ''
AND compaction = {'min_threshold': '4', 'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32'}
AND compression = {'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
cqlsh:test2 tracing on;
Tracing is already enabled. Use TRACING OFF to disable.
cqlsh:test2 select * from cas;
c1 | c2 | c3
++
1 | 2 | 1
(1 rows)
Tracing session: 8f0c8340-16ae-11e4-8ca3-fb429d8fb4a7
activity
| timestamp | source
| source_elapsed
---+++
Execute CQL3 query | 2014-07-28 16:26:10.804000 |
10.240.139.181 | 0
Parsing select * from
cas LIMIT 1; [SharedPool-Worker-1] | 2014-07-28 16:26:10.804000 |
10.240.139.181 | 78
Preparing statement [SharedPool-Worker-1] | 2014-07-28 16:26:10.804000 |
10.240.139.181 |282
Determining
replicas to query [SharedPool-Worker-1] | 2014-07-28 16:26:10.804000 |
10.240.139.181 |501
Enqueuing request
to /10.240.189.138 [SharedPool-Worker-1] | 2014-07-28 16:26:10.804000 |
10.240.139.181 |918
Sending message to
/10.240.189.138 [WRITE-/10.240.189.138] | 2014-07-28 16:26:10.805000 |
10.240.139.181 | 1095
Message
received from /10.240.139.181 [Thread-27] | 2014-07-28 16:26:10.805000 |
10.240.189.138 | 28
Executing seq scan across 0 sstables for [min(-9223372036854775808),
max(-4611686018427387904)] [SharedPool-Worker-1] | 2014-07-28 16:26:10.805000 |
10.240.189.138 |384
Scanned 0
rows and matched 0 [SharedPool-Worker-1] | 2014-07-28 16:26:10.805000 |
10.240.189.138 |481
Enqueuing response
to /10.240.139.181 [SharedPool-Worker-1] | 2014-07-28 16:26:10.805000 |
10.240.189.138 |570
Sending message to
/10.240.139.181 [WRITE-/10.240.139.181] | 2014-07-28 16:26:10.806000 |
10.240.189.138 |735
Message
received from /10.240.189.138 [Thread-30] | 2014-07-28 16:26:10.807000 |
10.240.139.181 | 3264
Processing response
from /10.240.189.138 [SharedPool-Worker-2] | 2014-07-28 16:26:10.807000 |
10.240.139.181 |
[jira] [Created] (CASSANDRA-7588) cqlsh error for query against collection index - list index out of range
dan jatnieks created CASSANDRA-7588:
---
Summary: cqlsh error for query against collection index - list
index out of range
Key: CASSANDRA-7588
URL: https://issues.apache.org/jira/browse/CASSANDRA-7588
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: dan jatnieks
This worked in 2.1 RC1
{noformat}
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 2.1.0-rc1-SNAPSHOT | CQL spec 3.1.7 | Native protocol
v3]
Use HELP for help.
cqlsh use k1;
cqlsh:k1 SELECT id, description FROM products WHERE categories CONTAINS 'hdtv';
id| description
---+-
29412 |32-inch LED HDTV (black)
34134 | 120-inch 1080p 3D plasma TV
(2 rows)
{noformat}
But fails with RC4:
{noformat}
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 2.1.0-rc4-SNAPSHOT | CQL spec 3.2.0 | Native protocol
v3]
Use HELP for help.
cqlsh use k1;
cqlsh:k1 SELECT id, description FROM products WHERE categories CONTAINS 'hdtv';
list index out of range
cqlsh:k1
{noformat}
This is using the example from the blog post
http://www.datastax.com/dev/blog/cql-in-2-1
A more complete repro:
{noformat}
cqlsh:k1
cqlsh:k1 CREATE KEYSPACE cat_index_test
... WITH replication = {'class': 'SimpleStrategy', 'replication_factor':
1};
cqlsh:k1 USE cat_index_test;
cqlsh:cat_index_test
cqlsh:cat_index_test CREATE TABLE IF NOT EXISTS products (
... id int PRIMARY KEY,
... description text,
... price int,
... categories settext,
... features maptext, text
... );
cqlsh:cat_index_test
cqlsh:cat_index_test CREATE INDEX IF NOT EXISTS cat_index ON
products(categories);
cqlsh:cat_index_test CREATE INDEX IF NOT EXISTS feat_index ON
products(features);
cqlsh:cat_index_test
cqlsh:cat_index_test INSERT INTO products(id, description, price,
categories, features)
...VALUES (34134,
...'120-inch 1080p 3D plasma TV',
...,
...{'tv', '3D', 'hdtv'},
...{'screen' : '120-inch', 'refresh-rate' :
'400hz', 'techno' : 'plasma'});
cqlsh:cat_index_test
cqlsh:cat_index_test INSERT INTO products(id, description, price,
categories, features)
...VALUES (29412,
...'32-inch LED HDTV (black)',
...929,
...{'tv', 'hdtv'},
...{'screen' : '32-inch', 'techno' : 'LED'});
cqlsh:cat_index_test
cqlsh:cat_index_test INSERT INTO products(id, description, price,
categories, features)
...VALUES (38471,
...'32-inch LCD TV',
...110,
...{'tv', 'used'},
...{'screen' : '32-inch', 'techno' : 'LCD'});
cqlsh:cat_index_test SELECT id, description FROM products WHERE categories
CONTAINS 'hdtv';
list index out of range
cqlsh:cat_index_test SELECT id, description FROM products WHERE features
CONTAINS '32-inch';
list index out of range
cqlsh:cat_index_test DROP INDEX feat_index;
cqlsh:cat_index_test CREATE INDEX feat_key_index ON products(KEYS(features));
cqlsh:cat_index_test
cqlsh:cat_index_test SELECT id, description
... FROM products
... WHERE features CONTAINS KEY 'refresh-rate';
list index out of range
{noformat}
This appears to be a cqlsh issue, since these queries still work when executed
using DevCenter.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7588) cqlsh error for query against collection index - list index out of range
[
https://issues.apache.org/jira/browse/CASSANDRA-7588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14070789#comment-14070789
]
dan jatnieks commented on CASSANDRA-7588:
-
[~brandon.williams] You're right, I was on cassandra-2.1, and I was going by
what cqlsh reported. Sorry for the confusion. And, yes, I tagged as rc3 because
I didn't find rc4.
cqlsh error for query against collection index - list index out of range
Key: CASSANDRA-7588
URL: https://issues.apache.org/jira/browse/CASSANDRA-7588
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: dan jatnieks
Assignee: Robert Stupp
Labels: cqlsh
Fix For: 2.1.1
Attachments: 7588-cqlsh-6910-contains-allow-filtering.txt
This worked in 2.1 RC1
{noformat}
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 2.1.0-rc1-SNAPSHOT | CQL spec 3.1.7 | Native
protocol v3]
Use HELP for help.
cqlsh use k1;
cqlsh:k1 SELECT id, description FROM products WHERE categories CONTAINS
'hdtv';
id| description
---+-
29412 |32-inch LED HDTV (black)
34134 | 120-inch 1080p 3D plasma TV
(2 rows)
{noformat}
But fails with RC4:
{noformat}
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 2.1.0-rc4-SNAPSHOT | CQL spec 3.2.0 | Native
protocol v3]
Use HELP for help.
cqlsh use k1;
cqlsh:k1 SELECT id, description FROM products WHERE categories CONTAINS
'hdtv';
list index out of range
cqlsh:k1
{noformat}
This is using the example from the blog post
http://www.datastax.com/dev/blog/cql-in-2-1
A more complete repro:
{noformat}
cqlsh:k1
cqlsh:k1 CREATE KEYSPACE cat_index_test
... WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 1};
cqlsh:k1 USE cat_index_test;
cqlsh:cat_index_test
cqlsh:cat_index_test CREATE TABLE IF NOT EXISTS products (
... id int PRIMARY KEY,
... description text,
... price int,
... categories settext,
... features maptext, text
... );
cqlsh:cat_index_test
cqlsh:cat_index_test CREATE INDEX IF NOT EXISTS cat_index ON
products(categories);
cqlsh:cat_index_test CREATE INDEX IF NOT EXISTS feat_index ON
products(features);
cqlsh:cat_index_test
cqlsh:cat_index_test INSERT INTO products(id, description, price,
categories, features)
...VALUES (34134,
...'120-inch 1080p 3D plasma TV',
...,
...{'tv', '3D', 'hdtv'},
...{'screen' : '120-inch', 'refresh-rate' :
'400hz', 'techno' : 'plasma'});
cqlsh:cat_index_test
cqlsh:cat_index_test INSERT INTO products(id, description, price,
categories, features)
...VALUES (29412,
...'32-inch LED HDTV (black)',
...929,
...{'tv', 'hdtv'},
...{'screen' : '32-inch', 'techno' :
'LED'});
cqlsh:cat_index_test
cqlsh:cat_index_test INSERT INTO products(id, description, price,
categories, features)
...VALUES (38471,
...'32-inch LCD TV',
...110,
...{'tv', 'used'},
...{'screen' : '32-inch', 'techno' :
'LCD'});
cqlsh:cat_index_test SELECT id, description FROM products WHERE categories
CONTAINS 'hdtv';
list index out of range
cqlsh:cat_index_test SELECT id, description FROM products WHERE features
CONTAINS '32-inch';
list index out of range
cqlsh:cat_index_test DROP INDEX feat_index;
cqlsh:cat_index_test CREATE INDEX feat_key_index ON products(KEYS(features));
cqlsh:cat_index_test
cqlsh:cat_index_test SELECT id, description
... FROM products
... WHERE features CONTAINS KEY 'refresh-rate';
list index out of range
{noformat}
This appears to be a cqlsh issue, since these queries still work when
executed using DevCenter.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7588) cqlsh error for query against collection index - list index out of range
[
https://issues.apache.org/jira/browse/CASSANDRA-7588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks updated CASSANDRA-7588:
Description:
This worked in 2.1 RC1
{noformat}
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 2.1.0-rc1-SNAPSHOT | CQL spec 3.1.7 | Native protocol
v3]
Use HELP for help.
cqlsh use k1;
cqlsh:k1 SELECT id, description FROM products WHERE categories CONTAINS 'hdtv';
id| description
---+-
29412 |32-inch LED HDTV (black)
34134 | 120-inch 1080p 3D plasma TV
(2 rows)
{noformat}
But fails on 2.1:
{noformat}
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 2.1.0-rc4-SNAPSHOT | CQL spec 3.2.0 | Native protocol
v3]
Use HELP for help.
cqlsh use k1;
cqlsh:k1 SELECT id, description FROM products WHERE categories CONTAINS 'hdtv';
list index out of range
cqlsh:k1
{noformat}
This is using the example from the blog post
http://www.datastax.com/dev/blog/cql-in-2-1
A more complete repro:
{noformat}
cqlsh:k1
cqlsh:k1 CREATE KEYSPACE cat_index_test
... WITH replication = {'class': 'SimpleStrategy', 'replication_factor':
1};
cqlsh:k1 USE cat_index_test;
cqlsh:cat_index_test
cqlsh:cat_index_test CREATE TABLE IF NOT EXISTS products (
... id int PRIMARY KEY,
... description text,
... price int,
... categories settext,
... features maptext, text
... );
cqlsh:cat_index_test
cqlsh:cat_index_test CREATE INDEX IF NOT EXISTS cat_index ON
products(categories);
cqlsh:cat_index_test CREATE INDEX IF NOT EXISTS feat_index ON
products(features);
cqlsh:cat_index_test
cqlsh:cat_index_test INSERT INTO products(id, description, price,
categories, features)
...VALUES (34134,
...'120-inch 1080p 3D plasma TV',
...,
...{'tv', '3D', 'hdtv'},
...{'screen' : '120-inch', 'refresh-rate' :
'400hz', 'techno' : 'plasma'});
cqlsh:cat_index_test
cqlsh:cat_index_test INSERT INTO products(id, description, price,
categories, features)
...VALUES (29412,
...'32-inch LED HDTV (black)',
...929,
...{'tv', 'hdtv'},
...{'screen' : '32-inch', 'techno' : 'LED'});
cqlsh:cat_index_test
cqlsh:cat_index_test INSERT INTO products(id, description, price,
categories, features)
...VALUES (38471,
...'32-inch LCD TV',
...110,
...{'tv', 'used'},
...{'screen' : '32-inch', 'techno' : 'LCD'});
cqlsh:cat_index_test SELECT id, description FROM products WHERE categories
CONTAINS 'hdtv';
list index out of range
cqlsh:cat_index_test SELECT id, description FROM products WHERE features
CONTAINS '32-inch';
list index out of range
cqlsh:cat_index_test DROP INDEX feat_index;
cqlsh:cat_index_test CREATE INDEX feat_key_index ON products(KEYS(features));
cqlsh:cat_index_test
cqlsh:cat_index_test SELECT id, description
... FROM products
... WHERE features CONTAINS KEY 'refresh-rate';
list index out of range
{noformat}
This appears to be a cqlsh issue, since these queries still work when executed
using DevCenter.
was:
This worked in 2.1 RC1
{noformat}
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 2.1.0-rc1-SNAPSHOT | CQL spec 3.1.7 | Native protocol
v3]
Use HELP for help.
cqlsh use k1;
cqlsh:k1 SELECT id, description FROM products WHERE categories CONTAINS 'hdtv';
id| description
---+-
29412 |32-inch LED HDTV (black)
34134 | 120-inch 1080p 3D plasma TV
(2 rows)
{noformat}
But fails with RC4:
{noformat}
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 2.1.0-rc4-SNAPSHOT | CQL spec 3.2.0 | Native protocol
v3]
Use HELP for help.
cqlsh use k1;
cqlsh:k1 SELECT id, description FROM products WHERE categories CONTAINS 'hdtv';
list index out of range
cqlsh:k1
{noformat}
This is using the example from the blog post
http://www.datastax.com/dev/blog/cql-in-2-1
A more complete repro:
{noformat}
cqlsh:k1
cqlsh:k1 CREATE KEYSPACE cat_index_test
... WITH replication = {'class': 'SimpleStrategy', 'replication_factor':
1};
cqlsh:k1 USE cat_index_test;
cqlsh:cat_index_test
cqlsh:cat_index_test CREATE TABLE IF NOT EXISTS products (
... id int PRIMARY KEY,
... description text,
... price int,
... categories settext,
[jira] [Created] (CASSANDRA-7487) Create index does not reject duplicate index name on tables with the same name but different keyspace
dan jatnieks created CASSANDRA-7487:
---
Summary: Create index does not reject duplicate index name on
tables with the same name but different keyspace
Key: CASSANDRA-7487
URL: https://issues.apache.org/jira/browse/CASSANDRA-7487
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: dan jatnieks
Priority: Minor
Came across this while experimenting with how index names are managed.
Create index tries to manage index names globally - even though indexes are
maintained per keyspace (table?) - and attempts to create indexes with the same
name are generally rejected.
However, given two tables with the same name in different keyspaces, it's
possible to create an index of the same name on each of those tables.
Here's the scenario:
{noformat}
cqlsh CREATE KEYSPACE ks1 WITH replication = {'class': 'SimpleStrategy',
'replication_factor': '1'};
cqlsh CREATE KEYSPACE ks2 WITH replication = {'class': 'SimpleStrategy',
'replication_factor': '1'};
cqlsh CREATE TABLE ks1.t1 (c1 int PRIMARY KEY, c2 int );
cqlsh CREATE TABLE ks1.t2 (c1 int PRIMARY KEY, c2 int );
cqlsh CREATE TABLE ks2.t1 (c1 int PRIMARY KEY, c2 int );
// First index okay
cqlsh create index t_idx on ks1.t1 (c2) ;
// Duplicate index on different table rejected - ok
cqlsh create index t_idx on ks1.t2 (c2) ;
ErrorMessage code=2300 [Query invalid because of configuration issue]
message=Duplicate index name t_idx
// Duplicate index on a different table in another keyspace rejected - ok
cqlsh create index t_idx on ks2.t2 (c2) ;
ErrorMessage code=2300 [Query invalid because of configuration issue]
message=Duplicate index name t_idx
// Duplicate index name in another keyspace works - wrong?
cqlsh create index t_idx on ks2.t1 (c2) ;
{noformat}
Describing keyspaces ks1 and ks2 shows that the table t1 in each has an index
named t_idx, so two indexes were created. Also dropping one of the indexes does
not affect the other.
{{CFMetaData.validate()}} calls {{existingIndexNames(cfName)}} to get all the
known index names, but filters out indexes for the current table.
However, using only the {{cfName}}, this masks indexes that may exist on tables
of the same name in another keyspace. The keyspace is needed to properly filter
index names for the table.
Issue CASSANDRA-7314 is related as it adds an optional {{keyspace}} argument to
{{DROP INDEX}}.
So it seems that {{CREATE INDEX}} should either:
- not scope index names globally, but rather by keyspace, or
- continue to mamaging index names globally by rejecting the duplicate index
name created in the above scenario.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-6357) Flush memtables to separate directory
[ https://issues.apache.org/jira/browse/CASSANDRA-6357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dan jatnieks updated CASSANDRA-6357: Attachment: c6357-2.1-stress-write-adj-ops-sec.png c6357-2.1-stress-write-latency-median.png c6357-2.1-stress-write-latency-99th.png Flush memtables to separate directory - Key: CASSANDRA-6357 URL: https://issues.apache.org/jira/browse/CASSANDRA-6357 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Patrick McFadin Assignee: Jonathan Ellis Priority: Minor Labels: performance Fix For: 2.1 beta1 Attachments: 6357-v2.txt, 6357.txt, c6357-2.1-stress-write-adj-ops-sec.png, c6357-2.1-stress-write-latency-99th.png, c6357-2.1-stress-write-latency-median.png, c6357-stress-write-latency-99th-1.png Flush writers are a critical element for keeping a node healthy. When several compactions run on systems with low performing data directories, IO becomes a premium. Once the disk subsystem is saturated, write IO is blocked which will cause flush writer threads to backup. Since memtables are large blocks of memory in the JVM, too much blocking can cause excessive GC over time degrading performance. In the worst case causing an OOM. Since compaction is running on the data directories. My proposal is to create a separate directory for flushing memtables. Potentially we can use the same methodology of keeping the commit log separate and minimize disk contention against the critical function of the flushwriter. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6357) Flush memtables to separate directory
[
https://issues.apache.org/jira/browse/CASSANDRA-6357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13945219#comment-13945219
]
dan jatnieks commented on CASSANDRA-6357:
-
Retested this using trunk (as of Mar 13). I switched to different hardware with
7200rpm disks because the slow 5400rpm disks on the other system just couldn't
keep up without throttling stress op/s.
The machine this time was a Dell with two quad core hyper-threaded Intel Xeon
E5620 CPU, 32Gb memory, and 8 disks (500Gb, 7200 rpm).
The two scenarios were the same as last time and used the same stress
parameters.
The results were much less dramatic than the 2.0 test.
The base test results (data and flush on the same device) :
{noformat}
real op rate : 9433
adjusted op rate : 9435
adjusted op rate stderr : 0
key rate : 9433
latency mean : 5.3
latency median: 1.4
latency 95th percentile : 5.4
latency 99th percentile : 12.9
latency 99.9th percentile : 259.4
latency max : 20873.9
Total operation time : 00:17:40
{noformat}
The flush test results (data and flush on separate devices):
{noformat}
real op rate : 10391
adjusted op rate : 10391
adjusted op rate stderr : 0
key rate : 10391
latency mean : 4.8
latency median: 1.4
latency 95th percentile : 5.4
latency 99th percentile : 14.2
latency 99.9th percentile : 245.0
latency max : 17035.2
Total operation time : 00:16:02
{noformat}
See attached graphs:
[2.1 Stress Write Latency 99.9th
Percentile|^c6357-2.1-stress-write-latency-99th.png]
[2.1 Stress Write Median Latency|^c6357-2.1-stress-write-latency-median.png]
[2.1 Stress Write Adjusted Ops/sec|^c6357-2.1-stress-write-adj-ops-sec.png]
Flush memtables to separate directory
-
Key: CASSANDRA-6357
URL: https://issues.apache.org/jira/browse/CASSANDRA-6357
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Patrick McFadin
Assignee: Jonathan Ellis
Priority: Minor
Labels: performance
Fix For: 2.1 beta1
Attachments: 6357-v2.txt, 6357.txt,
c6357-2.1-stress-write-adj-ops-sec.png,
c6357-2.1-stress-write-latency-99th.png,
c6357-2.1-stress-write-latency-median.png,
c6357-stress-write-latency-99th-1.png
Flush writers are a critical element for keeping a node healthy. When several
compactions run on systems with low performing data directories, IO becomes a
premium. Once the disk subsystem is saturated, write IO is blocked which will
cause flush writer threads to backup. Since memtables are large blocks of
memory in the JVM, too much blocking can cause excessive GC over time
degrading performance. In the worst case causing an OOM.
Since compaction is running on the data directories. My proposal is to create
a separate directory for flushing memtables. Potentially we can use the same
methodology of keeping the commit log separate and minimize disk contention
against the critical function of the flushwriter.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6357) Flush memtables to separate directory
[ https://issues.apache.org/jira/browse/CASSANDRA-6357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13945705#comment-13945705 ] dan jatnieks commented on CASSANDRA-6357: - I can also go back and re-run 2.0 w/patch on the same machine used for 2.1 ... but that patch isn't quite the same as the 2.1 code either... Flush memtables to separate directory - Key: CASSANDRA-6357 URL: https://issues.apache.org/jira/browse/CASSANDRA-6357 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Patrick McFadin Assignee: Jonathan Ellis Priority: Minor Labels: performance Fix For: 2.1 beta1 Attachments: 6357-v2.txt, 6357.txt, c6357-2.1-stress-write-adj-ops-sec.png, c6357-2.1-stress-write-latency-99th.png, c6357-2.1-stress-write-latency-median.png, c6357-stress-write-latency-99th-1.png Flush writers are a critical element for keeping a node healthy. When several compactions run on systems with low performing data directories, IO becomes a premium. Once the disk subsystem is saturated, write IO is blocked which will cause flush writer threads to backup. Since memtables are large blocks of memory in the JVM, too much blocking can cause excessive GC over time degrading performance. In the worst case causing an OOM. Since compaction is running on the data directories. My proposal is to create a separate directory for flushing memtables. Potentially we can use the same methodology of keeping the commit log separate and minimize disk contention against the critical function of the flushwriter. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-6357) Flush memtables to separate directory
[ https://issues.apache.org/jira/browse/CASSANDRA-6357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13945705#comment-13945705 ] dan jatnieks edited comment on CASSANDRA-6357 at 3/24/14 9:19 PM: -- I can also go back and re-run 2.0 w/patch on the same machine used for 2.1 ... but that patch isn't quite the same as the 2.1 code either... I was wondering if changes to flushing and/or compaction in 2.1 already lessen the contention that was present in 2.0? was (Author: djatnieks): I can also go back and re-run 2.0 w/patch on the same machine used for 2.1 ... but that patch isn't quite the same as the 2.1 code either... Flush memtables to separate directory - Key: CASSANDRA-6357 URL: https://issues.apache.org/jira/browse/CASSANDRA-6357 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Patrick McFadin Assignee: Jonathan Ellis Priority: Minor Labels: performance Fix For: 2.1 beta1 Attachments: 6357-v2.txt, 6357.txt, c6357-2.1-stress-write-adj-ops-sec.png, c6357-2.1-stress-write-latency-99th.png, c6357-2.1-stress-write-latency-median.png, c6357-stress-write-latency-99th-1.png Flush writers are a critical element for keeping a node healthy. When several compactions run on systems with low performing data directories, IO becomes a premium. Once the disk subsystem is saturated, write IO is blocked which will cause flush writer threads to backup. Since memtables are large blocks of memory in the JVM, too much blocking can cause excessive GC over time degrading performance. In the worst case causing an OOM. Since compaction is running on the data directories. My proposal is to create a separate directory for flushing memtables. Potentially we can use the same methodology of keeping the commit log separate and minimize disk contention against the critical function of the flushwriter. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6823) TimedOutException/dropped mutations running stress on 2.1
[
https://issues.apache.org/jira/browse/CASSANDRA-6823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13929983#comment-13929983
]
dan jatnieks commented on CASSANDRA-6823:
-
Yes, it could be flushing or compaction maybe; anyway something that seems to
be to be maxing out the IO capabilities of this blade server with slow drives.
So probably not a C* or stress issue then; but I used the same blade servers
not long ago without problems. I ran this same scenario in Dec using C* 2.0.3
and stress was able to complete without errors - although there were frequent
periods of time where nothing was being loaded, e.g.
{noformat}
9920313,0,0,1.1,6.1,4860.3,3984
9920313,0,0,1.1,6.1,4860.3,3994
9920313,0,0,1.1,6.1,4860.3,4004
9920313,0,0,1.1,6.1,4860.3,4014
9924219,390,390,1.1,5.9,4860.3,4024
9941460,1724,1724,1.1,6.7,4860.3,4035
1000,5854,5854,0.9,5.0,4860.3,4042
{noformat}
I wanted to re-run the same scenario again with C* 2.1 and that's when I
started getting the TimedOutException's described above.
BTW, I did try the new (C* 2.1) stress against C* 2.0 and still got the
timeouts.
Then I went back to C* 2.0, now at 2.0.6, and even then am getting timeouts,
e.g.
{noformat}
6978866,0,0,1.1,7.3,21019.4,1193
6978866,0,0,1.1,7.3,21019.4,1203
6978866,0,0,1.1,7.3,21019.4,1213
6978866,0,0,1.1,7.3,21019.4,1223
6978866,0,0,1.1,7.3,21019.4,1234
Operation [6978905] retried 100 times - error inserting key 06978905
((TimedOutException))
Operation [6978883] retried 100 times - error inserting key 06978883
((TimedOutException))
{noformat}
Maybe something's changed between 2.0.3 and 2.0.6/2.1 to improve the throughput
of C* and therefore exceeding the disk I/O? (watching with iostat confirms the
disk 90-100% a lot of the time)
Comparing the earlier 2.0.3 stress results with current 2.0.6 shows almost 1.5x
more total keys after 5 minutes with 2.0.6. I'm not aware of anything
significant has changed on the blade server to account for this.
2.0.3 (Dec '13)
{noformat}
total,interval_op_rate,interval_key_rate,latency,95th,99.9th,elapsed_time
101059,10105,10105,1.0,9.8,240.6,10
195848,9478,9478,1.1,8.1,156.3,20
303346,10749,10749,1.1,6.5,156.3,30
353340,4999,4999,1.1,5.1,156.2,40
391734,3839,3839,1.1,5.1,4165.0,50
503239,11150,11150,1.1,6.0,4164.7,60
603452,10021,10021,1.2,6.4,4164.7,70
603741,28,28,1.2,6.4,4164.7,80
631263,2752,2752,1.2,6.4,126.9,91
745765,11450,11450,1.1,6.6,3655.2,101
804784,5901,5901,1.0,6.7,3749.8,111
825932,2114,2114,1.0,6.7,3749.8,121
865002,3907,3907,1.0,7.4,3749.8,131
953287,8828,8828,1.0,7.2,175.3,141
1030450,7716,7716,1.0,7.2,175.0,151
1035041,459,459,1.0,7.6,10645.7,161
1035301,26,26,1.0,7.6,10645.7,171
1082020,4671,4671,1.1,8.6,10645.7,182
1203203,12118,12118,1.1,7.6,10645.7,192
1205520,231,231,1.1,7.6,10645.7,202
1231013,2549,2549,1.1,7.7,10645.7,212
1231013,0,0,1.1,7.7,10645.7,222
1231013,0,0,1.1,7.7,10645.7,232
1231013,0,0,1.1,7.7,10645.7,242
1231013,0,0,1.1,7.7,10645.7,252
1282460,5144,5144,1.1,6.7,49538.0,262
1346346,6388,6388,1.1,7.1,2228.0,273
1482054,13570,13570,1.1,5.4,310.0,283
1522362,4030,4030,1.1,5.5,780.0,293
1559749,3738,3738,1.1,5.8,776.4,303
{noformat}
2.0.6 (Mar '14)
{noformat}
total,interval_op_rate,interval_key_rate,latency,95th,99.9th,elapsed_time
81582,8158,8158,1.4,12.4,882.2,10
166315,8473,8473,1.2,9.0,125.6,20
286042,11972,11972,1.2,7.8,1827.1,30
370722,8468,8468,1.2,7.0,1827.5,40
434601,6387,6387,1.2,6.1,1860.1,50
501459,6685,6685,1.2,5.8,1860.1,60
584545,8308,8308,1.2,6.9,1860.1,70
692765,10822,10822,1.2,6.9,1287.8,80
805827,11306,11306,1.1,7.2,1287.8,91
880074,7424,7424,1.1,6.8,1260.0,101
965474,8540,8540,1.2,7.2,1500.5,111
1057880,9240,9240,1.2,6.6,1500.5,121
1137539,7965,7965,1.2,6.3,1472.5,131
1213965,7642,7642,1.2,6.1,1467.8,141
1288224,7425,7425,1.2,5.9,1467.8,151
1324108,3588,3588,1.2,6.0,4041.8,161
1422788,9868,9868,1.1,5.6,1467.8,171
1525673,10288,10288,1.1,5.4,1467.8,182
1592155,6648,6648,1.1,5.9,1467.9,192
1653758,6160,6160,1.2,6.1,1467.8,202
1788367,13460,13460,1.1,5.9,1467.8,212
1829188,4082,4082,1.1,5.7,159.5,222
1924749,9556,9556,1.1,4.9,159.5,232
1991759,6701,6701,1.1,5.3,202.2,242
2057482,6572,6572,1.1,5.2,202.2,252
2190652,13317,13317,1.1,5.9,202.2,263
2234147,4349,4349,1.1,6.0,4639.1,273
2312015,7786,7786,1.1,5.9,4639.1,283
2393938,8192,8192,1.1,5.5,4639.1,293
2454516,6057,6057,1.1,5.5,4639.1,303
{noformat}
And with 2.1 there is an even greater op/s rate, at least initially (due to the
warm-up?), but I just don't think the blade server disks can keep up and it
drops off pretty fast.
One note about these 2.1 results is that data and flush directories have been
put on separate disks. In fact, on this server, I'm not seeing a significant
difference in the stress results when data/flush are on the same or different
devices. But that is a different ticket (CASSANDRA-6357).
{noformat}
ops ,op/s,adj op/s, key/s,mean, med,
[jira] [Comment Edited] (CASSANDRA-6823) TimedOutException/dropped mutations running stress on 2.1
[
https://issues.apache.org/jira/browse/CASSANDRA-6823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13929983#comment-13929983
]
dan jatnieks edited comment on CASSANDRA-6823 at 3/11/14 6:06 AM:
--
Yes, it could be flushing or compaction maybe; anyway something that seems to
be to be maxing out the IO capabilities of this blade server with slow drives.
So probably not a C* or stress issue then; but I used the same blade servers
not long ago without problems. I ran this same scenario in Dec using C* 2.0.3
and stress was able to complete without errors - although there were frequent
periods of time where nothing was being loaded, e.g.
{noformat}
9920313,0,0,1.1,6.1,4860.3,3984
9920313,0,0,1.1,6.1,4860.3,3994
9920313,0,0,1.1,6.1,4860.3,4004
9920313,0,0,1.1,6.1,4860.3,4014
9924219,390,390,1.1,5.9,4860.3,4024
9941460,1724,1724,1.1,6.7,4860.3,4035
1000,5854,5854,0.9,5.0,4860.3,4042
{noformat}
I wanted to re-run the same scenario again with C* 2.1 and that's when I
started getting the TimedOutException's described above.
BTW, I did try the new (C* 2.1) stress against C* 2.0 and still got the
timeouts.
Then I went back to C* 2.0, now at 2.0.6, and even then am getting timeouts,
e.g.
{noformat}
6978866,0,0,1.1,7.3,21019.4,1193
6978866,0,0,1.1,7.3,21019.4,1203
6978866,0,0,1.1,7.3,21019.4,1213
6978866,0,0,1.1,7.3,21019.4,1223
6978866,0,0,1.1,7.3,21019.4,1234
Operation [6978905] retried 100 times - error inserting key 06978905
((TimedOutException))
Operation [6978883] retried 100 times - error inserting key 06978883
((TimedOutException))
{noformat}
Maybe something's changed between 2.0.3 and 2.0.6/2.1 to improve the throughput
of C* and therefore exceeding the disk I/O? (watching with iostat confirms the
disk 90-100% a lot of the time)
Comparing the earlier 2.0.3 stress results with current 2.0.6 shows almost 1.5x
more total keys after 5 minutes with 2.0.6. I'm not aware of anything
significant has changed on the blade server to account for this.
2.0.3 (Dec '13)
{noformat}
total,interval_op_rate,interval_key_rate,latency,95th,99.9th,elapsed_time
101059,10105,10105,1.0,9.8,240.6,10
195848,9478,9478,1.1,8.1,156.3,20
303346,10749,10749,1.1,6.5,156.3,30
353340,4999,4999,1.1,5.1,156.2,40
391734,3839,3839,1.1,5.1,4165.0,50
503239,11150,11150,1.1,6.0,4164.7,60
603452,10021,10021,1.2,6.4,4164.7,70
603741,28,28,1.2,6.4,4164.7,80
631263,2752,2752,1.2,6.4,126.9,91
745765,11450,11450,1.1,6.6,3655.2,101
804784,5901,5901,1.0,6.7,3749.8,111
825932,2114,2114,1.0,6.7,3749.8,121
865002,3907,3907,1.0,7.4,3749.8,131
953287,8828,8828,1.0,7.2,175.3,141
1030450,7716,7716,1.0,7.2,175.0,151
1035041,459,459,1.0,7.6,10645.7,161
1035301,26,26,1.0,7.6,10645.7,171
1082020,4671,4671,1.1,8.6,10645.7,182
1203203,12118,12118,1.1,7.6,10645.7,192
1205520,231,231,1.1,7.6,10645.7,202
1231013,2549,2549,1.1,7.7,10645.7,212
1231013,0,0,1.1,7.7,10645.7,222
1231013,0,0,1.1,7.7,10645.7,232
1231013,0,0,1.1,7.7,10645.7,242
1231013,0,0,1.1,7.7,10645.7,252
1282460,5144,5144,1.1,6.7,49538.0,262
1346346,6388,6388,1.1,7.1,2228.0,273
1482054,13570,13570,1.1,5.4,310.0,283
1522362,4030,4030,1.1,5.5,780.0,293
1559749,3738,3738,1.1,5.8,776.4,303
...
{noformat}
2.0.6 (Mar '14)
{noformat}
total,interval_op_rate,interval_key_rate,latency,95th,99.9th,elapsed_time
81582,8158,8158,1.4,12.4,882.2,10
166315,8473,8473,1.2,9.0,125.6,20
286042,11972,11972,1.2,7.8,1827.1,30
370722,8468,8468,1.2,7.0,1827.5,40
434601,6387,6387,1.2,6.1,1860.1,50
501459,6685,6685,1.2,5.8,1860.1,60
584545,8308,8308,1.2,6.9,1860.1,70
692765,10822,10822,1.2,6.9,1287.8,80
805827,11306,11306,1.1,7.2,1287.8,91
880074,7424,7424,1.1,6.8,1260.0,101
965474,8540,8540,1.2,7.2,1500.5,111
1057880,9240,9240,1.2,6.6,1500.5,121
1137539,7965,7965,1.2,6.3,1472.5,131
1213965,7642,7642,1.2,6.1,1467.8,141
1288224,7425,7425,1.2,5.9,1467.8,151
1324108,3588,3588,1.2,6.0,4041.8,161
1422788,9868,9868,1.1,5.6,1467.8,171
1525673,10288,10288,1.1,5.4,1467.8,182
1592155,6648,6648,1.1,5.9,1467.9,192
1653758,6160,6160,1.2,6.1,1467.8,202
1788367,13460,13460,1.1,5.9,1467.8,212
1829188,4082,4082,1.1,5.7,159.5,222
1924749,9556,9556,1.1,4.9,159.5,232
1991759,6701,6701,1.1,5.3,202.2,242
2057482,6572,6572,1.1,5.2,202.2,252
2190652,13317,13317,1.1,5.9,202.2,263
2234147,4349,4349,1.1,6.0,4639.1,273
2312015,7786,7786,1.1,5.9,4639.1,283
2393938,8192,8192,1.1,5.5,4639.1,293
2454516,6057,6057,1.1,5.5,4639.1,303
...
(eventually fails)
{noformat}
And with 2.1 there is an even greater op/s rate, at least initially (due to the
warm-up?), but I just don't think the blade server disks can keep up and it
drops off pretty fast.
One note about these 2.1 results is that data and flush directories have been
put on separate disks. In fact, on this server, I'm not seeing a significant
difference in the stress results when data/flush are on the same or different
devices. But that is a different ticket
[jira] [Commented] (CASSANDRA-6823) TimedOutException/dropped mutations running stress on 2.1
[
https://issues.apache.org/jira/browse/CASSANDRA-6823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13930507#comment-13930507
]
dan jatnieks commented on CASSANDRA-6823:
-
yup, thanks Benedict
TimedOutException/dropped mutations running stress on 2.1
--
Key: CASSANDRA-6823
URL: https://issues.apache.org/jira/browse/CASSANDRA-6823
Project: Cassandra
Issue Type: Bug
Reporter: dan jatnieks
Priority: Minor
Labels: stress
Attachments: stress.log, system.log
While testing CASSANDRA-6357, I am seeing TimedOutException errors running
stress on both 2.1 and trunk, and system log is showing dropped mutation
messages.
{noformat}
$ ant -Dversion=2.1.0-SNAPSHOT jar
$ ./bin/cassandra
$ ./cassandra-2.1/tools/bin/cassandra-stress write n=1000
Created keyspaces. Sleeping 1s for propagation.
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
Sleeping 2s...
Running WRITE with 50 threads for 1000 iterations
ops ,op/s,adj op/s, key/s,mean, med, .95, .99,
.999, max, time, stderr
74597 , 74590, 74590, 74590, 0.7, 0.3, 1.7, 7.8,
39.4, 156.0,1.0, 0.0
175807, 100469, 111362, 100469, 0.5, 0.3, 1.0, 2.2,
16.4, 105.2,2.0, 0.0
278037, 100483, 110412, 100483, 0.5, 0.4, 0.9, 2.2,
15.9,95.4,3.0, 0.13983
366806, 86301, 86301, 86301, 0.6, 0.4, 0.9, 2.4,
97.6, 107.0,4.1, 0.10002
473244, 105209, 115906, 105209, 0.5, 0.3, 1.0, 2.2,
10.2,99.6,5.1, 0.08246
574363, 99939, 112606, 99939, 0.5, 0.3, 1.0, 2.2,
8.4, 115.3,6.1, 0.07297
665162, 89343, 89343, 89343, 0.6, 0.3, 1.1, 2.3,
12.5, 116.4,7.1, 0.06256
768575, 102028, 102028, 102028, 0.5, 0.3, 1.0, 2.1,
10.7, 116.0,8.1, 0.05703
870318, 100383, 112278, 100383, 0.5, 0.4, 1.0, 2.1,
8.2, 109.1,9.1, 0.04984
972584, 100496, 111616, 100496, 0.5, 0.3, 1.0, 2.3,
10.3, 109.1, 10.1, 0.04542
1063466 , 88566, 88566, 88566, 0.6, 0.3, 1.1, 2.5,
107.3, 116.9, 11.2, 0.04152
1163218 , 98512, 107549, 98512, 0.5, 0.3, 1.2, 3.4,
17.9,92.9, 12.2, 0.04007
1257989 , 93578, 103808, 93578, 0.5, 0.3, 1.4, 3.8,
12.6, 105.6, 13.2, 0.03687
1349628 , 90205, 99257, 90205, 0.6, 0.3, 1.2, 2.9,
20.3,99.6, 14.2, 0.03401
1448125 , 97133, 106429, 97133, 0.5, 0.3, 1.2, 2.9,
11.9, 102.2, 15.2, 0.03170
1536662 , 87137, 95464, 87137, 0.6, 0.4, 1.1, 2.9,
83.7,94.0, 16.2, 0.02964
1632373 , 94446, 102735, 94446, 0.5, 0.4, 1.1, 2.6,
11.7,85.5, 17.2, 0.02818
1717028 , 83533, 83533, 83533, 0.6, 0.4, 1.1, 2.7,
87.4, 101.8, 18.3, 0.02651
1817081 , 97807, 108004, 97807, 0.5, 0.3, 1.1, 2.5,
14.5,99.1, 19.3, 0.02712
1904103 , 85634, 94846, 85634, 0.6, 0.3, 1.2, 3.0,
92.4, 105.3, 20.3, 0.02585
2001438 , 95991, 104822, 95991, 0.5, 0.3, 1.2, 2.7,
13.5,95.3, 21.3, 0.02482
2086571 , 89121, 99429, 89121, 0.6, 0.3, 1.2, 3.2,
30.9, 103.3, 22.3, 0.02367
2184096 , 88718, 97020, 88718, 0.6, 0.3, 1.3, 3.2,
85.6,98.0, 23.4, 0.02262
2276823 , 91795, 91795, 91795, 0.5, 0.3, 1.3, 3.5,
81.1, 102.1, 24.4, 0.02174
2381493 , 101074, 101074, 101074, 0.5, 0.3, 1.3, 3.3,
12.9,99.1, 25.4, 0.02123
2466415 , 83368, 92292, 83368, 0.6, 0.4, 1.2, 3.0,
14.3, 188.5, 26.4, 0.02037
2567406 , 100099, 109267, 100099, 0.5, 0.3, 1.4, 3.3,
10.9,94.2, 27.4, 0.01989
2653040 , 84476, 91922, 84476, 0.6, 0.3, 1.4, 3.2,
77.0, 100.3, 28.5, 0.01937
TimedOutException(acknowledged_by:0)
TimedOutException(acknowledged_by:0)
TimedOutException(acknowledged_by:0)
TimedOutException(acknowledged_by:0)
TimedOutException(acknowledged_by:0)
TimedOutException(acknowledged_by:0)
TimedOutException(acknowledged_by:0)
TimedOutException(acknowledged_by:0)
...
9825371 , 84636, 91716, 84636, 0.6, 0.3, 1.4, 4.5,
23.4,86.4, 125.7, 0.00894
9915317 , 87803,
[jira] [Created] (CASSANDRA-6823) TimedOutException/dropped mutations running stress on 2.1
dan jatnieks created CASSANDRA-6823:
---
Summary: TimedOutException/dropped mutations running stress on 2.1
Key: CASSANDRA-6823
URL: https://issues.apache.org/jira/browse/CASSANDRA-6823
Project: Cassandra
Issue Type: Bug
Reporter: dan jatnieks
Priority: Minor
Attachments: stress.log, system.log
While testing CASSANDRA-6357, I am seeing TimedOutException errors running
stress on both 2.1 and trunk, and system log is showing dropped mutation
messages.
{noformat}
$ ant -Dversion=2.1.0-SNAPSHOT jar
$ ./bin/cassandra
$ ./cassandra-2.1/tools/bin/cassandra-stress write n=1000
Created keyspaces. Sleeping 1s for propagation.
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
Sleeping 2s...
Running WRITE with 50 threads for 1000 iterations
ops ,op/s,adj op/s, key/s,mean, med, .95, .99,
.999, max, time, stderr
74597 , 74590, 74590, 74590, 0.7, 0.3, 1.7, 7.8,
39.4, 156.0,1.0, 0.0
175807, 100469, 111362, 100469, 0.5, 0.3, 1.0, 2.2,
16.4, 105.2,2.0, 0.0
278037, 100483, 110412, 100483, 0.5, 0.4, 0.9, 2.2,
15.9,95.4,3.0, 0.13983
366806, 86301, 86301, 86301, 0.6, 0.4, 0.9, 2.4,
97.6, 107.0,4.1, 0.10002
473244, 105209, 115906, 105209, 0.5, 0.3, 1.0, 2.2,
10.2,99.6,5.1, 0.08246
574363, 99939, 112606, 99939, 0.5, 0.3, 1.0, 2.2,
8.4, 115.3,6.1, 0.07297
665162, 89343, 89343, 89343, 0.6, 0.3, 1.1, 2.3,
12.5, 116.4,7.1, 0.06256
768575, 102028, 102028, 102028, 0.5, 0.3, 1.0, 2.1,
10.7, 116.0,8.1, 0.05703
870318, 100383, 112278, 100383, 0.5, 0.4, 1.0, 2.1,
8.2, 109.1,9.1, 0.04984
972584, 100496, 111616, 100496, 0.5, 0.3, 1.0, 2.3,
10.3, 109.1, 10.1, 0.04542
1063466 , 88566, 88566, 88566, 0.6, 0.3, 1.1, 2.5,
107.3, 116.9, 11.2, 0.04152
1163218 , 98512, 107549, 98512, 0.5, 0.3, 1.2, 3.4,
17.9,92.9, 12.2, 0.04007
1257989 , 93578, 103808, 93578, 0.5, 0.3, 1.4, 3.8,
12.6, 105.6, 13.2, 0.03687
1349628 , 90205, 99257, 90205, 0.6, 0.3, 1.2, 2.9,
20.3,99.6, 14.2, 0.03401
1448125 , 97133, 106429, 97133, 0.5, 0.3, 1.2, 2.9,
11.9, 102.2, 15.2, 0.03170
1536662 , 87137, 95464, 87137, 0.6, 0.4, 1.1, 2.9,
83.7,94.0, 16.2, 0.02964
1632373 , 94446, 102735, 94446, 0.5, 0.4, 1.1, 2.6,
11.7,85.5, 17.2, 0.02818
1717028 , 83533, 83533, 83533, 0.6, 0.4, 1.1, 2.7,
87.4, 101.8, 18.3, 0.02651
1817081 , 97807, 108004, 97807, 0.5, 0.3, 1.1, 2.5,
14.5,99.1, 19.3, 0.02712
1904103 , 85634, 94846, 85634, 0.6, 0.3, 1.2, 3.0,
92.4, 105.3, 20.3, 0.02585
2001438 , 95991, 104822, 95991, 0.5, 0.3, 1.2, 2.7,
13.5,95.3, 21.3, 0.02482
2086571 , 89121, 99429, 89121, 0.6, 0.3, 1.2, 3.2,
30.9, 103.3, 22.3, 0.02367
2184096 , 88718, 97020, 88718, 0.6, 0.3, 1.3, 3.2,
85.6,98.0, 23.4, 0.02262
2276823 , 91795, 91795, 91795, 0.5, 0.3, 1.3, 3.5,
81.1, 102.1, 24.4, 0.02174
2381493 , 101074, 101074, 101074, 0.5, 0.3, 1.3, 3.3,
12.9,99.1, 25.4, 0.02123
2466415 , 83368, 92292, 83368, 0.6, 0.4, 1.2, 3.0,
14.3, 188.5, 26.4, 0.02037
2567406 , 100099, 109267, 100099, 0.5, 0.3, 1.4, 3.3,
10.9,94.2, 27.4, 0.01989
2653040 , 84476, 91922, 84476, 0.6, 0.3, 1.4, 3.2,
77.0, 100.3, 28.5, 0.01937
TimedOutException(acknowledged_by:0)
TimedOutException(acknowledged_by:0)
TimedOutException(acknowledged_by:0)
TimedOutException(acknowledged_by:0)
TimedOutException(acknowledged_by:0)
TimedOutException(acknowledged_by:0)
TimedOutException(acknowledged_by:0)
TimedOutException(acknowledged_by:0)
...
9825371 , 84636, 91716, 84636, 0.6, 0.3, 1.4, 4.5,
23.4,86.4, 125.7, 0.00894
9915317 , 87803, 93938, 87803, 0.6, 0.3, 1.3, 4.2,
62.4,87.2, 126.7, 0.00888
1000 , 93564, 101405, 93564, 0.5, 0.3, 1.4, 4.2,
16.2,83.1, 127.6, 0.00880
Results:
real op rate : 78378
adjusted op rate : 78378
adjusted op rate stderr : 0
key rate : 78378
latency mean
[jira] [Updated] (CASSANDRA-6823) TimedOutException/dropped mutations running stress on 2.1
[
https://issues.apache.org/jira/browse/CASSANDRA-6823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks updated CASSANDRA-6823:
Attachment: system.log
stress.log
TimedOutException/dropped mutations running stress on 2.1
--
Key: CASSANDRA-6823
URL: https://issues.apache.org/jira/browse/CASSANDRA-6823
Project: Cassandra
Issue Type: Bug
Reporter: dan jatnieks
Priority: Minor
Labels: stress
Attachments: stress.log, system.log
While testing CASSANDRA-6357, I am seeing TimedOutException errors running
stress on both 2.1 and trunk, and system log is showing dropped mutation
messages.
{noformat}
$ ant -Dversion=2.1.0-SNAPSHOT jar
$ ./bin/cassandra
$ ./cassandra-2.1/tools/bin/cassandra-stress write n=1000
Created keyspaces. Sleeping 1s for propagation.
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
Sleeping 2s...
Running WRITE with 50 threads for 1000 iterations
ops ,op/s,adj op/s, key/s,mean, med, .95, .99,
.999, max, time, stderr
74597 , 74590, 74590, 74590, 0.7, 0.3, 1.7, 7.8,
39.4, 156.0,1.0, 0.0
175807, 100469, 111362, 100469, 0.5, 0.3, 1.0, 2.2,
16.4, 105.2,2.0, 0.0
278037, 100483, 110412, 100483, 0.5, 0.4, 0.9, 2.2,
15.9,95.4,3.0, 0.13983
366806, 86301, 86301, 86301, 0.6, 0.4, 0.9, 2.4,
97.6, 107.0,4.1, 0.10002
473244, 105209, 115906, 105209, 0.5, 0.3, 1.0, 2.2,
10.2,99.6,5.1, 0.08246
574363, 99939, 112606, 99939, 0.5, 0.3, 1.0, 2.2,
8.4, 115.3,6.1, 0.07297
665162, 89343, 89343, 89343, 0.6, 0.3, 1.1, 2.3,
12.5, 116.4,7.1, 0.06256
768575, 102028, 102028, 102028, 0.5, 0.3, 1.0, 2.1,
10.7, 116.0,8.1, 0.05703
870318, 100383, 112278, 100383, 0.5, 0.4, 1.0, 2.1,
8.2, 109.1,9.1, 0.04984
972584, 100496, 111616, 100496, 0.5, 0.3, 1.0, 2.3,
10.3, 109.1, 10.1, 0.04542
1063466 , 88566, 88566, 88566, 0.6, 0.3, 1.1, 2.5,
107.3, 116.9, 11.2, 0.04152
1163218 , 98512, 107549, 98512, 0.5, 0.3, 1.2, 3.4,
17.9,92.9, 12.2, 0.04007
1257989 , 93578, 103808, 93578, 0.5, 0.3, 1.4, 3.8,
12.6, 105.6, 13.2, 0.03687
1349628 , 90205, 99257, 90205, 0.6, 0.3, 1.2, 2.9,
20.3,99.6, 14.2, 0.03401
1448125 , 97133, 106429, 97133, 0.5, 0.3, 1.2, 2.9,
11.9, 102.2, 15.2, 0.03170
1536662 , 87137, 95464, 87137, 0.6, 0.4, 1.1, 2.9,
83.7,94.0, 16.2, 0.02964
1632373 , 94446, 102735, 94446, 0.5, 0.4, 1.1, 2.6,
11.7,85.5, 17.2, 0.02818
1717028 , 83533, 83533, 83533, 0.6, 0.4, 1.1, 2.7,
87.4, 101.8, 18.3, 0.02651
1817081 , 97807, 108004, 97807, 0.5, 0.3, 1.1, 2.5,
14.5,99.1, 19.3, 0.02712
1904103 , 85634, 94846, 85634, 0.6, 0.3, 1.2, 3.0,
92.4, 105.3, 20.3, 0.02585
2001438 , 95991, 104822, 95991, 0.5, 0.3, 1.2, 2.7,
13.5,95.3, 21.3, 0.02482
2086571 , 89121, 99429, 89121, 0.6, 0.3, 1.2, 3.2,
30.9, 103.3, 22.3, 0.02367
2184096 , 88718, 97020, 88718, 0.6, 0.3, 1.3, 3.2,
85.6,98.0, 23.4, 0.02262
2276823 , 91795, 91795, 91795, 0.5, 0.3, 1.3, 3.5,
81.1, 102.1, 24.4, 0.02174
2381493 , 101074, 101074, 101074, 0.5, 0.3, 1.3, 3.3,
12.9,99.1, 25.4, 0.02123
2466415 , 83368, 92292, 83368, 0.6, 0.4, 1.2, 3.0,
14.3, 188.5, 26.4, 0.02037
2567406 , 100099, 109267, 100099, 0.5, 0.3, 1.4, 3.3,
10.9,94.2, 27.4, 0.01989
2653040 , 84476, 91922, 84476, 0.6, 0.3, 1.4, 3.2,
77.0, 100.3, 28.5, 0.01937
TimedOutException(acknowledged_by:0)
TimedOutException(acknowledged_by:0)
TimedOutException(acknowledged_by:0)
TimedOutException(acknowledged_by:0)
TimedOutException(acknowledged_by:0)
TimedOutException(acknowledged_by:0)
TimedOutException(acknowledged_by:0)
TimedOutException(acknowledged_by:0)
...
9825371 , 84636, 91716, 84636, 0.6, 0.3, 1.4, 4.5,
23.4,86.4, 125.7, 0.00894
9915317 , 87803, 93938, 87803,
[jira] [Comment Edited] (CASSANDRA-6777) 2.1 w/java-driver 2.0 and stress write using thrift results in ArithmeticException / by zero errors
[
https://issues.apache.org/jira/browse/CASSANDRA-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13915193#comment-13915193
]
dan jatnieks edited comment on CASSANDRA-6777 at 2/27/14 11:24 PM:
---
Attaching patch that should work with released version of java-driver 2.0.
Note: this also fixes a minor issue with using the {{\-schema ks=}} option.
was (Author: djatnieks):
Attaching patch that should work with released version of java-driver 2.0.
2.1 w/java-driver 2.0 and stress write using thrift results in
ArithmeticException / by zero errors
---
Key: CASSANDRA-6777
URL: https://issues.apache.org/jira/browse/CASSANDRA-6777
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: Mac OSX, java 1.7.0_51
Reporter: dan jatnieks
Priority: Minor
Labels: stress
Attachments: Quote_keyspace_name.patch, logs.tar.gz
Running stress write (thrift) on 2.1 branch is resulting in the following.
Note: this is after working around
[JAVA-276|https://datastax-oss.atlassian.net/browse/JAVA-276] causing stress
to fail to connect to 2.1.
{noformat}
$ ./tools/bin/cassandra-stress write n=500
Unable to create stress keyspace: Keyspace names must be case-insensitively
unique (Keyspace1 conflicts with Keyspace1)
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
...
java.lang.Arithmjava.io.IOException: Operation [220] x10 key DD Error
executing: (ArithmeticException): / by zero
eticException: / by zero
java.lang.ArithmeticException: / by zero
at org.apache.cassandra.stress.Operation.error(Operation.java:237)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:216)
at
org.apache.cassandra.stress.operations.ThriftInserter.run(ThriftInserter.java:72)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:302)java.lang.ArithmeticException:
/ by zero
java.lang.ArithmeticException: / by zero
java.io.IOException: Operation [200] x10 key C9 Error executing:
(ArithmeticException): / by zero
at org.apache.cassandra.stress.Operation.error(Operation.java:237)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:216)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.operations.ThriftInserter.run(ThriftInserter.java:72)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:302)
...
{noformat}
Seems to be just a thrift issue, as running stress write using the native
protocol works:
{noformat}
$ ./tools/bin/cassandra-stress write n=500 -mode native cql3
Unable to create stress keyspace: Keyspace names must be case-insensitively
unique (Keyspace1 conflicts with Keyspace1)
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
Sleeping 2s...
Running WRITE with 50 threads for 500 iterations
ops ,op/s,adj op/s, key/s,mean, med, .95, .99,
.999, max, time, stderr
29342 , 29340, 30903, 29340, 1.7, 1.4, 3.1, 5.7,
51.5,54.4,1.0, 0.0
56353 , 26968, 28523, 26968, 1.8, 1.5, 3.6, 7.0,
56.5,57.2,2.0, 0.0
...
500 , 29358, 29358, 29358, 1.7, 1.3, 3.6, 8.4,
10.1,11.6, 168.8, 0.00828
Results:
real op rate : 29620
adjusted op rate : 29629
adjusted op rate stderr : 0
key rate : 29620
latency mean : 1.7
latency median: 1.4
latency 95th percentile : 3.0
latency 99th percentile : 5.4
latency 99.9th percentile : 56.5
latency max : 305.7
Total operation time : 00:02:48
END
{noformat}
Attaching stress write and system logs as well.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6777) 2.1 w/java-driver 2.0 and stress write using thrift results in ArithmeticException / by zero errors
[
https://issues.apache.org/jira/browse/CASSANDRA-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks updated CASSANDRA-6777:
Attachment: Quote_keyspace_name.patch
Attaching patch that should work with released version of java-driver 2.0.
2.1 w/java-driver 2.0 and stress write using thrift results in
ArithmeticException / by zero errors
---
Key: CASSANDRA-6777
URL: https://issues.apache.org/jira/browse/CASSANDRA-6777
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: Mac OSX, java 1.7.0_51
Reporter: dan jatnieks
Priority: Minor
Labels: stress
Attachments: Quote_keyspace_name.patch, logs.tar.gz
Running stress write (thrift) on 2.1 branch is resulting in the following.
Note: this is after working around
[JAVA-276|https://datastax-oss.atlassian.net/browse/JAVA-276] causing stress
to fail to connect to 2.1.
{noformat}
$ ./tools/bin/cassandra-stress write n=500
Unable to create stress keyspace: Keyspace names must be case-insensitively
unique (Keyspace1 conflicts with Keyspace1)
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
...
java.lang.Arithmjava.io.IOException: Operation [220] x10 key DD Error
executing: (ArithmeticException): / by zero
eticException: / by zero
java.lang.ArithmeticException: / by zero
at org.apache.cassandra.stress.Operation.error(Operation.java:237)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:216)
at
org.apache.cassandra.stress.operations.ThriftInserter.run(ThriftInserter.java:72)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:302)java.lang.ArithmeticException:
/ by zero
java.lang.ArithmeticException: / by zero
java.io.IOException: Operation [200] x10 key C9 Error executing:
(ArithmeticException): / by zero
at org.apache.cassandra.stress.Operation.error(Operation.java:237)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:216)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.operations.ThriftInserter.run(ThriftInserter.java:72)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:302)
...
{noformat}
Seems to be just a thrift issue, as running stress write using the native
protocol works:
{noformat}
$ ./tools/bin/cassandra-stress write n=500 -mode native cql3
Unable to create stress keyspace: Keyspace names must be case-insensitively
unique (Keyspace1 conflicts with Keyspace1)
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
Sleeping 2s...
Running WRITE with 50 threads for 500 iterations
ops ,op/s,adj op/s, key/s,mean, med, .95, .99,
.999, max, time, stderr
29342 , 29340, 30903, 29340, 1.7, 1.4, 3.1, 5.7,
51.5,54.4,1.0, 0.0
56353 , 26968, 28523, 26968, 1.8, 1.5, 3.6, 7.0,
56.5,57.2,2.0, 0.0
...
500 , 29358, 29358, 29358, 1.7, 1.3, 3.6, 8.4,
10.1,11.6, 168.8, 0.00828
Results:
real op rate : 29620
adjusted op rate : 29629
adjusted op rate stderr : 0
key rate : 29620
latency mean : 1.7
latency median: 1.4
latency 95th percentile : 3.0
latency 99th percentile : 5.4
latency 99.9th percentile : 56.5
latency max : 305.7
Total operation time : 00:02:48
END
{noformat}
Attaching stress write and system logs as well.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Created] (CASSANDRA-6777) stress write using thrift results in ArithmeticException / by zero errors
dan jatnieks created CASSANDRA-6777:
---
Summary: stress write using thrift results in ArithmeticException
/ by zero errors
Key: CASSANDRA-6777
URL: https://issues.apache.org/jira/browse/CASSANDRA-6777
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: Mac OSX, java 1.7.0_51
Reporter: dan jatnieks
Priority: Minor
Attachments: logs.tar.gz
Running stress write (thrift) on 2.1 branch is resulting in the following.
Note: this is after working around
[JAVA-276|https://datastax-oss.atlassian.net/browse/JAVA-276] causing stress to
fail to connect to 2.1.
{noformat}
$ ./tools/bin/cassandra-stress write n=500
Unable to create stress keyspace: Keyspace names must be case-insensitively
unique (Keyspace1 conflicts with Keyspace1)
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
...
java.lang.Arithmjava.io.IOException: Operation [220] x10 key DD Error
executing: (ArithmeticException): / by zero
eticException: / by zero
java.lang.ArithmeticException: / by zero
at org.apache.cassandra.stress.Operation.error(Operation.java:237)
java.lang.ArithmeticException: / by zeroat
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:216)
at
org.apache.cassandra.stress.operations.ThriftInserter.run(ThriftInserter.java:72)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:302)java.lang.ArithmeticException:
/ by zero
java.lang.ArithmeticException: / by zero
java.io.IOException: Operation [200] x10 key C9 Error executing:
(ArithmeticException): / by zero
at org.apache.cassandra.stress.Operation.error(Operation.java:237)
java.lang.ArithmeticException: / by zeroat
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:216)
java.lang.ArithmeticException: / by zeroat
org.apache.cassandra.stress.operations.ThriftInserter.run(ThriftInserter.java:72)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:302)
...
{noformat}
Seems to be just a thrift issue, as running stress write using the native
protocol works:
{noformat}
$ ./tools/bin/cassandra-stress write n=500 -mode native cql3
Unable to create stress keyspace: Keyspace names must be case-insensitively
unique (Keyspace1 conflicts with Keyspace1)
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
Sleeping 2s...
Running WRITE with 50 threads for 500 iterations
ops ,op/s,adj op/s, key/s,mean, med, .95, .99,
.999, max, time, stderr
29342 , 29340, 30903, 29340, 1.7, 1.4, 3.1, 5.7,
51.5,54.4,1.0, 0.0
56353 , 26968, 28523, 26968, 1.8, 1.5, 3.6, 7.0,
56.5,57.2,2.0, 0.0
...
500 , 29358, 29358, 29358, 1.7, 1.3, 3.6, 8.4,
10.1,11.6, 168.8, 0.00828
Results:
real op rate : 29620
adjusted op rate : 29629
adjusted op rate stderr : 0
key rate : 29620
latency mean : 1.7
latency median: 1.4
latency 95th percentile : 3.0
latency 99th percentile : 5.4
latency 99.9th percentile : 56.5
latency max : 305.7
Total operation time : 00:02:48
END
{noformat}
Attaching stress write and system logs as well.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Resolved] (CASSANDRA-6777) stress write using thrift results in ArithmeticException / by zero errors
[
https://issues.apache.org/jira/browse/CASSANDRA-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks resolved CASSANDRA-6777.
-
Resolution: Invalid
stress write using thrift results in ArithmeticException / by zero errors
-
Key: CASSANDRA-6777
URL: https://issues.apache.org/jira/browse/CASSANDRA-6777
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: Mac OSX, java 1.7.0_51
Reporter: dan jatnieks
Priority: Minor
Labels: stress
Attachments: logs.tar.gz
Running stress write (thrift) on 2.1 branch is resulting in the following.
Note: this is after working around
[JAVA-276|https://datastax-oss.atlassian.net/browse/JAVA-276] causing stress
to fail to connect to 2.1.
{noformat}
$ ./tools/bin/cassandra-stress write n=500
Unable to create stress keyspace: Keyspace names must be case-insensitively
unique (Keyspace1 conflicts with Keyspace1)
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
...
java.lang.Arithmjava.io.IOException: Operation [220] x10 key DD Error
executing: (ArithmeticException): / by zero
eticException: / by zero
java.lang.ArithmeticException: / by zero
at org.apache.cassandra.stress.Operation.error(Operation.java:237)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:216)
at
org.apache.cassandra.stress.operations.ThriftInserter.run(ThriftInserter.java:72)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:302)java.lang.ArithmeticException:
/ by zero
java.lang.ArithmeticException: / by zero
java.io.IOException: Operation [200] x10 key C9 Error executing:
(ArithmeticException): / by zero
at org.apache.cassandra.stress.Operation.error(Operation.java:237)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:216)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.operations.ThriftInserter.run(ThriftInserter.java:72)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:302)
...
{noformat}
Seems to be just a thrift issue, as running stress write using the native
protocol works:
{noformat}
$ ./tools/bin/cassandra-stress write n=500 -mode native cql3
Unable to create stress keyspace: Keyspace names must be case-insensitively
unique (Keyspace1 conflicts with Keyspace1)
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
Sleeping 2s...
Running WRITE with 50 threads for 500 iterations
ops ,op/s,adj op/s, key/s,mean, med, .95, .99,
.999, max, time, stderr
29342 , 29340, 30903, 29340, 1.7, 1.4, 3.1, 5.7,
51.5,54.4,1.0, 0.0
56353 , 26968, 28523, 26968, 1.8, 1.5, 3.6, 7.0,
56.5,57.2,2.0, 0.0
...
500 , 29358, 29358, 29358, 1.7, 1.3, 3.6, 8.4,
10.1,11.6, 168.8, 0.00828
Results:
real op rate : 29620
adjusted op rate : 29629
adjusted op rate stderr : 0
key rate : 29620
latency mean : 1.7
latency median: 1.4
latency 95th percentile : 3.0
latency 99th percentile : 5.4
latency 99.9th percentile : 56.5
latency max : 305.7
Total operation time : 00:02:48
END
{noformat}
Attaching stress write and system logs as well.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6777) stress write using thrift results in ArithmeticException / by zero errors
[
https://issues.apache.org/jira/browse/CASSANDRA-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13914083#comment-13914083
]
dan jatnieks commented on CASSANDRA-6777:
-
Ah. To get around JAVA-276, I had dropped in a modified java driver that could
parse the Cassandra version 2.1.0-beta1-SNAPSHOT. However doing that on the
head of the java driver 2.0 branch is somehow causing these ArithmeticException
/ by zero errors.
Restoring the {{cassandra-driver-core-2.0.0-rc3-SNAPSHOT.jar}} in tools/lib and
instead building C* with {{ant -Dbase.version=2.1.0}} works around JAVA-276 and
doesn't cause any problems for stress.
So closing this issue as it seems to be caused by java-driver.
stress write using thrift results in ArithmeticException / by zero errors
-
Key: CASSANDRA-6777
URL: https://issues.apache.org/jira/browse/CASSANDRA-6777
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: Mac OSX, java 1.7.0_51
Reporter: dan jatnieks
Priority: Minor
Labels: stress
Attachments: logs.tar.gz
Running stress write (thrift) on 2.1 branch is resulting in the following.
Note: this is after working around
[JAVA-276|https://datastax-oss.atlassian.net/browse/JAVA-276] causing stress
to fail to connect to 2.1.
{noformat}
$ ./tools/bin/cassandra-stress write n=500
Unable to create stress keyspace: Keyspace names must be case-insensitively
unique (Keyspace1 conflicts with Keyspace1)
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
...
java.lang.Arithmjava.io.IOException: Operation [220] x10 key DD Error
executing: (ArithmeticException): / by zero
eticException: / by zero
java.lang.ArithmeticException: / by zero
at org.apache.cassandra.stress.Operation.error(Operation.java:237)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:216)
at
org.apache.cassandra.stress.operations.ThriftInserter.run(ThriftInserter.java:72)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:302)java.lang.ArithmeticException:
/ by zero
java.lang.ArithmeticException: / by zero
java.io.IOException: Operation [200] x10 key C9 Error executing:
(ArithmeticException): / by zero
at org.apache.cassandra.stress.Operation.error(Operation.java:237)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:216)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.operations.ThriftInserter.run(ThriftInserter.java:72)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:302)
...
{noformat}
Seems to be just a thrift issue, as running stress write using the native
protocol works:
{noformat}
$ ./tools/bin/cassandra-stress write n=500 -mode native cql3
Unable to create stress keyspace: Keyspace names must be case-insensitively
unique (Keyspace1 conflicts with Keyspace1)
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
Sleeping 2s...
Running WRITE with 50 threads for 500 iterations
ops ,op/s,adj op/s, key/s,mean, med, .95, .99,
.999, max, time, stderr
29342 , 29340, 30903, 29340, 1.7, 1.4, 3.1, 5.7,
51.5,54.4,1.0, 0.0
56353 , 26968, 28523, 26968, 1.8, 1.5, 3.6, 7.0,
56.5,57.2,2.0, 0.0
...
500 , 29358, 29358, 29358, 1.7, 1.3, 3.6, 8.4,
10.1,11.6, 168.8, 0.00828
Results:
real op rate : 29620
adjusted op rate : 29629
adjusted op rate stderr : 0
key rate : 29620
latency mean : 1.7
latency median: 1.4
latency 95th percentile : 3.0
latency 99th percentile : 5.4
latency 99.9th percentile : 56.5
latency max : 305.7
Total operation time : 00:02:48
END
{noformat}
Attaching stress write and system logs as well.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6777) stress write using thrift results in ArithmeticException / by zero errors
[
https://issues.apache.org/jira/browse/CASSANDRA-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13914117#comment-13914117
]
dan jatnieks commented on CASSANDRA-6777:
-
A recent change to java-driver (revision f6a179d) caused this.
In stress, {{SmartThriftClient#get()}} gets a set of hosts by calling
{{metadata.getReplicas(keyspace, pk)}}.
However, in java-driver, {{Metadata#getReplicas}} has changed and now calls
{{handleId(keyspace)}} which essentially forces values to lowercase unless they
are quoted.
This likely means the caller of {{Metadata#getReplicas}} is responsible to
first quote case-sensitive keyspace names, such as the default Keyspace1 used
by stress, e.g. {{metadata.getReplicas(metadata.quote(keyspace), pk)}}
stress write using thrift results in ArithmeticException / by zero errors
-
Key: CASSANDRA-6777
URL: https://issues.apache.org/jira/browse/CASSANDRA-6777
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: Mac OSX, java 1.7.0_51
Reporter: dan jatnieks
Priority: Minor
Labels: stress
Attachments: logs.tar.gz
Running stress write (thrift) on 2.1 branch is resulting in the following.
Note: this is after working around
[JAVA-276|https://datastax-oss.atlassian.net/browse/JAVA-276] causing stress
to fail to connect to 2.1.
{noformat}
$ ./tools/bin/cassandra-stress write n=500
Unable to create stress keyspace: Keyspace names must be case-insensitively
unique (Keyspace1 conflicts with Keyspace1)
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
...
java.lang.Arithmjava.io.IOException: Operation [220] x10 key DD Error
executing: (ArithmeticException): / by zero
eticException: / by zero
java.lang.ArithmeticException: / by zero
at org.apache.cassandra.stress.Operation.error(Operation.java:237)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:216)
at
org.apache.cassandra.stress.operations.ThriftInserter.run(ThriftInserter.java:72)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:302)java.lang.ArithmeticException:
/ by zero
java.lang.ArithmeticException: / by zero
java.io.IOException: Operation [200] x10 key C9 Error executing:
(ArithmeticException): / by zero
at org.apache.cassandra.stress.Operation.error(Operation.java:237)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:216)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.operations.ThriftInserter.run(ThriftInserter.java:72)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:302)
...
{noformat}
Seems to be just a thrift issue, as running stress write using the native
protocol works:
{noformat}
$ ./tools/bin/cassandra-stress write n=500 -mode native cql3
Unable to create stress keyspace: Keyspace names must be case-insensitively
unique (Keyspace1 conflicts with Keyspace1)
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
Sleeping 2s...
Running WRITE with 50 threads for 500 iterations
ops ,op/s,adj op/s, key/s,mean, med, .95, .99,
.999, max, time, stderr
29342 , 29340, 30903, 29340, 1.7, 1.4, 3.1, 5.7,
51.5,54.4,1.0, 0.0
56353 , 26968, 28523, 26968, 1.8, 1.5, 3.6, 7.0,
56.5,57.2,2.0, 0.0
...
500 , 29358, 29358, 29358, 1.7, 1.3, 3.6, 8.4,
10.1,11.6, 168.8, 0.00828
Results:
real op rate : 29620
adjusted op rate : 29629
adjusted op rate stderr : 0
key rate : 29620
latency mean : 1.7
latency median: 1.4
latency 95th percentile : 3.0
latency 99th percentile : 5.4
latency 99.9th percentile : 56.5
latency max : 305.7
Total operation time : 00:02:48
END
{noformat}
Attaching stress write and system logs as well.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Comment Edited] (CASSANDRA-6777) stress write using thrift results in ArithmeticException / by zero errors
[
https://issues.apache.org/jira/browse/CASSANDRA-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13914117#comment-13914117
]
dan jatnieks edited comment on CASSANDRA-6777 at 2/27/14 6:07 AM:
--
A recent change to java-driver (revision f6a179d, for JAVA-269) caused this.
In stress, {{SmartThriftClient#get()}} gets a set of hosts by calling
{{metadata.getReplicas(keyspace, pk)}}.
However, in java-driver, {{Metadata#getReplicas}} has changed and now calls
{{handleId(keyspace)}} which essentially forces values to lowercase unless they
are quoted.
This likely means the caller of {{Metadata#getReplicas}} is responsible to
first quote case-sensitive keyspace names, such as the default Keyspace1 used
by stress, e.g. {{metadata.getReplicas(metadata.quote(keyspace), pk)}}
was (Author: djatnieks):
A recent change to java-driver (revision f6a179d) caused this.
In stress, {{SmartThriftClient#get()}} gets a set of hosts by calling
{{metadata.getReplicas(keyspace, pk)}}.
However, in java-driver, {{Metadata#getReplicas}} has changed and now calls
{{handleId(keyspace)}} which essentially forces values to lowercase unless they
are quoted.
This likely means the caller of {{Metadata#getReplicas}} is responsible to
first quote case-sensitive keyspace names, such as the default Keyspace1 used
by stress, e.g. {{metadata.getReplicas(metadata.quote(keyspace), pk)}}
stress write using thrift results in ArithmeticException / by zero errors
-
Key: CASSANDRA-6777
URL: https://issues.apache.org/jira/browse/CASSANDRA-6777
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: Mac OSX, java 1.7.0_51
Reporter: dan jatnieks
Priority: Minor
Labels: stress
Attachments: logs.tar.gz
Running stress write (thrift) on 2.1 branch is resulting in the following.
Note: this is after working around
[JAVA-276|https://datastax-oss.atlassian.net/browse/JAVA-276] causing stress
to fail to connect to 2.1.
{noformat}
$ ./tools/bin/cassandra-stress write n=500
Unable to create stress keyspace: Keyspace names must be case-insensitively
unique (Keyspace1 conflicts with Keyspace1)
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
...
java.lang.Arithmjava.io.IOException: Operation [220] x10 key DD Error
executing: (ArithmeticException): / by zero
eticException: / by zero
java.lang.ArithmeticException: / by zero
at org.apache.cassandra.stress.Operation.error(Operation.java:237)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:216)
at
org.apache.cassandra.stress.operations.ThriftInserter.run(ThriftInserter.java:72)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:302)java.lang.ArithmeticException:
/ by zero
java.lang.ArithmeticException: / by zero
java.io.IOException: Operation [200] x10 key C9 Error executing:
(ArithmeticException): / by zero
at org.apache.cassandra.stress.Operation.error(Operation.java:237)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:216)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.operations.ThriftInserter.run(ThriftInserter.java:72)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:302)
...
{noformat}
Seems to be just a thrift issue, as running stress write using the native
protocol works:
{noformat}
$ ./tools/bin/cassandra-stress write n=500 -mode native cql3
Unable to create stress keyspace: Keyspace names must be case-insensitively
unique (Keyspace1 conflicts with Keyspace1)
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
Sleeping 2s...
Running WRITE with 50 threads for 500 iterations
ops ,op/s,adj op/s, key/s,mean, med, .95, .99,
.999, max, time, stderr
29342 , 29340, 30903, 29340, 1.7, 1.4, 3.1, 5.7,
51.5,54.4,1.0, 0.0
56353 , 26968, 28523, 26968, 1.8, 1.5, 3.6, 7.0,
56.5,57.2,2.0, 0.0
...
500 , 29358, 29358, 29358, 1.7, 1.3, 3.6, 8.4,
10.1,11.6, 168.8, 0.00828
Results:
real op rate : 29620
adjusted op rate : 29629
[jira] [Comment Edited] (CASSANDRA-6777) stress write using thrift results in ArithmeticException / by zero errors
[
https://issues.apache.org/jira/browse/CASSANDRA-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13914117#comment-13914117
]
dan jatnieks edited comment on CASSANDRA-6777 at 2/27/14 6:12 AM:
--
A recent change to java-driver (revision f6a179d, for
[JAVA-269|https://datastax-oss.atlassian.net/browse/JAVA-269]) caused this.
In stress, {{SmartThriftClient#get()}} gets a set of hosts by calling
{{metadata.getReplicas(keyspace, pk)}}.
However, in java-driver, {{Metadata#getReplicas}} has changed and now calls
{{handleId(keyspace)}} which essentially forces values to lowercase unless they
are quoted.
This likely means the caller of {{Metadata#getReplicas}} is responsible to
first quote case-sensitive keyspace names, such as the default Keyspace1 used
by stress, e.g. {{metadata.getReplicas(metadata.quote(keyspace), pk)}}
was (Author: djatnieks):
A recent change to java-driver (revision f6a179d, for JAVA-269) caused this.
In stress, {{SmartThriftClient#get()}} gets a set of hosts by calling
{{metadata.getReplicas(keyspace, pk)}}.
However, in java-driver, {{Metadata#getReplicas}} has changed and now calls
{{handleId(keyspace)}} which essentially forces values to lowercase unless they
are quoted.
This likely means the caller of {{Metadata#getReplicas}} is responsible to
first quote case-sensitive keyspace names, such as the default Keyspace1 used
by stress, e.g. {{metadata.getReplicas(metadata.quote(keyspace), pk)}}
stress write using thrift results in ArithmeticException / by zero errors
-
Key: CASSANDRA-6777
URL: https://issues.apache.org/jira/browse/CASSANDRA-6777
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: Mac OSX, java 1.7.0_51
Reporter: dan jatnieks
Priority: Minor
Labels: stress
Attachments: logs.tar.gz
Running stress write (thrift) on 2.1 branch is resulting in the following.
Note: this is after working around
[JAVA-276|https://datastax-oss.atlassian.net/browse/JAVA-276] causing stress
to fail to connect to 2.1.
{noformat}
$ ./tools/bin/cassandra-stress write n=500
Unable to create stress keyspace: Keyspace names must be case-insensitively
unique (Keyspace1 conflicts with Keyspace1)
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
...
java.lang.Arithmjava.io.IOException: Operation [220] x10 key DD Error
executing: (ArithmeticException): / by zero
eticException: / by zero
java.lang.ArithmeticException: / by zero
at org.apache.cassandra.stress.Operation.error(Operation.java:237)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:216)
at
org.apache.cassandra.stress.operations.ThriftInserter.run(ThriftInserter.java:72)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:302)java.lang.ArithmeticException:
/ by zero
java.lang.ArithmeticException: / by zero
java.io.IOException: Operation [200] x10 key C9 Error executing:
(ArithmeticException): / by zero
at org.apache.cassandra.stress.Operation.error(Operation.java:237)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:216)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.operations.ThriftInserter.run(ThriftInserter.java:72)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:302)
...
{noformat}
Seems to be just a thrift issue, as running stress write using the native
protocol works:
{noformat}
$ ./tools/bin/cassandra-stress write n=500 -mode native cql3
Unable to create stress keyspace: Keyspace names must be case-insensitively
unique (Keyspace1 conflicts with Keyspace1)
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
Sleeping 2s...
Running WRITE with 50 threads for 500 iterations
ops ,op/s,adj op/s, key/s,mean, med, .95, .99,
.999, max, time, stderr
29342 , 29340, 30903, 29340, 1.7, 1.4, 3.1, 5.7,
51.5,54.4,1.0, 0.0
56353 , 26968, 28523, 26968, 1.8, 1.5, 3.6, 7.0,
56.5,57.2,2.0, 0.0
...
500 , 29358, 29358, 29358, 1.7, 1.3, 3.6, 8.4,
10.1,11.6, 168.8, 0.00828
Results:
[jira] [Updated] (CASSANDRA-6777) 2.1 w/java-driver 2.0 and stress write using thrift results in ArithmeticException / by zero errors
[
https://issues.apache.org/jira/browse/CASSANDRA-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks updated CASSANDRA-6777:
Summary: 2.1 w/java-driver 2.0 and stress write using thrift results in
ArithmeticException / by zero errors (was: stress write using thrift results
in ArithmeticException / by zero errors)
2.1 w/java-driver 2.0 and stress write using thrift results in
ArithmeticException / by zero errors
---
Key: CASSANDRA-6777
URL: https://issues.apache.org/jira/browse/CASSANDRA-6777
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: Mac OSX, java 1.7.0_51
Reporter: dan jatnieks
Priority: Minor
Labels: stress
Attachments: logs.tar.gz
Running stress write (thrift) on 2.1 branch is resulting in the following.
Note: this is after working around
[JAVA-276|https://datastax-oss.atlassian.net/browse/JAVA-276] causing stress
to fail to connect to 2.1.
{noformat}
$ ./tools/bin/cassandra-stress write n=500
Unable to create stress keyspace: Keyspace names must be case-insensitively
unique (Keyspace1 conflicts with Keyspace1)
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
...
java.lang.Arithmjava.io.IOException: Operation [220] x10 key DD Error
executing: (ArithmeticException): / by zero
eticException: / by zero
java.lang.ArithmeticException: / by zero
at org.apache.cassandra.stress.Operation.error(Operation.java:237)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:216)
at
org.apache.cassandra.stress.operations.ThriftInserter.run(ThriftInserter.java:72)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:302)java.lang.ArithmeticException:
/ by zero
java.lang.ArithmeticException: / by zero
java.io.IOException: Operation [200] x10 key C9 Error executing:
(ArithmeticException): / by zero
at org.apache.cassandra.stress.Operation.error(Operation.java:237)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:216)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.operations.ThriftInserter.run(ThriftInserter.java:72)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:302)
...
{noformat}
Seems to be just a thrift issue, as running stress write using the native
protocol works:
{noformat}
$ ./tools/bin/cassandra-stress write n=500 -mode native cql3
Unable to create stress keyspace: Keyspace names must be case-insensitively
unique (Keyspace1 conflicts with Keyspace1)
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
Sleeping 2s...
Running WRITE with 50 threads for 500 iterations
ops ,op/s,adj op/s, key/s,mean, med, .95, .99,
.999, max, time, stderr
29342 , 29340, 30903, 29340, 1.7, 1.4, 3.1, 5.7,
51.5,54.4,1.0, 0.0
56353 , 26968, 28523, 26968, 1.8, 1.5, 3.6, 7.0,
56.5,57.2,2.0, 0.0
...
500 , 29358, 29358, 29358, 1.7, 1.3, 3.6, 8.4,
10.1,11.6, 168.8, 0.00828
Results:
real op rate : 29620
adjusted op rate : 29629
adjusted op rate stderr : 0
key rate : 29620
latency mean : 1.7
latency median: 1.4
latency 95th percentile : 3.0
latency 99th percentile : 5.4
latency 99.9th percentile : 56.5
latency max : 305.7
Total operation time : 00:02:48
END
{noformat}
Attaching stress write and system logs as well.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Reopened] (CASSANDRA-6777) 2.1 w/java-driver 2.0 and stress write using thrift results in ArithmeticException / by zero errors
[
https://issues.apache.org/jira/browse/CASSANDRA-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks reopened CASSANDRA-6777:
-
Re-opening as there does appear to be something to fix in stress to make it
work with java-driver 2.0.
At the moment the 2.1 branch is using {{cassandra-driver-core-2.0.0-rc3}} but I
think when this is updated to the release version of 2.0 this error will occur.
See my previous comment for the proposed stress fix to quote the keyspace name
when calling java-driver {{Metadata#getReplicas}}.
2.1 w/java-driver 2.0 and stress write using thrift results in
ArithmeticException / by zero errors
---
Key: CASSANDRA-6777
URL: https://issues.apache.org/jira/browse/CASSANDRA-6777
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: Mac OSX, java 1.7.0_51
Reporter: dan jatnieks
Priority: Minor
Labels: stress
Attachments: logs.tar.gz
Running stress write (thrift) on 2.1 branch is resulting in the following.
Note: this is after working around
[JAVA-276|https://datastax-oss.atlassian.net/browse/JAVA-276] causing stress
to fail to connect to 2.1.
{noformat}
$ ./tools/bin/cassandra-stress write n=500
Unable to create stress keyspace: Keyspace names must be case-insensitively
unique (Keyspace1 conflicts with Keyspace1)
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
java.lang.ArithmeticException: / by zero
...
java.lang.Arithmjava.io.IOException: Operation [220] x10 key DD Error
executing: (ArithmeticException): / by zero
eticException: / by zero
java.lang.ArithmeticException: / by zero
at org.apache.cassandra.stress.Operation.error(Operation.java:237)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:216)
at
org.apache.cassandra.stress.operations.ThriftInserter.run(ThriftInserter.java:72)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:302)java.lang.ArithmeticException:
/ by zero
java.lang.ArithmeticException: / by zero
java.io.IOException: Operation [200] x10 key C9 Error executing:
(ArithmeticException): / by zero
at org.apache.cassandra.stress.Operation.error(Operation.java:237)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:216)
java.lang.ArithmeticException: / by zero at
org.apache.cassandra.stress.operations.ThriftInserter.run(ThriftInserter.java:72)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:302)
...
{noformat}
Seems to be just a thrift issue, as running stress write using the native
protocol works:
{noformat}
$ ./tools/bin/cassandra-stress write n=500 -mode native cql3
Unable to create stress keyspace: Keyspace names must be case-insensitively
unique (Keyspace1 conflicts with Keyspace1)
Warming up WRITE with 5 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
Sleeping 2s...
Running WRITE with 50 threads for 500 iterations
ops ,op/s,adj op/s, key/s,mean, med, .95, .99,
.999, max, time, stderr
29342 , 29340, 30903, 29340, 1.7, 1.4, 3.1, 5.7,
51.5,54.4,1.0, 0.0
56353 , 26968, 28523, 26968, 1.8, 1.5, 3.6, 7.0,
56.5,57.2,2.0, 0.0
...
500 , 29358, 29358, 29358, 1.7, 1.3, 3.6, 8.4,
10.1,11.6, 168.8, 0.00828
Results:
real op rate : 29620
adjusted op rate : 29629
adjusted op rate stderr : 0
key rate : 29620
latency mean : 1.7
latency median: 1.4
latency 95th percentile : 3.0
latency 99th percentile : 5.4
latency 99.9th percentile : 56.5
latency max : 305.7
Total operation time : 00:02:48
END
{noformat}
Attaching stress write and system logs as well.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6357) Flush memtables to separate directory
[ https://issues.apache.org/jira/browse/CASSANDRA-6357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881433#comment-13881433 ] dan jatnieks commented on CASSANDRA-6357: - I did some stress write testing on a single node physical machine with multiple rotating disks. The machine was a quad-code Intel Xeon E3-1230, 32Gb memory, and 4 disks (2Tb, 5400 rpm). Two scenarios were measured using stress-write for 10 million records (cassandra-stress --num-keys 1000 --columns=50 --operation=INSERT): Base scenario: * O/S on separate device * Commitlog on separate device * Data and flush directories on the same device Flush scenario: * O/S on separate device * Commitlog on separate device * Data directory on separate device * Flush directory on separate device The result of splitting the flush directory to another device on the stress-write results was: * an elapsed time improvement of about 57% * a much lower, and consistent, 99.9th percentile latency (3308 ms vs 15963 ms) See attached graph: [Stress Write Latency 99.9th Percentile|^c6357-stress-write-latency-99th-1.png] Flush memtables to separate directory - Key: CASSANDRA-6357 URL: https://issues.apache.org/jira/browse/CASSANDRA-6357 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Patrick McFadin Assignee: Jonathan Ellis Attachments: 6357.txt Flush writers are a critical element for keeping a node healthy. When several compactions run on systems with low performing data directories, IO becomes a premium. Once the disk subsystem is saturated, write IO is blocked which will cause flush writer threads to backup. Since memtables are large blocks of memory in the JVM, too much blocking can cause excessive GC over time degrading performance. In the worst case causing an OOM. Since compaction is running on the data directories. My proposal is to create a separate directory for flushing memtables. Potentially we can use the same methodology of keeping the commit log separate and minimize disk contention against the critical function of the flushwriter. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Comment Edited] (CASSANDRA-6357) Flush memtables to separate directory
[ https://issues.apache.org/jira/browse/CASSANDRA-6357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881433#comment-13881433 ] dan jatnieks edited comment on CASSANDRA-6357 at 1/24/14 9:20 PM: -- I did some stress write testing on a single node physical machine with multiple rotating disks. The machine was a quad-core Intel Xeon E3-1230, 32Gb memory, and 4 disks (2Tb, 5400 rpm). Two scenarios were measured using stress-write for 10 million records (cassandra-stress --num-keys 1000 --columns=50 --operation=INSERT): Base scenario: * O/S on separate device * Commitlog on separate device * Data and flush directories on the same device Flush scenario: * O/S on separate device * Commitlog on separate device * Data directory on separate device * Flush directory on separate device The result of splitting the flush directory to another device on the stress-write results was: * an elapsed time improvement of about 57% * a much lower, and consistent, 99.9th percentile latency (3308 ms vs 15963 ms) See attached graph: [Stress Write Latency 99.9th Percentile|^c6357-stress-write-latency-99th-1.png] was (Author: djatnieks): I did some stress write testing on a single node physical machine with multiple rotating disks. The machine was a quad-code Intel Xeon E3-1230, 32Gb memory, and 4 disks (2Tb, 5400 rpm). Two scenarios were measured using stress-write for 10 million records (cassandra-stress --num-keys 1000 --columns=50 --operation=INSERT): Base scenario: * O/S on separate device * Commitlog on separate device * Data and flush directories on the same device Flush scenario: * O/S on separate device * Commitlog on separate device * Data directory on separate device * Flush directory on separate device The result of splitting the flush directory to another device on the stress-write results was: * an elapsed time improvement of about 57% * a much lower, and consistent, 99.9th percentile latency (3308 ms vs 15963 ms) See attached graph: [Stress Write Latency 99.9th Percentile|^c6357-stress-write-latency-99th-1.png] Flush memtables to separate directory - Key: CASSANDRA-6357 URL: https://issues.apache.org/jira/browse/CASSANDRA-6357 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Patrick McFadin Assignee: Jonathan Ellis Attachments: 6357.txt, c6357-stress-write-latency-99th-1.png Flush writers are a critical element for keeping a node healthy. When several compactions run on systems with low performing data directories, IO becomes a premium. Once the disk subsystem is saturated, write IO is blocked which will cause flush writer threads to backup. Since memtables are large blocks of memory in the JVM, too much blocking can cause excessive GC over time degrading performance. In the worst case causing an OOM. Since compaction is running on the data directories. My proposal is to create a separate directory for flushing memtables. Potentially we can use the same methodology of keeping the commit log separate and minimize disk contention against the critical function of the flushwriter. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6357) Flush memtables to separate directory
[ https://issues.apache.org/jira/browse/CASSANDRA-6357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dan jatnieks updated CASSANDRA-6357: Attachment: c6357-stress-write-latency-99th-1.png Flush memtables to separate directory - Key: CASSANDRA-6357 URL: https://issues.apache.org/jira/browse/CASSANDRA-6357 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Patrick McFadin Assignee: Jonathan Ellis Attachments: 6357.txt, c6357-stress-write-latency-99th-1.png Flush writers are a critical element for keeping a node healthy. When several compactions run on systems with low performing data directories, IO becomes a premium. Once the disk subsystem is saturated, write IO is blocked which will cause flush writer threads to backup. Since memtables are large blocks of memory in the JVM, too much blocking can cause excessive GC over time degrading performance. In the worst case causing an OOM. Since compaction is running on the data directories. My proposal is to create a separate directory for flushing memtables. Potentially we can use the same methodology of keeping the commit log separate and minimize disk contention against the critical function of the flushwriter. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Comment Edited] (CASSANDRA-6357) Flush memtables to separate directory
[ https://issues.apache.org/jira/browse/CASSANDRA-6357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881433#comment-13881433 ] dan jatnieks edited comment on CASSANDRA-6357 at 1/24/14 9:22 PM: -- I did some stress write testing on a single node physical machine with multiple rotating disks. The machine was a quad-core Intel Xeon E3-1230, 32Gb memory, and 4 disks (2Tb, 5400 rpm). Two scenarios were measured using stress-write for 10 million records (cassandra-stress --num-keys 1000 --columns=50 --operation=INSERT): Base scenario: * O/S on separate device * Commitlog on separate device * Data and flush directories on the same device Flush scenario: * O/S on separate device * Commitlog on separate device * Data directory on separate device * Flush directory on separate device The result of splitting the flush directory to another device on the stress-write results was: * an elapsed time improvement of about 57% * about double the op/sec rate * a much lower, and consistent, 99.9th percentile latency (3308 ms vs 15963 ms) See attached graph: [Stress Write Latency 99.9th Percentile|^c6357-stress-write-latency-99th-1.png] was (Author: djatnieks): I did some stress write testing on a single node physical machine with multiple rotating disks. The machine was a quad-core Intel Xeon E3-1230, 32Gb memory, and 4 disks (2Tb, 5400 rpm). Two scenarios were measured using stress-write for 10 million records (cassandra-stress --num-keys 1000 --columns=50 --operation=INSERT): Base scenario: * O/S on separate device * Commitlog on separate device * Data and flush directories on the same device Flush scenario: * O/S on separate device * Commitlog on separate device * Data directory on separate device * Flush directory on separate device The result of splitting the flush directory to another device on the stress-write results was: * an elapsed time improvement of about 57% * a much lower, and consistent, 99.9th percentile latency (3308 ms vs 15963 ms) See attached graph: [Stress Write Latency 99.9th Percentile|^c6357-stress-write-latency-99th-1.png] Flush memtables to separate directory - Key: CASSANDRA-6357 URL: https://issues.apache.org/jira/browse/CASSANDRA-6357 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Patrick McFadin Assignee: Jonathan Ellis Attachments: 6357.txt, c6357-stress-write-latency-99th-1.png Flush writers are a critical element for keeping a node healthy. When several compactions run on systems with low performing data directories, IO becomes a premium. Once the disk subsystem is saturated, write IO is blocked which will cause flush writer threads to backup. Since memtables are large blocks of memory in the JVM, too much blocking can cause excessive GC over time degrading performance. In the worst case causing an OOM. Since compaction is running on the data directories. My proposal is to create a separate directory for flushing memtables. Potentially we can use the same methodology of keeping the commit log separate and minimize disk contention against the critical function of the flushwriter. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-5978) stressd broken by ClientEncriptionOptions
[
https://issues.apache.org/jira/browse/CASSANDRA-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13832793#comment-13832793
]
dan jatnieks commented on CASSANDRA-5978:
-
Ran into this today ... my stack has line numbers:
{noformat}
./dse-3.2.1/resources/cassandra/tools/bin/cassandra-stress -K 100 -t 50 -R
org.apache.cassandra.locator.NetworkTopologyStrategy --num-keys=1000
--columns=50 -D nodelist -O Cassandra:3 --operation=INSERT --send-to 127.0.0.1
Exception in thread main java.io.NotSerializableException:
org.apache.cassandra.config.EncryptionOptions$ClientEncryptionOptions
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1181)
at
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1541)
at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1506)
at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1175)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
at org.apache.cassandra.stress.Stress.main(Unknown Source)
Control-C caught. Canceling running action and shutting down...
{noformat}
stressd broken by ClientEncriptionOptions
-
Key: CASSANDRA-5978
URL: https://issues.apache.org/jira/browse/CASSANDRA-5978
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Jeremiah Jordan
Priority: Minor
The ClientEncryptionOptions object added to
org.apache.cassandra.stress.Session is not Serializable. So if you try to
use stress with stressd, the Session can't be serialized to be passed over to
stressd:
{noformat}
Exception in thread main java.io.NotSerializableException:
org.apache.cassandra.config.EncryptionOptions$ClientEncryptionOptions
at java.io.ObjectOutputStream.writeObject0(Unknown Source)
at java.io.ObjectOutputStream.defaultWriteFields(Unknown Source)
at java.io.ObjectOutputStream.writeSerialData(Unknown Source)
at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)
at java.io.ObjectOutputStream.writeObject0(Unknown Source)
at java.io.ObjectOutputStream.writeObject(Unknown Source)
at org.apache.cassandra.stress.Stress.main(Unknown Source)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (CASSANDRA-6103) ConcurrentModificationException in TokenMetadata.cloneOnlyTokenMap
[
https://issues.apache.org/jira/browse/CASSANDRA-6103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13780651#comment-13780651
]
dan jatnieks commented on CASSANDRA-6103:
-
I've run into the same error a couple times with a slightly different stack
trace with DSE 3.1.3 (C* 1.2.6 I believe).
+1 on {{updateHostId}} as a likely source from my non-expert look at the code.
{noformat}
Exception encountered during startup
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$EntryIterator.next(HashMap.java:834)
at java.util.HashMap$EntryIterator.next(HashMap.java:832)
at
com.google.common.collect.AbstractBiMap$EntrySet$1.next(AbstractBiMap.java:294)
at
com.google.common.collect.AbstractBiMap$EntrySet$1.next(AbstractBiMap.java:286)
at com.google.common.collect.AbstractBiMap.putAll(AbstractBiMap.java:160)
at com.google.common.collect.HashBiMap.putAll(HashBiMap.java:42)
at com.google.common.collect.HashBiMap.create(HashBiMap.java:72)
at
org.apache.cassandra.locator.TokenMetadata.cloneOnlyTokenMap(TokenMetadata.java:561)
at
org.apache.cassandra.locator.TokenMetadata.cloneAfterAllLeft(TokenMetadata.java:582)
at
org.apache.cassandra.service.StorageService.calculatePendingRanges(StorageService.java:1708)
at
org.apache.cassandra.service.StorageService.calculatePendingRanges(StorageService.java:1686)
at
org.apache.cassandra.service.StorageService.handleStateBootstrap(StorageService.java:1306)
at
org.apache.cassandra.service.StorageService.onChange(StorageService.java:1207)
at org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:935)
at
org.apache.cassandra.gms.Gossiper.addLocalApplicationState(Gossiper.java:1102)
at
org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:911)
at
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:699)
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:554)
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:451)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:342)
at com.datastax.bdp.server.DseDaemon.setup(DseDaemon.java:137)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:441)
at com.datastax.bdp.server.DseDaemon.main(DseDaemon.java:334)
{noformat}
ConcurrentModificationException in TokenMetadata.cloneOnlyTokenMap
--
Key: CASSANDRA-6103
URL: https://issues.apache.org/jira/browse/CASSANDRA-6103
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Mike Schrag
Fix For: 1.2.11
This isn't reproducible for me, but it happened to one of the servers in our
cluster while starting up. It went away on a restart, but I figured it was
worth filing anyway:
ERROR [main] 2013-09-26 08:04:02,478 CassandraDaemon.java (line 464)
Exception encountered during startup
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$EntryIterator.next(HashMap.java:834)
at java.util.HashMap$EntryIterator.next(HashMap.java:832)
at
com.google.common.collect.AbstractBiMap$EntrySet$1.next(AbstractBiMap.java:294)
at
com.google.common.collect.AbstractBiMap$EntrySet$1.next(AbstractBiMap.java:286)
at
com.google.common.collect.AbstractBiMap.putAll(AbstractBiMap.java:160)
at com.google.common.collect.HashBiMap.putAll(HashBiMap.java:42)
at com.google.common.collect.HashBiMap.create(HashBiMap.java:72)
at
org.apache.cassandra.locator.TokenMetadata.cloneOnlyTokenMap(TokenMetadata.java:561)
at
org.apache.cassandra.locator.AbstractReplicationStrategy.getAddressRanges(AbstractReplicationStrategy.java:192)
at
org.apache.cassandra.service.StorageService.calculatePendingRanges(StorageService.java:1711)
at
org.apache.cassandra.service.StorageService.calculatePendingRanges(StorageService.java:1692)
at
org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:1461)
at
org.apache.cassandra.service.StorageService.onChange(StorageService.java:1228)
at
org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:949)
at
org.apache.cassandra.gms.Gossiper.addLocalApplicationState(Gossiper.java:1116)
at
org.apache.cassandra.service.StorageService.setTokens(StorageService.java:214)
at
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:802)
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:554)
at
[jira] [Updated] (CASSANDRA-4536) Ability for CQL3 to list partition keys
[
https://issues.apache.org/jira/browse/CASSANDRA-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks updated CASSANDRA-4536:
Attachment: cassandra-4536_1.2.5.patch
Uploaded patch file (cassandra-4536_1.2.5) based on 1.2 branch.
Ability for CQL3 to list partition keys
---
Key: CASSANDRA-4536
URL: https://issues.apache.org/jira/browse/CASSANDRA-4536
Project: Cassandra
Issue Type: New Feature
Components: API
Affects Versions: 1.1.0
Reporter: Jonathan Ellis
Assignee: dan jatnieks
Priority: Minor
Labels: cql3
Fix For: 1.2.7
Attachments: cassandra-4536_1.1.0.patch, cassandra-4536_1.2.2.patch,
cassandra-4536_1.2.5.patch
It can be useful to know the set of in-use partition keys (storage engine row
keys). One example given to me was where application data was modeled as a
few 10s of 1000s of wide rows, where the app required presenting these rows
to the user sorted based on information in the partition key. The partition
count is small enough to do the sort client-side in memory, which is what the
app did with the Thrift API--a range slice with an empty columns list.
This was a problem when migrating to CQL3. {{SELECT mykey FROM mytable}}
includes all the logical rows, which makes the resultset too large to make
this a reasonable approach, even with paging.
One way to add support would be to allow DISTINCT in the special case of
{{SELECT DISTINCT mykey FROM mytable}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4536) Ability for CQL3 to list partition keys
[
https://issues.apache.org/jira/browse/CASSANDRA-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks updated CASSANDRA-4536:
Attachment: cassandra-4536_1.2.2.patch
Attaching patch file (cassandra-4536_1.2.2.patch).
Ability for CQL3 to list partition keys
---
Key: CASSANDRA-4536
URL: https://issues.apache.org/jira/browse/CASSANDRA-4536
Project: Cassandra
Issue Type: New Feature
Components: API
Affects Versions: 1.1.0
Reporter: Jonathan Ellis
Assignee: dan jatnieks
Priority: Minor
Labels: cql3
Fix For: 1.2.4
Attachments: cassandra-4536_1.1.0.patch, cassandra-4536_1.2.2.patch
It can be useful to know the set of in-use partition keys (storage engine row
keys). One example given to me was where application data was modeled as a
few 10s of 1000s of wide rows, where the app required presenting these rows
to the user sorted based on information in the partition key. The partition
count is small enough to do the sort client-side in memory, which is what the
app did with the Thrift API--a range slice with an empty columns list.
This was a problem when migrating to CQL3. {{SELECT mykey FROM mytable}}
includes all the logical rows, which makes the resultset too large to make
this a reasonable approach, even with paging.
One way to add support would be to allow DISTINCT in the special case of
{{SELECT DISTINCT mykey FROM mytable}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4536) Ability for CQL3 to list partition keys
[
https://issues.apache.org/jira/browse/CASSANDRA-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks updated CASSANDRA-4536:
Attachment: cassandra-4536_1.1.0.patch
Attaching patch file (cassandra-4536-1.1.0.patch) that I worked on based on
version 1.1.
I'll post a 1.2 version soon as I get it completed.
Ability for CQL3 to list partition keys
---
Key: CASSANDRA-4536
URL: https://issues.apache.org/jira/browse/CASSANDRA-4536
Project: Cassandra
Issue Type: New Feature
Components: API
Affects Versions: 1.1.0
Reporter: Jonathan Ellis
Priority: Minor
Labels: cql3
Fix For: 1.2.4
Attachments: cassandra-4536_1.1.0.patch
It can be useful to know the set of in-use partition keys (storage engine row
keys). One example given to me was where application data was modeled as a
few 10s of 1000s of wide rows, where the app required presenting these rows
to the user sorted based on information in the partition key. The partition
count is small enough to do the sort client-side in memory, which is what the
app did with the Thrift API--a range slice with an empty columns list.
This was a problem when migrating to CQL3. {{SELECT mykey FROM mytable}}
includes all the logical rows, which makes the resultset too large to make
this a reasonable approach, even with paging.
One way to add support would be to allow DISTINCT in the special case of
{{SELECT DISTINCT mykey FROM mytable}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
