[jira] [Commented] (CASSANDRA-15733) jvm dtest builder should be provided to the factory and expose state
[ https://issues.apache.org/jira/browse/CASSANDRA-15733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094160#comment-17094160 ] David Capwell commented on CASSANDRA-15733: --- updated all branches to include CASSANDRA-15756 [trunk|https://app.circleci.com/pipelines/github/dcapwell/cassandra/223/workflows/de8e2ce0-6326-4dc8-8aab-667572896814] [3.11|https://app.circleci.com/pipelines/github/dcapwell/cassandra/224/workflows/be3760bd-5f8b-42cc-adb4-e835d402bd69] [3.0|https://app.circleci.com/pipelines/github/dcapwell/cassandra/225/workflows/fae15e4a-dc73-4a52-bbe5-106d4f726d54] [2.2|https://app.circleci.com/pipelines/github/dcapwell/cassandra/226/workflows/be1d3ea6-12dc-4908-8b5b-3147775cd277] > jvm dtest builder should be provided to the factory and expose state > > > Key: CASSANDRA-15733 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15733 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Time Spent: 3h > Remaining Estimate: 0h > > Currently the builder is rather heavy and creates configs plus call the > factory with specific fields only, this isn’t that flexible and makes it > harder to have custom cluster definitions which require additional fields to > be defined. To solve this we should make the builder be sent to the factory > and expose the state so the factory can get all the fields it needs, the > factory should also be in charge of creating the configs -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-22) RESTEasy integration for Cassandra Sidecar
[ https://issues.apache.org/jira/browse/CASSANDRASC-22?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094142#comment-17094142 ] Dinesh Joshi commented on CASSANDRASC-22: - FYI: Contains some circleci changes that fixed my test failures. > RESTEasy integration for Cassandra Sidecar > -- > > Key: CASSANDRASC-22 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-22 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Dinesh Joshi >Assignee: Dinesh Joshi >Priority: Normal > > Add support for JAX-RS based routing via RESTEasy to Cassandra Sidecar. This > also dynamically generates swagger documentation and adds the swagger UI. > [Branch|https://github.com/dineshjoshi/cassandra-sidecar/tree/resteasy-swagger] > [Tests|https://circleci.com/workflow-run/a7888146-a22d-45af-983a-8833b77eef59] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-22) RESTEasy integration for Cassandra Sidecar
[ https://issues.apache.org/jira/browse/CASSANDRASC-22?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi updated CASSANDRASC-22: Description: Add support for JAX-RS based routing via RESTEasy to Cassandra Sidecar. This also dynamically generates swagger documentation and adds the swagger UI. [Branch|https://github.com/dineshjoshi/cassandra-sidecar/tree/resteasy-swagger] [Tests|https://circleci.com/workflow-run/a7888146-a22d-45af-983a-8833b77eef59] was: Add support for JAX-RS based routing via RESTEasy to Cassandra Sidecar. This also dynamically generates swagger documentation and adds the swagger UI. [Branch|https://github.com/dineshjoshi/cassandra-sidecar/tree/resteasy-swagger] [Tests|https://circleci.com/gh/dineshjoshi/cassandra-sidecar/tree/resteasy-swagger] > RESTEasy integration for Cassandra Sidecar > -- > > Key: CASSANDRASC-22 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-22 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Dinesh Joshi >Assignee: Dinesh Joshi >Priority: Normal > > Add support for JAX-RS based routing via RESTEasy to Cassandra Sidecar. This > also dynamically generates swagger documentation and adds the swagger UI. > [Branch|https://github.com/dineshjoshi/cassandra-sidecar/tree/resteasy-swagger] > [Tests|https://circleci.com/workflow-run/a7888146-a22d-45af-983a-8833b77eef59] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14831) Nodetool repair hangs with java.net.SocketException: End-of-stream reached
[
https://issues.apache.org/jira/browse/CASSANDRA-14831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094137#comment-17094137
]
David Capwell commented on CASSANDRA-14831:
---
Sorry. Just saw this. I’ll check tomorrow but I don’t remember a big difference
from 3.0 so possibly
> Nodetool repair hangs with java.net.SocketException: End-of-stream reached
> --
>
> Key: CASSANDRA-14831
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14831
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair
>Reporter: Tania S Engel
>Assignee: David Capwell
>Priority: Normal
> Fix For: 3.11.x
>
> Attachments: Cassandra - 14831 Logs.mht
>
>
> Using Cassandra 3.11.1.
> Ran >nodetool repair on a small 3 node cluster from node
> 3eef. Node 9160 and 3f5e experienced a stream failure.
> *NODE 9160:*
> ERROR [STREAM-IN-/fd70:616e:6761:6561:ae1f:6bff:fe12:3f5e:7000] 2018-10-16
> 01:45:00,400 StreamSession.java:593 - [Stream
> #103fe070-d0e5-11e8-a993-5929a1c131b4] Streaming error occurred on session
> with peer fd70:616e:6761:6561:ae1f:6bff:fe12:3f5e
> *java.net.SocketException: End-of-stream reached*
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:71)
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:311)
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_152]
>
> *NODE 3f5e:*
> ERROR [STREAM-IN-/fd70:616e:6761:6561:ec4:7aff:fece:9160:59676] 2018-10-16
> 01:45:09,474 StreamSession.java:593 - [Stream
> #103ef610-d0e5-11e8-a993-5929a1c131b4] Streaming error occurred on session
> with peer fd70:616e:6761:6561:ec4:7aff:fece:9160
> java.io.IOException: An existing connection was forcibly closed by the remote
> host
> at sun.nio.ch.SocketDispatcher.read0(Native Method) ~[na:1.8.0_152]
> at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43) ~[na:1.8.0_152]
> at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) ~[na:1.8.0_152]
> at sun.nio.ch.IOUtil.read(IOUtil.java:197) ~[na:1.8.0_152]
> at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
> ~[na:1.8.0_152]
> at sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:206)
> ~[na:1.8.0_152]
> at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
> ~[na:1.8.0_152]
> at java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)
> ~[na:1.8.0_152]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:56)
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:311)
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_152]
>
> *NODE 3EEF:*
> ERROR [RepairJobTask:14] 2018-10-16 01:45:00,457 RepairSession.java:281 -
> [repair #f2ab3eb0-d0e4-11e8-9926-bf64f35712c1] Session completed with the
> following error
> org.apache.cassandra.exceptions.RepairException: [repair
> #f2ab3eb0-d0e4-11e8-9926-bf64f35712c1 on logs/{color:#33}XX{color},
> [(-8271925838625565988,-8266397600493941101],
> (2290821710735817606,2299380749828706426]
> …(-8701313305140908434,-8686533141993948378]]] Sync failed between
> /fd70:616e:6761:6561:ec4:7aff:fece:9160 and
> /fd70:616e:6761:6561:ae1f:6bff:fe12:3f5e
> at
> org.apache.cassandra.repair.RemoteSyncTask.syncComplete(RemoteSyncTask.java:67)
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> at
> org.apache.cassandra.repair.RepairSession.syncComplete(RepairSession.java:202)
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> at
> org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:495)
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> at
> org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:162)
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66)
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_152]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_152]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [na:1.8.0_152]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [na:1.8.0_152]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
> [apache-cassandra-3.11.1.jar:3.11.1]
> at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_152]
>
> ERROR [RepairJobTask:14]
[jira] [Updated] (CASSANDRA-13994) Remove COMPACT STORAGE internals before 4.0 release
[
https://issues.apache.org/jira/browse/CASSANDRA-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dinesh Joshi updated CASSANDRA-13994:
-
Test and Documentation Plan: existing tests
Status: Patch Available (was: In Progress)
> Remove COMPACT STORAGE internals before 4.0 release
> ---
>
> Key: CASSANDRA-13994
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13994
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Local Write-Read Paths
>Reporter: Alex Petrov
>Assignee: Ekaterina Dimitrova
>Priority: Low
> Fix For: 4.0, 4.0-rc
>
>
> 4.0 comes without thrift (after [CASSANDRA-5]) and COMPACT STORAGE (after
> [CASSANDRA-10857]), and since Compact Storage flags are now disabled, all of
> the related functionality is useless.
> There are still some things to consider:
> 1. One of the system tables (built indexes) was compact. For now, we just
> added {{value}} column to it to make sure it's backwards-compatible, but we
> might want to make sure it's just a "normal" table and doesn't have redundant
> columns.
> 2. Compact Tables were building indexes in {{KEYS}} mode. Removing it is
> trivial, but this would mean that all built indexes will be defunct. We could
> log a warning for now and ask users to migrate off those for now and
> completely remove it from future releases. It's just a couple of classes
> though.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13994) Remove COMPACT STORAGE internals before 4.0 release
[
https://issues.apache.org/jira/browse/CASSANDRA-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dinesh Joshi updated CASSANDRA-13994:
-
Reviewers: Dinesh Joshi, Dinesh Joshi (was: Dinesh Joshi)
Dinesh Joshi, Dinesh Joshi (was: Dinesh Joshi)
Status: Review In Progress (was: Patch Available)
> Remove COMPACT STORAGE internals before 4.0 release
> ---
>
> Key: CASSANDRA-13994
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13994
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Local Write-Read Paths
>Reporter: Alex Petrov
>Assignee: Ekaterina Dimitrova
>Priority: Low
> Fix For: 4.0, 4.0-rc
>
>
> 4.0 comes without thrift (after [CASSANDRA-5]) and COMPACT STORAGE (after
> [CASSANDRA-10857]), and since Compact Storage flags are now disabled, all of
> the related functionality is useless.
> There are still some things to consider:
> 1. One of the system tables (built indexes) was compact. For now, we just
> added {{value}} column to it to make sure it's backwards-compatible, but we
> might want to make sure it's just a "normal" table and doesn't have redundant
> columns.
> 2. Compact Tables were building indexes in {{KEYS}} mode. Removing it is
> trivial, but this would mean that all built indexes will be defunct. We could
> log a warning for now and ask users to migrate off those for now and
> completely remove it from future releases. It's just a couple of classes
> though.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15718) Improve BatchMetricsTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094135#comment-17094135
]
Dinesh Joshi commented on CASSANDRA-15718:
--
FYI: CI was green
[here|https://circleci.com/workflow-run/10c17f89-a4f3-4b37-9bf4-7abd6834579f].
> Improve BatchMetricsTest
> -
>
> Key: CASSANDRA-15718
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15718
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Stephen Mallette
>Assignee: Stephen Mallette
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15582-test-cleanup.patch
>
>
> As noted in CASSANDRA-15582 {{BatchMetricsTest}} should test
> {{BatchStatement.Type.COUNTER}} to cover all the {{BatchMetrics}}. Some
> changes were introduced to make this improvement at:
> https://github.com/apache/cassandra/compare/trunk...spmallette:CASSANDRA-15582-trunk-batchmetrics
> and the following suggestions were made in review (in addition to the
> suggestion that a separate JIRA be created for this change) by [~dcapwell]:
> {quote}
> * I like the usage of BatchStatement.Type for the tests
> * honestly feel quick theories is better than random, but glad you added the
> seed to all asserts =). Would still be better as a quick theories test since
> you basically wrote a property anyways!
> *
> https://github.com/apache/cassandra/compare/trunk...spmallette:CASSANDRA-15582-trunk-batchmetrics#diff-8948cec1f9d33f10b15c38de80141548R131
> feel you should rename to expectedPartitionsPerLoggedBatch
> {Count,Logged,Unlogged}
> * . pre is what the value is, post is what the value is expected to be
> (rather than what it is).
> *
> *
> https://github.com/apache/cassandra/compare/trunk...spmallette:CASSANDRA-15582-trunk-batchmetrics#diff-8948cec1f9d33f10b15c38de80141548R150
> this doesn't look correct. the batch has distinctPartitions mutations, so
> shouldn't max reflect that? I ran the current test in a debugger and see that
> that is the case (aka current test is wrong).
> most of the comments are nit picks, but the last one looks like a test bug to
> me
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15718) Improve BatchMetricsTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dinesh Joshi updated CASSANDRA-15718:
-
Fix Version/s: 4.0-alpha
Source Control Link:
https://github.com/apache/cassandra/commit/d497c8c9f3f9a19e0193c1c463f684b75aeb7081
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed. Thanks for your contribution [~spmallette]! Thanks for the review
[~dcapwell]!
> Improve BatchMetricsTest
> -
>
> Key: CASSANDRA-15718
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15718
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Stephen Mallette
>Assignee: Stephen Mallette
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15582-test-cleanup.patch
>
>
> As noted in CASSANDRA-15582 {{BatchMetricsTest}} should test
> {{BatchStatement.Type.COUNTER}} to cover all the {{BatchMetrics}}. Some
> changes were introduced to make this improvement at:
> https://github.com/apache/cassandra/compare/trunk...spmallette:CASSANDRA-15582-trunk-batchmetrics
> and the following suggestions were made in review (in addition to the
> suggestion that a separate JIRA be created for this change) by [~dcapwell]:
> {quote}
> * I like the usage of BatchStatement.Type for the tests
> * honestly feel quick theories is better than random, but glad you added the
> seed to all asserts =). Would still be better as a quick theories test since
> you basically wrote a property anyways!
> *
> https://github.com/apache/cassandra/compare/trunk...spmallette:CASSANDRA-15582-trunk-batchmetrics#diff-8948cec1f9d33f10b15c38de80141548R131
> feel you should rename to expectedPartitionsPerLoggedBatch
> {Count,Logged,Unlogged}
> * . pre is what the value is, post is what the value is expected to be
> (rather than what it is).
> *
> *
> https://github.com/apache/cassandra/compare/trunk...spmallette:CASSANDRA-15582-trunk-batchmetrics#diff-8948cec1f9d33f10b15c38de80141548R150
> this doesn't look correct. the batch has distinctPartitions mutations, so
> shouldn't max reflect that? I ran the current test in a debugger and see that
> that is the case (aka current test is wrong).
> most of the comments are nit picks, but the last one looks like a test bug to
> me
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Improved testing for batch metrics.
This is an automated email from the ASF dual-hosted git repository.
djoshi pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new d497c8c Improved testing for batch metrics.
d497c8c is described below
commit d497c8c9f3f9a19e0193c1c463f684b75aeb7081
Author: Stephen Mallette
AuthorDate: Tue Mar 31 13:47:44 2020 -0400
Improved testing for batch metrics.
Refactored the test to include assertion metrics for counter batches
in addition to what was previously tested in logged and unlogged batches.
Modified tests to assert random ranges of batches, partitions and
statements,
printing the seed for the Random on failure so that the error state could be
recreated.
Patch by Stephen Mallette; Reviewed by David Capwell and Dinesh Joshi for
CASSANDRA-15718
---
.../DecayingEstimatedHistogramReservoir.java | 64 +++--
.../apache/cassandra/metrics/BatchMetricsTest.java | 160 +++--
2 files changed, 171 insertions(+), 53 deletions(-)
diff --git
a/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
b/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
index 6d24314..6dd1687 100644
---
a/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
+++
b/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
@@ -22,6 +22,7 @@ import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.io.PrintWriter;
import java.nio.charset.StandardCharsets;
+import java.util.Objects;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicLongArray;
@@ -41,7 +42,7 @@ import static java.lang.Math.min;
* collected in the previous minute. Measured values are collected in variable
sized buckets, using small buckets in the
* lower range and larger buckets in the upper range. Use this histogram when
you want to know if the distribution of
* the underlying data stream has changed recently and you want high
resolution on values in the lower range.
- *
+ *
* The histogram use forward decay [1] to make recent values more significant.
The forward decay factor will be doubled
* every minute (half-life time set to 60 seconds) [2]. The forward decay
landmark is reset every 30 minutes (or at
* first read/update after 30 minutes). During landmark reset, updates and
reads in the reservoir will be blocked in a
@@ -49,29 +50,31 @@ import static java.lang.Math.min;
* assumption that in an extreme case we would have to collect a metric 1M
times for a single bucket each second. By the
* end of the 30:th minute all collected values will roughly add up to
1.000.000 * 60 * pow(2, 30) which can be
* represented with 56 bits giving us some head room in a signed 64 bit long.
- *
+ *
* Internally two reservoirs are maintained, one with decay and one without
decay. All public getters in a {@link Snapshot}
* will expose the decay functionality with the exception of the {@link
Snapshot#getValues()} which will return values
* from the reservoir without decay. This makes it possible for the caller to
maintain precise deltas in an interval of
- * its choise.
- *
+ * its choice.
+ *
* The bucket size starts at 1 and grows by 1.2 each time (rounding and
removing duplicates). It goes from 1 to around
* 18T by default (creating 164+1 buckets), which will give a timing
resolution from microseconds to roughly 210 days,
* with less precision as the numbers get larger.
- *
+ *
* The series of values to which the counts in `decayingBuckets` correspond:
* 1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 14, 17, 20, 24, 29, 35, 42, 50, 60, 72 etc.
* Thus, a `decayingBuckets` of [0, 0, 1, 10] would mean we had seen 1 value
of 3 and 10 values of 4.
- *
+ *
* Each bucket represents values from (previous bucket offset, current offset].
- *
+ *
* To reduce contention each logical bucket is striped accross a configurable
number of stripes (default: 2). Threads are
* assigned to specific stripes. In addition, logical buckets are distributed
across the physical storage to reduce conention
* when logically adjacent buckets are updated. See CASSANDRA-15213.
- *
- * [1]: http://dimacs.rutgers.edu/~graham/pubs/papers/fwddecay.pdf
- * [2]: https://en.wikipedia.org/wiki/Half-life
- * [3]:
https://github.com/dropwizard/metrics/blob/v3.1.2/metrics-core/src/main/java/com/codahale/metrics/ExponentiallyDecayingReservoir.java
+ *
+ *
+ * [1]: http://dimacs.rutgers.edu/~graham/pubs/papers/fwddecay.pdf
+ * [2]: https://en.wikipedia.org/wiki/Half-life
+ * [3]:
https://github.com/dropwizard/metrics/blob/v3.1.2/metrics-core/src/main/java/com/codahale/metrics/ExponentiallyDecayingReservoir.java
+ *
*/
public class DecayingEstimatedHistogramReservoir implements Reservoir
{
@@ -522,6 +525,15 @@ public
[jira] [Updated] (CASSANDRA-15718) Improve BatchMetricsTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dinesh Joshi updated CASSANDRA-15718:
-
Status: Ready to Commit (was: Changes Suggested)
> Improve BatchMetricsTest
> -
>
> Key: CASSANDRA-15718
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15718
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Stephen Mallette
>Assignee: Stephen Mallette
>Priority: Normal
> Attachments: CASSANDRA-15582-test-cleanup.patch
>
>
> As noted in CASSANDRA-15582 {{BatchMetricsTest}} should test
> {{BatchStatement.Type.COUNTER}} to cover all the {{BatchMetrics}}. Some
> changes were introduced to make this improvement at:
> https://github.com/apache/cassandra/compare/trunk...spmallette:CASSANDRA-15582-trunk-batchmetrics
> and the following suggestions were made in review (in addition to the
> suggestion that a separate JIRA be created for this change) by [~dcapwell]:
> {quote}
> * I like the usage of BatchStatement.Type for the tests
> * honestly feel quick theories is better than random, but glad you added the
> seed to all asserts =). Would still be better as a quick theories test since
> you basically wrote a property anyways!
> *
> https://github.com/apache/cassandra/compare/trunk...spmallette:CASSANDRA-15582-trunk-batchmetrics#diff-8948cec1f9d33f10b15c38de80141548R131
> feel you should rename to expectedPartitionsPerLoggedBatch
> {Count,Logged,Unlogged}
> * . pre is what the value is, post is what the value is expected to be
> (rather than what it is).
> *
> *
> https://github.com/apache/cassandra/compare/trunk...spmallette:CASSANDRA-15582-trunk-batchmetrics#diff-8948cec1f9d33f10b15c38de80141548R150
> this doesn't look correct. the batch has distinctPartitions mutations, so
> shouldn't max reflect that? I ran the current test in a debugger and see that
> that is the case (aka current test is wrong).
> most of the comments are nit picks, but the last one looks like a test bug to
> me
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13994) Remove COMPACT STORAGE internals before 4.0 release
[
https://issues.apache.org/jira/browse/CASSANDRA-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094110#comment-17094110
]
Ekaterina Dimitrova commented on CASSANDRA-13994:
-
Current version of the patch
[here|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-13994]
Fixed the few tests which were failing.
Have some ccm issues when I tried to upgrade locally and check the indexes
defunct properly. Will try again tomorrow.
> Remove COMPACT STORAGE internals before 4.0 release
> ---
>
> Key: CASSANDRA-13994
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13994
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Local Write-Read Paths
>Reporter: Alex Petrov
>Assignee: Ekaterina Dimitrova
>Priority: Low
> Fix For: 4.0, 4.0-rc
>
>
> 4.0 comes without thrift (after [CASSANDRA-5]) and COMPACT STORAGE (after
> [CASSANDRA-10857]), and since Compact Storage flags are now disabled, all of
> the related functionality is useless.
> There are still some things to consider:
> 1. One of the system tables (built indexes) was compact. For now, we just
> added {{value}} column to it to make sure it's backwards-compatible, but we
> might want to make sure it's just a "normal" table and doesn't have redundant
> columns.
> 2. Compact Tables were building indexes in {{KEYS}} mode. Removing it is
> trivial, but this would mean that all built indexes will be defunct. We could
> log a warning for now and ask users to migrate off those for now and
> completely remove it from future releases. It's just a couple of classes
> though.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15262) server_encryption_options is not backwards compatible with 3.11
[
https://issues.apache.org/jira/browse/CASSANDRA-15262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094104#comment-17094104
]
Ekaterina Dimitrova edited comment on CASSANDRA-15262 at 4/28/20, 3:05 AM:
---
Looks like the tests in sslnodetonode_test.py fail - 5 out of 8 locally. I
guess they should be corrected too and it is not the patch but it's late now.
I am running now the full CI in CirclCI to get the full picture. I will
proceed tomorrow.
was (Author: e.dimitrova):
Looks like the tests in sslnodetonode_test.py fail - 5 out of 8 locally.
I am running now the full CI in CirclCI to get the full picture. Will proceed
tomorrow.
> server_encryption_options is not backwards compatible with 3.11
> ---
>
> Key: CASSANDRA-15262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15262
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Joey Lynch
>Assignee: Joey Lynch
>Priority: Normal
> Fix For: 4.0, 4.0-alpha
>
>
> The current `server_encryption_options` configuration options are as follows:
> {noformat}
> server_encryption_options:
> # set to true for allowing secure incoming connections
> enabled: false
> # If enabled and optional are both set to true, encrypted and unencrypted
> connections are handled on the storage_port
> optional: false
> # if enabled, will open up an encrypted listening socket on
> ssl_storage_port. Should be used
> # during upgrade to 4.0; otherwise, set to false.
> enable_legacy_ssl_storage_port: false
> # on outbound connections, determine which type of peers to securely
> connect to. 'enabled' must be set to true.
> internode_encryption: none
> keystore: conf/.keystore
> keystore_password: cassandra
> truststore: conf/.truststore
> truststore_password: cassandra
> # More advanced defaults below:
> # protocol: TLS
> # store_type: JKS
> # cipher_suites:
> [TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA]
> # require_client_auth: false
> # require_endpoint_verification: false
> {noformat}
> A couple of issues here:
> 1. optional defaults to false, which will break existing TLS configurations
> for (from what I can tell) no particularly good reason
> 2. The provided protocol and cipher suites are not good ideas (in particular
> encouraging anyone to use CBC ciphers is a bad plan
> I propose that before the 4.0 cut we fixup server_encryption_options and even
> client_encryption_options :
> # Change the default {{optional}} setting to true. As the new Netty code
> intelligently decides to open a TLS connection or not this is the more
> sensible default (saves operators a step while transitioning to TLS as well)
> # Update the defaults to what netty actually defaults to
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15262) server_encryption_options is not backwards compatible with 3.11
[
https://issues.apache.org/jira/browse/CASSANDRA-15262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094104#comment-17094104
]
Ekaterina Dimitrova commented on CASSANDRA-15262:
-
Looks like the tests in sslnodetonode_test.py fail - 5 out of 8 locally.
I am running now the full CI in CirclCI to get the full picture. Will proceed
tomorrow.
> server_encryption_options is not backwards compatible with 3.11
> ---
>
> Key: CASSANDRA-15262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15262
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Joey Lynch
>Assignee: Joey Lynch
>Priority: Normal
> Fix For: 4.0, 4.0-alpha
>
>
> The current `server_encryption_options` configuration options are as follows:
> {noformat}
> server_encryption_options:
> # set to true for allowing secure incoming connections
> enabled: false
> # If enabled and optional are both set to true, encrypted and unencrypted
> connections are handled on the storage_port
> optional: false
> # if enabled, will open up an encrypted listening socket on
> ssl_storage_port. Should be used
> # during upgrade to 4.0; otherwise, set to false.
> enable_legacy_ssl_storage_port: false
> # on outbound connections, determine which type of peers to securely
> connect to. 'enabled' must be set to true.
> internode_encryption: none
> keystore: conf/.keystore
> keystore_password: cassandra
> truststore: conf/.truststore
> truststore_password: cassandra
> # More advanced defaults below:
> # protocol: TLS
> # store_type: JKS
> # cipher_suites:
> [TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA]
> # require_client_auth: false
> # require_endpoint_verification: false
> {noformat}
> A couple of issues here:
> 1. optional defaults to false, which will break existing TLS configurations
> for (from what I can tell) no particularly good reason
> 2. The provided protocol and cipher suites are not good ideas (in particular
> encouraging anyone to use CBC ciphers is a bad plan
> I propose that before the 4.0 cut we fixup server_encryption_options and even
> client_encryption_options :
> # Change the default {{optional}} setting to true. As the new Netty code
> intelligently decides to open a TLS connection or not this is the more
> sensible default (saves operators a step while transitioning to TLS as well)
> # Update the defaults to what netty actually defaults to
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-22) RESTEasy integration for Cassandra Sidecar
[ https://issues.apache.org/jira/browse/CASSANDRASC-22?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094058#comment-17094058 ] Dinesh Joshi commented on CASSANDRASC-22: - sure, go for it [~yifanc]. > RESTEasy integration for Cassandra Sidecar > -- > > Key: CASSANDRASC-22 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-22 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Dinesh Joshi >Assignee: Dinesh Joshi >Priority: Normal > > Add support for JAX-RS based routing via RESTEasy to Cassandra Sidecar. This > also dynamically generates swagger documentation and adds the swagger UI. > [Branch|https://github.com/dineshjoshi/cassandra-sidecar/tree/resteasy-swagger] > [Tests|https://circleci.com/gh/dineshjoshi/cassandra-sidecar/tree/resteasy-swagger] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-22) RESTEasy integration for Cassandra Sidecar
[ https://issues.apache.org/jira/browse/CASSANDRASC-22?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094039#comment-17094039 ] Yifan Cai commented on CASSANDRASC-22: -- Need a second reviewer? > RESTEasy integration for Cassandra Sidecar > -- > > Key: CASSANDRASC-22 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-22 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Dinesh Joshi >Assignee: Dinesh Joshi >Priority: Normal > > Add support for JAX-RS based routing via RESTEasy to Cassandra Sidecar. This > also dynamically generates swagger documentation and adds the swagger UI. > [Branch|https://github.com/dineshjoshi/cassandra-sidecar/tree/resteasy-swagger] > [Tests|https://circleci.com/gh/dineshjoshi/cassandra-sidecar/tree/resteasy-swagger] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-22) RESTEasy integration for Cassandra Sidecar
[ https://issues.apache.org/jira/browse/CASSANDRASC-22?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Haddad updated CASSANDRASC-22: -- Reviewers: Jon Haddad Status: Review In Progress (was: Patch Available) Reviewing (still don't have Reviewers field working properly) > RESTEasy integration for Cassandra Sidecar > -- > > Key: CASSANDRASC-22 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-22 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Dinesh Joshi >Assignee: Dinesh Joshi >Priority: Normal > > Add support for JAX-RS based routing via RESTEasy to Cassandra Sidecar. This > also dynamically generates swagger documentation and adds the swagger UI. > [Branch|https://github.com/dineshjoshi/cassandra-sidecar/tree/resteasy-swagger] > [Tests|https://circleci.com/gh/dineshjoshi/cassandra-sidecar/tree/resteasy-swagger] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-22) RESTEasy integration for Cassandra Sidecar
[ https://issues.apache.org/jira/browse/CASSANDRASC-22?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi updated CASSANDRASC-22: Description: Add support for JAX-RS based routing via RESTEasy to Cassandra Sidecar. This also dynamically generates swagger documentation and adds the swagger UI. [Branch|https://github.com/dineshjoshi/cassandra-sidecar/tree/resteasy-swagger] [Tests|https://circleci.com/gh/dineshjoshi/cassandra-sidecar/tree/resteasy-swagger] was: Add support for JAX-RS based routing via RESTEasy to Cassandra Sidecar. This also dynamically generates swagger documentation and adds the swagger UI. [Branch|https://github.com/dineshjoshi/cassandra-sidecar/tree/resteasy-swagger] > RESTEasy integration for Cassandra Sidecar > -- > > Key: CASSANDRASC-22 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-22 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Dinesh Joshi >Assignee: Dinesh Joshi >Priority: Normal > > Add support for JAX-RS based routing via RESTEasy to Cassandra Sidecar. This > also dynamically generates swagger documentation and adds the swagger UI. > [Branch|https://github.com/dineshjoshi/cassandra-sidecar/tree/resteasy-swagger] > [Tests|https://circleci.com/gh/dineshjoshi/cassandra-sidecar/tree/resteasy-swagger] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15765) Get-by-index introduced in CASSANDRA-15394 could have negative performance impact on non-RandomAccess List
[
https://issues.apache.org/jira/browse/CASSANDRA-15765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093980#comment-17093980
]
Yifan Cai commented on CASSANDRA-15765:
---
It is a potential performance issue. Those methods' signatures accepts
{{List}}, meanwhile the implementations assume the type of input is {{List}}
with {{RandomAccess}}.
The current code runs just fine. The input should be all array-based lists, as
far as I can see.
> Get-by-index introduced in CASSANDRA-15394 could have negative performance
> impact on non-RandomAccess List
> --
>
> Key: CASSANDRA-15765
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15765
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Core
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
>
> CASSANDRA-15394 replaced the iterator based iteration with the get-by-index
> one to avoid allocation iterators.
> It works for the lists that support RandomAccess, i.e. the big O of {{get()}}
> is {{O(1)}}.
> However, it fails when the list does not support RandomAccess. The {{get()}}
> method's time complexity can be linear, and it leads to {{O(n^2)}} for the
> overall iteration.
> The implementation should provide different behaviors based on the property.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15765) Get-by-index introduced in CASSANDRA-15394 could have negative performance impact on non-RandomAccess List
[
https://issues.apache.org/jira/browse/CASSANDRA-15765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093976#comment-17093976
]
David Capwell commented on CASSANDRA-15765:
---
is there a code path you are seeing which doesn't use a list which supports
random access? Is this a potential performance issue or a monitored issue (and
if so under which code path)?
> Get-by-index introduced in CASSANDRA-15394 could have negative performance
> impact on non-RandomAccess List
> --
>
> Key: CASSANDRA-15765
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15765
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Core
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
>
> CASSANDRA-15394 replaced the iterator based iteration with the get-by-index
> one to avoid allocation iterators.
> It works for the lists that support RandomAccess, i.e. the big O of {{get()}}
> is {{O(1)}}.
> However, it fails when the list does not support RandomAccess. The {{get()}}
> method's time complexity can be linear, and it leads to {{O(n^2)}} for the
> overall iteration.
> The implementation should provide different behaviors based on the property.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15762) Update Bundled Python Driver to the latest release
[ https://issues.apache.org/jira/browse/CASSANDRA-15762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-15762: -- Change Category: Semantic Complexity: Normal Component/s: Tool/cqlsh Status: Open (was: Triage Needed) > Update Bundled Python Driver to the latest release > -- > > Key: CASSANDRA-15762 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15762 > Project: Cassandra > Issue Type: Task > Components: Tool/cqlsh >Reporter: Alan Boudreault >Priority: Normal > Fix For: 4.0-rc > > > This is not urgent for now since there are some missing 4.x features that > need to be implemented on the driver (checksum). This ticket is to make sure > we update the bundled driver before GA. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch master updated: ninja-fix: undo https://github.com/apache/cassandra-website/commit/48a7cd3a4d3c68751f43a25bf42a8ffc813b9e56#diff-60713fb19f6c50a2473835df63ff17f9 (only the l
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-website.git
The following commit(s) were added to refs/heads/master by this push:
new 9bc538d ninja-fix: undo
https://github.com/apache/cassandra-website/commit/48a7cd3a4d3c68751f43a25bf42a8ffc813b9e56#diff-60713fb19f6c50a2473835df63ff17f9
(only the latest 3.11 docs are to get generated (as well as trunk))
9bc538d is described below
commit 9bc538d3a67609984364ded9532e35a50a107f61
Author: mck
AuthorDate: Mon Apr 27 22:20:44 2020 +0200
ninja-fix: undo
https://github.com/apache/cassandra-website/commit/48a7cd3a4d3c68751f43a25bf42a8ffc813b9e56#diff-60713fb19f6c50a2473835df63ff17f9
(only the latest 3.11 docs are to get generated (as well as trunk))
---
docker-entrypoint.sh | 11 ---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/docker-entrypoint.sh b/docker-entrypoint.sh
index 658be92..02b5afd 100755
--- a/docker-entrypoint.sh
+++ b/docker-entrypoint.sh
@@ -17,9 +17,14 @@ make add-latest-doc
# Make sure we have the latest commit of Cassandra 3.11
-pushd ${CASSANDRA_DIR} ; ant realclean ; git checkout cassandra-3.11.5 ; popd
; make .build-doc
-pushd ${CASSANDRA_DIR} ; ant realclean ; git checkout cassandra-4.0-alpha1 ;
popd ; make .build-doc
-pushd ${CASSANDRA_DIR} ; ant realclean ; git checkout cassandra-4.0-alpha2 ;
popd ; make .build-doc
+pushd ${CASSANDRA_DIR}
+ant realclean
+git checkout cassandra-3.11
+git pull --rebase --prune
+popd
+
+# Now make the docs for 3.11
+make .build-doc
# Relink the 3.11 version
LATEST_VERSION=$(basename $(find ./doc -iname 3.11* -type d | tail -n 1))
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-22) RESTEasy integration for Cassandra Sidecar
[ https://issues.apache.org/jira/browse/CASSANDRASC-22?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi updated CASSANDRASC-22: Authors: Dinesh Joshi Test and Documentation Plan: Existing test cases Status: Patch Available (was: Open) > RESTEasy integration for Cassandra Sidecar > -- > > Key: CASSANDRASC-22 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-22 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Dinesh Joshi >Assignee: Dinesh Joshi >Priority: Normal > > Add support for JAX-RS based routing via RESTEasy to Cassandra Sidecar. This > also dynamically generates swagger documentation and adds the swagger UI. > [Branch|https://github.com/dineshjoshi/cassandra-sidecar/tree/resteasy-swagger] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-22) RESTEasy integration for Cassandra Sidecar
[ https://issues.apache.org/jira/browse/CASSANDRASC-22?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi updated CASSANDRASC-22: Change Category: Code Clarity Complexity: Low Hanging Fruit Status: Open (was: Triage Needed) > RESTEasy integration for Cassandra Sidecar > -- > > Key: CASSANDRASC-22 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-22 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Dinesh Joshi >Assignee: Dinesh Joshi >Priority: Normal > > Add support for JAX-RS based routing via RESTEasy to Cassandra Sidecar. This > also dynamically generates swagger documentation and adds the swagger UI. > [Branch|https://github.com/dineshjoshi/cassandra-sidecar/tree/resteasy-swagger] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRASC-22) RESTEasy integration for Cassandra Sidecar
Dinesh Joshi created CASSANDRASC-22: --- Summary: RESTEasy integration for Cassandra Sidecar Key: CASSANDRASC-22 URL: https://issues.apache.org/jira/browse/CASSANDRASC-22 Project: Sidecar for Apache Cassandra Issue Type: Improvement Components: Rest API Reporter: Dinesh Joshi Assignee: Dinesh Joshi Add support for JAX-RS based routing via RESTEasy to Cassandra Sidecar. This also dynamically generates swagger documentation and adds the swagger UI. [Branch|https://github.com/dineshjoshi/cassandra-sidecar/tree/resteasy-swagger] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15765) Get-by-index introduced in CASSANDRA-15394 could have negative performance impact on non-RandomAccess List
[
https://issues.apache.org/jira/browse/CASSANDRA-15765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifan Cai updated CASSANDRA-15765:
--
Description:
CASSANDRA-15394 replaced the iterator based iteration with the get-by-index one
to avoid allocation iterators.
It works for the lists that support RandomAccess, i.e. the big O of {{get()}}
is {{O(1)}}.
However, it fails when the list does not support RandomAccess. The {{get()}}
method's time complexity can be linear, and it leads to {{O(n^2)}} for the
overall iteration.
The implementation should provide different behaviors based on the property.
was:
CASSANDRA-15394 replaced the iterator based iteration with the get-by-index one
to avoid allocation iterators.
It works for the lists that support RandomAccess, i.e. the big O of {{get()}}
is {{O(1)}}.
However, it fails when the list does not support RandomAccess. The {{get()}}
method's time complexity can be {{O(n)}}, and it leads to {{O(n^2)}} for the
overall iteration.
The implementation should provide different behaviors based on the property.
> Get-by-index introduced in CASSANDRA-15394 could have negative performance
> impact on non-RandomAccess List
> --
>
> Key: CASSANDRA-15765
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15765
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Core
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
>
> CASSANDRA-15394 replaced the iterator based iteration with the get-by-index
> one to avoid allocation iterators.
> It works for the lists that support RandomAccess, i.e. the big O of {{get()}}
> is {{O(1)}}.
> However, it fails when the list does not support RandomAccess. The {{get()}}
> method's time complexity can be linear, and it leads to {{O(n^2)}} for the
> overall iteration.
> The implementation should provide different behaviors based on the property.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15756) in-jvm dtest IInstance and ICoordinator should use QueryResult as the base API
[
https://issues.apache.org/jira/browse/CASSANDRA-15756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093877#comment-17093877
]
Alex Petrov commented on CASSANDRA-15756:
-
Committed an in-jvm dtest API change with
[cc3e43c710a0fb683b7e955f641e221ccc2e5d54|https://github.com/apache/cassandra-in-jvm-dtest-api/commit/cc3e43c710a0fb683b7e955f641e221ccc2e5d54]
and released {{0.0.2-f2dbed3-SNAPSHOT}}.
> in-jvm dtest IInstance and ICoordinator should use QueryResult as the base API
> --
>
> Key: CASSANDRA-15756
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15756
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> ICoordinator was modified to have a executeWithResult which returns a
> QueryResult, but not all query APIs have this. We should fix it so the base
> interface is executeWithResult and return QueryResult, but update all
> Object[][] methods to default to calling this.
> This would be a breaking change so would need to be updated in each branch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-in-jvm-dtest-api] 01/02: In-jvm dtest IInstance and ICoordinator should use QueryResult as the base API
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-in-jvm-dtest-api.git
commit cc3e43c710a0fb683b7e955f641e221ccc2e5d54
Author: David Capwell
AuthorDate: Wed Apr 22 19:14:24 2020 -0700
In-jvm dtest IInstance and ICoordinator should use QueryResult as the base
API
Patch by David Capwell; reviewed by Alex Petrov for CASSANDRA-15756.
---
pom.xml| 23 +++
.../cassandra/distributed/api/ICoordinator.java| 28 ++-
.../cassandra/distributed/api/IInstance.java | 7 +-
.../cassandra/distributed/api/NodeToolResult.java | 3 +-
.../cassandra/distributed/api/QueryResult.java | 91 ++---
.../cassandra/distributed/api/QueryResults.java| 204 +
.../org/apache/cassandra/distributed/api/Row.java | 116 ++--
.../{QueryResult.java => SimpleQueryResult.java} | 46 +++--
.../cassandra/distributed/shared/AssertUtils.java | 37
.../cassandra/distributed/shared/FutureUtils.java | 95 ++
.../cassandra/distributed/api/QueryResultTest.java | 198
11 files changed, 740 insertions(+), 108 deletions(-)
diff --git a/pom.xml b/pom.xml
index df733c6..62c90d8 100644
--- a/pom.xml
+++ b/pom.xml
@@ -50,6 +50,25 @@
slf4j-api
1.7.25
+
+
+org.junit.jupiter
+junit-jupiter-api
+5.6.2
+test
+
+
+org.junit.jupiter
+junit-jupiter-engine
+5.6.2
+test
+
+
+org.assertj
+assertj-core
+3.15.0
+test
+
@@ -65,6 +84,10 @@
+
+maven-surefire-plugin
+2.22.2
+
diff --git
a/src/main/java/org/apache/cassandra/distributed/api/ICoordinator.java
b/src/main/java/org/apache/cassandra/distributed/api/ICoordinator.java
index 34087d0..3a96b63 100644
--- a/src/main/java/org/apache/cassandra/distributed/api/ICoordinator.java
+++ b/src/main/java/org/apache/cassandra/distributed/api/ICoordinator.java
@@ -22,6 +22,8 @@ import java.util.Iterator;
import java.util.UUID;
import java.util.concurrent.Future;
+import org.apache.cassandra.distributed.shared.FutureUtils;
+
// The cross-version API requires that a Coordinator can be constructed
without any constructor arguments
public interface ICoordinator
{
@@ -30,13 +32,31 @@ public interface ICoordinator
return executeWithResult(query, consistencyLevel,
boundValues).toObjectArrays();
}
-QueryResult executeWithResult(String query, ConsistencyLevel
consistencyLevel, Object... boundValues);
+SimpleQueryResult executeWithResult(String query, ConsistencyLevel
consistencyLevel, Object... boundValues);
+
+default Iterator executeWithPaging(String query,
ConsistencyLevel consistencyLevel, int pageSize, Object... boundValues)
+{
+return executeWithPagingWithResult(query, consistencyLevel, pageSize,
boundValues).map(Row::toObjectArray);
+}
+
+QueryResult executeWithPagingWithResult(String query, ConsistencyLevel
consistencyLevel, int pageSize, Object... boundValues);
+
+default Future asyncExecuteWithTracing(UUID sessionId, String
query, ConsistencyLevel consistencyLevel, Object... boundValues)
+{
+return FutureUtils.map(asyncExecuteWithTracingWithResult(sessionId,
query, consistencyLevel, boundValues), r -> r.toObjectArrays());
+}
-Iterator executeWithPaging(String query, ConsistencyLevel
consistencyLevel, int pageSize, Object... boundValues);
+Future asyncExecuteWithTracingWithResult(UUID
sessionId, String query, ConsistencyLevel consistencyLevel, Object...
boundValues);
-Future asyncExecuteWithTracing(UUID sessionId, String query,
ConsistencyLevel consistencyLevel, Object... boundValues);
+default Object[][] executeWithTracing(UUID sessionId, String query,
ConsistencyLevel consistencyLevel, Object... boundValues)
+{
+return executeWithTracingWithResult(sessionId, query,
consistencyLevel, boundValues).toObjectArrays();
+}
-Object[][] executeWithTracing(UUID sessionId, String query,
ConsistencyLevel consistencyLevel, Object... boundValues);
+default SimpleQueryResult executeWithTracingWithResult(UUID sessionId,
String query, ConsistencyLevel consistencyLevel, Object... boundValues)
+{
+return FutureUtils.waitOn(asyncExecuteWithTracingWithResult(sessionId,
query, consistencyLevel, boundValues));
+}
IInstance instance();
}
diff --git a/src/main/java/org/apache/cassandra/distributed/api/IInstance.java
b/src/main/java/org/apache/cassandra/distributed/api/IInstance.java
index 90c8242..8895d94
[cassandra-in-jvm-dtest-api] 02/02: Add CHANGES file
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/cassandra-in-jvm-dtest-api.git commit f2dbed37bb2a1cc070e7ff9296cb87983eb777ca Author: Alex Petrov AuthorDate: Mon Apr 27 21:28:05 2020 +0200 Add CHANGES file --- CHANGES.txt | 7 +++ 1 file changed, 7 insertions(+) diff --git a/CHANGES.txt b/CHANGES.txt new file mode 100644 index 000..59f056e --- /dev/null +++ b/CHANGES.txt @@ -0,0 +1,7 @@ +# 0.0.2 + +CASSANDRA-15684: Improve error codes in NodeToolResult to produce better errors and to allow Any style message checks +CASSANDRA-15713: Make shared class filter for InstanceClassLoader pluggable +CASSANDRA-15714: Add support for replacing logback with alternate logger config (like log4j2) +CASSANDRA-15756: In-jvm dtest IInstance and ICoordinator should use QueryResult as the base API +CASSANDRA-15733: Cluster builder should be provided to the factory and expose state - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-in-jvm-dtest-api] branch master updated (326045f -> f2dbed3)
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a change to branch master
in repository
https://gitbox.apache.org/repos/asf/cassandra-in-jvm-dtest-api.git.
from 326045f Cluster builder should be provided to the factory and expose
state
new cc3e43c In-jvm dtest IInstance and ICoordinator should use
QueryResult as the base API
new f2dbed3 Add CHANGES file
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
CHANGES.txt| 7 +
pom.xml| 23 +++
.../cassandra/distributed/api/ICoordinator.java| 28 ++-
.../cassandra/distributed/api/IInstance.java | 7 +-
.../cassandra/distributed/api/NodeToolResult.java | 3 +-
.../cassandra/distributed/api/QueryResult.java | 91 ++---
.../cassandra/distributed/api/QueryResults.java| 204 +
.../org/apache/cassandra/distributed/api/Row.java | 116 ++--
.../{QueryResult.java => SimpleQueryResult.java} | 46 +++--
.../cassandra/distributed/shared/AssertUtils.java | 37
.../cassandra/distributed/shared/FutureUtils.java | 95 ++
.../cassandra/distributed/api/QueryResultTest.java | 198
12 files changed, 747 insertions(+), 108 deletions(-)
create mode 100644 CHANGES.txt
create mode 100644
src/main/java/org/apache/cassandra/distributed/api/QueryResults.java
copy src/main/java/org/apache/cassandra/distributed/api/{QueryResult.java =>
SimpleQueryResult.java} (76%)
create mode 100644
src/main/java/org/apache/cassandra/distributed/shared/FutureUtils.java
create mode 100644
src/test/java/org/apache/cassandra/distributed/api/QueryResultTest.java
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15729) Jenkins Test Results Report in plaintext for ASF ML
[
https://issues.apache.org/jira/browse/CASSANDRA-15729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093874#comment-17093874
]
Michael Semb Wever commented on CASSANDRA-15729:
Thanks for the review [~dcapwell]. Looks like you jumped a bit in front of me,
as some of those changes were still wip. Only the stuff
[here|https://github.com/thelastpickle/cassandra/commit/ed0a7e7e2f3e2a67825b8d560ee9ffbb60983336]
is ready to go (and needs to go out first in advance).
But I can reply to your comments anyway :-)
The `ant jar` is only temporarily commented out. The `apt install` stuff will
happen using docker instead. The clean block will get removed (if it doesn't
get used). The `cassandra-test-report.sh` scripts is available because on the
line before the `${buildRepo}` (ie cassandra-builds) is cloned. And the
calculations for the 1MB limit was roughly what I did too. And there's no way
to know if we over, unless the jenkins email plugin has some of check (before
sending).
> Jenkins Test Results Report in plaintext for ASF ML
> ---
>
> Key: CASSANDRA-15729
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15729
> Project: Cassandra
> Issue Type: Task
> Components: Build, CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Labels: Jenkins
> Fix For: 4.0-beta
>
>
> The Jenkins pipeline builds now aggregate all test reports.
> For example:
> - https://ci-cassandra.apache.org/job/Cassandra-trunk/68/testReport/
> -
> https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-trunk/detail/Cassandra-trunk/68/tests
> But Jenkins can only keep a limited amount of build history, so those links
> are not permanent, can't be used as references, and don't help for bisecting
> and blame on regressions (and flakey tests) over a longer period of time.
> The builds@ ML can provide a permanent record of test results.
> This was first brought up in these two threads:
> -
> https://lists.apache.org/thread.html/re8122e4fdd8629e7fbca2abf27d72054b3bc0e3690ece8b8e66f618b%40%3Cdev.cassandra.apache.org%3E
> -
> https://lists.apache.org/thread.html/ra5f6aeea89546825fe7ccc4a80898c62f8ed57decabf709d81d9c720%40%3Cdev.cassandra.apache.org%3E
> An example plaintext report, to demonstrate feasibility, is available here:
> https://lists.apache.org/thread.html/r80d13f7af706bf8dfbf2387fab46004c1fbd3917b7bc339c49e69aa8%40%3Cbuilds.cassandra.apache.org%3E
> Hurdles:
> - the ASF mailing lists won't accept html, attachments, or any message body
> over 1MB.
> - packages are used as a differentiator in the final aggregated report. The
> cqlsh and dtests currently don't specify it. It needs to be added as a
> "dot-separated" prefix to the testsuite and testcase name.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15262) server_encryption_options is not backwards compatible with 3.11
[
https://issues.apache.org/jira/browse/CASSANDRA-15262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093869#comment-17093869
]
Ekaterina Dimitrova commented on CASSANDRA-15262:
-
Setter added to solve the issue:
[branch|https://github.com/jolynch/cassandra/commits/CASSANDRA-15262]
[commit|https://github.com/jolynch/cassandra/commit/2eea573e63b3d6505e1d79bf552c9a2867f10f14]
> server_encryption_options is not backwards compatible with 3.11
> ---
>
> Key: CASSANDRA-15262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15262
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Joey Lynch
>Assignee: Joey Lynch
>Priority: Normal
> Fix For: 4.0, 4.0-alpha
>
>
> The current `server_encryption_options` configuration options are as follows:
> {noformat}
> server_encryption_options:
> # set to true for allowing secure incoming connections
> enabled: false
> # If enabled and optional are both set to true, encrypted and unencrypted
> connections are handled on the storage_port
> optional: false
> # if enabled, will open up an encrypted listening socket on
> ssl_storage_port. Should be used
> # during upgrade to 4.0; otherwise, set to false.
> enable_legacy_ssl_storage_port: false
> # on outbound connections, determine which type of peers to securely
> connect to. 'enabled' must be set to true.
> internode_encryption: none
> keystore: conf/.keystore
> keystore_password: cassandra
> truststore: conf/.truststore
> truststore_password: cassandra
> # More advanced defaults below:
> # protocol: TLS
> # store_type: JKS
> # cipher_suites:
> [TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA]
> # require_client_auth: false
> # require_endpoint_verification: false
> {noformat}
> A couple of issues here:
> 1. optional defaults to false, which will break existing TLS configurations
> for (from what I can tell) no particularly good reason
> 2. The provided protocol and cipher suites are not good ideas (in particular
> encouraging anyone to use CBC ciphers is a bad plan
> I propose that before the 4.0 cut we fixup server_encryption_options and even
> client_encryption_options :
> # Change the default {{optional}} setting to true. As the new Netty code
> intelligently decides to open a TLS connection or not this is the more
> sensible default (saves operators a step while transitioning to TLS as well)
> # Update the defaults to what netty actually defaults to
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15158) Wait for schema agreement rather then in flight schema requests when bootstrapping
[ https://issues.apache.org/jira/browse/CASSANDRA-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093752#comment-17093752 ] Stefan Miklosovic edited comment on CASSANDRA-15158 at 4/27/20, 7:19 PM: - Hi [~bdeggleston] I took the patch and rewored it little bit [https://github.com/smiklosovic/cassandra/commits/CASSANDRA-15158] You have said that there is not any way how to confirm that any in flight migration tasks have been completed and applied. I do not know how to verify this was indeed done but what I did is that I have exposed the information if a particular migration task has failed or not to process (based on onFailure callback) so we can work on this further if necessary. Logically, it is same stuff as it was before in the original patch but the code is reorganised a bit. The "escape hatch" is one global bootstrap timeout, if it is passed and schemas are still not in agreement, it is still uknown to me what we want to do - either fail completely and halt that node or we allow to proceed with big fat warning. Looking forward to have some feedback! was (Author: stefan.miklosovic): Hi [~bdeggleston] I took the patch and rewored it little bit [https://github.com/smiklosovic/cassandra/commits/CASSANDRA-15158] You have said that there is not any way how to confirm that any in flight migration tasks have been completed and applied. I do not know how to verify this was indeed done but what I did is that I have exposed the information if a particular migration task has failed or nor to process (based on onFailure callback) so we can work on this further if necessary. Logically, it is same stuff as it was before in the original patch but the code is reorganised a bit. The "escape hatch" is one global bootstrap timeout, if it is passed and schemas are still not in agreement, it is still uknown to me what we want to do - either fail completely and halt that service or we allow to proceed with big fat warning. Looking forward to have some feedback! > Wait for schema agreement rather then in flight schema requests when > bootstrapping > -- > > Key: CASSANDRA-15158 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15158 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip, Cluster/Schema >Reporter: Vincent White >Assignee: Ben Bromhead >Priority: Normal > > Currently when a node is bootstrapping we use a set of latches > (org.apache.cassandra.service.MigrationTask#inflightTasks) to keep track of > in-flight schema pull requests, and we don't proceed with > bootstrapping/stream until all the latches are released (or we timeout > waiting for each one). One issue with this is that if we have a large schema, > or the retrieval of the schema from the other nodes was unexpectedly slow > then we have no explicit check in place to ensure we have actually received a > schema before we proceed. > While it's possible to increase "migration_task_wait_in_seconds" to force the > node to wait on each latche longer, there are cases where this doesn't help > because the callbacks for the schema pull requests have expired off the > messaging service's callback map > (org.apache.cassandra.net.MessagingService#callbacks) after > request_timeout_in_ms (default 10 seconds) before the other nodes were able > to respond to the new node. > This patch checks for schema agreement between the bootstrapping node and the > rest of the live nodes before proceeding with bootstrapping. It also adds a > check to prevent the new node from flooding existing nodes with simultaneous > schema pull requests as can happen in large clusters. > Removing the latch system should also prevent new nodes in large clusters > getting stuck for extended amounts of time as they wait > `migration_task_wait_in_seconds` on each of the latches left orphaned by the > timed out callbacks. > > ||3.11|| > |[PoC|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:check_for_schema]| > |[dtest|https://github.com/apache/cassandra-dtest/compare/master...vincewhite:wait_for_schema_agreement]| > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15729) Jenkins Test Results Report in plaintext for ASF ML
[ https://issues.apache.org/jira/browse/CASSANDRA-15729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093858#comment-17093858 ] David Capwell commented on CASSANDRA-15729: --- Cassandra Changes * https://github.com/thelastpickle/cassandra/commit/ed0a7e7e2f3e2a67825b8d560ee9ffbb60983336#diff-90e40e02845884b66e9006b25250ea5cR49. Why comment out the ant jar command? Build Changes * https://github.com/apache/cassandra-builds/compare/master...thelastpickle:mck/jenkins-test-report-format#diff-ed43728c5b4a5f326dc69a04e9cf57edR9 should we install this outside the build? feels wrong to me to install packages as part of a build. * https://github.com/apache/cassandra-builds/compare/master...thelastpickle:mck/jenkins-test-report-format#diff-ed43728c5b4a5f326dc69a04e9cf57edR27 dead block? * https://github.com/apache/cassandra-builds/compare/master...thelastpickle:mck/jenkins-test-report-format#diff-a76ac7fa36779199fe7e4fe917ac41bcR233 will unified file be accessible cross builds, or only able to view the workspace for the last build? in the linked build I don't have access to look at the file (if it was generated) For the build-scripts/cassandra-test-report.xsl file, I need to go learn that in order to review it... right now I can only black box review by trying to look at the output. cassandra-test-report.txt is limited to 900KB, which leaves just under 100KB for https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_15729#diff-9b83915ce4a53a5b3cc01b9f39d6a984R357-R367. ran a hacky test to show that 100KB is roughly 5000 times "this is just a test", so if we assume all test have a name of 4x that, that leaves 1,250 space for changes, so this looks fine to me. I assume though Jenkins won't know the email was rejected since I normally expect that to be a reply back email... > Jenkins Test Results Report in plaintext for ASF ML > --- > > Key: CASSANDRA-15729 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15729 > Project: Cassandra > Issue Type: Task > Components: Build, CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Labels: Jenkins > Fix For: 4.0-beta > > > The Jenkins pipeline builds now aggregate all test reports. > For example: > - https://ci-cassandra.apache.org/job/Cassandra-trunk/68/testReport/ > - > https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-trunk/detail/Cassandra-trunk/68/tests > But Jenkins can only keep a limited amount of build history, so those links > are not permanent, can't be used as references, and don't help for bisecting > and blame on regressions (and flakey tests) over a longer period of time. > The builds@ ML can provide a permanent record of test results. > This was first brought up in these two threads: > - > https://lists.apache.org/thread.html/re8122e4fdd8629e7fbca2abf27d72054b3bc0e3690ece8b8e66f618b%40%3Cdev.cassandra.apache.org%3E > - > https://lists.apache.org/thread.html/ra5f6aeea89546825fe7ccc4a80898c62f8ed57decabf709d81d9c720%40%3Cdev.cassandra.apache.org%3E > An example plaintext report, to demonstrate feasibility, is available here: > https://lists.apache.org/thread.html/r80d13f7af706bf8dfbf2387fab46004c1fbd3917b7bc339c49e69aa8%40%3Cbuilds.cassandra.apache.org%3E > Hurdles: > - the ASF mailing lists won't accept html, attachments, or any message body > over 1MB. > - packages are used as a differentiator in the final aggregated report. The > cqlsh and dtests currently don't specify it. It needs to be added as a > "dot-separated" prefix to the testsuite and testcase name. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15765) Get-by-index introduced in CASSANDRA-15394 could have negative performance impact on non-RandomAccess List
[
https://issues.apache.org/jira/browse/CASSANDRA-15765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifan Cai updated CASSANDRA-15765:
--
Change Category: Performance
Complexity: Normal
Status: Open (was: Triage Needed)
> Get-by-index introduced in CASSANDRA-15394 could have negative performance
> impact on non-RandomAccess List
> --
>
> Key: CASSANDRA-15765
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15765
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Core
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
>
> CASSANDRA-15394 replaced the iterator based iteration with the get-by-index
> one to avoid allocation iterators.
> It works for the lists that support RandomAccess, i.e. the big O of {{get()}}
> is {{O(1)}}.
> However, it fails when the list does not support RandomAccess. The {{get()}}
> method's time complexity can be {{O(n)}}, and it leads to {{O(n^2)}} for the
> overall iteration.
> The implementation should provide different behaviors based on the property.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15765) Get-by-index introduced in CASSANDRA-15394 could have negative performance impact on non-RandomAccess List
[
https://issues.apache.org/jira/browse/CASSANDRA-15765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifan Cai updated CASSANDRA-15765:
--
Description:
CASSANDRA-15394 replaced the iterator based iteration with the get-by-index one
to avoid allocation iterators.
It works for the lists that support RandomAccess, i.e. the big O of {{get()}}
is {{O(1)}}.
However, it fails when the list does not support RandomAccess. The {{get()}}
method's time complexity can be {{O(n)}}, and it leads to {{O(n^2)}} for the
overall iteration.
The implementation should provide different behaviors based on the property.
> Get-by-index introduced in CASSANDRA-15394 could have negative performance
> impact on non-RandomAccess List
> --
>
> Key: CASSANDRA-15765
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15765
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Core
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
>
> CASSANDRA-15394 replaced the iterator based iteration with the get-by-index
> one to avoid allocation iterators.
> It works for the lists that support RandomAccess, i.e. the big O of {{get()}}
> is {{O(1)}}.
> However, it fails when the list does not support RandomAccess. The {{get()}}
> method's time complexity can be {{O(n)}}, and it leads to {{O(n^2)}} for the
> overall iteration.
> The implementation should provide different behaviors based on the property.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15765) Get-by-index introduced in CASSANDRA-15394 could have negative performance impact on non-RandomAccess List
Yifan Cai created CASSANDRA-15765: - Summary: Get-by-index introduced in CASSANDRA-15394 could have negative performance impact on non-RandomAccess List Key: CASSANDRA-15765 URL: https://issues.apache.org/jira/browse/CASSANDRA-15765 Project: Cassandra Issue Type: Improvement Components: Legacy/Core Reporter: Yifan Cai Assignee: Yifan Cai -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15764) Optimize logging with lazy log parameter evaluation
[ https://issues.apache.org/jira/browse/CASSANDRA-15764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRA-15764: -- Change Category: Performance Complexity: Normal Component/s: Observability/Logging Status: Open (was: Triage Needed) > Optimize logging with lazy log parameter evaluation > --- > > Key: CASSANDRA-15764 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15764 > Project: Cassandra > Issue Type: Improvement > Components: Observability/Logging >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > > Multiple logging statements in the source evaluate methods and use the return > values as log parameter. > It does no harm when the log level permits for the statement. > However, for the logs that are permitted, the evaluation is a wastes of CPU. > The log parameters are not needed, but still being evaluated! > For such cases, lazy evaluation (by leveraging lambda) can be introduced to > skip competing the string parameter. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15764) Optimize logging with lazy log parameter evaluation
Yifan Cai created CASSANDRA-15764: - Summary: Optimize logging with lazy log parameter evaluation Key: CASSANDRA-15764 URL: https://issues.apache.org/jira/browse/CASSANDRA-15764 Project: Cassandra Issue Type: Improvement Reporter: Yifan Cai Assignee: Yifan Cai Multiple logging statements in the source evaluate methods and use the return values as log parameter. It does no harm when the log level permits for the statement. However, for the logs that are permitted, the evaluation is a wastes of CPU. The log parameters are not needed, but still being evaluated! For such cases, lazy evaluation (by leveraging lambda) can be introduced to skip competing the string parameter. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15763) Tunable RangeTombstoneList initial size and resize factor
Yifan Cai created CASSANDRA-15763: - Summary: Tunable RangeTombstoneList initial size and resize factor Key: CASSANDRA-15763 URL: https://issues.apache.org/jira/browse/CASSANDRA-15763 Project: Cassandra Issue Type: Improvement Components: Local/Config Reporter: Yifan Cai Assignee: Yifan Cai Cassandra allocates a backing array with a fixed size of one when creating a new RangeTombstoneList object. The resize factor is fixed to 1.5. The initial size and resize factor could be made tunable so it could reduce cost in some cases. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15763) Tunable RangeTombstoneList initial size and resize factor
[ https://issues.apache.org/jira/browse/CASSANDRA-15763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRA-15763: -- Change Category: Operability Complexity: Low Hanging Fruit Status: Open (was: Triage Needed) > Tunable RangeTombstoneList initial size and resize factor > - > > Key: CASSANDRA-15763 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15763 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > > Cassandra allocates a backing array with a fixed size of one when creating a > new RangeTombstoneList object. The resize factor is fixed to 1.5. > The initial size and resize factor could be made tunable so it could reduce > cost in some cases. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15762) Update Bundled Python Driver to the latest release
[ https://issues.apache.org/jira/browse/CASSANDRA-15762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Boudreault updated CASSANDRA-15762: Fix Version/s: 4.0-rc > Update Bundled Python Driver to the latest release > -- > > Key: CASSANDRA-15762 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15762 > Project: Cassandra > Issue Type: Task >Reporter: Alan Boudreault >Priority: Normal > Fix For: 4.0-rc > > > This is not urgent for now since there are some missing 4.x features that > need to be implemented on the driver (checksum). This ticket is to make sure > we update the bundled driver before GA. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15762) Update Bundled Python Driver to the latest release
Alan Boudreault created CASSANDRA-15762: --- Summary: Update Bundled Python Driver to the latest release Key: CASSANDRA-15762 URL: https://issues.apache.org/jira/browse/CASSANDRA-15762 Project: Cassandra Issue Type: Task Reporter: Alan Boudreault This is not urgent for now since there are some missing 4.x features that need to be implemented on the driver (checksum). This ticket is to make sure we update the bundled driver before GA. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15761) Make SASI query safe for concurrent compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-15761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-15761:
Bug Category: Parent values: Correctness(12982)Level 1 values: Recoverable
Corruption / Loss(12986)
Complexity: Normal
Component/s: (was: Feature/2i Index)
Feature/SASI
Discovered By: Unit Test
Severity: Critical
Status: Open (was: Triage Needed)
> Make SASI query safe for concurrent compaction
> --
>
> Key: CASSANDRA-15761
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15761
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SASI
>Reporter: ZhaoYang
>Priority: Normal
> Fix For: 3.11.x, 4.x
>
>
> Read Request Thread:
> 1.) In {{QueryController#getIndexes}}, read request gets the current {{view}}
> of sstable indexes.
> 2.) Request will try to reference the {{SSTableIndex}} from the {{view}} in
> {{TermIterator#build}} and skips the {{SSTableIndex}} if it cannot be
> referenced.
> Compaction Thread:
> If a compaction finishes between 1) and 2) and no other in-flight queries are
> holding the sstable index reference, compacted {{SSTableIndex}} will be
> released.
> Then in step 2, read request will not be able to reference the released
> {{SSTableIndex}}. We won't see a corruption error, but we'll silently not
> read all data.
>
>
>
> Proposed fix: Read Request should make sure all {{SSTableIndexes}} from the
> {{View}} can be referenced; if not, it should retry with latest {{View}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15158) Wait for schema agreement rather then in flight schema requests when bootstrapping
[ https://issues.apache.org/jira/browse/CASSANDRA-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093752#comment-17093752 ] Stefan Miklosovic commented on CASSANDRA-15158: --- Hi [~bdeggleston] I took the patch and rewored it little bit [https://github.com/smiklosovic/cassandra/commits/CASSANDRA-15158] You have said that there is not any way how to confirm that any in flight migration tasks have been completed and applied. I do not know how to verify this was indeed done but what I did is that I have exposed the information if a particular migration task has failed or nor to process (based on onFailure callback) so we can work on this further if necessary. Logically, it is same stuff as it was before in the original patch but the code is reorganised a bit. The "escape hatch" is one global bootstrap timeout, if it is passed and schemas are still not in agreement, it is still uknown to me what we want to do - either fail completely and halt that service or we allow to proceed with big fat warning. Looking forward to have some feedback! > Wait for schema agreement rather then in flight schema requests when > bootstrapping > -- > > Key: CASSANDRA-15158 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15158 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip, Cluster/Schema >Reporter: Vincent White >Assignee: Ben Bromhead >Priority: Normal > > Currently when a node is bootstrapping we use a set of latches > (org.apache.cassandra.service.MigrationTask#inflightTasks) to keep track of > in-flight schema pull requests, and we don't proceed with > bootstrapping/stream until all the latches are released (or we timeout > waiting for each one). One issue with this is that if we have a large schema, > or the retrieval of the schema from the other nodes was unexpectedly slow > then we have no explicit check in place to ensure we have actually received a > schema before we proceed. > While it's possible to increase "migration_task_wait_in_seconds" to force the > node to wait on each latche longer, there are cases where this doesn't help > because the callbacks for the schema pull requests have expired off the > messaging service's callback map > (org.apache.cassandra.net.MessagingService#callbacks) after > request_timeout_in_ms (default 10 seconds) before the other nodes were able > to respond to the new node. > This patch checks for schema agreement between the bootstrapping node and the > rest of the live nodes before proceeding with bootstrapping. It also adds a > check to prevent the new node from flooding existing nodes with simultaneous > schema pull requests as can happen in large clusters. > Removing the latch system should also prevent new nodes in large clusters > getting stuck for extended amounts of time as they wait > `migration_task_wait_in_seconds` on each of the latches left orphaned by the > timed out callbacks. > > ||3.11|| > |[PoC|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:check_for_schema]| > |[dtest|https://github.com/apache/cassandra-dtest/compare/master...vincewhite:wait_for_schema_agreement]| > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-17) Ensure sidecar can control multiple Cassandra instances
[ https://issues.apache.org/jira/browse/CASSANDRASC-17?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093732#comment-17093732 ] Jon Haddad commented on CASSANDRASC-17: --- I've been looking at this a bit more and the work isn't particularly difficult. I think I have a way of making the routing optional as well, so most people won't have to worry about it. Considering this is (for better or worse) a Cassandra feature that I personally use, I'm inclined to do it now. > Ensure sidecar can control multiple Cassandra instances > --- > > Key: CASSANDRASC-17 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-17 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Configuration >Reporter: Jon Haddad >Priority: Normal > > Since we can run multiple hosts per node, we should allow a single sidecar > process to control multiple Cassandra nodes. > I am not sure if we should encode the id of the node in the URL or as a > parameter that would have to be present in every request if using > 1 node. > I lean towards the latter - meaning it’s a slight inconvenience for a very > small group, rather than messing with the URL scheme for everyone else. I > don’t hold this opinion very strongly though. I’d like to discuss before > doing any work here. > Thoughts? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15595) Many errors of "java.lang.AssertionError: Illegal bounds"
[
https://issues.apache.org/jira/browse/CASSANDRA-15595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093717#comment-17093717
]
ZhaoYang commented on CASSANDRA-15595:
--
[~brandon.williams] thanks for the review
> Many errors of "java.lang.AssertionError: Illegal bounds"
> -
>
> Key: CASSANDRA-15595
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15595
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Caching
>Reporter: Yakir Gibraltar
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 3.0.21, 3.11.7, 4.0-beta
>
>
> Hi, i'm running cassandra 3.11.6 and getting on all hosts many errors of:
> {code}
> ERROR [ReadStage-6] 2020-02-24 13:53:34,528
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[ReadStage-6,5,main]
> java.lang.AssertionError: Illegal bounds [-2102982480..-2102982472); size:
> 2761628520
> at org.apache.cassandra.io.util.Memory.checkBounds(Memory.java:345)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at org.apache.cassandra.io.util.Memory.getLong(Memory.java:254)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.compress.CompressionMetadata.chunkFor(CompressionMetadata.java:234)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.CompressedChunkReader$Standard.readChunk(CompressedChunkReader.java:114)
> ~[apache-cassandra-3.11.6.ja
> r:3.11.6]
> at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:158)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:39)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalLoadingCache.lambda$new$0(BoundedLocalCache.java:2949)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$15(BoundedLocalCache.java:1807)
> ~[caffeine-2.2.6.jar:na]
> at
> java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
> ~[na:1.8.0-zing_19.12.102.0]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:1805)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:1788)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:97)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:66)
> ~[caffeine-2.2.6.jar:na]
> at
> org.apache.cassandra.cache.ChunkCache$CachingRebufferer.rebuffer(ChunkCache.java:236)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.cache.ChunkCache$CachingRebufferer.rebuffer(ChunkCache.java:214)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.RandomAccessReader.reBufferAt(RandomAccessReader.java:65)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReader.java:207)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.FileHandle.createReader(FileHandle.java:150)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.getFileDataInput(SSTableReader.java:1807)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.(AbstractSSTableIterator.java:103)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.columniterator.SSTableIterator.(SSTableIterator.java:49)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:72)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:65)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.StorageHook$1.makeRowIterator(StorageHook.java:100)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndSSTablesInTimestampOrder(SinglePartitionReadCommand.java:982)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(SinglePartitionReadCommand.java:693)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDisk(SinglePartitionReadCommand.java:670)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
>
[jira] [Updated] (CASSANDRA-15761) Make SASI query safe for concurrent compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-15761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhaoYang updated CASSANDRA-15761:
-
Fix Version/s: 4.x
3.11.x
> Make SASI query safe for concurrent compaction
> --
>
> Key: CASSANDRA-15761
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15761
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: ZhaoYang
>Priority: Normal
> Fix For: 3.11.x, 4.x
>
>
> Read Request Thread:
> 1.) In {{QueryController#getIndexes}}, read request gets the current {{view}}
> of sstable indexes.
> 2.) Request will try to reference the {{SSTableIndex}} from the {{view}} in
> {{TermIterator#build}} and skips the {{SSTableIndex}} if it cannot be
> referenced.
> Compaction Thread:
> If a compaction finishes between 1) and 2) and no other in-flight queries are
> holding the sstable index reference, compacted {{SSTableIndex}} will be
> released.
> Then in step 2, read request will not be able to reference the released
> {{SSTableIndex}}. We won't see a corruption error, but we'll silently not
> read all data.
>
>
>
> Proposed fix: Read Request should make sure all {{SSTableIndexes}} from the
> {{View}} can be referenced; if not, it should retry with latest {{View}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15761) Make SASI query safe for concurrent compaction
ZhaoYang created CASSANDRA-15761:
Summary: Make SASI query safe for concurrent compaction
Key: CASSANDRA-15761
URL: https://issues.apache.org/jira/browse/CASSANDRA-15761
Project: Cassandra
Issue Type: Bug
Components: Feature/2i Index
Reporter: ZhaoYang
Read Request Thread:
1.) In {{QueryController#getIndexes}}, read request gets the current {{view}}
of sstable indexes.
2.) Request will try to reference the {{SSTableIndex}} from the {{view}} in
{{TermIterator#build}} and skips the {{SSTableIndex}} if it cannot be
referenced.
Compaction Thread:
If a compaction finishes between 1) and 2) and no other in-flight queries are
holding the sstable index reference, compacted {{SSTableIndex}} will be
released.
Then in step 2, read request will not be able to reference the released
{{SSTableIndex}}. We won't see a corruption error, but we'll silently not read
all data.
Proposed fix: Read Request should make sure all {{SSTableIndexes}} from the
{{View}} can be referenced; if not, it should retry with latest {{View}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15595) Many errors of "java.lang.AssertionError: Illegal bounds"
[
https://issues.apache.org/jira/browse/CASSANDRA-15595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-15595:
-
Since Version: 3.0.0
Source Control Link:
https://github.com/apache/cassandra/commit/4886968c4e973488df4f2da480785beff09b562b
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed, thanks!
> Many errors of "java.lang.AssertionError: Illegal bounds"
> -
>
> Key: CASSANDRA-15595
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15595
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Caching
>Reporter: Yakir Gibraltar
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 3.0.21, 3.11.7, 4.0-beta
>
>
> Hi, i'm running cassandra 3.11.6 and getting on all hosts many errors of:
> {code}
> ERROR [ReadStage-6] 2020-02-24 13:53:34,528
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[ReadStage-6,5,main]
> java.lang.AssertionError: Illegal bounds [-2102982480..-2102982472); size:
> 2761628520
> at org.apache.cassandra.io.util.Memory.checkBounds(Memory.java:345)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at org.apache.cassandra.io.util.Memory.getLong(Memory.java:254)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.compress.CompressionMetadata.chunkFor(CompressionMetadata.java:234)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.CompressedChunkReader$Standard.readChunk(CompressedChunkReader.java:114)
> ~[apache-cassandra-3.11.6.ja
> r:3.11.6]
> at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:158)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:39)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalLoadingCache.lambda$new$0(BoundedLocalCache.java:2949)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$15(BoundedLocalCache.java:1807)
> ~[caffeine-2.2.6.jar:na]
> at
> java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
> ~[na:1.8.0-zing_19.12.102.0]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:1805)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:1788)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:97)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:66)
> ~[caffeine-2.2.6.jar:na]
> at
> org.apache.cassandra.cache.ChunkCache$CachingRebufferer.rebuffer(ChunkCache.java:236)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.cache.ChunkCache$CachingRebufferer.rebuffer(ChunkCache.java:214)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.RandomAccessReader.reBufferAt(RandomAccessReader.java:65)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReader.java:207)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.FileHandle.createReader(FileHandle.java:150)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.getFileDataInput(SSTableReader.java:1807)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.(AbstractSSTableIterator.java:103)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.columniterator.SSTableIterator.(SSTableIterator.java:49)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:72)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:65)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.StorageHook$1.makeRowIterator(StorageHook.java:100)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndSSTablesInTimestampOrder(SinglePartitionReadCommand.java:982)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(SinglePartitionReadCommand.java:693)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
>
[jira] [Updated] (CASSANDRA-15595) Many errors of "java.lang.AssertionError: Illegal bounds"
[
https://issues.apache.org/jira/browse/CASSANDRA-15595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-15595:
-
Status: Ready to Commit (was: Review In Progress)
> Many errors of "java.lang.AssertionError: Illegal bounds"
> -
>
> Key: CASSANDRA-15595
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15595
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Caching
>Reporter: Yakir Gibraltar
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 3.0.21, 3.11.7, 4.0-beta
>
>
> Hi, i'm running cassandra 3.11.6 and getting on all hosts many errors of:
> {code}
> ERROR [ReadStage-6] 2020-02-24 13:53:34,528
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[ReadStage-6,5,main]
> java.lang.AssertionError: Illegal bounds [-2102982480..-2102982472); size:
> 2761628520
> at org.apache.cassandra.io.util.Memory.checkBounds(Memory.java:345)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at org.apache.cassandra.io.util.Memory.getLong(Memory.java:254)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.compress.CompressionMetadata.chunkFor(CompressionMetadata.java:234)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.CompressedChunkReader$Standard.readChunk(CompressedChunkReader.java:114)
> ~[apache-cassandra-3.11.6.ja
> r:3.11.6]
> at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:158)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:39)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalLoadingCache.lambda$new$0(BoundedLocalCache.java:2949)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$15(BoundedLocalCache.java:1807)
> ~[caffeine-2.2.6.jar:na]
> at
> java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
> ~[na:1.8.0-zing_19.12.102.0]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:1805)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:1788)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:97)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:66)
> ~[caffeine-2.2.6.jar:na]
> at
> org.apache.cassandra.cache.ChunkCache$CachingRebufferer.rebuffer(ChunkCache.java:236)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.cache.ChunkCache$CachingRebufferer.rebuffer(ChunkCache.java:214)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.RandomAccessReader.reBufferAt(RandomAccessReader.java:65)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReader.java:207)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.FileHandle.createReader(FileHandle.java:150)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.getFileDataInput(SSTableReader.java:1807)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.(AbstractSSTableIterator.java:103)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.columniterator.SSTableIterator.(SSTableIterator.java:49)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:72)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:65)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.StorageHook$1.makeRowIterator(StorageHook.java:100)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndSSTablesInTimestampOrder(SinglePartitionReadCommand.java:982)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(SinglePartitionReadCommand.java:693)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDisk(SinglePartitionReadCommand.java:670)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
>
[cassandra] branch cassandra-3.0 updated: Fix chunk index overflow due to large sstable with small chunk length
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-3.0 by this push:
new 4886968 Fix chunk index overflow due to large sstable with small
chunk length
4886968 is described below
commit 4886968c4e973488df4f2da480785beff09b562b
Author: Zhao Yang
AuthorDate: Mon Apr 27 01:09:27 2020 +0800
Fix chunk index overflow due to large sstable with small chunk length
patch by ZhaoYang; reviewed by brandonwilliams for CASSANDRA-15595
---
CHANGES.txt| 1 +
.../cassandra/io/compress/CompressionMetadata.java | 6 +-
.../compress/CompressedRandomAccessReaderTest.java | 100 +++--
3 files changed, 80 insertions(+), 27 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index efe35a7..b906f15 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
3.0.21
* Fix infinite loop on index query paging in tables with clustering
(CASSANDRA-14242)
+ * Fix chunk index overflow due to large sstable with small chunk length
(CASSANDRA-15595)
* cqlsh return non-zero status when STDIN CQL fails (CASSANDRA-15623)
* Don't skip sstables in slice queries based only on local min/max/deletion
timestamp (CASSANDRA-15690)
* Memtable memory allocations may deadlock (CASSANDRA-15367)
diff --git a/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java
b/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java
index 101f722..10d1ae9 100644
--- a/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java
+++ b/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java
@@ -227,11 +227,15 @@ public class CompressionMetadata
public Chunk chunkFor(long position)
{
// position of the chunk
-int idx = 8 * (int) (position / parameters.chunkLength());
+long idx = 8 * (position / parameters.chunkLength());
if (idx >= chunkOffsetsSize)
throw new CorruptSSTableException(new EOFException(),
indexFilePath);
+if (idx < 0)
+throw new CorruptSSTableException(new
IllegalArgumentException(String.format("Invalid negative chunk index %d with
position %d", idx, position)),
+ indexFilePath);
+
long chunkOffset = chunkOffsets.getLong(idx);
long nextChunkOffset = (idx + 8 == chunkOffsetsSize)
? compressedFileLength
diff --git
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
index 0c96327..34ff94f 100644
---
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
+++
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
@@ -18,6 +18,7 @@
*/
package org.apache.cassandra.io.compress;
+import java.io.EOFException;
import java.io.File;
import java.io.IOException;
import java.io.RandomAccessFile;
@@ -43,6 +44,7 @@ import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertSame;
import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
public class CompressedRandomAccessReaderTest
{
@@ -75,11 +77,11 @@ public class CompressedRandomAccessReaderTest
{
File f = File.createTempFile("compressed6791_", "3");
String filename = f.getAbsolutePath();
-try(ChannelProxy channel = new ChannelProxy(f))
+try (ChannelProxy channel = new ChannelProxy(f))
{
MetadataCollector sstableMetadataCollector = new
MetadataCollector(new ClusteringComparator(BytesType.instance));
-try(CompressedSequentialWriter writer = new
CompressedSequentialWriter(f, filename + ".metadata",
CompressionParams.snappy(32), sstableMetadataCollector))
+try (CompressedSequentialWriter writer = new
CompressedSequentialWriter(f, filename + ".metadata",
CompressionParams.snappy(32), sstableMetadataCollector))
{
for (int i = 0; i < 20; i++)
@@ -97,9 +99,9 @@ public class CompressedRandomAccessReaderTest
writer.finish();
}
-try(RandomAccessReader reader = new
CompressedRandomAccessReader.Builder(channel,
-
new CompressionMetadata(filename + ".metadata", f.length(),
ChecksumType.CRC32))
-.build())
+try (RandomAccessReader reader = new
CompressedRandomAccessReader.Builder(channel,
+
new CompressionMetadata(filename + ".metadata", f.length(),
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 90f68a5f9439ed41e585e4946381c91ab1de6599
Merge: 78de6ac c190ab9
Author: Brandon Williams
AuthorDate: Mon Apr 27 11:17:55 2020 -0500
Merge branch 'cassandra-3.11' into trunk
CHANGES.txt| 1 +
.../cassandra/io/compress/CompressionMetadata.java | 6 +-
.../compress/CompressedRandomAccessReaderTest.java | 83 --
3 files changed, 69 insertions(+), 21 deletions(-)
diff --cc CHANGES.txt
index 11bc713,b3c593b..afd8c24
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,21 -1,7 +1,22 @@@
-3.11.7
- * Allow sstableloader to use SSL on the native port (CASSANDRA-14904)
+4.0-alpha5
+ * Avoid race condition when completing stream sessions (CASSANDRA-15666)
+ * Flush with fast compressors by default (CASSANDRA-15379)
+ * Fix CqlInputFormat regression from the switch to system.size_estimates
(CASSANDRA-15637)
+ * Allow sending Entire SSTables over SSL (CASSANDRA-15740)
+ * Fix CQLSH UTF-8 encoding issue for Python 2/3 compatibility
(CASSANDRA-15739)
+ * Fix batch statement preparation when multiple tables and parameters are
used (CASSANDRA-15730)
+ * Fix regression with traceOutgoingMessage printing message size
(CASSANDRA-15687)
+ * Ensure repaired data tracking reads a consistent amount of data across
replicas (CASSANDRA-15601)
+ * Fix CQLSH to avoid arguments being evaluated (CASSANDRA-15660)
+ * Correct Visibility and Improve Safety of Methods in LatencyMetrics
(CASSANDRA-15597)
+ * Allow cqlsh to run with Python2.7/Python3.6+
(CASSANDRA-15659,CASSANDRA-15573)
+ * Improve logging around incremental repair (CASSANDRA-15599)
+ * Do not check cdc_raw_directory filesystem space if CDC disabled
(CASSANDRA-15688)
+ * Replace array iterators with get by index (CASSANDRA-15394)
+ * Minimize BTree iterator allocations (CASSANDRA-15389)
+Merged from 3.11:
Merged from 3.0:
+ * Fix chunk index overflow due to large sstable with small chunk length
(CASSANDRA-15595)
* Allow selecting static column only when querying static index
(CASSANDRA-14242)
* cqlsh return non-zero status when STDIN CQL fails (CASSANDRA-15623)
* Don't skip sstables in slice queries based only on local min/max/deletion
timestamp (CASSANDRA-15690)
diff --cc
test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
index 5153310,c718147..d3d81f0
---
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
+++
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
@@@ -24,6 -25,6 +25,7 @@@ import java.io.RandomAccessFile
import java.util.Arrays;
import java.util.Random;
++import org.assertj.core.api.Assertions;
import org.junit.BeforeClass;
import org.junit.Test;
@@@ -35,8 -36,9 +37,9 @@@ import org.apache.cassandra.io.sstable.
import org.apache.cassandra.io.sstable.metadata.MetadataCollector;
import org.apache.cassandra.io.util.*;
import org.apache.cassandra.schema.CompressionParams;
-import org.apache.cassandra.utils.ChecksumType;
import org.apache.cassandra.utils.SyncUtil;
++import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertSame;
@@@ -136,33 -125,52 +139,46 @@@ public class CompressedRandomAccessRead
}
}
- private static void testResetAndTruncate(File f, boolean compressed,
boolean usemmap, int junkSize, double minCompressRatio) throws IOException
+ /**
+ * JIRA: CASSANDRA-15595 verify large position with small chunk length
won't overflow chunk index
+ */
+ @Test
+ public void testChunkIndexOverflow() throws IOException
{
- final String filename = f.getAbsolutePath();
- MetadataCollector sstableMetadataCollector = new
MetadataCollector(new ClusteringComparator(BytesType.instance));
- try(SequentialWriter writer = compressed
- ? new CompressedSequentialWriter(f, filename + ".metadata",
- null, SequentialWriterOption.DEFAULT,
- CompressionParams.snappy(), sstableMetadataCollector)
- : new SequentialWriter(f))
+ File file = File.createTempFile("chunk_idx_overflow", "1");
+ String filename = file.getAbsolutePath();
+ int chunkLength = 4096; // 4k
+
+ try
{
- writer.write("The quick ".getBytes());
- DataPosition mark = writer.mark();
- writer.write("blue fox jumps over the lazy dog".getBytes());
+ writeSSTable(file, CompressionParams.snappy(chunkLength), 10);
-CompressionMetadata metadata = new CompressionMetadata(filename +
".metadata", file.length(), ChecksumType.CRC32);
++
[cassandra] branch trunk updated (78de6ac -> 90f68a5)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 78de6ac Merge branch 'cassandra-3.11' into trunk new 04be736 Fix chunk index overflow due to large sstable with small chunk length new c190ab9 Merge branch 'cassandra-3.0' into cassandra-3.11 new 90f68a5 Merge branch 'cassandra-3.11' into trunk The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + .../cassandra/io/compress/CompressionMetadata.java | 6 +- .../compress/CompressedRandomAccessReaderTest.java | 83 -- 3 files changed, 69 insertions(+), 21 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 02/02: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit c190ab9fcbc8a993e570e6cdafc2b90219f80109
Merge: 377ceb6 04be736
Author: Brandon Williams
AuthorDate: Mon Apr 27 11:15:30 2020 -0500
Merge branch 'cassandra-3.0' into cassandra-3.11
CHANGES.txt| 1 +
.../cassandra/io/compress/CompressionMetadata.java | 6 +-
.../compress/CompressedRandomAccessReaderTest.java | 86 +-
3 files changed, 73 insertions(+), 20 deletions(-)
diff --cc CHANGES.txt
index fd7ad6a,99f540c..b3c593b
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,7 -1,5 +1,8 @@@
-3.0.21
+3.11.7
+ * Allow sstableloader to use SSL on the native port (CASSANDRA-14904)
+Merged from 3.0:
+ * Fix chunk index overflow due to large sstable with small chunk length
(CASSANDRA-15595)
+ * Allow selecting static column only when querying static index
(CASSANDRA-14242)
* cqlsh return non-zero status when STDIN CQL fails (CASSANDRA-15623)
* Don't skip sstables in slice queries based only on local min/max/deletion
timestamp (CASSANDRA-15690)
* Memtable memory allocations may deadlock (CASSANDRA-15367)
diff --cc
test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
index 0d2a9fb,34ff94f..c718147
---
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
+++
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
@@@ -123,42 -118,77 +125,61 @@@ public class CompressedRandomAccessRead
}
}
- private static void testResetAndTruncate(File f, boolean compressed,
boolean usemmap, int junkSize) throws IOException
+ /**
+ * JIRA: CASSANDRA-15595 verify large position with small chunk length
won't overflow chunk index
+ */
+ @Test
+ public void testChunkIndexOverflow() throws IOException
{
- final String filename = f.getAbsolutePath();
- MetadataCollector sstableMetadataCollector = new
MetadataCollector(new ClusteringComparator(BytesType.instance));
- try(SequentialWriter writer = compressed
- ? new CompressedSequentialWriter(f, filename + ".metadata",
- null, SequentialWriterOption.DEFAULT,
- CompressionParams.snappy(), sstableMetadataCollector)
- : new SequentialWriter(f))
+ File file = File.createTempFile("chunk_idx_overflow", "1");
+ String filename = file.getAbsolutePath();
+ int chunkLength = 4096; // 4k
+
+ try
{
- writer.write("The quick ".getBytes());
- DataPosition mark = writer.mark();
- writer.write("blue fox jumps over the lazy dog".getBytes());
+ writeSSTable(file, CompressionParams.snappy(chunkLength), 10);
+ CompressionMetadata metadata = new CompressionMetadata(filename +
".metadata", file.length(), ChecksumType.CRC32);
- // write enough to be sure to change chunk
- for (int i = 0; i < junkSize; ++i)
+ long chunks = 2761628520L;
+ long midPosition = (chunks / 2L) * chunkLength;
+ int idx = 8 * (int) (midPosition / chunkLength); // before patch
+ assertTrue("Expect integer overflow", idx < 0);
+
+ try
{
- writer.write((byte) 1);
+ metadata.chunkFor(midPosition);
+ fail("Expected to throw EOF exception with chunk idx larger
than total number of chunks in the sstable");
+ }
+ catch (CorruptSSTableException e)
+ {
+ assertTrue("Expect EOF, but got " + e.getCause(),
e.getCause() instanceof EOFException);
}
-
- writer.resetAndTruncate(mark);
- writer.write("brown fox jumps over the lazy dog".getBytes());
- writer.finish();
}
- assert f.exists();
+ finally
+ {
+ if (file.exists())
+ assertTrue(file.delete());
+ File metadata = new File(filename + ".metadata");
+ if (metadata.exists())
+ metadata.delete();
+ }
+ }
+
+ private static void testResetAndTruncate(File f, boolean compressed,
boolean usemmap, int junkSize) throws IOException
+ {
+ final String filename = f.getAbsolutePath();
-try(ChannelProxy channel = new ChannelProxy(f))
-{
-writeSSTable(f, compressed ? CompressionParams.snappy() : null,
junkSize);
-
-CompressionMetadata compressionMetadata = compressed ? new
CompressionMetadata(filename + ".metadata", f.length(), ChecksumType.CRC32) :
null;
-RandomAccessReader.Builder builder = compressed
- ? new
[cassandra] branch cassandra-3.11 updated (377ceb6 -> c190ab9)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 377ceb6 Merge branch 'cassandra-3.0' into cassandra-3.11 new 04be736 Fix chunk index overflow due to large sstable with small chunk length new c190ab9 Merge branch 'cassandra-3.0' into cassandra-3.11 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + .../cassandra/io/compress/CompressionMetadata.java | 6 +- .../compress/CompressedRandomAccessReaderTest.java | 86 +- 3 files changed, 73 insertions(+), 20 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/02: Fix chunk index overflow due to large sstable with small chunk length
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 04be7366fcf1b8edeb3495adcf37a171aa188d5c
Author: Zhao Yang
AuthorDate: Mon Apr 27 01:09:27 2020 +0800
Fix chunk index overflow due to large sstable with small chunk length
patch by ZhaoYang; reviewed by brandonwilliams for CASSANDRA-15595
---
CHANGES.txt| 1 +
.../cassandra/io/compress/CompressionMetadata.java | 6 +-
.../compress/CompressedRandomAccessReaderTest.java | 100 +++--
3 files changed, 80 insertions(+), 27 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 8d1c8ef..99f540c 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.21
+ * Fix chunk index overflow due to large sstable with small chunk length
(CASSANDRA-15595)
* cqlsh return non-zero status when STDIN CQL fails (CASSANDRA-15623)
* Don't skip sstables in slice queries based only on local min/max/deletion
timestamp (CASSANDRA-15690)
* Memtable memory allocations may deadlock (CASSANDRA-15367)
diff --git a/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java
b/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java
index 101f722..10d1ae9 100644
--- a/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java
+++ b/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java
@@ -227,11 +227,15 @@ public class CompressionMetadata
public Chunk chunkFor(long position)
{
// position of the chunk
-int idx = 8 * (int) (position / parameters.chunkLength());
+long idx = 8 * (position / parameters.chunkLength());
if (idx >= chunkOffsetsSize)
throw new CorruptSSTableException(new EOFException(),
indexFilePath);
+if (idx < 0)
+throw new CorruptSSTableException(new
IllegalArgumentException(String.format("Invalid negative chunk index %d with
position %d", idx, position)),
+ indexFilePath);
+
long chunkOffset = chunkOffsets.getLong(idx);
long nextChunkOffset = (idx + 8 == chunkOffsetsSize)
? compressedFileLength
diff --git
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
index 0c96327..34ff94f 100644
---
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
+++
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
@@ -18,6 +18,7 @@
*/
package org.apache.cassandra.io.compress;
+import java.io.EOFException;
import java.io.File;
import java.io.IOException;
import java.io.RandomAccessFile;
@@ -43,6 +44,7 @@ import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertSame;
import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
public class CompressedRandomAccessReaderTest
{
@@ -75,11 +77,11 @@ public class CompressedRandomAccessReaderTest
{
File f = File.createTempFile("compressed6791_", "3");
String filename = f.getAbsolutePath();
-try(ChannelProxy channel = new ChannelProxy(f))
+try (ChannelProxy channel = new ChannelProxy(f))
{
MetadataCollector sstableMetadataCollector = new
MetadataCollector(new ClusteringComparator(BytesType.instance));
-try(CompressedSequentialWriter writer = new
CompressedSequentialWriter(f, filename + ".metadata",
CompressionParams.snappy(32), sstableMetadataCollector))
+try (CompressedSequentialWriter writer = new
CompressedSequentialWriter(f, filename + ".metadata",
CompressionParams.snappy(32), sstableMetadataCollector))
{
for (int i = 0; i < 20; i++)
@@ -97,9 +99,9 @@ public class CompressedRandomAccessReaderTest
writer.finish();
}
-try(RandomAccessReader reader = new
CompressedRandomAccessReader.Builder(channel,
-
new CompressionMetadata(filename + ".metadata", f.length(),
ChecksumType.CRC32))
-.build())
+try (RandomAccessReader reader = new
CompressedRandomAccessReader.Builder(channel,
+
new CompressionMetadata(filename + ".metadata", f.length(),
ChecksumType.CRC32))
+.build())
{
String res = reader.readLine();
assertEquals(res, "");
@@ -110,37 +112,58 @@ public class CompressedRandomAccessReaderTest
{
if
[jira] [Commented] (CASSANDRA-15750) Upgrade Cassandra Java Driver to 4.6
[ https://issues.apache.org/jira/browse/CASSANDRA-15750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093691#comment-17093691 ] Josh McKenzie commented on CASSANDRA-15750: --- bq. As a side-note, it's very sad that all we have in-tree is SimpleDriver for testing and as a reference implemenetation of the protocol spec, and we have to rely on external projects even in tests, but this is probably a lengthier conversation that deserves a dev mailing list thread. If I'm not mistaken there already is one. :) > Upgrade Cassandra Java Driver to 4.6 > > > Key: CASSANDRA-15750 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15750 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies >Reporter: Eduard Tudenhoefner >Assignee: Eduard Tudenhoefner >Priority: Normal > Fix For: 4.x > > > After releasing C* 4.0 I think it would be a good idea to upgrade to the > latest Cassandra Java Driver that is being used internally. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15624) Avoid lazy initializing shut down instances when trying to send them messages
[
https://issues.apache.org/jira/browse/CASSANDRA-15624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gianluca Righetto updated CASSANDRA-15624:
--
Fix Version/s: (was: 4.0-alpha)
4.0-rc
> Avoid lazy initializing shut down instances when trying to send them messages
> -
>
> Key: CASSANDRA-15624
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15624
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Marcus Eriksson
>Assignee: Gianluca Righetto
>Priority: Normal
> Fix For: 4.0-rc
>
>
> We currently use {{to.broadcastAddressAndPort()}} when figuring out if we
> should send a message to an instance, if that instance has been shut down it
> will get re-initialized but not startup:ed which makes the tests fail.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14242) Indexed static column returns inconsistent results
[
https://issues.apache.org/jira/browse/CASSANDRA-14242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-14242:
--
Fix Version/s: (was: 4.0-beta)
(was: 3.11.x)
(was: 3.0.x)
4.0-alpha
3.11.7
3.0.21
Since Version: 3.0.0
Source Control Link:
https://ci-cassandra.apache.org/job/Cassandra-devbranch-test/61/
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Indexed static column returns inconsistent results
> --
>
> Key: CASSANDRA-14242
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14242
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
> Environment: Cassandra 3.11.2
> Java driver 3.4.0
> Ubuntu - 4.4.0-112-generic
>Reporter: Ross Black
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 3.0.21, 3.11.7, 4.0-alpha
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> I am using Cassandra 3.11.2, and the Java driver 3.4.0
> I have a table that has a static column, where the static column has a
> secondary index.
> When querying the table I get incomplete or duplicated results, depending on
> the fetch size.
> e.g.
> {code:java}
> CREATE KEYSPACE hack WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> CREATE TABLE hack.stuff (id int, kind text, chunk int static, val1 int,
> PRIMARY KEY (id, kind));
> CREATE INDEX stuff_chunk_index ON hack.stuff (chunk);{code}
> -- repeat with thousands of values for id =>
> {code:java}
> INSERT INTO hack.stuff (id, chunk, kind, val1 ) VALUES (${id}, 777, 'A',
> 123);{code}
> Querying from Java:
> {code:java}
> final SimpleStatement statement = new SimpleStatement("SELECT id, kind,
> val1 FROM hack.stuff WHERE chunk = " + chunk);
> statement.setFetchSize(fetchSize);
> statement.setConsistencyLevel(ConsistencyLevel.ALL);
> final ResultSet resultSet = connection.getSession().execute(statement);
> for (Row row : resultSet) {
> final int id = row.getInt("id");
> }{code}
> *The number of results returned depends on the fetch-size.*
> e.g. For 30k values inserted, I get the following:
> ||fetch-size||result-size||
> |4|3|
> |2|30001|
> |5000|30006|
> |100|30303|
> In production, I have a much larger table where the correct result size for a
> specific chunk is 20019, but some fetch sizes will return _significantly
> fewer_ results.
> ||fetch-size||result-size|| ||
> |25000|20019| |
> |5000||*<== this one is has far fewer results*|
> |5001|20026| |
> (so far been unable to reproduce this with the simpler test table)
> Thanks,
> Ross
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14242) Indexed static column returns inconsistent results
[
https://issues.apache.org/jira/browse/CASSANDRA-14242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093650#comment-17093650
]
Andres de la Peña commented on CASSANDRA-14242:
---
Thanks for the review. Committed to 3.0 as
[6cd2d07d9ae00fafa460fa1613264c43a283e24d|https://github.com/apache/cassandra/commit/6cd2d07d9ae00fafa460fa1613264c43a283e24d]
and merged to
[cassandra-3.11|https://github.com/apache/cassandra/commit/377ceb675c9c64afcfb3decbd4659f02b6f584a5]
and
[trunk|https://github.com/apache/cassandra/commit/78de6aca166e10524abf7f362efeb65751b78170].
> Indexed static column returns inconsistent results
> --
>
> Key: CASSANDRA-14242
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14242
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
> Environment: Cassandra 3.11.2
> Java driver 3.4.0
> Ubuntu - 4.4.0-112-generic
>Reporter: Ross Black
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> I am using Cassandra 3.11.2, and the Java driver 3.4.0
> I have a table that has a static column, where the static column has a
> secondary index.
> When querying the table I get incomplete or duplicated results, depending on
> the fetch size.
> e.g.
> {code:java}
> CREATE KEYSPACE hack WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> CREATE TABLE hack.stuff (id int, kind text, chunk int static, val1 int,
> PRIMARY KEY (id, kind));
> CREATE INDEX stuff_chunk_index ON hack.stuff (chunk);{code}
> -- repeat with thousands of values for id =>
> {code:java}
> INSERT INTO hack.stuff (id, chunk, kind, val1 ) VALUES (${id}, 777, 'A',
> 123);{code}
> Querying from Java:
> {code:java}
> final SimpleStatement statement = new SimpleStatement("SELECT id, kind,
> val1 FROM hack.stuff WHERE chunk = " + chunk);
> statement.setFetchSize(fetchSize);
> statement.setConsistencyLevel(ConsistencyLevel.ALL);
> final ResultSet resultSet = connection.getSession().execute(statement);
> for (Row row : resultSet) {
> final int id = row.getInt("id");
> }{code}
> *The number of results returned depends on the fetch-size.*
> e.g. For 30k values inserted, I get the following:
> ||fetch-size||result-size||
> |4|3|
> |2|30001|
> |5000|30006|
> |100|30303|
> In production, I have a much larger table where the correct result size for a
> specific chunk is 20019, but some fetch sizes will return _significantly
> fewer_ results.
> ||fetch-size||result-size|| ||
> |25000|20019| |
> |5000||*<== this one is has far fewer results*|
> |5001|20026| |
> (so far been unable to reproduce this with the simpler test table)
> Thanks,
> Ross
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15595) Many errors of "java.lang.AssertionError: Illegal bounds"
[
https://issues.apache.org/jira/browse/CASSANDRA-15595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-15595:
-
Reviewers: Brandon Williams, Brandon Williams (was: Brandon Williams)
Brandon Williams, Brandon Williams
Status: Review In Progress (was: Patch Available)
> Many errors of "java.lang.AssertionError: Illegal bounds"
> -
>
> Key: CASSANDRA-15595
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15595
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Caching
>Reporter: Yakir Gibraltar
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 3.0.21, 3.11.7, 4.0-beta
>
>
> Hi, i'm running cassandra 3.11.6 and getting on all hosts many errors of:
> {code}
> ERROR [ReadStage-6] 2020-02-24 13:53:34,528
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[ReadStage-6,5,main]
> java.lang.AssertionError: Illegal bounds [-2102982480..-2102982472); size:
> 2761628520
> at org.apache.cassandra.io.util.Memory.checkBounds(Memory.java:345)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at org.apache.cassandra.io.util.Memory.getLong(Memory.java:254)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.compress.CompressionMetadata.chunkFor(CompressionMetadata.java:234)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.CompressedChunkReader$Standard.readChunk(CompressedChunkReader.java:114)
> ~[apache-cassandra-3.11.6.ja
> r:3.11.6]
> at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:158)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:39)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalLoadingCache.lambda$new$0(BoundedLocalCache.java:2949)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$15(BoundedLocalCache.java:1807)
> ~[caffeine-2.2.6.jar:na]
> at
> java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
> ~[na:1.8.0-zing_19.12.102.0]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:1805)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:1788)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:97)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:66)
> ~[caffeine-2.2.6.jar:na]
> at
> org.apache.cassandra.cache.ChunkCache$CachingRebufferer.rebuffer(ChunkCache.java:236)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.cache.ChunkCache$CachingRebufferer.rebuffer(ChunkCache.java:214)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.RandomAccessReader.reBufferAt(RandomAccessReader.java:65)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReader.java:207)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.FileHandle.createReader(FileHandle.java:150)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.getFileDataInput(SSTableReader.java:1807)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.(AbstractSSTableIterator.java:103)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.columniterator.SSTableIterator.(SSTableIterator.java:49)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:72)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:65)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.StorageHook$1.makeRowIterator(StorageHook.java:100)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndSSTablesInTimestampOrder(SinglePartitionReadCommand.java:982)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(SinglePartitionReadCommand.java:693)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
>
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 377ceb675c9c64afcfb3decbd4659f02b6f584a5
Merge: d833df8 6cd2d07
Author: Andrés de la Peña
AuthorDate: Mon Apr 27 16:21:45 2020 +0100
Merge branch 'cassandra-3.0' into cassandra-3.11
# Conflicts:
# src/java/org/apache/cassandra/service/pager/AbstractQueryPager.java
# src/java/org/apache/cassandra/service/pager/MultiPartitionPager.java
#
test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
CHANGES.txt| 3 +-
.../cql3/restrictions/StatementRestrictions.java | 9 +-
.../index/internal/CassandraIndexSearcher.java | 7 +-
.../internal/composites/CompositesSearcher.java| 2 +-
.../service/pager/AbstractQueryPager.java | 28 ++-
.../service/pager/PartitionRangeQueryPager.java| 5 +
.../cassandra/cql3/DistinctQueryPagingTest.java| 262 +
.../cassandra/cql3/IndexQueryPagingTest.java | 54 +
.../validation/entities/StaticColumnsTest.java | 5 +-
.../validation/operations/SelectLimitTest.java | 156
.../cql3/validation/operations/SelectTest.java | 161 +
.../index/internal/CassandraIndexTest.java | 26 +-
12 files changed, 536 insertions(+), 182 deletions(-)
diff --cc CHANGES.txt
index cd69117,efe35a7..fd7ad6a
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,8 -1,5 +1,7 @@@
-3.0.21
- * Fix infinite loop on index query paging in tables with clustering
(CASSANDRA-14242)
+3.11.7
+ * Allow sstableloader to use SSL on the native port (CASSANDRA-14904)
+Merged from 3.0:
- ===
- 3.0.21
++ * Allow selecting static column only when querying static index
(CASSANDRA-14242)
* cqlsh return non-zero status when STDIN CQL fails (CASSANDRA-15623)
* Don't skip sstables in slice queries based only on local min/max/deletion
timestamp (CASSANDRA-15690)
* Memtable memory allocations may deadlock (CASSANDRA-15367)
diff --cc
src/java/org/apache/cassandra/cql3/restrictions/StatementRestrictions.java
index 3b7504f,84c6958..7636ecc
--- a/src/java/org/apache/cassandra/cql3/restrictions/StatementRestrictions.java
+++ b/src/java/org/apache/cassandra/cql3/restrictions/StatementRestrictions.java
@@@ -268,7 -252,7 +268,7 @@@ public final class StatementRestriction
}
if (usesSecondaryIndexing)
--validateSecondaryIndexSelections(selectsOnlyStaticColumns);
++validateSecondaryIndexSelections();
}
private void addRestriction(Restriction restriction)
@@@ -814,15 -832,15 +814,10 @@@
&& nonPrimaryKeyRestrictions.hasMultipleContains());
}
--private void validateSecondaryIndexSelections(boolean
selectsOnlyStaticColumns)
++private void validateSecondaryIndexSelections()
{
checkFalse(keyIsInRelation(),
"Select on indexed columns and with IN clause for the
PRIMARY KEY are not supported");
--// When the user only select static columns, the intent is that we
don't query the whole partition but just
--// the static parts. But 1) we don't have an easy way to do that with
2i and 2) since we don't support index on
--// static columns
--// so far, 2i means that you've restricted a non static column, so
the query is somewhat non-sensical.
--checkFalse(selectsOnlyStaticColumns, "Queries using 2ndary indexes
don't support selecting only static columns");
}
/**
diff --cc
src/java/org/apache/cassandra/index/internal/CassandraIndexSearcher.java
index 7b622e3,d6b39e6..7c23345
--- a/src/java/org/apache/cassandra/index/internal/CassandraIndexSearcher.java
+++ b/src/java/org/apache/cassandra/index/internal/CassandraIndexSearcher.java
@@@ -132,14 -132,14 +132,15 @@@ public abstract class CassandraIndexSea
DecoratedKey startKey = (DecoratedKey) range.left;
DecoratedKey endKey = (DecoratedKey) range.right;
-Slice.Bound start = Slice.Bound.BOTTOM;
-Slice.Bound end = Slice.Bound.TOP;
+ClusteringBound start = ClusteringBound.BOTTOM;
+ClusteringBound end = ClusteringBound.TOP;
/*
-- * For index queries over a range, we can't do a whole
lot better than querying everything for the key range, though for
-- * slice queries where we can slightly restrict the
beginning and end.
++ * For index queries over a range, we can't do a whole
lot better than querying everything for the
++ * key range, though for slice queries where we can
slightly restrict the beginning and end. We can
++ * not do this optimisation for static
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 78de6aca166e10524abf7f362efeb65751b78170 Merge: 001deb0 377ceb6 Author: Andrés de la Peña AuthorDate: Mon Apr 27 16:36:32 2020 +0100 Merge branch 'cassandra-3.11' into trunk # Conflicts: # CHANGES.txt # src/java/org/apache/cassandra/service/pager/AbstractQueryPager.java # test/unit/org/apache/cassandra/cql3/validation/entities/StaticColumnsTest.java # test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java CHANGES.txt| 1 + .../cql3/restrictions/StatementRestrictions.java | 9 +- .../index/internal/CassandraIndexSearcher.java | 7 +- .../internal/composites/CompositesSearcher.java| 2 +- .../service/pager/AbstractQueryPager.java | 28 ++- .../service/pager/PartitionRangeQueryPager.java| 5 + .../cassandra/cql3/DistinctQueryPagingTest.java| 231 + .../cassandra/cql3/IndexQueryPagingTest.java | 54 + .../validation/entities/StaticColumnsTest.java | 6 +- .../validation/operations/SelectLimitTest.java | 156 ++ .../index/internal/CassandraIndexTest.java | 25 +++ 11 files changed, 504 insertions(+), 20 deletions(-) diff --cc CHANGES.txt index 929dd70,fd7ad6a..11bc713 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,71 -1,9 +1,72 @@@ -3.11.7 - * Allow sstableloader to use SSL on the native port (CASSANDRA-14904) +4.0-alpha5 + * Avoid race condition when completing stream sessions (CASSANDRA-15666) + * Flush with fast compressors by default (CASSANDRA-15379) + * Fix CqlInputFormat regression from the switch to system.size_estimates (CASSANDRA-15637) + * Allow sending Entire SSTables over SSL (CASSANDRA-15740) + * Fix CQLSH UTF-8 encoding issue for Python 2/3 compatibility (CASSANDRA-15739) + * Fix batch statement preparation when multiple tables and parameters are used (CASSANDRA-15730) + * Fix regression with traceOutgoingMessage printing message size (CASSANDRA-15687) + * Ensure repaired data tracking reads a consistent amount of data across replicas (CASSANDRA-15601) + * Fix CQLSH to avoid arguments being evaluated (CASSANDRA-15660) + * Correct Visibility and Improve Safety of Methods in LatencyMetrics (CASSANDRA-15597) + * Allow cqlsh to run with Python2.7/Python3.6+ (CASSANDRA-15659,CASSANDRA-15573) + * Improve logging around incremental repair (CASSANDRA-15599) + * Do not check cdc_raw_directory filesystem space if CDC disabled (CASSANDRA-15688) + * Replace array iterators with get by index (CASSANDRA-15394) + * Minimize BTree iterator allocations (CASSANDRA-15389) +Merged from 3.11: Merged from 3.0: + * Allow selecting static column only when querying static index (CASSANDRA-14242) * cqlsh return non-zero status when STDIN CQL fails (CASSANDRA-15623) * Don't skip sstables in slice queries based only on local min/max/deletion timestamp (CASSANDRA-15690) +Merged from 2.2: + * Duplicate results with DISTINCT queries in mixed mode (CASSANDRA-15501) + +4.0-alpha4 + * Add client request size server metrics (CASSANDRA-15704) + * Add additional logging around FileUtils and compaction leftover cleanup (CASSANDRA-15705) + * Mark system_views/system_virtual_schema as non-alterable keyspaces in cqlsh (CASSANDRA-15711) + * Fail incremental repair if an old version sstable is involved (CASSANDRA-15612) + * Fix overflows on StreamingTombstoneHistogramBuilder produced by large deletion times (CASSANDRA-14773) + * Mark system_views/system_virtual_schema as system keyspaces in cqlsh (CASSANDRA-15706) + * Avoid unnecessary collection/iterator allocations during btree construction (CASSANDRA-15390) + * Repair history tables should have TTL and TWCS (CASSANDRA-12701) + * Fix cqlsh erroring out on Python 3.7 due to webbrowser module being absent (CASSANDRA-15572) + * Fix IMH#acquireCapacity() to return correct Outcome when endpoint reserve runs out (CASSANDRA-15607) + * Fix nodetool describering output (CASSANDRA-15682) + * Only track ideal CL failure when request CL met (CASSANDRA-15696) + * Fix flaky CoordinatorMessagingTest and docstring in OutboundSink and ConsistentSession (CASSANDRA-15672) + * Fix force compaction of wrapping ranges (CASSANDRA-15664) + * Expose repair streaming metrics (CASSANDRA-15656) + * Set now in seconds in the future for validation repairs (CASSANDRA-15655) + * Emit metric on preview repair failure (CASSANDRA-15654) + * Use more appropriate logging levels (CASSANDRA-15661) + * Fixed empty check in TrieMemIndex due to potential state inconsistency in ConcurrentSkipListMap (CASSANDRA-15526) + * Added UnleveledSSTables global and table level metric (CASSANDRA-15620) + * Added Virtual Table exposing Cassandra relevant system properties (CASSANDRA-15616,
[cassandra] branch cassandra-3.0 updated: Fix infinite loop on index query paging in tables with clustering
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-3.0 by this push:
new 6cd2d07 Fix infinite loop on index query paging in tables with
clustering
6cd2d07 is described below
commit 6cd2d07d9ae00fafa460fa1613264c43a283e24d
Author: Andrés de la Peña
AuthorDate: Mon Apr 27 15:01:31 2020 +0100
Fix infinite loop on index query paging in tables with clustering
patch by Andres de la Peña; reviewed by Benjamin Lerer for CASSANDRA-14242
---
CHANGES.txt| 1 +
.../service/pager/AbstractQueryPager.java | 26 ++-
.../service/pager/MultiPartitionPager.java | 6 +-
.../service/pager/PartitionRangeQueryPager.java| 5 +
.../cassandra/cql3/DistinctQueryPagingTest.java| 260 +
.../cassandra/cql3/IndexQueryPagingTest.java | 26 +++
.../cql3/validation/operations/SelectTest.java | 158 -
7 files changed, 318 insertions(+), 164 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 8d1c8ef..efe35a7 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.21
+ * Fix infinite loop on index query paging in tables with clustering
(CASSANDRA-14242)
* cqlsh return non-zero status when STDIN CQL fails (CASSANDRA-15623)
* Don't skip sstables in slice queries based only on local min/max/deletion
timestamp (CASSANDRA-15690)
* Memtable memory allocations may deadlock (CASSANDRA-15367)
diff --git
a/src/java/org/apache/cassandra/service/pager/AbstractQueryPager.java
b/src/java/org/apache/cassandra/service/pager/AbstractQueryPager.java
index f44aa24..2eecfee 100644
--- a/src/java/org/apache/cassandra/service/pager/AbstractQueryPager.java
+++ b/src/java/org/apache/cassandra/service/pager/AbstractQueryPager.java
@@ -67,7 +67,13 @@ abstract class AbstractQueryPager implements QueryPager
pageSize = Math.min(pageSize, remaining);
Pager pager = new RowPager(limits.forPaging(pageSize),
command.nowInSec());
-return
Transformation.apply(nextPageReadCommand(pageSize).execute(consistency,
clientState), pager);
+ReadCommand readCommand = nextPageReadCommand(pageSize);
+if (readCommand == null)
+{
+exhausted = true;
+return EmptyIterators.partition();
+}
+return Transformation.apply(readCommand.execute(consistency,
clientState), pager);
}
public PartitionIterator fetchPageInternal(int pageSize, ReadOrderGroup
orderGroup) throws RequestValidationException, RequestExecutionException
@@ -77,7 +83,13 @@ abstract class AbstractQueryPager implements QueryPager
pageSize = Math.min(pageSize, remaining);
RowPager pager = new RowPager(limits.forPaging(pageSize),
command.nowInSec());
-return
Transformation.apply(nextPageReadCommand(pageSize).executeInternal(orderGroup),
pager);
+ReadCommand readCommand = nextPageReadCommand(pageSize);
+if (readCommand == null)
+{
+exhausted = true;
+return EmptyIterators.partition();
+}
+return Transformation.apply(readCommand.executeInternal(orderGroup),
pager);
}
public UnfilteredPartitionIterator fetchPageUnfiltered(CFMetaData cfm, int
pageSize, ReadOrderGroup orderGroup)
@@ -86,9 +98,15 @@ abstract class AbstractQueryPager implements QueryPager
return EmptyIterators.unfilteredPartition(cfm, false);
pageSize = Math.min(pageSize, remaining);
-UnfilteredPager pager = new
UnfilteredPager(limits.forPaging(pageSize), command.nowInSec());
-return
Transformation.apply(nextPageReadCommand(pageSize).executeLocally(orderGroup),
pager);
+ReadCommand readCommand = nextPageReadCommand(pageSize);
+if (readCommand == null)
+{
+exhausted = true;
+return EmptyIterators.unfilteredPartition(cfm, false);
+}
+UnfilteredPager pager = new
UnfilteredPager(limits.forPaging(pageSize), command.nowInSec());
+return Transformation.apply(readCommand.executeLocally(orderGroup),
pager);
}
private class UnfilteredPager extends Pager
diff --git
a/src/java/org/apache/cassandra/service/pager/MultiPartitionPager.java
b/src/java/org/apache/cassandra/service/pager/MultiPartitionPager.java
index 344c64d..aa268ab 100644
--- a/src/java/org/apache/cassandra/service/pager/MultiPartitionPager.java
+++ b/src/java/org/apache/cassandra/service/pager/MultiPartitionPager.java
@@ -90,8 +90,10 @@ public class MultiPartitionPager implements QueryPager
if (isExhausted())
return null;
-PagingState state = pagers[current].state();
-return new PagingState(pagers[current].key(), state == null ? null :
state.rowMark, remaining, Integer.MAX_VALUE);
+
[cassandra] branch trunk updated (001deb0 -> 78de6ac)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 001deb0 revert to previous circleci config new 6cd2d07 Fix infinite loop on index query paging in tables with clustering new 377ceb6 Merge branch 'cassandra-3.0' into cassandra-3.11 new 78de6ac Merge branch 'cassandra-3.11' into trunk The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + .../cql3/restrictions/StatementRestrictions.java | 9 +- .../index/internal/CassandraIndexSearcher.java | 7 +- .../internal/composites/CompositesSearcher.java| 2 +- .../service/pager/AbstractQueryPager.java | 28 ++- .../service/pager/PartitionRangeQueryPager.java| 5 + .../cassandra/cql3/DistinctQueryPagingTest.java| 231 + .../cassandra/cql3/IndexQueryPagingTest.java | 54 + .../validation/entities/StaticColumnsTest.java | 6 +- .../validation/operations/SelectLimitTest.java | 156 ++ .../index/internal/CassandraIndexTest.java | 25 +++ 11 files changed, 504 insertions(+), 20 deletions(-) create mode 100644 test/unit/org/apache/cassandra/cql3/DistinctQueryPagingTest.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (d833df8 -> 377ceb6)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from d833df8 Merge branch 'cassandra-3.0' into cassandra-3.11 new 6cd2d07 Fix infinite loop on index query paging in tables with clustering new 377ceb6 Merge branch 'cassandra-3.0' into cassandra-3.11 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 3 +- .../cql3/restrictions/StatementRestrictions.java | 9 +- .../index/internal/CassandraIndexSearcher.java | 7 +- .../internal/composites/CompositesSearcher.java| 2 +- .../service/pager/AbstractQueryPager.java | 28 ++- .../service/pager/PartitionRangeQueryPager.java| 5 + .../cassandra/cql3/DistinctQueryPagingTest.java| 262 + .../cassandra/cql3/IndexQueryPagingTest.java | 54 + .../validation/entities/StaticColumnsTest.java | 5 +- .../validation/operations/SelectLimitTest.java | 156 .../cql3/validation/operations/SelectTest.java | 161 + .../index/internal/CassandraIndexTest.java | 26 +- 12 files changed, 536 insertions(+), 182 deletions(-) create mode 100644 test/unit/org/apache/cassandra/cql3/DistinctQueryPagingTest.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15595) Many errors of "java.lang.AssertionError: Illegal bounds"
[
https://issues.apache.org/jira/browse/CASSANDRA-15595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093559#comment-17093559
]
ZhaoYang commented on CASSANDRA-15595:
--
[~brandon.williams] do you mind reviewing?
> Many errors of "java.lang.AssertionError: Illegal bounds"
> -
>
> Key: CASSANDRA-15595
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15595
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Caching
>Reporter: Yakir Gibraltar
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 3.0.21, 3.11.7, 4.0-beta
>
>
> Hi, i'm running cassandra 3.11.6 and getting on all hosts many errors of:
> {code}
> ERROR [ReadStage-6] 2020-02-24 13:53:34,528
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[ReadStage-6,5,main]
> java.lang.AssertionError: Illegal bounds [-2102982480..-2102982472); size:
> 2761628520
> at org.apache.cassandra.io.util.Memory.checkBounds(Memory.java:345)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at org.apache.cassandra.io.util.Memory.getLong(Memory.java:254)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.compress.CompressionMetadata.chunkFor(CompressionMetadata.java:234)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.CompressedChunkReader$Standard.readChunk(CompressedChunkReader.java:114)
> ~[apache-cassandra-3.11.6.ja
> r:3.11.6]
> at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:158)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:39)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalLoadingCache.lambda$new$0(BoundedLocalCache.java:2949)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$15(BoundedLocalCache.java:1807)
> ~[caffeine-2.2.6.jar:na]
> at
> java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
> ~[na:1.8.0-zing_19.12.102.0]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:1805)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:1788)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:97)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:66)
> ~[caffeine-2.2.6.jar:na]
> at
> org.apache.cassandra.cache.ChunkCache$CachingRebufferer.rebuffer(ChunkCache.java:236)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.cache.ChunkCache$CachingRebufferer.rebuffer(ChunkCache.java:214)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.RandomAccessReader.reBufferAt(RandomAccessReader.java:65)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReader.java:207)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.FileHandle.createReader(FileHandle.java:150)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.getFileDataInput(SSTableReader.java:1807)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.(AbstractSSTableIterator.java:103)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.columniterator.SSTableIterator.(SSTableIterator.java:49)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:72)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:65)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.StorageHook$1.makeRowIterator(StorageHook.java:100)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndSSTablesInTimestampOrder(SinglePartitionReadCommand.java:982)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(SinglePartitionReadCommand.java:693)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDisk(SinglePartitionReadCommand.java:670)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
>
[jira] [Commented] (CASSANDRA-15718) Improve BatchMetricsTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093529#comment-17093529
]
Stephen Mallette commented on CASSANDRA-15718:
--
[~djoshi] I'm fine with the introduction of {{Range}} there rather than using
{{Pair}} +1
> Improve BatchMetricsTest
> -
>
> Key: CASSANDRA-15718
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15718
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Stephen Mallette
>Assignee: Stephen Mallette
>Priority: Normal
> Attachments: CASSANDRA-15582-test-cleanup.patch
>
>
> As noted in CASSANDRA-15582 {{BatchMetricsTest}} should test
> {{BatchStatement.Type.COUNTER}} to cover all the {{BatchMetrics}}. Some
> changes were introduced to make this improvement at:
> https://github.com/apache/cassandra/compare/trunk...spmallette:CASSANDRA-15582-trunk-batchmetrics
> and the following suggestions were made in review (in addition to the
> suggestion that a separate JIRA be created for this change) by [~dcapwell]:
> {quote}
> * I like the usage of BatchStatement.Type for the tests
> * honestly feel quick theories is better than random, but glad you added the
> seed to all asserts =). Would still be better as a quick theories test since
> you basically wrote a property anyways!
> *
> https://github.com/apache/cassandra/compare/trunk...spmallette:CASSANDRA-15582-trunk-batchmetrics#diff-8948cec1f9d33f10b15c38de80141548R131
> feel you should rename to expectedPartitionsPerLoggedBatch
> {Count,Logged,Unlogged}
> * . pre is what the value is, post is what the value is expected to be
> (rather than what it is).
> *
> *
> https://github.com/apache/cassandra/compare/trunk...spmallette:CASSANDRA-15582-trunk-batchmetrics#diff-8948cec1f9d33f10b15c38de80141548R150
> this doesn't look correct. the batch has distinctPartitions mutations, so
> shouldn't max reflect that? I ran the current test in a debugger and see that
> that is the case (aka current test is wrong).
> most of the comments are nit picks, but the last one looks like a test bug to
> me
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-17) Ensure sidecar can control multiple Cassandra instances
[ https://issues.apache.org/jira/browse/CASSANDRASC-17?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093512#comment-17093512 ] T Jake Luciani commented on CASSANDRASC-17: --- Back to this ticket I do think the scope here is more of a v2 thing. We should really be focused on getting everything for a single instance per node correct before moving onto many nodes. With a versioned api (I agree we should always have a /api/vX/ root) adding this over time is less of a concern. Same goes for batching IMO. > Ensure sidecar can control multiple Cassandra instances > --- > > Key: CASSANDRASC-17 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-17 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Configuration >Reporter: Jon Haddad >Priority: Normal > > Since we can run multiple hosts per node, we should allow a single sidecar > process to control multiple Cassandra nodes. > I am not sure if we should encode the id of the node in the URL or as a > parameter that would have to be present in every request if using > 1 node. > I lean towards the latter - meaning it’s a slight inconvenience for a very > small group, rather than messing with the URL scheme for everyone else. I > don’t hold this opinion very strongly though. I’d like to discuss before > doing any work here. > Thoughts? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15595) Many errors of "java.lang.AssertionError: Illegal bounds"
[
https://issues.apache.org/jira/browse/CASSANDRA-15595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093479#comment-17093479
]
Piotr Ryba Rybicki commented on CASSANDRA-15595:
Ok, I confirm, that this patch solves my 'Illegal bounds' problem on C* 3.11.6
Thank You!
> Many errors of "java.lang.AssertionError: Illegal bounds"
> -
>
> Key: CASSANDRA-15595
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15595
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Caching
>Reporter: Yakir Gibraltar
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 3.0.21, 3.11.7, 4.0-beta
>
>
> Hi, i'm running cassandra 3.11.6 and getting on all hosts many errors of:
> {code}
> ERROR [ReadStage-6] 2020-02-24 13:53:34,528
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[ReadStage-6,5,main]
> java.lang.AssertionError: Illegal bounds [-2102982480..-2102982472); size:
> 2761628520
> at org.apache.cassandra.io.util.Memory.checkBounds(Memory.java:345)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at org.apache.cassandra.io.util.Memory.getLong(Memory.java:254)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.compress.CompressionMetadata.chunkFor(CompressionMetadata.java:234)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.CompressedChunkReader$Standard.readChunk(CompressedChunkReader.java:114)
> ~[apache-cassandra-3.11.6.ja
> r:3.11.6]
> at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:158)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:39)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalLoadingCache.lambda$new$0(BoundedLocalCache.java:2949)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$15(BoundedLocalCache.java:1807)
> ~[caffeine-2.2.6.jar:na]
> at
> java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
> ~[na:1.8.0-zing_19.12.102.0]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:1805)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:1788)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:97)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:66)
> ~[caffeine-2.2.6.jar:na]
> at
> org.apache.cassandra.cache.ChunkCache$CachingRebufferer.rebuffer(ChunkCache.java:236)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.cache.ChunkCache$CachingRebufferer.rebuffer(ChunkCache.java:214)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.RandomAccessReader.reBufferAt(RandomAccessReader.java:65)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReader.java:207)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.FileHandle.createReader(FileHandle.java:150)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.getFileDataInput(SSTableReader.java:1807)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.(AbstractSSTableIterator.java:103)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.columniterator.SSTableIterator.(SSTableIterator.java:49)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:72)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:65)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.StorageHook$1.makeRowIterator(StorageHook.java:100)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndSSTablesInTimestampOrder(SinglePartitionReadCommand.java:982)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(SinglePartitionReadCommand.java:693)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDisk(SinglePartitionReadCommand.java:670)
>
[jira] [Commented] (CASSANDRA-15595) Many errors of "java.lang.AssertionError: Illegal bounds"
[
https://issues.apache.org/jira/browse/CASSANDRA-15595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093216#comment-17093216
]
Piotr Ryba Rybicki commented on CASSANDRA-15595:
Thanks. Checking that.
That indeed has to be the reason.
I'm also on 4K compression chunk length. Also this problematic node has huge
(5TB) data on just 1 storage there. Other nodes have many smaller disks, and
does not produce those "Illegal bounds" errors.
Will let U know shortly, if that patch fixes problem for me.
> Many errors of "java.lang.AssertionError: Illegal bounds"
> -
>
> Key: CASSANDRA-15595
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15595
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Caching
>Reporter: Yakir Gibraltar
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 3.0.21, 3.11.7, 4.0-beta
>
>
> Hi, i'm running cassandra 3.11.6 and getting on all hosts many errors of:
> {code}
> ERROR [ReadStage-6] 2020-02-24 13:53:34,528
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[ReadStage-6,5,main]
> java.lang.AssertionError: Illegal bounds [-2102982480..-2102982472); size:
> 2761628520
> at org.apache.cassandra.io.util.Memory.checkBounds(Memory.java:345)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at org.apache.cassandra.io.util.Memory.getLong(Memory.java:254)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.compress.CompressionMetadata.chunkFor(CompressionMetadata.java:234)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.CompressedChunkReader$Standard.readChunk(CompressedChunkReader.java:114)
> ~[apache-cassandra-3.11.6.ja
> r:3.11.6]
> at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:158)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:39)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalLoadingCache.lambda$new$0(BoundedLocalCache.java:2949)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$15(BoundedLocalCache.java:1807)
> ~[caffeine-2.2.6.jar:na]
> at
> java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
> ~[na:1.8.0-zing_19.12.102.0]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:1805)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:1788)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:97)
> ~[caffeine-2.2.6.jar:na]
> at
> com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:66)
> ~[caffeine-2.2.6.jar:na]
> at
> org.apache.cassandra.cache.ChunkCache$CachingRebufferer.rebuffer(ChunkCache.java:236)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.cache.ChunkCache$CachingRebufferer.rebuffer(ChunkCache.java:214)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.RandomAccessReader.reBufferAt(RandomAccessReader.java:65)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReader.java:207)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.util.FileHandle.createReader(FileHandle.java:150)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.getFileDataInput(SSTableReader.java:1807)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.(AbstractSSTableIterator.java:103)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.columniterator.SSTableIterator.(SSTableIterator.java:49)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:72)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:65)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.StorageHook$1.makeRowIterator(StorageHook.java:100)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndSSTablesInTimestampOrder(SinglePartitionReadCommand.java:982)
> ~[apache-cassandra-3.11.6.jar:3.11.6]
> at
>
[jira] [Commented] (CASSANDRA-15750) Upgrade Cassandra Java Driver to 4.6
[
https://issues.apache.org/jira/browse/CASSANDRA-15750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093164#comment-17093164
]
Alex Petrov commented on CASSANDRA-15750:
-
I'm -1 on updating the driver to 4.6, and am strongly +1 for considering 3.6.x
series.
As a side-note, it's very sad that all we have in-tree is {{SimpleDriver}} for
testing and as a reference implemenetation of the protocol spec, and we have to
rely on external projects even in tests, but this is probably a lengthier
conversation that deserves a dev mailing list thread.
> Upgrade Cassandra Java Driver to 4.6
>
>
> Key: CASSANDRA-15750
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15750
> Project: Cassandra
> Issue Type: Improvement
> Components: Dependencies
>Reporter: Eduard Tudenhoefner
>Assignee: Eduard Tudenhoefner
>Priority: Normal
> Fix For: 4.x
>
>
> After releasing C* 4.0 I think it would be a good idea to upgrade to the
> latest Cassandra Java Driver that is being used internally.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15750) Upgrade Cassandra Java Driver to 4.6
[ https://issues.apache.org/jira/browse/CASSANDRA-15750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eduard Tudenhoefner updated CASSANDRA-15750: Description: After releasing C* 4.0 I think it would be a good idea to upgrade to the latest Cassandra Java Driver that is being used internally. (was: Before releasing C* 4.0 I think it would be a good idea to upgrade to the latest Cassandra Java Driver that is being used internally.) > Upgrade Cassandra Java Driver to 4.6 > > > Key: CASSANDRA-15750 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15750 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies >Reporter: Eduard Tudenhoefner >Assignee: Eduard Tudenhoefner >Priority: Normal > Fix For: 4.x > > > After releasing C* 4.0 I think it would be a good idea to upgrade to the > latest Cassandra Java Driver that is being used internally. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15750) Upgrade Cassandra Java Driver to 4.6
[ https://issues.apache.org/jira/browse/CASSANDRA-15750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093148#comment-17093148 ] Eduard Tudenhoefner commented on CASSANDRA-15750: - Makes sense to not update to 4.6 now. Fwiw, C* is already using latest 3.6.0 Driver (there's however 3.8.0 available). I'll leave the ticket open and we can upgrade to 4.6/in-tree driver at a later point. > Upgrade Cassandra Java Driver to 4.6 > > > Key: CASSANDRA-15750 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15750 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies >Reporter: Eduard Tudenhoefner >Assignee: Eduard Tudenhoefner >Priority: Normal > Fix For: 4.x > > > Before releasing C* 4.0 I think it would be a good idea to upgrade to the > latest Cassandra Java Driver that is being used internally. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14764) Test Messaging Refactor with: 12 Node Breaking Point, compression=none, encryption=none, coalescing=off
[
https://issues.apache.org/jira/browse/CASSANDRA-14764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-14764:
---
Summary: Test Messaging Refactor with: 12 Node Breaking Point,
compression=none, encryption=none, coalescing=off (was: Evaluate 12 Node
Breaking Point, compression=none, encryption=none, coalescing=off)
> Test Messaging Refactor with: 12 Node Breaking Point, compression=none,
> encryption=none, coalescing=off
> ---
>
> Key: CASSANDRA-14764
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14764
> Project: Cassandra
> Issue Type: Sub-task
> Components: Legacy/Streaming and Messaging
>Reporter: Joey Lynch
>Assignee: Vinay Chella
>Priority: Normal
> Fix For: 4.0-rc
>
> Attachments: i-03341e1c52de6ea3e-after-queue-change.svg,
> i-07cd92e844d66d801-after-queue-bound.svg, i-07cd92e844d66d801-hint-play.svg,
> i-07cd92e844d66d801-uninlined-with-jvm-methods.svg, ttop.txt
>
>
> *Setup:*
> * Cassandra: 12 (2*6) node i3.xlarge AWS instance (4 cpu cores, 30GB ram)
> running cassandra trunk off of jasobrown/14503 jdd7ec5a2 (Jasons patched
> internode messaging branch) vs the same footprint running 3.0.17
> * Two datacenters with 100ms latency between them
> * No compression, encryption, or coalescing turned on
> *Test #1:*
> ndbench sent 1.5k QPS at a coordinator level to one datacenter (RF=3*2 = 6 so
> 3k global replica QPS) of 4kb single partition BATCH mutations at LOCAL_ONE.
> This represents about 250 QPS per coordinator in the first datacenter or 60
> QPS per core. The goal was to observe P99 write and read latencies under
> various QPS.
> *Result:*
> The good news is since the CASSANDRA-14503 changes, instead of keeping the
> mutations on heap we put the message into hints instead and don't run out of
> memory. The bad news is that the {{MessagingService-NettyOutbound-Thread's}}
> would occasionally enter a degraded state where they would just spin on a
> core. I've attached flame graphs showing the CPU state as [~jasobrown]
> applied fixes to the {{OutboundMessagingConnection}} class.
> *Follow Ups:*
> [~jasobrown] has committed a number of fixes onto his
> {{jasobrown/14503-collab}} branch including:
> 1. Limiting the amount of time spent dequeuing messages if they are expired
> (previously if messages entered the queue faster than we could dequeue them
> we'd just inifinte loop on the consumer side)
> 2. Don't call {{dequeueMessages}} from within {{dequeueMessages}} created
> callbacks.
> We're continuing to use CPU flamegraphs to figure out where we're looping and
> fixing bugs as we find them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14764) Evaluate 12 Node Breaking Point, compression=none, encryption=none, coalescing=off
[
https://issues.apache.org/jira/browse/CASSANDRA-14764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-14764:
---
Fix Version/s: (was: 4.0-beta)
4.0-rc
> Evaluate 12 Node Breaking Point, compression=none, encryption=none,
> coalescing=off
> --
>
> Key: CASSANDRA-14764
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14764
> Project: Cassandra
> Issue Type: Sub-task
> Components: Legacy/Streaming and Messaging
>Reporter: Joey Lynch
>Assignee: Vinay Chella
>Priority: Normal
> Fix For: 4.0-rc
>
> Attachments: i-03341e1c52de6ea3e-after-queue-change.svg,
> i-07cd92e844d66d801-after-queue-bound.svg, i-07cd92e844d66d801-hint-play.svg,
> i-07cd92e844d66d801-uninlined-with-jvm-methods.svg, ttop.txt
>
>
> *Setup:*
> * Cassandra: 12 (2*6) node i3.xlarge AWS instance (4 cpu cores, 30GB ram)
> running cassandra trunk off of jasobrown/14503 jdd7ec5a2 (Jasons patched
> internode messaging branch) vs the same footprint running 3.0.17
> * Two datacenters with 100ms latency between them
> * No compression, encryption, or coalescing turned on
> *Test #1:*
> ndbench sent 1.5k QPS at a coordinator level to one datacenter (RF=3*2 = 6 so
> 3k global replica QPS) of 4kb single partition BATCH mutations at LOCAL_ONE.
> This represents about 250 QPS per coordinator in the first datacenter or 60
> QPS per core. The goal was to observe P99 write and read latencies under
> various QPS.
> *Result:*
> The good news is since the CASSANDRA-14503 changes, instead of keeping the
> mutations on heap we put the message into hints instead and don't run out of
> memory. The bad news is that the {{MessagingService-NettyOutbound-Thread's}}
> would occasionally enter a degraded state where they would just spin on a
> core. I've attached flame graphs showing the CPU state as [~jasobrown]
> applied fixes to the {{OutboundMessagingConnection}} class.
> *Follow Ups:*
> [~jasobrown] has committed a number of fixes onto his
> {{jasobrown/14503-collab}} branch including:
> 1. Limiting the amount of time spent dequeuing messages if they are expired
> (previously if messages entered the queue faster than we could dequeue them
> we'd just inifinte loop on the consumer side)
> 2. Don't call {{dequeueMessages}} from within {{dequeueMessages}} created
> callbacks.
> We're continuing to use CPU flamegraphs to figure out where we're looping and
> fixing bugs as we find them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15750) Upgrade Cassandra Java Driver to 4.6
[ https://issues.apache.org/jira/browse/CASSANDRA-15750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eduard Tudenhoefner updated CASSANDRA-15750: Fix Version/s: (was: 4.0-rc) (was: 4.0-beta) 4.x > Upgrade Cassandra Java Driver to 4.6 > > > Key: CASSANDRA-15750 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15750 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies >Reporter: Eduard Tudenhoefner >Assignee: Eduard Tudenhoefner >Priority: Normal > Fix For: 4.x > > > Before releasing C* 4.0 I think it would be a good idea to upgrade to the > latest Cassandra Java Driver that is being used internally. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15749) Make clear in the documentation that stress is not a secured tool
[ https://issues.apache.org/jira/browse/CASSANDRA-15749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093110#comment-17093110 ] Benjamin Lerer commented on CASSANDRA-15749: [~akin-tekeoglu] Yes :-) > Make clear in the documentation that stress is not a secured tool > - > > Key: CASSANDRA-15749 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15749 > Project: Cassandra > Issue Type: Bug > Components: Documentation/Website >Reporter: Benjamin Lerer >Priority: Normal > Fix For: 4.0, 2.2.x, 3.0.x, 3.11.x > > > It has been reported that the serialization used by the stress server was > unsafe. > As Stress has never intended to be a secure application, the problem should > not be fixed but the documentation should be updated to reflect that fact. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
