[jira] [Commented] (CASSANDRA-13810) Overload because of hint pressure + MVs
[
https://issues.apache.org/jira/browse/CASSANDRA-13810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17127079#comment-17127079
]
Surbhi Gupta commented on CASSANDRA-13810:
--
We have upgraded to 3.11.5 and still see the same issue.
> Overload because of hint pressure + MVs
> ---
>
> Key: CASSANDRA-13810
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13810
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Materialized Views
>Reporter: Tom van der Woerdt
>Priority: Urgent
> Labels: materializedviews
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Cluster setup: 3 DCs, 20 Cassandra nodes each, all 3.0.14, with approx. 200GB
> data per machine. Many tables have MVs associated.
> During some maintenance we did a rolling restart of all nodes in the cluster.
> This caused a buildup of hints/batches, as expected. Most nodes came back
> just fine, except for two nodes.
> These two nodes came back with a loadavg of >100, and 'nodetool tpstats'
> showed a million (not exaggerating) MutationStage tasks per second(!). It was
> clear that these were mostly (all?) mutations coming from hints, as indicated
> by thousands of log entries per second in debug.log :
> {noformat}
> DEBUG [SharedPool-Worker-107] 2017-08-27 13:16:51,098 HintVerbHandler.java:95
> - Failed to apply hint
> java.util.concurrent.CompletionException:
> org.apache.cassandra.exceptions.WriteTimeoutException: Operation timed out -
> received only 0 responses.
> at
> java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
> ~[na:1.8.0_144]
> at
> java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
> ~[na:1.8.0_144]
> at
> java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:647)
> ~[na:1.8.0_144]
> at
> java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:632)
> ~[na:1.8.0_144]
> at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> ~[na:1.8.0_144]
> at
> java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
> ~[na:1.8.0_144]
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:481)
> ~[apache-cassandra-3.0.14.jar:3.0.14]
> at
> org.apache.cassandra.db.Keyspace.lambda$applyInternal$0(Keyspace.java:495)
> ~[apache-cassandra-3.0.14.jar:3.0.14]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_144]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
> ~[apache-cassandra-3.0.14.jar:3.0.14]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> ~[apache-cassandra-3.0.14.jar:3.0.14]
> at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_144]
> Caused by: org.apache.cassandra.exceptions.WriteTimeoutException: Operation
> timed out - received only 0 responses.
> ... 6 common frames omitted
> {noformat}
> After reading the relevant code, it seems that a hint is considered
> droppable, and in the mutation path when the table contains a MV and the lock
> fails to acquire and the mutation is droppable, it throws a WTE without
> waiting until the timeout expires. This explains why Cassandra is able to
> process a million mutations per second without actually considering them
> 'dropped' in the 'nodetool tpstats' output.
> I managed to recover the two nodes by stopping handoffs on all nodes in the
> cluster and reenabling them one at a time. It's likely that the hint/batchlog
> settings were sub-optimal on this cluster, but I think that the retry
> behavior(?) of hints should be improved as it's hard to express hint
> throughput in kb/s when the mutations can involve MVs.
> More data available upon request -- I'm not sure which bits are relevant and
> which aren't.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13810) Overload because of hint pressure + MVs
[
https://issues.apache.org/jira/browse/CASSANDRA-13810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17032585#comment-17032585
]
Surbhi Gupta edited comment on CASSANDRA-13810 at 6/5/20, 8:35 PM:
---
We are on 3.11.0 and also facing the same issue ...
Even lowering the hinted_handoff_throttle_in_kb to 100 doesn't help much
even when we have hinted handoff in very small amount .
CPU gets 100% busy and load becomes very high but if we truncate the hints then
CPU and Load goes down immediately .
Can it be prioritized ?
Any other suggestion?
was (Author: surbhigupta01):
We are on 3.11.1 and also facing the same issue ...
Even lowering the hinted_handoff_throttle_in_kb to 100 doesn't help much
even when we have hinted handoff in very small amount .
CPU gets 100% busy and load becomes very high but if we truncate the hints then
CPU and Load goes down immediately .
Can it be prioritized ?
Any other suggestion?
> Overload because of hint pressure + MVs
> ---
>
> Key: CASSANDRA-13810
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13810
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Materialized Views
>Reporter: Tom van der Woerdt
>Priority: Urgent
> Labels: materializedviews
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Cluster setup: 3 DCs, 20 Cassandra nodes each, all 3.0.14, with approx. 200GB
> data per machine. Many tables have MVs associated.
> During some maintenance we did a rolling restart of all nodes in the cluster.
> This caused a buildup of hints/batches, as expected. Most nodes came back
> just fine, except for two nodes.
> These two nodes came back with a loadavg of >100, and 'nodetool tpstats'
> showed a million (not exaggerating) MutationStage tasks per second(!). It was
> clear that these were mostly (all?) mutations coming from hints, as indicated
> by thousands of log entries per second in debug.log :
> {noformat}
> DEBUG [SharedPool-Worker-107] 2017-08-27 13:16:51,098 HintVerbHandler.java:95
> - Failed to apply hint
> java.util.concurrent.CompletionException:
> org.apache.cassandra.exceptions.WriteTimeoutException: Operation timed out -
> received only 0 responses.
> at
> java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
> ~[na:1.8.0_144]
> at
> java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
> ~[na:1.8.0_144]
> at
> java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:647)
> ~[na:1.8.0_144]
> at
> java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:632)
> ~[na:1.8.0_144]
> at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> ~[na:1.8.0_144]
> at
> java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
> ~[na:1.8.0_144]
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:481)
> ~[apache-cassandra-3.0.14.jar:3.0.14]
> at
> org.apache.cassandra.db.Keyspace.lambda$applyInternal$0(Keyspace.java:495)
> ~[apache-cassandra-3.0.14.jar:3.0.14]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_144]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
> ~[apache-cassandra-3.0.14.jar:3.0.14]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> ~[apache-cassandra-3.0.14.jar:3.0.14]
> at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_144]
> Caused by: org.apache.cassandra.exceptions.WriteTimeoutException: Operation
> timed out - received only 0 responses.
> ... 6 common frames omitted
> {noformat}
> After reading the relevant code, it seems that a hint is considered
> droppable, and in the mutation path when the table contains a MV and the lock
> fails to acquire and the mutation is droppable, it throws a WTE without
> waiting until the timeout expires. This explains why Cassandra is able to
> process a million mutations per second without actually considering them
> 'dropped' in the 'nodetool tpstats' output.
> I managed to recover the two nodes by stopping handoffs on all nodes in the
> cluster and reenabling them one at a time. It's likely that the hint/batchlog
> settings were sub-optimal on this cluster, but I think that the retry
> behavior(?) of hints should be improved as it's hard to express hint
> throughput in kb/s when the mutations can involve MVs.
> More data available upon request -- I'm not sure which bits are relevant and
> which aren't.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe,
[jira] [Comment Edited] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17127003#comment-17127003
]
Michael Semb Wever edited comment on CASSANDRA-15234 at 6/5/20, 7:29 PM:
-
Thanks for the summary post [~e.dimitrova].
For the first
[part|https://github.com/apache/cassandra/compare/trunk...ekaterinadimitrova2:CASSANDRA-15234-3-rebase]
(including the dtest and ccm changes), the ASF CI pipeline results are
[here|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/144/].
In addition the ASF CI dtest upgrade test results are
[here|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch-dtest-upgrade/10/].
For the second
[part|https://github.com/apache/cassandra/compare/trunk...ekaterinadimitrova2:CASSANDRA-15234-5]
the ASF CI results are
[here|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/145/pipeline].
was (Author: michaelsembwever):
Thanks for the summary post [~e.dimitrova].
For the first
[part|https://github.com/apache/cassandra/compare/trunk...ekaterinadimitrova2:CASSANDRA-15234-3-rebase]
(including the dtest and ccm changes), the ASF CI pipeline results are
[here|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/144/].
In addition the ASF CI dtest upgrade test results are
[here|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch-dtest-upgrade/9/].
For the second
[part|https://github.com/apache/cassandra/compare/trunk...ekaterinadimitrova2:CASSANDRA-15234-5]
the ASF CI results are
[here|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/145/pipeline].
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15821) Metrics Documentation Enhancements
[
https://issues.apache.org/jira/browse/CASSANDRA-15821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-15821:
--
Fix Version/s: 4.0-beta
> Metrics Documentation Enhancements
> --
>
> Key: CASSANDRA-15821
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15821
> Project: Cassandra
> Issue Type: Improvement
> Components: Documentation/Website
>Reporter: Stephen Mallette
>Assignee: Stephen Mallette
>Priority: Normal
> Fix For: 4.0-beta
>
>
> CASSANDRA-15582 involves quality around metrics and it was mentioned that
> reviewing and [improving
> documentation|https://github.com/apache/cassandra/blob/trunk/doc/source/operating/metrics.rst]
> around metrics would fall into that scope. Please consider some of this
> analysis in determining what improvements to make here:
> Please see [this
> spreadsheet|https://docs.google.com/spreadsheets/d/1iPWfCMIG75CI6LbYuDtCTjEOvZw-5dyH-e08bc63QnI/edit?usp=sharing]
> that itemizes almost all of cassandra's metrics and whether they are
> documented or not (and other notes). That spreadsheet is "almost all"
> because there are some metrics that don't seem to initialize as part of
> Cassandra startup (i was able to trigger some to initialize, but all were not
> immediately obvious). The missing metrics seem to be related to the following:
> * ThreadPool metrics - only some initialize at startup the list of which
> follow below
> * Streaming Metrics
> * HintedHandoff Metrics
> * HintsService Metrics
> Here are the ThreadPool scopes that get listed:
> {code}
> AntiEntropyStage
> CacheCleanupExecutor
> CompactionExecutor
> GossipStage
> HintsDispatcher
> MemtableFlushWriter
> MemtablePostFlush
> MemtableReclaimMemory
> MigrationStage
> MutationStage
> Native-Transport-Requests
> PendingRangeCalculator
> PerDiskMemtableFlushWriter_0
> ReadStage
> Repair-Task
> RequestResponseStage
> Sampler
> SecondaryIndexManagement
> ValidationExecutor
> ViewBuildExecutor
> {code}
> I noticed that Keyspace Metrics have this note: "Most of these metrics are
> the same as the Table Metrics above, only they are aggregated at the Keyspace
> level." I think I've isolated those metrics on table that are not on keyspace
> to specifically be:
> {code}
> BloomFilterFalsePositives
> BloomFilterFalseRatio
> BytesAnticompacted
> BytesFlushed
> BytesMutatedAnticompaction
> BytesPendingRepair
> BytesRepaired
> BytesUnrepaired
> CompactionBytesWritten
> CompressionRatio
> CoordinatorReadLatency
> CoordinatorScanLatency
> CoordinatorWriteLatency
> EstimatedColumnCountHistogram
> EstimatedPartitionCount
> EstimatedPartitionSizeHistogram
> KeyCacheHitRate
> LiveSSTableCount
> MaxPartitionSize
> MeanPartitionSize
> MinPartitionSize
> MutatedAnticompactionGauge
> PercentRepaired
> RowCacheHitOutOfRange
> RowCacheHit
> RowCacheMiss
> SpeculativeSampleLatencyNanos
> SyncTime
> WaitingOnFreeMemtableSpace
> DroppedMutations
> {code}
> Someone with greater knowledge of this area might consider it worth the
> effort to see if any of these metrics should be aggregated to the keyspace
> level in case they were inadvertently missed. In any case, perhaps the
> documentation could easily now reflect which metric names could be expected
> on Keyspace.
> The DroppedMessage metrics have a much larger body of scopes than just what
> were documented:
> {code}
> ASYMMETRIC_SYNC_REQ
> BATCH_REMOVE_REQ
> BATCH_REMOVE_RSP
> BATCH_STORE_REQ
> BATCH_STORE_RSP
> CLEANUP_MSG
> COUNTER_MUTATION_REQ
> COUNTER_MUTATION_RSP
> ECHO_REQ
> ECHO_RSP
> FAILED_SESSION_MSG
> FAILURE_RSP
> FINALIZE_COMMIT_MSG
> FINALIZE_PROMISE_MSG
> FINALIZE_PROPOSE_MSG
> GOSSIP_DIGEST_ACK
> GOSSIP_DIGEST_ACK2
> GOSSIP_DIGEST_SYN
> GOSSIP_SHUTDOWN
> HINT_REQ
> HINT_RSP
> INTERNAL_RSP
> MUTATION_REQ
> MUTATION_RSP
> PAXOS_COMMIT_REQ
> PAXOS_COMMIT_RSP
> PAXOS_PREPARE_REQ
> PAXOS_PREPARE_RSP
> PAXOS_PROPOSE_REQ
> PAXOS_PROPOSE_RSP
> PING_REQ
> PING_RSP
> PREPARE_CONSISTENT_REQ
> PREPARE_CONSISTENT_RSP
> PREPARE_MSG
> RANGE_REQ
> RANGE_RSP
> READ_REPAIR_REQ
> READ_REPAIR_RSP
> READ_REQ
> READ_RSP
> REPAIR_RSP
> REPLICATION_DONE_REQ
> REPLICATION_DONE_RSP
> REQUEST_RSP
> SCHEMA_PULL_REQ
> SCHEMA_PULL_RSP
> SCHEMA_PUSH_REQ
> SCHEMA_PUSH_RSP
> SCHEMA_VERSION_REQ
> SCHEMA_VERSION_RSP
> SNAPSHOT_MSG
> SNAPSHOT_REQ
> SNAPSHOT_RSP
> STATUS_REQ
> STATUS_RSP
> SYNC_REQ
> SYNC_RSP
> TRUNCATE_REQ
> TRUNCATE_RSP
> VALIDATION_REQ
> VALIDATION_RSP
> _SAMPLE
> _TEST_1
> _TEST_2
> _TRACE
> {code}
> I suppose I may yet be missing some metrics as my knowledge of what's
> available is limited to what I can get from JMX after cassandra
> initialization (and some initial starting commands) and what's int he
> documentation. If something is present that is missing from both then I won't
> know it's there. Anyway, perhaps
[jira] [Updated] (CASSANDRA-15821) Metrics Documentation Enhancements
[
https://issues.apache.org/jira/browse/CASSANDRA-15821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-15821:
--
Change Category: Code Clarity
Complexity: Low Hanging Fruit
Status: Open (was: Triage Needed)
> Metrics Documentation Enhancements
> --
>
> Key: CASSANDRA-15821
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15821
> Project: Cassandra
> Issue Type: Improvement
> Components: Documentation/Website
>Reporter: Stephen Mallette
>Assignee: Stephen Mallette
>Priority: Normal
>
> CASSANDRA-15582 involves quality around metrics and it was mentioned that
> reviewing and [improving
> documentation|https://github.com/apache/cassandra/blob/trunk/doc/source/operating/metrics.rst]
> around metrics would fall into that scope. Please consider some of this
> analysis in determining what improvements to make here:
> Please see [this
> spreadsheet|https://docs.google.com/spreadsheets/d/1iPWfCMIG75CI6LbYuDtCTjEOvZw-5dyH-e08bc63QnI/edit?usp=sharing]
> that itemizes almost all of cassandra's metrics and whether they are
> documented or not (and other notes). That spreadsheet is "almost all"
> because there are some metrics that don't seem to initialize as part of
> Cassandra startup (i was able to trigger some to initialize, but all were not
> immediately obvious). The missing metrics seem to be related to the following:
> * ThreadPool metrics - only some initialize at startup the list of which
> follow below
> * Streaming Metrics
> * HintedHandoff Metrics
> * HintsService Metrics
> Here are the ThreadPool scopes that get listed:
> {code}
> AntiEntropyStage
> CacheCleanupExecutor
> CompactionExecutor
> GossipStage
> HintsDispatcher
> MemtableFlushWriter
> MemtablePostFlush
> MemtableReclaimMemory
> MigrationStage
> MutationStage
> Native-Transport-Requests
> PendingRangeCalculator
> PerDiskMemtableFlushWriter_0
> ReadStage
> Repair-Task
> RequestResponseStage
> Sampler
> SecondaryIndexManagement
> ValidationExecutor
> ViewBuildExecutor
> {code}
> I noticed that Keyspace Metrics have this note: "Most of these metrics are
> the same as the Table Metrics above, only they are aggregated at the Keyspace
> level." I think I've isolated those metrics on table that are not on keyspace
> to specifically be:
> {code}
> BloomFilterFalsePositives
> BloomFilterFalseRatio
> BytesAnticompacted
> BytesFlushed
> BytesMutatedAnticompaction
> BytesPendingRepair
> BytesRepaired
> BytesUnrepaired
> CompactionBytesWritten
> CompressionRatio
> CoordinatorReadLatency
> CoordinatorScanLatency
> CoordinatorWriteLatency
> EstimatedColumnCountHistogram
> EstimatedPartitionCount
> EstimatedPartitionSizeHistogram
> KeyCacheHitRate
> LiveSSTableCount
> MaxPartitionSize
> MeanPartitionSize
> MinPartitionSize
> MutatedAnticompactionGauge
> PercentRepaired
> RowCacheHitOutOfRange
> RowCacheHit
> RowCacheMiss
> SpeculativeSampleLatencyNanos
> SyncTime
> WaitingOnFreeMemtableSpace
> DroppedMutations
> {code}
> Someone with greater knowledge of this area might consider it worth the
> effort to see if any of these metrics should be aggregated to the keyspace
> level in case they were inadvertently missed. In any case, perhaps the
> documentation could easily now reflect which metric names could be expected
> on Keyspace.
> The DroppedMessage metrics have a much larger body of scopes than just what
> were documented:
> {code}
> ASYMMETRIC_SYNC_REQ
> BATCH_REMOVE_REQ
> BATCH_REMOVE_RSP
> BATCH_STORE_REQ
> BATCH_STORE_RSP
> CLEANUP_MSG
> COUNTER_MUTATION_REQ
> COUNTER_MUTATION_RSP
> ECHO_REQ
> ECHO_RSP
> FAILED_SESSION_MSG
> FAILURE_RSP
> FINALIZE_COMMIT_MSG
> FINALIZE_PROMISE_MSG
> FINALIZE_PROPOSE_MSG
> GOSSIP_DIGEST_ACK
> GOSSIP_DIGEST_ACK2
> GOSSIP_DIGEST_SYN
> GOSSIP_SHUTDOWN
> HINT_REQ
> HINT_RSP
> INTERNAL_RSP
> MUTATION_REQ
> MUTATION_RSP
> PAXOS_COMMIT_REQ
> PAXOS_COMMIT_RSP
> PAXOS_PREPARE_REQ
> PAXOS_PREPARE_RSP
> PAXOS_PROPOSE_REQ
> PAXOS_PROPOSE_RSP
> PING_REQ
> PING_RSP
> PREPARE_CONSISTENT_REQ
> PREPARE_CONSISTENT_RSP
> PREPARE_MSG
> RANGE_REQ
> RANGE_RSP
> READ_REPAIR_REQ
> READ_REPAIR_RSP
> READ_REQ
> READ_RSP
> REPAIR_RSP
> REPLICATION_DONE_REQ
> REPLICATION_DONE_RSP
> REQUEST_RSP
> SCHEMA_PULL_REQ
> SCHEMA_PULL_RSP
> SCHEMA_PUSH_REQ
> SCHEMA_PUSH_RSP
> SCHEMA_VERSION_REQ
> SCHEMA_VERSION_RSP
> SNAPSHOT_MSG
> SNAPSHOT_REQ
> SNAPSHOT_RSP
> STATUS_REQ
> STATUS_RSP
> SYNC_REQ
> SYNC_RSP
> TRUNCATE_REQ
> TRUNCATE_RSP
> VALIDATION_REQ
> VALIDATION_RSP
> _SAMPLE
> _TEST_1
> _TEST_2
> _TRACE
> {code}
> I suppose I may yet be missing some metrics as my knowledge of what's
> available is limited to what I can get from JMX after cassandra
> initialization (and some initial starting commands) and what's int he
> documentation. If something is present that is missing
[jira] [Comment Edited] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126974#comment-17126974 ] Ekaterina Dimitrova edited comment on CASSANDRA-15234 at 6/5/20, 6:35 PM: -- *Summary:* This branch contains: - [first commit |https://github.com/ekaterinadimitrova2/cassandra/commit/a8a9145a032a9f2d9b74369f2e4474a210aa5b99] which contains parameters name changes as requested; backward compatibility with the old names; Proposed new structure into sections for cassandra.yaml. - the [second commit |https://github.com/ekaterinadimitrova2/cassandra/commit/032fbd8cca1d7b5b96738bd96d9672864426d718] implements the functionality which gives the end users the opportunity to attach units suffixes to Memory, Duration, Rates parameters. - [third commit |https://github.com/ekaterinadimitrova2/cassandra/commit/68f89503e6e6f30e8f6df1374cb553f243f0068e] is a ninja fix of one line which was changed in a wrong way during the latest rebase. - [fourth commit |https://github.com/ekaterinadimitrova2/cassandra/commit/8389010d704a9524a1c36d96f9e4d8179a8e2044] is to return the default config.yml This patch requires custom [ [DTests |https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] ] which run successfully with the following patch of [CCM |https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3] DTests - requirements.txt should be updated back to the original one before commit! The DTests are not backward compatible with the yaml file without units suffixes. This part is covered by a unit test which tests the successful load of the values of the parameters without units as per the old format. [ [trunk |https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-3-rebase] ] [ [DTests |https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] ] [ [CCM |https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3] ] [ [JAVA8 |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/718c8a04-385b-4ddc-ac2e-9291c37b1f1e] ] [ [JAVA11 |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/6cd6048d-5daf-4490-a5d3-b5c3178a0757] ] Attached to the ticket is the log of the successful runs of the few failing JAVA8 DTests in CircleCI. I ran them locally after fixing lost line of code as it didn't make sense to me to rerun the whole suite of tests. *Last update:* after rebase and a few nits on my end, we have only 1 DTest and 1 Unit test failing in JAVA11. Unrelated known failures. CQLSH JAVA11 tests are not currently available in CircleCI. *WARNING:* Before commit return to the default requirements.txt file *Order of commits:* 1) [ [CASSANDRA branch |https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-3-rebase] ] 2) [ [CCM branch |https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3] ] 3) [ [DTests branch |https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] ] This [branch |https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-5] is a class of Accessors for System properties. *This patch could be considered separate of the previous one, it doesn't matter first or second.* It also doesn't require custom DTests or custom CCM. [ [trunk |https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-5] ] [ [JAVA8 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/ebd771ac-d71c-49ea-872b-00b534d9d5c6] ] [ [JAVA11 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/19a3254b-cd7a-4e4f-856f-f314546ff9d7] ] There are 1 DTest and 1 Unit unrelated failing tests in JAVA11. Should check for tickets logged. was (Author: e.dimitrova): *Summary:* This branch contains: - [first commit | https://github.com/ekaterinadimitrova2/cassandra/commit/a8a9145a032a9f2d9b74369f2e4474a210aa5b99] which contains parameters name changes as requested; backward compatibility with the old names; Proposed new structure into [sections |https://issues.apache.org/jira/browse/CASSANDRA-15234?focusedCommentId=17057969=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17057969] for cassandra.yaml. - the [second commit | https://github.com/ekaterinadimitrova2/cassandra/commit/032fbd8cca1d7b5b96738bd96d9672864426d718] implements the functionality which gives the end users the opportunity to attach units suffixes to Memory, Duration, Rates parameters. - [third commit |https://github.com/ekaterinadimitrova2/cassandra/commit/68f89503e6e6f30e8f6df1374cb553f243f0068e] is a ninja fix of one line which was changed in a wrong way during the latest rebase. - [fourth commit
[jira] [Comment Edited] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126974#comment-17126974 ] Ekaterina Dimitrova edited comment on CASSANDRA-15234 at 6/5/20, 6:28 PM: -- *Summary:* This branch contains: - [first commit | https://github.com/ekaterinadimitrova2/cassandra/commit/a8a9145a032a9f2d9b74369f2e4474a210aa5b99] which contains parameters name changes as requested; backward compatibility with the old names; Proposed new structure into [sections |https://issues.apache.org/jira/browse/CASSANDRA-15234?focusedCommentId=17057969=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17057969] for cassandra.yaml. - the [second commit | https://github.com/ekaterinadimitrova2/cassandra/commit/032fbd8cca1d7b5b96738bd96d9672864426d718] implements the functionality which gives the end users the opportunity to attach units suffixes to Memory, Duration, Rates parameters. - [third commit |https://github.com/ekaterinadimitrova2/cassandra/commit/68f89503e6e6f30e8f6df1374cb553f243f0068e] is a ninja fix of one line which was changed in a wrong way during the latest rebase. - [fourth commit |https://github.com/ekaterinadimitrova2/cassandra/commit/8389010d704a9524a1c36d96f9e4d8179a8e2044] is to return the default config.yml This patch requires custom [ [DTests | https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] ] which run successfully with the following patch of [CCM |https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3] DTests - requirements.txt should be updated back to the original one before commit! The DTests are not backward compatible with the yaml file without units suffixes. This part is covered by a unit test which tests the successful load of the values of the parameters without units as per the old format. [ [trunk |https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-3-rebase ] ] [ [DTests | https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] ] [ [CCM |https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3] ] [ [JAVA8 |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/718c8a04-385b-4ddc-ac2e-9291c37b1f1e] ] [ [JAVA11 |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/6cd6048d-5daf-4490-a5d3-b5c3178a0757 ] ] Attached to the ticket is the log of the successful runs of the few failing JAVA8 DTests in CircleCI. I ran them locally after fixing lost line of code as it didn't make sense to me to rerun the whole suite of tests. *Last update:* after rebase and a few nits on my end, we have only 1 DTest and 1 Unit test failing in JAVA11. Unrelated known failures. CQLSH JAVA11 tests are not currently available in CircleCI. *WARNING:* Before commit return to the default requirements.txt file This [branch | https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-5] is a class of Accessors for System properties. *This patch could be considered separate of the previous one.* It doesn't require custom DTests or custom CCM. [ [trunk |https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-5] ] [ [JAVA8 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/ebd771ac-d71c-49ea-872b-00b534d9d5c6] ] [ [JAVA11 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/19a3254b-cd7a-4e4f-856f-f314546ff9d7] ] There are 1 DTest and 1 Unit unrelated failing tests in JAVA11. Should check for tickets logged. was (Author: e.dimitrova): *Summary:* This branch contains: - [first commit | https://github.com/ekaterinadimitrova2/cassandra/commit/a8a9145a032a9f2d9b74369f2e4474a210aa5b99] which contains parameters name changes as requested; backward compatibility with the old names; Proposed new structure into [sections |https://issues.apache.org/jira/browse/CASSANDRA-15234?focusedCommentId=17057969=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17057969] for cassandra.yaml. - the [second commit | https://github.com/ekaterinadimitrova2/cassandra/commit/032fbd8cca1d7b5b96738bd96d9672864426d718] implements the functionality which gives the end users the opportunity to attach units suffixes to Memory, Duration, Rates parameters. - [third commit |https://github.com/ekaterinadimitrova2/cassandra/commit/68f89503e6e6f30e8f6df1374cb553f243f0068e] is a ninja fix of one line which was changed in a wrong way during the latest rebase. - [fourth commit |https://github.com/ekaterinadimitrova2/cassandra/commit/8389010d704a9524a1c36d96f9e4d8179a8e2044] is to return the default config.yml This patch requires custom [ [DTests | https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] ] which run successfully
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17127003#comment-17127003
]
Michael Semb Wever commented on CASSANDRA-15234:
Thanks for the summary post [~e.dimitrova].
For the first
[part|https://github.com/apache/cassandra/compare/trunk...ekaterinadimitrova2:CASSANDRA-15234-3-rebase]
(including the dtest and ccm changes), the ASF CI pipeline results are
[here|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/144/].
In addition the ASF CI dtest upgrade test results are
[here|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch-dtest-upgrade/9/].
For the second
[part|https://github.com/apache/cassandra/compare/trunk...ekaterinadimitrova2:CASSANDRA-15234-5]
the ASF CI results are
[here|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/145/pipeline].
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126974#comment-17126974 ] Ekaterina Dimitrova edited comment on CASSANDRA-15234 at 6/5/20, 5:50 PM: -- *Summary:* This branch contains: - [first commit | https://github.com/ekaterinadimitrova2/cassandra/commit/a8a9145a032a9f2d9b74369f2e4474a210aa5b99] which contains parameters name changes as requested; backward compatibility with the old names; Proposed new structure into [sections |https://issues.apache.org/jira/browse/CASSANDRA-15234?focusedCommentId=17057969=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17057969] for cassandra.yaml. - the [second commit | https://github.com/ekaterinadimitrova2/cassandra/commit/032fbd8cca1d7b5b96738bd96d9672864426d718] implements the functionality which gives the end users the opportunity to attach units suffixes to Memory, Duration, Rates parameters. - [third commit |https://github.com/ekaterinadimitrova2/cassandra/commit/68f89503e6e6f30e8f6df1374cb553f243f0068e] is a ninja fix of one line which was changed in a wrong way during the latest rebase. - [fourth commit |https://github.com/ekaterinadimitrova2/cassandra/commit/8389010d704a9524a1c36d96f9e4d8179a8e2044] is to return the default config.yml This patch requires custom [ [DTests | https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] ] which run successfully with the following patch of [CCM |https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3] DTests - requirements.txt should be updated back to the original one before commit! The DTests are not backward compatible with the yaml file without units suffixes. This part is covered by a unit test which tests the successful load of the values of the parameters without units as per the old format. [ [trunk |https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-3-rebase ] ] [ [DTests | https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] ] [ [CCM |https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3] ] [ [JAVA8 |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/718c8a04-385b-4ddc-ac2e-9291c37b1f1e] ] [ [JAVA11 |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/6cd6048d-5daf-4490-a5d3-b5c3178a0757 ] ] Attached to the ticket is the log of the successful runs of the few failing JAVA8 DTests in CircleCI. I ran them locally after fixing lost line of code as it didn't make sense to me to rerun the whole suite of tests. *Last update:* after rebase and a few nits on my end, we have only 1 DTest and 1 Unit test failing in JAVA11. Unrelated known failures. CQLSH JAVA11 tests are not currently available in CircleCI. This branch is a class of Accessors for System properties. *This patch could be considered separate of the previous one.* It doesn't require custom DTests or custom CCM. [ [trunk |https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-5] ] [ [JAVA8 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/ebd771ac-d71c-49ea-872b-00b534d9d5c6] ] [ [JAVA11 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/19a3254b-cd7a-4e4f-856f-f314546ff9d7] ] There are 1 DTest and 1 Unit unrelated failing tests in JAVA11. Should check for tickets logged. *WARNING:* Before commit return to the default requirements.txt file was (Author: e.dimitrova): *Summary:* This branch contains: - [first commit | https://github.com/ekaterinadimitrova2/cassandra/commit/a8a9145a032a9f2d9b74369f2e4474a210aa5b99] which contains parameters name changes as requested; backward compatibility with the old names; Proposed new structure into [sections |https://issues.apache.org/jira/browse/CASSANDRA-15234?focusedCommentId=17057969=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17057969] for cassandra.yaml. - the [second commit | https://github.com/ekaterinadimitrova2/cassandra/commit/032fbd8cca1d7b5b96738bd96d9672864426d718] implements the functionality which gives the end users the opportunity to attach units suffixes to Memory, Duration, Rates parameters. - [third commit |https://github.com/ekaterinadimitrova2/cassandra/commit/68f89503e6e6f30e8f6df1374cb553f243f0068e] is a ninja fix of one line which was changed in a wrong way during the latest rebase. - [fourth commit |https://github.com/ekaterinadimitrova2/cassandra/commit/8389010d704a9524a1c36d96f9e4d8179a8e2044] is to return the default config.yml This patch requires custom [ [DTests | https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] ] which run successfully with the following patch of [CCM
[jira] [Comment Edited] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126974#comment-17126974 ] Ekaterina Dimitrova edited comment on CASSANDRA-15234 at 6/5/20, 5:50 PM: -- *Summary:* This branch contains: - [first commit | https://github.com/ekaterinadimitrova2/cassandra/commit/a8a9145a032a9f2d9b74369f2e4474a210aa5b99] which contains parameters name changes as requested; backward compatibility with the old names; Proposed new structure into [sections |https://issues.apache.org/jira/browse/CASSANDRA-15234?focusedCommentId=17057969=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17057969] for cassandra.yaml. - the [second commit | https://github.com/ekaterinadimitrova2/cassandra/commit/032fbd8cca1d7b5b96738bd96d9672864426d718] implements the functionality which gives the end users the opportunity to attach units suffixes to Memory, Duration, Rates parameters. - [third commit |https://github.com/ekaterinadimitrova2/cassandra/commit/68f89503e6e6f30e8f6df1374cb553f243f0068e] is a ninja fix of one line which was changed in a wrong way during the latest rebase. - [fourth commit |https://github.com/ekaterinadimitrova2/cassandra/commit/8389010d704a9524a1c36d96f9e4d8179a8e2044] is to return the default config.yml This patch requires custom [ [DTests | https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] ] which run successfully with the following patch of [CCM |https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3] DTests - requirements.txt should be updated back to the original one before commit! The DTests are not backward compatible with the yaml file without units suffixes. This part is covered by a unit test which tests the successful load of the values of the parameters without units as per the old format. [ [trunk |https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-3-rebase ] ] [ [DTests | https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] ] [ [CCM |https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3] ] [ [JAVA8 |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/718c8a04-385b-4ddc-ac2e-9291c37b1f1e] ] [ [JAVA11 |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/6cd6048d-5daf-4490-a5d3-b5c3178a0757 ] ] Attached to the ticket is the log of the successful runs of the few failing JAVA8 DTests in CircleCI. I ran them locally after fixing lost line of code as it didn't make sense to me to rerun the whole suite of tests. *Last update:* after rebase and a few nits on my end, we have only 1 DTest and 1 Unit test failing in JAVA11. Unrelated known failures. CQLSH JAVA11 tests are not currently available in CircleCI. *WARNING:* Before commit return to the default requirements.txt file This branch is a class of Accessors for System properties. *This patch could be considered separate of the previous one.* It doesn't require custom DTests or custom CCM. [ [trunk |https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-5] ] [ [JAVA8 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/ebd771ac-d71c-49ea-872b-00b534d9d5c6] ] [ [JAVA11 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/19a3254b-cd7a-4e4f-856f-f314546ff9d7] ] There are 1 DTest and 1 Unit unrelated failing tests in JAVA11. Should check for tickets logged. was (Author: e.dimitrova): *Summary:* This branch contains: - [first commit | https://github.com/ekaterinadimitrova2/cassandra/commit/a8a9145a032a9f2d9b74369f2e4474a210aa5b99] which contains parameters name changes as requested; backward compatibility with the old names; Proposed new structure into [sections |https://issues.apache.org/jira/browse/CASSANDRA-15234?focusedCommentId=17057969=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17057969] for cassandra.yaml. - the [second commit | https://github.com/ekaterinadimitrova2/cassandra/commit/032fbd8cca1d7b5b96738bd96d9672864426d718] implements the functionality which gives the end users the opportunity to attach units suffixes to Memory, Duration, Rates parameters. - [third commit |https://github.com/ekaterinadimitrova2/cassandra/commit/68f89503e6e6f30e8f6df1374cb553f243f0068e] is a ninja fix of one line which was changed in a wrong way during the latest rebase. - [fourth commit |https://github.com/ekaterinadimitrova2/cassandra/commit/8389010d704a9524a1c36d96f9e4d8179a8e2044] is to return the default config.yml This patch requires custom [ [DTests | https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] ] which run successfully with the following patch of [CCM
[jira] [Comment Edited] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126974#comment-17126974 ] Ekaterina Dimitrova edited comment on CASSANDRA-15234 at 6/5/20, 5:49 PM: -- *Summary:* This branch contains: - [first commit | https://github.com/ekaterinadimitrova2/cassandra/commit/a8a9145a032a9f2d9b74369f2e4474a210aa5b99] which contains parameters name changes as requested; backward compatibility with the old names; Proposed new structure into [sections |https://issues.apache.org/jira/browse/CASSANDRA-15234?focusedCommentId=17057969=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17057969] for cassandra.yaml. - the [second commit | https://github.com/ekaterinadimitrova2/cassandra/commit/032fbd8cca1d7b5b96738bd96d9672864426d718] implements the functionality which gives the end users the opportunity to attach units suffixes to Memory, Duration, Rates parameters. - [third commit |https://github.com/ekaterinadimitrova2/cassandra/commit/68f89503e6e6f30e8f6df1374cb553f243f0068e] is a ninja fix of one line which was changed in a wrong way during the latest rebase. - [fourth commit |https://github.com/ekaterinadimitrova2/cassandra/commit/8389010d704a9524a1c36d96f9e4d8179a8e2044] is to return the default config.yml This patch requires custom [ [DTests | https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] ] which run successfully with the following patch of [CCM |https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3] DTests - requirements.txt should be updated back to the original one before commit! The DTests are not backward compatible with the yaml file without units suffixes. This part is covered by a unit test which tests the successful load of the values of the parameters without units as per the old format. [ [trunk |https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-3-rebase ] ] [ [DTests | https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] ] [ [CCM |https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3] ] [ [JAVA8 |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/718c8a04-385b-4ddc-ac2e-9291c37b1f1e] ] [ [JAVA11 |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/6cd6048d-5daf-4490-a5d3-b5c3178a0757 ] ] Attached to the ticket is the log of the successful runs of the few failing JAVA8 DTests in CircleCI. I ran them locally after fixing lost line of code as it didn't make sense to me to rerun the whole suite of tests. *Last update:* after rebase and a few nits on my end, we have only 1 DTest and 1 Unit test failing in JAVA11. Unrelated known failures. CQLSH JAVA11 tests are not currently available in CircleCI. This branch is a class of Accessors System properties. *This patch could be considered separate of the previous one.* It doesn't require custom DTests or custom CCM. [ [trunk |https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-5] ] [ [JAVA8 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/ebd771ac-d71c-49ea-872b-00b534d9d5c6] ] [ [JAVA11 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/19a3254b-cd7a-4e4f-856f-f314546ff9d7] ] There are 1 DTest and 1 Unit unrelated failing tests in JAVA11. Should check for tickets logged. *WARNING:* Before commit return to the default requirements.txt file was (Author: e.dimitrova): *Summary:* This branch contains: - [first commit | https://github.com/ekaterinadimitrova2/cassandra/commit/a8a9145a032a9f2d9b74369f2e4474a210aa5b99] which contains parameters name changes as requested; backward compatibility with the old names; Proposed new structure into [sections |https://issues.apache.org/jira/browse/CASSANDRA-15234?focusedCommentId=17057969=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17057969] for cassandra.yaml. - the [second commit | https://github.com/ekaterinadimitrova2/cassandra/commit/032fbd8cca1d7b5b96738bd96d9672864426d718] implements the functionality which gives the end users the opportunity to attach units suffixes to Memory, Duration, Rates parameters. - [third commit |https://github.com/ekaterinadimitrova2/cassandra/commit/68f89503e6e6f30e8f6df1374cb553f243f0068e] is a ninja fix of one line which was changed in a wrong way during the latest rebase. - [fourth commit |https://github.com/ekaterinadimitrova2/cassandra/commit/8389010d704a9524a1c36d96f9e4d8179a8e2044] is to return the default config.yml This patch requires custom [ [DTests | https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] ] which run successfully with the following patch of [CCM
[jira] [Updated] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15234:
Test and Documentation Plan:
https://issues.apache.org/jira/browse/CASSANDRA-15234?focusedCommentId=17126974=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17126974
(was: [ [trunk
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-3-rebase
] ] [ [DTests |
https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3]
] [ [CCM |https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3] ]
[ [JAVA8 CI
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/718c8a04-385b-4ddc-ac2e-9291c37b1f1e]
] [ [JAVA11 CI
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/6cd6048d-5daf-4490-a5d3-b5c3178a0757
] ]
Attached to the ticket is the log of the successful runs of the few failing
JAVA8 DTests in CircleCI. I ran them locally after fixing lost line of code as
it didn't make sense to me to rerun the whole suite of tests.
*Last update:* after rebase and a few nits on my end, we have only 1 DTest and
1 Unit test failing in JAVA11. Unrelated known failures.
- added additional note on the possible units to be used as suffixes in both
NEWS.txt and cassandra.yaml
CQLSH JAVA11 tests are not currently available in CircleCI.
The last part is all green after rebase and CI run.
- "line.separator" getter added
[ [trunk
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-5] ] [
[JAVA8 CI
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/ebd771ac-d71c-49ea-872b-00b534d9d5c6]
] [ [JAVA11 CI
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/19a3254b-cd7a-4e4f-856f-f314546ff9d7]
]
There are 1 DTest and 1 Unit unrelated failing tests in JAVA11. Should check
for tickets logged.
*NOTE:* Before commit return to the default config.yml and requirements.txt
files)
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126974#comment-17126974
]
Ekaterina Dimitrova commented on CASSANDRA-15234:
-
*Summary:*
This branch contains:
- [first commit |
https://github.com/ekaterinadimitrova2/cassandra/commit/a8a9145a032a9f2d9b74369f2e4474a210aa5b99]
which contains parameters name changes as requested; backward compatibility
with the old names; Proposed new structure into [sections
|https://issues.apache.org/jira/browse/CASSANDRA-15234?focusedCommentId=17057969=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17057969]
for cassandra.yaml.
- the [second commit |
https://github.com/ekaterinadimitrova2/cassandra/commit/032fbd8cca1d7b5b96738bd96d9672864426d718]
implements the functionality which gives the end users the opportunity to
attach units suffixes to Memory, Duration, Rates parameters.
- [third commit
|https://github.com/ekaterinadimitrova2/cassandra/commit/68f89503e6e6f30e8f6df1374cb553f243f0068e]
is a ninja fix of one line which was changed in a wrong way during the latest
rebase.
- [fourth commit
|https://github.com/ekaterinadimitrova2/cassandra/commit/8389010d704a9524a1c36d96f9e4d8179a8e2044]
is to return the default config.yml
This patch requires custom [ [DTests |
https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3]
] which run successfully with the following patch of [CCM
|https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3]
DTests - requirements.txt should be updated back to the original one before
commit!
The tests are not backward compatible with the yaml file without units suffixes.
This part is covered by a unit test which tests the successful load of the
values of the parameters without units as per the old format.
[ [trunk
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-3-rebase
] ] [ [DTests |
https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3]
] [ [CCM |https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3] ]
[ [JAVA8
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/718c8a04-385b-4ddc-ac2e-9291c37b1f1e]
] [ [JAVA11
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/6cd6048d-5daf-4490-a5d3-b5c3178a0757
] ]
Attached to the ticket is the log of the successful runs of the few failing
JAVA8 DTests in CircleCI. I ran them locally after fixing lost line of code as
it didn't make sense to me to rerun the whole suite of tests.
*Last update:* after rebase and a few nits on my end, we have only 1 DTest and
1 Unit test failing in JAVA11. Unrelated known failures.
CQLSH JAVA11 tests are not currently available in CircleCI.
This branch is a class of Accessors System properties. *This patch could be
considered separate of the previous one.* It doesn't require custom DTests or
custom CCM.
[ [trunk
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-5] ] [
[JAVA8 CI
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/ebd771ac-d71c-49ea-872b-00b534d9d5c6]
] [ [JAVA11 CI
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/19a3254b-cd7a-4e4f-856f-f314546ff9d7]
]
There are 1 DTest and 1 Unit unrelated failing tests in JAVA11. Should check
for tickets logged.
*WARNING:* Before commit return to the default requirements.txt file
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the
[jira] [Updated] (CASSANDRA-15733) jvm dtest builder should be provided to the factory and expose state
[ https://issues.apache.org/jira/browse/CASSANDRA-15733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-15733: Fix Version/s: 4.0 3.11.7 3.0.21 2.2.17 Source Control Link: https://github.com/apache/cassandra/commit/4fef917f985ea13ea46f68aae00645cb3592ffcf Resolution: Fixed Status: Resolved (was: Ready to Commit) > jvm dtest builder should be provided to the factory and expose state > > > Key: CASSANDRA-15733 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15733 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Fix For: 2.2.17, 3.0.21, 3.11.7, 4.0 > > Time Spent: 3h > Remaining Estimate: 0h > > Currently the builder is rather heavy and creates configs plus call the > factory with specific fields only, this isn’t that flexible and makes it > harder to have custom cluster definitions which require additional fields to > be defined. To solve this we should make the builder be sent to the factory > and expose the state so the factory can get all the fields it needs, the > factory should also be in charge of creating the configs -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15733) jvm dtest builder should be provided to the factory and expose state
[ https://issues.apache.org/jira/browse/CASSANDRA-15733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-15733: Status: Ready to Commit (was: Review In Progress) > jvm dtest builder should be provided to the factory and expose state > > > Key: CASSANDRA-15733 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15733 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Time Spent: 3h > Remaining Estimate: 0h > > Currently the builder is rather heavy and creates configs plus call the > factory with specific fields only, this isn’t that flexible and makes it > harder to have custom cluster definitions which require additional fields to > be defined. To solve this we should make the builder be sent to the factory > and expose the state so the factory can get all the fields it needs, the > factory should also be in charge of creating the configs -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15733) jvm dtest builder should be provided to the factory and expose state
[ https://issues.apache.org/jira/browse/CASSANDRA-15733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-15733: Reviewers: Alex Petrov, Alex Petrov (was: Alex Petrov) Alex Petrov, Alex Petrov (was: Alex Petrov) Status: Review In Progress (was: Patch Available) > jvm dtest builder should be provided to the factory and expose state > > > Key: CASSANDRA-15733 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15733 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Time Spent: 3h > Remaining Estimate: 0h > > Currently the builder is rather heavy and creates configs plus call the > factory with specific fields only, this isn’t that flexible and makes it > harder to have custom cluster definitions which require additional fields to > be defined. To solve this we should make the builder be sent to the factory > and expose the state so the factory can get all the fields it needs, the > factory should also be in charge of creating the configs -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15733) jvm dtest builder should be provided to the factory and expose state
[ https://issues.apache.org/jira/browse/CASSANDRA-15733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126972#comment-17126972 ] Alex Petrov commented on CASSANDRA-15733: - Sorry for the delay. +1, and thank you for the patch. Committed to 2.2 with [4fef917f985ea13ea46f68aae00645cb3592ffcf |https://github.com/apache/cassandra/commit/4fef917f985ea13ea46f68aae00645cb3592ffcf], and merged up to [3.0|https://github.com/apache/cassandra/commit/880b07c2e63f6e1dea92995a1baa14707e3c2e5e], [3.11|https://github.com/apache/cassandra/commit/04b00498312c2627bd180485565fbd1b18e1fbc3], and [trunk|https://github.com/apache/cassandra/commit/7bbc97e5835b64c49fd418581f5a45429c1806b1]. > jvm dtest builder should be provided to the factory and expose state > > > Key: CASSANDRA-15733 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15733 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Time Spent: 3h > Remaining Estimate: 0h > > Currently the builder is rather heavy and creates configs plus call the > factory with specific fields only, this isn’t that flexible and makes it > harder to have custom cluster definitions which require additional fields to > be defined. To solve this we should make the builder be sent to the factory > and expose the state so the factory can get all the fields it needs, the > factory should also be in charge of creating the configs -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (dd255ff -> 880b07c)
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from dd255ff Fix replica-side filtering returning stale data with CL > 1 new 4fef917 In-jvm dtest builder should be provided to the factory and expose state new 880b07c Merge branch 'cassandra-2.2' into cassandra-3.0 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: build.xml | 2 +- .../org/apache/cassandra/distributed/Cluster.java | 38 +++- .../cassandra/distributed/UpgradeableCluster.java | 33 +++--- .../distributed/impl/AbstractCluster.java | 53 ++ .../cassandra/distributed/impl/Coordinator.java| 33 ++ .../impl/DelegatingInvokableInstance.java | 6 +++ .../cassandra/distributed/impl/Instance.java | 10 ++-- .../apache/cassandra/distributed/impl/RowUtil.java | 22 + .../cassandra/distributed/test/BootstrapTest.java | 18 +++- .../cassandra/distributed/test/TestBaseImpl.java | 9 ++-- .../distributed/upgrade/UpgradeTestBase.java | 10 ++-- 11 files changed, 139 insertions(+), 95 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.2 updated: In-jvm dtest builder should be provided to the factory and expose state
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch cassandra-2.2
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-2.2 by this push:
new 4fef917 In-jvm dtest builder should be provided to the factory and
expose state
4fef917 is described below
commit 4fef917f985ea13ea46f68aae00645cb3592ffcf
Author: David Capwell
AuthorDate: Wed Apr 15 13:33:45 2020 -0700
In-jvm dtest builder should be provided to the factory and expose state
Patch by David Capwell; reviewed by Alex Petrov for CASSANDRA-15733.
---
build.xml | 2 +-
.../org/apache/cassandra/distributed/Cluster.java | 38 +++-
.../cassandra/distributed/UpgradeableCluster.java | 33 +++---
.../distributed/impl/AbstractCluster.java | 53 ++
.../cassandra/distributed/impl/Coordinator.java| 28 +---
.../impl/DelegatingInvokableInstance.java | 6 +++
.../cassandra/distributed/impl/Instance.java | 10 ++--
.../apache/cassandra/distributed/impl/RowUtil.java | 22 +
.../cassandra/distributed/test/BootstrapTest.java | 18 +++-
.../cassandra/distributed/test/TestBaseImpl.java | 9 ++--
.../distributed/upgrade/UpgradeTestBase.java | 10 ++--
11 files changed, 136 insertions(+), 93 deletions(-)
diff --git a/build.xml b/build.xml
index 028210b..0fa2380 100644
--- a/build.xml
+++ b/build.xml
@@ -395,7 +395,7 @@
-
+
diff --git a/test/distributed/org/apache/cassandra/distributed/Cluster.java
b/test/distributed/org/apache/cassandra/distributed/Cluster.java
index d657638..95ead50 100644
--- a/test/distributed/org/apache/cassandra/distributed/Cluster.java
+++ b/test/distributed/org/apache/cassandra/distributed/Cluster.java
@@ -18,17 +18,13 @@
package org.apache.cassandra.distributed;
-import java.io.File;
import java.io.IOException;
-import java.util.List;
import java.util.function.Consumer;
import org.apache.cassandra.distributed.api.IInstanceConfig;
import org.apache.cassandra.distributed.api.IInvokableInstance;
import org.apache.cassandra.distributed.impl.AbstractCluster;
-import org.apache.cassandra.distributed.impl.InstanceConfig;
-import org.apache.cassandra.distributed.shared.Builder;
-import org.apache.cassandra.distributed.shared.NetworkTopology;
+import org.apache.cassandra.distributed.shared.AbstractBuilder;
import org.apache.cassandra.distributed.shared.Versions;
/**
@@ -38,9 +34,9 @@ import org.apache.cassandra.distributed.shared.Versions;
public class Cluster extends AbstractCluster
{
-private Cluster(File root, Versions.Version version, List
configs, ClassLoader sharedClassLoader)
+private Cluster(Builder builder)
{
-super(root, version, configs, sharedClassLoader);
+super(builder);
}
protected IInvokableInstance newInstanceWrapper(int generation,
Versions.Version version, IInstanceConfig config)
@@ -48,22 +44,12 @@ public class Cluster extends
AbstractCluster
return new Wrapper(generation, version, config);
}
-public static Builder build()
+public static Builder build()
{
-return new Builder(Cluster::new)
-{
-{
-withVersion(CURRENT_VERSION);
-}
-
-protected IInstanceConfig generateConfig(int nodeNum, String
ipAddress, NetworkTopology networkTopology, File root, String token, String
seedIp)
-{
-return InstanceConfig.generate(nodeNum, ipAddress,
networkTopology, root, token, seedIp);
-}
-};
+return new Builder();
}
-public static Builder build(int nodeCount)
+public static Builder build(int nodeCount)
{
return build().withNodes(nodeCount);
}
@@ -77,4 +63,14 @@ public class Cluster extends
AbstractCluster
{
return build(nodeCount).start();
}
-}
\ No newline at end of file
+
+public static final class Builder extends
AbstractBuilder
+{
+public Builder()
+{
+super(Cluster::new);
+withVersion(CURRENT_VERSION);
+}
+}
+}
+
diff --git
a/test/distributed/org/apache/cassandra/distributed/UpgradeableCluster.java
b/test/distributed/org/apache/cassandra/distributed/UpgradeableCluster.java
index 71f3f8c..bde5d4e 100644
--- a/test/distributed/org/apache/cassandra/distributed/UpgradeableCluster.java
+++ b/test/distributed/org/apache/cassandra/distributed/UpgradeableCluster.java
@@ -18,17 +18,13 @@
package org.apache.cassandra.distributed;
-import java.io.File;
import java.io.IOException;
-import java.util.List;
import java.util.function.Consumer;
import org.apache.cassandra.distributed.api.IInstanceConfig;
-import
[cassandra] branch trunk updated (69ea5ff -> 7bbc97e)
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 69ea5ff Merge branch 'cassandra-3.11' into trunk new 4fef917 In-jvm dtest builder should be provided to the factory and expose state new 880b07c Merge branch 'cassandra-2.2' into cassandra-3.0 new 04b0049 Merge branch 'cassandra-3.0' into cassandra-3.11 new 7bbc97e Merge branch 'cassandra-3.11' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: build.xml | 2 +- .../org/apache/cassandra/distributed/Cluster.java | 36 ++- .../cassandra/distributed/UpgradeableCluster.java | 34 +++--- .../distributed/impl/AbstractCluster.java | 53 ++ .../cassandra/distributed/impl/Coordinator.java| 30 +--- .../impl/DelegatingInvokableInstance.java | 6 +++ .../cassandra/distributed/impl/Instance.java | 11 ++--- .../apache/cassandra/distributed/impl/RowUtil.java | 22 + .../cassandra/distributed/test/BootstrapTest.java | 18 +++- .../cassandra/distributed/test/TestBaseImpl.java | 6 +-- .../distributed/upgrade/UpgradeTestBase.java | 10 ++-- 11 files changed, 132 insertions(+), 96 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (7014cd5 -> 04b0049)
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 7014cd5 Merge branch 'cassandra-3.0' into cassandra-3.11 new 4fef917 In-jvm dtest builder should be provided to the factory and expose state new 880b07c Merge branch 'cassandra-2.2' into cassandra-3.0 new 04b0049 Merge branch 'cassandra-3.0' into cassandra-3.11 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: build.xml | 2 +- .../org/apache/cassandra/distributed/Cluster.java | 38 +++- .../cassandra/distributed/UpgradeableCluster.java | 33 +++--- .../distributed/impl/AbstractCluster.java | 53 ++ .../cassandra/distributed/impl/Coordinator.java| 30 +--- .../impl/DelegatingInvokableInstance.java | 6 +++ .../cassandra/distributed/impl/Instance.java | 10 ++-- .../apache/cassandra/distributed/impl/RowUtil.java | 22 + .../cassandra/distributed/test/BootstrapTest.java | 18 +++- .../cassandra/distributed/test/TestBaseImpl.java | 9 ++-- .../distributed/upgrade/UpgradeTestBase.java | 10 ++-- 11 files changed, 137 insertions(+), 94 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 7bbc97e5835b64c49fd418581f5a45429c1806b1
Merge: 69ea5ff 04b0049
Author: Alex Petrov
AuthorDate: Fri Jun 5 19:34:13 2020 +0200
Merge branch 'cassandra-3.11' into trunk
build.xml | 2 +-
.../org/apache/cassandra/distributed/Cluster.java | 36 ++-
.../cassandra/distributed/UpgradeableCluster.java | 34 +++---
.../distributed/impl/AbstractCluster.java | 53 ++
.../cassandra/distributed/impl/Coordinator.java| 30 +---
.../impl/DelegatingInvokableInstance.java | 6 +++
.../cassandra/distributed/impl/Instance.java | 11 ++---
.../apache/cassandra/distributed/impl/RowUtil.java | 22 +
.../cassandra/distributed/test/BootstrapTest.java | 18 +++-
.../cassandra/distributed/test/TestBaseImpl.java | 6 +--
.../distributed/upgrade/UpgradeTestBase.java | 10 ++--
11 files changed, 132 insertions(+), 96 deletions(-)
diff --cc build.xml
index 812efc0,3fd409b..7760385
--- a/build.xml
+++ b/build.xml
@@@ -534,24 -401,28 +534,24 @@@
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
-
+
-
-
-
-
-
-
-
-
+
+
+
-
+
diff --cc test/distributed/org/apache/cassandra/distributed/Cluster.java
index ee01874,95ead50..e8e6041
--- a/test/distributed/org/apache/cassandra/distributed/Cluster.java
+++ b/test/distributed/org/apache/cassandra/distributed/Cluster.java
@@@ -18,18 -18,13 +18,13 @@@
package org.apache.cassandra.distributed;
- import java.io.File;
import java.io.IOException;
- import java.util.List;
import java.util.function.Consumer;
- import org.apache.cassandra.distributed.api.ICluster;
import org.apache.cassandra.distributed.api.IInstanceConfig;
-import org.apache.cassandra.distributed.api.IInvokableInstance;
import org.apache.cassandra.distributed.impl.AbstractCluster;
+import org.apache.cassandra.distributed.api.IInvokableInstance;
- import org.apache.cassandra.distributed.impl.InstanceConfig;
- import org.apache.cassandra.distributed.shared.Builder;
- import org.apache.cassandra.distributed.shared.NetworkTopology;
+ import org.apache.cassandra.distributed.shared.AbstractBuilder;
import org.apache.cassandra.distributed.shared.Versions;
/**
diff --cc
test/distributed/org/apache/cassandra/distributed/UpgradeableCluster.java
index 57ace93,bde5d4e..532c1b1
--- a/test/distributed/org/apache/cassandra/distributed/UpgradeableCluster.java
+++ b/test/distributed/org/apache/cassandra/distributed/UpgradeableCluster.java
@@@ -81,5 -70,13 +70,14 @@@ public class UpgradeableCluster extend
{
return build(nodeCount).withVersion(version).start();
}
+
+ public static final class Builder extends
AbstractBuilder
+ {
+
+ public Builder()
+ {
+ super(UpgradeableCluster::new);
+ }
+ }
}
+
diff --cc
test/distributed/org/apache/cassandra/distributed/impl/Coordinator.java
index c1446d1,329fa37..2ee209d
--- a/test/distributed/org/apache/cassandra/distributed/impl/Coordinator.java
+++ b/test/distributed/org/apache/cassandra/distributed/impl/Coordinator.java
@@@ -79,10 -81,10 +81,10 @@@ public class Coordinator implements ICo
return
org.apache.cassandra.db.ConsistencyLevel.fromCode(cl.ordinal());
}
- private QueryResult executeInternal(String query, ConsistencyLevel
consistencyLevelOrigin, Object[] boundValues)
+ private SimpleQueryResult executeInternal(String query, ConsistencyLevel
consistencyLevelOrigin, Object[] boundValues)
{
ClientState clientState = makeFakeClientState();
-CQLStatement prepared = QueryProcessor.getStatement(query,
clientState).statement;
+CQLStatement prepared = QueryProcessor.getStatement(query,
clientState);
List boundBBValues = new ArrayList<>();
ConsistencyLevel consistencyLevel =
ConsistencyLevel.valueOf(consistencyLevelOrigin.name());
for (Object boundValue : boundValues)
@@@ -96,21 -98,10 +98,11 @@@
Integer.MAX_VALUE,
null,
null,
-
ProtocolVersion.CURRENT),
+
[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 880b07c2e63f6e1dea92995a1baa14707e3c2e5e
Merge: dd255ff 4fef917
Author: Alex Petrov
AuthorDate: Fri Jun 5 19:25:27 2020 +0200
Merge branch 'cassandra-2.2' into cassandra-3.0
build.xml | 2 +-
.../org/apache/cassandra/distributed/Cluster.java | 38 +++-
.../cassandra/distributed/UpgradeableCluster.java | 33 +++---
.../distributed/impl/AbstractCluster.java | 53 ++
.../cassandra/distributed/impl/Coordinator.java| 33 ++
.../impl/DelegatingInvokableInstance.java | 6 +++
.../cassandra/distributed/impl/Instance.java | 10 ++--
.../apache/cassandra/distributed/impl/RowUtil.java | 22 +
.../cassandra/distributed/test/BootstrapTest.java | 18 +++-
.../cassandra/distributed/test/TestBaseImpl.java | 9 ++--
.../distributed/upgrade/UpgradeTestBase.java | 10 ++--
11 files changed, 139 insertions(+), 95 deletions(-)
diff --cc
test/distributed/org/apache/cassandra/distributed/impl/Coordinator.java
index e2ebef0,6553fb9..2f2b525
--- a/test/distributed/org/apache/cassandra/distributed/impl/Coordinator.java
+++ b/test/distributed/org/apache/cassandra/distributed/impl/Coordinator.java
@@@ -35,9 -36,13 +36,11 @@@ import org.apache.cassandra.distributed
import org.apache.cassandra.distributed.api.ICoordinator;
import org.apache.cassandra.distributed.api.IInstance;
import org.apache.cassandra.distributed.api.QueryResult;
+ import org.apache.cassandra.distributed.api.QueryResults;
+ import org.apache.cassandra.distributed.api.SimpleQueryResult;
import org.apache.cassandra.service.ClientState;
import org.apache.cassandra.service.QueryState;
-import org.apache.cassandra.service.pager.Pageable;
import org.apache.cassandra.service.pager.QueryPager;
-import org.apache.cassandra.service.pager.QueryPagers;
import org.apache.cassandra.transport.Server;
import org.apache.cassandra.tracing.Tracing;
import org.apache.cassandra.transport.messages.ResultMessage;
@@@ -78,9 -83,9 +81,9 @@@ public class Coordinator implements ICo
return
org.apache.cassandra.db.ConsistencyLevel.fromCode(cl.ordinal());
}
- private QueryResult executeInternal(String query, ConsistencyLevel
consistencyLevelOrigin, Object[] boundValues)
+ private SimpleQueryResult executeInternal(String query, ConsistencyLevel
consistencyLevelOrigin, Object[] boundValues)
{
-ClientState clientState = ClientState.forInternalCalls();
+ClientState clientState = makeFakeClientState();
CQLStatement prepared = QueryProcessor.getStatement(query,
clientState).statement;
List boundBBValues = new ArrayList<>();
ConsistencyLevel consistencyLevel =
ConsistencyLevel.valueOf(consistencyLevelOrigin.name());
@@@ -136,29 -130,32 +129,30 @@@
boundBBValues.add(ByteBufferUtil.objectToBytes(boundValue));
}
-prepared.validate(QueryState.forInternalCalls().getClientState());
+prepared.validate(clientState);
assert prepared instanceof SelectStatement : "Only SELECT
statements can be executed with paging";
-ClientState clientState =
QueryState.forInternalCalls().getClientState();
SelectStatement selectStatement = (SelectStatement) prepared;
-QueryOptions queryOptions =
QueryOptions.create(toCassandraCL(consistencyLevel),
-boundBBValues,
-false,
-pageSize,
-null,
-null,
-
Server.CURRENT_VERSION);
-Pageable pageable =
selectStatement.getPageableCommand(queryOptions);
+
+QueryPager pager =
selectStatement.getQuery(QueryOptions.create(toCassandraCL(consistencyLevel),
+
boundBBValues,
+
false,
+
pageSize,
+
null,
+
null,
+
Server.CURRENT_VERSION),
+
FBUtilities.nowInSeconds())
- .getPager(null,
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 04b00498312c2627bd180485565fbd1b18e1fbc3
Merge: 7014cd5 880b07c
Author: Alex Petrov
AuthorDate: Fri Jun 5 19:26:56 2020 +0200
Merge branch 'cassandra-3.0' into cassandra-3.11
build.xml | 2 +-
.../org/apache/cassandra/distributed/Cluster.java | 38 +++-
.../cassandra/distributed/UpgradeableCluster.java | 33 +++---
.../distributed/impl/AbstractCluster.java | 53 ++
.../cassandra/distributed/impl/Coordinator.java| 30 +---
.../impl/DelegatingInvokableInstance.java | 6 +++
.../cassandra/distributed/impl/Instance.java | 10 ++--
.../apache/cassandra/distributed/impl/RowUtil.java | 22 +
.../cassandra/distributed/test/BootstrapTest.java | 18 +++-
.../cassandra/distributed/test/TestBaseImpl.java | 9 ++--
.../distributed/upgrade/UpgradeTestBase.java | 10 ++--
11 files changed, 137 insertions(+), 94 deletions(-)
diff --cc
test/distributed/org/apache/cassandra/distributed/impl/Coordinator.java
index d49679d,2f2b525..329fa37
--- a/test/distributed/org/apache/cassandra/distributed/impl/Coordinator.java
+++ b/test/distributed/org/apache/cassandra/distributed/impl/Coordinator.java
@@@ -64,8 -66,8 +66,8 @@@ public class Coordinator implements ICo
return instance.async(() -> {
try
{
-Tracing.instance.newSession(sessionId);
+Tracing.instance.newSession(sessionId,
Collections.emptyMap());
- return executeInternal(query, consistencyLevelOrigin,
boundValues).toObjectArrays();
+ return executeInternal(query, consistencyLevelOrigin,
boundValues);
}
finally
{
@@@ -96,20 -98,9 +98,10 @@@
Integer.MAX_VALUE,
null,
null,
-
Server.CURRENT_VERSION));
+
ProtocolVersion.CURRENT),
+ System.nanoTime());
- if (res != null && res.kind == ResultMessage.Kind.ROWS)
- {
- ResultMessage.Rows rows = (ResultMessage.Rows) res;
- String[] names = rows.result.metadata.names.stream().map(c ->
c.name.toString()).toArray(String[]::new);
- Object[][] results = RowUtil.toObjects(rows);
- return new QueryResult(names, results);
- }
- else
- {
- return QueryResult.EMPTY;
- }
+ return RowUtil.toQueryResult(res);
}
public Object[][] executeWithTracing(UUID sessionId, String query,
ConsistencyLevel consistencyLevelOrigin, Object... boundValues)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12126) CAS Reads Inconsistencies
[ https://issues.apache.org/jira/browse/CASSANDRA-12126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126937#comment-17126937 ] Sylvain Lebresne commented on CASSANDRA-12126: -- bq. The test cases I provided demonstrate several consistency violations during range movements. Yes, sorry I hadn't read them before commenting. And I certainly agree those are problematic (I was about to open a ticket so it's tracked, but I'd say CASSANDRA-15745 kind of cover those). bq. There are also (more debatably) issues with TTL on system.paxos Agreed this has always been a weak point. It does feel somewhat separated of other consistency points though, and maybe short term we can just offer a way to override the TTL (with documentation on the tradeoffs involved)? bq. Also, mixing LOCAL_SERIAL and SERIAL is entirely unsafe Yeah. I'm not sure how to fix that one without a breaking API change though (namely, limiting their unrestricted use together). It's not "that" different from the fact we allow unrestricted mixing of serial and non-serial operations. Which is something I don't like and I'm happy to discuss moving forward, but imo post-3.X material in the best of cases. bq. I think it is worth considering if we should instead aggressively try to remedy all of the known issues, have a strong verification push, and then roll out all of the changes at-once - including a fix for this that does not regress performance. It is certainly an option worth bringing, and thank you for that. I'm not sure how to really know what is the best option though, so I can only offer my current opinion. Which is that I feel this issue is a very serious issue. And I don't mean that in a way that diminishes the seriousness of the other problems you mentioned, I mean that in absolute terms (the range movement issues are also fairly bad imo for instance). But leaving less of our known serious unaddressed feels better than not, so I'd personally prefer fixing that issue ASAP. Basically, I'm worried that waiting for a more all-encompassing fix might take us quite some time, with no absolute guarantee that we'll be collectively at ease with pushing that to 3.X. Anyway, I'd like to move this forward personally. How do we decide if we do? > CAS Reads Inconsistencies > -- > > Key: CASSANDRA-12126 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12126 > Project: Cassandra > Issue Type: Bug > Components: Feature/Lightweight Transactions, Legacy/Coordination >Reporter: Sankalp Kohli >Assignee: Sylvain Lebresne >Priority: Normal > Labels: LWT, pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > While looking at the CAS code in Cassandra, I found a potential issue with > CAS Reads. Here is how it can happen with RF=3 > 1) You issue a CAS Write and it fails in the propose phase. A machine replies > true to a propose and saves the commit in accepted filed. The other two > machines B and C does not get to the accept phase. > Current state is that machine A has this commit in paxos table as accepted > but not committed and B and C does not. > 2) Issue a CAS Read and it goes to only B and C. You wont be able to read the > value written in step 1. This step is as if nothing is inflight. > 3) Issue another CAS Read and it goes to A and B. Now we will discover that > there is something inflight from A and will propose and commit it with the > current ballot. Now we can read the value written in step 1 as part of this > CAS read. > If we skip step 3 and instead run step 4, we will never learn about value > written in step 1. > 4. Issue a CAS Write and it involves only B and C. This will succeed and > commit a different value than step 1. Step 1 value will never be seen again > and was never seen before. > If you read the Lamport “paxos made simple” paper and read section 2.3. It > talks about this issue which is how learners can find out if majority of the > acceptors have accepted the proposal. > In step 3, it is correct that we propose the value again since we dont know > if it was accepted by majority of acceptors. When we ask majority of > acceptors, and more than one acceptors but not majority has something in > flight, we have no way of knowing if it is accepted by majority of acceptors. > So this behavior is correct. > However we need to fix step 2, since it caused reads to not be linearizable > with respect to writes and other reads. In this case, we know that majority > of acceptors have no inflight commit which means we have majority that > nothing was accepted by majority. I think we should run a propose step here > with empty commit and that will cause write written in step 1 to not be > visible ever after. > With this fix, we will either see data
[jira] [Updated] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15234:
---
Reviewers: Benjamin Lerer, Michael Semb Wever (was: Benjamin Lerer)
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15833) Unresolvable false digest mismatch during upgrade due to CASSANDRA-10657
[ https://issues.apache.org/jira/browse/CASSANDRA-15833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-15833: Reviewers: Jordan West, Jordan West (was: Jordan West) Jordan West, Jordan West Status: Review In Progress (was: Patch Available) Thanks for the patch [~jlewandowski]. I will take a look. > Unresolvable false digest mismatch during upgrade due to CASSANDRA-10657 > > > Key: CASSANDRA-15833 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15833 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Repair >Reporter: Jacek Lewandowski >Priority: Normal > Fix For: 3.11.x, 4.0-beta > > Attachments: CASSANDRA-15833-3.11.patch, CASSANDRA-15833-4.0.patch > > > CASSANDRA-10657 introduced changes in how the ColumnFilter is interpreted. > This results in digest mismatch when querying incomplete set of columns from > a table with consistency that requires reaching instances running pre > CASSANDRA-10657 from nodes that include CASSANDRA-10657 (it was introduced in > Cassandra 3.4). > The fix is to bring back the previous behaviour until there are no instances > running pre CASSANDRA-10657 version. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15833) Unresolvable false digest mismatch during upgrade due to CASSANDRA-10657
[ https://issues.apache.org/jira/browse/CASSANDRA-15833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West reassigned CASSANDRA-15833: --- Assignee: Jacek Lewandowski > Unresolvable false digest mismatch during upgrade due to CASSANDRA-10657 > > > Key: CASSANDRA-15833 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15833 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Repair >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 3.11.x, 4.0-beta > > Attachments: CASSANDRA-15833-3.11.patch, CASSANDRA-15833-4.0.patch > > > CASSANDRA-10657 introduced changes in how the ColumnFilter is interpreted. > This results in digest mismatch when querying incomplete set of columns from > a table with consistency that requires reaching instances running pre > CASSANDRA-10657 from nodes that include CASSANDRA-10657 (it was introduced in > Cassandra 3.4). > The fix is to bring back the previous behaviour until there are no instances > running pre CASSANDRA-10657 version. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch master updated: ninja-fix to e883522: Cassandra-devbranch-artifacts can also be concurrent (over different nodes)
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
The following commit(s) were added to refs/heads/master by this push:
new 4dce2b1 ninja-fix to e883522: Cassandra-devbranch-artifacts can also
be concurrent (over different nodes)
4dce2b1 is described below
commit 4dce2b180c82187d6b9e0e44745d6d604e2c12fa
Author: Mick Semb Wever
AuthorDate: Fri Jun 5 17:27:24 2020 +0200
ninja-fix to e883522: Cassandra-devbranch-artifacts can also be concurrent
(over different nodes)
---
jenkins-dsl/cassandra_job_dsl_seed.groovy | 1 +
1 file changed, 1 insertion(+)
diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy
b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index 48439d9..728e10f 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -476,6 +476,7 @@ cassandraBranches.each {
*/
job('Cassandra-devbranch-artifacts') {
description(jobDescription)
+concurrentBuild()
jdk(jdkLabel)
label(slaveLabel)
compressBuildLog()
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-8612) Read metrics should be updated on all types of reads
[ https://issues.apache.org/jira/browse/CASSANDRA-8612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126840#comment-17126840 ] Dean Z commented on CASSANDRA-8612: --- Hi [~cnlwsu] , you mean this ticket is resolved and nothing else need to be done right? > Read metrics should be updated on all types of reads > > > Key: CASSANDRA-8612 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8612 > Project: Cassandra > Issue Type: Improvement > Components: Observability/Metrics >Reporter: Chris Lohfink >Assignee: Dean Z >Priority: Low > Labels: lhf, metrics > Attachments: 0001-Update-read-metrics-on-all-types-of-reads.patch > > > Metrics like "sstables per read" are not updated on a range slice. Although > separating things out for each type of read could make sense like we do for > latencies, only exposing the metrics for one type can be a little confusing > when people do a query and see nothing increases. I think its sufficient to > use the same metrics for all reads. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15234:
---
Reviewers: Benjamin Lerer
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15146) Transitional TLS server configuration options are overly complex
[
https://issues.apache.org/jira/browse/CASSANDRA-15146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126834#comment-17126834
]
Ekaterina Dimitrova commented on CASSANDRA-15146:
-
[~jolynch], just to confirm what we talked and for the record, the final plan
is to rename only server_encryption_options to internode_encryption, right? Let
me know if there is any additional work that needs attention as part of this
ticket. Thanks
> Transitional TLS server configuration options are overly complex
>
>
> Key: CASSANDRA-15146
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15146
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Encryption, Local/Config
>Reporter: Joey Lynch
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> It appears as part of the port from transitional client TLS to transitional
> server TLS in CASSANDRA-10404 (the ability to switch a cluster to using
> {{internode_encryption}} without listening on two ports and without downtime)
> we carried the {{enabled}} setting over from the client implementation. I
> believe that the {{enabled}} option is redundant to {{internode_encryption}}
> and {{optional}} and it should therefore be removed prior to the 4.0 release
> where we will have to start respecting that interface.
> Current trunk yaml:
> {noformat}
> server_encryption_options:
>
> # set to true for allowing secure incoming connections
>
> enabled: false
>
> # If enabled and optional are both set to true, encrypted and unencrypted
> connections are handled on the storage_port
> optional: false
>
>
>
>
> # if enabled, will open up an encrypted listening socket on
> ssl_storage_port. Should be used
> # during upgrade to 4.0; otherwise, set to false.
>
> enable_legacy_ssl_storage_port: false
>
> # on outbound connections, determine which type of peers to securely
> connect to. 'enabled' must be set to true.
> internode_encryption: none
>
> keystore: conf/.keystore
>
> keystore_password: cassandra
>
> truststore: conf/.truststore
>
> truststore_password: cassandra
> {noformat}
> I propose we eliminate {{enabled}} and just use {{optional}} and
> {{internode_encryption}} to determine the listener setup. I also propose we
> change the default of {{optional}} to true. We could also re-name
> {{optional}} since it's a new option but I think it's good to stay consistent
> with the client and use {{optional}}.
> ||optional||internode_encryption||description||
> |true|none|(default) No encryption is used but if a server reaches out with
> it we'll use it|
> |false|dc|Encryption is required for inter-dc communication, but not intra-dc|
> |false|all|Encryption is required for all communication|
> |false|none|We only listen for unencrypted connections|
> |true|dc|Encryption is used for inter-dc communication but is not required|
> |true|all|Encryption is used for all communication but is not required|
> From these states it is clear when we should be accepting TLS connections
> (all except for false and none) as well as when we must enforce it.
> To transition without downtime from an un-encrypted cluster to an encrypted
> cluster the user would do the following:
> 1. After adding valid truststores, change {{internode_encryption}} to the
> desired level of encryption (recommended {{all}}) and restart Cassandra
> 2. Change {{optional=false}} and restart Cassandra to enforce #1
> If {{optional}} defaulted to {{false}} as it does right now we'd need a third
> restart to first change {{optional}} to {{true}}, which given my
> understanding of the OptionalSslHandler isn't really relevant.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15685) flaky testWithMismatchingPending - org.apache.cassandra.distributed.test.PreviewRepairTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh McKenzie updated CASSANDRA-15685:
--
Fix Version/s: (was: 4.0-alpha)
4.0-beta
> flaky testWithMismatchingPending -
> org.apache.cassandra.distributed.test.PreviewRepairTest
> --
>
> Key: CASSANDRA-15685
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15685
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Kevin Gallardo
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-beta
>
> Attachments: log-CASSANDRA-15685.txt, output
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Observed in:
> https://app.circleci.com/pipelines/github/newkek/cassandra/34/workflows/1c6b157d-13c3-48a9-85fb-9fe8c153256b/jobs/191/tests
> Failure:
> {noformat}
> testWithMismatchingPending -
> org.apache.cassandra.distributed.test.PreviewRepairTest
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.distributed.test.PreviewRepairTest.testWithMismatchingPending(PreviewRepairTest.java:97)
> {noformat}
> [Circle
> CI|https://circleci.com/gh/dcapwell/cassandra/tree/bug%2FCASSANDRA-15685]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126815#comment-17126815
]
Ekaterina Dimitrova commented on CASSANDRA-15234:
-
This one is ready for final review I think.
[~mck] I am not a committer but I think it would be great also to run the tests
in Jenkins. Thank you
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15234:
Test and Documentation Plan:
[ [trunk
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-3-rebase
] ] [ [DTests |
https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3]
] [ [CCM |https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3] ]
[ [JAVA8 CI
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/718c8a04-385b-4ddc-ac2e-9291c37b1f1e]
] [ [JAVA11 CI
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/6cd6048d-5daf-4490-a5d3-b5c3178a0757
] ]
Attached to the ticket is the log of the successful runs of the few failing
JAVA8 DTests in CircleCI. I ran them locally after fixing lost line of code as
it didn't make sense to me to rerun the whole suite of tests.
*Last update:* after rebase and a few nits on my end, we have only 1 DTest and
1 Unit test failing in JAVA11. Unrelated known failures.
- added additional note on the possible units to be used as suffixes in both
NEWS.txt and cassandra.yaml
CQLSH JAVA11 tests are not currently available in CircleCI.
The last part is all green after rebase and CI run.
- "line.separator" getter added
[ [trunk
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-5] ] [
[JAVA8 CI
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/ebd771ac-d71c-49ea-872b-00b534d9d5c6]
] [ [JAVA11 CI
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/19a3254b-cd7a-4e4f-856f-f314546ff9d7]
]
There are 1 DTest and 1 Unit unrelated failing tests in JAVA11. Should check
for tickets logged.
*NOTE:* Before commit return to the default config.yml and requirements.txt
files
was:
[ [trunk
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-3-rebase
] ] [ [DTests |
https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3]
] [ [CCM |https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3] ]
[ [JAVA8
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/718c8a04-385b-4ddc-ac2e-9291c37b1f1e]
] [ [JAVA11
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/6cd6048d-5daf-4490-a5d3-b5c3178a0757
] ]
Attached to the ticket is the log of the successful runs of the few failing
JAVA8 DTests in CircleCI. I ran them locally after fixing lost line of code as
it didn't make sense to me to rerun the whole suite of tests.
*Last update:* after rebase and a few nits on my end, we have only 1 DTest and
1 Unit test failing in JAVA11. Unrelated known failures.
- added additional note on the possible units to be used as suffixes in both
NEWS.txt and cassandra.yaml
CQLSH JAVA11 tests are not currently available in CircleCI.
The last part is all green after rebase and CI run.
- "line.separator" getter added
[ [trunk
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-5] ] [
[JAVA8 CI
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/ebd771ac-d71c-49ea-872b-00b534d9d5c6]
] [ [JAVA11 CI
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/19a3254b-cd7a-4e4f-856f-f314546ff9d7]
]
There are 1 DTest and 1 Unit unrelated failing tests in JAVA11. Should check
for tickets logged.
*NOTE:* Before commit return to the default config.yml and requirements.txt
files
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are
[jira] [Comment Edited] (CASSANDRA-15685) flaky testWithMismatchingPending - org.apache.cassandra.distributed.test.PreviewRepairTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126807#comment-17126807
]
Ekaterina Dimitrova edited comment on CASSANDRA-15685 at 6/5/20, 2:20 PM:
--
[~jmckenzie], personally, I think we can move it to 4.0-beta. My reasoning is
that we might not be able to run preview repair(the participant is still not
aware of the IR being completed already) and have to start it again later but
there is no data loss or something. [~dcapwell] are you ok with this reasoning?
was (Author: e.dimitrova):
[~jmckenzie]I personally think we can move it to 4.0-beta. My reasoning is that
we might not be able to run preview repair(the participant is still not aware
of the IR being completed already) and have to start it again later but there
is no data loss or something. [~dcapwell] are you ok with this reasoning?
> flaky testWithMismatchingPending -
> org.apache.cassandra.distributed.test.PreviewRepairTest
> --
>
> Key: CASSANDRA-15685
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15685
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Kevin Gallardo
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Attachments: log-CASSANDRA-15685.txt, output
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Observed in:
> https://app.circleci.com/pipelines/github/newkek/cassandra/34/workflows/1c6b157d-13c3-48a9-85fb-9fe8c153256b/jobs/191/tests
> Failure:
> {noformat}
> testWithMismatchingPending -
> org.apache.cassandra.distributed.test.PreviewRepairTest
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.distributed.test.PreviewRepairTest.testWithMismatchingPending(PreviewRepairTest.java:97)
> {noformat}
> [Circle
> CI|https://circleci.com/gh/dcapwell/cassandra/tree/bug%2FCASSANDRA-15685]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15685) flaky testWithMismatchingPending - org.apache.cassandra.distributed.test.PreviewRepairTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126807#comment-17126807
]
Ekaterina Dimitrova commented on CASSANDRA-15685:
-
[~jmckenzie]I personally think we can move it to 4.0-beta. My reasoning is that
we might not be able to run preview repair(the participant is still not aware
of the IR being completed already) and have to start it again later but there
is no data loss or something. [~dcapwell] are you ok with this reasoning?
> flaky testWithMismatchingPending -
> org.apache.cassandra.distributed.test.PreviewRepairTest
> --
>
> Key: CASSANDRA-15685
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15685
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Kevin Gallardo
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Attachments: log-CASSANDRA-15685.txt, output
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Observed in:
> https://app.circleci.com/pipelines/github/newkek/cassandra/34/workflows/1c6b157d-13c3-48a9-85fb-9fe8c153256b/jobs/191/tests
> Failure:
> {noformat}
> testWithMismatchingPending -
> org.apache.cassandra.distributed.test.PreviewRepairTest
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.distributed.test.PreviewRepairTest.testWithMismatchingPending(PreviewRepairTest.java:97)
> {noformat}
> [Circle
> CI|https://circleci.com/gh/dcapwell/cassandra/tree/bug%2FCASSANDRA-15685]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15842) Fix flaky org.apache.cassandra.schema.SchemaTest.testTransKsMigration-cdc
[
https://issues.apache.org/jira/browse/CASSANDRA-15842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126767#comment-17126767
]
Benjamin Lerer commented on CASSANDRA-15842:
The fix looks good. I just wonder if we should not simply make the
{{CDCSizeTracker.shutdown()}} more robust. Something like:
{code:java}
public void shutdown()
{
if (cdcSizeCalculationExecutor != null &&
!cdcSizeCalculationExecutor.isShutdown())
{
cdcSizeCalculationExecutor.shutdown();
}
}
{code}
Throwing a NPE when the {{CDCSizeTracker}} is not started is not the nicest
behavior.
> Fix flaky org.apache.cassandra.schema.SchemaTest.testTransKsMigration-cdc
> -
>
> Key: CASSANDRA-15842
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15842
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Failing for the past [28
> builds|https://ci-cassandra.apache.org/job/Cassandra-trunk/153/testReport/org.apache.cassandra.schema/SchemaTest/testTransKsMigration_cdc/]
> java.lang.NullPointerException
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDC$CDCSizeTracker.shutdown(CommitLogSegmentManagerCDC.java:312)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDC.shutdown(CommitLogSegmentManagerCDC.java:89)
> at
> org.apache.cassandra.db.commitlog.AbstractCommitLogSegmentManager.stopUnsafe(AbstractCommitLogSegmentManager.java:413)
> at
> org.apache.cassandra.db.commitlog.CommitLog.stopUnsafe(CommitLog.java:467)
> at
> org.apache.cassandra.SchemaLoader.cleanupAndLeaveDirs(SchemaLoader.java:785)
> at
> org.apache.cassandra.schema.SchemaTest.testTransKsMigration(SchemaTest.java:46)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15842) Fix flaky org.apache.cassandra.schema.SchemaTest.testTransKsMigration-cdc
[ https://issues.apache.org/jira/browse/CASSANDRA-15842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-15842: --- Reviewers: Benjamin Lerer, Benjamin Lerer (was: Benjamin Lerer) Benjamin Lerer, Benjamin Lerer (was: Benjamin Lerer) Status: Review In Progress (was: Patch Available) > Fix flaky org.apache.cassandra.schema.SchemaTest.testTransKsMigration-cdc > - > > Key: CASSANDRA-15842 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15842 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 20m > Remaining Estimate: 0h > > Failing for the past [28 > builds|https://ci-cassandra.apache.org/job/Cassandra-trunk/153/testReport/org.apache.cassandra.schema/SchemaTest/testTransKsMigration_cdc/] > java.lang.NullPointerException > at > org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDC$CDCSizeTracker.shutdown(CommitLogSegmentManagerCDC.java:312) > at > org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDC.shutdown(CommitLogSegmentManagerCDC.java:89) > at > org.apache.cassandra.db.commitlog.AbstractCommitLogSegmentManager.stopUnsafe(AbstractCommitLogSegmentManager.java:413) > at > org.apache.cassandra.db.commitlog.CommitLog.stopUnsafe(CommitLog.java:467) > at > org.apache.cassandra.SchemaLoader.cleanupAndLeaveDirs(SchemaLoader.java:785) > at > org.apache.cassandra.schema.SchemaTest.testTransKsMigration(SchemaTest.java:46) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15369) Fake row deletions and range tombstones, causing digest mismatch and sstable growth
[
https://issues.apache.org/jira/browse/CASSANDRA-15369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhaoYang updated CASSANDRA-15369:
-
Test and Documentation Plan:
Added unit tests to read RT with named queries and slice queries.
[CIrcle
CI|https://circleci.com/workflow-run/2d7af5ed-1ee6-4188-af62-90cb7f55b697]
running
was:
Added unit tests to read RT with named queries and slice queries.
[CIrcle
CI|https://circleci.com/workflow-run/5b72aec3-9a21-4393-9641-1c4c36f03b0c]
running
> Fake row deletions and range tombstones, causing digest mismatch and sstable
> growth
> ---
>
> Key: CASSANDRA-15369
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15369
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination, Local/Memtable, Local/SSTable
>Reporter: Benedict Elliott Smith
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> As assessed in CASSANDRA-15363, we generate fake row deletions and fake
> tombstone markers under various circumstances:
> * If we perform a clustering key query (or select a compact column):
> * Serving from a {{Memtable}}, we will generate fake row deletions
> * Serving from an sstable, we will generate fake row tombstone markers
> * If we perform a slice query, we will generate only fake row tombstone
> markers for any range tombstone that begins or ends outside of the limit of
> the requested slice
> * If we perform a multi-slice or IN query, this will occur for each
> slice/clustering
> Unfortunately, these different behaviours can lead to very different data
> stored in sstables until a full repair is run. When we read-repair, we only
> send these fake deletions or range tombstones. A fake row deletion,
> clustering RT and slice RT, each produces a different digest. So for each
> single point lookup we can produce a digest mismatch twice, and until a full
> repair is run we can encounter an unlimited number of digest mismatches
> across different overlapping queries.
> Relatedly, this seems a more problematic variant of our atomicity failures
> caused by our monotonic reads, since RTs can have an atomic effect across (up
> to) the entire partition, whereas the propagation may happen on an
> arbitrarily small portion. If the RT exists on only one node, this could
> plausibly lead to fairly problematic scenario if that node fails before the
> range can be repaired.
> At the very least, this behaviour can lead to an almost unlimited amount of
> extraneous data being stored until the range is repaired and compaction
> happens to overwrite the sub-range RTs and row deletions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15369) Fake row deletions and range tombstones, causing digest mismatch and sstable growth
[
https://issues.apache.org/jira/browse/CASSANDRA-15369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhaoYang updated CASSANDRA-15369:
-
Reviewers: Andres de la Peña, Marcus Eriksson, ZhaoYang (was: Andres de la
Peña, Marcus Eriksson)
Status: Review In Progress (was: Patch Available)
> Fake row deletions and range tombstones, causing digest mismatch and sstable
> growth
> ---
>
> Key: CASSANDRA-15369
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15369
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination, Local/Memtable, Local/SSTable
>Reporter: Benedict Elliott Smith
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> As assessed in CASSANDRA-15363, we generate fake row deletions and fake
> tombstone markers under various circumstances:
> * If we perform a clustering key query (or select a compact column):
> * Serving from a {{Memtable}}, we will generate fake row deletions
> * Serving from an sstable, we will generate fake row tombstone markers
> * If we perform a slice query, we will generate only fake row tombstone
> markers for any range tombstone that begins or ends outside of the limit of
> the requested slice
> * If we perform a multi-slice or IN query, this will occur for each
> slice/clustering
> Unfortunately, these different behaviours can lead to very different data
> stored in sstables until a full repair is run. When we read-repair, we only
> send these fake deletions or range tombstones. A fake row deletion,
> clustering RT and slice RT, each produces a different digest. So for each
> single point lookup we can produce a digest mismatch twice, and until a full
> repair is run we can encounter an unlimited number of digest mismatches
> across different overlapping queries.
> Relatedly, this seems a more problematic variant of our atomicity failures
> caused by our monotonic reads, since RTs can have an atomic effect across (up
> to) the entire partition, whereas the propagation may happen on an
> arbitrarily small portion. If the RT exists on only one node, this could
> plausibly lead to fairly problematic scenario if that node fails before the
> range can be repaired.
> At the very least, this behaviour can lead to an almost unlimited amount of
> extraneous data being stored until the range is repaired and compaction
> happens to overwrite the sub-range RTs and row deletions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15855) Mark test_populate_mv_after_insert_wide_rows as flaky
[ https://issues.apache.org/jira/browse/CASSANDRA-15855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126753#comment-17126753 ] Andres de la Peña commented on CASSANDRA-15855: --- bq. You made me think the test should run for <4.0 were the implementation for MVs is different. So I amended my commit to instead of skipping always, running it up to 3X and xfail for 4.0. This is much better imo.wdyt? Sounds good to me. New dtest runs for the three involved branches: || cassandra-3.0 || cassandra-3.11 || trunk || |[147|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/147/]|[148|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/148/]|[149|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/149/]| I think this can be run on CircleCI creating and pushing new branches from 3.0, 3.11 and trunk where the only change is the CircleCI config file, and never merge those branches. I guess it would be possible to remove the unit tests from that file, so CircleCI only runs the dtests, although I have never done it. > Mark test_populate_mv_after_insert_wide_rows as flaky > - > > Key: CASSANDRA-15855 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15855 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 10m > Remaining Estimate: 0h > > See CASSANDRA-15845. This test can still fail in a flaky way so we better > mark it as such to avoid confusion and dup investigation efforts on failing > tests -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-8612) Read metrics should be updated on all types of reads
[ https://issues.apache.org/jira/browse/CASSANDRA-8612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dean Z reassigned CASSANDRA-8612: - Assignee: Dean Z > Read metrics should be updated on all types of reads > > > Key: CASSANDRA-8612 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8612 > Project: Cassandra > Issue Type: Improvement > Components: Observability/Metrics >Reporter: Chris Lohfink >Assignee: Dean Z >Priority: Low > Labels: lhf, metrics > Attachments: 0001-Update-read-metrics-on-all-types-of-reads.patch > > > Metrics like "sstables per read" are not updated on a range slice. Although > separating things out for each type of read could make sense like we do for > latencies, only exposing the metrics for one type can be a little confusing > when people do a query and see nothing increases. I think its sufficient to > use the same metrics for all reads. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-8272) 2ndary indexes can return stale data
[
https://issues.apache.org/jira/browse/CASSANDRA-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126726#comment-17126726
]
Andres de la Peña edited comment on CASSANDRA-8272 at 6/5/20, 12:37 PM:
Committed to {{cassandra-3.0}} as
[dd255ffa07d0263521a1ca863fc2192db19bc04c|https://github.com/apache/cassandra/commit/dd255ffa07d0263521a1ca863fc2192db19bc04c]
and merged up to {{cassandra-3.11}} and {{trunk}}.
Dtests committed as
[68f05b02842ccf4b2859d35a057d3be77d3313ab|https://github.com/apache/cassandra-dtest/commit/68f05b02842ccf4b2859d35a057d3be77d3313ab].
Thanks for the reviews.
was (Author: adelapena):
Committed to {{cassandra-3.0}} as
[dd255ffa07d0263521a1ca863fc2192db19bc04c|https://github.com/apache/cassandra/commit/dd255ffa07d0263521a1ca863fc2192db19bc04c]
and merged up to {{cassandra-3.0}} and {{trunk}}.
Dtests committed as
[68f05b02842ccf4b2859d35a057d3be77d3313ab|https://github.com/apache/cassandra-dtest/commit/68f05b02842ccf4b2859d35a057d3be77d3313ab].
Thanks for the reviews.
> 2ndary indexes can return stale data
>
>
> Key: CASSANDRA-8272
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8272
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Sylvain Lebresne
>Assignee: Andres de la Peña
>Priority: Normal
> Labels: pull-request-available
> Fix For: 3.0.21, 3.11.7, 4.0-alpha5
>
> Time Spent: 7h
> Remaining Estimate: 0h
>
> When replica return 2ndary index results, it's possible for a single replica
> to return a stale result and that result will be sent back to the user,
> potentially failing the CL contract.
> For instance, consider 3 replicas A, B and C, and the following situation:
> {noformat}
> CREATE TABLE test (k int PRIMARY KEY, v text);
> CREATE INDEX ON test(v);
> INSERT INTO test(k, v) VALUES (0, 'foo');
> {noformat}
> with every replica up to date. Now, suppose that the following queries are
> done at {{QUORUM}}:
> {noformat}
> UPDATE test SET v = 'bar' WHERE k = 0;
> SELECT * FROM test WHERE v = 'foo';
> {noformat}
> then, if A and B acknowledge the insert but C respond to the read before
> having applied the insert, then the now stale result will be returned (since
> C will return it and A or B will return nothing).
> A potential solution would be that when we read a tombstone in the index (and
> provided we make the index inherit the gcGrace of it's parent CF), instead of
> skipping that tombstone, we'd insert in the result a corresponding range
> tombstone.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-8273) Allow filtering queries can return stale data
[
https://issues.apache.org/jira/browse/CASSANDRA-8273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-8273:
-
Resolution: Duplicate
Status: Resolved (was: Open)
The fix for CASSANDRA-8272 includes this case too.
> Allow filtering queries can return stale data
> -
>
> Key: CASSANDRA-8273
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8273
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Sylvain Lebresne
>Assignee: Andres de la Peña
>Priority: Normal
>
> Data filtering is done replica side. That means that a single replica with
> stale data may make the whole query return that stale data.
> For instance, consider 3 replicas A, B and C, and the following situation:
> {noformat}
> CREATE TABLE test (k int PRIMARY KEY, v1 text, v2 int);
> CREATE INDEX ON test(v1);
> INSERT INTO test(k, v1, v2) VALUES (0, 'foo', 1);
> {noformat}
> with every replica up to date. Now, suppose that the following queries are
> done at {{QUORUM}}:
> {noformat}
> UPDATE test SET v2 = 2 WHERE k = 0;
> SELECT * FROM test WHERE v1 = 'foo' AND v2 = 1;
> {noformat}
> then, if A and B acknowledge the insert but C respond to the read before
> having applied the insert, then the now stale result will be returned. Let's
> note that this is a problem related to filtering, not 2ndary indexes.
> This issue share similarity with CASSANDRA-8272 but contrarily to that former
> issue, I'm not sure how to fix it. Obviously, moving the filtering to the
> coordinator would remove that problem, but doing so would, on top of not
> being trivial to implmenent, have serious performance impact since we can't
> know in advance how much data will be filtered and we may have to redo query
> to replica multiple times.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-8272) 2ndary indexes can return stale data
[
https://issues.apache.org/jira/browse/CASSANDRA-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-8272:
-
Fix Version/s: (was: 4.0-beta)
(was: 3.11.x)
(was: 3.0.x)
4.0-alpha5
3.11.7
3.0.21
Since Version: 0.3
Source Control Link:
https://github.com/apache/cassandra/commit/dd255ffa07d0263521a1ca863fc2192db19bc04c
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> 2ndary indexes can return stale data
>
>
> Key: CASSANDRA-8272
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8272
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Sylvain Lebresne
>Assignee: Andres de la Peña
>Priority: Normal
> Labels: pull-request-available
> Fix For: 3.0.21, 3.11.7, 4.0-alpha5
>
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> When replica return 2ndary index results, it's possible for a single replica
> to return a stale result and that result will be sent back to the user,
> potentially failing the CL contract.
> For instance, consider 3 replicas A, B and C, and the following situation:
> {noformat}
> CREATE TABLE test (k int PRIMARY KEY, v text);
> CREATE INDEX ON test(v);
> INSERT INTO test(k, v) VALUES (0, 'foo');
> {noformat}
> with every replica up to date. Now, suppose that the following queries are
> done at {{QUORUM}}:
> {noformat}
> UPDATE test SET v = 'bar' WHERE k = 0;
> SELECT * FROM test WHERE v = 'foo';
> {noformat}
> then, if A and B acknowledge the insert but C respond to the read before
> having applied the insert, then the now stale result will be returned (since
> C will return it and A or B will return nothing).
> A potential solution would be that when we read a tombstone in the index (and
> provided we make the index inherit the gcGrace of it's parent CF), instead of
> skipping that tombstone, we'd insert in the result a corresponding range
> tombstone.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-8272) 2ndary indexes can return stale data
[
https://issues.apache.org/jira/browse/CASSANDRA-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126726#comment-17126726
]
Andres de la Peña commented on CASSANDRA-8272:
--
Committed to {{cassandra-3.0}} as
[dd255ffa07d0263521a1ca863fc2192db19bc04c|https://github.com/apache/cassandra/commit/dd255ffa07d0263521a1ca863fc2192db19bc04c]
and merged up to {{cassandra-3.0}} and {{trunk}}.
Dtests committed as
[68f05b02842ccf4b2859d35a057d3be77d3313ab|https://github.com/apache/cassandra-dtest/commit/68f05b02842ccf4b2859d35a057d3be77d3313ab].
Thanks for the reviews.
> 2ndary indexes can return stale data
>
>
> Key: CASSANDRA-8272
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8272
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Sylvain Lebresne
>Assignee: Andres de la Peña
>Priority: Normal
> Labels: pull-request-available
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> When replica return 2ndary index results, it's possible for a single replica
> to return a stale result and that result will be sent back to the user,
> potentially failing the CL contract.
> For instance, consider 3 replicas A, B and C, and the following situation:
> {noformat}
> CREATE TABLE test (k int PRIMARY KEY, v text);
> CREATE INDEX ON test(v);
> INSERT INTO test(k, v) VALUES (0, 'foo');
> {noformat}
> with every replica up to date. Now, suppose that the following queries are
> done at {{QUORUM}}:
> {noformat}
> UPDATE test SET v = 'bar' WHERE k = 0;
> SELECT * FROM test WHERE v = 'foo';
> {noformat}
> then, if A and B acknowledge the insert but C respond to the read before
> having applied the insert, then the now stale result will be returned (since
> C will return it and A or B will return nothing).
> A potential solution would be that when we read a tombstone in the index (and
> provided we make the index inherit the gcGrace of it's parent CF), instead of
> skipping that tombstone, we'd insert in the result a corresponding range
> tombstone.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-dtest] branch master updated: Add tests for CASSANDRA-8272 and CASSANDRA-8273
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/master by this push:
new 68f05b0 Add tests for CASSANDRA-8272 and CASSANDRA-8273
68f05b0 is described below
commit 68f05b02842ccf4b2859d35a057d3be77d3313ab
Author: Andrés de la Peña
AuthorDate: Tue Mar 17 20:11:35 2020 +
Add tests for CASSANDRA-8272 and CASSANDRA-8273
---
replica_side_filtering_test.py | 558 +
1 file changed, 558 insertions(+)
diff --git a/replica_side_filtering_test.py b/replica_side_filtering_test.py
new file mode 100644
index 000..c38fcfc
--- /dev/null
+++ b/replica_side_filtering_test.py
@@ -0,0 +1,558 @@
+from abc import abstractmethod
+
+import pytest
+from cassandra import ConsistencyLevel as CL
+from cassandra.query import SimpleStatement
+
+from dtest import Tester, create_ks
+from tools.assertions import (assert_all, assert_none, assert_one)
+
+since = pytest.mark.since
+keyspace = 'ks'
+
+class ReplicaSideFiltering(Tester):
+"""
+@jira_ticket CASSANDRA-8272, CASSANDRA-8273
+Base consistency test for queries involving replica-side filtering when
some of the replicas have stale data.
+"""
+__test__ = False
+
+def _prepare_cluster(self, create_table, create_index=None,
both_nodes=None, only_node1=None, only_node2=None):
+"""
+:param create_table a table creation CQL query
+:param create_index an index creation CQL query, that will be executed
depending on ``_create_index`` method
+:param both_nodes queries to be executed in both nodes with CL=ALL
+:param only_node1 queries to be executed in the first node only, with
CL=ONE, while the second node is stopped
+:param only_node2 queries to be executed in the second node only, with
CL=ONE, while the first node is stopped
+:return: a session connected exclusively to the first node with CL=ALL
+"""
+cluster = self.cluster
+
+# Disable hinted handoff and set batch commit log so this doesn't
interfere with the test
+if only_node1 or only_node2:
+
cluster.set_configuration_options(values={'hinted_handoff_enabled': False})
+cluster.set_batch_commitlog(enabled=True)
+
+cluster.populate(2)
+node1, node2 = cluster.nodelist()
+cluster.start()
+
+session = self.patient_exclusive_cql_connection(node1,
consistency_level=CL.ALL)
+create_ks(session, keyspace, 2)
+session.execute("USE " + keyspace)
+
+# create the table
+session.execute(create_table)
+
+# create the index if it's required
+if self.create_index():
+session.execute(create_index)
+
+# execute the queries for both nodes with CL=ALL
+if both_nodes:
+for q in both_nodes:
+session.execute(q)
+
+# execute the queries for the first node only with the second node
stopped
+if only_node1:
+self._execute_isolated(node_to_update=node1, node_to_stop=node2,
queries=only_node1)
+
+# execute the queries for the second node only with the first node
stopped
+if only_node2:
+self._execute_isolated(node_to_update=node2, node_to_stop=node1,
queries=only_node2)
+
+# set the session with CL=ALL for testing queries with the created
scenario
+self.session = self.patient_exclusive_cql_connection(node1,
keyspace=keyspace, consistency_level=CL.ALL)
+
+def _execute_isolated(self, node_to_update, node_to_stop, queries):
+node_to_stop.flush()
+node_to_stop.stop()
+session = self.patient_cql_connection(node_to_update, keyspace,
consistency_level=CL.ONE)
+for q in queries:
+session.execute(q)
+node_to_stop.start(wait_other_notice=True)
+
+def _assert_none(self, query):
+"""
+Assert query returns nothing.
+@param query Query to run
+"""
+decorated_query = self._decorate_query(query)
+assert_none(self.session, decorated_query)
+
+def _assert_one(self, query, row):
+"""
+Assert query returns one row.
+@param query Query to run
+@param row Expected result row from query
+"""
+decorated_query = self._decorate_query(query)
+assert_one(self.session, decorated_query, row)
+
+def _assert_all(self, query, rows):
+"""
+Assert query returns all expected rows in the correct order.
+@param query Query to run
+@param rows Expected result rows from query
+"""
+decorated_query = self._decorate_query(query)
+assert_all(self.session, decorated_query, rows)
+
+def _decorate_query(self, query):
+return query if self.create_index() else query + " ALLOW
[cassandra] branch trunk updated (da8fe7a -> 69ea5ff)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from da8fe7a Merge branch 'cassandra-3.11' into trunk new dd255ff Fix replica-side filtering returning stale data with CL > 1 new 7014cd5 Merge branch 'cassandra-3.0' into cassandra-3.11 new 69ea5ff Merge branch 'cassandra-3.11' into trunk The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + src/java/org/apache/cassandra/db/DataRange.java| 10 + .../cassandra/db/PartitionRangeReadCommand.java| 5 + src/java/org/apache/cassandra/db/ReadCommand.java | 6 + .../cassandra/db/SinglePartitionReadCommand.java | 5 + .../db/compaction/CompactionIterator.java | 3 +- .../org/apache/cassandra/db/filter/RowFilter.java | 41 +- .../db/partitions/PartitionIterators.java | 12 + .../partitions/UnfilteredPartitionIterators.java | 21 +- .../cassandra/db/rows/UnfilteredRowIterators.java | 34 +- .../org/apache/cassandra/metrics/TableMetrics.java | 18 +- .../cassandra/service/reads/DataResolver.java | 197 +++-- .../service/reads/ReplicaFilteringProtection.java | 480 + .../reads/repair/RowIteratorMergeListener.java | 6 +- .../cassandra/utils/concurrent/Accumulator.java| 13 + .../utils/concurrent/AccumulatorTest.java | 54 ++- 16 files changed, 820 insertions(+), 86 deletions(-) create mode 100644 src/java/org/apache/cassandra/service/reads/ReplicaFilteringProtection.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 7014cd5ab02f3d0ae28b0783f7df34e05fb79f2a
Merge: 056c9ef dd255ff
Author: Andrés de la Peña
AuthorDate: Fri Jun 5 12:25:28 2020 +0100
Merge branch 'cassandra-3.0' into cassandra-3.11
# Conflicts:
# CHANGES.txt
# src/java/org/apache/cassandra/db/filter/RowFilter.java
CHANGES.txt| 1 +
src/java/org/apache/cassandra/db/DataRange.java| 10 +
.../cassandra/db/PartitionRangeReadCommand.java| 5 +
src/java/org/apache/cassandra/db/ReadCommand.java | 7 +
.../cassandra/db/SinglePartitionReadCommand.java | 5 +
.../db/compaction/CompactionIterator.java | 3 +-
.../org/apache/cassandra/db/filter/RowFilter.java | 92 ++--
.../partitions/UnfilteredPartitionIterators.java | 11 +-
.../cassandra/db/rows/UnfilteredRowIterators.java | 32 +-
.../org/apache/cassandra/metrics/TableMetrics.java | 18 +-
.../org/apache/cassandra/service/DataResolver.java | 175 ++--
.../service/ReplicaFilteringProtection.java| 468 +
.../cassandra/utils/concurrent/Accumulator.java| 13 +
.../utils/concurrent/AccumulatorTest.java | 54 ++-
14 files changed, 800 insertions(+), 94 deletions(-)
diff --cc CHANGES.txt
index a809016,ff00579..d64df0f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,7 -1,5 +1,8 @@@
-3.0.21
+3.11.7
+ * Fix CQL formatting of read command restrictions for slow query log
(CASSANDRA-15503)
+ * Allow sstableloader to use SSL on the native port (CASSANDRA-14904)
+Merged from 3.0:
+ * Fix replica-side filtering returning stale data with CL > ONE
(CASSANDRA-8272, CASSANDRA-8273)
* Fix duplicated row on 2.x upgrades when multi-rows range tombstones
interact with collection ones (CASSANDRA-15805)
* Rely on snapshotted session infos on StreamResultFuture.maybeComplete to
avoid race conditions (CASSANDRA-15667)
* EmptyType doesn't override writeValue so could attempt to write bytes when
expected not to (CASSANDRA-15790)
diff --cc src/java/org/apache/cassandra/db/filter/RowFilter.java
index 84a93ab,774e4d3..b4f8a7f
--- a/src/java/org/apache/cassandra/db/filter/RowFilter.java
+++ b/src/java/org/apache/cassandra/db/filter/RowFilter.java
@@@ -20,14 -20,8 +20,13 @@@ package org.apache.cassandra.db.filter
import java.io.IOException;
import java.nio.ByteBuffer;
import java.util.*;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.atomic.AtomicInteger;
import com.google.common.base.Objects;
- import com.google.common.collect.Iterables;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.config.ColumnDefinition;
@@@ -35,8 -29,7 +34,10 @@@ import org.apache.cassandra.cql3.Operat
import org.apache.cassandra.db.*;
import org.apache.cassandra.db.context.*;
import org.apache.cassandra.db.marshal.*;
-import org.apache.cassandra.db.partitions.*;
++import org.apache.cassandra.db.partitions.FilteredPartition;
+import org.apache.cassandra.db.partitions.ImmutableBTreePartition;
++import org.apache.cassandra.db.partitions.PartitionIterator;
+import org.apache.cassandra.db.partitions.UnfilteredPartitionIterator;
import org.apache.cassandra.db.rows.*;
import org.apache.cassandra.db.transform.Transformation;
import org.apache.cassandra.exceptions.InvalidRequestException;
@@@ -284,42 -281,29 +303,40 @@@ public abstract class RowFilter impleme
super(expressions);
}
- public UnfilteredPartitionIterator filter(UnfilteredPartitionIterator
iter, int nowInSec)
+ protected Transformation> filter(CFMetaData
metadata, int nowInSec)
{
- if (expressions.isEmpty())
- return iter;
-
- final CFMetaData metadata = iter.metadata();
-
-long numberOfStaticColumnExpressions =
expressions.stream().filter(e -> e.column.isStatic()).count();
-final boolean filterStaticColumns =
numberOfStaticColumnExpressions != 0;
-final boolean filterNonStaticColumns = (expressions.size() -
numberOfStaticColumnExpressions) > 0;
+List partitionLevelExpressions = new ArrayList<>();
+List rowLevelExpressions = new ArrayList<>();
+for (Expression e: expressions)
+{
+if (e.column.isStatic() || e.column.isPartitionKey())
+partitionLevelExpressions.add(e);
+else
+rowLevelExpressions.add(e);
+}
+
+long numberOfRegularColumnExpressions =
rowLevelExpressions.size();
+final boolean filterNonStaticColumns =
numberOfRegularColumnExpressions > 0;
-
[cassandra] branch cassandra-3.11 updated (056c9ef -> 7014cd5)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 056c9ef Merge branch 'cassandra-3.0' into cassandra-3.11 new dd255ff Fix replica-side filtering returning stale data with CL > 1 new 7014cd5 Merge branch 'cassandra-3.0' into cassandra-3.11 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + src/java/org/apache/cassandra/db/DataRange.java| 10 + .../cassandra/db/PartitionRangeReadCommand.java| 5 + src/java/org/apache/cassandra/db/ReadCommand.java | 7 + .../cassandra/db/SinglePartitionReadCommand.java | 5 + .../db/compaction/CompactionIterator.java | 3 +- .../org/apache/cassandra/db/filter/RowFilter.java | 92 ++-- .../partitions/UnfilteredPartitionIterators.java | 11 +- .../cassandra/db/rows/UnfilteredRowIterators.java | 32 +- .../org/apache/cassandra/metrics/TableMetrics.java | 18 +- .../org/apache/cassandra/service/DataResolver.java | 175 ++-- .../service/ReplicaFilteringProtection.java| 468 + .../cassandra/utils/concurrent/Accumulator.java| 13 + .../utils/concurrent/AccumulatorTest.java | 54 ++- 14 files changed, 800 insertions(+), 94 deletions(-) create mode 100644 src/java/org/apache/cassandra/service/ReplicaFilteringProtection.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 69ea5ffdbc5866419bc33b0a1b45bae580ee9dd1 Merge: da8fe7a 7014cd5 Author: Andrés de la Peña AuthorDate: Fri Jun 5 13:15:46 2020 +0100 Merge branch 'cassandra-3.11' into trunk # Conflicts: # CHANGES.txt # src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java # src/java/org/apache/cassandra/db/ReadCommand.java # src/java/org/apache/cassandra/db/SinglePartitionReadCommand.java # src/java/org/apache/cassandra/db/filter/RowFilter.java # src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java # src/java/org/apache/cassandra/metrics/TableMetrics.java # src/java/org/apache/cassandra/service/DataResolver.java # src/java/org/apache/cassandra/utils/concurrent/Accumulator.java CHANGES.txt| 1 + src/java/org/apache/cassandra/db/DataRange.java| 10 + .../cassandra/db/PartitionRangeReadCommand.java| 5 + src/java/org/apache/cassandra/db/ReadCommand.java | 6 + .../cassandra/db/SinglePartitionReadCommand.java | 5 + .../db/compaction/CompactionIterator.java | 3 +- .../org/apache/cassandra/db/filter/RowFilter.java | 41 +- .../db/partitions/PartitionIterators.java | 12 + .../partitions/UnfilteredPartitionIterators.java | 21 +- .../cassandra/db/rows/UnfilteredRowIterators.java | 34 +- .../org/apache/cassandra/metrics/TableMetrics.java | 18 +- .../cassandra/service/reads/DataResolver.java | 197 +++-- .../service/reads/ReplicaFilteringProtection.java | 480 + .../reads/repair/RowIteratorMergeListener.java | 6 +- .../cassandra/utils/concurrent/Accumulator.java| 13 + .../utils/concurrent/AccumulatorTest.java | 54 ++- 16 files changed, 820 insertions(+), 86 deletions(-) diff --cc CHANGES.txt index 565f362,d64df0f..4487420 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,44 -1,9 +1,45 @@@ -3.11.7 +4.0-alpha5 + * Add fqltool and auditlogviewer to rpm and deb packages (CASSANDRA-14712) + * Include DROPPED_COLUMNS in schema digest computation (CASSANDRA-15843) + * Fix Cassandra restart from rpm install (CASSANDRA-15830) + * Improve handling of 2i initialization failures (CASSANDRA-13606) + * Add completion_ratio column to sstable_tasks virtual table (CASANDRA-15759) + * Add support for adding custom Verbs (CASSANDRA-15725) + * Speed up entire-file-streaming file containment check and allow entire-file-streaming for all compaction strategies (CASSANDRA-15657,CASSANDRA-15783) + * Provide ability to configure IAuditLogger (CASSANDRA-15748) + * Fix nodetool enablefullquerylog blocking param parsing (CASSANDRA-15819) + * Add isTransient to SSTableMetadataView (CASSANDRA-15806) + * Fix tools/bin/fqltool for all shells (CASSANDRA-15820) + * Fix clearing of legacy size_estimates (CASSANDRA-15776) + * Update port when reconnecting to pre-4.0 SSL storage (CASSANDRA-15727) + * Only calculate dynamicBadnessThreshold once per loop in DynamicEndpointSnitch (CASSANDRA-15798) + * Cleanup redundant nodetool commands added in 4.0 (CASSANDRA-15256) + * Update to Python driver 3.23 for cqlsh (CASSANDRA-15793) + * Add tunable initial size and growth factor to RangeTombstoneList (CASSANDRA-15763) + * Improve debug logging in SSTableReader for index summary (CASSANDRA-15755) + * bin/sstableverify should support user provided token ranges (CASSANDRA-15753) + * Improve logging when mutation passed to commit log is too large (CASSANDRA-14781) + * replace LZ4FastDecompressor with LZ4SafeDecompressor (CASSANDRA-15560) + * Fix buffer pool NPE with concurrent release due to in-progress tiny pool eviction (CASSANDRA-15726) + * Avoid race condition when completing stream sessions (CASSANDRA-15666) + * Flush with fast compressors by default (CASSANDRA-15379) + * Fix CqlInputFormat regression from the switch to system.size_estimates (CASSANDRA-15637) + * Allow sending Entire SSTables over SSL (CASSANDRA-15740) + * Fix CQLSH UTF-8 encoding issue for Python 2/3 compatibility (CASSANDRA-15739) + * Fix batch statement preparation when multiple tables and parameters are used (CASSANDRA-15730) + * Fix regression with traceOutgoingMessage printing message size (CASSANDRA-15687) + * Ensure repaired data tracking reads a consistent amount of data across replicas (CASSANDRA-15601) + * Fix CQLSH to avoid arguments being evaluated (CASSANDRA-15660) + * Correct Visibility and Improve Safety of Methods in LatencyMetrics (CASSANDRA-15597) + * Allow cqlsh to run with Python2.7/Python3.6+ (CASSANDRA-15659,CASSANDRA-15573) + * Improve logging around incremental repair (CASSANDRA-15599) + * Do not check cdc_raw_directory filesystem space if CDC disabled (CASSANDRA-15688) + * Replace array iterators with get
[cassandra] branch cassandra-3.0 updated: Fix replica-side filtering returning stale data with CL > 1
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-3.0 by this push:
new dd255ff Fix replica-side filtering returning stale data with CL > 1
dd255ff is described below
commit dd255ffa07d0263521a1ca863fc2192db19bc04c
Author: Andrés de la Peña
AuthorDate: Wed May 27 11:01:42 2020 +0100
Fix replica-side filtering returning stale data with CL > 1
patch by Andres de la Peña; reviewed by Benjamin Lerer, Caleb Rackliffe and
ZhaoYang for CASSANDRA-8272
---
CHANGES.txt| 1 +
src/java/org/apache/cassandra/db/DataRange.java| 10 +
.../cassandra/db/PartitionRangeReadCommand.java| 5 +
src/java/org/apache/cassandra/db/ReadCommand.java | 7 +
.../cassandra/db/SinglePartitionReadCommand.java | 5 +
.../db/compaction/CompactionIterator.java | 3 +-
.../org/apache/cassandra/db/filter/RowFilter.java | 87 ++--
.../partitions/UnfilteredPartitionIterators.java | 30 +-
.../cassandra/db/rows/UnfilteredRowIterators.java | 32 +-
.../org/apache/cassandra/metrics/TableMetrics.java | 18 +-
.../org/apache/cassandra/service/DataResolver.java | 162 +--
.../service/ReplicaFilteringProtection.java| 465 +
.../cassandra/utils/concurrent/Accumulator.java| 13 +
.../utils/concurrent/AccumulatorTest.java | 54 ++-
14 files changed, 800 insertions(+), 92 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 46b3f56..ff00579 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.21
+ * Fix replica-side filtering returning stale data with CL > ONE
(CASSANDRA-8272, CASSANDRA-8273)
* Fix duplicated row on 2.x upgrades when multi-rows range tombstones
interact with collection ones (CASSANDRA-15805)
* Rely on snapshotted session infos on StreamResultFuture.maybeComplete to
avoid race conditions (CASSANDRA-15667)
* EmptyType doesn't override writeValue so could attempt to write bytes when
expected not to (CASSANDRA-15790)
diff --git a/src/java/org/apache/cassandra/db/DataRange.java
b/src/java/org/apache/cassandra/db/DataRange.java
index d2f9c76..f6776c4 100644
--- a/src/java/org/apache/cassandra/db/DataRange.java
+++ b/src/java/org/apache/cassandra/db/DataRange.java
@@ -196,6 +196,16 @@ public class DataRange
}
/**
+ * Whether the underlying {@code ClusteringIndexFilter} is reversed or not.
+ *
+ * @return whether the underlying {@code ClusteringIndexFilter} is
reversed or not.
+ */
+public boolean isReversed()
+{
+return clusteringIndexFilter.isReversed();
+}
+
+/**
* The clustering index filter to use for the provided key.
*
* This may or may not be the same filter for all keys (that is, paging
range
diff --git a/src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java
b/src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java
index 4f936cc..1da66c1 100644
--- a/src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java
+++ b/src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java
@@ -206,6 +206,11 @@ public class PartitionRangeReadCommand extends ReadCommand
return DatabaseDescriptor.getRangeRpcTimeout();
}
+public boolean isReversed()
+{
+return dataRange.isReversed();
+}
+
public boolean selectsKey(DecoratedKey key)
{
if (!dataRange().contains(key))
diff --git a/src/java/org/apache/cassandra/db/ReadCommand.java
b/src/java/org/apache/cassandra/db/ReadCommand.java
index b499daf..39a5402 100644
--- a/src/java/org/apache/cassandra/db/ReadCommand.java
+++ b/src/java/org/apache/cassandra/db/ReadCommand.java
@@ -330,6 +330,13 @@ public abstract class ReadCommand implements ReadQuery
protected abstract int oldestUnrepairedTombstone();
+/**
+ * Whether the underlying {@code ClusteringIndexFilter} is reversed or not.
+ *
+ * @return whether the underlying {@code ClusteringIndexFilter} is
reversed or not.
+ */
+public abstract boolean isReversed();
+
public ReadResponse createResponse(UnfilteredPartitionIterator iterator)
{
// validate that the sequence of RT markers is correct: open is
followed by close, deletion times for both
diff --git a/src/java/org/apache/cassandra/db/SinglePartitionReadCommand.java
b/src/java/org/apache/cassandra/db/SinglePartitionReadCommand.java
index 2e014ba..841c3b9 100644
--- a/src/java/org/apache/cassandra/db/SinglePartitionReadCommand.java
+++ b/src/java/org/apache/cassandra/db/SinglePartitionReadCommand.java
@@ -415,6 +415,11 @@ public class SinglePartitionReadCommand extends ReadCommand
return DatabaseDescriptor.getReadRpcTimeout();
}
+public boolean isReversed()
+{
+return clusteringIndexFilter.isReversed();
+}
[jira] [Commented] (CASSANDRA-14612) Please add OWASP Dependency Check to the build (pom.xml)
[ https://issues.apache.org/jira/browse/CASSANDRA-14612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1712#comment-1712 ] Stefan Miklosovic commented on CASSANDRA-14612: --- please let us know if this whole Gradle initiative is going to happen in reasonably near future (at most few weeks) so it is worth to wait for that in upstream and we do not need to patch our builds just to scan it easilly. If Gradle integration is going to be done in 6 months (as an example), I still believe this is something we could merge into trunk as it is. > Please add OWASP Dependency Check to the build (pom.xml) > > > Key: CASSANDRA-14612 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14612 > Project: Cassandra > Issue Type: New Feature > Components: Build > Environment: All development, build, test, environments. >Reporter: Albert Baker >Priority: Normal > Labels: build, easyfix, security > Fix For: 3.11.x, 4.x > > Original Estimate: 1h > Remaining Estimate: 1h > > Please add OWASP Dependency Check to the build (pom.xml). OWASP DC makes an > outbound REST call to MITRE Common Vulnerabilities & Exposures (CVE) to > perform a lookup for each dependant .jar to list any/all known > vulnerabilities for each jar. This step is needed because a manual MITRE CVE > lookup/check on the main component does not include checking for > vulnerabilities in components or in dependant libraries. > OWASP Dependency check : > https://www.owasp.org/index.php/OWASP_Dependency_Check has plug-ins for most > Java build/make types (ant, maven, ivy, gradle). > Also, add the appropriate command to the nightly build to generate a report > of all known vulnerabilities in any/all third party libraries/dependencies > that get pulled in. example : mvn -Powasp -Dtest=false -DfailIfNoTests=false > clean aggregate > Generating this report nightly/weekly will help inform the project's > development team if any dependant libraries have a reported known > vulnerailities. Project teams that keep up with removing vulnerabilities on a > weekly basis will help protect businesses that rely on these open source > componets. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15858) TestRepairDataSystemTable test_repair_parent_table and test_repair_table fail
[ https://issues.apache.org/jira/browse/CASSANDRA-15858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-15858: Fix Version/s: 4.0-beta > TestRepairDataSystemTable test_repair_parent_table and test_repair_table fail > - > > Key: CASSANDRA-15858 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15858 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 10m > Remaining Estimate: 0h > > They fail > [consistently|https://ci-cassandra.apache.org/job/Cassandra-trunk/159/testReport/] > and locally as well -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15858) TestRepairDataSystemTable test_repair_parent_table and test_repair_table fail
[ https://issues.apache.org/jira/browse/CASSANDRA-15858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126651#comment-17126651 ] Berenguer Blasi commented on CASSANDRA-15858: - They failed locally now they pass both individually and the full suite. > TestRepairDataSystemTable test_repair_parent_table and test_repair_table fail > - > > Key: CASSANDRA-15858 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15858 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Time Spent: 10m > Remaining Estimate: 0h > > They fail > [consistently|https://ci-cassandra.apache.org/job/Cassandra-trunk/159/testReport/] > and locally as well -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15858) TestRepairDataSystemTable test_repair_parent_table and test_repair_table fail
[ https://issues.apache.org/jira/browse/CASSANDRA-15858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-15858: Test and Documentation Plan: See PR Status: Patch Available (was: In Progress) > TestRepairDataSystemTable test_repair_parent_table and test_repair_table fail > - > > Key: CASSANDRA-15858 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15858 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Time Spent: 10m > Remaining Estimate: 0h > > They fail > [consistently|https://ci-cassandra.apache.org/job/Cassandra-trunk/159/testReport/] > and locally as well -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15858) TestRepairDataSystemTable test_repair_parent_table and test_repair_table fail
[ https://issues.apache.org/jira/browse/CASSANDRA-15858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-15858: Bug Category: Parent values: Correctness(12982)Level 1 values: Test Failure(12990) Complexity: Normal Discovered By: Unit Test Severity: Normal Status: Open (was: Triage Needed) > TestRepairDataSystemTable test_repair_parent_table and test_repair_table fail > - > > Key: CASSANDRA-15858 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15858 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > > They fail > [consistently|https://ci-cassandra.apache.org/job/Cassandra-trunk/159/testReport/] > and locally as well -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15858) TestRepairDataSystemTable test_repair_parent_table and test_repair_table fail
Berenguer Blasi created CASSANDRA-15858: --- Summary: TestRepairDataSystemTable test_repair_parent_table and test_repair_table fail Key: CASSANDRA-15858 URL: https://issues.apache.org/jira/browse/CASSANDRA-15858 Project: Cassandra Issue Type: Bug Components: Test/dtest Reporter: Berenguer Blasi Assignee: Berenguer Blasi They fail [consistently|https://ci-cassandra.apache.org/job/Cassandra-trunk/159/testReport/] and locally as well -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14612) Please add OWASP Dependency Check to the build (pom.xml)
[ https://issues.apache.org/jira/browse/CASSANDRA-14612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126580#comment-17126580 ] Michael Semb Wever commented on CASSANDRA-14612: I'm thinking it is best we hold off on this until the gradle build is introduced. (fyi [~snazy]) > Please add OWASP Dependency Check to the build (pom.xml) > > > Key: CASSANDRA-14612 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14612 > Project: Cassandra > Issue Type: New Feature > Components: Build > Environment: All development, build, test, environments. >Reporter: Albert Baker >Priority: Normal > Labels: build, easyfix, security > Fix For: 3.11.x, 4.x > > Original Estimate: 1h > Remaining Estimate: 1h > > Please add OWASP Dependency Check to the build (pom.xml). OWASP DC makes an > outbound REST call to MITRE Common Vulnerabilities & Exposures (CVE) to > perform a lookup for each dependant .jar to list any/all known > vulnerabilities for each jar. This step is needed because a manual MITRE CVE > lookup/check on the main component does not include checking for > vulnerabilities in components or in dependant libraries. > OWASP Dependency check : > https://www.owasp.org/index.php/OWASP_Dependency_Check has plug-ins for most > Java build/make types (ant, maven, ivy, gradle). > Also, add the appropriate command to the nightly build to generate a report > of all known vulnerabilities in any/all third party libraries/dependencies > that get pulled in. example : mvn -Powasp -Dtest=false -DfailIfNoTests=false > clean aggregate > Generating this report nightly/weekly will help inform the project's > development team if any dependant libraries have a reported known > vulnerailities. Project teams that keep up with removing vulnerabilities on a > weekly basis will help protect businesses that rely on these open source > componets. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch master updated (55a2d26 -> 94f2b78)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git. from 55a2d26 Add deb and rpm packaging to cassandra-artifacts.sh test script add 94f2b78 Add support for dtest upgrade tests No new revisions were added by this update. Summary of changes: build-scripts/cassandra-dtest-pytest.sh | 4 +- jenkins-dsl/cassandra_job_dsl_seed.groovy | 127 -- 2 files changed, 72 insertions(+), 59 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-dtest] branch master updated: Introduce --execute-upgrade-tests-only to only run upgrade tests
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/master by this push:
new 7da11dd Introduce --execute-upgrade-tests-only to only run upgrade
tests
7da11dd is described below
commit 7da11ddc2da9a5c4938044c21909ac4221580687
Author: Eduard Tudenhoefner
AuthorDate: Thu Jun 4 08:33:26 2020 +0200
Introduce --execute-upgrade-tests-only to only run upgrade tests
---
conftest.py | 44
dtest_config.py | 2 ++
run_dtests.py | 3 ++-
3 files changed, 28 insertions(+), 21 deletions(-)
diff --git a/conftest.py b/conftest.py
index 2629689..1fa5e22 100644
--- a/conftest.py
+++ b/conftest.py
@@ -1,32 +1,28 @@
-import pytest
+import copy
+import inspect
import logging
import os
+import platform
+import re
import shutil
import time
-import re
-import platform
-import copy
-import inspect
-import subprocess
-
-from dtest import running_in_docker,
cleanup_docker_environment_before_test_execution
-
from datetime import datetime
from distutils.version import LooseVersion
-from netifaces import AF_INET
-from psutil import virtual_memory
+# Python 3 imports
+from itertools import zip_longest
-import netifaces as ni
import ccmlib.repository
+import netifaces as ni
+import pytest
from ccmlib.common import validate_install_dir, is_win, get_version_from_build
+from netifaces import AF_INET
+from psutil import virtual_memory
+from dtest import running_in_docker,
cleanup_docker_environment_before_test_execution
from dtest_config import DTestConfig
from dtest_setup import DTestSetup
from dtest_setup_overrides import DTestSetupOverrides
-# Python 3 imports
-from itertools import zip_longest
-
logger = logging.getLogger(__name__)
def check_required_loopback_interfaces_available():
@@ -76,6 +72,8 @@ def pytest_addoption(parser):
help="Delete all generated logs created by a test after
the completion of a test.")
parser.addoption("--execute-upgrade-tests", action="store_true",
default=False,
help="Execute Cassandra Upgrade Tests (e.g. tests
annotated with the upgrade_test mark)")
+parser.addoption("--execute-upgrade-tests-only", action="store_true",
default=False,
+ help="Execute Cassandra Upgrade Tests without running any
other tests")
parser.addoption("--disable-active-log-watching", action="store_true",
default=False,
help="Disable ccm active log watching, which will cause
dtests to check for errors in the "
"logs in a single operation instead of semi-realtime
processing by consuming "
@@ -477,6 +475,11 @@ def pytest_collection_modifyitems(items, config):
for item in items:
deselect_test = False
+if config.getoption("--execute-upgrade-tests-only"):
+deselect_test = not item.get_closest_marker("upgrade_test")
+if deselect_test:
+logger.info("SKIP: Deselecting non-upgrade test %s because of
--execute-upgrade-tests-only" % item.name)
+
if item.get_closest_marker("resource_intensive") and not collect_only:
force_resource_intensive =
config.getoption("--force-resource-intensive-tests")
skip_resource_intensive =
config.getoption("--skip-resource-intensive-tests")
@@ -495,7 +498,6 @@ def pytest_collection_modifyitems(items, config):
deselect_test = True
logger.info("SKIP: Deselecting non resource_intensive test %s
as --only-resource-intensive-tests specified" % item.name)
-
if item.get_closest_marker("no_vnodes"):
if config.getoption("--use-vnodes"):
deselect_test = True
@@ -514,12 +516,10 @@ def pytest_collection_modifyitems(items, config):
for module_pytest_mark in test_item_class[1].pytestmark:
if module_pytest_mark.name == "upgrade_test":
-if not config.getoption("--execute-upgrade-tests"):
-deselect_test = True
+deselect_test = not _upgrade_testing_enabled(config)
if item.get_closest_marker("upgrade_test"):
-if not config.getoption("--execute-upgrade-tests"):
-deselect_test = True
+deselect_test = not _upgrade_testing_enabled(config)
if item.get_closest_marker("no_offheap_memtables"):
if config.getoption("use_off_heap_memtables"):
@@ -536,3 +536,7 @@ def pytest_collection_modifyitems(items, config):
config.hook.pytest_deselected(items=deselected_items)
items[:] = selected_items
+
+
+def _upgrade_testing_enabled(config):
+return config.getoption("--execute-upgrade-tests") or
config.getoption("--execute-upgrade-tests-only")
diff --git a/dtest_config.py b/dtest_config.py
index
[cassandra-dtest] branch master updated: Introduce --execute-upgrade-tests-only to only run upgrade tests
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/master by this push:
new 7da11dd Introduce --execute-upgrade-tests-only to only run upgrade
tests
7da11dd is described below
commit 7da11ddc2da9a5c4938044c21909ac4221580687
Author: Eduard Tudenhoefner
AuthorDate: Thu Jun 4 08:33:26 2020 +0200
Introduce --execute-upgrade-tests-only to only run upgrade tests
---
conftest.py | 44
dtest_config.py | 2 ++
run_dtests.py | 3 ++-
3 files changed, 28 insertions(+), 21 deletions(-)
diff --git a/conftest.py b/conftest.py
index 2629689..1fa5e22 100644
--- a/conftest.py
+++ b/conftest.py
@@ -1,32 +1,28 @@
-import pytest
+import copy
+import inspect
import logging
import os
+import platform
+import re
import shutil
import time
-import re
-import platform
-import copy
-import inspect
-import subprocess
-
-from dtest import running_in_docker,
cleanup_docker_environment_before_test_execution
-
from datetime import datetime
from distutils.version import LooseVersion
-from netifaces import AF_INET
-from psutil import virtual_memory
+# Python 3 imports
+from itertools import zip_longest
-import netifaces as ni
import ccmlib.repository
+import netifaces as ni
+import pytest
from ccmlib.common import validate_install_dir, is_win, get_version_from_build
+from netifaces import AF_INET
+from psutil import virtual_memory
+from dtest import running_in_docker,
cleanup_docker_environment_before_test_execution
from dtest_config import DTestConfig
from dtest_setup import DTestSetup
from dtest_setup_overrides import DTestSetupOverrides
-# Python 3 imports
-from itertools import zip_longest
-
logger = logging.getLogger(__name__)
def check_required_loopback_interfaces_available():
@@ -76,6 +72,8 @@ def pytest_addoption(parser):
help="Delete all generated logs created by a test after
the completion of a test.")
parser.addoption("--execute-upgrade-tests", action="store_true",
default=False,
help="Execute Cassandra Upgrade Tests (e.g. tests
annotated with the upgrade_test mark)")
+parser.addoption("--execute-upgrade-tests-only", action="store_true",
default=False,
+ help="Execute Cassandra Upgrade Tests without running any
other tests")
parser.addoption("--disable-active-log-watching", action="store_true",
default=False,
help="Disable ccm active log watching, which will cause
dtests to check for errors in the "
"logs in a single operation instead of semi-realtime
processing by consuming "
@@ -477,6 +475,11 @@ def pytest_collection_modifyitems(items, config):
for item in items:
deselect_test = False
+if config.getoption("--execute-upgrade-tests-only"):
+deselect_test = not item.get_closest_marker("upgrade_test")
+if deselect_test:
+logger.info("SKIP: Deselecting non-upgrade test %s because of
--execute-upgrade-tests-only" % item.name)
+
if item.get_closest_marker("resource_intensive") and not collect_only:
force_resource_intensive =
config.getoption("--force-resource-intensive-tests")
skip_resource_intensive =
config.getoption("--skip-resource-intensive-tests")
@@ -495,7 +498,6 @@ def pytest_collection_modifyitems(items, config):
deselect_test = True
logger.info("SKIP: Deselecting non resource_intensive test %s
as --only-resource-intensive-tests specified" % item.name)
-
if item.get_closest_marker("no_vnodes"):
if config.getoption("--use-vnodes"):
deselect_test = True
@@ -514,12 +516,10 @@ def pytest_collection_modifyitems(items, config):
for module_pytest_mark in test_item_class[1].pytestmark:
if module_pytest_mark.name == "upgrade_test":
-if not config.getoption("--execute-upgrade-tests"):
-deselect_test = True
+deselect_test = not _upgrade_testing_enabled(config)
if item.get_closest_marker("upgrade_test"):
-if not config.getoption("--execute-upgrade-tests"):
-deselect_test = True
+deselect_test = not _upgrade_testing_enabled(config)
if item.get_closest_marker("no_offheap_memtables"):
if config.getoption("use_off_heap_memtables"):
@@ -536,3 +536,7 @@ def pytest_collection_modifyitems(items, config):
config.hook.pytest_deselected(items=deselected_items)
items[:] = selected_items
+
+
+def _upgrade_testing_enabled(config):
+return config.getoption("--execute-upgrade-tests") or
config.getoption("--execute-upgrade-tests-only")
diff --git a/dtest_config.py b/dtest_config.py
index
