[PR] CASSANDRA-19562: DefaultLoadBalancingPolicy considers a TimeOut as a response [cassandra-java-driver]
SiyaoIsHiding opened a new pull request, #1927: URL: https://github.com/apache/cassandra-java-driver/pull/1927 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-19562) DefaultLoadBalancingPolicy considers a TimeOut as a response
[ https://issues.apache.org/jira/browse/CASSANDRA-19562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jane He reassigned CASSANDRA-19562: --- Assignee: Jane He (was: Henry Hughes) > DefaultLoadBalancingPolicy considers a TimeOut as a response > > > Key: CASSANDRA-19562 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19562 > Project: Cassandra > Issue Type: Bug > Components: Client/java-driver >Reporter: Jane He >Assignee: Jane He >Priority: Normal > > At this line: > [https://github.com/apache/cassandra-java-driver/blob/388a46b9c10b5653c71ac8840bcda0c91b59bce4/core/src/main/java/com/datastax/oss/driver/internal/core/loadbalancing/DefaultLoadBalancingPolicy.java#L255] > It considers a timeout as a response, when it is counting whether there are 2 > responses in the past 200ms to decide whether a node is slow and should be > excluded. This will result in the driver continue sending requests to a node > that is not responding and giving timeout exception. > It should be fixed just by an if statement saying if the error is not > DriverTimeoutException, then update response times. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-117) Record additional metrics for improved observability
[ https://issues.apache.org/jira/browse/CASSANDRASC-117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837911#comment-17837911 ] Yifan Cai commented on CASSANDRASC-117: --- +1 > Record additional metrics for improved observability > > > Key: CASSANDRASC-117 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-117 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Saranya Krishnakumar >Assignee: Saranya Krishnakumar >Priority: Normal > Labels: pull-request-available > > With this patch increasing the number of metrics captured on sidecar side for > improved observability. This includes, capturing resource level metrics, > Cassandra instance health metrics, SSTable import metrics etc. Also as part > of this patch, migrating existing metrics to Dropwizard registry. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-94) Reduce filesystem calls while streaming SSTables
[
https://issues.apache.org/jira/browse/CASSANDRASC-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837910#comment-17837910
]
Francisco Guerrero commented on CASSANDRASC-94:
---
After discussing with [~yifanc], we've decided to simplify the design of this
effort to two single caches.
1. *Cache all data file locations*: This is cached once and it will not change
during the lifecycle of the application. The values come from the Storage
Service MBean {{getAllDataFileLocations}} method.
2. *snapshot list cache*: to maintain a cache of recently listed snapshot files
under a snapshot directory. This cache avoids having to access the filesystem
every time a bulk read client list the snapshot directory. This is a short
lived cache.
> Reduce filesystem calls while streaming SSTables
>
>
> Key: CASSANDRASC-94
> URL: https://issues.apache.org/jira/browse/CASSANDRASC-94
> Project: Sidecar for Apache Cassandra
> Issue Type: Improvement
> Components: Configuration
>Reporter: Francisco Guerrero
>Assignee: Francisco Guerrero
>Priority: Normal
> Labels: pull-request-available
>
> When streaming snapshotted SSTables from Cassandra Sidecar, Sidecar will
> perform multiple filesystem calls:
> - Traverse the data directories to determine the keyspace / table path
> - Once found determine if the SSTable file exists under the snapshots
> directory
> - Read the filesystem to obtain the file type and file size
> - Read the requested range of the file and stream it
> The amount of filesystem calls is manageable for streaming a single SSTable,
> but when a client(s) read multiple SSTables, for example in the case of
> Cassandra Analytics bulk reads, hundred to thousand of requests are performed
> requiring every request to perform the above system calls.
> In this improvement, it is proposed introducing several caches to reduce the
> amount of system calls while streaming SSTables.
> - *snapshot list cache*: to maintain a cache of recently listed snapshot
> files under a snapshot directory. This cache avoids having to access the
> filesystem every time a bulk read client list the snapshot directory.
> - *table dir cache*: to maintain a cache of recently streamed table directory
> paths. This cache helps avoiding having to traverse the filesystem searching
> for the table directory while running bulk reads for example. Since bulk
> reads can stream tens to hundreds of SSTable components from a snapshot
> directory, this cache helps avoid having to resolve the table directory each
> time.
> - *snapshot path cache*: to maintain a cache of recently streamed snapshot
> SSTable components. This cache avoids having to resolve the snapshot SSTable
> component path during bulk reads. Since bulk reads streams sub-ranges of an
> SSTable component, the resolution can happen multiple times during bulk reads
> for a single SSTable component.
> - *file props cache*: to maintain a cache of FileProps of recently streamed
> files. This cache avoids having to validate file properties during bulk reads
> for example where sub-ranges of an SSTable component are streamed, therefore
> reading the file properties can occur multiple times during bulk reads of the
> same file.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19564) MemtablePostFlush deadlock leads to stuck nodes and crashes
[
https://issues.apache.org/jira/browse/CASSANDRA-19564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837894#comment-17837894
]
Jon Haddad commented on CASSANDRA-19564:
{noformat}
memtable_heap_space: 5120MiB
memtable_offheap_space: 5120MiB
memtable_allocation_type: offheap_objects
memtable_flush_writers: 4
{noformat}
> MemtablePostFlush deadlock leads to stuck nodes and crashes
> ---
>
> Key: CASSANDRA-19564
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19564
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction, Local/Memtable
>Reporter: Jon Haddad
>Priority: Urgent
> Fix For: 4.1.x
>
> Attachments: image-2024-04-16-11-55-54-750.png,
> image-2024-04-16-12-29-15-386.png, image-2024-04-16-13-43-11-064.png,
> image-2024-04-16-13-53-24-455.png
>
>
> I've run into an issue on a 4.1.4 cluster where an entire node has locked up
> due to what I believe is a deadlock in memtable flushing. Here's what I know
> so far. I've stitched together what happened based on conversations, logs,
> and some flame graphs.
> *Log reports memtable flushing*
> The last successful flush happens at 12:19.
> {noformat}
> INFO [NativePoolCleaner] 2024-04-16 12:19:53,634
> AbstractAllocatorMemtable.java:286 - Flushing largest CFS(Keyspace='ks',
> ColumnFamily='version') to free up room. Used total: 0.24/0.33, live:
> 0.16/0.20, flushing: 0.09/0.13, this: 0.13/0.15
> INFO [NativePoolCleaner] 2024-04-16 12:19:53,634 ColumnFamilyStore.java:1012
> - Enqueuing flush of ks.version, Reason: MEMTABLE_LIMIT, Usage: 660.521MiB
> (13%) on-heap, 790.606MiB (15%) off-heap
> {noformat}
> *MemtablePostFlush appears to be blocked*
> At this point, MemtablePostFlush completed tasks stops incrementing, active
> stays at 1 and pending starts to rise.
> {noformat}
> MemtablePostFlush 1 1 3446 0 0
> {noformat}
>
> The flame graph reveals that PostFlush.call is stuck. I don't have the line
> number, but I know we're stuck in
> {{org.apache.cassandra.db.ColumnFamilyStore.PostFlush#call}} given the visual
> below:
> *!image-2024-04-16-13-43-11-064.png!*
> *Memtable flushing is now blocked.*
> All MemtableFlushWriter threads are Parked waiting on
> {{{}OpOrder.Barrier.await{}}}. A wall clock profile of 30s reveals all time
> is spent here. Presumably we're waiting on the single threaded Post Flush.
> !image-2024-04-16-12-29-15-386.png!
> *Memtable allocations start to block*
> Eventually it looks like the NativeAllocator stops successfully allocating
> memory. I assume it's waiting on memory to be freed, but since memtable
> flushes are blocked, we wait indefinitely.
> Looking at a wall clock flame graph, all writer threads have reached the
> allocation failure path of {{MemtableAllocator.allocate()}}. I believe we're
> waiting on {{signal.awaitThrowUncheckedOnInterrupt()}}
> {noformat}
> MutationStage 48 828425 980253369 0 0{noformat}
> !image-2024-04-16-11-55-54-750.png!
>
> *Compaction Stops*
> Since we write to the compaction history table, and that requires memtables,
> compactions are now blocked as well.
>
> !image-2024-04-16-13-53-24-455.png!
>
> The node is now doing basically nothing and must be restarted.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19564) MemtablePostFlush deadlock leads to stuck nodes and crashes
[
https://issues.apache.org/jira/browse/CASSANDRA-19564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837889#comment-17837889
]
Brandon Williams commented on CASSANDRA-19564:
--
Can you note which memtable_allocation_type was used and how many
memtable_flush_writers?
> MemtablePostFlush deadlock leads to stuck nodes and crashes
> ---
>
> Key: CASSANDRA-19564
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19564
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction, Local/Memtable
>Reporter: Jon Haddad
>Priority: Urgent
> Fix For: 4.1.x
>
> Attachments: image-2024-04-16-11-55-54-750.png,
> image-2024-04-16-12-29-15-386.png, image-2024-04-16-13-43-11-064.png,
> image-2024-04-16-13-53-24-455.png
>
>
> I've run into an issue on a 4.1.4 cluster where an entire node has locked up
> due to what I believe is a deadlock in memtable flushing. Here's what I know
> so far. I've stitched together what happened based on conversations, logs,
> and some flame graphs.
> *Log reports memtable flushing*
> The last successful flush happens at 12:19.
> {noformat}
> INFO [NativePoolCleaner] 2024-04-16 12:19:53,634
> AbstractAllocatorMemtable.java:286 - Flushing largest CFS(Keyspace='ks',
> ColumnFamily='version') to free up room. Used total: 0.24/0.33, live:
> 0.16/0.20, flushing: 0.09/0.13, this: 0.13/0.15
> INFO [NativePoolCleaner] 2024-04-16 12:19:53,634 ColumnFamilyStore.java:1012
> - Enqueuing flush of ks.version, Reason: MEMTABLE_LIMIT, Usage: 660.521MiB
> (13%) on-heap, 790.606MiB (15%) off-heap
> {noformat}
> *MemtablePostFlush appears to be blocked*
> At this point, MemtablePostFlush completed tasks stops incrementing, active
> stays at 1 and pending starts to rise.
> {noformat}
> MemtablePostFlush 1 1 3446 0 0
> {noformat}
>
> The flame graph reveals that PostFlush.call is stuck. I don't have the line
> number, but I know we're stuck in
> {{org.apache.cassandra.db.ColumnFamilyStore.PostFlush#call}} given the visual
> below:
> *!image-2024-04-16-13-43-11-064.png!*
> *Memtable flushing is now blocked.*
> All MemtableFlushWriter threads are Parked waiting on
> {{{}OpOrder.Barrier.await{}}}. A wall clock profile of 30s reveals all time
> is spent here. Presumably we're waiting on the single threaded Post Flush.
> !image-2024-04-16-12-29-15-386.png!
> *Memtable allocations start to block*
> Eventually it looks like the NativeAllocator stops successfully allocating
> memory. I assume it's waiting on memory to be freed, but since memtable
> flushes are blocked, we wait indefinitely.
> Looking at a wall clock flame graph, all writer threads have reached the
> allocation failure path of {{MemtableAllocator.allocate()}}. I believe we're
> waiting on {{signal.awaitThrowUncheckedOnInterrupt()}}
> {noformat}
> MutationStage 48 828425 980253369 0 0{noformat}
> !image-2024-04-16-11-55-54-750.png!
>
> *Compaction Stops*
> Since we write to the compaction history table, and that requires memtables,
> compactions are now blocked as well.
>
> !image-2024-04-16-13-53-24-455.png!
>
> The node is now doing basically nothing and must be restarted.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19564) MemtablePostFlush deadlock leads to stuck nodes and crashes
[
https://issues.apache.org/jira/browse/CASSANDRA-19564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-19564:
---
Summary: MemtablePostFlush deadlock leads to stuck nodes and crashes (was:
MemtablePostFlush deadlock leads to stuck nodes)
> MemtablePostFlush deadlock leads to stuck nodes and crashes
> ---
>
> Key: CASSANDRA-19564
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19564
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction, Local/Memtable
>Reporter: Jon Haddad
>Priority: Urgent
> Fix For: 4.1.x
>
> Attachments: image-2024-04-16-11-55-54-750.png,
> image-2024-04-16-12-29-15-386.png, image-2024-04-16-13-43-11-064.png,
> image-2024-04-16-13-53-24-455.png
>
>
> I've run into an issue on a 4.1.4 cluster where an entire node has locked up
> due to what I believe is a deadlock in memtable flushing. Here's what I know
> so far. I've stitched together what happened based on conversations, logs,
> and some flame graphs.
> *Log reports memtable flushing*
> The last successful flush happens at 12:19.
> {noformat}
> INFO [NativePoolCleaner] 2024-04-16 12:19:53,634
> AbstractAllocatorMemtable.java:286 - Flushing largest CFS(Keyspace='ks',
> ColumnFamily='version') to free up room. Used total: 0.24/0.33, live:
> 0.16/0.20, flushing: 0.09/0.13, this: 0.13/0.15
> INFO [NativePoolCleaner] 2024-04-16 12:19:53,634 ColumnFamilyStore.java:1012
> - Enqueuing flush of ks.version, Reason: MEMTABLE_LIMIT, Usage: 660.521MiB
> (13%) on-heap, 790.606MiB (15%) off-heap
> {noformat}
> *MemtablePostFlush appears to be blocked*
> At this point, MemtablePostFlush completed tasks stops incrementing, active
> stays at 1 and pending starts to rise.
> {noformat}
> MemtablePostFlush 1 1 3446 0 0
> {noformat}
>
> The flame graph reveals that PostFlush.call is stuck. I don't have the line
> number, but I know we're stuck in
> {{org.apache.cassandra.db.ColumnFamilyStore.PostFlush#call}} given the visual
> below:
> *!image-2024-04-16-13-43-11-064.png!*
> *Memtable flushing is now blocked.*
> All MemtableFlushWriter threads are Parked waiting on
> {{{}OpOrder.Barrier.await{}}}. A wall clock profile of 30s reveals all time
> is spent here. Presumably we're waiting on the single threaded Post Flush.
> !image-2024-04-16-12-29-15-386.png!
> *Memtable allocations start to block*
> Eventually it looks like the NativeAllocator stops successfully allocating
> memory. I assume it's waiting on memory to be freed, but since memtable
> flushes are blocked, we wait indefinitely.
> Looking at a wall clock flame graph, all writer threads have reached the
> allocation failure path of {{MemtableAllocator.allocate()}}. I believe we're
> waiting on {{signal.awaitThrowUncheckedOnInterrupt()}}
> {noformat}
> MutationStage 48 828425 980253369 0 0{noformat}
> !image-2024-04-16-11-55-54-750.png!
>
> *Compaction Stops*
> Since we write to the compaction history table, and that requires memtables,
> compactions are now blocked as well.
>
> !image-2024-04-16-13-53-24-455.png!
>
> The node is now doing basically nothing and must be restarted.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19561) finish_release.sh does not handle jfrog errors
[ https://issues.apache.org/jira/browse/CASSANDRA-19561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837886#comment-17837886 ] Brandon Williams commented on CASSANDRA-19561: -- I don't really want to pull in jq just to use a non-standard error mechanism, but even so with the execution passed through the 'execute()' function, we can't pass the output to it or even grep. Even if we could, if the script aborts at that you still can't rerun it to try again and finish things without some hackery. > finish_release.sh does not handle jfrog errors > -- > > Key: CASSANDRA-19561 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19561 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Brandon Williams >Priority: Normal > > If there is any "soft" problem (not network) uploading to jfrog, > finish_release.sh will keep on going and then delete the packages regardless > of whether they could be uploaded or not. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19561) finish_release.sh does not handle jfrog errors
[ https://issues.apache.org/jira/browse/CASSANDRA-19561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837886#comment-17837886 ] Brandon Williams edited comment on CASSANDRA-19561 at 4/16/24 9:16 PM: --- I don't really want to pull in jq just to use a non-standard error mechanism, but even so with the execution passed through the 'execute()' function, we can't pass the output to it or even grep. Even if we could, if the script aborts at that point you still can't rerun it to try again and finish things without some hackery. was (Author: brandon.williams): I don't really want to pull in jq just to use a non-standard error mechanism, but even so with the execution passed through the 'execute()' function, we can't pass the output to it or even grep. Even if we could, if the script aborts at that you still can't rerun it to try again and finish things without some hackery. > finish_release.sh does not handle jfrog errors > -- > > Key: CASSANDRA-19561 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19561 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Brandon Williams >Priority: Normal > > If there is any "soft" problem (not network) uploading to jfrog, > finish_release.sh will keep on going and then delete the packages regardless > of whether they could be uploaded or not. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19564) MemtablePostFlush deadlock leads to stuck nodes
[
https://issues.apache.org/jira/browse/CASSANDRA-19564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19564:
-
Complexity: Normal
Component/s: Local/Memtable
Discovered By: User Report
Fix Version/s: 4.1.x
Status: Open (was: Triage Needed)
> MemtablePostFlush deadlock leads to stuck nodes
> ---
>
> Key: CASSANDRA-19564
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19564
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction, Local/Memtable
>Reporter: Jon Haddad
>Priority: Urgent
> Fix For: 4.1.x
>
> Attachments: image-2024-04-16-11-55-54-750.png,
> image-2024-04-16-12-29-15-386.png, image-2024-04-16-13-43-11-064.png,
> image-2024-04-16-13-53-24-455.png
>
>
> I've run into an issue on a 4.1.4 cluster where an entire node has locked up
> due to what I believe is a deadlock in memtable flushing. Here's what I know
> so far. I've stitched together what happened based on conversations, logs,
> and some flame graphs.
> *Log reports memtable flushing*
> The last successful flush happens at 12:19.
> {noformat}
> INFO [NativePoolCleaner] 2024-04-16 12:19:53,634
> AbstractAllocatorMemtable.java:286 - Flushing largest CFS(Keyspace='ks',
> ColumnFamily='version') to free up room. Used total: 0.24/0.33, live:
> 0.16/0.20, flushing: 0.09/0.13, this: 0.13/0.15
> INFO [NativePoolCleaner] 2024-04-16 12:19:53,634 ColumnFamilyStore.java:1012
> - Enqueuing flush of ks.version, Reason: MEMTABLE_LIMIT, Usage: 660.521MiB
> (13%) on-heap, 790.606MiB (15%) off-heap
> {noformat}
> *MemtablePostFlush appears to be blocked*
> At this point, MemtablePostFlush completed tasks stops incrementing, active
> stays at 1 and pending starts to rise.
> {noformat}
> MemtablePostFlush 1 1 3446 0 0
> {noformat}
>
> The flame graph reveals that PostFlush.call is stuck. I don't have the line
> number, but I know we're stuck in
> {{org.apache.cassandra.db.ColumnFamilyStore.PostFlush#call}} given the visual
> below:
> *!image-2024-04-16-13-43-11-064.png!*
> *Memtable flushing is now blocked.*
> All MemtableFlushWriter threads are Parked waiting on
> {{{}OpOrder.Barrier.await{}}}. A wall clock profile of 30s reveals all time
> is spent here. Presumably we're waiting on the single threaded Post Flush.
> !image-2024-04-16-12-29-15-386.png!
> *Memtable allocations start to block*
> Eventually it looks like the NativeAllocator stops successfully allocating
> memory. I assume it's waiting on memory to be freed, but since memtable
> flushes are blocked, we wait indefinitely.
> Looking at a wall clock flame graph, all writer threads have reached the
> allocation failure path of {{MemtableAllocator.allocate()}}. I believe we're
> waiting on {{signal.awaitThrowUncheckedOnInterrupt()}}
> {noformat}
> MutationStage 48 828425 980253369 0 0{noformat}
> !image-2024-04-16-11-55-54-750.png!
>
> *Compaction Stops*
> Since we write to the compaction history table, and that requires memtables,
> compactions are now blocked as well.
>
> !image-2024-04-16-13-53-24-455.png!
>
> The node is now doing basically nothing and must be restarted.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19564) MemtablePostFlush deadlock leads to stuck nodes
[
https://issues.apache.org/jira/browse/CASSANDRA-19564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-19564:
---
Bug Category: Parent values: Availability(12983)Level 1 values: Process
Crash(12992)
Severity: Critical
Since Version: 4.1.4
> MemtablePostFlush deadlock leads to stuck nodes
> ---
>
> Key: CASSANDRA-19564
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19564
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Jon Haddad
>Priority: Urgent
> Attachments: image-2024-04-16-11-55-54-750.png,
> image-2024-04-16-12-29-15-386.png, image-2024-04-16-13-43-11-064.png,
> image-2024-04-16-13-53-24-455.png
>
>
> I've run into an issue on a 4.1.4 cluster where an entire node has locked up
> due to what I believe is a deadlock in memtable flushing. Here's what I know
> so far. I've stitched together what happened based on conversations, logs,
> and some flame graphs.
> *Log reports memtable flushing*
> The last successful flush happens at 12:19.
> {noformat}
> INFO [NativePoolCleaner] 2024-04-16 12:19:53,634
> AbstractAllocatorMemtable.java:286 - Flushing largest CFS(Keyspace='ks',
> ColumnFamily='version') to free up room. Used total: 0.24/0.33, live:
> 0.16/0.20, flushing: 0.09/0.13, this: 0.13/0.15
> INFO [NativePoolCleaner] 2024-04-16 12:19:53,634 ColumnFamilyStore.java:1012
> - Enqueuing flush of ks.version, Reason: MEMTABLE_LIMIT, Usage: 660.521MiB
> (13%) on-heap, 790.606MiB (15%) off-heap
> {noformat}
> *MemtablePostFlush appears to be blocked*
> At this point, MemtablePostFlush completed tasks stops incrementing, active
> stays at 1 and pending starts to rise.
> {noformat}
> MemtablePostFlush 1 1 3446 0 0
> {noformat}
>
> The flame graph reveals that PostFlush.call is stuck. I don't have the line
> number, but I know we're stuck in
> {{org.apache.cassandra.db.ColumnFamilyStore.PostFlush#call}} given the visual
> below:
> *!image-2024-04-16-13-43-11-064.png!*
> *Memtable flushing is now blocked.*
> All MemtableFlushWriter threads are Parked waiting on
> {{{}OpOrder.Barrier.await{}}}. A wall clock profile of 30s reveals all time
> is spent here. Presumably we're waiting on the single threaded Post Flush.
> !image-2024-04-16-12-29-15-386.png!
> *Memtable allocations start to block*
> Eventually it looks like the NativeAllocator stops successfully allocating
> memory. I assume it's waiting on memory to be freed, but since memtable
> flushes are blocked, we wait indefinitely.
> Looking at a wall clock flame graph, all writer threads have reached the
> allocation failure path of {{MemtableAllocator.allocate()}}. I believe we're
> waiting on {{signal.awaitThrowUncheckedOnInterrupt()}}
> {noformat}
> MutationStage 48 828425 980253369 0 0{noformat}
> !image-2024-04-16-11-55-54-750.png!
>
> *Compaction Stops*
> Since we write to the compaction history table, and that requires memtables,
> compactions are now blocked as well.
>
> !image-2024-04-16-13-53-24-455.png!
>
> The node is now doing basically nothing and must be restarted.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19564) MemtablePostFlush deadlock leads to stuck nodes
[
https://issues.apache.org/jira/browse/CASSANDRA-19564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-19564:
---
Description:
I've run into an issue on a 4.1.4 cluster where an entire node has locked up
due to what I believe is a deadlock in memtable flushing. Here's what I know so
far. I've stitched together what happened based on conversations, logs, and
some flame graphs.
*Log reports memtable flushing*
The last successful flush happens at 12:19.
{noformat}
INFO [NativePoolCleaner] 2024-04-16 12:19:53,634
AbstractAllocatorMemtable.java:286 - Flushing largest CFS(Keyspace='ks',
ColumnFamily='version') to free up room. Used total: 0.24/0.33, live:
0.16/0.20, flushing: 0.09/0.13, this: 0.13/0.15
INFO [NativePoolCleaner] 2024-04-16 12:19:53,634 ColumnFamilyStore.java:1012 -
Enqueuing flush of ks.version, Reason: MEMTABLE_LIMIT, Usage: 660.521MiB (13%)
on-heap, 790.606MiB (15%) off-heap
{noformat}
*MemtablePostFlush appears to be blocked*
At this point, MemtablePostFlush completed tasks stops incrementing, active
stays at 1 and pending starts to rise.
{noformat}
MemtablePostFlush 1 1 3446 0 0
{noformat}
The flame graph reveals that PostFlush.call is stuck. I don't have the line
number, but I know we're stuck in
{{org.apache.cassandra.db.ColumnFamilyStore.PostFlush#call}} given the visual
below:
*!image-2024-04-16-13-43-11-064.png!*
*Memtable flushing is now blocked.*
All MemtableFlushWriter threads are Parked waiting on
{{{}OpOrder.Barrier.await{}}}. A wall clock profile of 30s reveals all time is
spent here. Presumably we're waiting on the single threaded Post Flush.
!image-2024-04-16-12-29-15-386.png!
*Memtable allocations start to block*
Eventually it looks like the NativeAllocator stops successfully allocating
memory. I assume it's waiting on memory to be freed, but since memtable flushes
are blocked, we wait indefinitely.
Looking at a wall clock flame graph, all writer threads have reached the
allocation failure path of {{MemtableAllocator.allocate()}}. I believe we're
waiting on {{signal.awaitThrowUncheckedOnInterrupt()}}
{noformat}
MutationStage 48 828425 980253369 0 0{noformat}
!image-2024-04-16-11-55-54-750.png!
*Compaction Stops*
Since we write to the compaction history table, and that requires memtables,
compactions are now blocked as well.
!image-2024-04-16-13-53-24-455.png!
The node is now doing basically nothing and must be restarted.
was:
I've run into an issue on a 4.1.4 cluster where an entire node has locked up
due to what I believe is a deadlock in memtable flushing. Here's what I know so
far. I've stitched together what happened based on conversations, logs, and
some flame graphs.
*Log reports memtable flushing*
The last successful flush happens at 12:19.
{noformat}
INFO [NativePoolCleaner] 2024-04-16 12:19:53,634
AbstractAllocatorMemtable.java:286 - Flushing largest CFS(Keyspace='ks',
ColumnFamily='version') to free up room. Used total: 0.24/0.33, live:
0.16/0.20, flushing: 0.09/0.13, this: 0.13/0.15
INFO [NativePoolCleaner] 2024-04-16 12:19:53,634 ColumnFamilyStore.java:1012 -
Enqueuing flush of ks.version, Reason: MEMTABLE_LIMIT, Usage: 660.521MiB (13%)
on-heap, 790.606MiB (15%) off-heap
{noformat}
*MemtablePostFlush appears to be blocked*
At this point, MemtablePostFlush completed tasks stops incrementing, active
stays at 1 and pending starts to rise.
{noformat}
MemtablePostFlush 1 1 3446 0 0{noformat}
The flame graph reveals that PostFlush.call is stuck. I don't have the line
number, but I know we're stuck in
{{org.apache.cassandra.db.ColumnFamilyStore.PostFlush#call}} given the visual
below:
*!image-2024-04-16-13-43-11-064.png!*
*Memtable flushing is now blocked.*
All MemtableFlushWriter threads are Parked waiting on
{{{}OpOrder.Barrier.await{}}}. A wall clock profile of 30s reveals all time is
spent here. Presumably we're waiting on the single threaded Post Flush.
!image-2024-04-16-12-29-15-386.png!
*Memtable allocations start to block*
Eventually it looks like the NativeAllocator stops successfully allocating
memory. I assume it's waiting on memory to be freed, but since memtable flushes
are blocked, we wait indefinitely.
Looking at a wall clock flame graph, all writer threads have reached the
allocation failure path of {{MemtableAllocator.allocate(). I believe we're
waiting on }}{{signal.awaitThrowUncheckedOnInterrupt()}}
{noformat}
MutationStage 48 828425 980253369 0 0{noformat}
!image-2024-04-16-11-55-54-750.png!
*Compaction Stops*
Since we write to the compaction history table, and that requires memtables,
compactions are now blocked as well.
!image-2024-04-16-13-53-24-455.png!
The node is now doing basically nothing and must be restarted.
> MemtablePostFlush deadlock leads to stuck nodes
>
[jira] [Created] (CASSANDRA-19564) MemtablePostFlush deadlock leads to stuck nodes
Jon Haddad created CASSANDRA-19564:
--
Summary: MemtablePostFlush deadlock leads to stuck nodes
Key: CASSANDRA-19564
URL: https://issues.apache.org/jira/browse/CASSANDRA-19564
Project: Cassandra
Issue Type: Bug
Components: Local/Compaction
Reporter: Jon Haddad
Attachments: image-2024-04-16-11-55-54-750.png,
image-2024-04-16-12-29-15-386.png, image-2024-04-16-13-43-11-064.png,
image-2024-04-16-13-53-24-455.png
I've run into an issue on a 4.1.4 cluster where an entire node has locked up
due to what I believe is a deadlock in memtable flushing. Here's what I know so
far. I've stitched together what happened based on conversations, logs, and
some flame graphs.
*Log reports memtable flushing*
The last successful flush happens at 12:19.
{noformat}
INFO [NativePoolCleaner] 2024-04-16 12:19:53,634
AbstractAllocatorMemtable.java:286 - Flushing largest CFS(Keyspace='ks',
ColumnFamily='version') to free up room. Used total: 0.24/0.33, live:
0.16/0.20, flushing: 0.09/0.13, this: 0.13/0.15
INFO [NativePoolCleaner] 2024-04-16 12:19:53,634 ColumnFamilyStore.java:1012 -
Enqueuing flush of ks.version, Reason: MEMTABLE_LIMIT, Usage: 660.521MiB (13%)
on-heap, 790.606MiB (15%) off-heap
{noformat}
*MemtablePostFlush appears to be blocked*
At this point, MemtablePostFlush completed tasks stops incrementing, active
stays at 1 and pending starts to rise.
{noformat}
MemtablePostFlush 1 1 3446 0 0{noformat}
The flame graph reveals that PostFlush.call is stuck. I don't have the line
number, but I know we're stuck in
{{org.apache.cassandra.db.ColumnFamilyStore.PostFlush#call}} given the visual
below:
*!image-2024-04-16-13-43-11-064.png!*
*Memtable flushing is now blocked.*
All MemtableFlushWriter threads are Parked waiting on
{{{}OpOrder.Barrier.await{}}}. A wall clock profile of 30s reveals all time is
spent here. Presumably we're waiting on the single threaded Post Flush.
!image-2024-04-16-12-29-15-386.png!
*Memtable allocations start to block*
Eventually it looks like the NativeAllocator stops successfully allocating
memory. I assume it's waiting on memory to be freed, but since memtable flushes
are blocked, we wait indefinitely.
Looking at a wall clock flame graph, all writer threads have reached the
allocation failure path of {{MemtableAllocator.allocate(). I believe we're
waiting on }}{{signal.awaitThrowUncheckedOnInterrupt()}}
{noformat}
MutationStage 48 828425 980253369 0 0{noformat}
!image-2024-04-16-11-55-54-750.png!
*Compaction Stops*
Since we write to the compaction history table, and that requires memtables,
compactions are now blocked as well.
!image-2024-04-16-13-53-24-455.png!
The node is now doing basically nothing and must be restarted.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19564) MemtablePostFlush deadlock leads to stuck nodes
[
https://issues.apache.org/jira/browse/CASSANDRA-19564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-19564:
---
Description:
I've run into an issue on a 4.1.4 cluster where an entire node has locked up
due to what I believe is a deadlock in memtable flushing. Here's what I know so
far. I've stitched together what happened based on conversations, logs, and
some flame graphs.
*Log reports memtable flushing*
The last successful flush happens at 12:19.
{noformat}
INFO [NativePoolCleaner] 2024-04-16 12:19:53,634
AbstractAllocatorMemtable.java:286 - Flushing largest CFS(Keyspace='ks',
ColumnFamily='version') to free up room. Used total: 0.24/0.33, live:
0.16/0.20, flushing: 0.09/0.13, this: 0.13/0.15
INFO [NativePoolCleaner] 2024-04-16 12:19:53,634 ColumnFamilyStore.java:1012 -
Enqueuing flush of ks.version, Reason: MEMTABLE_LIMIT, Usage: 660.521MiB (13%)
on-heap, 790.606MiB (15%) off-heap
{noformat}
*MemtablePostFlush appears to be blocked*
At this point, MemtablePostFlush completed tasks stops incrementing, active
stays at 1 and pending starts to rise.
{noformat}
MemtablePostFlush 1 1 3446 0 0{noformat}
The flame graph reveals that PostFlush.call is stuck. I don't have the line
number, but I know we're stuck in
{{org.apache.cassandra.db.ColumnFamilyStore.PostFlush#call}} given the visual
below:
*!image-2024-04-16-13-43-11-064.png!*
*Memtable flushing is now blocked.*
All MemtableFlushWriter threads are Parked waiting on
{{{}OpOrder.Barrier.await{}}}. A wall clock profile of 30s reveals all time is
spent here. Presumably we're waiting on the single threaded Post Flush.
!image-2024-04-16-12-29-15-386.png!
*Memtable allocations start to block*
Eventually it looks like the NativeAllocator stops successfully allocating
memory. I assume it's waiting on memory to be freed, but since memtable flushes
are blocked, we wait indefinitely.
Looking at a wall clock flame graph, all writer threads have reached the
allocation failure path of {{MemtableAllocator.allocate(). I believe we're
waiting on }}{{signal.awaitThrowUncheckedOnInterrupt()}}
{noformat}
MutationStage 48 828425 980253369 0 0{noformat}
!image-2024-04-16-11-55-54-750.png!
*Compaction Stops*
Since we write to the compaction history table, and that requires memtables,
compactions are now blocked as well.
!image-2024-04-16-13-53-24-455.png!
The node is now doing basically nothing and must be restarted.
was:
I've run into an issue on a 4.1.4 cluster where an entire node has locked up
due to what I believe is a deadlock in memtable flushing. Here's what I know so
far. I've stitched together what happened based on conversations, logs, and
some flame graphs.
*Log reports memtable flushing*
The last successful flush happens at 12:19.
{noformat}
INFO [NativePoolCleaner] 2024-04-16 12:19:53,634
AbstractAllocatorMemtable.java:286 - Flushing largest CFS(Keyspace='ks',
ColumnFamily='version') to free up room. Used total: 0.24/0.33, live:
0.16/0.20, flushing: 0.09/0.13, this: 0.13/0.15
INFO [NativePoolCleaner] 2024-04-16 12:19:53,634 ColumnFamilyStore.java:1012 -
Enqueuing flush of ks.version, Reason: MEMTABLE_LIMIT, Usage: 660.521MiB (13%)
on-heap, 790.606MiB (15%) off-heap
{noformat}
*MemtablePostFlush appears to be blocked*
At this point, MemtablePostFlush completed tasks stops incrementing, active
stays at 1 and pending starts to rise.
{noformat}
MemtablePostFlush 1 1 3446 0 0{noformat}
The flame graph reveals that PostFlush.call is stuck. I don't have the line
number, but I know we're stuck in
{{org.apache.cassandra.db.ColumnFamilyStore.PostFlush#call}} given the visual
below:
*!image-2024-04-16-13-43-11-064.png!*
*Memtable flushing is now blocked.*
All MemtableFlushWriter threads are Parked waiting on
{{{}OpOrder.Barrier.await{}}}. A wall clock profile of 30s reveals all time is
spent here. Presumably we're waiting on the single threaded Post Flush.
!image-2024-04-16-12-29-15-386.png!
*Memtable allocations start to block*
Eventually it looks like the NativeAllocator stops successfully allocating
memory. I assume it's waiting on memory to be freed, but since memtable flushes
are blocked, we wait indefinitely.
Looking at a wall clock flame graph, all writer threads have reached the
allocation failure path of {{MemtableAllocator.allocate(). I believe we're
waiting on }}{{signal.awaitThrowUncheckedOnInterrupt()}}
{noformat}
MutationStage 48 828425 980253369 0 0{noformat}
!image-2024-04-16-11-55-54-750.png!
*Compaction Stops*
Since we write to the compaction history table, and that requires memtables,
compactions are now blocked as well.
!image-2024-04-16-13-53-24-455.png!
The node is now doing basically nothing and must be restarted.
> MemtablePostFlush deadlock leads to stuck nodes
>
(cassandra-java-driver) branch 4.x updated: Changelog updates to reflect work that went out in 4.18.0
This is an automated email from the ASF dual-hosted git repository. absurdfarce pushed a commit to branch 4.x in repository https://gitbox.apache.org/repos/asf/cassandra-java-driver.git The following commit(s) were added to refs/heads/4.x by this push: new c8b17ac38 Changelog updates to reflect work that went out in 4.18.0 c8b17ac38 is described below commit c8b17ac38b48ca580b4862571cdfd7b7633a5793 Author: Bret McGuire AuthorDate: Mon Feb 19 23:07:18 2024 -0600 Changelog updates to reflect work that went out in 4.18.0 Patch by Bret McGuire; reviewed by Bret McGuire, Alexandre Dutra for PR 1914 --- changelog/README.md | 10 ++ 1 file changed, 10 insertions(+) diff --git a/changelog/README.md b/changelog/README.md index 8ff2913b7..7807ef15f 100644 --- a/changelog/README.md +++ b/changelog/README.md @@ -21,6 +21,16 @@ under the License. +### 4.18.0 + +- [improvement] PR 1689: Add support for publishing percentile time series for the histogram metrics (nparaddi-walmart) +- [improvement] JAVA-3104: Do not eagerly pre-allocate array when deserializing CqlVector +- [improvement] JAVA-3111: upgrade jackson-databind to 2.13.4.2 to address gradle dependency issue +- [improvement] PR 1617: Improve ByteBufPrimitiveCodec readBytes (chibenwa) +- [improvement] JAVA-3095: Fix CREATE keyword in vector search example in upgrade guide +- [improvement] JAVA-3100: Update jackson-databind to 2.13.4.1 and jackson-jaxrs-json-provider to 2.13.4 to address recent CVEs +- [improvement] JAVA-3089: Forbid wildcard imports + ### 4.17.0 - [improvement] JAVA-3070: Make CqlVector and CqlDuration serializable - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] Changelog updates to reflect work that went out in 4.18.0 [cassandra-java-driver]
absurdfarce merged PR #1914: URL: https://github.com/apache/cassandra-java-driver/pull/1914 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19130) Implement transactional table truncation
[
https://issues.apache.org/jira/browse/CASSANDRA-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837875#comment-17837875
]
Stefan Miklosovic edited comment on CASSANDRA-19130 at 4/16/24 8:38 PM:
[~aratnofsky] feel free to expand that patch in the direction you fancy if you
are interested in that. I am very open to the cooperation on this.
?? TCM, then in compaction remove entire SSTables with maxDataAge <=
truncatedAt, and remove all cells with earlier mutation timestamps. When we're
handling a Truncation transformation in TCM (via TableTruncationListener), kick
off a local truncation compaction??
isnt this quite a heavy operation? compacting as part of truncation? how would
that look like when a node is restarted and truncations are replayed? You would
compact on every such truncation record until all is replayed?
EDIT: I think that the logic would be same as now, basically ... it would
truncate only if timestamp in TCM is bigger than what is persisted in system's
truncatedAt field.
Anyway, as I said, just show what you mean exactly, it is a little bit
confusing for to me replicate what you mean.
was (Author: smiklosovic):
[~aratnofsky] feel free to expand that patch in the direction you fancy if you
are interested in that. I am very open to the cooperation on this.
?? TCM, then in compaction remove entire SSTables with maxDataAge <=
truncatedAt, and remove all cells with earlier mutation timestamps. When we're
handling a Truncation transformation in TCM (via TableTruncationListener), kick
off a local truncation compaction??
isnt this quite a heavy operation? compacting as part of truncation? how would
that look like when a node is restarted and truncations are replayed? You would
compact on every such truncation record until all is replayed?
> Implement transactional table truncation
>
>
> Key: CASSANDRA-19130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19130
> Project: Cassandra
> Issue Type: New Feature
> Components: Consistency/Coordination
>Reporter: Marcus Eriksson
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> TRUNCATE table should leverage cluster metadata to ensure consistent
> truncation timestamps across all replicas. The current implementation depends
> on all nodes being available, but this could be reimplemented as a
> {{Transformation}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19130) Implement transactional table truncation
[
https://issues.apache.org/jira/browse/CASSANDRA-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837875#comment-17837875
]
Stefan Miklosovic edited comment on CASSANDRA-19130 at 4/16/24 8:33 PM:
[~aratnofsky] feel free to expand that patch in the direction you fancy if you
are interested in that. I am very open to the cooperation on this.
?? TCM, then in compaction remove entire SSTables with maxDataAge <=
truncatedAt, and remove all cells with earlier mutation timestamps. When we're
handling a Truncation transformation in TCM (via TableTruncationListener), kick
off a local truncation compaction??
isnt this quite a heavy operation? compacting as part of truncation? how would
that look like when a node is restarted and truncations are replayed? You would
compact on every such truncation record until all is replayed?
was (Author: smiklosovic):
[~aratnofsky] feel free to expand that patch in the direction you fancy if you
are interested in that. I am very open to the cooperation on this.
> Implement transactional table truncation
>
>
> Key: CASSANDRA-19130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19130
> Project: Cassandra
> Issue Type: New Feature
> Components: Consistency/Coordination
>Reporter: Marcus Eriksson
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> TRUNCATE table should leverage cluster metadata to ensure consistent
> truncation timestamps across all replicas. The current implementation depends
> on all nodes being available, but this could be reimplemented as a
> {{Transformation}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19130) Implement transactional table truncation
[
https://issues.apache.org/jira/browse/CASSANDRA-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837875#comment-17837875
]
Stefan Miklosovic commented on CASSANDRA-19130:
---
[~aratnofsky] feel free to expand that patch in the direction you fancy if you
are interested in that. I am very open to the cooperation on this.
> Implement transactional table truncation
>
>
> Key: CASSANDRA-19130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19130
> Project: Cassandra
> Issue Type: New Feature
> Components: Consistency/Coordination
>Reporter: Marcus Eriksson
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> TRUNCATE table should leverage cluster metadata to ensure consistent
> truncation timestamps across all replicas. The current implementation depends
> on all nodes being available, but this could be reimplemented as a
> {{Transformation}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19476) CQL Management API
[
https://issues.apache.org/jira/browse/CASSANDRA-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837675#comment-17837675
]
Maxim Muzafarov edited comment on CASSANDRA-19476 at 4/16/24 8:20 PM:
--
I've spent some time prototyping the changes and the plan may be the following,
splitting by the issues:
# _Replace the airline with the picocli._ CASSANDRA-17445 The airlift library
is now used to expose the management commands via the CLI. We can reuse this
approach to keep the number of changes low, however, we need to replace the
library with the picocli as the airlift has drawbacks. It prevents us from
building JMX and CQL APIs in follows:
## The airlift library is no longer supported. It doesn't make sense to build
a new code on top of deprecated annotations;
## The library handles the `[ ...]` as a single input
argument, but we need to handle it independently as two input arguments
`keyspace` and `tables` to build JMX and CQL APIs;
## All commands right now have host, port, username, and password options etc.
which are not relevant to other types of public APIs;
# {_}Create an adapter that builds dynamic JMX MBeans{_}, and parses the
picocli annotations of the commands we already have. This will expose these
commands with the same parameters and options that they have in the CLI,
matching the nodetool CLI input. The management MBeans of the internal
components e.g. StorageServiceMBean can be injected into the commands directly
when the command context is initialized.
# _Create a CQL command adapter._ Based on the commands annotations we can
create a new adapter that handles command parameters received from the binary
protocol, this is basically the same as new dynamic MBeans with the caveat that
the binary protocol have to be improved to support new operation types.
Options 2 and 3 can be implemented independently.
was (Author: mmuzaf):
I've spent some time prototyping the changes and the plan may be the following,
splitting by the issues:
# _Replace the airline with the picocli._ CASSANDRA-17445 The airlift library
is now used to expose the management commands via the CLI. We can reuse this
approach to keep the number of changes low, however, we need to replace the
library with the picocli as the airlift has drawbacks. It prevents us from
building JMX and CQL APIs in follows:
## The airlift library is no longer supported;
## The library handles the `[ ...]` as a single input
argument, but we need to handle it independently as two input arguments
`keyspace` and `tables` to build JMX and CQL APIs;
## All commands right now have host, port, username, and password options etc.
which are not relevant to other types of interfaces;
# {_}Create an adapter that builds dynamic JMX MBeans{_}, and parses the
picocli annotations of the commands we already have. This will expose these
commands with the same parameters and options that they have in the CLI,
matching the nodetool CLI input. The management MBeans of the internal
components e.g. StorageServiceMBean can be injected into the commands directly
when the command context is initialized.
# _Create a CQL command adapter._ Based on the commands annotations we can
create a new adapter that handles command parameters received from the binary
protocol, this is basically the same as new dynamic MBeans with the caveat that
the binary protocol have to be improved to support new operation types.
Options 2 and 3 can be implemented independently.
> CQL Management API
> --
>
> Key: CASSANDRA-19476
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19476
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL/Interpreter, CQL/Syntax, Tool/nodetool
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: High
> Labels: CEP-38
> Fix For: 5.x
>
>
> We want to run management commands via CQL.
> The goals are:
> * To provide a way to run predefined management commands via CQL;
> * To provide a mechanism for retrieving command definitions and metadata via
> CQL;
> * To provide information on all available management commands via virtual
> tables;
> * To provide a registry that stores all C* commands and their metadata
> accordingly;
> * To internal instrumentation and a reasonable plan for migrating cluster
> management from JMX to CQL, taking into account backward compatibility and
> adopted deprecation policies;
> The discussion on the ML:
> https://lists.apache.org/thread/pow83q92m666nqtwyw4m3b18nnkgj2y8
> The design document:
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-38%3A+CQL+Management+API
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe,
[jira] [Commented] (CASSANDRA-19130) Implement transactional table truncation
[
https://issues.apache.org/jira/browse/CASSANDRA-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837866#comment-17837866
]
Abe Ratnofsky commented on CASSANDRA-19130:
---
I actually wonder if we should make truncation "lazier": rather than removing
SSTables within TruncateVerbHandler (and keeping SSTables with maxDataAge >
truncatedAt), just record the agreed-upon truncation timestamp in TCM, then in
compaction remove entire SSTables with maxDataAge <= truncatedAt, and remove
all cells with earlier mutation timestamps. When we're handling a Truncation
transformation in TCM (via TableTruncationListener), kick off a local
truncation compaction. Once the compaction completes, we can purged the
truncatedAt marker in the system table.
On the read path, we'd also filter returned cells by mutation timestamps, and
exclude SSTables with maxDataAge <= truncatedAt.
The benefits of this are: clearer semantics (don't need to depend on flush /
snapshot timing), friendlier to the key and row cache (could purge only as
necessary). The main downside would be that truncate couldn't be used for
emergency disk reclaim, but I think that use-case belongs in nodetool not CQL
anyway.
> Implement transactional table truncation
>
>
> Key: CASSANDRA-19130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19130
> Project: Cassandra
> Issue Type: New Feature
> Components: Consistency/Coordination
>Reporter: Marcus Eriksson
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> TRUNCATE table should leverage cluster metadata to ensure consistent
> truncation timestamps across all replicas. The current implementation depends
> on all nodes being available, but this could be reimplemented as a
> {{Transformation}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19493) Optionally fail writes when SAI refuses to index a term value exceeding a configured maximum size
[
https://issues.apache.org/jira/browse/CASSANDRA-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837864#comment-17837864
]
Caleb Rackliffe commented on CASSANDRA-19493:
-
Trunk CI looks good. We're hitting CASSANDRA-19514, CASSANDRA-19101, and
CASSANDRA-19520 though.
[~bereng] [~smiklosovic] Anything else we need before committing?
> Optionally fail writes when SAI refuses to index a term value exceeding a
> configured maximum size
> -
>
> Key: CASSANDRA-19493
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19493
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Attachments: ci_summary-1.html, ci_summary.html
>
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> SAI currently emits a client warning when we try to index a text value larger
> than {{{}cassandra.sai.max_string_term_size{}}}. It might be nice to have a
> hard limit as well, above which we can reject the mutation entirely.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19493) Optionally fail writes when SAI refuses to index a term value exceeding a configured maximum size
[
https://issues.apache.org/jira/browse/CASSANDRA-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-19493:
Attachment: ci_summary-1.html
> Optionally fail writes when SAI refuses to index a term value exceeding a
> configured maximum size
> -
>
> Key: CASSANDRA-19493
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19493
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Attachments: ci_summary-1.html, ci_summary.html
>
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> SAI currently emits a client warning when we try to index a text value larger
> than {{{}cassandra.sai.max_string_term_size{}}}. It might be nice to have a
> hard limit as well, above which we can reject the mutation entirely.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19520) Test failure: upgrade_tests.py::upgrade_through_versions_test::test_bootstrap_multidc failing counter checks
[
https://issues.apache.org/jira/browse/CASSANDRA-19520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-19520:
Fix Version/s: 5.1-alpha
> Test failure:
> upgrade_tests.py::upgrade_through_versions_test::test_bootstrap_multidc
> failing counter checks
>
>
> Key: CASSANDRA-19520
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19520
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Bootstrap and Decommission,
> Feature/Counters, Test/dtest/python
>Reporter: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.1-alpha
>
> Attachments: TESTS-TestSuites.xml, ci_summary.html
>
>
> This appears to be a new failure in {{{}test_bootstrap_multidc{}}}, around
> {{upgrade_scenario}} verifying counter updates. I can't reproduce locally, so
> perhaps there is a counter race condition lurking somewhere...
> {noformat}
> assert 249 == 248 +249 -248
> self =
> object at 0x7fbae9b96550>
> def test_bootstrap_multidc(self):
> # try and add a new node
> # multi dc, 2 nodes in each dc
> self.prepare()
> cluster = self.cluster
> if cluster.version() >= '5.0':
> cluster.set_configuration_options({'user_defined_functions_threads_enabled':
> 'true',
> 'scripted_user_defined_functions_enabled': 'false'})
> elif cluster.version() >= '3.0':
> cluster.set_configuration_options({'enable_user_defined_functions': 'true',
> 'enable_scripted_user_defined_functions': 'true'})
> elif cluster.version() >= '2.2':
> cluster.set_configuration_options({'enable_user_defined_functions': 'true'})
> cluster.populate([2, 2])
> [node.start(use_jna=True, wait_for_binary_proto=True) for node in
> self.cluster.nodelist()]
> self._multidc_schema_create()
> > self.upgrade_scenario(populate=False, create_schema=False,
> > after_upgrade_call=(self._bootstrap_new_node_multidc,))
> upgrade_tests/upgrade_through_versions_test.py:865:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> upgrade_tests/upgrade_through_versions_test.py:506: in upgrade_scenario
> call()
> upgrade_tests/upgrade_through_versions_test.py:841: in
> _bootstrap_new_node_multidc
> self._check_counters()
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> self =
> object at 0x7fbae9b96550>
> def _check_counters(self):
> logger.debug("Checking counter values...")
> session = self.patient_cql_connection(self.node2,
> protocol_version=self.protocol_version)
> session.execute("use upgrade;")
> for key1 in list(self.expected_counts.keys()):
> for key2 in list(self.expected_counts[key1].keys()):
> expected_value = self.expected_counts[key1][key2]
> query = SimpleStatement("SELECT c from countertable where k1='{key1}' and
> k2={key2};".format(key1=key1, key2=key2),
> consistency_level=ConsistencyLevel.ONE)
> results = session.execute(query)
> if results is not None:
> actual_value = results[0][0]
> else:
> # counter wasn't found
> actual_value = None
> > assert actual_value == expected_value
> E assert 249 == 248
> E +249
> E -248
> upgrade_tests/upgrade_through_versions_test.py:789: AssertionError
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19130) Implement transactional table truncation
[
https://issues.apache.org/jira/browse/CASSANDRA-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837854#comment-17837854
]
Abe Ratnofsky commented on CASSANDRA-19130:
---
> What happens with the truncation when there is a mixed cluster in place?
I would think that we should continue handle TruncateStatement in the pre-TCM
way, even when coordinated by an instance running with TCM, until the entire
cluster is known to be on TCM. Specifically, that would mean checking that all
instances are available, sending TRUNCATE_REQ to all peers and waiting for
responses, etc. Once the entire cluster is on TCM, we stop sending TRUNCATE_REQ
and ignore received TRUNCATE_REQ[1], and from then onwards all execution of
truncate happens via the TCM log listener.
[1]: We should warn if we receive a TRUNCATE_REQ after the entire cluster is on
TCM, since it should only be possible to receive one if a strange message
delivery happens from a pre-upgraded instance
> Implement transactional table truncation
>
>
> Key: CASSANDRA-19130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19130
> Project: Cassandra
> Issue Type: New Feature
> Components: Consistency/Coordination
>Reporter: Marcus Eriksson
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> TRUNCATE table should leverage cluster metadata to ensure consistent
> truncation timestamps across all replicas. The current implementation depends
> on all nodes being available, but this could be reimplemented as a
> {{Transformation}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19563) [Analytics] Support bulk write via S3
[ https://issues.apache.org/jira/browse/CASSANDRA-19563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRA-19563: -- Test and Documentation Plan: ci; unit + integration Status: Patch Available (was: Open) PR: [https://github.com/apache/cassandra-analytics/pull/53] CI: [https://app.circleci.com/pipelines/github/yifan-c/cassandra-analytics?branch=CASSANDRA-19563%2Ftrunk] > [Analytics] Support bulk write via S3 > - > > Key: CASSANDRA-19563 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19563 > Project: Cassandra > Issue Type: New Feature > Components: Analytics Library >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Time Spent: 10m > Remaining Estimate: 0h > > I would like to propose a new write option in Cassandra Analytics to bulk > write SSTables via S3, in addition to the previously-implemented "direct > upload to all sidecars" (now known as the "Direct" transport). > The new write option, now being implemented, is the "S3_COMPAT" transport, > which allows the job to upload the generated SSTables to an S3-compatible > storage system, and then inform the Cassandra Sidecar that those files are > available for download & commit. > Additionally, a plug-in system was added to allow communications between > custom transport hooks and the job, so the custom hook can provide updated > credentials and out-of-band status updates on S3-related issues. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[PR] CASSANDRA-19563: Support bulk write via S3 [cassandra-analytics]
yifan-c opened a new pull request, #53: URL: https://github.com/apache/cassandra-analytics/pull/53 This commit adds a configuration (writer) option to pick a transport other than the previously-implemented "direct upload to all sidecars" (now known as the "Direct" transport). The second transport, now being implemented, is the "S3_COMPAT" transport, which allows the job to upload the generated SSTables to an S3-compatible storage system, and then inform the Cassandra Sidecar that those files are available for download & commit. Additionally, a plug-in system was added to allow communications between custom transport hooks and the job, so the custom hook can provide updated credentials and out-of-band status updates on S3-related issues. Co-Authored-By: Yifan Cai Co-Authored-By: Doug Rohrer Co-Authored-By: Francisco Guerrero Co-Authored-By: Saranya Krishnakumar Patch by Yifan Cai, Doug Rohrer, Francisco Guerrero, Saranya Krishnakumar; Reviewed by TBD for CASSANDRA-19563 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19553) get/set guardrails configuration via CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-19553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-19553:
--
Status: Needs Committer (was: Patch Available)
> get/set guardrails configuration via CQL
>
>
> Key: CASSANDRA-19553
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19553
> Project: Cassandra
> Issue Type: New Feature
> Components: Feature/Virtual Tables
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> In CASSANDRA-19552, there is a patch which modifies guardrails via nodetool
> and via reflection.
> I think that it will be better if we do that via CQL.
> So far, I have this:
> {code}
> cqlsh> select * from system_views.guardrails_threshold ;
> name | fail | warn
> ---+---+---
>collection_size | 0 | 0
> column_value_size |-1 |-1
> columns_per_table |-1 |-1
> fields_per_udt |-1 |-1
>in_select_cartesian_product |-1 |-1
> items_per_collection |-1 |-1
> keyspaces |-1 |-1
> materialized_views_per_table |-1 |-1
> maximum_replication_factor |-1 |-1
> maximum_timestamp | 0 | 0
> minimum_replication_factor |-1 |-1
> minimum_timestamp | 0 | 0
> page_size |-1 |-1
> partition_keys_in_select |-1 |-1
> partition_size | 40960 | 20480
> partition_tombstones |-1 |-1
> sai_sstable_indexes_per_query |-1 |32
>secondary_indexes_per_table |-1 |-1
> tables |-1 |-1
> vector_dimensions |-1 |-1
> {code}
> {code}
> cqlsh> update system_views.guardrails_threshold SET warn = 16, fail = 20
> where name = 'sai_sstable_indexes_per_query';
> {code}
> {code}
> cqlsh> select * from system_views.guardrails_threshold where name =
> 'sai_sstable_indexes_per_query';
> name | fail | warn
> ---+--+--
> sai_sstable_indexes_per_query | 20 | 16
> {code}
> {code}
> INFO [Native-Transport-Requests-1] 2024-04-11 11:51:24,483
> GuardrailsOptions.java:998 - Updated
> sai_sstable_indexes_per_query_warn_threshold from 32 to 16
> INFO [Native-Transport-Requests-1] 2024-04-11 11:51:24,483
> GuardrailsOptions.java:998 - Updated
> sai_sstable_indexes_per_query_fail_threshold from -1 to 20
> {code}
> {code}
> cqlsh> update system_views.guardrails_threshold SET warn = 10, fail = 5 where
> name = 'sai_sstable_indexes_per_query';
> InvalidRequest: Error from server: code=2200 [Invalid query] message="The
> warn threshold 10 for sai_sstable_indexes_per_query_warn_threshold should be
> lower than the fail threshold 5"
> {code}
> We can easily extend this to EnableFlag guardrails for which we would
> dedicate a separate table (system_views.guadrails_flag).
> Other guardrail types are being investigated.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-builds) branch trunk updated: Update ccm versions in docker
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
The following commit(s) were added to refs/heads/trunk by this push:
new 3a843c5 Update ccm versions in docker
3a843c5 is described below
commit 3a843c54d1950adf12dc21a12eefe3f050a25ff0
Author: Brandon Williams
AuthorDate: Tue Apr 16 13:59:46 2024 -0500
Update ccm versions in docker
---
docker/testing/ubuntu2004_j11_w_dependencies.docker | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/docker/testing/ubuntu2004_j11_w_dependencies.docker
b/docker/testing/ubuntu2004_j11_w_dependencies.docker
index 0e692a3..cee2633 100644
--- a/docker/testing/ubuntu2004_j11_w_dependencies.docker
+++ b/docker/testing/ubuntu2004_j11_w_dependencies.docker
@@ -63,8 +63,8 @@ RUN rm -fr ~/cassandra
RUN bash -c 'source ~/env3.6/bin/activate && \
for i in {1..22} ; do echo $i ; ccm create --quiet -n 1 -v binary:2.1.$i
test && ccm remove test ; done && \
for i in {1..19} ; do echo $i ; ccm create --quiet -n 1 -v binary:2.2.$i
test && ccm remove test ; done && \
-for i in {1..29} ; do echo $i ; ccm create --quiet -n 1 -v binary:3.0.$i
test && ccm remove test ; done && \
-for i in {1..15} ; do echo $i ; ccm create --quiet -n 1 -v binary:3.11.$i
test && ccm remove test ; done && \
+for i in {1..30} ; do echo $i ; ccm create --quiet -n 1 -v binary:3.0.$i
test && ccm remove test ; done && \
+for i in {1..17} ; do echo $i ; ccm create --quiet -n 1 -v binary:3.11.$i
test && ccm remove test ; done && \
for i in {1..11} ; do echo $i ; ccm create --quiet -n 1 -v binary:4.0.$i
test && ccm remove test ; done && \
-for i in {1..3} ; do echo $i ; ccm create --quiet -n 1 -v binary:4.1.$i
test && ccm remove test ; done'
+for i in {1..4} ; do echo $i ; ccm create --quiet -n 1 -v binary:4.1.$i
test && ccm remove test ; done'
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-dtest) branch trunk updated: Increment version for 3.11
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git The following commit(s) were added to refs/heads/trunk by this push: new d51d4e1b Increment version for 3.11 d51d4e1b is described below commit d51d4e1b5436d744b196f189445a52b258a6cbb8 Author: Brandon Williams AuthorDate: Tue Apr 16 13:29:31 2024 -0500 Increment version for 3.11 --- upgrade_tests/upgrade_manifest.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/upgrade_tests/upgrade_manifest.py b/upgrade_tests/upgrade_manifest.py index 760255ed..8a83173b 100644 --- a/upgrade_tests/upgrade_manifest.py +++ b/upgrade_tests/upgrade_manifest.py @@ -166,7 +166,7 @@ indev_3_0_x = VersionMeta(name='indev_3_0_x', family=CASSANDRA_3_0, variant='ind current_3_0_x = VersionMeta(name='current_3_0_x', family=CASSANDRA_3_0, variant='current', version='3.0.30', min_proto_v=3, max_proto_v=4, java_versions=(8,)) indev_3_11_x = VersionMeta(name='indev_3_11_x', family=CASSANDRA_3_11, variant='indev', version='github:apache/cassandra-3.11', min_proto_v=3, max_proto_v=4, java_versions=(8,)) -current_3_11_x = VersionMeta(name='current_3_11_x', family=CASSANDRA_3_11, variant='current', version='3.11.16', min_proto_v=3, max_proto_v=4, java_versions=(8,)) +current_3_11_x = VersionMeta(name='current_3_11_x', family=CASSANDRA_3_11, variant='current', version='3.11.17', min_proto_v=3, max_proto_v=4, java_versions=(8,)) indev_4_0_x = VersionMeta(name='indev_4_0_x', family=CASSANDRA_4_0, variant='indev', version='github:apache/cassandra-4.0', min_proto_v=3, max_proto_v=4, java_versions=(8,11)) current_4_0_x = VersionMeta(name='current_4_0_x', family=CASSANDRA_4_0, variant='current', version='4.0.12', min_proto_v=4, max_proto_v=5, java_versions=(8,11)) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-website) branch asf-site updated (d21cd2d6 -> 25cd5019)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard d21cd2d6 generate docs for 3774363b add 607350d3 Minor release 3.0.30 add 1d4732f3 Minor release 3.11.17 add 25cd5019 generate docs for 1d4732f3 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (d21cd2d6) \ N -- N -- N refs/heads/asf-site (25cd5019) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: content/_/download.html| 16 +- .../cassandra/configuration/cass_yaml_file.html| 16 + .../cassandra/configuration/cass_yaml_file.html| 16 + .../cassandra/configuration/cass_yaml_file.html| 16 + .../cassandra/configuration/cass_yaml_file.html| 16 + .../cassandra/configuration/cass_yaml_file.html| 16 + .../cassandra/configuration/cass_yaml_file.html| 16 + .../cassandra/configuration/cass_yaml_file.html| 16 + .../cassandra/configuration/cass_yaml_file.html| 16 + .../cassandra/configuration/cass_yaml_file.html| 16 + .../cassandra/configuration/cass_yaml_file.html| 16 + .../cassandra/configuration/cass_yaml_file.html| 16 + .../cassandra/configuration/cass_yaml_file.html| 16 + .../cassandra/configuration/cass_yaml_file.html| 16 + .../cassandra/configuration/cass_yaml_file.html| 16 + .../cassandra/configuration/cass_yaml_file.html| 16 + .../cassandra/configuration/cass_yaml_file.html| 16 + .../cassandra/configuration/cass_yaml_file.html| 16 + .../cassandra/configuration/cass_yaml_file.html| 16 + .../doc/5.1/cassandra/developing/cql/triggers.html |5 + .../managing/configuration/cass_yaml_file.html | 14 + .../5.1/cassandra/managing/tools/nodetool/cms.html | 10 +- .../managing/tools/nodetool/nodetool.html |4 - .../managing/tools/nodetool/sealperiod.html| 1112 .../cassandra/configuration/cass_yaml_file.html| 16 + .../trunk/cassandra/developing/cql/triggers.html |5 + .../managing/configuration/cass_yaml_file.html | 14 + .../cassandra/managing/tools/nodetool/cms.html | 10 +- .../managing/tools/nodetool/nodetool.html |4 - .../managing/tools/nodetool/sealperiod.html| 1112 content/search-index.js|2 +- .../source/modules/ROOT/pages/download.adoc| 16 +- site-ui/build/ui-bundle.zip| Bin 4883646 -> 4883646 bytes 33 files changed, 377 insertions(+), 2251 deletions(-) delete mode 100644 content/doc/5.1/cassandra/managing/tools/nodetool/sealperiod.html delete mode 100644 content/doc/trunk/cassandra/managing/tools/nodetool/sealperiod.html - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-website) branch asf-staging updated (1e32839c -> 25cd5019)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard 1e32839c generate docs for 1d4732f3 new 25cd5019 generate docs for 1d4732f3 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (1e32839c) \ N -- N -- N refs/heads/asf-staging (25cd5019) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: site-ui/build/ui-bundle.zip | Bin 4883646 -> 4883646 bytes 1 file changed, 0 insertions(+), 0 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14572) Expose all table metrics in virtual table
[
https://issues.apache.org/jira/browse/CASSANDRA-14572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837832#comment-17837832
]
Caleb Rackliffe commented on CASSANDRA-14572:
-
Hi [~mmuzaf].
After merging this to trunk, I've noticed a bit of an issue w/
{{o.a.c.distributed.test.ReadRepairTest}} failing sporadically due to OOM in my
CI environment, and looking at the heap situation, the bulk of it is references
to {{CassandraMetricsRegistry}}. I am able to stabilize things by just giving
the tests more heap or by reverting this patch entirely, but I'd also like to
understand, if possible, the impact we expect this patch to have on persistent
heap usage.
Thanks!
P.S. If I were to experiment with something like a feature flag for my own
testing, what would be the easiest place to insert that?
> Expose all table metrics in virtual table
> -
>
> Key: CASSANDRA-14572
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14572
> Project: Cassandra
> Issue Type: New Feature
> Components: Legacy/Observability, Observability/Metrics
>Reporter: Chris Lohfink
>Assignee: Maxim Muzafarov
>Priority: Low
> Labels: virtual-tables
> Fix For: 5.1
>
> Attachments: flight_recording_1270017199_13.jfr, keyspayces_group
> responses times.png, keyspayces_group summary.png, select keyspaces_group by
> string prefix.png, select keyspaces_group compare with wo.png, select
> keyspaces_group without value.png, systemv_views.metrics_dropped_message.png,
> thread_pools benchmark.png
>
> Time Spent: 7h 50m
> Remaining Estimate: 0h
>
> While we want a number of virtual tables to display data in a way thats great
> and intuitive like in nodetool. There is also much for being able to expose
> the metrics we have for tooling via CQL instead of JMX. This is more for the
> tooling and adhoc advanced users who know exactly what they are looking for.
> *Schema:*
> Initial idea is to expose data via {{((keyspace, table), metric)}} with a
> column for each metric value. Could also use a Map or UDT instead of the
> column based that can be a bit more specific to each metric type. To that end
> there can be a {{metric_type}} column and then a UDT for each metric type
> filled in, or a single value with more of a Map style. I am
> purposing the column type though as with {{ALLOW FILTERING}} it does allow
> more extensive query capabilities.
> *Implementations:*
> * Use reflection to grab all the metrics from TableMetrics (see:
> CASSANDRA-7622 impl). This is easiest and least abrasive towards new metric
> implementors... but its reflection and a kinda a bad idea.
> * Add a hook in TableMetrics to register with this virtual table when
> registering
> * Pull from the CassandraMetrics registery (either reporter or iterate
> through metrics query on read of virtual table)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19563) [Analytics] Support bulk write via S3
[ https://issues.apache.org/jira/browse/CASSANDRA-19563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRA-19563: -- Change Category: Semantic Complexity: Normal Component/s: Analytics Library Status: Open (was: Triage Needed) > [Analytics] Support bulk write via S3 > - > > Key: CASSANDRA-19563 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19563 > Project: Cassandra > Issue Type: New Feature > Components: Analytics Library >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > > I would like to propose a new write option in Cassandra Analytics to bulk > write SSTables via S3, in addition to the previously-implemented "direct > upload to all sidecars" (now known as the "Direct" transport). > The new write option, now being implemented, is the "S3_COMPAT" transport, > which allows the job to upload the generated SSTables to an S3-compatible > storage system, and then inform the Cassandra Sidecar that those files are > available for download & commit. > Additionally, a plug-in system was added to allow communications between > custom transport hooks and the job, so the custom hook can provide updated > credentials and out-of-band status updates on S3-related issues. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-website) branch asf-staging updated (8df8dc70 -> 1e32839c)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard 8df8dc70 generate docs for 607350d3 add 1d4732f3 Minor release 3.11.17 new 1e32839c generate docs for 1d4732f3 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (8df8dc70) \ N -- N -- N refs/heads/asf-staging (1e32839c) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: content/_/download.html| 8 .../doc/5.1/cassandra/developing/cql/triggers.html | 5 + .../managing/configuration/cass_yaml_file.html | 14 ++ .../trunk/cassandra/developing/cql/triggers.html | 5 + .../managing/configuration/cass_yaml_file.html | 14 ++ content/search-index.js| 2 +- .../source/modules/ROOT/pages/download.adoc| 8 site-ui/build/ui-bundle.zip| Bin 4883646 -> 4883646 bytes 8 files changed, 47 insertions(+), 9 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19563) [Analytics] Support bulk write via S3
Yifan Cai created CASSANDRA-19563: - Summary: [Analytics] Support bulk write via S3 Key: CASSANDRA-19563 URL: https://issues.apache.org/jira/browse/CASSANDRA-19563 Project: Cassandra Issue Type: New Feature Reporter: Yifan Cai Assignee: Yifan Cai I would like to propose a new write option in Cassandra Analytics to bulk write SSTables via S3, in addition to the previously-implemented "direct upload to all sidecars" (now known as the "Direct" transport). The new write option, now being implemented, is the "S3_COMPAT" transport, which allows the job to upload the generated SSTables to an S3-compatible storage system, and then inform the Cassandra Sidecar that those files are available for download & commit. Additionally, a plug-in system was added to allow communications between custom transport hooks and the job, so the custom hook can provide updated credentials and out-of-band status updates on S3-related issues. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) 01/01: Merge branch 'cassandra-3.11' into cassandra-4.0
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch cassandra-4.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 0662094b84c13db8046ef51964551f2e0288dfcd Merge: 53f07f9610 2856a34636 Author: Brandon Williams AuthorDate: Tue Apr 16 13:27:12 2024 -0500 Merge branch 'cassandra-3.11' into cassandra-4.0 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) 01/01: Merge branch 'cassandra-4.1' into cassandra-5.0
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch cassandra-5.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 0c34234f67edc954695237df6c79295a017551a2 Merge: e127ddf7dd 643d7f1751 Author: Brandon Williams AuthorDate: Tue Apr 16 13:27:24 2024 -0500 Merge branch 'cassandra-4.1' into cassandra-5.0 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch cassandra-5.0 updated (e127ddf7dd -> 0c34234f67)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch cassandra-5.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from e127ddf7dd Merge branch 'cassandra-4.1' into cassandra-5.0 new 2856a34636 Increment version new 0662094b84 Merge branch 'cassandra-3.11' into cassandra-4.0 new 643d7f1751 Merge branch 'cassandra-4.0' into cassandra-4.1 new 0c34234f67 Merge branch 'cassandra-4.1' into cassandra-5.0 The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) 01/01: Merge branch 'cassandra-4.0' into cassandra-4.1
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch cassandra-4.1 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 643d7f1751fcac3f732301f87fe6b7e7d1a9d6b7 Merge: a6209d0cfc 0662094b84 Author: Brandon Williams AuthorDate: Tue Apr 16 13:27:18 2024 -0500 Merge branch 'cassandra-4.0' into cassandra-4.1 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch cassandra-3.11 updated: Increment version
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/cassandra-3.11 by this push: new 2856a34636 Increment version 2856a34636 is described below commit 2856a34636f66747eae514e131b6a00f5f998b6d Author: Brandon Williams AuthorDate: Tue Apr 16 13:26:59 2024 -0500 Increment version --- CHANGES.txt | 4 build.xml | 2 +- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/CHANGES.txt b/CHANGES.txt index 7227024396..886bf069de 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,3 +1,7 @@ +3.11.18 + * + + 3.11.17 * Move ClientWarn.State#warnings to a thread-safe list (CASSANDRA-19427) * Fix delayed SSTable release with unsafe_aggressive_sstable_expiration (CASSANDRA-18756) diff --git a/build.xml b/build.xml index 22ddfba959..ab94b82302 100644 --- a/build.xml +++ b/build.xml @@ -34,7 +34,7 @@ - + https://gitbox.apache.org/repos/asf/cassandra.git"/> https://gitbox.apache.org/repos/asf/cassandra.git"/> https://gitbox.apache.org/repos/asf?p=cassandra.git"/> - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch cassandra-4.0 updated (53f07f9610 -> 0662094b84)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch cassandra-4.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 53f07f9610 Merge branch 'cassandra-3.11' into cassandra-4.0 new 2856a34636 Increment version new 0662094b84 Merge branch 'cassandra-3.11' into cassandra-4.0 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) 01/01: Merge branch 'cassandra-5.0' into trunk
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit f194c0772c8fd4250faa0b300bf5f59be17c6a83 Merge: 8d705b31e9 0c34234f67 Author: Brandon Williams AuthorDate: Tue Apr 16 13:27:33 2024 -0500 Merge branch 'cassandra-5.0' into trunk - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch trunk updated (8d705b31e9 -> f194c0772c)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from 8d705b31e9 Add new TriggersPolicy configuration to allow operators to disable triggers new 2856a34636 Increment version new 0662094b84 Merge branch 'cassandra-3.11' into cassandra-4.0 new 643d7f1751 Merge branch 'cassandra-4.0' into cassandra-4.1 new 0c34234f67 Merge branch 'cassandra-4.1' into cassandra-5.0 new f194c0772c Merge branch 'cassandra-5.0' into trunk The 5 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch cassandra-4.1 updated (a6209d0cfc -> 643d7f1751)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch cassandra-4.1 in repository https://gitbox.apache.org/repos/asf/cassandra.git from a6209d0cfc Merge branch 'cassandra-4.0' into cassandra-4.1 new 2856a34636 Increment version new 0662094b84 Merge branch 'cassandra-3.11' into cassandra-4.0 new 643d7f1751 Merge branch 'cassandra-4.0' into cassandra-4.1 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-website) branch trunk updated: Minor release 3.11.17
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-website.git The following commit(s) were added to refs/heads/trunk by this push: new 1d4732f3 Minor release 3.11.17 1d4732f3 is described below commit 1d4732f3bcbba8d0b217880cb7de307a307bc8c6 Author: Brandon Williams AuthorDate: Tue Apr 16 13:14:36 2024 -0500 Minor release 3.11.17 https://lists.apache.org/thread/56dj6zzzv7pxmgsr2p6v5h19yo4r56j1 --- site-content/source/modules/ROOT/pages/download.adoc | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/site-content/source/modules/ROOT/pages/download.adoc b/site-content/source/modules/ROOT/pages/download.adoc index ad04d53d..ac225060 100644 --- a/site-content/source/modules/ROOT/pages/download.adoc +++ b/site-content/source/modules/ROOT/pages/download.adoc @@ -81,15 +81,15 @@ https://www.apache.org/dyn/closer.lua/cassandra/4.0.12/apache-cassandra-4.0.12-b [discrete] Apache Cassandra 3.11 [discrete] - Latest release on 2023-08-20 + Latest release on 2023-04-16 [discrete] Maintained until 5.0.0 release (Nov-Dec 2023) [.btn.btn--alt] -https://www.apache.org/dyn/closer.lua/cassandra/3.11.16/apache-cassandra-3.11.16-bin.tar.gz[3.11.16,window=blank] +https://www.apache.org/dyn/closer.lua/cassandra/3.11.17/apache-cassandra-3.11.17-bin.tar.gz[3.11.17,window=blank] -(https://downloads.apache.org/cassandra/3.11.16/apache-cassandra-3.11.16-bin.tar.gz.asc[pgp,window=blank], https://downloads.apache.org/cassandra/3.11.16/apache-cassandra-3.11.16-bin.tar.gz.sha256[sha256,window=blank], https://downloads.apache.org/cassandra/3.11.16/apache-cassandra-3.11.16-bin.tar.gz.sha512[sha512,window=blank]) + -(https://www.apache.org/dyn/closer.lua/cassandra/3.11.16/apache-cassandra-3.11.16-src.tar.gz[source,window=blank]: https://downloads.apache.org/cassandra/3.11.16/apache-cassandra-3.11.16-bin.tar.gz.asc[pgp,window=blank], https://downloads.apache.org/cassandra/3.11.16/apache-cassandra-3.11.16-bin.tar.gz.sha256[sha256,window=blank], https://downloads.apache.org/cassandra/3.11.16/apache-cassandra-3.11.16-bin.tar.gz.sha512[sha512,window=blank]) +(https://downloads.apache.org/cassandra/3.11.17/apache-cassandra-3.11.17-bin.tar.gz.asc[pgp,window=blank], https://downloads.apache.org/cassandra/3.11.17/apache-cassandra-3.11.17-bin.tar.gz.sha256[sha256,window=blank], https://downloads.apache.org/cassandra/3.11.17/apache-cassandra-3.11.17-bin.tar.gz.sha512[sha512,window=blank]) + +(https://www.apache.org/dyn/closer.lua/cassandra/3.11.17/apache-cassandra-3.11.17-src.tar.gz[source,window=blank]: https://downloads.apache.org/cassandra/3.11.17/apache-cassandra-3.11.17-bin.tar.gz.asc[pgp,window=blank], https://downloads.apache.org/cassandra/3.11.17/apache-cassandra-3.11.17-bin.tar.gz.sha256[sha256,window=blank], https://downloads.apache.org/cassandra/3.11.17/apache-cassandra-3.11.17-bin.tar.gz.sha512[sha512,window=blank]) -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19493) Optionally fail writes when SAI refuses to index a term value exceeding a configured maximum size
[
https://issues.apache.org/jira/browse/CASSANDRA-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837814#comment-17837814
]
Caleb Rackliffe edited comment on CASSANDRA-19493 at 4/16/24 6:11 PM:
--
trunk PR: https://github.com/apache/cassandra/pull/3256 (CI in progress...)
It merged cleanly up, w/ the exception of some import ordering stuff and the
CHANGES file.
was (Author: maedhroz):
trunk PR: https://github.com/apache/cassandra/pull/3256 (CI in progress...)
> Optionally fail writes when SAI refuses to index a term value exceeding a
> configured maximum size
> -
>
> Key: CASSANDRA-19493
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19493
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Attachments: ci_summary.html
>
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> SAI currently emits a client warning when we try to index a text value larger
> than {{{}cassandra.sai.max_string_term_size{}}}. It might be nice to have a
> hard limit as well, above which we can reject the mutation entirely.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19493) Optionally fail writes when SAI refuses to index a term value exceeding a configured maximum size
[
https://issues.apache.org/jira/browse/CASSANDRA-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837814#comment-17837814
]
Caleb Rackliffe commented on CASSANDRA-19493:
-
trunk PR: https://github.com/apache/cassandra/pull/3256 (CI in progress...)
> Optionally fail writes when SAI refuses to index a term value exceeding a
> configured maximum size
> -
>
> Key: CASSANDRA-19493
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19493
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Attachments: ci_summary.html
>
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> SAI currently emits a client warning when we try to index a text value larger
> than {{{}cassandra.sai.max_string_term_size{}}}. It might be nice to have a
> hard limit as well, above which we can reject the mutation entirely.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19562) DefaultLoadBalancingPolicy considers a TimeOut as a response
Jane He created CASSANDRA-19562: --- Summary: DefaultLoadBalancingPolicy considers a TimeOut as a response Key: CASSANDRA-19562 URL: https://issues.apache.org/jira/browse/CASSANDRA-19562 Project: Cassandra Issue Type: Bug Components: Client/java-driver Reporter: Jane He Assignee: Henry Hughes At this line: [https://github.com/apache/cassandra-java-driver/blob/388a46b9c10b5653c71ac8840bcda0c91b59bce4/core/src/main/java/com/datastax/oss/driver/internal/core/loadbalancing/DefaultLoadBalancingPolicy.java#L255] It considers a timeout as a response, when it is counting whether there are 2 responses in the past 200ms to decide whether a node is slow and should be excluded. This will result in the driver continue sending requests to a node that is not responding and giving timeout exception. It should be fixed just by an if statement saying if the error is not DriverTimeoutException, then update response times. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19551) CCM nodes share the same environment variable map breaking upgrade tests
[
https://issues.apache.org/jira/browse/CASSANDRA-19551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837791#comment-17837791
]
Ariel Weisberg commented on CASSANDRA-19551:
TY!
> CCM nodes share the same environment variable map breaking upgrade tests
>
>
> Key: CASSANDRA-19551
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19551
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.31, 3.11.17, 4.0.13, 5.0-beta2, 5.1
>
> Attachments: ci_summary.html
>
>
> In {{node.py}} {{__environment_variables}} is generally always set with a map
> that is passed in from {{cluster.py}} so it is [shared between
> nodes|https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L151]
> and if nodes modify the map, such as in {{start}} when [updating the Java
> version|https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L860]
> then when {{get_env}} runs it will [overwrite the Java
> version|https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L244]
> that is selected by {{update_java_version}}.
> This results in {{nodetool drain}} failing when upgrading from 3.11 to 4.0 in
> some of the upgrade tests because after the first node upgrades to 4.0 it's
> not longer possible for the subsequent nodes to select a Java version that
> isn't 11 because it's overridden by {{__environment_variables}}.
> I'm not even 100% clear on why the code in {{start}} should update
> {{__environment_variables}} at all if we calculate the correct java version
> on every invocation of other tools.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19551) CCM nodes share the same environment variable map breaking upgrade tests
[
https://issues.apache.org/jira/browse/CASSANDRA-19551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19551:
-
Fix Version/s: (was: 5.x)
> CCM nodes share the same environment variable map breaking upgrade tests
>
>
> Key: CASSANDRA-19551
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19551
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.31, 3.11.17, 4.0.13, 5.0-beta2, 5.1
>
> Attachments: ci_summary.html
>
>
> In {{node.py}} {{__environment_variables}} is generally always set with a map
> that is passed in from {{cluster.py}} so it is [shared between
> nodes|https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L151]
> and if nodes modify the map, such as in {{start}} when [updating the Java
> version|https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L860]
> then when {{get_env}} runs it will [overwrite the Java
> version|https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L244]
> that is selected by {{update_java_version}}.
> This results in {{nodetool drain}} failing when upgrading from 3.11 to 4.0 in
> some of the upgrade tests because after the first node upgrades to 4.0 it's
> not longer possible for the subsequent nodes to select a Java version that
> isn't 11 because it's overridden by {{__environment_variables}}.
> I'm not even 100% clear on why the code in {{start}} should update
> {{__environment_variables}} at all if we calculate the correct java version
> on every invocation of other tools.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19551) CCM nodes share the same environment variable map breaking upgrade tests
[
https://issues.apache.org/jira/browse/CASSANDRA-19551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19551:
-
Fix Version/s: 3.0.31
3.11.17
4.0.13
5.0-beta2
5.1
Since Version: NA
Source Control Link:
https://github.com/riptano/ccm/commit/ce69e9df94f9e44f3f5b36f8ed4e2d07f734195f
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
I committed this and updated the cassandra-test tag.
> CCM nodes share the same environment variable map breaking upgrade tests
>
>
> Key: CASSANDRA-19551
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19551
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.31, 3.11.17, 4.0.13, 5.0-beta2, 5.x, 5.1
>
> Attachments: ci_summary.html
>
>
> In {{node.py}} {{__environment_variables}} is generally always set with a map
> that is passed in from {{cluster.py}} so it is [shared between
> nodes|https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L151]
> and if nodes modify the map, such as in {{start}} when [updating the Java
> version|https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L860]
> then when {{get_env}} runs it will [overwrite the Java
> version|https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L244]
> that is selected by {{update_java_version}}.
> This results in {{nodetool drain}} failing when upgrading from 3.11 to 4.0 in
> some of the upgrade tests because after the first node upgrades to 4.0 it's
> not longer possible for the subsequent nodes to select a Java version that
> isn't 11 because it's overridden by {{__environment_variables}}.
> I'm not even 100% clear on why the code in {{start}} should update
> {{__environment_variables}} at all if we calculate the correct java version
> on every invocation of other tools.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19551) CCM nodes share the same environment variable map breaking upgrade tests
[
https://issues.apache.org/jira/browse/CASSANDRA-19551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-19551:
---
Description:
In {{node.py}} {{__environment_variables}} is generally always set with a map
that is passed in from {{cluster.py}} so it is [shared between
nodes|https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L151]
and if nodes modify the map, such as in {{start}} when [updating the Java
version|https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L860]
then when {{get_env}} runs it will [overwrite the Java
version|https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L244]
that is selected by {{update_java_version}}.
This results in {{nodetool drain}} failing when upgrading from 3.11 to 4.0 in
some of the upgrade tests because after the first node upgrades to 4.0 it's not
longer possible for the subsequent nodes to select a Java version that isn't 11
because it's overridden by {{__environment_variables}}.
I'm not even 100% clear on why the code in {{start}} should update
{{__environment_variables}} at all if we calculate the correct java version on
every invocation of other tools.
was:
In {{node.py}} {{__environment_variables}} is generally always set with a map
that is passed in from {{cluster.py}} so it is [shared between
nodes](https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L151)
and if nodes modify the map, such as in {{start}} when [updating the Java
version](https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L860)
then when {{get_env}} runs it will [overwrite the Java
version](https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L244)
that is selected by {{update_java_version}}.
This results in {{nodetool drain}} failing when upgrading from 3.11 to 4.0 in
some of the upgrade tests because after the first node upgrades to 4.0 it's not
longer possible for the subsequent nodes to select a Java version that isn't 11
because it's overridden by {{__environment_variables}}.
I'm not even 100% clear on why the code in {{start}} should update
{{__environment_variables}} at all if we calculate the correct java version on
every invocation of other tools.
> CCM nodes share the same environment variable map breaking upgrade tests
>
>
> Key: CASSANDRA-19551
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19551
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 5.x
>
> Attachments: ci_summary.html
>
>
> In {{node.py}} {{__environment_variables}} is generally always set with a map
> that is passed in from {{cluster.py}} so it is [shared between
> nodes|https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L151]
> and if nodes modify the map, such as in {{start}} when [updating the Java
> version|https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L860]
> then when {{get_env}} runs it will [overwrite the Java
> version|https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L244]
> that is selected by {{update_java_version}}.
> This results in {{nodetool drain}} failing when upgrading from 3.11 to 4.0 in
> some of the upgrade tests because after the first node upgrades to 4.0 it's
> not longer possible for the subsequent nodes to select a Java version that
> isn't 11 because it's overridden by {{__environment_variables}}.
> I'm not even 100% clear on why the code in {{start}} should update
> {{__environment_variables}} at all if we calculate the correct java version
> on every invocation of other tools.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19429) Remove lock contention generated by getCapacity function in SSTableReader
[
https://issues.apache.org/jira/browse/CASSANDRA-19429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837784#comment-17837784
]
Dipietro Salvatore commented on CASSANDRA-19429:
[~smiklosovic] What if we provide the instance for few days? Would you be able
to test this?
> Remove lock contention generated by getCapacity function in SSTableReader
> -
>
> Key: CASSANDRA-19429
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19429
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Dipietro Salvatore
>Assignee: Dipietro Salvatore
>Priority: Normal
> Fix For: 4.0.x, 4.1.x
>
> Attachments: Screenshot 2024-02-26 at 10.27.10.png, Screenshot
> 2024-02-27 at 11.29.41.png, Screenshot 2024-03-19 at 15.22.50.png,
> asprof_cass4.1.3__lock_20240216052912lock.html,
> image-2024-03-08-15-51-30-439.png, image-2024-03-08-15-52-07-902.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Profiling Cassandra 4.1.3 on large AWS instances, a high number of lock
> acquires is measured in the `getCapacity` function from
> `org/apache/cassandra/cache/InstrumentingCache` (1.9M lock acquires per 60
> seconds). Based on our tests on r8g.24xlarge instances (using Ubuntu 22.04),
> this limits the CPU utilization of the system to under 50% when testing at
> full load and therefore limits the achieved throughput.
> Removing the lock contention from the SSTableReader.java file by replacing
> the call to `getCapacity` with `size` achieves up to 2.95x increase in
> throughput on r8g.24xlarge and 2x on r7i.24xlarge:
> |Instance type|Cass 4.1.3|Cass 4.1.3 patched|
> |r8g.24xlarge|168k ops|496k ops (2.95x)|
> |r7i.24xlarge|153k ops|304k ops (1.98x)|
>
> Instructions to reproduce:
> {code:java}
> ## Requirements for Ubuntu 22.04
> sudo apt install -y ant git openjdk-11-jdk
> ## Build and run
> CASSANDRA_USE_JDK11=true ant realclean && CASSANDRA_USE_JDK11=true ant jar &&
> CASSANDRA_USE_JDK11=true ant stress-build && rm -rf data && bin/cassandra -f
> -R
> # Run
> bin/cqlsh -e 'drop table if exists keyspace1.standard1;' && \
> bin/cqlsh -e 'drop keyspace if exists keyspace1;' && \
> bin/nodetool clearsnapshot --all && tools/bin/cassandra-stress write
> n=1000 cl=ONE -rate threads=384 -node 127.0.0.1 -log file=cload.log
> -graph file=cload.html && \
> bin/nodetool compact keyspace1 && sleep 30s && \
> tools/bin/cassandra-stress mixed ratio\(write=10,read=90\) duration=10m
> cl=ONE -rate threads=406 -node localhost -log file=result.log -graph
> file=graph.html
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-19101) Test Failure: org.apache.cassandra.db.commitlog.CommitlogShutdownTest failed on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-19101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh McKenzie reassigned CASSANDRA-19101:
-
Assignee: Josh McKenzie
> Test Failure: org.apache.cassandra.db.commitlog.CommitlogShutdownTest failed
> on trunk
> -
>
> Key: CASSANDRA-19101
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19101
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Jacek Lewandowski
>Assignee: Josh McKenzie
>Priority: Normal
> Fix For: 5.x
>
>
> {noformat}
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.cassandra.db.commitlog.CommitlogShutdownTest.testShutdownWithPendingTasks(CommitlogShutdownTest.java:96)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.jboss.byteman.contrib.bmunit.BMUnitRunner$10.evaluate(BMUnitRunner.java:393)
> at
> org.jboss.byteman.contrib.bmunit.BMUnitRunner$6.evaluate(BMUnitRunner.java:263)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.jboss.byteman.contrib.bmunit.BMUnitRunner$1.evaluate(BMUnitRunner.java:97)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
> at
> com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
> at
> com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
> at
> com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
> at
> com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
> at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)
> {noformat}
> Manual testing to confirm issues found by CircleCI when testing
> CASSANDRA-18464. Run with Java 11 / IntelliJ
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19221) CMS: Nodes can restart with new ipaddress already defined in the cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-19221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837763#comment-17837763
]
Alex Petrov commented on CASSANDRA-19221:
-
I've had a closer look at it, and wanted to mention that 5.0 behaviour is most
likely uninteded; it contains at least one bug, and is potentially dangeroud.
In short, my test was to spin up a 3 node cluster: {{127.0.0.1}},
{{127.0.0.2}}, {{127.0.0.3}}, and swap IP addresses for the two latter nodes
({{.2}} and {{.3}}. As a result of this test, nodes have in fact swapped their
IPs, but:
* if you would shut down {{.2}} and {{.3}}, and start {{.2}}, and then
{{.3}}, {{.3}} startup won't even begin because ccm considers its IP address to
be occupied, so an entire test can work only if you start the two nodes in
parallel
* after swapping ip addresses, ccm breaks, since it attempts to search {{UP}}
message for a specific IP address for a node, which it doesn't find if you
merely change the address in the conf file
* peers table for {{.2}} whose address is now {{.3}} will still have {{.3}}
in its peers table.
In general, since we are using ip addresses for node identity, I am weary of
allowing identity transfers for the occupied pars. By this I mean if {{ip <->
node id}} pair exists in the directory, we have to free up the IP address
before the other node can claim it. So the test would look as follows:
So for swapping {{.2}} and {{.3}}, one of the nodes would have to migrate to
{{.4}} first, and only then can the freed up IP address be occupied again.
Submitting a patch that fixes the peers table behaviour and codifies a
requirement of a separate node for swapping addresses.
> CMS: Nodes can restart with new ipaddress already defined in the cluster
>
>
> Key: CASSANDRA-19221
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19221
> Project: Cassandra
> Issue Type: Bug
> Components: Transactional Cluster Metadata
>Reporter: Paul Chandler
>Assignee: Alex Petrov
>Priority: Normal
> Fix For: 5.1-alpha1
>
>
> I am simulating running a cluster in Kubernetes and testing what happens when
> several pods go down and ip addresses are swapped between nodes. In 4.0 this
> is blocked and the node cannot be restarted.
> To simulate this I create a 3 node cluster on a local machine using 3
> loopback addresses
> {code}
> 127.0.0.1
> 127.0.0.2
> 127.0.0.3
> {code}
> The nodes are created correctly and the first node is assigned as a CMS node
> as shown:
> {code}
> bin/nodetool -p 7199 describecms
> {code}
> Cluster Metadata Service:
> {code}
> Members: /127.0.0.1:7000
> Is Member: true
> Service State: LOCAL
> {code}
> At this point I bring down the nodes 127.0.0.2 and 127.0.0.3 and swap the ip
> addresses for the rpc_address and listen_address
>
> The nodes come back as normal, but the nodeid has now been swapped against
> the ip address:
> Before:
> {code}
> Datacenter: datacenter1
> ===
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- Address Load Tokens Owns (effective) Host ID
> Rack
> UN 127.0.0.3 75.2 KiB 16 76.0%
> 6d194555-f6eb-41d0-c000-0003 rack1
> UN 127.0.0.2 86.77 KiB 16 59.3%
> 6d194555-f6eb-41d0-c000-0002 rack1
> UN 127.0.0.1 80.88 KiB 16 64.7%
> 6d194555-f6eb-41d0-c000-0001 rack1
> {code}
> After:
> {code}
> Datacenter: datacenter1
> ===
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- Address Load Tokens Owns (effective) Host ID
> Rack
> UN 127.0.0.3 149.62 KiB 16 76.0%
> 6d194555-f6eb-41d0-c000-0003 rack1
> UN 127.0.0.2 155.48 KiB 16 59.3%
> 6d194555-f6eb-41d0-c000-0002 rack1
> UN 127.0.0.1 75.74 KiB 16 64.7%
> 6d194555-f6eb-41d0-c000-0001 rack1
> {code}
> On previous tests of this I have created a table with a replication factor of
> 1, inserted some data before the swap. After the swap the data on nodes 2
> and 3 is now missing.
> One theory I have is that I am using different port numbers for the different
> nodes, and I am only swapping the ip addresses and not the port numbers, so
> the ip:port still looks unique
> i.e. 127.0.0.2:9043 becomes 127.0.0.2:9044
> and 127.0.0.3:9044 becomes 127.0.0.3:9043
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19221) CMS: Nodes can restart with new ipaddress already defined in the cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-19221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837763#comment-17837763
]
Alex Petrov edited comment on CASSANDRA-19221 at 4/16/24 3:28 PM:
--
I've had a closer look at it, and wanted to mention that 5.0 behaviour is most
likely uninteded; it contains at least one bug, and is potentially dangeroud.
In short, my test was to spin up a 3 node cluster: {{127.0.0.1}},
{{127.0.0.2}}, {{127.0.0.3}}, and swap IP addresses for the two latter nodes
({{.2}} and {{.3}}). As a result of this test, nodes have in fact swapped their
IPs, but:
* if you would shut down {{.2}} and {{.3}}, and start {{.2}}, and then
{{.3}}, {{.3}} startup won't even begin because ccm considers its IP address to
be occupied, so an entire test can work only if you start the two nodes in
parallel
* after swapping ip addresses, ccm breaks, since it attempts to search {{UP}}
message for a specific IP address for a node, which it doesn't find if you
merely change the address in the conf file
* peers table for {{.2}} whose address is now {{.3}} will still have {{.3}}
in its peers table.
In general, since we are using ip addresses for node identity, I am weary of
allowing identity transfers for the occupied pars. By this I mean if {{ip <->
node id}} pair exists in the directory, we have to free up the IP address
before the other node can claim it. So the test would look as follows:
So for swapping {{.2}} and {{.3}}, one of the nodes would have to migrate to
{{.4}} first, and only then can the freed up IP address be occupied again.
Submitting a patch that fixes the peers table behaviour and codifies a
requirement of a separate node for swapping addresses.
was (Author: ifesdjeen):
I've had a closer look at it, and wanted to mention that 5.0 behaviour is most
likely uninteded; it contains at least one bug, and is potentially dangeroud.
In short, my test was to spin up a 3 node cluster: {{127.0.0.1}},
{{127.0.0.2}}, {{127.0.0.3}}, and swap IP addresses for the two latter nodes
({{.2}} and {{.3}}. As a result of this test, nodes have in fact swapped their
IPs, but:
* if you would shut down {{.2}} and {{.3}}, and start {{.2}}, and then
{{.3}}, {{.3}} startup won't even begin because ccm considers its IP address to
be occupied, so an entire test can work only if you start the two nodes in
parallel
* after swapping ip addresses, ccm breaks, since it attempts to search {{UP}}
message for a specific IP address for a node, which it doesn't find if you
merely change the address in the conf file
* peers table for {{.2}} whose address is now {{.3}} will still have {{.3}}
in its peers table.
In general, since we are using ip addresses for node identity, I am weary of
allowing identity transfers for the occupied pars. By this I mean if {{ip <->
node id}} pair exists in the directory, we have to free up the IP address
before the other node can claim it. So the test would look as follows:
So for swapping {{.2}} and {{.3}}, one of the nodes would have to migrate to
{{.4}} first, and only then can the freed up IP address be occupied again.
Submitting a patch that fixes the peers table behaviour and codifies a
requirement of a separate node for swapping addresses.
> CMS: Nodes can restart with new ipaddress already defined in the cluster
>
>
> Key: CASSANDRA-19221
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19221
> Project: Cassandra
> Issue Type: Bug
> Components: Transactional Cluster Metadata
>Reporter: Paul Chandler
>Assignee: Alex Petrov
>Priority: Normal
> Fix For: 5.1-alpha1
>
>
> I am simulating running a cluster in Kubernetes and testing what happens when
> several pods go down and ip addresses are swapped between nodes. In 4.0 this
> is blocked and the node cannot be restarted.
> To simulate this I create a 3 node cluster on a local machine using 3
> loopback addresses
> {code}
> 127.0.0.1
> 127.0.0.2
> 127.0.0.3
> {code}
> The nodes are created correctly and the first node is assigned as a CMS node
> as shown:
> {code}
> bin/nodetool -p 7199 describecms
> {code}
> Cluster Metadata Service:
> {code}
> Members: /127.0.0.1:7000
> Is Member: true
> Service State: LOCAL
> {code}
> At this point I bring down the nodes 127.0.0.2 and 127.0.0.3 and swap the ip
> addresses for the rpc_address and listen_address
>
> The nodes come back as normal, but the nodeid has now been swapped against
> the ip address:
> Before:
> {code}
> Datacenter: datacenter1
> ===
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- Address Load Tokens Owns (effective) Host ID
> Rack
> UN 127.0.0.3 75.2 KiB
[jira] [Updated] (CASSANDRA-19561) finish_release.sh does not handle jfrog errors
[ https://issues.apache.org/jira/browse/CASSANDRA-19561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19561: - Description: If there is any "soft" problem (not network) uploading to jfrog, finish_release.sh will keep on going and then delete the packages regardless of whether they could be uploaded or not. (was: If there is any problem uploading to jfrog, finish_release.sh will keep on going and then delete the packages regardless of whether they could be uploaded or not.) > finish_release.sh does not handle jfrog errors > -- > > Key: CASSANDRA-19561 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19561 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Brandon Williams >Priority: Normal > > If there is any "soft" problem (not network) uploading to jfrog, > finish_release.sh will keep on going and then delete the packages regardless > of whether they could be uploaded or not. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19561) finish_release.sh does not handle jfrog errors
[
https://issues.apache.org/jira/browse/CASSANDRA-19561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837751#comment-17837751
]
Brandon Williams commented on CASSANDRA-19561:
--
The root cause seems to be that jfrog returns its errors in json instead of
http return codes, so curl will always exit with zero:
{noformat}
drift@phoenix:/tmp$ curl -X PUT -T foo -ubad:password
https://apache.jfrog.io/artifactory/cassandra/whatever?override=1
{
"errors" : [ {
"status" : 401,
"message" : "Bad credentials"
} ]
}drift@phoenix:/tmp$ echo $?
0
{noformat}
> finish_release.sh does not handle jfrog errors
> --
>
> Key: CASSANDRA-19561
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19561
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Brandon Williams
>Priority: Normal
>
> If there is any problem uploading to jfrog, finish_release.sh will keep on
> going and then delete the packages regardless of whether they could be
> uploaded or not.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19561) finish_release.sh does not handle jfrog errors
[ https://issues.apache.org/jira/browse/CASSANDRA-19561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19561: - Bug Category: Parent values: Degradation(12984)Level 1 values: Other Exception(12998) Complexity: Normal Component/s: Build Discovered By: User Report Severity: Normal Status: Open (was: Triage Needed) > finish_release.sh does not handle jfrog errors > -- > > Key: CASSANDRA-19561 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19561 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Brandon Williams >Priority: Normal > > If there is any problem uploading to jfrog, finish_release.sh will keep on > going and then delete the packages regardless of whether they could be > uploaded or not. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19561) finish_release.sh does not handle jfrog errors
Brandon Williams created CASSANDRA-19561: Summary: finish_release.sh does not handle jfrog errors Key: CASSANDRA-19561 URL: https://issues.apache.org/jira/browse/CASSANDRA-19561 Project: Cassandra Issue Type: Bug Reporter: Brandon Williams If there is any problem uploading to jfrog, finish_release.sh will keep on going and then delete the packages regardless of whether they could be uploaded or not. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
svn commit: r68582 - /release/cassandra/3.11.17/redhat/
Author: brandonwilliams Date: Tue Apr 16 14:43:07 2024 New Revision: 68582 Log: Apache Cassandra 3.11.17 redhat artifacts Removed: release/cassandra/3.11.17/redhat/ - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
svn commit: r68581 - /release/cassandra/3.11.17/debian/
Author: brandonwilliams Date: Tue Apr 16 14:42:34 2024 New Revision: 68581 Log: Apache Cassandra 3.11.17 debian artifacts Removed: release/cassandra/3.11.17/debian/ - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
svn commit: r68580 - /dev/cassandra/3.11.17/ /release/cassandra/3.11.17/
Author: brandonwilliams Date: Tue Apr 16 14:41:54 2024 New Revision: 68580 Log: Apache Cassandra 3.11.17 release Added: release/cassandra/3.11.17/ - copied from r68579, dev/cassandra/3.11.17/ Removed: dev/cassandra/3.11.17/ - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) tag 3.11.17-tentative deleted (was d079952c50)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to tag 3.11.17-tentative in repository https://gitbox.apache.org/repos/asf/cassandra.git *** WARNING: tag 3.11.17-tentative was deleted! *** was d079952c50 Prepare debian changelog for 3.11.17 The revisions that were on this tag are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) annotated tag cassandra-3.11.17 created (now 0fb5e9cb91)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to annotated tag cassandra-3.11.17 in repository https://gitbox.apache.org/repos/asf/cassandra.git at 0fb5e9cb91 (tag) tagging d079952c50019c3c8c96e341a01f7bc0554e1733 (commit) replaces cassandra-3.11.16 by Brandon Williams on Tue Apr 16 09:41:43 2024 -0500 - Log - Apache Cassandra 3.11.17 release --- No new revisions were added by this update. - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18661) Update cassandra-stress to use Apache Commons CLI
[
https://issues.apache.org/jira/browse/CASSANDRA-18661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837738#comment-17837738
]
Josh McKenzie commented on CASSANDRA-18661:
---
{quote}Perhaps that is a topic for the ML
{quote}
+1. [~rustyrazorblade] - you able to / willing to open up a [DISCUSS] thread on
this? I've just started more aggressively using tlp-stress internally and I
think the time is ripe for us to talk about this as a project.
> Update cassandra-stress to use Apache Commons CLI
> -
>
> Key: CASSANDRA-18661
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18661
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/stress
>Reporter: Brad Schoening
>Assignee: Claude Warren
>Priority: Normal
> Labels: lhf
>
> The Apache Commons CLI library provides an API for parsing command line
> options with the package org.apache.commons.cli and this is already used by a
> dozen of existing Cassandra utilities including:
> {quote}SSTableMetadataViewer, StandaloneScrubber, StandaloneSplitter,
> SSTableExport, BulkLoader, and others.
> {quote}
> However, cassandra-stress is an outlier which uses its own custom classes to
> parse command line options with classes such as OptionsSimple. In addition,
> the options syntax for username, password, and others are not aligned with
> the format used by CQLSH.
> Currently, there are > 5K lines of code in 'settings' which appears to just
> process command line args.
> This suggestion is to:
>
> a) Upgrade cassandra-stress to use Apache Commons CLI (no new dependencies
> are required as this library is already used by the project)
>
> b) Align the cassandra-stress CLI options with those in CQLSH,
>
> {quote}For example, using the new syntax like CQLSH:
> {quote}
>
> cassandra-stress -username foo -password bar
> {quote}and replacing the old syntax:
> {quote}
> cassandra-stress -mode username=foo and password=bar
>
> This will simplify and unify the code base, eliminate code and reduce the
> confusion between similar named classes such as
> org.apache.cassandra.stress.settings.\{Option, OptionsMulti, OptionsSimple}
> and org.apache.commons.cli.{Option, OptionGroup, Options)
>
> Note: documentation will need to be updated as well
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18831) Enable Cassandra to build and run under JDK21
[ https://issues.apache.org/jira/browse/CASSANDRA-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837735#comment-17837735 ] Josh McKenzie commented on CASSANDRA-18831: --- Sorry, think this is me abusing colloquialisms of US english. Basically, I'm only reactively updating dependencies where required to get them to work on JDK21. I'm not proactively going to update any dependencies unless required by the JDK upgrade and environment. The 3 heuristics I use to determine whether to advocate for a dependency bump: # Is there a CVE? # Do we need to bump the dep to support some other change (another lib bump, JDK version change, etc)? # Is there new functionality we need from either a new dep or a version update? In the case of things working on JDK21, I'm limiting my considerations to #2 above; only updating where required to have things work on JDK21. My guess is the gap from 17->21 is smaller than the gap from 11->17 was so I'm not running into as many breakages as you did w/that work, for which I am very grateful to you. :) > Enable Cassandra to build and run under JDK21 > - > > Key: CASSANDRA-18831 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18831 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Achilles Benetopoulos >Assignee: Josh McKenzie >Priority: Normal > Fix For: 5.x > > Attachments: jdk21-patch > > > This patch builds on the work in CASSANDRA-16895 that added JDK17 to the list > of supported Java versions, and extends that work to enable building and > running Cassandra under JDK21. > The following commits comprise the changes included in the attached patch: > - > [https://github.com/apache/cassandra/commit/b15d4d6980e787ab5f3405ca8cb17a9c92a4aa47] > - > [https://github.com/apache/cassandra/commit/0c5df38dafe58bfec8924e81507bb604e1543897] > - > [https://github.com/apache/cassandra/commit/6506b7279d98eed4b2b65b71e0c6f41eb78c6913] > - > [https://github.com/apache/cassandra/commit/564cbd534c5a975cda0c629c14c68c8745b41451] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19551) CCM nodes share the same environment variable map breaking upgrade tests
[
https://issues.apache.org/jira/browse/CASSANDRA-19551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Berenguer Blasi updated CASSANDRA-19551:
Reviewers: Berenguer Blasi, Josh McKenzie, Joshua McKenzie (was: Josh
McKenzie, Joshua McKenzie)
> CCM nodes share the same environment variable map breaking upgrade tests
>
>
> Key: CASSANDRA-19551
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19551
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 5.x
>
> Attachments: ci_summary.html
>
>
> In {{node.py}} {{__environment_variables}} is generally always set with a map
> that is passed in from {{cluster.py}} so it is [shared between
> nodes](https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L151)
> and if nodes modify the map, such as in {{start}} when [updating the Java
> version](https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L860)
> then when {{get_env}} runs it will [overwrite the Java
> version](https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L244)
> that is selected by {{update_java_version}}.
> This results in {{nodetool drain}} failing when upgrading from 3.11 to 4.0 in
> some of the upgrade tests because after the first node upgrades to 4.0 it's
> not longer possible for the subsequent nodes to select a Java version that
> isn't 11 because it's overridden by {{__environment_variables}}.
> I'm not even 100% clear on why the code in {{start}} should update
> {{__environment_variables}} at all if we calculate the correct java version
> on every invocation of other tools.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19551) CCM nodes share the same environment variable map breaking upgrade tests
[
https://issues.apache.org/jira/browse/CASSANDRA-19551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh McKenzie updated CASSANDRA-19551:
--
Status: Ready to Commit (was: Review In Progress)
> CCM nodes share the same environment variable map breaking upgrade tests
>
>
> Key: CASSANDRA-19551
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19551
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 5.x
>
> Attachments: ci_summary.html
>
>
> In {{node.py}} {{__environment_variables}} is generally always set with a map
> that is passed in from {{cluster.py}} so it is [shared between
> nodes](https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L151)
> and if nodes modify the map, such as in {{start}} when [updating the Java
> version](https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L860)
> then when {{get_env}} runs it will [overwrite the Java
> version](https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L244)
> that is selected by {{update_java_version}}.
> This results in {{nodetool drain}} failing when upgrading from 3.11 to 4.0 in
> some of the upgrade tests because after the first node upgrades to 4.0 it's
> not longer possible for the subsequent nodes to select a Java version that
> isn't 11 because it's overridden by {{__environment_variables}}.
> I'm not even 100% clear on why the code in {{start}} should update
> {{__environment_variables}} at all if we calculate the correct java version
> on every invocation of other tools.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19551) CCM nodes share the same environment variable map breaking upgrade tests
[
https://issues.apache.org/jira/browse/CASSANDRA-19551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh McKenzie updated CASSANDRA-19551:
--
Reviewers: Joshua McKenzie, Josh McKenzie (was: Joshua McKenzie)
Joshua McKenzie, Josh McKenzie (was: Josh McKenzie, Joshua
McKenzie)
Status: Review In Progress (was: Patch Available)
> CCM nodes share the same environment variable map breaking upgrade tests
>
>
> Key: CASSANDRA-19551
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19551
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 5.x
>
> Attachments: ci_summary.html
>
>
> In {{node.py}} {{__environment_variables}} is generally always set with a map
> that is passed in from {{cluster.py}} so it is [shared between
> nodes](https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L151)
> and if nodes modify the map, such as in {{start}} when [updating the Java
> version](https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L860)
> then when {{get_env}} runs it will [overwrite the Java
> version](https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L244)
> that is selected by {{update_java_version}}.
> This results in {{nodetool drain}} failing when upgrading from 3.11 to 4.0 in
> some of the upgrade tests because after the first node upgrades to 4.0 it's
> not longer possible for the subsequent nodes to select a Java version that
> isn't 11 because it's overridden by {{__environment_variables}}.
> I'm not even 100% clear on why the code in {{start}} should update
> {{__environment_variables}} at all if we calculate the correct java version
> on every invocation of other tools.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19551) CCM nodes share the same environment variable map breaking upgrade tests
[
https://issues.apache.org/jira/browse/CASSANDRA-19551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837721#comment-17837721
]
Josh McKenzie commented on CASSANDRA-19551:
---
+1 on the PR. As I mentioned in the PR notes, looks like the cluster object is
the only other mutable thing we assign by ref but I can't think of a situation
in which that'd be mutated externally so should be fine.
> CCM nodes share the same environment variable map breaking upgrade tests
>
>
> Key: CASSANDRA-19551
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19551
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 5.x
>
> Attachments: ci_summary.html
>
>
> In {{node.py}} {{__environment_variables}} is generally always set with a map
> that is passed in from {{cluster.py}} so it is [shared between
> nodes](https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L151)
> and if nodes modify the map, such as in {{start}} when [updating the Java
> version](https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L860)
> then when {{get_env}} runs it will [overwrite the Java
> version](https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L244)
> that is selected by {{update_java_version}}.
> This results in {{nodetool drain}} failing when upgrading from 3.11 to 4.0 in
> some of the upgrade tests because after the first node upgrades to 4.0 it's
> not longer possible for the subsequent nodes to select a Java version that
> isn't 11 because it's overridden by {{__environment_variables}}.
> I'm not even 100% clear on why the code in {{start}} should update
> {{__environment_variables}} at all if we calculate the correct java version
> on every invocation of other tools.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19558) Standalone jenkinsfile first round bug fixes
[ https://issues.apache.org/jira/browse/CASSANDRA-19558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837698#comment-17837698 ] Michael Semb Wever commented on CASSANDRA-19558: CI - 5.0 post-commit internal [^CASSANDRA-19558-5.0_#16_ci_summary.html] , [^CASSANDRA-19558-5.0_#16_results_details.tar.xz] (green run, took 2.5hrs) - 5.0 …wip… - trunk …wip… > Standalone jenkinsfile first round bug fixes > > > Key: CASSANDRA-19558 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19558 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 5.0.x, 5.x > > Attachments: CASSANDRA-19558_50_#5_ci_summary.html, > CASSANDRA-19558_50_#5_results_details.tar.xz, > CASSANDRA-19558-5.0_#13_ci_summary.html, > CASSANDRA-19558-5.0_#13_results_details.tar.xz, > CASSANDRA-19558-5.0_#16_ci_summary.html, > CASSANDRA-19558-5.0_#16_results_details.tar.xz, > CASSANDRA-19558_#8_ci_summary.html, CASSANDRA-19558_#8_results_details.tar.xz > > > A few follow up improvements and bug fixes for the standalone jenkinsfile. > - add at top a list of test failures in ci_summary.html > - docker scripts always try to login (as base images need to be pulled too) > - move simulator-dtests to large containers (they need 8g just heap) > - in ubuntu2004_test.docker make sure /home/cassandra exists and has correct > perms (from marcuse) > - persist the jenkinsfile parameters from run to run (important for the > post-commit jobs to keep their non-default branch and profile values) (was > CASSANDRA-19536) > - increase jvm-dtest splits from 8 to 12 > - when on ci-cassandra, replace use of copyArtifacts in Jenkinsfile > generateTestReports() with manual wget of test files, allowing the summary > phase to be run on any agent (copyArtifact would take >4hrs otherwise) (was > INFRA-25694) > - copy ci_summary.html and results_details.tar.xz to nightlies -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19558) Standalone jenkinsfile first round bug fixes
[ https://issues.apache.org/jira/browse/CASSANDRA-19558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-19558: --- Attachment: CASSANDRA-19558-5.0_#16_ci_summary.html CASSANDRA-19558-5.0_#16_results_details.tar.xz > Standalone jenkinsfile first round bug fixes > > > Key: CASSANDRA-19558 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19558 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 5.0.x, 5.x > > Attachments: CASSANDRA-19558_50_#5_ci_summary.html, > CASSANDRA-19558_50_#5_results_details.tar.xz, > CASSANDRA-19558-5.0_#13_ci_summary.html, > CASSANDRA-19558-5.0_#13_results_details.tar.xz, > CASSANDRA-19558-5.0_#16_ci_summary.html, > CASSANDRA-19558-5.0_#16_results_details.tar.xz, > CASSANDRA-19558_#8_ci_summary.html, CASSANDRA-19558_#8_results_details.tar.xz > > > A few follow up improvements and bug fixes for the standalone jenkinsfile. > - add at top a list of test failures in ci_summary.html > - docker scripts always try to login (as base images need to be pulled too) > - move simulator-dtests to large containers (they need 8g just heap) > - in ubuntu2004_test.docker make sure /home/cassandra exists and has correct > perms (from marcuse) > - persist the jenkinsfile parameters from run to run (important for the > post-commit jobs to keep their non-default branch and profile values) (was > CASSANDRA-19536) > - increase jvm-dtest splits from 8 to 12 > - when on ci-cassandra, replace use of copyArtifacts in Jenkinsfile > generateTestReports() with manual wget of test files, allowing the summary > phase to be run on any agent (copyArtifact would take >4hrs otherwise) (was > INFRA-25694) > - copy ci_summary.html and results_details.tar.xz to nightlies -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19476) CQL Management API
[
https://issues.apache.org/jira/browse/CASSANDRA-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837675#comment-17837675
]
Maxim Muzafarov edited comment on CASSANDRA-19476 at 4/16/24 12:08 PM:
---
I've spent some time prototyping the changes and the plan may be the following,
splitting by the issues:
# _Replace the airline with the picocli._ CASSANDRA-17445 The airlift library
is now used to expose the management commands via the CLI. We can reuse this
approach to keep the number of changes low, however, we need to replace the
library with the picocli as the airlift has drawbacks. It prevents us from
building JMX and CQL APIs in follows:
## The airlift library is no longer supported;
## The library handles the `[ ...]` as a single input
argument, but we need to handle it independently as two input arguments
`keyspace` and `tables` to build JMX and CQL APIs;
## All commands right now have host, port, username, and password options etc.
which are not relevant to other types of interfaces;
# {_}Create an adapter that builds dynamic JMX MBeans{_}, and parses the
picocli annotations of the commands we already have. This will expose these
commands with the same parameters and options that they have in the CLI,
matching the nodetool CLI input. The management MBeans of the internal
components e.g. StorageServiceMBean can be injected into the commands directly
when the command context is initialized.
# _Create a CQL command adapter._ Based on the commands annotations we can
create a new adapter that handles command parameters received from the binary
protocol, this is basically the same as new dynamic MBeans with the caveat that
the binary protocol have to be improved to support new operation types.
Options 2 and 3 can be implemented independently.
was (Author: mmuzaf):
I've spent some time prototyping the changes and the plan may be the following,
splitting by the issues:
# _Replace the airline with the picocli._ CASSANDRA-17445 The airlift library
is now used to expose the management commands via the CLI. We can reuse this
approach to keep the number of changes low, however, we need to replace the
library with the picocli as the airlift has drawbacks. It prevents us from
building JMX and CQL APIs in follows: the airlift library is no longer
supported, the library handles the `[ ...]` as a single input
argument, but we need to handle it independently as two input arguments
`keyspace` and `tables` to build JMX and CQL APIs.
# _Create an adapter that builds dynamic JMX MBeans_, and parses the picocli
annotations of the commands we already have. This will expose these commands
with the same parameters and options that they have in the CLI, matching the
nodetool CLI input. The management MBeans of the internal components e.g.
StorageServiceMBean can be injected into the commands directly when the command
context is initialized.
# _Create a CQL command adapter._ Based on the commands annotations we can
create a new adapter that handles command parameters received from the binary
protocol, this is basically the same as new dynamic MBeans with the caveat that
the binary protocol have to be improved to support new operation types.
Options 2 and 3 can be implemented independently.
> CQL Management API
> --
>
> Key: CASSANDRA-19476
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19476
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL/Interpreter, CQL/Syntax, Tool/nodetool
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: High
> Labels: CEP-38
> Fix For: 5.x
>
>
> We want to run management commands via CQL.
> The goals are:
> * To provide a way to run predefined management commands via CQL;
> * To provide a mechanism for retrieving command definitions and metadata via
> CQL;
> * To provide information on all available management commands via virtual
> tables;
> * To provide a registry that stores all C* commands and their metadata
> accordingly;
> * To internal instrumentation and a reasonable plan for migrating cluster
> management from JMX to CQL, taking into account backward compatibility and
> adopted deprecation policies;
> The discussion on the ML:
> https://lists.apache.org/thread/pow83q92m666nqtwyw4m3b18nnkgj2y8
> The design document:
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-38%3A+CQL+Management+API
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19559) prepare_release.sh should check for mvn
[ https://issues.apache.org/jira/browse/CASSANDRA-19559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19559: - Status: Review In Progress (was: Needs Committer) > prepare_release.sh should check for mvn > --- > > Key: CASSANDRA-19559 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19559 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: NA > > > Part of the 'prepare' phase of releasing includes publishing Maven artifacts, > which requires that it be installed. The script should check for this since > it's quite easy to miss. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19559) prepare_release.sh should check for mvn
[ https://issues.apache.org/jira/browse/CASSANDRA-19559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19559: - Reviewers: Berenguer Blasi > prepare_release.sh should check for mvn > --- > > Key: CASSANDRA-19559 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19559 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: NA > > > Part of the 'prepare' phase of releasing includes publishing Maven artifacts, > which requires that it be installed. The script should check for this since > it's quite easy to miss. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19559) prepare_release.sh should check for mvn
[ https://issues.apache.org/jira/browse/CASSANDRA-19559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19559: - Status: Ready to Commit (was: Review In Progress) > prepare_release.sh should check for mvn > --- > > Key: CASSANDRA-19559 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19559 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: NA > > > Part of the 'prepare' phase of releasing includes publishing Maven artifacts, > which requires that it be installed. The script should check for this since > it's quite easy to miss. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19559) prepare_release.sh should check for mvn
[ https://issues.apache.org/jira/browse/CASSANDRA-19559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19559: - Fix Version/s: 3.0.31 3.11.17 4.0.13 4.1.5 5.0-beta2 5.1 (was: NA) Since Version: NA Source Control Link: https://github.com/apache/cassandra-builds/commit/cdf0ec532635ce069c3406a176846209c9be8af5 Resolution: Fixed Status: Resolved (was: Ready to Commit) Thanks for the review! Committed. > prepare_release.sh should check for mvn > --- > > Key: CASSANDRA-19559 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19559 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.31, 3.11.17, 4.0.13, 4.1.5, 5.0-beta2, 5.1 > > > Part of the 'prepare' phase of releasing includes publishing Maven artifacts, > which requires that it be installed. The script should check for this since > it's quite easy to miss. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19559) prepare_release.sh should check for mvn
[ https://issues.apache.org/jira/browse/CASSANDRA-19559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19559: - Status: Needs Committer (was: Patch Available) > prepare_release.sh should check for mvn > --- > > Key: CASSANDRA-19559 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19559 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: NA > > > Part of the 'prepare' phase of releasing includes publishing Maven artifacts, > which requires that it be installed. The script should check for this since > it's quite easy to miss. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-builds) branch trunk updated: Fix error when gpg_key is not set and add check for mvn
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
The following commit(s) were added to refs/heads/trunk by this push:
new cdf0ec5 Fix error when gpg_key is not set and add check for mvn
cdf0ec5 is described below
commit cdf0ec532635ce069c3406a176846209c9be8af5
Author: Brandon Williams
AuthorDate: Mon Apr 15 12:32:08 2024 -0500
Fix error when gpg_key is not set and add check for mvn
Patch by brandonwilliams; reviewed by bereng for CASSANDRA-19559
---
cassandra-release/prepare_release.sh | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/cassandra-release/prepare_release.sh
b/cassandra-release/prepare_release.sh
index a9e6ceb..cff842e 100755
--- a/cassandra-release/prepare_release.sh
+++ b/cassandra-release/prepare_release.sh
@@ -9,7 +9,7 @@ if [ -z "$gpg_key" ]; then
fi
if [ "$gpg_key" = "" ]; then
-exit -e "Gpg key is unset. Pleae set gpg_key variable."
+echo >&2 "Gpg key is unset. Pleae set gpg_key variable." && exit 1
fi
# The name of remote for the asf remote in your git repo
@@ -28,6 +28,7 @@ command -v reprepro >/dev/null 2>&1 || { echo >&2 "reprepro
needs to be installe
command -v rpmsign >/dev/null 2>&1 || { echo >&2 "rpmsign needs to be
installed"; exit 1; }
command -v docker >/dev/null 2>&1 || { echo >&2 "docker needs to be
installed"; exit 1; }
command -v createrepo_c >/dev/null 2>&1 || { echo >&2 "createrepo_c needs to
be installed"; exit 1; }
+command -v mvn >/dev/null 2>&1 || { echo >&2 "mvn needs to be installed"; exit
1; }
(docker info >/dev/null 2>&1) || { echo >&2 "docker needs to running"; exit 1;
}
###
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19476) CQL Management API
[ https://issues.apache.org/jira/browse/CASSANDRA-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837675#comment-17837675 ] Maxim Muzafarov edited comment on CASSANDRA-19476 at 4/16/24 11:18 AM: --- I've spent some time prototyping the changes and the plan may be the following, splitting by the issues: # _Replace the airline with the picocli._ CASSANDRA-17445 The airlift library is now used to expose the management commands via the CLI. We can reuse this approach to keep the number of changes low, however, we need to replace the library with the picocli as the airlift has drawbacks. It prevents us from building JMX and CQL APIs in follows: the airlift library is no longer supported, the library handles the `[ ...]` as a single input argument, but we need to handle it independently as two input arguments `keyspace` and `tables` to build JMX and CQL APIs. # _Create an adapter that builds dynamic JMX MBeans_, and parses the picocli annotations of the commands we already have. This will expose these commands with the same parameters and options that they have in the CLI, matching the nodetool CLI input. The management MBeans of the internal components e.g. StorageServiceMBean can be injected into the commands directly when the command context is initialized. # _Create a CQL command adapter._ Based on the commands annotations we can create a new adapter that handles command parameters received from the binary protocol, this is basically the same as new dynamic MBeans with the caveat that the binary protocol have to be improved to support new operation types. Options 2 and 3 can be implemented independently. was (Author: mmuzaf): I've spent some time prototyping the changes and the plan may be the following, splitting by the issues: # _Replace the airline with the picocli._ CASSANDRA-17445 The airlift library is now used to expose the management commands via the CLI. We can reuse this approach to keep the number of changes low, however, we need to replace the library with the picocli as the airlift has drawbacks. It prevents us from building JMX and CQL APIs in follows: the airlift library is no longer supported, the library handles the `[ ...]` as a single input argument, but we need to handle it independently for JMX and CQL APIs. # _Create an adapter that builds dynamic JMX MBeans_, and parses the picocli annotations of the commands we already have. This will expose these commands with the same parameters and options that they have in the CLI, matching the nodetool CLI input. The management MBeans of the internal components e.g. StorageServiceMBean can be injected into the commands directly when the command context is initialized. # _Create a CQL command adapter._ Based on the commands annotations we can create a new adapter that handles command parameters received from the binary protocol, this is basically the same as new dynamic MBeans with the caveat that the binary protocol have to be improved to support new operation types. Options 2 and 3 can be implemented independently. > CQL Management API > -- > > Key: CASSANDRA-19476 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19476 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Interpreter, CQL/Syntax, Tool/nodetool >Reporter: Maxim Muzafarov >Assignee: Maxim Muzafarov >Priority: High > Labels: CEP-38 > Fix For: 5.x > > > We want to run management commands via CQL. > The goals are: > * To provide a way to run predefined management commands via CQL; > * To provide a mechanism for retrieving command definitions and metadata via > CQL; > * To provide information on all available management commands via virtual > tables; > * To provide a registry that stores all C* commands and their metadata > accordingly; > * To internal instrumentation and a reasonable plan for migrating cluster > management from JMX to CQL, taking into account backward compatibility and > adopted deprecation policies; > The discussion on the ML: > https://lists.apache.org/thread/pow83q92m666nqtwyw4m3b18nnkgj2y8 > The design document: > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-38%3A+CQL+Management+API -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19476) CQL Management API
[ https://issues.apache.org/jira/browse/CASSANDRA-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837675#comment-17837675 ] Maxim Muzafarov commented on CASSANDRA-19476: - I've spent some time prototyping the changes and the plan may be the following, splitting by the issues: # _Replace the airline with the picocli._ CASSANDRA-17445 The airlift library is now used to expose the management commands via the CLI. We can reuse this approach to keep the number of changes low, however, we need to replace the library with the picocli as the airlift has drawbacks. It prevents us from building JMX and CQL APIs in follows: the airlift library is no longer supported, the library handles the `[ ...]` as a single input argument, but we need to handle it independently for JMX and CQL APIs. # _Create an adapter that builds dynamic JMX MBeans_, and parses the picocli annotations of the commands we already have. This will expose these commands with the same parameters and options that they have in the CLI, matching the nodetool CLI input. The management MBeans of the internal components e.g. StorageServiceMBean can be injected into the commands directly when the command context is initialized. # _Create a CQL command adapter._ Based on the commands annotations we can create a new adapter that handles command parameters received from the binary protocol, this is basically the same as new dynamic MBeans with the caveat that the binary protocol have to be improved to support new operation types. Options 2 and 3 can be implemented independently. > CQL Management API > -- > > Key: CASSANDRA-19476 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19476 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Interpreter, CQL/Syntax, Tool/nodetool >Reporter: Maxim Muzafarov >Assignee: Maxim Muzafarov >Priority: High > Labels: CEP-38 > Fix For: 5.x > > > We want to run management commands via CQL. > The goals are: > * To provide a way to run predefined management commands via CQL; > * To provide a mechanism for retrieving command definitions and metadata via > CQL; > * To provide information on all available management commands via virtual > tables; > * To provide a registry that stores all C* commands and their metadata > accordingly; > * To internal instrumentation and a reasonable plan for migrating cluster > management from JMX to CQL, taking into account backward compatibility and > adopted deprecation policies; > The discussion on the ML: > https://lists.apache.org/thread/pow83q92m666nqtwyw4m3b18nnkgj2y8 > The design document: > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-38%3A+CQL+Management+API -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19429) Remove lock contention generated by getCapacity function in SSTableReader
[
https://issues.apache.org/jira/browse/CASSANDRA-19429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-19429:
--
Status: Patch Available (was: Review In Progress)
> Remove lock contention generated by getCapacity function in SSTableReader
> -
>
> Key: CASSANDRA-19429
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19429
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Dipietro Salvatore
>Assignee: Dipietro Salvatore
>Priority: Normal
> Fix For: 4.0.x, 4.1.x
>
> Attachments: Screenshot 2024-02-26 at 10.27.10.png, Screenshot
> 2024-02-27 at 11.29.41.png, Screenshot 2024-03-19 at 15.22.50.png,
> asprof_cass4.1.3__lock_20240216052912lock.html,
> image-2024-03-08-15-51-30-439.png, image-2024-03-08-15-52-07-902.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Profiling Cassandra 4.1.3 on large AWS instances, a high number of lock
> acquires is measured in the `getCapacity` function from
> `org/apache/cassandra/cache/InstrumentingCache` (1.9M lock acquires per 60
> seconds). Based on our tests on r8g.24xlarge instances (using Ubuntu 22.04),
> this limits the CPU utilization of the system to under 50% when testing at
> full load and therefore limits the achieved throughput.
> Removing the lock contention from the SSTableReader.java file by replacing
> the call to `getCapacity` with `size` achieves up to 2.95x increase in
> throughput on r8g.24xlarge and 2x on r7i.24xlarge:
> |Instance type|Cass 4.1.3|Cass 4.1.3 patched|
> |r8g.24xlarge|168k ops|496k ops (2.95x)|
> |r7i.24xlarge|153k ops|304k ops (1.98x)|
>
> Instructions to reproduce:
> {code:java}
> ## Requirements for Ubuntu 22.04
> sudo apt install -y ant git openjdk-11-jdk
> ## Build and run
> CASSANDRA_USE_JDK11=true ant realclean && CASSANDRA_USE_JDK11=true ant jar &&
> CASSANDRA_USE_JDK11=true ant stress-build && rm -rf data && bin/cassandra -f
> -R
> # Run
> bin/cqlsh -e 'drop table if exists keyspace1.standard1;' && \
> bin/cqlsh -e 'drop keyspace if exists keyspace1;' && \
> bin/nodetool clearsnapshot --all && tools/bin/cassandra-stress write
> n=1000 cl=ONE -rate threads=384 -node 127.0.0.1 -log file=cload.log
> -graph file=cload.html && \
> bin/nodetool compact keyspace1 && sleep 30s && \
> tools/bin/cassandra-stress mixed ratio\(write=10,read=90\) duration=10m
> cl=ONE -rate threads=406 -node localhost -log file=result.log -graph
> file=graph.html
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19429) Remove lock contention generated by getCapacity function in SSTableReader
[
https://issues.apache.org/jira/browse/CASSANDRA-19429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837621#comment-17837621
]
Stefan Miklosovic commented on CASSANDRA-19429:
---
I asked internally our techops to test this and we do not have such a big
instances we run Cassandra on regularly so it is of a low interest to deal with
this for now.
> Remove lock contention generated by getCapacity function in SSTableReader
> -
>
> Key: CASSANDRA-19429
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19429
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Dipietro Salvatore
>Assignee: Dipietro Salvatore
>Priority: Normal
> Fix For: 4.0.x, 4.1.x
>
> Attachments: Screenshot 2024-02-26 at 10.27.10.png, Screenshot
> 2024-02-27 at 11.29.41.png, Screenshot 2024-03-19 at 15.22.50.png,
> asprof_cass4.1.3__lock_20240216052912lock.html,
> image-2024-03-08-15-51-30-439.png, image-2024-03-08-15-52-07-902.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Profiling Cassandra 4.1.3 on large AWS instances, a high number of lock
> acquires is measured in the `getCapacity` function from
> `org/apache/cassandra/cache/InstrumentingCache` (1.9M lock acquires per 60
> seconds). Based on our tests on r8g.24xlarge instances (using Ubuntu 22.04),
> this limits the CPU utilization of the system to under 50% when testing at
> full load and therefore limits the achieved throughput.
> Removing the lock contention from the SSTableReader.java file by replacing
> the call to `getCapacity` with `size` achieves up to 2.95x increase in
> throughput on r8g.24xlarge and 2x on r7i.24xlarge:
> |Instance type|Cass 4.1.3|Cass 4.1.3 patched|
> |r8g.24xlarge|168k ops|496k ops (2.95x)|
> |r7i.24xlarge|153k ops|304k ops (1.98x)|
>
> Instructions to reproduce:
> {code:java}
> ## Requirements for Ubuntu 22.04
> sudo apt install -y ant git openjdk-11-jdk
> ## Build and run
> CASSANDRA_USE_JDK11=true ant realclean && CASSANDRA_USE_JDK11=true ant jar &&
> CASSANDRA_USE_JDK11=true ant stress-build && rm -rf data && bin/cassandra -f
> -R
> # Run
> bin/cqlsh -e 'drop table if exists keyspace1.standard1;' && \
> bin/cqlsh -e 'drop keyspace if exists keyspace1;' && \
> bin/nodetool clearsnapshot --all && tools/bin/cassandra-stress write
> n=1000 cl=ONE -rate threads=384 -node 127.0.0.1 -log file=cload.log
> -graph file=cload.html && \
> bin/nodetool compact keyspace1 && sleep 30s && \
> tools/bin/cassandra-stress mixed ratio\(write=10,read=90\) duration=10m
> cl=ONE -rate threads=406 -node localhost -log file=result.log -graph
> file=graph.html
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19532) Allow operators to disable the execution of triggers
[ https://issues.apache.org/jira/browse/CASSANDRA-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-19532: -- Fix Version/s: 5.1-alpha1 (was: 5.x) Source Control Link: https://github.com/apache/cassandra/commit/8d705b31e9a3f213e5eb296d4308e5b22b0e996c Resolution: Fixed Status: Resolved (was: Ready to Commit) > Allow operators to disable the execution of triggers > > > Key: CASSANDRA-19532 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19532 > Project: Cassandra > Issue Type: Improvement > Components: Local/Other >Reporter: Abe Ratnofsky >Assignee: Abe Ratnofsky >Priority: Normal > Fix For: 5.1-alpha1 > > Attachments: ci_summary-1.html > > Time Spent: 2h > Remaining Estimate: 0h > > Currently, triggers are discouraged but there's no explicit way to disable > them. Similar configuration already exists to disable other features, such as > "conf.materialized_views_enabled". There should be a means for operators to > gracefully disable the creation and execution of triggers. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch trunk updated: Add new TriggersPolicy configuration to allow operators to disable triggers
This is an automated email from the ASF dual-hosted git repository.
smiklosovic pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 8d705b31e9 Add new TriggersPolicy configuration to allow operators to
disable triggers
8d705b31e9 is described below
commit 8d705b31e9a3f213e5eb296d4308e5b22b0e996c
Author: Abe Ratnofsky
AuthorDate: Thu Apr 4 17:00:08 2024 -0400
Add new TriggersPolicy configuration to allow operators to disable triggers
patch by Abe Ratnofsky; reviewed by Stefan Miklosovic and Sam Tunnicliffe
for CASSANDRA-19532
---
CHANGES.txt| 1 +
conf/cassandra.yaml| 6 ++
conf/cassandra_latest.yaml | 6 ++
.../cassandra/pages/developing/cql/triggers.adoc | 4 +
src/java/org/apache/cassandra/config/Config.java | 12 +++
.../cassandra/config/DatabaseDescriptor.java | 12 +++
.../statements/schema/CreateTriggerStatement.java | 7 +-
.../triggers/TriggerDisabledException.java | 29 +++
.../apache/cassandra/triggers/TriggerExecutor.java | 35
.../config/DatabaseDescriptorRefTest.java | 1 +
.../apache/cassandra/triggers/TriggersTest.java| 98 --
11 files changed, 202 insertions(+), 9 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index cfbbc651ef..ab257c3baf 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
5.1
+ * Add new TriggersPolicy configuration to allow operators to disable triggers
(CASSANDRA-19532)
* Use Transformation.Kind.id in local and distributed log tables
(CASSANDRA-19516)
* Remove period field from ClusterMetadata and metadata log tables
(CASSANDRA-19482)
* Enrich system_views.pending_hints vtable with hints sizes (CASSANDRA-19486)
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index 42f311b0eb..bfc28162d5 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -1691,6 +1691,12 @@ trace_type_repair_ttl: 7d
# As of Cassandra 3.0 there is a sandbox in place that should prevent
execution of evil code.
user_defined_functions_enabled: false
+# Triggers are enabled by default.
+# `enabled` executes queries and their triggers.
+# `disabled` executes queries but skips trigger execution, and logs a warning.
+# `forbidden` fails queries that would execute triggers with
TriggerDisabledException.
+triggers_policy: enabled
+
# Enables encrypting data at-rest (on disk). Different key providers can be
plugged in, but the default reads from
# a JCE-style keystore. A single keystore can hold multiple keys, but the one
referenced by
# the "key_alias" is the only key that will be used for encrypt opertaions;
previously used keys
diff --git a/conf/cassandra_latest.yaml b/conf/cassandra_latest.yaml
index 585e70f598..2e996d9468 100644
--- a/conf/cassandra_latest.yaml
+++ b/conf/cassandra_latest.yaml
@@ -1655,6 +1655,12 @@ trace_type_repair_ttl: 7d
# As of Cassandra 3.0 there is a sandbox in place that should prevent
execution of evil code.
user_defined_functions_enabled: false
+# Triggers are enabled by default.
+# `enabled` executes queries and their triggers.
+# `disabled` executes queries but skips trigger execution, and logs a warning.
+# `forbidden` fails queries that would execute triggers with
TriggerDisabledException.
+triggers_policy: enabled
+
# Enables encrypting data at-rest (on disk). Different key providers can be
plugged in, but the default reads from
# a JCE-style keystore. A single keystore can hold multiple keys, but the one
referenced by
# the "key_alias" is the only key that will be used for encrypt opertaions;
previously used keys
diff --git a/doc/modules/cassandra/pages/developing/cql/triggers.adoc
b/doc/modules/cassandra/pages/developing/cql/triggers.adoc
index 89247206a5..45497ca78d 100644
--- a/doc/modules/cassandra/pages/developing/cql/triggers.adoc
+++ b/doc/modules/cassandra/pages/developing/cql/triggers.adoc
@@ -48,3 +48,7 @@ For instance:
include::cassandra:example$CQL/drop_trigger.cql[]
+
+Triggers can be disabled in two steps. `triggers_policy` is `enabled` by
default, which runs all created triggers as
+mutations are executed. `disabled` skips trigger execution but otherwise
executes query operations as normal (and logs a
+warning). `forbidden` will fail queries that would execute triggers.
\ No newline at end of file
diff --git a/src/java/org/apache/cassandra/config/Config.java
b/src/java/org/apache/cassandra/config/Config.java
index feadbc21bd..47f879eb07 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -1300,4 +1300,16 @@ public class Config
public volatile DurationSpec.LongMillisecondsBound
progress_barrier_backoff = new DurationSpec.LongMillisecondsBound("1000ms");
public volatile DurationSpec.LongSecondsBound
[jira] [Commented] (CASSANDRA-19559) prepare_release.sh should check for mvn
[ https://issues.apache.org/jira/browse/CASSANDRA-19559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837618#comment-17837618 ] Berenguer Blasi commented on CASSANDRA-19559: - +1 > prepare_release.sh should check for mvn > --- > > Key: CASSANDRA-19559 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19559 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: NA > > > Part of the 'prepare' phase of releasing includes publishing Maven artifacts, > which requires that it be installed. The script should check for this since > it's quite easy to miss. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19128) The result of applying a metadata snapshot via ForceSnapshot should return the correct set of modified keys
[ https://issues.apache.org/jira/browse/CASSANDRA-19128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-19128: Source Control Link: https://github.com/apache/cassandra/commit/7623e4678b8ef131434f1de3522c6425c092dff9 Resolution: Fixed Status: Resolved (was: Ready to Commit) > The result of applying a metadata snapshot via ForceSnapshot should return > the correct set of modified keys > --- > > Key: CASSANDRA-19128 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19128 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Membership >Reporter: Marcus Eriksson >Assignee: Alex Petrov >Priority: High > Fix For: 5.1-alpha1 > > Attachments: ci_summary-1.html, ci_summary.html > > Time Spent: 50m > Remaining Estimate: 0h > > It should use the same logic as Transformer::build to compare the updated CM > with the previous to derive the modified keys -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19128) The result of applying a metadata snapshot via ForceSnapshot should return the correct set of modified keys
[ https://issues.apache.org/jira/browse/CASSANDRA-19128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-19128: Reviewers: Marcus Eriksson Status: Review In Progress (was: Patch Available) > The result of applying a metadata snapshot via ForceSnapshot should return > the correct set of modified keys > --- > > Key: CASSANDRA-19128 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19128 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Membership >Reporter: Marcus Eriksson >Assignee: Alex Petrov >Priority: High > Fix For: 5.1-alpha1 > > Attachments: ci_summary-1.html, ci_summary.html > > Time Spent: 50m > Remaining Estimate: 0h > > It should use the same logic as Transformer::build to compare the updated CM > with the previous to derive the modified keys -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19128) The result of applying a metadata snapshot via ForceSnapshot should return the correct set of modified keys
[ https://issues.apache.org/jira/browse/CASSANDRA-19128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837615#comment-17837615 ] Alex Petrov commented on CASSANDRA-19128: - [~marcuse] left his +1 on the pull request. > The result of applying a metadata snapshot via ForceSnapshot should return > the correct set of modified keys > --- > > Key: CASSANDRA-19128 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19128 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Membership >Reporter: Marcus Eriksson >Assignee: Alex Petrov >Priority: High > Fix For: 5.1-alpha1 > > Attachments: ci_summary-1.html, ci_summary.html > > Time Spent: 50m > Remaining Estimate: 0h > > It should use the same logic as Transformer::build to compare the updated CM > with the previous to derive the modified keys -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19128) The result of applying a metadata snapshot via ForceSnapshot should return the correct set of modified keys
[ https://issues.apache.org/jira/browse/CASSANDRA-19128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-19128: Status: Ready to Commit (was: Review In Progress) > The result of applying a metadata snapshot via ForceSnapshot should return > the correct set of modified keys > --- > > Key: CASSANDRA-19128 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19128 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Membership >Reporter: Marcus Eriksson >Assignee: Alex Petrov >Priority: High > Fix For: 5.1-alpha1 > > Attachments: ci_summary-1.html, ci_summary.html > > Time Spent: 50m > Remaining Estimate: 0h > > It should use the same logic as Transformer::build to compare the updated CM > with the previous to derive the modified keys -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19560) Implement support for virtual types
[
https://issues.apache.org/jira/browse/CASSANDRA-19560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-19560:
--
Change Category: Operability
Complexity: Normal
Fix Version/s: 5.x
Status: Open (was: Triage Needed)
> Implement support for virtual types
> ---
>
> Key: CASSANDRA-19560
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19560
> Project: Cassandra
> Issue Type: New Feature
> Components: Feature/Virtual Tables
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
>
> We can not use user types in virtual tables. While it might look strange at
> first to ask for the support of this, currently we can not do something like
> this:
> {code}
> cqlsh> select * from system_guardrails.thresholds ;
> name | value
> ---+--
>collection_size | {warn: 0, fail: 0}
> column_value_size | {warn: -1, fail: -1}
> columns_per_table | {warn: -1, fail: -1}
> ...
> {code}
> {code}
> VIRTUAL TABLE system_guardrails.thresholds (
> name text PRIMARY KEY,
> value settings
> ) WITH comment = 'Guardrails configuration table for thresholds';
> {code}
> because "settings" is a UDT
> {code}
> cqlsh> DESCRIBE type system_guardrails.settings ;
> CREATE TYPE system_guardrails.settings (
> warn bigint,
> fail bigint
> );
> {code}
> While this is not absolutely necessary to implement and it might be worked
> around by a simple tuple, it feels sad that user experience suffers.
> Supporting custom types for vtables is indeed possible so we should just do
> that.
> There is additional work needed to do this, because virtual types are not
> supported in python-driver, I had to do this:
> https://github.com/smiklosovic/python-driver/commit/14b236bb471bc4a7e251a9f6b694de7885b339de
> python-driver reads system_schema.types in order to show the correct output
> in cqlsh, as it has not recognized virtual types, it was alwasy displaying
> the results like
> {code}
> settings(warn=-1, fail -1)
> {code}
> because it could not evaluate it differently.
> With the patch for python-driver, it will fetch it from
> system_virtual_schema.types, where it is like:
> {code}
> cqlsh> select * from system_virtual_schema.types ;
> keyspace_name | type_name | field_names | field_types
> ---+---+--+--
> system_guardrails | settings | ['warn', 'fail'] | ['bigint', 'bigint']
> (1 rows)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19344) Range movements involving transient replicas must safely enact changes to read and write replica sets
[
https://issues.apache.org/jira/browse/CASSANDRA-19344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837573#comment-17837573
]
Alex Petrov edited comment on CASSANDRA-19344 at 4/16/24 8:04 AM:
--
Wanted to point out a somewhat unintuitive albeit correct behaviour that
involves Transient Replicas. I think it is worth talking through such things
because pending ranges with transient replicas work slightly differently from
their "normal" counterparts.
We have a four node cluster with nodes 1,2,3,4 owning tokens 100,200,300,400,
and 4 moving from 400 to 350.
Original/start state (READ/WRITE placements):
{code}
(400,MIN] -> [Full(/127.0.0.1:7012,(400,MIN]),
Full(/127.0.0.2:7012,(400,MIN]), Transient(/127.0.0.3:7012,(400,MIN])]}
(MIN,100] -> [Full(/127.0.0.1:7012,(MIN,100]),
Full(/127.0.0.2:7012,(MIN,100]), Transient(/127.0.0.3:7012,(MIN,100])]}
(100,200] -> [Full(/127.0.0.2:7012,(100,200]),
Full(/127.0.0.3:7012,(100,200]), Transient(/127.0.0.4:7012,(100,200])]}
(200,300] -> [Full(/127.0.0.3:7012,(200,300]),
Full(/127.0.0.4:7012,(200,300]), Transient(/127.0.0.1:7012,(200,300])]}
(300,350] -> [Full(/127.0.0.1:7012,(300,350]),
Transient(/127.0.0.2:7012,(300,350]), Full(/127.0.0.4:7012,(300,350])]}
(350,400] -> [Full(/127.0.0.4:7012,(350,400]),
Full(/127.0.0.1:7012,(350,400]), Transient(/127.0.0.2:7012,(350,400])]}
{code}
State after {{START_MOVE}} (which is the point at which streaming starts, so
think of additional replicas as pending), for WRITE placements:
{code}
(400,MIN] -> [Full(/127.0.0.1:7012,(400,MIN]),
Full(/127.0.0.2:7012,(400,MIN]), Full(/127.0.0.3:701a2,(400,MIN])]}
(MIN,100] -> [Full(/127.0.0.1:7012,(MIN,100]),
Full(/127.0.0.2:7012,(MIN,100]), Full(/127.0.0.3:7012,(MIN,100])]}
(100,200] -> [Full(/127.0.0.2:7012,(100,200]),
Full(/127.0.0.3:7012,(100,200]), Transient(/127.0.0.4:7012,(100,200]),
Transient(/127.0.0.1:7012,(100,200])]}
(200,300] -> [Full(/127.0.0.3:7012,(200,300]),
Full(/127.0.0.4:7012,(200,300]), Full(/127.0.0.1:7012,(200,300])]}
(300,350] -> [Full(/127.0.0.1:7012,(300,350]),
Transient(/127.0.0.2:7012,(300,350]), Full(/127.0.0.4:7012,(300,350])]}
(350,400] -> [Full(/127.0.0.4:7012,(350,400]),
Full(/127.0.0.1:7012,(350,400]), Full(/127.0.0.2:7012,(350,400]),
Transient(/127.0.0.3:7012,(350,400])]}
{code}
READ placements at the same moment:
{code}
(400,MIN] -> [Transient(/127.0.0.1:7012,(400,MIN]),
Full(/127.0.0.2:7012,(400,MIN]), Full(/127.0.0.3:7012,(400,MIN])]}
(MIN,100] -> [Transient(/127.0.0.1:7012,(MIN,100]),
Full(/127.0.0.2:7012,(MIN,100]), Full(/127.0.0.3:7012,(MIN,100])]}
(100,200] -> [Full(/127.0.0.2:7012,(100,200]),
Full(/127.0.0.3:7012,(100,200]), Transient(/127.0.0.1:7012,(100,200])]}
(200,300] -> [Full(/127.0.0.3:7012,(200,300]),
Full(/127.0.0.1:7012,(200,300]), Transient(/127.0.0.4:7012,(200,300])]}
(300,350] -> [Full(/127.0.0.1:7012,(300,350]),
Transient(/127.0.0.2:7012,(300,350]), Full(/127.0.0.4:7012,(300,350])]}
(350,400] -> [Full(/127.0.0.4:7012,(350,400]),
Full(/127.0.0.2:7012,(350,400]), Transient(/127.0.0.3:7012,(350,400])]}
{code}
Please note that READ placements are always a subset of WRITE ones (or, well,
in a way: we can technically read from full to satisfy a transient read)
after FINISH_MOVE, we get for both READ and WRITE:
{code}
(400,MIN] -> [Full(/127.0.0.2:7012,(400,MIN]),
Full(/127.0.0.3:7012,(400,MIN]), Transient(/127.0.0.1:7012,(400,MIN])]}
(MIN,200] -> [Full(/127.0.0.2:7012,(MIN,200]),
Transient(/127.0.0.1:7012,(MIN,200]), Full(/127.0.0.3:7012,(MIN,200])]}
(200,300] -> [Full(/127.0.0.3:7012,(200,300]),
Full(/127.0.0.1:7012,(200,300]), Transient(/127.0.0.4:7012,(200,300])]}
(300,350] -> [Full(/127.0.0.1:7012,(300,350]),
Transient(/127.0.0.2:7012,(300,350]), Full(/127.0.0.4:7012,(300,350])]}
(350,400] -> [Full(/127.0.0.4:7012,(350,400]),
Full(/127.0.0.2:7012,(350,400]), Transient(/127.0.0.3:7012,(350,400])]}
{code}
After executing START_MOVE, we get 3 full and no transient nodes for
{{(200,300]}}. If we put transitions together, we see:
{code}
1. (200,300] -> [Full(/127.0.0.3:7012,(200,300]),
Full(/127.0.0.4:7012,(200,300]), Transient(/127.0.0.1:7012,(200,300])]}
2. (200,300] -> [Full(/127.0.0.3:7012,(200,300]),
Full(/127.0.0.4:7012,(200,300]), Full(/127.0.0.1:7012,(200,300])]}
3. (200,300] -> [Full(/127.0.0.3:7012,(200,300]),
Full(/127.0.0.1:7012,(200,300]), Transient(/127.0.0.4:7012,(200,300])]}
{code}
In {{2.}}, you see that {{127.0.0.1}} went from transient to full, since it is
now gaining a range, and should be a target for pending writes for this range.
At the same time, it remains a _transient read replica_. In {{3.}},
{{127.0.0.04}} went from full to transient; it was kept full up till now since
it was a streaming source, and to keep consistency levels correct, we
What is unintuitive here
[jira] [Commented] (CASSANDRA-19344) Range movements involving transient replicas must safely enact changes to read and write replica sets
[
https://issues.apache.org/jira/browse/CASSANDRA-19344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837573#comment-17837573
]
Alex Petrov commented on CASSANDRA-19344:
-
Wanted to point out a somewhat unintuitive albeit correct behaviour that
involves Transient Replicas. I think it is worth talking through such things
because pending ranges with transient replicas work slightly differently from
their "normal" counterparts.
We have a four node cluster with nodes 1,2,3,4 owning tokens 100,200,300,400,
and 4 moving from 400 to 350.
Original/start state (READ/WRITE placements):
{code]
(400,MIN] -> [Full(/127.0.0.1:7012,(400,MIN]),
Full(/127.0.0.2:7012,(400,MIN]), Transient(/127.0.0.3:7012,(400,MIN])]}
(MIN,100] -> [Full(/127.0.0.1:7012,(MIN,100]),
Full(/127.0.0.2:7012,(MIN,100]), Transient(/127.0.0.3:7012,(MIN,100])]}
(100,200] -> [Full(/127.0.0.2:7012,(100,200]),
Full(/127.0.0.3:7012,(100,200]), Transient(/127.0.0.4:7012,(100,200])]}
(200,300] -> [Full(/127.0.0.3:7012,(200,300]),
Full(/127.0.0.4:7012,(200,300]), Transient(/127.0.0.1:7012,(200,300])]}
(300,350] -> [Full(/127.0.0.1:7012,(300,350]),
Transient(/127.0.0.2:7012,(300,350]), Full(/127.0.0.4:7012,(300,350])]}
(350,400] -> [Full(/127.0.0.4:7012,(350,400]),
Full(/127.0.0.1:7012,(350,400]), Transient(/127.0.0.2:7012,(350,400])]}
{code}
State after {{START_MOVE}} (which is the point at which streaming starts, so
think of additional replicas as pending), for WRITE placements:
{code}
(400,MIN] -> [Full(/127.0.0.1:7012,(400,MIN]),
Full(/127.0.0.2:7012,(400,MIN]), Full(/127.0.0.3:701a2,(400,MIN])]}
(MIN,100] -> [Full(/127.0.0.1:7012,(MIN,100]),
Full(/127.0.0.2:7012,(MIN,100]), Full(/127.0.0.3:7012,(MIN,100])]}
(100,200] -> [Full(/127.0.0.2:7012,(100,200]),
Full(/127.0.0.3:7012,(100,200]), Transient(/127.0.0.4:7012,(100,200]),
Transient(/127.0.0.1:7012,(100,200])]}
(200,300] -> [Full(/127.0.0.3:7012,(200,300]),
Full(/127.0.0.4:7012,(200,300]), Full(/127.0.0.1:7012,(200,300])]}
(300,350] -> [Full(/127.0.0.1:7012,(300,350]),
Transient(/127.0.0.2:7012,(300,350]), Full(/127.0.0.4:7012,(300,350])]}
(350,400] -> [Full(/127.0.0.4:7012,(350,400]),
Full(/127.0.0.1:7012,(350,400]), Full(/127.0.0.2:7012,(350,400]),
Transient(/127.0.0.3:7012,(350,400])]}
{code}
READ placements at the same moment:
{code}
(400,MIN] -> [Transient(/127.0.0.1:7012,(400,MIN]),
Full(/127.0.0.2:7012,(400,MIN]), Full(/127.0.0.3:7012,(400,MIN])]}
(MIN,100] -> [Transient(/127.0.0.1:7012,(MIN,100]),
Full(/127.0.0.2:7012,(MIN,100]), Full(/127.0.0.3:7012,(MIN,100])]}
(100,200] -> [Full(/127.0.0.2:7012,(100,200]),
Full(/127.0.0.3:7012,(100,200]), Transient(/127.0.0.1:7012,(100,200])]}
(200,300] -> [Full(/127.0.0.3:7012,(200,300]),
Full(/127.0.0.1:7012,(200,300]), Transient(/127.0.0.4:7012,(200,300])]}
(300,350] -> [Full(/127.0.0.1:7012,(300,350]),
Transient(/127.0.0.2:7012,(300,350]), Full(/127.0.0.4:7012,(300,350])]}
(350,400] -> [Full(/127.0.0.4:7012,(350,400]),
Full(/127.0.0.2:7012,(350,400]), Transient(/127.0.0.3:7012,(350,400])]}
{code}
Please note that READ placements are always a subset of WRITE ones (or, well,
in a way: we can technically read from full to satisfy a transient read)
after FINISH_MOVE, we get for both READ and WRITE:
{code}
(400,MIN] -> [Full(/127.0.0.2:7012,(400,MIN]),
Full(/127.0.0.3:7012,(400,MIN]), Transient(/127.0.0.1:7012,(400,MIN])]}
(MIN,200] -> [Full(/127.0.0.2:7012,(MIN,200]),
Transient(/127.0.0.1:7012,(MIN,200]), Full(/127.0.0.3:7012,(MIN,200])]}
(200,300] -> [Full(/127.0.0.3:7012,(200,300]),
Full(/127.0.0.1:7012,(200,300]), Transient(/127.0.0.4:7012,(200,300])]}
(300,350] -> [Full(/127.0.0.1:7012,(300,350]),
Transient(/127.0.0.2:7012,(300,350]), Full(/127.0.0.4:7012,(300,350])]}
(350,400] -> [Full(/127.0.0.4:7012,(350,400]),
Full(/127.0.0.2:7012,(350,400]), Transient(/127.0.0.3:7012,(350,400])]}
{code}
After executing START_MOVE, we get 3 full and no transient nodes for
{{(200,300]}}. If we put transitions together, we see:
{code}
1. (200,300] -> [Full(/127.0.0.3:7012,(200,300]),
Full(/127.0.0.4:7012,(200,300]), Transient(/127.0.0.1:7012,(200,300])]}
2. (200,300] -> [Full(/127.0.0.3:7012,(200,300]),
Full(/127.0.0.4:7012,(200,300]), Full(/127.0.0.1:7012,(200,300])]}
3. (200,300] -> [Full(/127.0.0.3:7012,(200,300]),
Full(/127.0.0.1:7012,(200,300]), Transient(/127.0.0.4:7012,(200,300])]}
{code}
In {{2.}}, you see that {{127.0.0.1}} went from transient to full, since it is
now gaining a range, and should be a target for pending writes for this range.
At the same time, it remains a _transient read replica_. In {{3.}},
{{127.0.0.04}} went from full to transient; it was kept full up till now since
it was a streaming source, and to keep consistency levels correct, we
What is unintuitive here is that usually, with replication factor of 3, we
[jira] [Updated] (CASSANDRA-19190) ForceSnapshot transformations should not be persisted in the local log table
[
https://issues.apache.org/jira/browse/CASSANDRA-19190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-19190:
Attachment: ci_summary.html
> ForceSnapshot transformations should not be persisted in the local log table
>
>
> Key: CASSANDRA-19190
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19190
> Project: Cassandra
> Issue Type: Improvement
> Components: Transactional Cluster Metadata
>Reporter: Marcus Eriksson
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 5.1-alpha1
>
> Attachments: ci_summary.html
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Per its inline comments, ForceSnapshot is a synthetic transformation whose
> purpose it to enable the local log to jump missing epochs. A common use for
> this is when replaying persisted events from the metadata log at startup. The
> log is initialised with {{Epoch.EMPTY}} but rather that replaying every
> single entry since the beginning of history, we select the most recent
> snapshot held locally and start the replay from that point. Likewise, when
> catching up from a peer, a node may receive a snapshot plus subsequent log
> entries. In order to bring local metadata to the same state as the snapshot,
> a {{ForceSnapshot}} with the same epoch as the snapshot is inserted into the
> {{LocalLog}} and enacted like any other other transformation. These synthetic
> transformations should not be persisted in the `system.local_metadata_log`,
> as they do not exist in the distributed metadata log. We _should_ persist the
> snapshot itself in {{system.metadata_snapshots}} so that we can avoid having
> to re-fetch remote snapshots (i.e. if a node were to restart shortly after
> receiving a catchup from a peer).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19190) ForceSnapshot transformations should not be persisted in the local log table
[
https://issues.apache.org/jira/browse/CASSANDRA-19190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-19190:
Test and Documentation Plan: ci run
Status: Patch Available (was: Open)
ci looks bad right now, will rerun after CASSANDRA-19514 has been committed
> ForceSnapshot transformations should not be persisted in the local log table
>
>
> Key: CASSANDRA-19190
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19190
> Project: Cassandra
> Issue Type: Improvement
> Components: Transactional Cluster Metadata
>Reporter: Marcus Eriksson
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 5.1-alpha1
>
> Attachments: ci_summary.html
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Per its inline comments, ForceSnapshot is a synthetic transformation whose
> purpose it to enable the local log to jump missing epochs. A common use for
> this is when replaying persisted events from the metadata log at startup. The
> log is initialised with {{Epoch.EMPTY}} but rather that replaying every
> single entry since the beginning of history, we select the most recent
> snapshot held locally and start the replay from that point. Likewise, when
> catching up from a peer, a node may receive a snapshot plus subsequent log
> entries. In order to bring local metadata to the same state as the snapshot,
> a {{ForceSnapshot}} with the same epoch as the snapshot is inserted into the
> {{LocalLog}} and enacted like any other other transformation. These synthetic
> transformations should not be persisted in the `system.local_metadata_log`,
> as they do not exist in the distributed metadata log. We _should_ persist the
> snapshot itself in {{system.metadata_snapshots}} so that we can avoid having
> to re-fetch remote snapshots (i.e. if a node were to restart shortly after
> receiving a catchup from a peer).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19551) CCM nodes share the same environment variable map breaking upgrade tests
[
https://issues.apache.org/jira/browse/CASSANDRA-19551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837534#comment-17837534
]
Berenguer Blasi commented on CASSANDRA-19551:
-
Let's take a look and use
[this|https://ci-cassandra.apache.org/job/Cassandra-trunk/1859/#showFailuresLink
as baseline:
- distributed.test.log.FetchLogFromPeersTest: Timeout. Run locally and see if
it passes. Fails on baseline as well.
- fuzz.harry.integration.model.ConcurrentQuiescentCheckerIntegrationTest: Can't
help here but I don't think ccm plays a part.
- fuzz.ring.ConsistentBootstrapTest: Can't help here but I don't think ccm
plays a part.
- distributed.upgrade.ClusterMetadataUpgradeHarryTest: Can't help here but I
don't think ccm plays a part.
- distributed.test.log.FetchLogFromPeersTest: Timeout. Run locally and see if
it passes. Fails on baseline as well.
- fuzz.ring.ConsistentBootstrapTest: Timeout. Run locally and see if it passes
- distributed.upgrade.ClusterMetadataUpgradeHarryTest: Can't help here but I
don't think ccm plays a part.
- bootstrap_test.TestBootstrap: CASSANDRA-18098
- gossip_test.TestGossip: They fail on baseline and CASSANDRA-19538
- largecolumn_test.TestLargeColumn: Where do you see it failing consistenly?
-
upgrade_tests.upgrade_through_versions_test.TestProtoV3Upgrade_AllVersions_EndsAt_Trunk_HEAD:
Counters in upgrades are known problems CASSANDRA-19520
^If my checks are correct I don't think your PR introduces new failures. CI is
in a sorry state and I don't think we can do any better.
> CCM nodes share the same environment variable map breaking upgrade tests
>
>
> Key: CASSANDRA-19551
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19551
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 5.x
>
> Attachments: ci_summary.html
>
>
> In {{node.py}} {{__environment_variables}} is generally always set with a map
> that is passed in from {{cluster.py}} so it is [shared between
> nodes](https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L151)
> and if nodes modify the map, such as in {{start}} when [updating the Java
> version](https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L860)
> then when {{get_env}} runs it will [overwrite the Java
> version](https://github.com/riptano/ccm/blob/ac264706c8ca007cc584871ce907d48db334d36d/ccmlib/node.py#L244)
> that is selected by {{update_java_version}}.
> This results in {{nodetool drain}} failing when upgrading from 3.11 to 4.0 in
> some of the upgrade tests because after the first node upgrades to 4.0 it's
> not longer possible for the subsequent nodes to select a Java version that
> isn't 11 because it's overridden by {{__environment_variables}}.
> I'm not even 100% clear on why the code in {{start}} should update
> {{__environment_variables}} at all if we calculate the correct java version
> on every invocation of other tools.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19538) Test Failure: test_assassinate_valid_node
[
https://issues.apache.org/jira/browse/CASSANDRA-19538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-19538:
Attachment: ci_summary.html
> Test Failure: test_assassinate_valid_node
> -
>
> Key: CASSANDRA-19538
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19538
> Project: Cassandra
> Issue Type: Bug
> Components: CI, Test/dtest/python
>Reporter: Ekaterina Dimitrova
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 5.x
>
> Attachments: ci_summary.html
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Failing consistently on trunk:
> {code:java}
> ccmlib.node.TimeoutError: 03 Apr 2024 19:39:32 [node1] after 120.11/120
> seconds Missing: ['127.0.0.4:7000.* is now UP'] not found in system.log:
> Head: INFO [Messaging-EventLoop-3-1] 2024-04-03 19:37:3
> Tail: ... some nodes were not ready
> INFO [OptionalTasks:1] 2024-04-03 19:39:30,454 CassandraRoleManager.java:484
> - Setup task failed with error, rescheduling
> self =
> def test_assassinate_valid_node(self):
> """
> @jira_ticket CASSANDRA-16588
> Test that after taking two non-seed nodes down and assassinating
> one of them, the other can come back up.
> """
> cluster = self.cluster
>
> cluster.populate(5).start()
> node1 = cluster.nodelist()[0]
> node3 = cluster.nodelist()[2]
>
> self.cluster.set_configuration_options({
> 'seed_provider': [{'class_name':
> 'org.apache.cassandra.locator.SimpleSeedProvider',
>'parameters': [{'seeds': node1.address()}]
> }]
> })
>
> non_seed_nodes = cluster.nodelist()[-2:]
> for node in non_seed_nodes:
> node.stop()
>
> assassination_target = non_seed_nodes[0]
> logger.debug("Assassinating non-seed node
> {}".format(assassination_target.address()))
> out, err, _ = node1.nodetool("assassinate
> {}".format(assassination_target.address()))
> assert_stderr_clean(err)
>
> logger.debug("Starting non-seed nodes")
> for node in non_seed_nodes:
> > node.start()
> gossip_test.py:78:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> ../env3.8/lib/python3.8/site-packages/ccmlib/node.py:915: in start
> node.watch_log_for_alive(self, from_mark=mark)
> ../env3.8/lib/python3.8/site-packages/ccmlib/node.py:684: in
> watch_log_for_alive
> self.watch_log_for(tofind, from_mark=from_mark, timeout=timeout,
> filename=filename)
> ../env3.8/lib/python3.8/site-packages/ccmlib/node.py:608: in watch_log_for
> TimeoutError.raise_if_passed(start=start, timeout=timeout, node=self.name,
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> start = 1712173052.8186479, timeout = 120
> msg = "Missing: ['127.0.0.4:7000.* is now UP'] not found in system.log:\n
> Head: INFO [Messaging-EventLoop-3-1] 2024-04-03 1...[OptionalTasks:1]
> 2024-04-03 19:39:30,454 CassandraRoleManager.java:484 - Setup task failed
> with error, rescheduling\n"
> node = 'node1'
> @staticmethod
> def raise_if_passed(start, timeout, msg, node=None):
> if start + timeout < time.time():
> > raise TimeoutError.create(start, timeout, msg, node)
> E ccmlib.node.TimeoutError: 03 Apr 2024 19:39:32 [node1] after
> 120.11/120 seconds Missing: ['127.0.0.4:7000.* is now UP'] not found in
> system.log:
> EHead: INFO [Messaging-EventLoop-3-1] 2024-04-03 19:37:3
> ETail: ... some nodes were not ready
> E INFO [OptionalTasks:1] 2024-04-03 19:39:30,454
> CassandraRoleManager.java:484 - Setup task failed with error, rescheduling
> ../env3.8/lib/python3.8/site-packages/ccmlib/node.py:56: TimeoutError
> {code}
> https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2680/workflows/8b1c0d0a-7458-4b43-9bba-ac96b9bfe64f/jobs/58929/tests#failed-test-0
> https://ci-cassandra.apache.org/job/Cassandra-trunk/1859/#showFailuresLink
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19538) Test Failure: test_assassinate_valid_node
[
https://issues.apache.org/jira/browse/CASSANDRA-19538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-19538:
Test and Documentation Plan: ci run
Status: Patch Available (was: Open)
> Test Failure: test_assassinate_valid_node
> -
>
> Key: CASSANDRA-19538
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19538
> Project: Cassandra
> Issue Type: Bug
> Components: CI, Test/dtest/python
>Reporter: Ekaterina Dimitrova
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 5.x
>
> Attachments: ci_summary.html
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Failing consistently on trunk:
> {code:java}
> ccmlib.node.TimeoutError: 03 Apr 2024 19:39:32 [node1] after 120.11/120
> seconds Missing: ['127.0.0.4:7000.* is now UP'] not found in system.log:
> Head: INFO [Messaging-EventLoop-3-1] 2024-04-03 19:37:3
> Tail: ... some nodes were not ready
> INFO [OptionalTasks:1] 2024-04-03 19:39:30,454 CassandraRoleManager.java:484
> - Setup task failed with error, rescheduling
> self =
> def test_assassinate_valid_node(self):
> """
> @jira_ticket CASSANDRA-16588
> Test that after taking two non-seed nodes down and assassinating
> one of them, the other can come back up.
> """
> cluster = self.cluster
>
> cluster.populate(5).start()
> node1 = cluster.nodelist()[0]
> node3 = cluster.nodelist()[2]
>
> self.cluster.set_configuration_options({
> 'seed_provider': [{'class_name':
> 'org.apache.cassandra.locator.SimpleSeedProvider',
>'parameters': [{'seeds': node1.address()}]
> }]
> })
>
> non_seed_nodes = cluster.nodelist()[-2:]
> for node in non_seed_nodes:
> node.stop()
>
> assassination_target = non_seed_nodes[0]
> logger.debug("Assassinating non-seed node
> {}".format(assassination_target.address()))
> out, err, _ = node1.nodetool("assassinate
> {}".format(assassination_target.address()))
> assert_stderr_clean(err)
>
> logger.debug("Starting non-seed nodes")
> for node in non_seed_nodes:
> > node.start()
> gossip_test.py:78:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> ../env3.8/lib/python3.8/site-packages/ccmlib/node.py:915: in start
> node.watch_log_for_alive(self, from_mark=mark)
> ../env3.8/lib/python3.8/site-packages/ccmlib/node.py:684: in
> watch_log_for_alive
> self.watch_log_for(tofind, from_mark=from_mark, timeout=timeout,
> filename=filename)
> ../env3.8/lib/python3.8/site-packages/ccmlib/node.py:608: in watch_log_for
> TimeoutError.raise_if_passed(start=start, timeout=timeout, node=self.name,
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> start = 1712173052.8186479, timeout = 120
> msg = "Missing: ['127.0.0.4:7000.* is now UP'] not found in system.log:\n
> Head: INFO [Messaging-EventLoop-3-1] 2024-04-03 1...[OptionalTasks:1]
> 2024-04-03 19:39:30,454 CassandraRoleManager.java:484 - Setup task failed
> with error, rescheduling\n"
> node = 'node1'
> @staticmethod
> def raise_if_passed(start, timeout, msg, node=None):
> if start + timeout < time.time():
> > raise TimeoutError.create(start, timeout, msg, node)
> E ccmlib.node.TimeoutError: 03 Apr 2024 19:39:32 [node1] after
> 120.11/120 seconds Missing: ['127.0.0.4:7000.* is now UP'] not found in
> system.log:
> EHead: INFO [Messaging-EventLoop-3-1] 2024-04-03 19:37:3
> ETail: ... some nodes were not ready
> E INFO [OptionalTasks:1] 2024-04-03 19:39:30,454
> CassandraRoleManager.java:484 - Setup task failed with error, rescheduling
> ../env3.8/lib/python3.8/site-packages/ccmlib/node.py:56: TimeoutError
> {code}
> https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2680/workflows/8b1c0d0a-7458-4b43-9bba-ac96b9bfe64f/jobs/58929/tests#failed-test-0
> https://ci-cassandra.apache.org/job/Cassandra-trunk/1859/#showFailuresLink
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
