[jira] [Commented] (CASSANDRA-19429) Remove lock contention generated by getCapacity function in SSTableReader
[
https://issues.apache.org/jira/browse/CASSANDRA-19429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824889#comment-17824889
]
Jon Haddad commented on CASSANDRA-19429:
Unfortunately I don't have the time or a budget to do anymore testing on this.
I'm still not sure why [~dipiets] was unable to push Cassandra past 50% CPU.
I'm seeing 72 CPUs at 90%+ on my tests. My CPU flame graphs are virtually
identical between the two versions as well.
It would be better if we could demonstrate the difference using JMH, as it's
pretty expensive for me to fire up those big nodes.
> Remove lock contention generated by getCapacity function in SSTableReader
> -
>
> Key: CASSANDRA-19429
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19429
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Dipietro Salvatore
>Assignee: Dipietro Salvatore
>Priority: Normal
> Fix For: 4.0.x, 4.1.x
>
> Attachments: Screenshot 2024-02-26 at 10.27.10.png, Screenshot
> 2024-02-27 at 11.29.41.png, asprof_cass4.1.3__lock_20240216052912lock.html,
> image-2024-03-08-15-51-30-439.png, image-2024-03-08-15-52-07-902.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Profiling Cassandra 4.1.3 on large AWS instances, a high number of lock
> acquires is measured in the `getCapacity` function from
> `org/apache/cassandra/cache/InstrumentingCache` (1.9M lock acquires per 60
> seconds). Based on our tests on r8g.24xlarge instances (using Ubuntu 22.04),
> this limits the CPU utilization of the system to under 50% when testing at
> full load and therefore limits the achieved throughput.
> Removing the lock contention from the SSTableReader.java file by replacing
> the call to `getCapacity` with `size` achieves up to 2.95x increase in
> throughput on r8g.24xlarge and 2x on r7i.24xlarge:
> |Instance type|Cass 4.1.3|Cass 4.1.3 patched|
> |r8g.24xlarge|168k ops|496k ops (2.95x)|
> |r7i.24xlarge|153k ops|304k ops (1.98x)|
>
> Instructions to reproduce:
> {code:java}
> ## Requirements for Ubuntu 22.04
> sudo apt install -y ant git openjdk-11-jdk
> ## Build and run
> CASSANDRA_USE_JDK11=true ant realclean && CASSANDRA_USE_JDK11=true ant jar &&
> CASSANDRA_USE_JDK11=true ant stress-build && rm -rf data && bin/cassandra -f
> -R
> # Run
> bin/cqlsh -e 'drop table if exists keyspace1.standard1;' && \
> bin/cqlsh -e 'drop keyspace if exists keyspace1;' && \
> bin/nodetool clearsnapshot --all && tools/bin/cassandra-stress write
> n=1000 cl=ONE -rate threads=384 -node 127.0.0.1 -log file=cload.log
> -graph file=cload.html && \
> bin/nodetool compact keyspace1 && sleep 30s && \
> tools/bin/cassandra-stress mixed ratio\(write=10,read=90\) duration=10m
> cl=ONE -rate threads=406 -node localhost -log file=result.log -graph
> file=graph.html
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19429) Remove lock contention generated by getCapacity function in SSTableReader
[
https://issues.apache.org/jira/browse/CASSANDRA-19429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824886#comment-17824886
]
Jon Haddad commented on CASSANDRA-19429:
I've tried several different workloads and I am unable to replicate the
performance results from above. 4.1 and the branch are behaving almost
identically for both workloads.
4.1
!image-2024-03-08-15-51-30-439.png!
branch:
!image-2024-03-08-15-52-07-902.png!
> Remove lock contention generated by getCapacity function in SSTableReader
> -
>
> Key: CASSANDRA-19429
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19429
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Dipietro Salvatore
>Assignee: Dipietro Salvatore
>Priority: Normal
> Fix For: 4.0.x, 4.1.x
>
> Attachments: Screenshot 2024-02-26 at 10.27.10.png, Screenshot
> 2024-02-27 at 11.29.41.png, asprof_cass4.1.3__lock_20240216052912lock.html,
> image-2024-03-08-15-51-30-439.png, image-2024-03-08-15-52-07-902.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Profiling Cassandra 4.1.3 on large AWS instances, a high number of lock
> acquires is measured in the `getCapacity` function from
> `org/apache/cassandra/cache/InstrumentingCache` (1.9M lock acquires per 60
> seconds). Based on our tests on r8g.24xlarge instances (using Ubuntu 22.04),
> this limits the CPU utilization of the system to under 50% when testing at
> full load and therefore limits the achieved throughput.
> Removing the lock contention from the SSTableReader.java file by replacing
> the call to `getCapacity` with `size` achieves up to 2.95x increase in
> throughput on r8g.24xlarge and 2x on r7i.24xlarge:
> |Instance type|Cass 4.1.3|Cass 4.1.3 patched|
> |r8g.24xlarge|168k ops|496k ops (2.95x)|
> |r7i.24xlarge|153k ops|304k ops (1.98x)|
>
> Instructions to reproduce:
> {code:java}
> ## Requirements for Ubuntu 22.04
> sudo apt install -y ant git openjdk-11-jdk
> ## Build and run
> CASSANDRA_USE_JDK11=true ant realclean && CASSANDRA_USE_JDK11=true ant jar &&
> CASSANDRA_USE_JDK11=true ant stress-build && rm -rf data && bin/cassandra -f
> -R
> # Run
> bin/cqlsh -e 'drop table if exists keyspace1.standard1;' && \
> bin/cqlsh -e 'drop keyspace if exists keyspace1;' && \
> bin/nodetool clearsnapshot --all && tools/bin/cassandra-stress write
> n=1000 cl=ONE -rate threads=384 -node 127.0.0.1 -log file=cload.log
> -graph file=cload.html && \
> bin/nodetool compact keyspace1 && sleep 30s && \
> tools/bin/cassandra-stress mixed ratio\(write=10,read=90\) duration=10m
> cl=ONE -rate threads=406 -node localhost -log file=result.log -graph
> file=graph.html
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19429) Remove lock contention generated by getCapacity function in SSTableReader
[
https://issues.apache.org/jira/browse/CASSANDRA-19429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-19429:
---
Attachment: image-2024-03-08-15-52-07-902.png
> Remove lock contention generated by getCapacity function in SSTableReader
> -
>
> Key: CASSANDRA-19429
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19429
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Dipietro Salvatore
>Assignee: Dipietro Salvatore
>Priority: Normal
> Fix For: 4.0.x, 4.1.x
>
> Attachments: Screenshot 2024-02-26 at 10.27.10.png, Screenshot
> 2024-02-27 at 11.29.41.png, asprof_cass4.1.3__lock_20240216052912lock.html,
> image-2024-03-08-15-51-30-439.png, image-2024-03-08-15-52-07-902.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Profiling Cassandra 4.1.3 on large AWS instances, a high number of lock
> acquires is measured in the `getCapacity` function from
> `org/apache/cassandra/cache/InstrumentingCache` (1.9M lock acquires per 60
> seconds). Based on our tests on r8g.24xlarge instances (using Ubuntu 22.04),
> this limits the CPU utilization of the system to under 50% when testing at
> full load and therefore limits the achieved throughput.
> Removing the lock contention from the SSTableReader.java file by replacing
> the call to `getCapacity` with `size` achieves up to 2.95x increase in
> throughput on r8g.24xlarge and 2x on r7i.24xlarge:
> |Instance type|Cass 4.1.3|Cass 4.1.3 patched|
> |r8g.24xlarge|168k ops|496k ops (2.95x)|
> |r7i.24xlarge|153k ops|304k ops (1.98x)|
>
> Instructions to reproduce:
> {code:java}
> ## Requirements for Ubuntu 22.04
> sudo apt install -y ant git openjdk-11-jdk
> ## Build and run
> CASSANDRA_USE_JDK11=true ant realclean && CASSANDRA_USE_JDK11=true ant jar &&
> CASSANDRA_USE_JDK11=true ant stress-build && rm -rf data && bin/cassandra -f
> -R
> # Run
> bin/cqlsh -e 'drop table if exists keyspace1.standard1;' && \
> bin/cqlsh -e 'drop keyspace if exists keyspace1;' && \
> bin/nodetool clearsnapshot --all && tools/bin/cassandra-stress write
> n=1000 cl=ONE -rate threads=384 -node 127.0.0.1 -log file=cload.log
> -graph file=cload.html && \
> bin/nodetool compact keyspace1 && sleep 30s && \
> tools/bin/cassandra-stress mixed ratio\(write=10,read=90\) duration=10m
> cl=ONE -rate threads=406 -node localhost -log file=result.log -graph
> file=graph.html
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19429) Remove lock contention generated by getCapacity function in SSTableReader
[
https://issues.apache.org/jira/browse/CASSANDRA-19429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-19429:
---
Attachment: image-2024-03-08-15-51-30-439.png
> Remove lock contention generated by getCapacity function in SSTableReader
> -
>
> Key: CASSANDRA-19429
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19429
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Dipietro Salvatore
>Assignee: Dipietro Salvatore
>Priority: Normal
> Fix For: 4.0.x, 4.1.x
>
> Attachments: Screenshot 2024-02-26 at 10.27.10.png, Screenshot
> 2024-02-27 at 11.29.41.png, asprof_cass4.1.3__lock_20240216052912lock.html,
> image-2024-03-08-15-51-30-439.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Profiling Cassandra 4.1.3 on large AWS instances, a high number of lock
> acquires is measured in the `getCapacity` function from
> `org/apache/cassandra/cache/InstrumentingCache` (1.9M lock acquires per 60
> seconds). Based on our tests on r8g.24xlarge instances (using Ubuntu 22.04),
> this limits the CPU utilization of the system to under 50% when testing at
> full load and therefore limits the achieved throughput.
> Removing the lock contention from the SSTableReader.java file by replacing
> the call to `getCapacity` with `size` achieves up to 2.95x increase in
> throughput on r8g.24xlarge and 2x on r7i.24xlarge:
> |Instance type|Cass 4.1.3|Cass 4.1.3 patched|
> |r8g.24xlarge|168k ops|496k ops (2.95x)|
> |r7i.24xlarge|153k ops|304k ops (1.98x)|
>
> Instructions to reproduce:
> {code:java}
> ## Requirements for Ubuntu 22.04
> sudo apt install -y ant git openjdk-11-jdk
> ## Build and run
> CASSANDRA_USE_JDK11=true ant realclean && CASSANDRA_USE_JDK11=true ant jar &&
> CASSANDRA_USE_JDK11=true ant stress-build && rm -rf data && bin/cassandra -f
> -R
> # Run
> bin/cqlsh -e 'drop table if exists keyspace1.standard1;' && \
> bin/cqlsh -e 'drop keyspace if exists keyspace1;' && \
> bin/nodetool clearsnapshot --all && tools/bin/cassandra-stress write
> n=1000 cl=ONE -rate threads=384 -node 127.0.0.1 -log file=cload.log
> -graph file=cload.html && \
> bin/nodetool compact keyspace1 && sleep 30s && \
> tools/bin/cassandra-stress mixed ratio\(write=10,read=90\) duration=10m
> cl=ONE -rate threads=406 -node localhost -log file=result.log -graph
> file=graph.html
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19429) Remove lock contention generated by getCapacity function in SSTableReader
[
https://issues.apache.org/jira/browse/CASSANDRA-19429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824885#comment-17824885
]
Jon Haddad edited comment on CASSANDRA-19429 at 3/8/24 11:37 PM:
-
When I try to spin up those instance types in us-west-2 I get an error that
they're invalid, so I'm running a test with c5.18xlarge.
I'm working with a single node, with compaction disabled, and I reduced my
memtable space to 16MB in order to constantly flush. I wrote 10m rows and have
1928 SStables. These boxes have 72 CPU. I'm using G1GC with a 24GB heap.
I've tested concurrent_reads at 64 and 128 since there's enough cores on here
to handle and we don't need to bottleneck on reads.
So, right off the bat, I'm not able to duplicate the original observation about
CPU not going over 50% utilization. 4.1 has reached 90+% CPU utilization
consistently:
{noformat}
23:29:57 CPU %usr %nice %sys %iowait %irq %soft %steal
%guest %gnice %idle
23:29:58 all 90.11 0.00 1.07 0.00 0.00 0.24 0.01
0.00 0.00 8.57
23:29:59 all 90.12 0.00 0.84 0.00 0.00 0.27 0.00
0.00 0.00 8.77
23:30:00 all 89.82 0.00 0.83 0.03 0.00 0.38 0.01
0.00 0.00 8.93
23:30:01 all 90.13 0.03 1.08 0.00 0.00 0.30 0.00
0.00 0.00 8.47
23:30:02 all 89.95 0.00 0.89 0.00 0.00 0.34 0.01
0.00 0.00 8.82
23:30:03 all 89.86 0.00 1.08 0.00 0.00 0.24 0.00
0.00 0.00 8.83
23:30:04 all 87.90 0.00 0.97 0.00 0.00 0.24 0.01
0.00 0.00 10.88 {noformat}
Using a variety of easy-cass-stress KeyValue workloads with different settings
for --rate, I'm unable to see any meaningful difference, performance-wise.
{noformat}
easy-cass-stress run KeyValue -d 20m --rate 20k -p 10m -t 16 -r 1{noformat}
For each workload I've run, I've seen virtually identical results. Both are
pushing C* to use 90% CPU and achieve roughly 25K reads / second.
was (Author: rustyrazorblade):
When I try to spin up those instance types in us-west-2 I get an error that
they're invalid, so I'm running a test with c5.18xlarge.
I'm working with a single node, with compaction disabled, and I reduced my
memtable space to 16MB in order to constantly flush. I wrote 10m rows and have
1928 SStables. These boxes have 72 CPU. I'm using G1GC with a 24GB heap.
I've tested concurrent_reads at 64 and 128 since there's enough cores on here
to handle and we don't need to bottleneck on reads.
So, right off the bat, I'm not able to duplicate the original observation about
CPU not going over 50% utilization. 4.1 has reached 90+% CPU utilization:
{noformat}
23:29:57 CPU %usr %nice %sys %iowait %irq %soft %steal
%guest %gnice %idle
23:29:58 all 90.11 0.00 1.07 0.00 0.00 0.24 0.01
0.00 0.00 8.57
23:29:59 all 90.12 0.00 0.84 0.00 0.00 0.27 0.00
0.00 0.00 8.77
23:30:00 all 89.82 0.00 0.83 0.03 0.00 0.38 0.01
0.00 0.00 8.93
23:30:01 all 90.13 0.03 1.08 0.00 0.00 0.30 0.00
0.00 0.00 8.47
23:30:02 all 89.95 0.00 0.89 0.00 0.00 0.34 0.01
0.00 0.00 8.82
23:30:03 all 89.86 0.00 1.08 0.00 0.00 0.24 0.00
0.00 0.00 8.83
23:30:04 all 87.90 0.00 0.97 0.00 0.00 0.24 0.01
0.00 0.00 10.88 {noformat}
Using a variety of easy-cass-stress KeyValue workloads with different settings
for --rate, I'm unable to see any meaningful difference, performance-wise.
{noformat}
easy-cass-stress run KeyValue -d 20m --rate 20k -p 10m -t 16 -r 1{noformat}
For each workload I've run, I've seen virtually identical results. Both are
pushing C* to use 90% CPU and achieve roughly 25K reads / second.
> Remove lock contention generated by getCapacity function in SSTableReader
> -
>
> Key: CASSANDRA-19429
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19429
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Dipietro Salvatore
>Assignee: Dipietro Salvatore
>Priority: Normal
> Fix For: 4.0.x, 4.1.x
>
> Attachments: Screenshot 2024-02-26 at 10.27.10.png, Screenshot
> 2024-02-27 at 11.29.41.png, asprof_cass4.1.3__lock_20240216052912lock.html
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Profiling Cassandra 4.1.3 on large AWS instances, a high number of lock
> acquires is measured in the `getCapacity` function from
> `org/apache/cassandra/cache/InstrumentingCache` (1.9M lock acquires per 60
> seconds). Based
[jira] [Commented] (CASSANDRA-19429) Remove lock contention generated by getCapacity function in SSTableReader
[
https://issues.apache.org/jira/browse/CASSANDRA-19429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824885#comment-17824885
]
Jon Haddad commented on CASSANDRA-19429:
When I try to spin up those instance types in us-west-2 I get an error that
they're invalid, so I'm running a test with c5.18xlarge.
I'm working with a single node, with compaction disabled, and I reduced my
memtable space to 16MB in order to constantly flush. I wrote 10m rows and have
1928 SStables. These boxes have 72 CPU. I'm using G1GC with a 24GB heap.
I've tested concurrent_reads at 64 and 128 since there's enough cores on here
to handle and we don't need to bottleneck on reads.
So, right off the bat, I'm not able to duplicate the original observation about
CPU not going over 50% utilization. 4.1 has reached 90+% CPU utilization:
{noformat}
23:29:57 CPU %usr %nice %sys %iowait %irq %soft %steal
%guest %gnice %idle
23:29:58 all 90.11 0.00 1.07 0.00 0.00 0.24 0.01
0.00 0.00 8.57
23:29:59 all 90.12 0.00 0.84 0.00 0.00 0.27 0.00
0.00 0.00 8.77
23:30:00 all 89.82 0.00 0.83 0.03 0.00 0.38 0.01
0.00 0.00 8.93
23:30:01 all 90.13 0.03 1.08 0.00 0.00 0.30 0.00
0.00 0.00 8.47
23:30:02 all 89.95 0.00 0.89 0.00 0.00 0.34 0.01
0.00 0.00 8.82
23:30:03 all 89.86 0.00 1.08 0.00 0.00 0.24 0.00
0.00 0.00 8.83
23:30:04 all 87.90 0.00 0.97 0.00 0.00 0.24 0.01
0.00 0.00 10.88 {noformat}
Using a variety of easy-cass-stress KeyValue workloads with different settings
for --rate, I'm unable to see any meaningful difference, performance-wise.
{noformat}
easy-cass-stress run KeyValue -d 20m --rate 20k -p 10m -t 16 -r 1{noformat}
For each workload I've run, I've seen virtually identical results. Both are
pushing C* to use 90% CPU and achieve roughly 25K reads / second.
> Remove lock contention generated by getCapacity function in SSTableReader
> -
>
> Key: CASSANDRA-19429
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19429
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Dipietro Salvatore
>Assignee: Dipietro Salvatore
>Priority: Normal
> Fix For: 4.0.x, 4.1.x
>
> Attachments: Screenshot 2024-02-26 at 10.27.10.png, Screenshot
> 2024-02-27 at 11.29.41.png, asprof_cass4.1.3__lock_20240216052912lock.html
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Profiling Cassandra 4.1.3 on large AWS instances, a high number of lock
> acquires is measured in the `getCapacity` function from
> `org/apache/cassandra/cache/InstrumentingCache` (1.9M lock acquires per 60
> seconds). Based on our tests on r8g.24xlarge instances (using Ubuntu 22.04),
> this limits the CPU utilization of the system to under 50% when testing at
> full load and therefore limits the achieved throughput.
> Removing the lock contention from the SSTableReader.java file by replacing
> the call to `getCapacity` with `size` achieves up to 2.95x increase in
> throughput on r8g.24xlarge and 2x on r7i.24xlarge:
> |Instance type|Cass 4.1.3|Cass 4.1.3 patched|
> |r8g.24xlarge|168k ops|496k ops (2.95x)|
> |r7i.24xlarge|153k ops|304k ops (1.98x)|
>
> Instructions to reproduce:
> {code:java}
> ## Requirements for Ubuntu 22.04
> sudo apt install -y ant git openjdk-11-jdk
> ## Build and run
> CASSANDRA_USE_JDK11=true ant realclean && CASSANDRA_USE_JDK11=true ant jar &&
> CASSANDRA_USE_JDK11=true ant stress-build && rm -rf data && bin/cassandra -f
> -R
> # Run
> bin/cqlsh -e 'drop table if exists keyspace1.standard1;' && \
> bin/cqlsh -e 'drop keyspace if exists keyspace1;' && \
> bin/nodetool clearsnapshot --all && tools/bin/cassandra-stress write
> n=1000 cl=ONE -rate threads=384 -node 127.0.0.1 -log file=cload.log
> -graph file=cload.html && \
> bin/nodetool compact keyspace1 && sleep 30s && \
> tools/bin/cassandra-stress mixed ratio\(write=10,read=90\) duration=10m
> cl=ONE -rate threads=406 -node localhost -log file=result.log -graph
> file=graph.html
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19457) Object reference in Micrometer metrics prevent GC from reclaiming Session instances
[
https://issues.apache.org/jira/browse/CASSANDRA-19457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bret McGuire updated CASSANDRA-19457:
-
Summary: Object reference in Micrometer metrics prevent GC from reclaiming
Session instances (was: Memory Leak of `DefaultSession`)
> Object reference in Micrometer metrics prevent GC from reclaiming Session
> instances
> ---
>
> Key: CASSANDRA-19457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19457
> Project: Cassandra
> Issue Type: Bug
> Components: Client/java-driver
>Reporter: Jane He
>Assignee: Jane He
>Priority: Normal
> Attachments: Screenshot 2024-03-06 at 2.07.01 PM.png, Screenshot
> 2024-03-06 at 2.07.13 PM.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> There is a memory leak of previous closed {{{}DefaultSession{}}}s. It can be
> reproduced by this:
> {code:java}
> public static void main(String[] args) throws InterruptedException {
> Semaphore sema = new Semaphore(20);
> for (int i = 0; i < 1; i++) {
> new Thread(() -> {
> try {
> sema.acquire();
> try(CqlSession session = CqlSession.builder()
>
> .withCloudSecureConnectBundle(Paths.get("bundle.zip"))
> .withAuthCredentials("token", "")
> .build()) {
> // Do stuff
> }
> } catch (Exception e) {
> System.out.println(e);
> } finally {
> sema.release();
> }
> }).start();
> }
> }{code}
> On initial investigation, it seems like
> {{MicrometerMetricUpdater.initializeGauge()}} uses
> {{{}Gauge.{}}}{{{}_builder()_{}}} _using_ {{_Supplier_}} _._ This creates a
> strong reference that is causing the issue.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] JAVA-2967: Support native_transport_(address|port) + native_transport_port_ssl for DSE 6.8 (4.x edition) [cassandra-java-driver]
absurdfarce commented on PR #1913: URL: https://github.com/apache/cassandra-java-driver/pull/1913#issuecomment-1986449928 Can't add my +1 for some reason but I approve this message as well. :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] Changelog updates to reflect work that went out in 4.18.0 [cassandra-java-driver]
absurdfarce commented on PR #1914: URL: https://github.com/apache/cassandra-java-driver/pull/1914#issuecomment-1986448006 Can't add my +1 for some reason but I approve this message as well. :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] CASSANDRA-19457: Memory Leak of `DefaultSession` [cassandra-java-driver]
SiyaoIsHiding commented on PR #1916: URL: https://github.com/apache/cassandra-java-driver/pull/1916#issuecomment-1986407007 I find it will lead to some missing metrics like `connected-nodes`. We need another way to fix this for sure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-114) Make hash algorithm implementation pluggable
[ https://issues.apache.org/jira/browse/CASSANDRASC-114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CASSANDRASC-114: --- Labels: pull-request-available (was: ) > Make hash algorithm implementation pluggable > > > Key: CASSANDRASC-114 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-114 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Labels: pull-request-available > > I would like to propose to make the hash algorithm implementation in sidecar > pluggable. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-114) Make hash algorithm implementation pluggable
[ https://issues.apache.org/jira/browse/CASSANDRASC-114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRASC-114: -- Test and Documentation Plan: ci; unit Status: Patch Available (was: Open) PR: https://github.com/apache/cassandra-sidecar/pull/106 CI: https://app.circleci.com/pipelines/github/yifan-c/cassandra-sidecar?branch=CASSANDRASC-114%2Ftrunk > Make hash algorithm implementation pluggable > > > Key: CASSANDRASC-114 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-114 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Labels: pull-request-available > > I would like to propose to make the hash algorithm implementation in sidecar > pluggable. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-114) Make hash algorithm implementation pluggable
[ https://issues.apache.org/jira/browse/CASSANDRASC-114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRASC-114: -- Change Category: Code Clarity Complexity: Normal Component/s: Rest API Status: Open (was: Triage Needed) > Make hash algorithm implementation pluggable > > > Key: CASSANDRASC-114 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-114 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > > I would like to propose to make the hash algorithm implementation in sidecar > pluggable. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRASC-114) Make hash algorithm implementation pluggable
Yifan Cai created CASSANDRASC-114: - Summary: Make hash algorithm implementation pluggable Key: CASSANDRASC-114 URL: https://issues.apache.org/jira/browse/CASSANDRASC-114 Project: Sidecar for Apache Cassandra Issue Type: Improvement Reporter: Yifan Cai Assignee: Yifan Cai I would like to propose to make the hash algorithm implementation in sidecar pluggable. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-builds) branch trunk updated: Update puppet.conf
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git The following commit(s) were added to refs/heads/trunk by this push: new 5310698 Update puppet.conf 5310698 is described below commit 531069897cb1f47f4b48efa6e9a394fda50d6c4f Author: Brandon Williams AuthorDate: Fri Mar 8 13:40:59 2024 -0600 Update puppet.conf Patch by brandonwilliams for INFRA-25584 --- jenkins-dsl/agent-install.sh | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/jenkins-dsl/agent-install.sh b/jenkins-dsl/agent-install.sh index 4f7f611..656c85f 100644 --- a/jenkins-dsl/agent-install.sh +++ b/jenkins-dsl/agent-install.sh @@ -83,6 +83,9 @@ apt-get install -y puppet-agent sh -c 'cat >> /etc/puppetlabs/puppet/puppet.conf << EOF [main] -server = pm01-lw-us.apache.org +use_srv_records = true +srv_domain = apache.org + +environment = production EOF' - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14476) ShortType and ByteType are incorrectly considered variable-length types
[ https://issues.apache.org/jira/browse/CASSANDRA-14476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824853#comment-17824853 ] David Capwell commented on CASSANDRA-14476: --- [~jlewandowski], when I first found out about the fixed length types being variable length I wanted to fix but this trickles down to the SSTable format as well, so requires special handling to avoid upgrade/downgrades > ShortType and ByteType are incorrectly considered variable-length types > --- > > Key: CASSANDRA-14476 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14476 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core >Reporter: Vladimir Krivopalov >Assignee: Jacek Lewandowski >Priority: Low > Labels: lhf > Fix For: 5.0.x, 5.1 > > > The AbstractType class has a method valueLengthIfFixed() that returns -1 for > data types with a variable length and a positive value for types with a fixed > length. This is primarily used for efficient serialization and > deserialization. > > It turns out that there is an inconsistency in types ShortType and ByteType > as those are in fact fixed-length types (2 bytes and 1 byte, respectively) > but they don't have the valueLengthIfFixed() method overloaded and it returns > -1 as if they were of variable length. > > It would be good to fix that at some appropriate point, for example, when > introducing a new version of SSTables format, to keep the meaning of the > function consistent across data types. Saving some bytes in serialized format > is a minor but pleasant bonus. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19462) cqlsh does not autocomplete table names in MacOS
[
https://issues.apache.org/jira/browse/CASSANDRA-19462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824613#comment-17824613
]
Brad Schoening edited comment on CASSANDRA-19462 at 3/8/24 7:22 PM:
Tab completion on trunk works for me with python 3.11 and Ventura, but w/out
pyreadline.
{quote}% ./bin/cqlsh
Connected to *Test Cluster* at 127.0.0.1:9042
[cqlsh 6.3.0 | Cassandra 5.0-beta1 | CQL spec 3.4.7 | Native protocol v5]
cassandra@cqlsh> select * from system_auth.
cidr_groups identity_to_role
resource_role_permissons_index role_permissions cidr_permissions
network_permissions role_members roles
{quote}
When I installed pyreadline 2.1 <2015-09-16>, tab completion stopped working
with trunk, but it does work with pip installed cqlsh.
The cqlsh with pip is based on Cassandra version 4.1.1
was (Author: bschoeni):
Tab completion on trunk works for me with python 3.11 and Ventura, but w/out
pyreadline.
{quote}% ./bin/cqlsh
Connected to *Test Cluster* at 127.0.0.1:9042
[cqlsh 6.3.0 | Cassandra 5.0-beta1 | CQL spec 3.4.7 | Native protocol v5]
cassandra@cqlsh> select * from system_auth.
cidr_groups identity_to_role
resource_role_permissons_index role_permissions cidr_permissions
network_permissions role_members roles
{quote}
When I installed pyreadline, tab completion stopped working with trunk, but it
does work with pip installed cqlsh.
The cqlsh with pip is based on Cassandra version 4.1.1
> cqlsh does not autocomplete table names in MacOS
>
>
> Key: CASSANDRA-19462
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19462
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
>Reporter: Stefan Miklosovic
>Assignee: Brad Schoening
>Priority: Normal
>
> I am on MacBook with Sonoma
> {code}
> $ uname -a
> Darwin stefanm-mac-0 23.3.0 Darwin Kernel Version 23.3.0: Wed Dec 20 21:30:59
> PST 2023; root:xnu-10002.81.5~7/RELEASE_ARM64_T6030 arm64
> {code}
> What is interesting is that ./bin/cqlsh works but when I am tabbing like
> {code}
> select * from system_auth.
> {code}
> This does not work.
> It does work when I install cqlsh from pip though.
> I have pyreadline installed,
> I have two pythons 3.10 and 3.9.6. I installed 3.10 from brew and 3.9.6 is
> from Sonoma itself.
> Could somebody with Mac verify that tab completion works (or does not) when
> invoking ./bin/cqlsh from Cassandra?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-19462) cqlsh does not autocomplete table names in MacOS
[
https://issues.apache.org/jira/browse/CASSANDRA-19462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brad Schoening reassigned CASSANDRA-19462:
--
Assignee: Brad Schoening
> cqlsh does not autocomplete table names in MacOS
>
>
> Key: CASSANDRA-19462
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19462
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
>Reporter: Stefan Miklosovic
>Assignee: Brad Schoening
>Priority: Normal
>
> I am on MacBook with Sonoma
> {code}
> $ uname -a
> Darwin stefanm-mac-0 23.3.0 Darwin Kernel Version 23.3.0: Wed Dec 20 21:30:59
> PST 2023; root:xnu-10002.81.5~7/RELEASE_ARM64_T6030 arm64
> {code}
> What is interesting is that ./bin/cqlsh works but when I am tabbing like
> {code}
> select * from system_auth.
> {code}
> This does not work.
> It does work when I install cqlsh from pip though.
> I have pyreadline installed,
> I have two pythons 3.10 and 3.9.6. I installed 3.10 from brew and 3.9.6 is
> from Sonoma itself.
> Could somebody with Mac verify that tab completion works (or does not) when
> invoking ./bin/cqlsh from Cassandra?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19462) cqlsh does not autocomplete table names in MacOS
[
https://issues.apache.org/jira/browse/CASSANDRA-19462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824613#comment-17824613
]
Brad Schoening edited comment on CASSANDRA-19462 at 3/8/24 7:16 PM:
Tab completion on trunk works for me with python 3.11 and Ventura, but w/out
pyreadline.
{quote}% ./bin/cqlsh
Connected to *Test Cluster* at 127.0.0.1:9042
[cqlsh 6.3.0 | Cassandra 5.0-beta1 | CQL spec 3.4.7 | Native protocol v5]
cassandra@cqlsh> select * from system_auth.
cidr_groups identity_to_role
resource_role_permissons_index role_permissions cidr_permissions
network_permissions role_members roles
{quote}
When I installed pyreadline, it tab completion stopped working, but it does
work with pip installed cqlsh.
The cqlsh with pip is based on Cassandra version 4.1.1
was (Author: bschoeni):
Odd, tab completion on trunk works for me with python 3.11 and Ventura.
{quote}% ./bin/cqlsh
Connected to *Test Cluster* at 127.0.0.1:9042
[cqlsh 6.3.0 | Cassandra 5.0-beta1 | CQL spec 3.4.7 | Native protocol v5]
cassandra@cqlsh> select * from system_auth.
cidr_groups identity_to_role
resource_role_permissons_index role_permissions cidr_permissions
network_permissions role_members roles
{quote}
The cqlsh with pip is based on Cassandra version 4.1.1
> cqlsh does not autocomplete table names in MacOS
>
>
> Key: CASSANDRA-19462
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19462
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
>Reporter: Stefan Miklosovic
>Priority: Normal
>
> I am on MacBook with Sonoma
> {code}
> $ uname -a
> Darwin stefanm-mac-0 23.3.0 Darwin Kernel Version 23.3.0: Wed Dec 20 21:30:59
> PST 2023; root:xnu-10002.81.5~7/RELEASE_ARM64_T6030 arm64
> {code}
> What is interesting is that ./bin/cqlsh works but when I am tabbing like
> {code}
> select * from system_auth.
> {code}
> This does not work.
> It does work when I install cqlsh from pip though.
> I have pyreadline installed,
> I have two pythons 3.10 and 3.9.6. I installed 3.10 from brew and 3.9.6 is
> from Sonoma itself.
> Could somebody with Mac verify that tab completion works (or does not) when
> invoking ./bin/cqlsh from Cassandra?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19462) cqlsh does not autocomplete table names in MacOS
[
https://issues.apache.org/jira/browse/CASSANDRA-19462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824613#comment-17824613
]
Brad Schoening edited comment on CASSANDRA-19462 at 3/8/24 7:17 PM:
Tab completion on trunk works for me with python 3.11 and Ventura, but w/out
pyreadline.
{quote}% ./bin/cqlsh
Connected to *Test Cluster* at 127.0.0.1:9042
[cqlsh 6.3.0 | Cassandra 5.0-beta1 | CQL spec 3.4.7 | Native protocol v5]
cassandra@cqlsh> select * from system_auth.
cidr_groups identity_to_role
resource_role_permissons_index role_permissions cidr_permissions
network_permissions role_members roles
{quote}
When I installed pyreadline, tab completion stopped working with trunk, but it
does work with pip installed cqlsh.
The cqlsh with pip is based on Cassandra version 4.1.1
was (Author: bschoeni):
Tab completion on trunk works for me with python 3.11 and Ventura, but w/out
pyreadline.
{quote}% ./bin/cqlsh
Connected to *Test Cluster* at 127.0.0.1:9042
[cqlsh 6.3.0 | Cassandra 5.0-beta1 | CQL spec 3.4.7 | Native protocol v5]
cassandra@cqlsh> select * from system_auth.
cidr_groups identity_to_role
resource_role_permissons_index role_permissions cidr_permissions
network_permissions role_members roles
{quote}
When I installed pyreadline, it tab completion stopped working, but it does
work with pip installed cqlsh.
The cqlsh with pip is based on Cassandra version 4.1.1
> cqlsh does not autocomplete table names in MacOS
>
>
> Key: CASSANDRA-19462
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19462
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
>Reporter: Stefan Miklosovic
>Priority: Normal
>
> I am on MacBook with Sonoma
> {code}
> $ uname -a
> Darwin stefanm-mac-0 23.3.0 Darwin Kernel Version 23.3.0: Wed Dec 20 21:30:59
> PST 2023; root:xnu-10002.81.5~7/RELEASE_ARM64_T6030 arm64
> {code}
> What is interesting is that ./bin/cqlsh works but when I am tabbing like
> {code}
> select * from system_auth.
> {code}
> This does not work.
> It does work when I install cqlsh from pip though.
> I have pyreadline installed,
> I have two pythons 3.10 and 3.9.6. I installed 3.10 from brew and 3.9.6 is
> from Sonoma itself.
> Could somebody with Mac verify that tab completion works (or does not) when
> invoking ./bin/cqlsh from Cassandra?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-builds) branch trunk updated: Ensure ~/.m2/repository exists
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
The following commit(s) were added to refs/heads/trunk by this push:
new 363cdeb Ensure ~/.m2/repository exists
363cdeb is described below
commit 363cdeb99ca978fe08746732b99dee89191b074b
Author: Brandon Williams
AuthorDate: Fri Mar 8 09:35:51 2024 -0600
Ensure ~/.m2/repository exists
Patch by brandonwilliams; reviwed by mck for INFRA-25584
---
build-scripts/cassandra-deb-packaging.sh | 1 +
build-scripts/cassandra-rpm-packaging.sh | 1 +
2 files changed, 2 insertions(+)
diff --git a/build-scripts/cassandra-deb-packaging.sh
b/build-scripts/cassandra-deb-packaging.sh
index b73b10e..83f7656 100755
--- a/build-scripts/cassandra-deb-packaging.sh
+++ b/build-scripts/cassandra-deb-packaging.sh
@@ -39,6 +39,7 @@ until docker build --build-arg
CASSANDRA_GIT_URL=$CASSANDRA_GIT_URL --build-arg
# Run build script through docker (specify branch, tag, or sha)
+mkdir -p ~/.m2/repository
docker run --rm -v "${deb_dir}":/dist -v
~/.m2/repository/:/home/build/.m2/repository/
cassandra-artifacts-bullseye:${sha} /home/build/build-debs.sh ${sha}
${java_version}
popd
diff --git a/build-scripts/cassandra-rpm-packaging.sh
b/build-scripts/cassandra-rpm-packaging.sh
index 3ed2255..c8ba817 100755
--- a/build-scripts/cassandra-rpm-packaging.sh
+++ b/build-scripts/cassandra-rpm-packaging.sh
@@ -49,6 +49,7 @@ pushd $cassandra_builds_dir
until docker build --build-arg CASSANDRA_GIT_URL=$CASSANDRA_GIT_URL
--build-arg UID_ARG=`id -u` --build-arg GID_ARG=`id -g` -t
cassandra-artifacts-${dist_name}:${sha} -f docker/${dist_name}-image.docker
docker/ ; do echo "docker build failed… trying again in 10s… " ; sleep 10 ;
done
# Run build script through docker (specify branch, tag, or sha)
+mkdir -p ~/.m2/repository
docker run --rm -v "${rpm_dir}":/dist -v
~/.m2/repository/:/home/build/.m2/repository/
cassandra-artifacts-${dist_name}:${sha} /home/build/build-rpms.sh ${sha}
${java_version} ${rpm_dist}
popd
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14476) ShortType and ByteType are incorrectly considered variable-length types
[ https://issues.apache.org/jira/browse/CASSANDRA-14476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824810#comment-17824810 ] Jacek Lewandowski commented on CASSANDRA-14476: --- Here it is: https://github.com/apache/cassandra/pull/3169 > ShortType and ByteType are incorrectly considered variable-length types > --- > > Key: CASSANDRA-14476 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14476 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core >Reporter: Vladimir Krivopalov >Assignee: Jacek Lewandowski >Priority: Low > Labels: lhf > Fix For: 5.0.x, 5.1 > > > The AbstractType class has a method valueLengthIfFixed() that returns -1 for > data types with a variable length and a positive value for types with a fixed > length. This is primarily used for efficient serialization and > deserialization. > > It turns out that there is an inconsistency in types ShortType and ByteType > as those are in fact fixed-length types (2 bytes and 1 byte, respectively) > but they don't have the valueLengthIfFixed() method overloaded and it returns > -1 as if they were of variable length. > > It would be good to fix that at some appropriate point, for example, when > introducing a new version of SSTables format, to keep the meaning of the > function consistent across data types. Saving some bytes in serialized format > is a minor but pleasant bonus. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19429) Remove lock contention generated by getCapacity function in SSTableReader
[
https://issues.apache.org/jira/browse/CASSANDRA-19429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824809#comment-17824809
]
Jon Haddad commented on CASSANDRA-19429:
Looking at this today.
> Remove lock contention generated by getCapacity function in SSTableReader
> -
>
> Key: CASSANDRA-19429
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19429
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Dipietro Salvatore
>Assignee: Dipietro Salvatore
>Priority: Normal
> Fix For: 4.0.x, 4.1.x
>
> Attachments: Screenshot 2024-02-26 at 10.27.10.png, Screenshot
> 2024-02-27 at 11.29.41.png, asprof_cass4.1.3__lock_20240216052912lock.html
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Profiling Cassandra 4.1.3 on large AWS instances, a high number of lock
> acquires is measured in the `getCapacity` function from
> `org/apache/cassandra/cache/InstrumentingCache` (1.9M lock acquires per 60
> seconds). Based on our tests on r8g.24xlarge instances (using Ubuntu 22.04),
> this limits the CPU utilization of the system to under 50% when testing at
> full load and therefore limits the achieved throughput.
> Removing the lock contention from the SSTableReader.java file by replacing
> the call to `getCapacity` with `size` achieves up to 2.95x increase in
> throughput on r8g.24xlarge and 2x on r7i.24xlarge:
> |Instance type|Cass 4.1.3|Cass 4.1.3 patched|
> |r8g.24xlarge|168k ops|496k ops (2.95x)|
> |r7i.24xlarge|153k ops|304k ops (1.98x)|
>
> Instructions to reproduce:
> {code:java}
> ## Requirements for Ubuntu 22.04
> sudo apt install -y ant git openjdk-11-jdk
> ## Build and run
> CASSANDRA_USE_JDK11=true ant realclean && CASSANDRA_USE_JDK11=true ant jar &&
> CASSANDRA_USE_JDK11=true ant stress-build && rm -rf data && bin/cassandra -f
> -R
> # Run
> bin/cqlsh -e 'drop table if exists keyspace1.standard1;' && \
> bin/cqlsh -e 'drop keyspace if exists keyspace1;' && \
> bin/nodetool clearsnapshot --all && tools/bin/cassandra-stress write
> n=1000 cl=ONE -rate threads=384 -node 127.0.0.1 -log file=cload.log
> -graph file=cload.html && \
> bin/nodetool compact keyspace1 && sleep 30s && \
> tools/bin/cassandra-stress mixed ratio\(write=10,read=90\) duration=10m
> cl=ONE -rate threads=406 -node localhost -log file=result.log -graph
> file=graph.html
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14476) ShortType and ByteType are incorrectly considered variable-length types
[ https://issues.apache.org/jira/browse/CASSANDRA-14476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824785#comment-17824785 ] Jacek Lewandowski commented on CASSANDRA-14476: --- I'm going to first submit some tests which will validate serialization and comparisons, as well asl backward compatibility. > ShortType and ByteType are incorrectly considered variable-length types > --- > > Key: CASSANDRA-14476 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14476 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core >Reporter: Vladimir Krivopalov >Assignee: Jacek Lewandowski >Priority: Low > Labels: lhf > Fix For: 5.0.x, 5.1 > > > The AbstractType class has a method valueLengthIfFixed() that returns -1 for > data types with a variable length and a positive value for types with a fixed > length. This is primarily used for efficient serialization and > deserialization. > > It turns out that there is an inconsistency in types ShortType and ByteType > as those are in fact fixed-length types (2 bytes and 1 byte, respectively) > but they don't have the valueLengthIfFixed() method overloaded and it returns > -1 as if they were of variable length. > > It would be good to fix that at some appropriate point, for example, when > introducing a new version of SSTables format, to keep the meaning of the > function consistent across data types. Saving some bytes in serialized format > is a minor but pleasant bonus. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14476) ShortType and ByteType are incorrectly considered variable-length types
[
https://issues.apache.org/jira/browse/CASSANDRA-14476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17822240#comment-17822240
]
Jacek Lewandowski edited comment on CASSANDRA-14476 at 3/8/24 1:05 PM:
---
There are more problems with type compatibility:
1. Fixed length types reported as variable length: *ByteType*, *ShortType*,
*CounterColumnType*, *SimpleDateType*, *TimeType*, and types like *TupleType*,
*UserType* when all subtypes are of fixed length
2. Value compatibility issues:
* *IntegerType* should be compatible with *ShortType*, *ByteType*,
*SimpleDateType*, and *TimeType* - all of them are simple integers serialized
with Big-Endian byte order
* *LongType* is compatible with *TimestampType* and *TimestampType* is
compatible with *LongType*, which makes a cycle in the type compatibility
hierarchy - I don't know if it is ok because the relation
{{isValueCompatibleWith}} is used when merging data from different sources to
determine the resulting type. It may end up with a result depending on the
order of data sources. Is it ok for compaction and querying? - I don't know.
* *TimeType* is compatible with *LongType*, but it should be opposite as the
*LongType* is more generic than *TimeType*
* *SimpleDateType* is compatible with *Int32Type*, but is should be opposite as
the *Int32Type* is more generic than *SimpleDateType*
3. Painful lack of tests for this stuff
4. {{isCompatibleWith}} seems to be used for very few things:
* validating the return type of the replaced function or aggregate
* validating the new table metadata against the previous metadata - the new
metadata must have all the types compatible with the previous metadata.
Some conclusions:
* for the return type of functions and aggregates, it does not matter whether
the compared types are multi-cell or not, all in all we deal with opaque value
- it would be enough to ensure value compatibility (compose/decompose) and
comparison consistency.
* I suspect a bug there, though - the return type is required to satisfy
{{returnType.isCompatibleWith(existingAggregate.returnType())}} condition. I
believe the condition should be the opposite - assuming that relation
{{isCompatibleWith}} is a partial order, the *existing return type should be
the same or more generic than the new type* so that the function will continue
to work correctly with the existing usages. If we allow changing the type from,
say, {{UTF8}} to {{Bytes}} (which is valid according to the current condition),
the usages expecting {{UTF8}} return type will stop working.
* For the metadata compatibility checks, we never use multi-cell serialized
values for sorting. If a multi-cell type is ever used in an order requiring
context (part of the primary key), it is always frozen. Therefore, there is no
need to consider different rules for multi-cell / frozen variants.
---
I haven't investigated the compatibility of complex types yet
was (Author: jlewandowski):
There are more problems with type compatibility:
1. Fixed length types reported as variable length: *ByteType*, *ShortType*,
*CounterColumnType*, *SimpleDateType*, *TimeType*, and types like *TupleType*,
*UserType* when all subtypes are of fixed length
2. Value compatibility issues:
* *IntegerType* should be compatible with *ShortType*, *ByteType*,
*SimpleDateType* and *TimeType* - all of them are simple integers serialized
with Big-Endian byte order
* *LongType* is compatible with *TimestampType* and *TimestampType* is
compatible with *LongType* which makes a cycle in the type compatibility
hierarchy - I don't know if it is ok because the relation
{{isValueCompatibleWith}} is used when merging data from different sources in
order to determine the resulting type. It may end up with a result depending on
the order of data sources - is it ok for compaction and querying?
* *TimeType* is compatible with *LongType*, but it should be opposite as the
*LongType* is more generic than *TimeType*
* *SimpleDateType* is compatible with *Int32Type*, but is should be opposite as
the *Int32Type* is more generic than *SimpleDateType*
3. Painful lack of tests for this stuff
---
I haven't investigated the compatibility of complex types yet
> ShortType and ByteType are incorrectly considered variable-length types
> ---
>
> Key: CASSANDRA-14476
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14476
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core
>Reporter: Vladimir Krivopalov
>Assignee: Jacek Lewandowski
>Priority: Low
> Labels: lhf
> Fix For: 5.0.x, 5.1
>
>
> The AbstractType class has a method valueLengthIfFixed() that returns -1 for
> data types with a variable length and a positive value
[jira] [Updated] (CASSANDRA-19384) Avoid exposing intermediate node state during startup
[ https://issues.apache.org/jira/browse/CASSANDRA-19384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-19384: Fix Version/s: 5.x Since Version: 5.x Source Control Link: https://github.com/apache/cassandra/commit/5d4bcc797af882c64736b3f842cbf8bedbba184b Resolution: Fixed Status: Resolved (was: Ready to Commit) committed, thanks > Avoid exposing intermediate node state during startup > - > > Key: CASSANDRA-19384 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19384 > Project: Cassandra > Issue Type: Bug > Components: Transactional Cluster Metadata >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 5.x > > Attachments: ci_summary-1.html, ci_summary.html, > result_details.tar-1.gz, result_details.tar.gz > > Time Spent: 20m > Remaining Estimate: 0h > > During startup we replay the local log, during this time we might expose > intermediate node states (via JMX for example). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch trunk updated: Avoid exposing intermediate state while replaying log during startup
This is an automated email from the ASF dual-hosted git repository.
marcuse pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 5d4bcc797a Avoid exposing intermediate state while replaying log
during startup
5d4bcc797a is described below
commit 5d4bcc797af882c64736b3f842cbf8bedbba184b
Author: Marcus Eriksson
AuthorDate: Thu Feb 1 10:16:41 2024 +0100
Avoid exposing intermediate state while replaying log during startup
Patch by marcuse; reviewed by Sam Tunnicliffe for CASSANDRA-19384
---
.../cassandra/cql3/functions/FunctionCall.java | 7 +++---
.../cassandra/cql3/functions/FunctionResolver.java | 24 --
.../cql3/functions/masking/ColumnMask.java | 11
.../cql3/selection/AbstractFunctionSelector.java | 3 ++-
.../cassandra/cql3/selection/Selectable.java | 6 ++---
.../statements/schema/AlterTableStatement.java | 29 ++
.../statements/schema/CreateTableStatement.java| 22
.../cassandra/io/sstable/CQLSSTableWriter.java | 6 ++---
.../org/apache/cassandra/schema/UserFunctions.java | 17 +
.../apache/cassandra/service/StorageService.java | 4 +--
src/java/org/apache/cassandra/tcm/Startup.java | 8 --
.../org/apache/cassandra/tcm/Transformation.java | 2 +-
.../org/apache/cassandra/tcm/log/LocalLog.java | 14 ++-
.../org/apache/cassandra/tcm/log/LogState.java | 13 ++
test/unit/org/apache/cassandra/cql3/CQLTester.java | 1 +
.../cql3/functions/NativeFunctionsTest.java| 4 ++-
.../cql3/functions/masking/ColumnMaskTester.java | 3 +--
.../db/ColumnFamilyStoreClientModeTest.java| 3 ++-
.../io/sstable/StressCQLSSTableWriter.java | 2 +-
19 files changed, 103 insertions(+), 76 deletions(-)
diff --git a/src/java/org/apache/cassandra/cql3/functions/FunctionCall.java
b/src/java/org/apache/cassandra/cql3/functions/FunctionCall.java
index 30d9d50437..c45964a997 100644
--- a/src/java/org/apache/cassandra/cql3/functions/FunctionCall.java
+++ b/src/java/org/apache/cassandra/cql3/functions/FunctionCall.java
@@ -32,6 +32,7 @@ import org.apache.cassandra.cql3.terms.Term;
import org.apache.cassandra.cql3.terms.Terms;
import org.apache.cassandra.db.marshal.*;
import org.apache.cassandra.exceptions.InvalidRequestException;
+import org.apache.cassandra.schema.UserFunctions;
import org.apache.cassandra.serializers.MarshalException;
import org.apache.cassandra.utils.ByteBufferUtil;
@@ -150,7 +151,7 @@ public class FunctionCall extends Term.NonTerminal
public Term prepare(String keyspace, ColumnSpecification receiver)
throws InvalidRequestException
{
-Function fun = FunctionResolver.get(keyspace, name, terms,
receiver.ksName, receiver.cfName, receiver.type);
+Function fun = FunctionResolver.get(keyspace, name, terms,
receiver.ksName, receiver.cfName, receiver.type,
UserFunctions.getCurrentUserFunctions(name, keyspace));
if (fun == null)
throw invalidRequest("Unknown function %s called", name);
if (fun.isAggregate())
@@ -194,7 +195,7 @@ public class FunctionCall extends Term.NonTerminal
// later with a more helpful error message that if we were to
return false here.
try
{
-Function fun = FunctionResolver.get(keyspace, name, terms,
receiver.ksName, receiver.cfName, receiver.type);
+Function fun = FunctionResolver.get(keyspace, name, terms,
receiver.ksName, receiver.cfName, receiver.type,
UserFunctions.getCurrentUserFunctions(name, keyspace));
// Because the return type of functions built by factories is
not fixed but depending on the types of
// their arguments, we'll always get EXACT_MATCH. To handle
potentially ambiguous function calls with
@@ -221,7 +222,7 @@ public class FunctionCall extends Term.NonTerminal
{
try
{
-Function fun = FunctionResolver.get(keyspace, name, terms,
null, null, null);
+Function fun = FunctionResolver.get(keyspace, name, terms,
null, null, null, UserFunctions.getCurrentUserFunctions(name, keyspace));
return fun == null ? null : fun.returnType();
}
catch (InvalidRequestException e)
diff --git a/src/java/org/apache/cassandra/cql3/functions/FunctionResolver.java
b/src/java/org/apache/cassandra/cql3/functions/FunctionResolver.java
index fd1c901b59..11cf24362c 100644
--- a/src/java/org/apache/cassandra/cql3/functions/FunctionResolver.java
+++ b/src/java/org/apache/cassandra/cql3/functions/FunctionResolver.java
@@ -29,7 +29,6 @@ import org.apache.cassandra.cql3.terms.Marker;
import org.apache.cassandra.cql3.AssignmentTestable;
import
[jira] [Commented] (CASSANDRA-19460) Fix tests to work with ULID SSTable identifiers to enable uuid_sstable_identifiers_enabled in cassandra-latest.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-19460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824756#comment-17824756 ] Branimir Lambov commented on CASSANDRA-19460: - LGTM > Fix tests to work with ULID SSTable identifiers to enable > uuid_sstable_identifiers_enabled in cassandra-latest.yaml > --- > > Key: CASSANDRA-19460 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19460 > Project: Cassandra > Issue Type: Task > Components: CI, Test/dtest/java, Test/unit >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 5.0-rc, 5.x > > Time Spent: 1h > Remaining Estimate: 0h > > In CASSANDRA-18753 we identified that we want to also set > uuid_sstable_identifiers_enabled to true, while running a CI with it turned > on, it failed (1). > Errors do not seem to be serious, it is just the test suite we have is not > prepared for the case when uuid_sstable_identifiers_enabled is set to true by > default. > We need to fix all these tests so we can have cassandra-latest.yaml > containing that property. > https://app.circleci.com/pipelines/github/blambov/cassandra/609/workflows/aef2b936-0551-4f3b-9d86-a49451c83947 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19460) Fix tests to work with ULID SSTable identifiers to enable uuid_sstable_identifiers_enabled in cassandra-latest.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-19460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Branimir Lambov updated CASSANDRA-19460: Reviewers: Branimir Lambov Status: Review In Progress (was: Needs Committer) > Fix tests to work with ULID SSTable identifiers to enable > uuid_sstable_identifiers_enabled in cassandra-latest.yaml > --- > > Key: CASSANDRA-19460 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19460 > Project: Cassandra > Issue Type: Task > Components: CI, Test/dtest/java, Test/unit >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 5.0-rc, 5.x > > Time Spent: 1h > Remaining Estimate: 0h > > In CASSANDRA-18753 we identified that we want to also set > uuid_sstable_identifiers_enabled to true, while running a CI with it turned > on, it failed (1). > Errors do not seem to be serious, it is just the test suite we have is not > prepared for the case when uuid_sstable_identifiers_enabled is set to true by > default. > We need to fix all these tests so we can have cassandra-latest.yaml > containing that property. > https://app.circleci.com/pipelines/github/blambov/cassandra/609/workflows/aef2b936-0551-4f3b-9d86-a49451c83947 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19460) Fix tests to work with ULID SSTable identifiers to enable uuid_sstable_identifiers_enabled in cassandra-latest.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-19460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Branimir Lambov updated CASSANDRA-19460: Status: Ready to Commit (was: Review In Progress) > Fix tests to work with ULID SSTable identifiers to enable > uuid_sstable_identifiers_enabled in cassandra-latest.yaml > --- > > Key: CASSANDRA-19460 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19460 > Project: Cassandra > Issue Type: Task > Components: CI, Test/dtest/java, Test/unit >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 5.0-rc, 5.x > > Time Spent: 1h > Remaining Estimate: 0h > > In CASSANDRA-18753 we identified that we want to also set > uuid_sstable_identifiers_enabled to true, while running a CI with it turned > on, it failed (1). > Errors do not seem to be serious, it is just the test suite we have is not > prepared for the case when uuid_sstable_identifiers_enabled is set to true by > default. > We need to fix all these tests so we can have cassandra-latest.yaml > containing that property. > https://app.circleci.com/pipelines/github/blambov/cassandra/609/workflows/aef2b936-0551-4f3b-9d86-a49451c83947 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18635) Test failure: org.apache.cassandra.distributed.test.UpgradeSSTablesTest
[
https://issues.apache.org/jira/browse/CASSANDRA-18635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18635:
-
Status: Ready to Commit (was: Review In Progress)
> Test failure: org.apache.cassandra.distributed.test.UpgradeSSTablesTest
> ---
>
> Key: CASSANDRA-18635
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18635
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Brandon Williams
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> Seen here:
> https://app.circleci.com/pipelines/github/driftx/cassandra/1095/workflows/6114e2e3-8dcc-4bb0-b664-ae7d82c3349f/jobs/33405/tests
> {noformat}
> junit.framework.AssertionFailedError: expected:<0> but was:<2>
> at
> org.apache.cassandra.distributed.test.UpgradeSSTablesTest.upgradeSSTablesInterruptsOngoingCompaction(UpgradeSSTablesTest.java:86)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18635) Test failure: org.apache.cassandra.distributed.test.UpgradeSSTablesTest
[
https://issues.apache.org/jira/browse/CASSANDRA-18635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18635:
-
Status: Review In Progress (was: Needs Committer)
> Test failure: org.apache.cassandra.distributed.test.UpgradeSSTablesTest
> ---
>
> Key: CASSANDRA-18635
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18635
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Brandon Williams
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> Seen here:
> https://app.circleci.com/pipelines/github/driftx/cassandra/1095/workflows/6114e2e3-8dcc-4bb0-b664-ae7d82c3349f/jobs/33405/tests
> {noformat}
> junit.framework.AssertionFailedError: expected:<0> but was:<2>
> at
> org.apache.cassandra.distributed.test.UpgradeSSTablesTest.upgradeSSTablesInterruptsOngoingCompaction(UpgradeSSTablesTest.java:86)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18635) Test failure: org.apache.cassandra.distributed.test.UpgradeSSTablesTest
[
https://issues.apache.org/jira/browse/CASSANDRA-18635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824727#comment-17824727
]
Brandon Williams commented on CASSANDRA-18635:
--
That all looks unrelated to me and the repeated tests passed, so I'm +1
> Test failure: org.apache.cassandra.distributed.test.UpgradeSSTablesTest
> ---
>
> Key: CASSANDRA-18635
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18635
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Brandon Williams
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> Seen here:
> https://app.circleci.com/pipelines/github/driftx/cassandra/1095/workflows/6114e2e3-8dcc-4bb0-b664-ae7d82c3349f/jobs/33405/tests
> {noformat}
> junit.framework.AssertionFailedError: expected:<0> but was:<2>
> at
> org.apache.cassandra.distributed.test.UpgradeSSTablesTest.upgradeSSTablesInterruptsOngoingCompaction(UpgradeSSTablesTest.java:86)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18635) Test failure: org.apache.cassandra.distributed.test.UpgradeSSTablesTest
[
https://issues.apache.org/jira/browse/CASSANDRA-18635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824723#comment-17824723
]
Berenguer Blasi commented on CASSANDRA-18635:
-
I pushed, rebased both PRs and added CI. 5.0 is green besides what I believe is
a new flaky. Trunk it's just too difficult to tell and we can't match against
jenkins. I think we'll have to use our best guesswork and comparision against
18753's trunk run
https://app.circleci.com/pipelines/github/blambov/cassandra?branch=CASSANDRA-18753-trunk?
I think it is ok to merge and failures look unrelated but I am happy to hear
opinions.
> Test failure: org.apache.cassandra.distributed.test.UpgradeSSTablesTest
> ---
>
> Key: CASSANDRA-18635
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18635
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Brandon Williams
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> Seen here:
> https://app.circleci.com/pipelines/github/driftx/cassandra/1095/workflows/6114e2e3-8dcc-4bb0-b664-ae7d82c3349f/jobs/33405/tests
> {noformat}
> junit.framework.AssertionFailedError: expected:<0> but was:<2>
> at
> org.apache.cassandra.distributed.test.UpgradeSSTablesTest.upgradeSSTablesInterruptsOngoingCompaction(UpgradeSSTablesTest.java:86)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19429) Remove lock contention generated by getCapacity function in SSTableReader
[
https://issues.apache.org/jira/browse/CASSANDRA-19429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824698#comment-17824698
]
Stefan Miklosovic commented on CASSANDRA-19429:
---
hi [~rustyrazorblade] any progress here?
> Remove lock contention generated by getCapacity function in SSTableReader
> -
>
> Key: CASSANDRA-19429
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19429
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Dipietro Salvatore
>Assignee: Dipietro Salvatore
>Priority: Normal
> Fix For: 4.0.x, 4.1.x
>
> Attachments: Screenshot 2024-02-26 at 10.27.10.png, Screenshot
> 2024-02-27 at 11.29.41.png, asprof_cass4.1.3__lock_20240216052912lock.html
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Profiling Cassandra 4.1.3 on large AWS instances, a high number of lock
> acquires is measured in the `getCapacity` function from
> `org/apache/cassandra/cache/InstrumentingCache` (1.9M lock acquires per 60
> seconds). Based on our tests on r8g.24xlarge instances (using Ubuntu 22.04),
> this limits the CPU utilization of the system to under 50% when testing at
> full load and therefore limits the achieved throughput.
> Removing the lock contention from the SSTableReader.java file by replacing
> the call to `getCapacity` with `size` achieves up to 2.95x increase in
> throughput on r8g.24xlarge and 2x on r7i.24xlarge:
> |Instance type|Cass 4.1.3|Cass 4.1.3 patched|
> |r8g.24xlarge|168k ops|496k ops (2.95x)|
> |r7i.24xlarge|153k ops|304k ops (1.98x)|
>
> Instructions to reproduce:
> {code:java}
> ## Requirements for Ubuntu 22.04
> sudo apt install -y ant git openjdk-11-jdk
> ## Build and run
> CASSANDRA_USE_JDK11=true ant realclean && CASSANDRA_USE_JDK11=true ant jar &&
> CASSANDRA_USE_JDK11=true ant stress-build && rm -rf data && bin/cassandra -f
> -R
> # Run
> bin/cqlsh -e 'drop table if exists keyspace1.standard1;' && \
> bin/cqlsh -e 'drop keyspace if exists keyspace1;' && \
> bin/nodetool clearsnapshot --all && tools/bin/cassandra-stress write
> n=1000 cl=ONE -rate threads=384 -node 127.0.0.1 -log file=cload.log
> -graph file=cload.html && \
> bin/nodetool compact keyspace1 && sleep 30s && \
> tools/bin/cassandra-stress mixed ratio\(write=10,read=90\) duration=10m
> cl=ONE -rate threads=406 -node localhost -log file=result.log -graph
> file=graph.html
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19460) Fix tests to work with ULID SSTable identifiers to enable uuid_sstable_identifiers_enabled in cassandra-latest.yaml
[
https://issues.apache.org/jira/browse/CASSANDRA-19460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824689#comment-17824689
]
Stefan Miklosovic commented on CASSANDRA-19460:
---
[CASSANDRA-19460|https://github.com/instaclustr/cassandra/tree/CASSANDRA-19460]
{noformat}
java17_pre-commit_tests
✓ j17_build 4m 0s
✓ j17_cqlsh_dtests_py311 5m 56s
✓ j17_cqlsh_dtests_py311_vnode 6m 8s
✓ j17_cqlsh_dtests_py385m 56s
✓ j17_cqlsh_dtests_py38_vnode 6m 3s
✓ j17_cqlshlib_cython_tests7m 29s
✓ j17_cqlshlib_tests 6m 23s
✓ j17_dtests 33m 31s
✓ j17_dtests_latest 32m 40s
✓ j17_dtests_latest_repeat 0m 41s
✓ j17_dtests_vnode32m 27s
✓ j17_jvm_dtests 19m 22s
✓ j17_jvm_dtests_latest_vnode 14m 47s
✓ j17_jvm_dtests_latest_vnode_repeat 0m 31s
✓ j17_unit_tests 16m 20s
✓ j17_utests_latest 17m 14s
✓ j17_utests_latest_repeat 0m 32s
✓ j17_utests_oa 14m 48s
java17_separate_tests
java11_pre-commit_tests
✓ j11_build3m 52s
✓ j11_cqlsh_dtests_py3116m 1s
✓ j11_cqlsh_dtests_py311_vnode 6m 57s
✓ j11_cqlsh_dtests_py389m 48s
✓ j11_cqlsh_dtests_py38_vnode 7m 3s
✓ j11_cqlshlib_cython_tests9m 25s
✓ j11_cqlshlib_tests 6m 55s
✓ j11_dtests 33m 47s
✓ j11_dtests_latest 34m 34s
✓ j11_dtests_latest_repeat 4m 7s
✓ j11_dtests_vnode33m 15s
✓ j11_jvm_dtests 24m 12s
✓ j11_jvm_dtests_latest_vnode 15m 55s
✓ j11_jvm_dtests_latest_vnode_repeat 0m 34s
✓ j11_simulator_dtests 4m 36s
✓ j11_unit_tests 17m 0s
✓ j11_utests_latest 19m 28s
✓ j11_utests_latest_repeat 3m 54s
✓ j11_utests_oa 15m 56s
✓ j11_utests_system_keyspace_directory 18m 0s
✓ j17_cqlsh_dtests_py311 5m 54s
✓ j17_cqlsh_dtests_py311_vnode 6m 9s
✓ j17_cqlsh_dtests_py385m 56s
✓ j17_cqlsh_dtests_py38_vnode 6m 36s
✓ j17_cqlshlib_cython_tests7m 54s
✓ j17_cqlshlib_tests 6m 36s
✓ j17_dtests 33m 15s
✓ j17_dtests_latest 33m 52s
✓ j17_dtests_latest_repeat 1m 9s
✓ j17_dtests_vnode33m 16s
✓ j17_jvm_dtests 20m 15s
✓ j17_jvm_dtests_latest_vnode 15m 26s
✓ j17_jvm_dtests_latest_vnode_repeat 0m 37s
✓ j17_unit_tests 16m 19s
✓ j17_utests_latest 16m 26s
✓ j17_utests_latest_repeat 0m 20s
✓ j17_utests_oa15m 9s
java11_separate_tests
{noformat}
[java17_pre-commit_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3967/workflows/eeeabcb9-3499-438c-9af9-1f1717a0705c]
[java17_separate_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3967/workflows/9a437e65-5ec1-40c8-a69e-0291b12325d9]
[java11_pre-commit_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3967/workflows/04a11cec-977f-4cbd-894d-36123572c807]
[java11_separate_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3967/workflows/e6365105-089b-4e9f-bc6c-e961a40ba630]
> Fix tests to work with ULID SSTable identifiers to enable
> uuid_sstable_identifiers_enabled in cassandra-latest.yaml
> ---
>
> Key: CASSANDRA-19460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19460
> Project: Cassandra
> Issue Type: Task
> Components: CI, Test/dtest/java, Test/unit
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>
[jira] [Updated] (CASSANDRA-19391) Flush metadata snapshot table on every write
[ https://issues.apache.org/jira/browse/CASSANDRA-19391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-19391: Source Control Link: https://github.com/apache/cassandra/commit/f31d72e8758ee1018435612775c73f7addbb6ec4 Resolution: Fixed Status: Resolved (was: Ready to Commit) committed, thanks > Flush metadata snapshot table on every write > > > Key: CASSANDRA-19391 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19391 > Project: Cassandra > Issue Type: Improvement > Components: Transactional Cluster Metadata >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Low > Fix For: 5.x > > Attachments: ci_summary.html, result_details.tar.gz > > > We depend on the latest snapshot when starting up, flushing avoids gaps > between latest snapshot and the most recent local log entry -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19348) Fix serialization version check in InProgressSequences
[ https://issues.apache.org/jira/browse/CASSANDRA-19348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-19348: Since Version: 5.x Source Control Link: https://github.com/apache/cassandra/commit/60fe2dc61d3ea3d241e7e624f7a3ef94bd427b2f Resolution: Fixed Status: Resolved (was: Ready to Commit) committed, thanks > Fix serialization version check in InProgressSequences > -- > > Key: CASSANDRA-19348 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19348 > Project: Cassandra > Issue Type: Bug > Components: Transactional Cluster Metadata >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 5.x > > Attachments: ci_summary.html, result_details.tar.gz > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19390) Transformation.Kind should contain an explicit integer id
[ https://issues.apache.org/jira/browse/CASSANDRA-19390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-19390: Source Control Link: https://github.com/apache/cassandra/commit/0e5fb78c018245e1dfb25317d695665cb3317f3e Resolution: Fixed Status: Resolved (was: Ready to Commit) committed, thanks > Transformation.Kind should contain an explicit integer id > - > > Key: CASSANDRA-19390 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19390 > Project: Cassandra > Issue Type: Improvement > Components: Transactional Cluster Metadata >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Low > Fix For: 5.x > > Attachments: ci_summary.html, result_details.tar.gz > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) 02/03: Add an explicit id to Transformation.Kind
This is an automated email from the ASF dual-hosted git repository.
marcuse pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 0e5fb78c018245e1dfb25317d695665cb3317f3e
Author: Marcus Eriksson
AuthorDate: Mon Feb 12 12:57:30 2024 +0100
Add an explicit id to Transformation.Kind
Patch by marcuse; reviewed by Sam Tunnicliffe for CASSANDRA-19390
---
.../org/apache/cassandra/tcm/Transformation.java | 111 +
1 file changed, 67 insertions(+), 44 deletions(-)
diff --git a/src/java/org/apache/cassandra/tcm/Transformation.java
b/src/java/org/apache/cassandra/tcm/Transformation.java
index df15598b4d..943450aa53 100644
--- a/src/java/org/apache/cassandra/tcm/Transformation.java
+++ b/src/java/org/apache/cassandra/tcm/Transformation.java
@@ -20,6 +20,8 @@ package org.apache.cassandra.tcm;
import java.io.IOException;
import java.nio.ByteBuffer;
+import java.util.Arrays;
+import java.util.Comparator;
import java.util.function.Supplier;
import com.google.common.collect.ImmutableSet;
@@ -165,60 +167,81 @@ public interface Transformation
enum Kind
{
-PRE_INITIALIZE_CMS(() -> PreInitialize.serializer),
-INITIALIZE_CMS(() -> Initialize.serializer),
-FORCE_SNAPSHOT(() -> ForceSnapshot.serializer),
-SEAL_PERIOD(() -> SealPeriod.serializer),
-SCHEMA_CHANGE(() -> AlterSchema.serializer),
-REGISTER(() -> Register.serializer),
-UNREGISTER(() -> Unregister.serializer),
-
-UNSAFE_JOIN(() -> UnsafeJoin.serializer),
-PREPARE_JOIN(() -> PrepareJoin.serializer),
-START_JOIN(() -> PrepareJoin.StartJoin.serializer),
-MID_JOIN(() -> PrepareJoin.MidJoin.serializer),
-FINISH_JOIN(() -> PrepareJoin.FinishJoin.serializer),
-
-PREPARE_MOVE(() -> PrepareMove.serializer),
-START_MOVE(() -> PrepareMove.StartMove.serializer),
-MID_MOVE(() -> PrepareMove.MidMove.serializer),
-FINISH_MOVE(() -> PrepareMove.FinishMove.serializer),
-
-PREPARE_LEAVE(() -> PrepareLeave.serializer),
-START_LEAVE(() -> PrepareLeave.StartLeave.serializer),
-MID_LEAVE(() -> PrepareLeave.MidLeave.serializer),
-FINISH_LEAVE(() -> PrepareLeave.FinishLeave.serializer),
-ASSASSINATE(() -> Assassinate.serializer),
-
-PREPARE_REPLACE(() -> PrepareReplace.serializer),
-START_REPLACE(() -> PrepareReplace.StartReplace.serializer),
-MID_REPLACE(() -> PrepareReplace.MidReplace.serializer),
-FINISH_REPLACE(() -> PrepareReplace.FinishReplace.serializer),
-
-CANCEL_SEQUENCE(() -> CancelInProgressSequence.serializer),
+PRE_INITIALIZE_CMS(0, () -> PreInitialize.serializer),
+INITIALIZE_CMS(1, () -> Initialize.serializer),
+FORCE_SNAPSHOT(2, () -> ForceSnapshot.serializer),
+SEAL_PERIOD(3, () -> SealPeriod.serializer),
+SCHEMA_CHANGE(4, () -> AlterSchema.serializer),
+REGISTER(5, () -> Register.serializer),
+UNREGISTER(6, () -> Unregister.serializer),
+
+UNSAFE_JOIN(7, () -> UnsafeJoin.serializer),
+PREPARE_JOIN(8, () -> PrepareJoin.serializer),
+START_JOIN(9, () -> PrepareJoin.StartJoin.serializer),
+MID_JOIN(10, () -> PrepareJoin.MidJoin.serializer),
+FINISH_JOIN(11, () -> PrepareJoin.FinishJoin.serializer),
+
+PREPARE_MOVE(12, () -> PrepareMove.serializer),
+START_MOVE(13, () -> PrepareMove.StartMove.serializer),
+MID_MOVE(14, () -> PrepareMove.MidMove.serializer),
+FINISH_MOVE(15, () -> PrepareMove.FinishMove.serializer),
+
+PREPARE_LEAVE(16, () -> PrepareLeave.serializer),
+START_LEAVE(17, () -> PrepareLeave.StartLeave.serializer),
+MID_LEAVE(18, () -> PrepareLeave.MidLeave.serializer),
+FINISH_LEAVE(19, () -> PrepareLeave.FinishLeave.serializer),
+ASSASSINATE(20, () -> Assassinate.serializer),
+
+PREPARE_REPLACE(21, () -> PrepareReplace.serializer),
+START_REPLACE(22, () -> PrepareReplace.StartReplace.serializer),
+MID_REPLACE(23, () -> PrepareReplace.MidReplace.serializer),
+FINISH_REPLACE(24, () -> PrepareReplace.FinishReplace.serializer),
+
+CANCEL_SEQUENCE(25, () -> CancelInProgressSequence.serializer),
@Deprecated(since = "CEP-21")
-START_ADD_TO_CMS(() -> StartAddToCMS.serializer),
+START_ADD_TO_CMS(26, () -> StartAddToCMS.serializer),
@Deprecated(since = "CEP-21")
-FINISH_ADD_TO_CMS(() -> FinishAddToCMS.serializer),
+FINISH_ADD_TO_CMS(27, () -> FinishAddToCMS.serializer),
@Deprecated(since = "CEP-21")
-REMOVE_FROM_CMS(() -> RemoveFromCMS.serializer),
+REMOVE_FROM_CMS(28, () -> RemoveFromCMS.serializer),
-STARTUP(() -> Startup.serializer),
+STARTUP(29, () -> Startup.serializer),
-CUSTOM(() -> CustomTransformation.serializer),
+CUSTOM(30, () ->
(cassandra) 01/03: Fix version check in InProgressSequences serialization
This is an automated email from the ASF dual-hosted git repository.
marcuse pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 60fe2dc61d3ea3d241e7e624f7a3ef94bd427b2f
Author: Marcus Eriksson
AuthorDate: Wed Jan 31 12:52:16 2024 +0100
Fix version check in InProgressSequences serialization
Patch by marcuse; reviewed by Sam Tunnicliffe for CASSANDRA-19348
---
.../org/apache/cassandra/tcm/sequences/InProgressSequences.java | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git
a/src/java/org/apache/cassandra/tcm/sequences/InProgressSequences.java
b/src/java/org/apache/cassandra/tcm/sequences/InProgressSequences.java
index a8b2b16259..86b449c346 100644
--- a/src/java/org/apache/cassandra/tcm/sequences/InProgressSequences.java
+++ b/src/java/org/apache/cassandra/tcm/sequences/InProgressSequences.java
@@ -223,7 +223,7 @@ public class InProgressSequences implements
MetadataValue
out.writeInt(t.state.size());
for (Map.Entry> entry : t.state.entrySet())
{
-if (Version.UNKNOWN.isBefore(V2))
+if (version.isBefore(V2))
{
NodeId.serializer.serialize((NodeId) entry.getKey(), out,
version);
MultiStepOperation seq = entry.getValue();
@@ -249,7 +249,7 @@ public class InProgressSequences implements
MetadataValue
ImmutableMap.Builder> res = ImmutableMap.builder();
for (int i = 0; i < ipsSize; i++)
{
-if (Version.UNKNOWN.isBefore(V2))
+if (version.isBefore(V2))
{
NodeId nodeId = NodeId.serializer.deserialize(in, version);
MultiStepOperation.Kind kind =
MultiStepOperation.Kind.valueOf(in.readUTF());
@@ -273,7 +273,7 @@ public class InProgressSequences implements
MetadataValue
size += sizeof(t.state.size());
for (Map.Entry> entry : t.state.entrySet())
{
-if (Version.UNKNOWN.isBefore(V2))
+if (version.isBefore(V2))
{
size += NodeId.serializer.serializedSize((NodeId)
entry.getKey(), version);
MultiStepOperation seq = entry.getValue();
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) 03/03: Flush snapshot table on every write and introduce reversed long local partitioner.
This is an automated email from the ASF dual-hosted git repository.
marcuse pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit f31d72e8758ee1018435612775c73f7addbb6ec4
Author: Marcus Eriksson
AuthorDate: Mon Feb 12 14:23:25 2024 +0100
Flush snapshot table on every write and introduce reversed long local
partitioner.
Patch by marcuse; reviewed by Sam Tunnicliffe for CASSANDRA-19391
---
.../org/apache/cassandra/db/SystemKeyspace.java| 22 +-
.../dht/ReversedLongLocalPartitioner.java | 224 +
2 files changed, 241 insertions(+), 5 deletions(-)
diff --git a/src/java/org/apache/cassandra/db/SystemKeyspace.java
b/src/java/org/apache/cassandra/db/SystemKeyspace.java
index 9e6b53ee3c..ff54d226b0 100644
--- a/src/java/org/apache/cassandra/db/SystemKeyspace.java
+++ b/src/java/org/apache/cassandra/db/SystemKeyspace.java
@@ -62,7 +62,6 @@ import org.apache.cassandra.db.commitlog.CommitLogPosition;
import org.apache.cassandra.db.compaction.CompactionHistoryTabularData;
import org.apache.cassandra.db.marshal.BytesType;
import org.apache.cassandra.db.marshal.LongType;
-import org.apache.cassandra.db.marshal.ReversedType;
import org.apache.cassandra.db.marshal.TimeUUIDType;
import org.apache.cassandra.db.marshal.TupleType;
import org.apache.cassandra.db.marshal.UTF8Type;
@@ -73,6 +72,7 @@ import org.apache.cassandra.db.rows.Rows;
import org.apache.cassandra.dht.IPartitioner;
import org.apache.cassandra.dht.LocalPartitioner;
import org.apache.cassandra.dht.Range;
+import org.apache.cassandra.dht.ReversedLongLocalPartitioner;
import org.apache.cassandra.dht.Token;
import org.apache.cassandra.exceptions.ConfigurationException;
import org.apache.cassandra.gms.EndpointState;
@@ -505,7 +505,7 @@ public final class SystemKeyspace
"epoch bigint PRIMARY
KEY," +
"period bigint," +
"snapshot blob)")
- .partitioner(new
LocalPartitioner(ReversedType.getInstance(LongType.instance)))
+
.partitioner(ReversedLongLocalPartitioner.instance)
.build();
@Deprecated(since = "4.0")
@@ -1990,6 +1990,7 @@ public final class SystemKeyspace
logger.info("Storing snapshot of cluster metadata at epoch {} (period
{})", epoch, period);
String query = String.format("INSERT INTO %s.%s (epoch, period,
snapshot) VALUES (?, ?, ?)", SchemaConstants.SYSTEM_KEYSPACE_NAME,
SNAPSHOT_TABLE_NAME);
executeInternal(query, epoch.getEpoch(), period, snapshot);
+forceBlockingFlush(SNAPSHOT_TABLE_NAME);
}
public static ByteBuffer getSnapshot(Epoch epoch)
@@ -2003,12 +2004,23 @@ public final class SystemKeyspace
}
/**
- * WARNING: we use token(epoch) >= search in the query below - this is due
the fact that we use LocalPartitioner with a reversed LongToken
- * and this is not quite supported (yet), so the query is
actually `token(epoch) <= search` which is what we want here
+ * We use ReversedLongLocalPartitioner here, which calculates token as
Long.MAX_VALUE - key
+ *
+ * table is something like (assuming Long.MAX_VALUE is 1000 for easier
calculations...):
+ * epoch | token(epoch)
+ *
+ * 100 | 900
+ *90 | 910
+ *80 | 920
+ *70 | 970
+ *...
+ *
+ * so to find the first snapshot before epoch 85, we query the table with
token(epoch) >= token(85)=915. Which gives us
+ * epoch 80, 70... and the first row is the first snapshot before `search`
*/
public static ByteBuffer findSnapshotBefore(Epoch search)
{
-String query = String.format("SELECT snapshot FROM %s.%s WHERE
token(epoch) >= ? LIMIT 1", SchemaConstants.SYSTEM_KEYSPACE_NAME,
SNAPSHOT_TABLE_NAME);
+String query = String.format("SELECT snapshot FROM %s.%s WHERE
token(epoch) >= token(?) LIMIT 1", SchemaConstants.SYSTEM_KEYSPACE_NAME,
SNAPSHOT_TABLE_NAME);
UntypedResultSet res = executeInternal(query, search.getEpoch());
if (res != null && !res.isEmpty())
diff --git
a/src/java/org/apache/cassandra/dht/ReversedLongLocalPartitioner.java
b/src/java/org/apache/cassandra/dht/ReversedLongLocalPartitioner.java
new file mode 100644
index 00..c423ba7e13
--- /dev/null
+++ b/src/java/org/apache/cassandra/dht/ReversedLongLocalPartitioner.java
@@ -0,0 +1,224 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0
(cassandra) branch trunk updated (baef2447bb -> f31d72e875)
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from baef2447bb Merge branch 'cassandra-5.0' into trunk new 60fe2dc61d Fix version check in InProgressSequences serialization new 0e5fb78c01 Add an explicit id to Transformation.Kind new f31d72e875 Flush snapshot table on every write and introduce reversed long local partitioner. The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../org/apache/cassandra/db/SystemKeyspace.java| 22 +- .../dht/ReversedLongLocalPartitioner.java | 224 + .../org/apache/cassandra/tcm/Transformation.java | 111 ++ .../tcm/sequences/InProgressSequences.java | 6 +- 4 files changed, 311 insertions(+), 52 deletions(-) create mode 100644 src/java/org/apache/cassandra/dht/ReversedLongLocalPartitioner.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19348) Fix serialization version check in InProgressSequences
[ https://issues.apache.org/jira/browse/CASSANDRA-19348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-19348: Status: Ready to Commit (was: Review In Progress) +1 thanks! > Fix serialization version check in InProgressSequences > -- > > Key: CASSANDRA-19348 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19348 > Project: Cassandra > Issue Type: Bug > Components: Transactional Cluster Metadata >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 5.x > > Attachments: ci_summary.html, result_details.tar.gz > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
