[jira] [Commented] (CASSANDRA-14536) Bump the hints message version to match the current one
[
https://issues.apache.org/jira/browse/CASSANDRA-14536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16520001#comment-16520001
]
Jeff Jirsa commented on CASSANDRA-14536:
This isn't a formal review, but can you add a comment to the messaging service
version list to remind future developers to also consider bumping
{{HintsDescriptor.java}} when they bump messaging version?
> Bump the hints message version to match the current one
> ---
>
> Key: CASSANDRA-14536
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14536
> Project: Cassandra
> Issue Type: Improvement
> Components: Hints
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
>
> Currently in trunk the
> [{{Hints.messagingVersion()}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hints/HintsDescriptor.java#L217]
> is {{VERSION_30}}. The current {{messagingVersion}} is {{VERSION_40}}. Which
> causes ineffeicient hints dispatch:
> [{{HintsDispatcher.java:128}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hints/HintsDispatcher.java#L128]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-10540) RangeAwareCompaction
[
https://issues.apache.org/jira/browse/CASSANDRA-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519900#comment-16519900
]
Lerh Chuan Low edited comment on CASSANDRA-10540 at 6/22/18 3:16 AM:
-
Here is another benchmark run. It is still the same stressspec YAML. This time,
the process is to stop one of the nodes in a DC (Same as before, 3 in 1 DC and
2 in the other), and then insert for 10 minutes.
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=10m
cl=QUORUM ops\(insert=1\) -node file=nodelist.txt -rate threads=50 -log
file=insert.log > nohup.txt &
{code}
After that, trigger a stress but at the same time run a full repair in the DC:
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=1h
cl=QUORUM ops\(insert=10,simple1=10,range1=1\) -node file=nodelist.txt -rate
threads=50 -log file=mixed.log > nohup.txt &
nohup nodetool repair --full stresscql2 typestest > nohup.txt &
{code}
Here are the results:
|| ||RACS||Non RACS||
|Stress Result 1|Op rate : 244 op/s [insert: 116 op/s, range1: 12 op/s,
simple1: 116 op/s]
Partition rate : 243 pk/s [insert: 116 pk/s, range1: 10 pk/s, simple1: 116
pk/s]
Row rate : 274 row/s [insert: 116 row/s, range1: 41 row/s, simple1: 116 row/s]
Latency mean : 204.6 ms [insert: 2.5 ms, range1: 387.4 ms, simple1: 388.8 ms]
Latency median : 39.7 ms [insert: 2.0 ms, range1: 378.0 ms, simple1: 377.7 ms]
Latency 95th percentile : 706.2 ms [insert: 3.2 ms, range1: 802.2 ms, simple1:
805.3 ms]
Latency 99th percentile : 941.6 ms [insert: 19.7 ms, range1: 1,022.9 ms,
simple1: 1,022.4 ms]
Latency 99.9th percentile : 1183.8 ms [insert: 65.5 ms, range1: 1,232.1 ms,
simple1: 1,218.4 ms]
Latency max : 7314.9 ms [insert: 550.0 ms, range1: 1,472.2 ms, simple1:
7,314.9 ms]
Total partitions : 874,058 [insert: 419,116, range1: 36,428, simple1: 418,514]
Total errors : 0 [insert: 0, range1: 0, simple1: 0]
Total GC count : 0
Total GC memory : 0.000 KiB
Total GC time : 0.0 seconds
Avg GC time : NaN ms
StdDev GC time : 0.0 ms
Total operation time : 01:00:00|Op rate : 221 op/s [insert: 105 op/s, range1:
11 op/s, simple1: 105 op/s]
Partition rate : 220 pk/s [insert: 105 pk/s, range1: 9 pk/s, simple1: 105 pk/s]
Row rate : 248 row/s [insert: 105 row/s, range1: 38 row/s, simple1: 105 row/s]
Latency mean : 226.2 ms [insert: 2.7 ms, range1: 428.8 ms, simple1: 429.1 ms]
Latency median : 150.3 ms [insert: 2.0 ms, range1: 385.4 ms, simple1: 383.8 ms]
Latency 95th percentile : 716.2 ms [insert: 3.0 ms, range1: 837.3 ms, simple1:
841.5 ms]
Latency 99th percentile : 1047.5 ms [insert: 14.8 ms, range1: 1,210.1 ms,
simple1: 1,230.0 ms]
Latency 99.9th percentile : 1830.8 ms [insert: 57.5 ms, range1: 2,029.0 ms,
simple1: 2,063.6 ms]
Latency max : 7457.5 ms [insert: 6,358.6 ms, range1: 7,159.7 ms, simple1:
7,457.5 ms]
Total partitions : 790,543 [insert: 378,618, range1: 33,908, simple1: 378,017]
Total errors : 0 [insert: 0, range1: 0, simple1: 0]
Total GC count : 0
Total GC memory : 0.000 KiB
Total GC time : 0.0 seconds
Avg GC time : NaN ms
StdDev GC time : 0.0 ms
Total operation time : 01:00:00|
|Stress result 2|Op rate : 247 op/s [insert: 118 op/s, range1: 12 op/s,
simple1: 118 op/s]
Partition rate : 246 pk/s [insert: 118 pk/s, range1: 10 pk/s, simple1: 118 pk/s]
Row rate : 278 row/s [insert: 118 row/s, range1: 42 row/s, simple1: 118 row/s]
Latency mean : 201.9 ms [insert: 3.8 ms, range1: 384.5 ms, simple1: 382.3 ms]
Latency median : 45.0 ms [insert: 2.0 ms, range1: 374.1 ms, simple1: 372.2 ms]
Latency 95th percentile : 666.4 ms [insert: 10.0 ms, range1: 761.3 ms, simple1:
759.7 ms]
Latency 99th percentile : 888.7 ms [insert: 44.0 ms, range1: 973.1 ms, simple1:
968.9 ms]
Latency 99.9th percentile : 1135.6 ms [insert: 68.8 ms, range1: 1,182.8 ms,
simple1: 1,181.7 ms]
Latency max : 7101.0 ms [insert: 328.5 ms, range1: 6,970.9 ms, simple1: 7,101.0
ms]
Total partitions : 886,043 [insert: 425,188, range1: 36,972, simple1: 423,883]
Total errors : 0 [insert: 0, range1: 0, simple1: 0]
Total GC count : 0
Total GC memory : 0.000 KiB
Total GC time : 0.0 seconds
Avg GC time : NaN ms
StdDev GC time : 0.0 ms
Total operation time : 01:00:01|Op rate : 211 op/s [insert: 100 op/s, range1:
10 op/s, simple1: 101 op/s]
Partition rate : 210 pk/s [insert: 100 pk/s, range1: 9 pk/s, simple1: 101 pk/s]
Row rate : 236 row/s [insert: 100 row/s, range1: 35 row/s, simple1: 101 row/s]
Latency mean : 237.0 ms [insert: 2.5 ms, range1: 451.1 ms, simple1: 449.9 ms]
Latency median : 153.9 ms [insert: 2.0 ms, range1: 416.3 ms, simple1: 414.4 ms]
Latency 95th percentile : 753.4 ms [insert: 3.0 ms, range1: 867.7 ms, simple1:
864.0 ms]
Latency 99th percentile : 1016.6 ms [insert: 18.0 ms, range1: 1,118.8 ms,
simple1: 1,109.4 ms]
Latency 99.9th percentile : 1274.0 ms [insert: 58.6 ms, range1: 1,355.8 ms,
simple1: 1,344.3 ms]
[jira] [Assigned] (CASSANDRA-14421) Reenable upgrade tests
[
https://issues.apache.org/jira/browse/CASSANDRA-14421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Brown reassigned CASSANDRA-14421:
---
Assignee: Jason Brown
> Reenable upgrade tests
> --
>
> Key: CASSANDRA-14421
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14421
> Project: Cassandra
> Issue Type: Task
> Components: Testing
>Reporter: Sam Tunnicliffe
>Assignee: Jason Brown
>Priority: Major
> Fix For: 4.0
>
>
> Since dtests were switched to pytest & python3 in CASSANDRA-13134, the
> upgrade tests have been non-functional and are deselected by default (though
> even if you ran with the {{--execute-upgrade-tests}} they wouldn't work).
> They're further broken by CASSANDRA-14420, as {{upgrade_manifest}} relies on
> {{CASSANDRA_VERSION_FROM_BUILD}}. We need to get them, or something
> equivalent, up and running.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-10540) RangeAwareCompaction
[
https://issues.apache.org/jira/browse/CASSANDRA-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519900#comment-16519900
]
Lerh Chuan Low edited comment on CASSANDRA-10540 at 6/22/18 1:22 AM:
-
Here is another benchmark run. It is still the same stressspec YAML. This time,
the process is to stop one of the nodes in a DC (Same as before, 3 in 1 DC and
2 in the other), and then insert for 10 minutes.
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=10m
cl=QUORUM ops\(insert=1\) -node file=nodelist.txt -rate threads=50 -log
file=insert.log > nohup.txt &
{code}
After that, trigger a stress but at the same time run a full repair in the DC:
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=1h
cl=QUORUM ops\(insert=10,simple1=10,range1=1\) -node file=nodelist.txt -rate
threads=50 -log file=mixed.log > nohup.txt &
nohup nodetool repair --full stresscql2 typestest > nohup.txt &
{code}
Here are the results:
|| ||RACS||Non RACS||
|Stress Result|Op rate : 244 op/s [insert: 116 op/s, range1: 12 op/s, simple1:
116 op/s]
Partition rate : 243 pk/s [insert: 116 pk/s, range1: 10 pk/s, simple1: 116 pk/s]
Row rate : 274 row/s [insert: 116 row/s, range1: 41 row/s, simple1: 116 row/s]
Latency mean : 204.6 ms [insert: 2.5 ms, range1: 387.4 ms, simple1: 388.8 ms]
Latency median : 39.7 ms [insert: 2.0 ms, range1: 378.0 ms, simple1: 377.7 ms]
Latency 95th percentile : 706.2 ms [insert: 3.2 ms, range1: 802.2 ms, simple1:
805.3 ms]
Latency 99th percentile : 941.6 ms [insert: 19.7 ms, range1: 1,022.9 ms,
simple1: 1,022.4 ms]
Latency 99.9th percentile : 1183.8 ms [insert: 65.5 ms, range1: 1,232.1 ms,
simple1: 1,218.4 ms]
Latency max : 7314.9 ms [insert: 550.0 ms, range1: 1,472.2 ms, simple1: 7,314.9
ms]
Total partitions : 874,058 [insert: 419,116, range1: 36,428, simple1: 418,514]
Total errors : 0 [insert: 0, range1: 0, simple1: 0]
Total GC count : 0
Total GC memory : 0.000 KiB
Total GC time : 0.0 seconds
Avg GC time : NaN ms
StdDev GC time : 0.0 ms
Total operation time : 01:00:00|Op rate : 221 op/s [insert: 105 op/s, range1:
11 op/s, simple1: 105 op/s]
Partition rate : 220 pk/s [insert: 105 pk/s, range1: 9 pk/s, simple1: 105 pk/s]
Row rate : 248 row/s [insert: 105 row/s, range1: 38 row/s, simple1: 105 row/s]
Latency mean : 226.2 ms [insert: 2.7 ms, range1: 428.8 ms, simple1: 429.1 ms]
Latency median : 150.3 ms [insert: 2.0 ms, range1: 385.4 ms, simple1: 383.8 ms]
Latency 95th percentile : 716.2 ms [insert: 3.0 ms, range1: 837.3 ms, simple1:
841.5 ms]
Latency 99th percentile : 1047.5 ms [insert: 14.8 ms, range1: 1,210.1 ms,
simple1: 1,230.0 ms]
Latency 99.9th percentile : 1830.8 ms [insert: 57.5 ms, range1: 2,029.0 ms,
simple1: 2,063.6 ms]
Latency max : 7457.5 ms [insert: 6,358.6 ms, range1: 7,159.7 ms, simple1:
7,457.5 ms]
Total partitions : 790,543 [insert: 378,618, range1: 33,908, simple1: 378,017]
Total errors : 0 [insert: 0, range1: 0, simple1: 0]
Total GC count : 0
Total GC memory : 0.000 KiB
Total GC time : 0.0 seconds
Avg GC time : NaN ms
StdDev GC time : 0.0 ms
Total operation time : 01:00:00|
Big thanks to Jason Brown for the repair patch, works like a charm :)
was (Author: lerh low):
Here is another benchmark run. It is still the same stressspec YAML. This time,
the process is to stop one of the nodes in a DC (Same as before, 3 in 1 DC and
2 in the other), and then insert for 10 minutes.
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=10m
cl=QUORUM ops\(insert=1\) -node file=nodelist.txt -rate threads=50 -log
file=insert.log > nohup.txt &
{code}
After that, trigger a stress but at the same time run a full repair in the DC:
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=1h
cl=QUORUM ops\(insert=10,simple1=10,range1=1\) -node file=nodelist.txt -rate
threads=50 -log file=mixed.log > nohup.txt &
nohup nodetool repair --full stresscql2 typestest > nohup.txt &
{code}
Here are the results:
|| ||RACS||Non RACS||
|Stress Result|Op rate : 244 op/s [insert: 116 op/s, range1: 12 op/s, simple1:
116 op/s]
Partition rate : 243 pk/s [insert: 116 pk/s, range1: 10 pk/s, simple1: 116
pk/s]
Row rate : 274 row/s [insert: 116 row/s, range1: 41 row/s, simple1: 116 row/s]
Latency mean : 204.6 ms [insert: 2.5 ms, range1: 387.4 ms, simple1: 388.8 ms]
Latency median : 39.7 ms [insert: 2.0 ms, range1: 378.0 ms, simple1: 377.7 ms]
Latency 95th percentile : 706.2 ms [insert: 3.2 ms, range1: 802.2 ms, simple1:
805.3 ms] Latency 99th percentile : 941.6 ms [insert: 19.7 ms, range1: 1,022.9
ms, simple1: 1,022.4 ms] Latency 99.9th percentile : 1183.8 ms [insert: 65.5
ms, range1: 1,232.1 ms, simple1: 1,218.4 ms] Latency max : 7314.9 ms [insert:
550.0 ms, range1: 1,472.2 ms, simple1: 7,314.9 ms]
Total partitions : 874,058 [insert: 419,116, range1:
[jira] [Comment Edited] (CASSANDRA-10540) RangeAwareCompaction
[
https://issues.apache.org/jira/browse/CASSANDRA-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519900#comment-16519900
]
Lerh Chuan Low edited comment on CASSANDRA-10540 at 6/22/18 1:21 AM:
-
Here is another benchmark run. It is still the same stressspec YAML. This time,
the process is to stop one of the nodes in a DC (Same as before, 3 in 1 DC and
2 in the other), and then insert for 10 minutes.
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=10m
cl=QUORUM ops\(insert=1\) -node file=nodelist.txt -rate threads=50 -log
file=insert.log > nohup.txt &
{code}
After that, trigger a stress but at the same time run a full repair in the DC:
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=1h
cl=QUORUM ops\(insert=10,simple1=10,range1=1\) -node file=nodelist.txt -rate
threads=50 -log file=mixed.log > nohup.txt &
nohup nodetool repair --full stresscql2 typestest > nohup.txt &
{code}
Here are the results:
|| ||RACS||Non RACS||
|Stress Result|Op rate : 244 op/s [insert: 116 op/s, range1: 12 op/s, simple1:
116 op/s]
Partition rate : 243 pk/s [insert: 116 pk/s, range1: 10 pk/s, simple1: 116
pk/s]
Row rate : 274 row/s [insert: 116 row/s, range1: 41 row/s, simple1: 116 row/s]
Latency mean : 204.6 ms [insert: 2.5 ms, range1: 387.4 ms, simple1: 388.8 ms]
Latency median : 39.7 ms [insert: 2.0 ms, range1: 378.0 ms, simple1: 377.7 ms]
Latency 95th percentile : 706.2 ms [insert: 3.2 ms, range1: 802.2 ms, simple1:
805.3 ms] Latency 99th percentile : 941.6 ms [insert: 19.7 ms, range1: 1,022.9
ms, simple1: 1,022.4 ms] Latency 99.9th percentile : 1183.8 ms [insert: 65.5
ms, range1: 1,232.1 ms, simple1: 1,218.4 ms] Latency max : 7314.9 ms [insert:
550.0 ms, range1: 1,472.2 ms, simple1: 7,314.9 ms]
Total partitions : 874,058 [insert: 419,116, range1: 36,428, simple1: 418,514]
Total errors : 0 [insert: 0, range1: 0, simple1: 0]
Total GC count : 0
Total GC memory : 0.000 KiB
Total GC time : 0.0 seconds
Avg GC time : NaN ms
StdDev GC time : 0.0 ms
Total operation time : 01:00:00|Op rate : 221 op/s [insert: 105 op/s, range1:
11 op/s, simple1: 105 op/s]
Partition rate : 220 pk/s [insert: 105 pk/s, range1: 9 pk/s, simple1: 105 pk/s]
Row rate : 248 row/s [insert: 105 row/s, range1: 38 row/s, simple1: 105 row/s]
Latency mean : 226.2 ms [insert: 2.7 ms, range1: 428.8 ms, simple1: 429.1 ms]
Latency median : 150.3 ms [insert: 2.0 ms, range1: 385.4 ms, simple1: 383.8 ms]
Latency 95th percentile : 716.2 ms [insert: 3.0 ms, range1: 837.3 ms, simple1:
841.5 ms]
Latency 99th percentile : 1047.5 ms [insert: 14.8 ms, range1: 1,210.1 ms,
simple1: 1,230.0 ms]
Latency 99.9th percentile : 1830.8 ms [insert: 57.5 ms, range1: 2,029.0 ms,
simple1: 2,063.6 ms]
Latency max : 7457.5 ms [insert: 6,358.6 ms, range1: 7,159.7 ms, simple1:
7,457.5 ms]
Total partitions : 790,543 [insert: 378,618, range1: 33,908, simple1: 378,017]
Total errors : 0 [insert: 0, range1: 0, simple1: 0]
Total GC count : 0
Total GC memory : 0.000 KiB
Total GC time : 0.0 seconds
Avg GC time : NaN ms
StdDev GC time : 0.0 ms
Total operation time : 01:00:00|
Big thanks to Jason Brown for the repair patch, works like a charm :)
was (Author: lerh low):
Here is another benchmark run. It is still the same stressspec YAML. This time,
the process is to stop one of the nodes in a DC (Same as before, 3 in 1 DC and
2 in the other), and then insert for 10 minutes.
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=10m
cl=QUORUM ops\(insert=1\) -node file=nodelist.txt -rate threads=50 -log
file=insert.log > nohup.txt &
{code}
After that, trigger a stress but at the same time run a full repair in the DC:
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=1h
cl=QUORUM ops\(insert=10,simple1=10,range1=1\) -node file=nodelist.txt -rate
threads=50 -log file=mixed.log > nohup.txt &
nohup nodetool repair --full stresscql2 typestest > nohup.txt &
{code}
Here are the results:
|| ||RACS||Non RACS||
|Stress Result|Op rate : 244 op/s [insert: 116 op/s, range1: 12 op/s, simple1:
116 op/s]
Partition rate : 243 pk/s [insert: 116 pk/s, range1: 10 pk/s, simple1: 116
pk/s]
Row rate : 274 row/s [insert: 116 row/s, range1: 41 row/s, simple1: 116 row/s]
Latency mean : 204.6 ms [insert: 2.5 ms, range1: 387.4 ms, simple1: 388.8 ms]
Latency median : 39.7 ms [insert: 2.0 ms, range1: 378.0 ms, simple1: 377.7 ms]
Latency 95th percentile : 706.2 ms [insert: 3.2 ms, range1: 802.2 ms, simple1:
805.3 ms] Latency 99th percentile : 941.6 ms [insert: 19.7 ms, range1: 1,022.9
ms, simple1: 1,022.4 ms] Latency 99.9th percentile : 1183.8 ms [insert: 65.5
ms, range1: 1,232.1 ms, simple1: 1,218.4 ms] Latency max : 7314.9 ms [insert:
550.0 ms, range1: 1,472.2 ms, simple1: 7,314.9 ms]
Total partitions : 874,058 [insert: 419,116, range1:
[jira] [Commented] (CASSANDRA-10540) RangeAwareCompaction
[
https://issues.apache.org/jira/browse/CASSANDRA-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519900#comment-16519900
]
Lerh Chuan Low commented on CASSANDRA-10540:
Here is another benchmark run. It is still the same stressspec YAML. This time,
the process is to stop one of the nodes in a DC (Same as before, 3 in 1 DC and
2 in the other), and then insert for 10 minutes.
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=10m
cl=QUORUM ops\(insert=1\) -node file=nodelist.txt -rate threads=50 -log
file=insert.log > nohup.txt &
{code}
After that, trigger a stress but at the same time run a full repair in the DC:
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=1h
cl=QUORUM ops\(insert=10,simple1=10,range1=1\) -node file=nodelist.txt -rate
threads=50 -log file=mixed.log > nohup.txt &
nohup nodetool repair --full stresscql2 typestest > nohup.txt &
{code}
Here are the results:
|| ||RACS||Non RACS||
|Stress Result|Op rate : 244 op/s [insert: 116 op/s, range1: 12 op/s, simple1:
116 op/s]
Partition rate : 243 pk/s [insert: 116 pk/s, range1: 10 pk/s, simple1: 116
pk/s]
Row rate : 274 row/s [insert: 116 row/s, range1: 41 row/s, simple1: 116 row/s]
Latency mean : 204.6 ms [insert: 2.5 ms, range1: 387.4 ms, simple1: 388.8 ms]
Latency median : 39.7 ms [insert: 2.0 ms, range1: 378.0 ms, simple1: 377.7 ms]
Latency 95th percentile : 706.2 ms [insert: 3.2 ms, range1: 802.2 ms, simple1:
805.3 ms] Latency 99th percentile : 941.6 ms [insert: 19.7 ms, range1: 1,022.9
ms, simple1: 1,022.4 ms] Latency 99.9th percentile : 1183.8 ms [insert: 65.5
ms, range1: 1,232.1 ms, simple1: 1,218.4 ms] Latency max : 7314.9 ms [insert:
550.0 ms, range1: 1,472.2 ms, simple1: 7,314.9 ms]
Total partitions : 874,058 [insert: 419,116, range1: 36,428, simple1: 418,514]
Total errors : 0 [insert: 0, range1: 0, simple1: 0]
Total GC count : 0
Total GC memory : 0.000 KiB
Total GC time : 0.0 seconds
Avg GC time : NaN ms
StdDev GC time : 0.0 ms
Total operation time : 01:00:00|Op rate : 221 op/s [insert: 105 op/s, range1:
11 op/s, simple1: 105 op/s]
Partition rate : 220 pk/s [insert: 105 pk/s, range1: 9 pk/s, simple1: 105 pk/s]
Row rate : 248 row/s [insert: 105 row/s, range1: 38 row/s, simple1: 105 row/s]
Latency mean : 226.2 ms [insert: 2.7 ms, range1: 428.8 ms, simple1: 429.1 ms]
Latency median : 150.3 ms [insert: 2.0 ms, range1: 385.4 ms, simple1: 383.8 ms]
Latency 95th percentile : 716.2 ms [insert: 3.0 ms, range1: 837.3 ms, simple1:
841.5 ms]
Latency 99th percentile : 1047.5 ms [insert: 14.8 ms, range1: 1,210.1 ms,
simple1: 1,230.0 ms]
Latency 99.9th percentile : 1830.8 ms [insert: 57.5 ms, range1: 2,029.0 ms,
simple1: 2,063.6 ms]
Latency max : 7457.5 ms [insert: 6,358.6 ms, range1: 7,159.7 ms, simple1:
7,457.5 ms]
Total partitions : 790,543 [insert: 378,618, range1: 33,908, simple1: 378,017]
Total errors : 0 [insert: 0, range1: 0, simple1: 0]
Total GC count : 0
Total GC memory : 0.000 KiB
Total GC time : 0.0 seconds
Avg GC time : NaN ms
StdDev GC time : 0.0 ms
Total operation time : 01:00:00|
> RangeAwareCompaction
>
>
> Key: CASSANDRA-10540
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10540
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
> Labels: compaction, lcs, vnodes
> Fix For: 4.x
>
>
> Broken out from CASSANDRA-6696, we should split sstables based on ranges
> during compaction.
> Requirements;
> * dont create tiny sstables - keep them bunched together until a single vnode
> is big enough (configurable how big that is)
> * make it possible to run existing compaction strategies on the per-range
> sstables
> We should probably add a global compaction strategy parameter that states
> whether this should be enabled or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14536) Bump the hints message version to match the current one
[
https://issues.apache.org/jira/browse/CASSANDRA-14536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jay Zhuang updated CASSANDRA-14536:
---
Status: Patch Available (was: Open)
> Bump the hints message version to match the current one
> ---
>
> Key: CASSANDRA-14536
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14536
> Project: Cassandra
> Issue Type: Improvement
> Components: Hints
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
>
> Currently in trunk the
> [{{Hints.messagingVersion()}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hints/HintsDescriptor.java#L217]
> is {{VERSION_30}}. The current {{messagingVersion}} is {{VERSION_40}}. Which
> causes ineffeicient hints dispatch:
> [{{HintsDispatcher.java:128}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hints/HintsDispatcher.java#L128]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14536) Bump the hints message version to match the current one
[
https://issues.apache.org/jira/browse/CASSANDRA-14536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519890#comment-16519890
]
Jay Zhuang commented on CASSANDRA-14536:
{{3.11}} branch is fine, which is using the latest messaging version
{{VERSION_30}} or {{VERSION_3014}} in that branch.
Here is the patch to bump the hints message version, please review:
| Branch | uTest | dTest |
| [trunk|https://github.com/cooldoger/cassandra/tree/14536-trunk] |
[!https://circleci.com/gh/cooldoger/cassandra/tree/14536-trunk.svg?style=svg!|https://circleci.com/gh/cooldoger/cassandra/tree/14536-trunk]
|
[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/576/badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/576/]
|
> Bump the hints message version to match the current one
> ---
>
> Key: CASSANDRA-14536
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14536
> Project: Cassandra
> Issue Type: Improvement
> Components: Hints
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
>
> Currently in trunk the
> [{{Hints.messagingVersion()}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hints/HintsDescriptor.java#L217]
> is {{VERSION_30}}. The current {{messagingVersion}} is {{VERSION_40}}. Which
> causes ineffeicient hints dispatch:
> [{{HintsDispatcher.java:128}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hints/HintsDispatcher.java#L128]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14536) Bump the hints message version to match the current one
Jay Zhuang created CASSANDRA-14536:
--
Summary: Bump the hints message version to match the current one
Key: CASSANDRA-14536
URL: https://issues.apache.org/jira/browse/CASSANDRA-14536
Project: Cassandra
Issue Type: Improvement
Components: Hints
Reporter: Jay Zhuang
Assignee: Jay Zhuang
Currently in trunk the
[{{Hints.messagingVersion()}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hints/HintsDescriptor.java#L217]
is {{VERSION_30}}. The current {{messagingVersion}} is {{VERSION_40}}. Which
causes ineffeicient hints dispatch:
[{{HintsDispatcher.java:128}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hints/HintsDispatcher.java#L128]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14376) Limiting a clustering column with a range not allowed when using "group by"
[
https://issues.apache.org/jira/browse/CASSANDRA-14376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519847#comment-16519847
]
Alexander Ivakov commented on CASSANDRA-14376:
--
Cassandra currently doesn't implement group by on arbitrary columns. When
grouping by multiple columns, you can only group by primary key columns in the
order they are declared in the primary key, starting from the first.
The only exception allowed is if you have restricted the first n columns with
an equality restriction. Note, restricting a column with an "=" is selecting
one group, so there is nothing to group in that column, that's why Cassandra
allows this. You can then group by the remaining columns, in order and starting
from the next column (you can’t skip columns in between).
So to group by a column, all preceding primary key columns have to be either
restricted by "=" or be in the group by clause.
The range query like the one above fails because the sample column is being
restricted by a range, thus still having multiple groups, but is not in the
group by clause. LIKE and IN restrictions will also not work in this case.
This query will work:
select city, state, sum(count) from samples where name='bob' and partition=1
and sample>=1 and sample<=3 group by sample, city, state;
The partition key columns are restricted by "=" and all the clustering key
columns are in the group by clause in the same order as the schema. “Group by
sample” and “group by sample, city” will also work.
> Limiting a clustering column with a range not allowed when using "group by"
> ---
>
> Key: CASSANDRA-14376
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14376
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
> Environment: Cassandra 3.11.1
>Reporter: Chris mildebrandt
>Priority: Major
>
> I’m trying to use a range to limit a clustering column while at the same time
> using `group by` and running into issues. Here’s a sample table:
> {{create table if not exists samples (name text, partition int, sample int,
> city text, state text, count counter, primary key ((name, partition), sample,
> city, state)) with clustering order by (sample desc);}}
> When I filter `sample` by a range, I get an error:
> {{select city, state, sum(count) from samples where name='bob' and
> partition=1 and sample>=1 and sample<=3 group by city, state;}}
> {{{color:#ff}InvalidRequest: Error from server: code=2200 [Invalid
> query] message="Group by currently only support groups of columns following
> their declared order in the PRIMARY KEY"{color}}}
> However, it allows the query when I change from a range to an equals:
> {{select city, state, sum(count) from samples where name='bob' and
> partition=1 and sample=1 group by city, state;}}
> {{city | state | system.sum(count)}}
> {{++--}}
> {{ Austin | TX | 2}}
> {{ Denver | CO | 1}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14376) Limiting a clustering column with a range not allowed when using "group by"
[
https://issues.apache.org/jira/browse/CASSANDRA-14376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519847#comment-16519847
]
Alexander Ivakov edited comment on CASSANDRA-14376 at 6/21/18 11:41 PM:
Cassandra currently doesn't implement group by on arbitrary columns. When
grouping by multiple columns, you can only group by primary key columns in the
order they are declared in the primary key, starting from the first.
The only exception allowed is if you have restricted the first n columns with
an equality restriction. Note, restricting a column with an "=" is selecting
one group, so there is nothing to group in that column, that's why Cassandra
allows this. You can then group by the remaining columns, in order and starting
from the next column (you can’t skip columns in between).
So to group by a column, all preceding primary key columns have to be either
restricted by "=" or be in the group by clause.
The range query like the one above fails because the sample column is being
restricted by a range, thus still having multiple groups, but is not in the
group by clause. LIKE and IN restrictions will also not work in this case.
This query will work:
select city, state, sum(count) from samples where name='bob' and partition=1
and sample>=1 and sample<=3 group by sample, city, state;
The partition key columns are restricted by "=" and all the clustering key
columns are in the group by clause in the same order as the schema. “Group by
sample” and “group by sample, city” will also work.
was (Author: alex.ivakov):
Cassandra currently doesn't implement group by on arbitrary columns. When
grouping by multiple columns, you can only group by primary key columns in the
order they are declared in the primary key, starting from the first.
The only exception allowed is if you have restricted the first n columns with
an equality restriction. Note, restricting a column with an "=" is selecting
one group, so there is nothing to group in that column, that's why Cassandra
allows this. You can then group by the remaining columns, in order and starting
from the next column (you can’t skip columns in between).
So to group by a column, all preceding primary key columns have to be either
restricted by "=" or be in the group by clause.
The range query like the one above fails because the sample column is being
restricted by a range, thus still having multiple groups, but is not in the
group by clause. LIKE and IN restrictions will also not work in this case.
This query will work:
select city, state, sum(count) from samples where name='bob' and partition=1
and sample>=1 and sample<=3 group by sample, city, state;
The partition key columns are restricted by "=" and all the clustering key
columns are in the group by clause in the same order as the schema. “Group by
sample” and “group by sample, city” will also work.
> Limiting a clustering column with a range not allowed when using "group by"
> ---
>
> Key: CASSANDRA-14376
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14376
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
> Environment: Cassandra 3.11.1
>Reporter: Chris mildebrandt
>Priority: Major
>
> I’m trying to use a range to limit a clustering column while at the same time
> using `group by` and running into issues. Here’s a sample table:
> {{create table if not exists samples (name text, partition int, sample int,
> city text, state text, count counter, primary key ((name, partition), sample,
> city, state)) with clustering order by (sample desc);}}
> When I filter `sample` by a range, I get an error:
> {{select city, state, sum(count) from samples where name='bob' and
> partition=1 and sample>=1 and sample<=3 group by city, state;}}
> {{{color:#ff}InvalidRequest: Error from server: code=2200 [Invalid
> query] message="Group by currently only support groups of columns following
> their declared order in the PRIMARY KEY"{color}}}
> However, it allows the query when I change from a range to an equals:
> {{select city, state, sum(count) from samples where name='bob' and
> partition=1 and sample=1 group by city, state;}}
> {{city | state | system.sum(count)}}
> {{++--}}
> {{ Austin | TX | 2}}
> {{ Denver | CO | 1}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14532) Partition level deletions past GCGS are not propagated/merged on read
[
https://issues.apache.org/jira/browse/CASSANDRA-14532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kurt Greaves updated CASSANDRA-14532:
-
Priority: Minor (was: Major)
> Partition level deletions past GCGS are not propagated/merged on read
> -
>
> Key: CASSANDRA-14532
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14532
> Project: Cassandra
> Issue Type: Bug
> Components: Documentation and Website
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Minor
>
> So as [~jay.zhuang] mentioned on the mailing list
> [here|http://mail-archives.us.apache.org/mod_mbox/cassandra-dev/201806.mbox/],
> it appears that partition deletions that have passed GCGS are not
> propagated/merged properly on read, and also not repaired via read repair.
> Steps to reproduce:
> {code}
> create keyspace test WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 3};
> create table test.test (id int PRIMARY KEY , data text) WITH gc_grace_seconds
> = 10;
> CONSISTENCY ALL;
> INSERT INTO test.test (id, data) values (1, 'test');
> ccm node2 stop
> CONSISTENCY QUORUM;
> DELETE from test.test where id = 1; // wait 10 seconds so HH doesn't
> propagate tombstone when starting node2
> select * from test.test where id = 1 ;
> id | data
> +--
> (0 rows)
> ccm node2 start
> CONSISTENCY ALL;
> select * from test.test where id = 1 ;
> id | data
> +--
> 1 | test
> alter table test.test WITH gc_grace_seconds = 10; // GC
> select * from test.test where id = 1 ;
> id | data
> +--
> (0 rows)
> {code}
> We've also found a seemingly related issue in compaction where trying to
> compact an SSTable which contains the partition deletion post GCGS, the
> partition deletion won't be removed via compaction. Likely the same code is
> causing both bugs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14532) Partition level deletions past GCGS are not propagated/merged on read
[
https://issues.apache.org/jira/browse/CASSANDRA-14532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kurt Greaves updated CASSANDRA-14532:
-
Component/s: Documentation and Website
> Partition level deletions past GCGS are not propagated/merged on read
> -
>
> Key: CASSANDRA-14532
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14532
> Project: Cassandra
> Issue Type: Bug
> Components: Documentation and Website
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Minor
>
> So as [~jay.zhuang] mentioned on the mailing list
> [here|http://mail-archives.us.apache.org/mod_mbox/cassandra-dev/201806.mbox/],
> it appears that partition deletions that have passed GCGS are not
> propagated/merged properly on read, and also not repaired via read repair.
> Steps to reproduce:
> {code}
> create keyspace test WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 3};
> create table test.test (id int PRIMARY KEY , data text) WITH gc_grace_seconds
> = 10;
> CONSISTENCY ALL;
> INSERT INTO test.test (id, data) values (1, 'test');
> ccm node2 stop
> CONSISTENCY QUORUM;
> DELETE from test.test where id = 1; // wait 10 seconds so HH doesn't
> propagate tombstone when starting node2
> select * from test.test where id = 1 ;
> id | data
> +--
> (0 rows)
> ccm node2 start
> CONSISTENCY ALL;
> select * from test.test where id = 1 ;
> id | data
> +--
> 1 | test
> alter table test.test WITH gc_grace_seconds = 10; // GC
> select * from test.test where id = 1 ;
> id | data
> +--
> (0 rows)
> {code}
> We've also found a seemingly related issue in compaction where trying to
> compact an SSTable which contains the partition deletion post GCGS, the
> partition deletion won't be removed via compaction. Likely the same code is
> causing both bugs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14532) Partition level deletions past GCGS are not propagated/merged on read
[
https://issues.apache.org/jira/browse/CASSANDRA-14532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519843#comment-16519843
]
Kurt Greaves commented on CASSANDRA-14532:
--
bq. That's not really a bug in that this is working as designed. The very
reason for gc grace is to be a long enough time that we can guarantee any data
(including tombstone) has been propagated to all replica, and that's why you
must run repair within the gc grace window (otherwise other mechanism don't
truly guarantee that). So we should not have to propagate anything past gcgs
and doing so is at best an inefficiency.
Yeah we've come to that conclusion on the ML already. I haven't updated here
but I think this ticket really becomes a case of documenting the behaviour a
bit more explicitly. It's pretty hard to reason through, and having already
reasoned through it in the past and forgotten, and that it's happened to other
people seems to make this a good case for documentation. I'd like to put a
better explanation in the code, but more importantly, in the docs somewhere.
I'll look at that in the near future, but regarding docs we don't really have
much (anything?) on tombstones at the moment and it would be terribly out of
place to document this without a whole lot of general/background knowledge on
tombstones. Anyway, I'll get around to writing that up as well at some point...
bq. That would be a real bug, though - worth opening a JIRA there.
bq. Possibly. But remember that being post GCGS is not the only condition on
tombstone for them to be purged: compaction also needs to be able to prove the
tombstone cannot shadow something in another, non-compacted, sstable. So the
sentence is not precise enough to say this is a bug for sure either.
So I'll get exact reproduce steps sorted, but I left out some details in my
original comment which makes it significantly more minor. In this case what we
see is that you can create a single SSTable containing only a partition
deletion with no shadowed data in any other SSTable and when you compact the
tombstone won't be removed. However, as soon as you insert more data that
shadows the tombstone and compact with the SSTable it will be removed (there
may be more to it than this, have to test again).
Either way we figured that's not a major bug in the scheme of things, but I'll
get a test case up and running so can investigate a bit more.
> Partition level deletions past GCGS are not propagated/merged on read
> -
>
> Key: CASSANDRA-14532
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14532
> Project: Cassandra
> Issue Type: Bug
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Major
>
> So as [~jay.zhuang] mentioned on the mailing list
> [here|http://mail-archives.us.apache.org/mod_mbox/cassandra-dev/201806.mbox/],
> it appears that partition deletions that have passed GCGS are not
> propagated/merged properly on read, and also not repaired via read repair.
> Steps to reproduce:
> {code}
> create keyspace test WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 3};
> create table test.test (id int PRIMARY KEY , data text) WITH gc_grace_seconds
> = 10;
> CONSISTENCY ALL;
> INSERT INTO test.test (id, data) values (1, 'test');
> ccm node2 stop
> CONSISTENCY QUORUM;
> DELETE from test.test where id = 1; // wait 10 seconds so HH doesn't
> propagate tombstone when starting node2
> select * from test.test where id = 1 ;
> id | data
> +--
> (0 rows)
> ccm node2 start
> CONSISTENCY ALL;
> select * from test.test where id = 1 ;
> id | data
> +--
> 1 | test
> alter table test.test WITH gc_grace_seconds = 10; // GC
> select * from test.test where id = 1 ;
> id | data
> +--
> (0 rows)
> {code}
> We've also found a seemingly related issue in compaction where trying to
> compact an SSTable which contains the partition deletion post GCGS, the
> partition deletion won't be removed via compaction. Likely the same code is
> causing both bugs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13010) nodetool compactionstats should say which disk a compaction is writing to
[ https://issues.apache.org/jira/browse/CASSANDRA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519829#comment-16519829 ] Jon Haddad commented on CASSANDRA-13010: Hey Alex, I'll try to get to this tomorrow. > nodetool compactionstats should say which disk a compaction is writing to > - > > Key: CASSANDRA-13010 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13010 > Project: Cassandra > Issue Type: New Feature > Components: Compaction, Tools >Reporter: Jon Haddad >Assignee: Alex Lourie >Priority: Major > Labels: lhf > Attachments: cleanup.png, multiple operations.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14421) Reenable upgrade tests
[
https://issues.apache.org/jira/browse/CASSANDRA-14421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519816#comment-16519816
]
Dinesh Joshi commented on CASSANDRA-14421:
--
vstring was an attribute of Version -
[https://github.com/python/cpython/blob/master/Lib/distutils/version.py#L307]
We moved away from using it but code references vstring property causing it to
break. Looks like we can rely on raw version strings now.
> Reenable upgrade tests
> --
>
> Key: CASSANDRA-14421
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14421
> Project: Cassandra
> Issue Type: Task
> Components: Testing
>Reporter: Sam Tunnicliffe
>Priority: Major
> Fix For: 4.0
>
>
> Since dtests were switched to pytest & python3 in CASSANDRA-13134, the
> upgrade tests have been non-functional and are deselected by default (though
> even if you ran with the {{--execute-upgrade-tests}} they wouldn't work).
> They're further broken by CASSANDRA-14420, as {{upgrade_manifest}} relies on
> {{CASSANDRA_VERSION_FROM_BUILD}}. We need to get them, or something
> equivalent, up and running.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14535) dtest for checking orphan hint files post node removal
[ https://issues.apache.org/jira/browse/CASSANDRA-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519620#comment-16519620 ] Jaydeepkumar Chovatia commented on CASSANDRA-14535: --- Please find dtest here: ||dtest|| |[patch|https://github.com/apache/cassandra-dtest/compare/master...jaydeepkumar1984:14535-dtest?expand=1]| [~iamaleksey] Could you please review this dtest which I wrote for CASSANDRA-13740? > dtest for checking orphan hint files post node removal > -- > > Key: CASSANDRA-14535 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14535 > Project: Cassandra > Issue Type: Sub-task > Components: Testing >Reporter: Jaydeepkumar Chovatia >Assignee: Jaydeepkumar Chovatia >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14458) Add virtual table to list active connections
[
https://issues.apache.org/jira/browse/CASSANDRA-14458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-14458:
--
Resolution: Fixed
Fix Version/s: (was: 4.x)
4.0
Status: Resolved (was: Patch Available)
> Add virtual table to list active connections
>
>
> Key: CASSANDRA-14458
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14458
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.0
>
>
> List all active connections in virtual table like:
> {code:sql}
> cqlsh:system> select * from system_views.clients ;
>
> client_address | cipher | driver_name | driver_version | keyspace |
> protocol | requests | ssl | user | version
> --+---+-++--+---+--+---+---+-
> /127.0.0.1:63903 | undefined | undefined | undefined | |
> undefined | 13 | False | anonymous | 4
> /127.0.0.1:63904 | undefined | undefined | undefined | system |
> undefined | 16 | False | anonymous | 4
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14458) Add virtual table to list active connections
[
https://issues.apache.org/jira/browse/CASSANDRA-14458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519615#comment-16519615
]
Aleksey Yeschenko commented on CASSANDRA-14458:
---
Neat. +1.
Committed as
[46a5514c2aa7f377e8dc4cfd0d701b940f3137c7|https://github.com/apache/cassandra/commit/46a5514c2aa7f377e8dc4cfd0d701b940f3137c7]
to trunk.
> Add virtual table to list active connections
>
>
> Key: CASSANDRA-14458
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14458
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.0
>
>
> List all active connections in virtual table like:
> {code:sql}
> cqlsh:system> select * from system_views.clients ;
>
> client_address | cipher | driver_name | driver_version | keyspace |
> protocol | requests | ssl | user | version
> --+---+-++--+---+--+---+---+-
> /127.0.0.1:63903 | undefined | undefined | undefined | |
> undefined | 13 | False | anonymous | 4
> /127.0.0.1:63904 | undefined | undefined | undefined | system |
> undefined | 16 | False | anonymous | 4
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14535) dtest for checking orphan hint files post node removal
Jaydeepkumar Chovatia created CASSANDRA-14535: - Summary: dtest for checking orphan hint files post node removal Key: CASSANDRA-14535 URL: https://issues.apache.org/jira/browse/CASSANDRA-14535 Project: Cassandra Issue Type: Sub-task Components: Testing Reporter: Jaydeepkumar Chovatia Assignee: Jaydeepkumar Chovatia -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
cassandra git commit: Add a virtual table to expose active client connections
Repository: cassandra
Updated Branches:
refs/heads/trunk 53c71949d -> 46a5514c2
Add a virtual table to expose active client connections
patch by Chris Lohfink; reviewed by Aleksey Yeschenko for
CASSANDRA-14458
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/46a5514c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/46a5514c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/46a5514c
Branch: refs/heads/trunk
Commit: 46a5514c2aa7f377e8dc4cfd0d701b940f3137c7
Parents: 53c7194
Author: Chris Lohfink
Authored: Thu Jun 21 11:36:30 2018 -0500
Committer: Aleksey Yeshchenko
Committed: Thu Jun 21 18:30:50 2018 +0100
--
CHANGES.txt | 1 +
.../cassandra/db/virtual/ClientsTable.java | 86
.../db/virtual/SystemViewsKeyspace.java | 2 +-
.../apache/cassandra/metrics/ClientMetrics.java | 10 +++
.../cassandra/transport/ConnectedClient.java| 4 +-
5 files changed, 100 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/46a5514c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 598eaff..ce945df 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0
+ * Add a virtual table to expose active client connections (CASSANDRA-14458)
* Clean up and refactor client metrics (CASSANDRA-14524)
* Nodetool import row cache invalidation races with adding sstables to
tracker (CASSANDRA-14529)
* Fix assertions in LWTs after TableMetadata was made immutable
(CASSANDRA-14356)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/46a5514c/src/java/org/apache/cassandra/db/virtual/ClientsTable.java
--
diff --git a/src/java/org/apache/cassandra/db/virtual/ClientsTable.java
b/src/java/org/apache/cassandra/db/virtual/ClientsTable.java
new file mode 100644
index 000..98d1a28
--- /dev/null
+++ b/src/java/org/apache/cassandra/db/virtual/ClientsTable.java
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.db.virtual;
+
+import java.net.InetSocketAddress;
+
+import org.apache.cassandra.db.marshal.*;
+import org.apache.cassandra.metrics.ClientMetrics;
+import org.apache.cassandra.schema.TableMetadata;
+import org.apache.cassandra.transport.ConnectedClient;
+
+final class ClientsTable extends AbstractVirtualTable
+{
+private static final String ADDRESS = "address";
+private static final String PORT = "port";
+private static final String HOSTNAME = "hostname";
+private static final String USERNAME = "username";
+private static final String CONNECTION_STAGE = "connection_stage";
+private static final String PROTOCOL_VERSION = "protocol_version";
+private static final String DRIVER_NAME = "driver_name";
+private static final String DRIVER_VERSION = "driver_version";
+private static final String REQUEST_COUNT = "request_count";
+private static final String SSL_ENABLED = "ssl_enabled";
+private static final String SSL_PROTOCOL = "ssl_protocol";
+private static final String SSL_CIPHER_SUITE = "ssl_cipher_suite";
+
+ClientsTable(String keyspace)
+{
+super(TableMetadata.builder(keyspace, "clients")
+ .comment("currently connected clients")
+ .kind(TableMetadata.Kind.VIRTUAL)
+ .addPartitionKeyColumn(ADDRESS,
InetAddressType.instance)
+ .addClusteringColumn(PORT, Int32Type.instance)
+ .addRegularColumn(HOSTNAME, UTF8Type.instance)
+ .addRegularColumn(USERNAME, UTF8Type.instance)
+ .addRegularColumn(CONNECTION_STAGE,
UTF8Type.instance)
+ .addRegularColumn(PROTOCOL_VERSION,
Int32Type.instance)
+ .addRegularColumn(DRIVER_NAME, UTF8Type.instance)
+
[jira] [Commented] (CASSANDRA-14532) Partition level deletions past GCGS are not propagated/merged on read
[
https://issues.apache.org/jira/browse/CASSANDRA-14532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519521#comment-16519521
]
Sylvain Lebresne commented on CASSANDRA-14532:
--
bq. That would be a real bug, though - worth opening a JIRA there.
Possibly. But remember that being post GCGS is not the _only_ condition on
tombstone for them to be purged: compaction also needs to be able to prove the
tombstone cannot shadow something in another, non-compacted, sstable. So the
sentence is not precise enough to say this is a bug for sure either.
> Partition level deletions past GCGS are not propagated/merged on read
> -
>
> Key: CASSANDRA-14532
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14532
> Project: Cassandra
> Issue Type: Bug
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Major
>
> So as [~jay.zhuang] mentioned on the mailing list
> [here|http://mail-archives.us.apache.org/mod_mbox/cassandra-dev/201806.mbox/],
> it appears that partition deletions that have passed GCGS are not
> propagated/merged properly on read, and also not repaired via read repair.
> Steps to reproduce:
> {code}
> create keyspace test WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 3};
> create table test.test (id int PRIMARY KEY , data text) WITH gc_grace_seconds
> = 10;
> CONSISTENCY ALL;
> INSERT INTO test.test (id, data) values (1, 'test');
> ccm node2 stop
> CONSISTENCY QUORUM;
> DELETE from test.test where id = 1; // wait 10 seconds so HH doesn't
> propagate tombstone when starting node2
> select * from test.test where id = 1 ;
> id | data
> +--
> (0 rows)
> ccm node2 start
> CONSISTENCY ALL;
> select * from test.test where id = 1 ;
> id | data
> +--
> 1 | test
> alter table test.test WITH gc_grace_seconds = 10; // GC
> select * from test.test where id = 1 ;
> id | data
> +--
> (0 rows)
> {code}
> We've also found a seemingly related issue in compaction where trying to
> compact an SSTable which contains the partition deletion post GCGS, the
> partition deletion won't be removed via compaction. Likely the same code is
> causing both bugs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14524) Client metrics refactor
[ https://issues.apache.org/jira/browse/CASSANDRA-14524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519514#comment-16519514 ] Aleksey Yeschenko commented on CASSANDRA-14524: --- Committed to trunk as [53c71949d4a49d6062e43ff3b1a9ca3e94496cfb|https://github.com/apache/cassandra/commit/53c71949d4a49d6062e43ff3b1a9ca3e94496cfb], thanks. > Client metrics refactor > --- > > Key: CASSANDRA-14524 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14524 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Aleksey Yeschenko >Priority: Minor > Fix For: 4.0 > > > Moving refactoring out of CASSANDRA-14458 to be reviewed separately. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14524) Client metrics refactor
[ https://issues.apache.org/jira/browse/CASSANDRA-14524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14524: -- Resolution: Fixed Fix Version/s: (was: 4.0.x) 4.0 Status: Resolved (was: Patch Available) > Client metrics refactor > --- > > Key: CASSANDRA-14524 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14524 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Aleksey Yeschenko >Priority: Minor > Fix For: 4.0 > > > Moving refactoring out of CASSANDRA-14458 to be reviewed separately. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
cassandra git commit: Clean up and refactor client metrics
Repository: cassandra
Updated Branches:
refs/heads/trunk 2d8ae8373 -> 53c71949d
Clean up and refactor client metrics
patch by Aleksey Yeschenko and Chris Lohfink for CASSANDRA-14524
Co-authored-by: Chris Lohfink
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/53c71949
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/53c71949
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/53c71949
Branch: refs/heads/trunk
Commit: 53c71949d4a49d6062e43ff3b1a9ca3e94496cfb
Parents: 2d8ae83
Author: Aleksey Yeschenko
Authored: Sat Jun 16 22:05:51 2018 -0500
Committer: Aleksey Yeshchenko
Committed: Thu Jun 21 16:01:18 2018 +0100
--
CHANGES.txt | 1 +
.../apache/cassandra/metrics/AuthMetrics.java | 40 --
.../apache/cassandra/metrics/ClientMetrics.java | 109 +++---
.../service/NativeTransportService.java | 53 +--
.../org/apache/cassandra/tools/NodeProbe.java | 2 +-
.../cassandra/tools/nodetool/ClientStats.java | 23 ++-
.../apache/cassandra/transport/ClientStat.java | 56
.../cassandra/transport/ConnectedClient.java| 144 +++
.../cassandra/transport/ConnectionStage.java| 23 +++
.../transport/ProtocolVersionTracker.java | 75 --
.../org/apache/cassandra/transport/Server.java | 71 +++--
.../cassandra/transport/ServerConnection.java | 31 ++--
.../transport/messages/AuthResponse.java| 6 +-
.../transport/ProtocolVersionTrackerTest.java | 23 ++-
14 files changed, 409 insertions(+), 248 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/53c71949/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 041a655..598eaff 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0
+ * Clean up and refactor client metrics (CASSANDRA-14524)
* Nodetool import row cache invalidation races with adding sstables to
tracker (CASSANDRA-14529)
* Fix assertions in LWTs after TableMetadata was made immutable
(CASSANDRA-14356)
* Abort compactions quicker (CASSANDRA-14397)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/53c71949/src/java/org/apache/cassandra/metrics/AuthMetrics.java
--
diff --git a/src/java/org/apache/cassandra/metrics/AuthMetrics.java
b/src/java/org/apache/cassandra/metrics/AuthMetrics.java
deleted file mode 100644
index 126738c..000
--- a/src/java/org/apache/cassandra/metrics/AuthMetrics.java
+++ /dev/null
@@ -1,40 +0,0 @@
-package org.apache.cassandra.metrics;
-
-import com.codahale.metrics.Meter;
-
-/**
- * Metrics about authentication
- */
-public class AuthMetrics
-{
-
-public static final AuthMetrics instance = new AuthMetrics();
-
-public static void init()
-{
-// no-op, just used to force instance creation
-}
-
-/** Number and rate of successful logins */
-protected final Meter success;
-
-/** Number and rate of login failures */
-protected final Meter failure;
-
-private AuthMetrics()
-{
-
-success = ClientMetrics.instance.addMeter("AuthSuccess");
-failure = ClientMetrics.instance.addMeter("AuthFailure");
-}
-
-public void markSuccess()
-{
-success.mark();
-}
-
-public void markFailure()
-{
-failure.mark();
-}
-}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/53c71949/src/java/org/apache/cassandra/metrics/ClientMetrics.java
--
diff --git a/src/java/org/apache/cassandra/metrics/ClientMetrics.java
b/src/java/org/apache/cassandra/metrics/ClientMetrics.java
index 8ca3480..5e7720a 100644
--- a/src/java/org/apache/cassandra/metrics/ClientMetrics.java
+++ b/src/java/org/apache/cassandra/metrics/ClientMetrics.java
@@ -18,38 +18,113 @@
*/
package org.apache.cassandra.metrics;
-import java.util.concurrent.Callable;
+import java.util.*;
import com.codahale.metrics.Gauge;
import com.codahale.metrics.Meter;
+import org.apache.cassandra.transport.ClientStat;
+import org.apache.cassandra.transport.ConnectedClient;
+import org.apache.cassandra.transport.Server;
import static org.apache.cassandra.metrics.CassandraMetricsRegistry.Metrics;
-
-public class ClientMetrics
+public final class ClientMetrics
{
-private static final MetricNameFactory factory = new
DefaultNameFactory("Client");
-
public static final ClientMetrics instance = new ClientMetrics();
-
+
+private static final MetricNameFactory factory = new
DefaultNameFactory("Client");
+
+private volatile boolean initialized = false;
+private Collection servers =
[jira] [Commented] (CASSANDRA-14524) Client metrics refactor
[ https://issues.apache.org/jira/browse/CASSANDRA-14524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519473#comment-16519473 ] Chris Lohfink commented on CASSANDRA-14524: --- +1 lgtm thanks! > Client metrics refactor > --- > > Key: CASSANDRA-14524 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14524 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Aleksey Yeschenko >Priority: Minor > Fix For: 4.0.x > > > Moving refactoring out of CASSANDRA-14458 to be reviewed separately. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-8346) Paxos operation can use stale data during multiple range movements
[
https://issues.apache.org/jira/browse/CASSANDRA-8346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519472#comment-16519472
]
ASF GitHub Bot commented on CASSANDRA-8346:
---
Github user ifesdjeen commented on a diff in the pull request:
https://github.com/apache/cassandra/pull/224#discussion_r197133241
--- Diff: src/java/org/apache/cassandra/service/StorageProxy.java ---
@@ -344,47 +343,43 @@ private static void recordCasContention(int
contentions)
casWriteMetrics.contention.update(contentions);
}
-private static Predicate sameDCPredicateFor(final
String dc)
+private static Predicate sameDCPredicateFor(final String dc)
{
final IEndpointSnitch snitch =
DatabaseDescriptor.getEndpointSnitch();
-return new Predicate()
-{
-public boolean apply(InetAddressAndPort host)
-{
-return dc.equals(snitch.getDatacenter(host));
-}
-};
+return replica -> dc.equals(snitch.getDatacenter(replica));
}
private static PaxosParticipants getPaxosParticipants(TableMetadata
metadata, DecoratedKey key, ConsistencyLevel consistencyForPaxos) throws
UnavailableException
{
Token tk = key.getToken();
-List naturalEndpoints =
StorageService.instance.getNaturalEndpoints(metadata.keyspace, tk);
-Collection pendingEndpoints =
StorageService.instance.getTokenMetadata().pendingEndpointsFor(tk,
metadata.keyspace);
+ReplicaList naturalReplicas =
StorageService.instance.getNaturalReplicas(metadata.keyspace, tk);
+ReplicaList pendingReplicas = new
ReplicaList(StorageService.instance.getTokenMetadata().pendingEndpointsFor(tk,
metadata.keyspace));
if (consistencyForPaxos == ConsistencyLevel.LOCAL_SERIAL)
{
-// Restrict naturalEndpoints and pendingEndpoints to node in
the local DC only
+// Restrict naturalReplicas and pendingReplicas to node in the
local DC only
String localDc =
DatabaseDescriptor.getEndpointSnitch().getDatacenter(FBUtilities.getBroadcastAddressAndPort());
-Predicate isLocalDc =
sameDCPredicateFor(localDc);
-naturalEndpoints =
ImmutableList.copyOf(Iterables.filter(naturalEndpoints, isLocalDc));
-pendingEndpoints =
ImmutableList.copyOf(Iterables.filter(pendingEndpoints, isLocalDc));
+Predicate isLocalDc = sameDCPredicateFor(localDc);
+naturalReplicas =
ReplicaList.immutableCopyOf(naturalReplicas.filter(isLocalDc));
+pendingReplicas =
ReplicaList.immutableCopyOf(pendingReplicas.filter(isLocalDc));
}
-int participants = pendingEndpoints.size() +
naturalEndpoints.size();
+int participants = pendingReplicas.size() + naturalReplicas.size();
int requiredParticipants = participants / 2 + 1; // See
CASSANDRA-8346, CASSANDRA-833
-List liveEndpoints =
ImmutableList.copyOf(Iterables.filter(Iterables.concat(naturalEndpoints,
pendingEndpoints), IAsyncCallback.isAlive));
-if (liveEndpoints.size() < requiredParticipants)
-throw new UnavailableException(consistencyForPaxos,
requiredParticipants, liveEndpoints.size());
+
+Replicas concatenated =
Replicas.concatNaturalAndPending(naturalReplicas, pendingReplicas);
+ReplicaList liveReplicas =
ReplicaList.immutableCopyOf(Replicas.filter(concatenated,
IAsyncCallback.isReplicaAlive));
--- End diff --
Same here (with Immutable).
> Paxos operation can use stale data during multiple range movements
> --
>
> Key: CASSANDRA-8346
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8346
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
>Priority: Major
> Fix For: 2.0.12
>
> Attachments: 8346.txt
>
>
> Paxos operations correctly account for pending ranges for all operation
> pertaining to the Paxos state, but those pending ranges are not taken into
> account when reading the data to check for the conditions or during a serial
> read. It's thus possible to break the LWT guarantees by reading a stale
> value. This require 2 node movements (on the same token range) to be a
> problem though.
> Basically, we have {{RF}} replicas + {{P}} pending nodes. For the Paxos
> prepare/propose phases, the number of required participants (the "Paxos
> QUORUM") is {{(RF + P + 1) / 2}} ({{SP.getPaxosParticipants}}), but the read
> done to check conditions or for serial reads is done at a "normal" QUORUM (or
>
[jira] [Updated] (CASSANDRA-14524) Client metrics refactor
[ https://issues.apache.org/jira/browse/CASSANDRA-14524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14524: -- Reviewer: Chris Lohfink (was: Aleksey Yeschenko) > Client metrics refactor > --- > > Key: CASSANDRA-14524 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14524 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Aleksey Yeschenko >Priority: Minor > Fix For: 4.0.x > > > Moving refactoring out of CASSANDRA-14458 to be reviewed separately. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14524) Client metrics refactor
[ https://issues.apache.org/jira/browse/CASSANDRA-14524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko reassigned CASSANDRA-14524: - Assignee: Aleksey Yeschenko (was: Chris Lohfink) > Client metrics refactor > --- > > Key: CASSANDRA-14524 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14524 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Aleksey Yeschenko >Priority: Minor > Fix For: 4.0.x > > > Moving refactoring out of CASSANDRA-14458 to be reviewed separately. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14524) Client metrics refactor
[ https://issues.apache.org/jira/browse/CASSANDRA-14524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519462#comment-16519462 ] Aleksey Yeschenko commented on CASSANDRA-14524: --- Took initial patch as the base and pushed it a bit further [here|https://github.com/iamaleksey/cassandra/commits/14524-4.0]. CI [here|https://circleci.com/gh/iamaleksey/cassandra/365]. Also used this as an opportunity to clean up some recently added but sub-clean code. While it doesn't go as far as I'd like to, it's something. P.S. If I see another list of maps of strings to strings used internally to marshal objects, and not just at the edge, to format for JMX, I'm going to cut someone with a plastic spoon. > Client metrics refactor > --- > > Key: CASSANDRA-14524 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14524 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Minor > Fix For: 4.0.x > > > Moving refactoring out of CASSANDRA-14458 to be reviewed separately. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14532) Partition level deletions past GCGS are not propagated/merged on read
[
https://issues.apache.org/jira/browse/CASSANDRA-14532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519439#comment-16519439
]
Jeff Jirsa commented on CASSANDRA-14532:
{quote}
We've also found a seemingly related issue in compaction where trying to
compact an SSTable which contains the partition deletion post GCGS, the
partition deletion won't be removed via compaction.
{quote}
That would be a real bug, though - worth opening a JIRA there.
> Partition level deletions past GCGS are not propagated/merged on read
> -
>
> Key: CASSANDRA-14532
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14532
> Project: Cassandra
> Issue Type: Bug
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Major
>
> So as [~jay.zhuang] mentioned on the mailing list
> [here|http://mail-archives.us.apache.org/mod_mbox/cassandra-dev/201806.mbox/],
> it appears that partition deletions that have passed GCGS are not
> propagated/merged properly on read, and also not repaired via read repair.
> Steps to reproduce:
> {code}
> create keyspace test WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 3};
> create table test.test (id int PRIMARY KEY , data text) WITH gc_grace_seconds
> = 10;
> CONSISTENCY ALL;
> INSERT INTO test.test (id, data) values (1, 'test');
> ccm node2 stop
> CONSISTENCY QUORUM;
> DELETE from test.test where id = 1; // wait 10 seconds so HH doesn't
> propagate tombstone when starting node2
> select * from test.test where id = 1 ;
> id | data
> +--
> (0 rows)
> ccm node2 start
> CONSISTENCY ALL;
> select * from test.test where id = 1 ;
> id | data
> +--
> 1 | test
> alter table test.test WITH gc_grace_seconds = 10; // GC
> select * from test.test where id = 1 ;
> id | data
> +--
> (0 rows)
> {code}
> We've also found a seemingly related issue in compaction where trying to
> compact an SSTable which contains the partition deletion post GCGS, the
> partition deletion won't be removed via compaction. Likely the same code is
> causing both bugs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Resolved] (CASSANDRA-14054) testRegularColumnTimestampUpdates - org.apache.cassandra.cql3.ViewTest is flaky: expected <2> but got <1>
[ https://issues.apache.org/jira/browse/CASSANDRA-14054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Brown resolved CASSANDRA-14054. - Resolution: Cannot Reproduce > testRegularColumnTimestampUpdates - org.apache.cassandra.cql3.ViewTest is > flaky: expected <2> but got <1> > - > > Key: CASSANDRA-14054 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14054 > Project: Cassandra > Issue Type: Bug > Components: Testing >Reporter: Michael Kjellman >Assignee: Alex Lourie >Priority: Major > > testRegularColumnTimestampUpdates - org.apache.cassandra.cql3.ViewTest is > flaky: expected <2> but got <1> > Fails about 25% of the time. It is currently our only flaky unit test on > trunk so it would be great to get this one fixed up so we can be confident in > unit test failures going forward. > junit.framework.AssertionFailedError: Invalid value for row 0 column 0 (c of > type int), expected <2> but got <1> > at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:973) > at > org.apache.cassandra.cql3.ViewTest.testRegularColumnTimestampUpdates(ViewTest.java:380) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14056) Many dtests fail with ConfigurationException: offheap_objects are not available in 3.0 when OFFHEAP_MEMTABLES="true"
[
https://issues.apache.org/jira/browse/CASSANDRA-14056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Brown updated CASSANDRA-14056:
Resolution: Fixed
Reviewer: Jason Brown
Status: Resolved (was: Patch Available)
Updated commit lgtm. committed as {{2313e39ee19ea12f304d3e6dd04c41d4f03798ca}}.
Thanks!
[~alourie] Can you close the PR, as well?
> Many dtests fail with ConfigurationException: offheap_objects are not
> available in 3.0 when OFFHEAP_MEMTABLES="true"
>
>
> Key: CASSANDRA-14056
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14056
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
>Reporter: Michael Kjellman
>Assignee: Alex Lourie
>Priority: Major
>
> Tons of dtests are running when they shouldn't as it looks like the path is
> no longer supported.. we need to add a bunch of logic that's missing to fully
> support running dtests with off-heap memtables enabled (via the
> OFFHEAP_MEMTABLES="true" environment variable)
> {code}[node2 ERROR] java.lang.ExceptionInInitializerError
> at
> org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:394)
> at
> org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:361)
> at
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:577)
> at
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:554)
> at org.apache.cassandra.db.Keyspace.initCf(Keyspace.java:368)
> at org.apache.cassandra.db.Keyspace.(Keyspace.java:305)
> at org.apache.cassandra.db.Keyspace.open(Keyspace.java:129)
> at org.apache.cassandra.db.Keyspace.open(Keyspace.java:106)
> at
> org.apache.cassandra.db.SystemKeyspace.checkHealth(SystemKeyspace.java:887)
> at
> org.apache.cassandra.service.StartupChecks$9.execute(StartupChecks.java:354)
> at
> org.apache.cassandra.service.StartupChecks.verify(StartupChecks.java:110)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:179)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:569)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:697)
> Caused by: org.apache.cassandra.exceptions.ConfigurationException:
> offheap_objects are not available in 3.0. They will be re-introduced in a
> future release, see https://issues.apache.org/jira/browse/CASSANDRA-9472 for
> details
> at
> org.apache.cassandra.config.DatabaseDescriptor.getMemtableAllocatorPool(DatabaseDescriptor.java:1907)
> at org.apache.cassandra.db.Memtable.(Memtable.java:65)
> ... 14 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
cassandra-dtest git commit: Check --use-off-heap-memtables option and stop if not supported
Repository: cassandra-dtest
Updated Branches:
refs/heads/master 51c835202 -> 2313e39ee
Check --use-off-heap-memtables option and stop if not supported
patch by Alex Lourie; reviewed by jasobrown for CASSANDRA-14056
Project: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/commit/2313e39e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/tree/2313e39e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/diff/2313e39e
Branch: refs/heads/master
Commit: 2313e39ee19ea12f304d3e6dd04c41d4f03798ca
Parents: 51c8352
Author: Alex Lourie
Authored: Tue Feb 6 14:04:03 2018 +1030
Committer: Jason Brown
Committed: Thu Jun 21 06:37:27 2018 -0700
--
conftest.py | 14 --
run_dtests.py | 11 ++-
2 files changed, 22 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra-dtest/blob/2313e39e/conftest.py
--
diff --git a/conftest.py b/conftest.py
index 3e30942..1078989 100644
--- a/conftest.py
+++ b/conftest.py
@@ -18,7 +18,7 @@ from netifaces import AF_INET
from psutil import virtual_memory
import netifaces as ni
-from ccmlib.common import validate_install_dir, is_win
+from ccmlib.common import validate_install_dir, is_win, get_version_from_build
from dtest_config import DTestConfig
from dtest_setup import DTestSetup
@@ -105,7 +105,7 @@ def fixture_dtest_cluster_name():
Not exactly sure why :\ but, this fixture needs to be scoped to function level
and not
session or class. If you invoke pytest with tests across multiple test
classes, when scopped
at session, the root logger appears to get reset between each test class
invocation.
-this means that the first test to run not from the first test class (and all
subsequent
+this means that the first test to run not from the first test class (and all
subsequent
tests), will have the root logger reset and see a level of NOTSET. Scoping it
at the
class level seems to work, and I guess it's not that much extra overhead to
setup the
logger once per test class vs. once per session in the grand scheme of things.
@@ -415,6 +415,16 @@ def pytest_collection_modifyitems(items, config):
raise Exception("Required dtest arguments were missing! You must
provide either --cassandra-dir "
"or --cassandra-version. Refer to the
documentation or invoke the help with --help.")
+# Either cassandra_version or cassandra_dir is defined, so figure out the
version
+CASSANDRA_VERSION = config.getoption("--cassandra-version") or
get_version_from_build(config.getoption("--cassandra-dir"))
+
+# Check that use_off_heap_memtables is supported in this c* version
+if config.getoption("--use-off-heap-memtables") and ("3.0" <=
CASSANDRA_VERSION < "3.4"):
+raise Exception("The selected Cassandra version %s doesn't support the
provided option "
+"--use-off-heap-memtables, see
https://issues.apache.org/jira/browse/CASSANDRA-9472 "
+"for details" % CASSANDRA_VERSION)
+
+
selected_items = []
deselected_items = []
http://git-wip-us.apache.org/repos/asf/cassandra-dtest/blob/2313e39e/run_dtests.py
--
diff --git a/run_dtests.py b/run_dtests.py
index 198cde2..2d6ca20 100755
--- a/run_dtests.py
+++ b/run_dtests.py
@@ -41,6 +41,7 @@ from _pytest.config import Parser
import argparse
from conftest import pytest_addoption
+from dtest import get_version_from_build
logger = logging.getLogger(__name__)
@@ -94,6 +95,14 @@ class RunDTests():
logging.root.setLevel(logging.DEBUG)
logger.setLevel(logging.DEBUG)
+# Either cassandra_version or cassandra_dir is defined, so figure out
the version
+CASSANDRA_VERSION = args.cassandra_version or
get_version_from_build(args.cassandra_dir)
+
+if args.use_off_heap_memtables and ("3.0" <= CASSANDRA_VERSION <
"3.4"):
+raise Exception("The selected Cassandra version %s doesn't support
the provided option "
+"--use-off-heap-memtables, see
https://issues.apache.org/jira/browse/CASSANDRA-9472 "
+"for details" % CASSANDRA_VERSION)
+
# Get dictionaries corresponding to each point in the configuration
matrix
# we want to run, then generate a config object for each of them.
logger.debug('Generating configurations from the following
matrix:\n\t{}'.format(args))
@@ -211,7 +220,7 @@ def collect_test_modules(stdout):
test_collect_xml_lines.append(" ")
is_first_class = True
-has_closed_class= False
+has_closed_class
[jira] [Resolved] (CASSANDRA-14530) Unable to start repair process
[
https://issues.apache.org/jira/browse/CASSANDRA-14530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ALEXEY KASHINTSEV resolved CASSANDRA-14530.
---
Resolution: Fixed
> Unable to start repair process
> --
>
> Key: CASSANDRA-14530
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14530
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction, Core, Repair
> Environment: Description: Debian GNU/Linux 8.5 (jessie)
> Codename: jessie
> 16GB mem. 6 allocated by java only.
> Linux 04 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt25-2+deb8u3 (2016-07-02)
> x86_64 GNU/Linux
> cassandra 3.11.2
>
> Output of command
> XProf file attached below
> {code:java}
> X04:/var/lib/cassandra# nodetool repair archive statuses -full
> [2018-06-19 17:17:38,170] Starting repair command #1
> (84fd0590-73cb-11e8-9815-6d9cda004dd5), repairing keyspace archive with
> repair options (parallelism: parallel, primary range: false, incremental:
> false, job threads: 1, ColumnFamilies: [statuses], dataCenters: [], hosts:
> [], # of ranges: 967, pull repair: false)
> Exception occurred during clean-up.
> java.lang.reflect.UndeclaredThrowableException
> Cassandra has shutdown.
> error: [2018-06-19 17:17:42,233] JMX connection closed. You should check
> server log for repair status of keyspace archive(Subsequent keyspaces are not
> going to be repaired).
> -- StackTrace --
> java.io.IOException: [2018-06-19 17:17:42,233] JMX connection closed. You
> should check server log for repair status of keyspace archive(Subsequent
> keyspaces are not going to be repaired).
> at
> org.apache.cassandra.tools.RepairRunner.handleConnectionFailed(RepairRunner.java:98)
> at
> org.apache.cassandra.utils.progress.jmx.JMXNotificationProgressListener.handleNotification(JMXNotificationProgressListener.java:86)
> at
> javax.management.NotificationBroadcasterSupport.handleNotification(NotificationBroadcasterSupport.java:275)
> at
> javax.management.NotificationBroadcasterSupport$SendNotifJob.run(NotificationBroadcasterSupport.java:352)
> at
> javax.management.NotificationBroadcasterSupport$1.execute(NotificationBroadcasterSupport.java:337)
> at
> javax.management.NotificationBroadcasterSupport.sendNotification(NotificationBroadcasterSupport.java:248)
> at
> javax.management.remote.rmi.RMIConnector.sendNotification(RMIConnector.java:441)
> at javax.management.remote.rmi.RMIConnector.access$1200(RMIConnector.java:121)
> at
> javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.gotIOException(RMIConnector.java:1531)
> at
> javax.management.remote.rmi.RMIConnector$RMINotifClient.fetchNotifs(RMIConnector.java:1352)
> at
> com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchOneNotif(ClientNotifForwarder.java:655)
> at
> com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchNotifs(ClientNotifForwarder.java:607)
> at
> com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.doRun(ClientNotifForwarder.java:471)
> at
> com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.run(ClientNotifForwarder.java:452)
> at
> com.sun.jmx.remote.internal.ClientNotifForwarder$LinearExecutor$1.run(ClientNotifForwarder.java:108)
> {code}
>Reporter: ALEXEY KASHINTSEV
>Priority: Blocker
> Attachments: hs_err_1529417753.log
>
>
>
> Then repair starts, cassandra goes down.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14530) Unable to start repair process
[
https://issues.apache.org/jira/browse/CASSANDRA-14530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519266#comment-16519266
]
ALEXEY KASHINTSEV commented on CASSANDRA-14530:
---
Seems to be fixed by setting vm.max_map_count = 1048575
> Unable to start repair process
> --
>
> Key: CASSANDRA-14530
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14530
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction, Core, Repair
> Environment: Description: Debian GNU/Linux 8.5 (jessie)
> Codename: jessie
> 16GB mem. 6 allocated by java only.
> Linux 04 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt25-2+deb8u3 (2016-07-02)
> x86_64 GNU/Linux
> cassandra 3.11.2
>
> Output of command
> XProf file attached below
> {code:java}
> X04:/var/lib/cassandra# nodetool repair archive statuses -full
> [2018-06-19 17:17:38,170] Starting repair command #1
> (84fd0590-73cb-11e8-9815-6d9cda004dd5), repairing keyspace archive with
> repair options (parallelism: parallel, primary range: false, incremental:
> false, job threads: 1, ColumnFamilies: [statuses], dataCenters: [], hosts:
> [], # of ranges: 967, pull repair: false)
> Exception occurred during clean-up.
> java.lang.reflect.UndeclaredThrowableException
> Cassandra has shutdown.
> error: [2018-06-19 17:17:42,233] JMX connection closed. You should check
> server log for repair status of keyspace archive(Subsequent keyspaces are not
> going to be repaired).
> -- StackTrace --
> java.io.IOException: [2018-06-19 17:17:42,233] JMX connection closed. You
> should check server log for repair status of keyspace archive(Subsequent
> keyspaces are not going to be repaired).
> at
> org.apache.cassandra.tools.RepairRunner.handleConnectionFailed(RepairRunner.java:98)
> at
> org.apache.cassandra.utils.progress.jmx.JMXNotificationProgressListener.handleNotification(JMXNotificationProgressListener.java:86)
> at
> javax.management.NotificationBroadcasterSupport.handleNotification(NotificationBroadcasterSupport.java:275)
> at
> javax.management.NotificationBroadcasterSupport$SendNotifJob.run(NotificationBroadcasterSupport.java:352)
> at
> javax.management.NotificationBroadcasterSupport$1.execute(NotificationBroadcasterSupport.java:337)
> at
> javax.management.NotificationBroadcasterSupport.sendNotification(NotificationBroadcasterSupport.java:248)
> at
> javax.management.remote.rmi.RMIConnector.sendNotification(RMIConnector.java:441)
> at javax.management.remote.rmi.RMIConnector.access$1200(RMIConnector.java:121)
> at
> javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.gotIOException(RMIConnector.java:1531)
> at
> javax.management.remote.rmi.RMIConnector$RMINotifClient.fetchNotifs(RMIConnector.java:1352)
> at
> com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchOneNotif(ClientNotifForwarder.java:655)
> at
> com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchNotifs(ClientNotifForwarder.java:607)
> at
> com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.doRun(ClientNotifForwarder.java:471)
> at
> com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.run(ClientNotifForwarder.java:452)
> at
> com.sun.jmx.remote.internal.ClientNotifForwarder$LinearExecutor$1.run(ClientNotifForwarder.java:108)
> {code}
>Reporter: ALEXEY KASHINTSEV
>Priority: Blocker
> Attachments: hs_err_1529417753.log
>
>
>
> Then repair starts, cassandra goes down.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14532) Partition level deletions past GCGS are not propagated/merged on read
[
https://issues.apache.org/jira/browse/CASSANDRA-14532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519023#comment-16519023
]
Sylvain Lebresne commented on CASSANDRA-14532:
--
That's not really a bug in that this is working as designed. The very reason
for gc grace is to be a long enough time that we can guarantee any data
(including tombstone) has been propagated to all replica, and that's why you
must run repair within the gc grace window (otherwise other mechanism don't
truly guarantee that). So we should not have to propagate anything past gcgs
and doing so is at best an inefficiency.
Or another way to thing about it is, post-gcgs, a tombstone can be purged at
any time, including immediately, solely based on local compaction conditions.
So if not propagating post-gcgs tombstone was _a bug_, we'd be basically saying
the whole concept of ever purging tombstones is bugged.
And sending tombstone past gcgs is actually somewhat bad, exactly due to the
fact that such tombstone will be purged by different nodes at (possibly very)
different times. As every time a node has purged a tombstones while other
haven't and we reads, we'd digest mismatch, increasing the operation latency
and incurring more work on the system. And that for no reason whatsoever to any
user that actually properly configure gcgs and properly run repairs. Even for
those that don't, whether sending post-gcgs tombstone save their asses or not
will be at best totally random (and so largely useless imo).
Overall, I'd be kind of -1 (at least unless disprove my reasoning above) on a
patch that simply start sending post-gcgs tombstones on reads as it create
inefficiencies without buying any additional concrete guarantee.
> Partition level deletions past GCGS are not propagated/merged on read
> -
>
> Key: CASSANDRA-14532
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14532
> Project: Cassandra
> Issue Type: Bug
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Major
>
> So as [~jay.zhuang] mentioned on the mailing list
> [here|http://mail-archives.us.apache.org/mod_mbox/cassandra-dev/201806.mbox/],
> it appears that partition deletions that have passed GCGS are not
> propagated/merged properly on read, and also not repaired via read repair.
> Steps to reproduce:
> {code}
> create keyspace test WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 3};
> create table test.test (id int PRIMARY KEY , data text) WITH gc_grace_seconds
> = 10;
> CONSISTENCY ALL;
> INSERT INTO test.test (id, data) values (1, 'test');
> ccm node2 stop
> CONSISTENCY QUORUM;
> DELETE from test.test where id = 1; // wait 10 seconds so HH doesn't
> propagate tombstone when starting node2
> select * from test.test where id = 1 ;
> id | data
> +--
> (0 rows)
> ccm node2 start
> CONSISTENCY ALL;
> select * from test.test where id = 1 ;
> id | data
> +--
> 1 | test
> alter table test.test WITH gc_grace_seconds = 10; // GC
> select * from test.test where id = 1 ;
> id | data
> +--
> (0 rows)
> {code}
> We've also found a seemingly related issue in compaction where trying to
> compact an SSTable which contains the partition deletion post GCGS, the
> partition deletion won't be removed via compaction. Likely the same code is
> causing both bugs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14423) SSTables stop being compacted
[
https://issues.apache.org/jira/browse/CASSANDRA-14423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519010#comment-16519010
]
Stefan Podkowinski commented on CASSANDRA-14423:
I think this is going in the right direction.
As for log messages, the "does not intersect repaired ranges" message seems to
be now incorrectly shown for already repaired sstables.
{quote}Also, the assert at the end of performAntiCompaction had to be removed
as we're removing nonAnticompacting from the transaction which violates the
assert, this is fine however as the assert was effectively incorrect in the
first place, and only valid due to the bug.
{quote}
Help me out, in which case do we end up removing an sstable in
nonAnticompacting from txn.originals that would not also be removed from
sstableIterator?
> SSTables stop being compacted
> -
>
> Key: CASSANDRA-14423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14423
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Major
> Fix For: 2.2.13, 3.0.17, 3.11.3
>
>
> So seeing a problem in 3.11.0 where SSTables are being lost from the view and
> not being included in compactions/as candidates for compaction. It seems to
> get progressively worse until there's only 1-2 SSTables in the view which
> happen to be the most recent SSTables and thus compactions completely stop
> for that table.
> The SSTables seem to still be included in reads, just not compactions.
> The issue can be fixed by restarting C*, as it will reload all SSTables into
> the view, but this is only a temporary fix. User defined/major compactions
> still work - not clear if they include the result back in the view but is not
> a good work around.
> This also results in a discrepancy between SSTable count and SSTables in
> levels for any table using LCS.
> {code:java}
> Keyspace : xxx
> Read Count: 57761088
> Read Latency: 0.10527088681224288 ms.
> Write Count: 2513164
> Write Latency: 0.018211106398149903 ms.
> Pending Flushes: 0
> Table: xxx
> SSTable count: 10
> SSTables in each level: [2, 0, 0, 0, 0, 0, 0, 0, 0]
> Space used (live): 894498746
> Space used (total): 894498746
> Space used by snapshots (total): 0
> Off heap memory used (total): 11576197
> SSTable Compression Ratio: 0.6956629530569777
> Number of keys (estimate): 3562207
> Memtable cell count: 0
> Memtable data size: 0
> Memtable off heap memory used: 0
> Memtable switch count: 87
> Local read count: 57761088
> Local read latency: 0.108 ms
> Local write count: 2513164
> Local write latency: NaN ms
> Pending flushes: 0
> Percent repaired: 86.33
> Bloom filter false positives: 43
> Bloom filter false ratio: 0.0
> Bloom filter space used: 8046104
> Bloom filter off heap memory used: 8046024
> Index summary off heap memory used: 3449005
> Compression metadata off heap memory used: 81168
> Compacted partition minimum bytes: 104
> Compacted partition maximum bytes: 5722
> Compacted partition mean bytes: 175
> Average live cells per slice (last five minutes): 1.0
> Maximum live cells per slice (last five minutes): 1
> Average tombstones per slice (last five minutes): 1.0
> Maximum tombstones per slice (last five minutes): 1
> Dropped Mutations: 0
> {code}
> Also for STCS we've confirmed that SSTable count will be different to the
> number of SSTables reported in the Compaction Bucket's. In the below example
> there's only 3 SSTables in a single bucket - no more are listed for this
> table. Compaction thresholds haven't been modified for this table and it's a
> very basic KV schema.
> {code:java}
> Keyspace : yyy
> Read Count: 30485
> Read Latency: 0.06708991307200263 ms.
> Write Count: 57044
> Write Latency: 0.02204061776873992 ms.
> Pending Flushes: 0
> Table: yyy
> SSTable count: 19
> Space used (live): 18195482
> Space used (total): 18195482
> Space used by snapshots (total): 0
> Off heap memory used (total): 747376
> SSTable Compression Ratio: 0.7607394576769735
> Number of keys (estimate): 116074
> Memtable cell count: 0
> Memtable data size: 0
> Memtable off heap memory used: 0
> Memtable switch count: 39
> Local read count: 30485
> Local read latency: NaN ms
> Local write count: 57044
> Local write latency: NaN ms
> Pending flushes: 0
> Percent repaired: 79.76
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.0
> Bloom filter space used: 690912
> Bloom filter off heap memory used: 690760
> Index summary off heap memory used: 54736
> Compression metadata off heap memory used: 1880
> Compacted
[jira] [Commented] (CASSANDRA-14054) testRegularColumnTimestampUpdates - org.apache.cassandra.cql3.ViewTest is flaky: expected <2> but got <1>
[ https://issues.apache.org/jira/browse/CASSANDRA-14054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519001#comment-16519001 ] Alex Lourie commented on CASSANDRA-14054: - [~jasobrown] Just rerun this on trunk, 3.0 and 3.11, doesn't reproduce. Happy to close it off. > testRegularColumnTimestampUpdates - org.apache.cassandra.cql3.ViewTest is > flaky: expected <2> but got <1> > - > > Key: CASSANDRA-14054 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14054 > Project: Cassandra > Issue Type: Bug > Components: Testing >Reporter: Michael Kjellman >Assignee: Alex Lourie >Priority: Major > > testRegularColumnTimestampUpdates - org.apache.cassandra.cql3.ViewTest is > flaky: expected <2> but got <1> > Fails about 25% of the time. It is currently our only flaky unit test on > trunk so it would be great to get this one fixed up so we can be confident in > unit test failures going forward. > junit.framework.AssertionFailedError: Invalid value for row 0 column 0 (c of > type int), expected <2> but got <1> > at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:973) > at > org.apache.cassandra.cql3.ViewTest.testRegularColumnTimestampUpdates(ViewTest.java:380) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14496) TWCS erroneously disabling tombstone compactions when unchecked_tombstone_compaction=true
[
https://issues.apache.org/jira/browse/CASSANDRA-14496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Ivakov updated CASSANDRA-14496:
-
Status: Patch Available (was: In Progress)
[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...aivakov:CASSANDRA-14496]
Attaching a patch for the above. Simply checking if
unchecked_tombstone_compactions is set to true, regardless of
tombstone_compaction_interval and tombstone_threshold being set. Also added a
unit test.
> TWCS erroneously disabling tombstone compactions when
> unchecked_tombstone_compaction=true
> -
>
> Key: CASSANDRA-14496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14496
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Robert Tarrall
>Assignee: Alexander Ivakov
>Priority: Minor
>
> This code:
> {code:java}
> this.options = new TimeWindowCompactionStrategyOptions(options);
> if
> (!options.containsKey(AbstractCompactionStrategy.TOMBSTONE_COMPACTION_INTERVAL_OPTION)
> &&
> !options.containsKey(AbstractCompactionStrategy.TOMBSTONE_THRESHOLD_OPTION))
> {
> disableTombstoneCompactions = true;
> logger.debug("Disabling tombstone compactions for TWCS");
> }
> else
> logger.debug("Enabling tombstone compactions for TWCS");
> }
> {code}
> ... in TimeWindowCompactionStrategy.java disables tombstone compactions in
> TWCS if you have not *explicitly* set either tombstone_compaction_interval or
> tombstone_threshold. Adding 'tombstone_compaction_interval': '86400' to the
> compaction stanza in a table definition has the (to me unexpected) side
> effect of enabling tombstone compactions.
> This is surprising and does not appear to be mentioned in the docs.
> I would suggest that tombstone compactions should be run unless these options
> are both set to 0.
> If the concern is that (as with DTCS in CASSANDRA-9234) we don't want to
> waste time on tombstone compactions when we expect the tables to eventually
> be expired away, perhaps we should also check unchecked_tombstone_compaction
> and still enable tombstone compactions if that's set to true.
> May also make sense to set defaults for interval & threshold to 0 & disable
> if they're nonzero so that setting non-default values, rather than setting
> ANY value, is what determines whether tombstone compactions are enabled?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
