[jira] [Comment Edited] (CASSANDRA-10540) RangeAwareCompaction

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519900#comment-16519900
 ] 

Lerh Chuan Low edited comment on CASSANDRA-10540 at 6/22/18 3:16 AM:
-

Here is another benchmark run. It is still the same stressspec YAML. This time, 
the process is to stop one of the nodes in a DC (Same as before, 3 in 1 DC and 
2 in the other), and then insert for 10 minutes. 
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=10m 
cl=QUORUM ops\(insert=1\) -node file=nodelist.txt -rate threads=50 -log 
file=insert.log > nohup.txt &
{code}
After that, trigger a stress but at the same time run a full repair in the DC:
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=1h 
cl=QUORUM ops\(insert=10,simple1=10,range1=1\) -node file=nodelist.txt -rate 
threads=50 -log file=mixed.log > nohup.txt &


nohup nodetool repair --full stresscql2 typestest > nohup.txt &
{code}
Here are the results:
|| ||RACS||Non RACS||
|Stress Result 1|Op rate : 244 op/s [insert: 116 op/s, range1: 12 op/s, 
simple1: 116 op/s]
 Partition rate : 243 pk/s [insert: 116 pk/s, range1: 10 pk/s, simple1: 116 
pk/s]
 Row rate : 274 row/s [insert: 116 row/s, range1: 41 row/s, simple1: 116 row/s]
 Latency mean : 204.6 ms [insert: 2.5 ms, range1: 387.4 ms, simple1: 388.8 ms]
 Latency median : 39.7 ms [insert: 2.0 ms, range1: 378.0 ms, simple1: 377.7 ms]
 Latency 95th percentile : 706.2 ms [insert: 3.2 ms, range1: 802.2 ms, simple1: 
805.3 ms]
 Latency 99th percentile : 941.6 ms [insert: 19.7 ms, range1: 1,022.9 ms, 
simple1: 1,022.4 ms]
 Latency 99.9th percentile : 1183.8 ms [insert: 65.5 ms, range1: 1,232.1 ms, 
simple1: 1,218.4 ms]
 Latency max : 7314.9 ms [insert: 550.0 ms, range1: 1,472.2 ms, simple1: 
7,314.9 ms]
 Total partitions : 874,058 [insert: 419,116, range1: 36,428, simple1: 418,514]
 Total errors : 0 [insert: 0, range1: 0, simple1: 0]
 Total GC count : 0
 Total GC memory : 0.000 KiB
 Total GC time : 0.0 seconds
 Avg GC time : NaN ms
 StdDev GC time : 0.0 ms
 Total operation time : 01:00:00|Op rate : 221 op/s [insert: 105 op/s, range1: 
11 op/s, simple1: 105 op/s]
 Partition rate : 220 pk/s [insert: 105 pk/s, range1: 9 pk/s, simple1: 105 pk/s]
 Row rate : 248 row/s [insert: 105 row/s, range1: 38 row/s, simple1: 105 row/s]
 Latency mean : 226.2 ms [insert: 2.7 ms, range1: 428.8 ms, simple1: 429.1 ms]
 Latency median : 150.3 ms [insert: 2.0 ms, range1: 385.4 ms, simple1: 383.8 ms]
 Latency 95th percentile : 716.2 ms [insert: 3.0 ms, range1: 837.3 ms, simple1: 
841.5 ms]
 Latency 99th percentile : 1047.5 ms [insert: 14.8 ms, range1: 1,210.1 ms, 
simple1: 1,230.0 ms]
 Latency 99.9th percentile : 1830.8 ms [insert: 57.5 ms, range1: 2,029.0 ms, 
simple1: 2,063.6 ms]
 Latency max : 7457.5 ms [insert: 6,358.6 ms, range1: 7,159.7 ms, simple1: 
7,457.5 ms]
 Total partitions : 790,543 [insert: 378,618, range1: 33,908, simple1: 378,017]
 Total errors : 0 [insert: 0, range1: 0, simple1: 0]
 Total GC count : 0
 Total GC memory : 0.000 KiB
 Total GC time : 0.0 seconds
 Avg GC time : NaN ms
 StdDev GC time : 0.0 ms
 Total operation time : 01:00:00|
|Stress result 2|Op rate : 247 op/s [insert: 118 op/s, range1: 12 op/s, 
simple1: 118 op/s]
Partition rate : 246 pk/s [insert: 118 pk/s, range1: 10 pk/s, simple1: 118 pk/s]
Row rate : 278 row/s [insert: 118 row/s, range1: 42 row/s, simple1: 118 row/s]
Latency mean : 201.9 ms [insert: 3.8 ms, range1: 384.5 ms, simple1: 382.3 ms]
Latency median : 45.0 ms [insert: 2.0 ms, range1: 374.1 ms, simple1: 372.2 ms]
Latency 95th percentile : 666.4 ms [insert: 10.0 ms, range1: 761.3 ms, simple1: 
759.7 ms]
Latency 99th percentile : 888.7 ms [insert: 44.0 ms, range1: 973.1 ms, simple1: 
968.9 ms]
Latency 99.9th percentile : 1135.6 ms [insert: 68.8 ms, range1: 1,182.8 ms, 
simple1: 1,181.7 ms]
Latency max : 7101.0 ms [insert: 328.5 ms, range1: 6,970.9 ms, simple1: 7,101.0 
ms]
Total partitions : 886,043 [insert: 425,188, range1: 36,972, simple1: 423,883]
Total errors : 0 [insert: 0, range1: 0, simple1: 0]
Total GC count : 0
Total GC memory : 0.000 KiB
Total GC time : 0.0 seconds
Avg GC time : NaN ms
StdDev GC time : 0.0 ms
Total operation time : 01:00:01|Op rate : 211 op/s [insert: 100 op/s, range1: 
10 op/s, simple1: 101 op/s]
Partition rate : 210 pk/s [insert: 100 pk/s, range1: 9 pk/s, simple1: 101 pk/s]
Row rate : 236 row/s [insert: 100 row/s, range1: 35 row/s, simple1: 101 row/s]
Latency mean : 237.0 ms [insert: 2.5 ms, range1: 451.1 ms, simple1: 449.9 ms]
Latency median : 153.9 ms [insert: 2.0 ms, range1: 416.3 ms, simple1: 414.4 ms]
Latency 95th percentile : 753.4 ms [insert: 3.0 ms, range1: 867.7 ms, simple1: 
864.0 ms]
Latency 99th percentile : 1016.6 ms [insert: 18.0 ms, range1: 1,118.8 ms, 
simple1: 1,109.4 ms]
Latency 99.9th percentile : 1274.0 ms [insert: 58.6 ms, range1: 1,355.8 ms, 
simple1: 1,344.3 ms]

[jira] [Comment Edited] (CASSANDRA-10540) RangeAwareCompaction

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519900#comment-16519900
 ] 

Lerh Chuan Low edited comment on CASSANDRA-10540 at 6/22/18 1:22 AM:
-

Here is another benchmark run. It is still the same stressspec YAML. This time, 
the process is to stop one of the nodes in a DC (Same as before, 3 in 1 DC and 
2 in the other), and then insert for 10 minutes. 
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=10m 
cl=QUORUM ops\(insert=1\) -node file=nodelist.txt -rate threads=50 -log 
file=insert.log > nohup.txt &
{code}
After that, trigger a stress but at the same time run a full repair in the DC:
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=1h 
cl=QUORUM ops\(insert=10,simple1=10,range1=1\) -node file=nodelist.txt -rate 
threads=50 -log file=mixed.log > nohup.txt &


nohup nodetool repair --full stresscql2 typestest > nohup.txt &
{code}
Here are the results:
|| ||RACS||Non RACS||
|Stress Result|Op rate : 244 op/s [insert: 116 op/s, range1: 12 op/s, simple1: 
116 op/s]
Partition rate : 243 pk/s [insert: 116 pk/s, range1: 10 pk/s, simple1: 116 pk/s]
Row rate : 274 row/s [insert: 116 row/s, range1: 41 row/s, simple1: 116 row/s]
Latency mean : 204.6 ms [insert: 2.5 ms, range1: 387.4 ms, simple1: 388.8 ms]
Latency median : 39.7 ms [insert: 2.0 ms, range1: 378.0 ms, simple1: 377.7 ms]
Latency 95th percentile : 706.2 ms [insert: 3.2 ms, range1: 802.2 ms, simple1: 
805.3 ms]
Latency 99th percentile : 941.6 ms [insert: 19.7 ms, range1: 1,022.9 ms, 
simple1: 1,022.4 ms]
Latency 99.9th percentile : 1183.8 ms [insert: 65.5 ms, range1: 1,232.1 ms, 
simple1: 1,218.4 ms]
Latency max : 7314.9 ms [insert: 550.0 ms, range1: 1,472.2 ms, simple1: 7,314.9 
ms]
Total partitions : 874,058 [insert: 419,116, range1: 36,428, simple1: 418,514]
Total errors : 0 [insert: 0, range1: 0, simple1: 0]
Total GC count : 0
Total GC memory : 0.000 KiB
Total GC time : 0.0 seconds
Avg GC time : NaN ms
StdDev GC time : 0.0 ms
Total operation time : 01:00:00|Op rate : 221 op/s [insert: 105 op/s, range1: 
11 op/s, simple1: 105 op/s]
 Partition rate : 220 pk/s [insert: 105 pk/s, range1: 9 pk/s, simple1: 105 pk/s]
 Row rate : 248 row/s [insert: 105 row/s, range1: 38 row/s, simple1: 105 row/s]
 Latency mean : 226.2 ms [insert: 2.7 ms, range1: 428.8 ms, simple1: 429.1 ms]
 Latency median : 150.3 ms [insert: 2.0 ms, range1: 385.4 ms, simple1: 383.8 ms]
 Latency 95th percentile : 716.2 ms [insert: 3.0 ms, range1: 837.3 ms, simple1: 
841.5 ms]
 Latency 99th percentile : 1047.5 ms [insert: 14.8 ms, range1: 1,210.1 ms, 
simple1: 1,230.0 ms]
 Latency 99.9th percentile : 1830.8 ms [insert: 57.5 ms, range1: 2,029.0 ms, 
simple1: 2,063.6 ms]
 Latency max : 7457.5 ms [insert: 6,358.6 ms, range1: 7,159.7 ms, simple1: 
7,457.5 ms]
 Total partitions : 790,543 [insert: 378,618, range1: 33,908, simple1: 378,017]
 Total errors : 0 [insert: 0, range1: 0, simple1: 0]
 Total GC count : 0
 Total GC memory : 0.000 KiB
 Total GC time : 0.0 seconds
 Avg GC time : NaN ms
 StdDev GC time : 0.0 ms
 Total operation time : 01:00:00|

Big thanks to Jason Brown for the repair patch, works like a charm :)

 


was (Author: lerh low):
Here is another benchmark run. It is still the same stressspec YAML. This time, 
the process is to stop one of the nodes in a DC (Same as before, 3 in 1 DC and 
2 in the other), and then insert for 10 minutes. 
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=10m 
cl=QUORUM ops\(insert=1\) -node file=nodelist.txt -rate threads=50 -log 
file=insert.log > nohup.txt &
{code}
After that, trigger a stress but at the same time run a full repair in the DC:
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=1h 
cl=QUORUM ops\(insert=10,simple1=10,range1=1\) -node file=nodelist.txt -rate 
threads=50 -log file=mixed.log > nohup.txt &


nohup nodetool repair --full stresscql2 typestest > nohup.txt &
{code}
Here are the results:


|| ||RACS||Non RACS||
|Stress Result|Op rate : 244 op/s [insert: 116 op/s, range1: 12 op/s, simple1: 
116 op/s] 
Partition rate : 243 pk/s [insert: 116 pk/s, range1: 10 pk/s, simple1: 116 
pk/s] 
Row rate : 274 row/s [insert: 116 row/s, range1: 41 row/s, simple1: 116 row/s] 
Latency mean : 204.6 ms [insert: 2.5 ms, range1: 387.4 ms, simple1: 388.8 ms] 
Latency median : 39.7 ms [insert: 2.0 ms, range1: 378.0 ms, simple1: 377.7 ms] 
Latency 95th percentile : 706.2 ms [insert: 3.2 ms, range1: 802.2 ms, simple1: 
805.3 ms] Latency 99th percentile : 941.6 ms [insert: 19.7 ms, range1: 1,022.9 
ms, simple1: 1,022.4 ms] Latency 99.9th percentile : 1183.8 ms [insert: 65.5 
ms, range1: 1,232.1 ms, simple1: 1,218.4 ms] Latency max : 7314.9 ms [insert: 
550.0 ms, range1: 1,472.2 ms, simple1: 7,314.9 ms] 
Total partitions : 874,058 [insert: 419,116, range1: 

[jira] [Comment Edited] (CASSANDRA-10540) RangeAwareCompaction

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519900#comment-16519900
 ] 

Lerh Chuan Low edited comment on CASSANDRA-10540 at 6/22/18 1:21 AM:
-

Here is another benchmark run. It is still the same stressspec YAML. This time, 
the process is to stop one of the nodes in a DC (Same as before, 3 in 1 DC and 
2 in the other), and then insert for 10 minutes. 
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=10m 
cl=QUORUM ops\(insert=1\) -node file=nodelist.txt -rate threads=50 -log 
file=insert.log > nohup.txt &
{code}
After that, trigger a stress but at the same time run a full repair in the DC:
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=1h 
cl=QUORUM ops\(insert=10,simple1=10,range1=1\) -node file=nodelist.txt -rate 
threads=50 -log file=mixed.log > nohup.txt &


nohup nodetool repair --full stresscql2 typestest > nohup.txt &
{code}
Here are the results:


|| ||RACS||Non RACS||
|Stress Result|Op rate : 244 op/s [insert: 116 op/s, range1: 12 op/s, simple1: 
116 op/s] 
Partition rate : 243 pk/s [insert: 116 pk/s, range1: 10 pk/s, simple1: 116 
pk/s] 
Row rate : 274 row/s [insert: 116 row/s, range1: 41 row/s, simple1: 116 row/s] 
Latency mean : 204.6 ms [insert: 2.5 ms, range1: 387.4 ms, simple1: 388.8 ms] 
Latency median : 39.7 ms [insert: 2.0 ms, range1: 378.0 ms, simple1: 377.7 ms] 
Latency 95th percentile : 706.2 ms [insert: 3.2 ms, range1: 802.2 ms, simple1: 
805.3 ms] Latency 99th percentile : 941.6 ms [insert: 19.7 ms, range1: 1,022.9 
ms, simple1: 1,022.4 ms] Latency 99.9th percentile : 1183.8 ms [insert: 65.5 
ms, range1: 1,232.1 ms, simple1: 1,218.4 ms] Latency max : 7314.9 ms [insert: 
550.0 ms, range1: 1,472.2 ms, simple1: 7,314.9 ms] 
Total partitions : 874,058 [insert: 419,116, range1: 36,428, simple1: 418,514] 
Total errors : 0 [insert: 0, range1: 0, simple1: 0] 
Total GC count : 0 
Total GC memory : 0.000 KiB 
Total GC time : 0.0 seconds 
Avg GC time : NaN ms 
StdDev GC time : 0.0 ms 
Total operation time : 01:00:00|Op rate : 221 op/s [insert: 105 op/s, range1: 
11 op/s, simple1: 105 op/s]
Partition rate : 220 pk/s [insert: 105 pk/s, range1: 9 pk/s, simple1: 105 pk/s]
Row rate : 248 row/s [insert: 105 row/s, range1: 38 row/s, simple1: 105 row/s]
Latency mean : 226.2 ms [insert: 2.7 ms, range1: 428.8 ms, simple1: 429.1 ms]
Latency median : 150.3 ms [insert: 2.0 ms, range1: 385.4 ms, simple1: 383.8 ms]
Latency 95th percentile : 716.2 ms [insert: 3.0 ms, range1: 837.3 ms, simple1: 
841.5 ms]
Latency 99th percentile : 1047.5 ms [insert: 14.8 ms, range1: 1,210.1 ms, 
simple1: 1,230.0 ms]
Latency 99.9th percentile : 1830.8 ms [insert: 57.5 ms, range1: 2,029.0 ms, 
simple1: 2,063.6 ms]
Latency max : 7457.5 ms [insert: 6,358.6 ms, range1: 7,159.7 ms, simple1: 
7,457.5 ms]
Total partitions : 790,543 [insert: 378,618, range1: 33,908, simple1: 378,017]
Total errors : 0 [insert: 0, range1: 0, simple1: 0]
Total GC count : 0
Total GC memory : 0.000 KiB
Total GC time : 0.0 seconds
Avg GC time : NaN ms
StdDev GC time : 0.0 ms
Total operation time : 01:00:00|

Big thanks to Jason Brown for the repair patch, works like a charm :)

 


was (Author: lerh low):
Here is another benchmark run. It is still the same stressspec YAML. This time, 
the process is to stop one of the nodes in a DC (Same as before, 3 in 1 DC and 
2 in the other), and then insert for 10 minutes. 
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=10m 
cl=QUORUM ops\(insert=1\) -node file=nodelist.txt -rate threads=50 -log 
file=insert.log > nohup.txt &
{code}
After that, trigger a stress but at the same time run a full repair in the DC:
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=1h 
cl=QUORUM ops\(insert=10,simple1=10,range1=1\) -node file=nodelist.txt -rate 
threads=50 -log file=mixed.log > nohup.txt &


nohup nodetool repair --full stresscql2 typestest > nohup.txt &
{code}
Here are the results:


|| ||RACS||Non RACS||
|Stress Result|Op rate : 244 op/s [insert: 116 op/s, range1: 12 op/s, simple1: 
116 op/s] 
Partition rate : 243 pk/s [insert: 116 pk/s, range1: 10 pk/s, simple1: 116 
pk/s] 
Row rate : 274 row/s [insert: 116 row/s, range1: 41 row/s, simple1: 116 row/s] 
Latency mean : 204.6 ms [insert: 2.5 ms, range1: 387.4 ms, simple1: 388.8 ms] 
Latency median : 39.7 ms [insert: 2.0 ms, range1: 378.0 ms, simple1: 377.7 ms] 
Latency 95th percentile : 706.2 ms [insert: 3.2 ms, range1: 802.2 ms, simple1: 
805.3 ms] Latency 99th percentile : 941.6 ms [insert: 19.7 ms, range1: 1,022.9 
ms, simple1: 1,022.4 ms] Latency 99.9th percentile : 1183.8 ms [insert: 65.5 
ms, range1: 1,232.1 ms, simple1: 1,218.4 ms] Latency max : 7314.9 ms [insert: 
550.0 ms, range1: 1,472.2 ms, simple1: 7,314.9 ms] 
Total partitions : 874,058 [insert: 419,116, range1: 

[jira] [Commented] (CASSANDRA-10540) RangeAwareCompaction

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519900#comment-16519900
 ] 

Lerh Chuan Low commented on CASSANDRA-10540:


Here is another benchmark run. It is still the same stressspec YAML. This time, 
the process is to stop one of the nodes in a DC (Same as before, 3 in 1 DC and 
2 in the other), and then insert for 10 minutes. 
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=10m 
cl=QUORUM ops\(insert=1\) -node file=nodelist.txt -rate threads=50 -log 
file=insert.log > nohup.txt &
{code}
After that, trigger a stress but at the same time run a full repair in the DC:
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=1h 
cl=QUORUM ops\(insert=10,simple1=10,range1=1\) -node file=nodelist.txt -rate 
threads=50 -log file=mixed.log > nohup.txt &


nohup nodetool repair --full stresscql2 typestest > nohup.txt &
{code}
Here are the results:


|| ||RACS||Non RACS||
|Stress Result|Op rate : 244 op/s [insert: 116 op/s, range1: 12 op/s, simple1: 
116 op/s] 
Partition rate : 243 pk/s [insert: 116 pk/s, range1: 10 pk/s, simple1: 116 
pk/s] 
Row rate : 274 row/s [insert: 116 row/s, range1: 41 row/s, simple1: 116 row/s] 
Latency mean : 204.6 ms [insert: 2.5 ms, range1: 387.4 ms, simple1: 388.8 ms] 
Latency median : 39.7 ms [insert: 2.0 ms, range1: 378.0 ms, simple1: 377.7 ms] 
Latency 95th percentile : 706.2 ms [insert: 3.2 ms, range1: 802.2 ms, simple1: 
805.3 ms] Latency 99th percentile : 941.6 ms [insert: 19.7 ms, range1: 1,022.9 
ms, simple1: 1,022.4 ms] Latency 99.9th percentile : 1183.8 ms [insert: 65.5 
ms, range1: 1,232.1 ms, simple1: 1,218.4 ms] Latency max : 7314.9 ms [insert: 
550.0 ms, range1: 1,472.2 ms, simple1: 7,314.9 ms] 
Total partitions : 874,058 [insert: 419,116, range1: 36,428, simple1: 418,514] 
Total errors : 0 [insert: 0, range1: 0, simple1: 0] 
Total GC count : 0 
Total GC memory : 0.000 KiB 
Total GC time : 0.0 seconds 
Avg GC time : NaN ms 
StdDev GC time : 0.0 ms 
Total operation time : 01:00:00|Op rate : 221 op/s [insert: 105 op/s, range1: 
11 op/s, simple1: 105 op/s]
Partition rate : 220 pk/s [insert: 105 pk/s, range1: 9 pk/s, simple1: 105 pk/s]
Row rate : 248 row/s [insert: 105 row/s, range1: 38 row/s, simple1: 105 row/s]
Latency mean : 226.2 ms [insert: 2.7 ms, range1: 428.8 ms, simple1: 429.1 ms]
Latency median : 150.3 ms [insert: 2.0 ms, range1: 385.4 ms, simple1: 383.8 ms]
Latency 95th percentile : 716.2 ms [insert: 3.0 ms, range1: 837.3 ms, simple1: 
841.5 ms]
Latency 99th percentile : 1047.5 ms [insert: 14.8 ms, range1: 1,210.1 ms, 
simple1: 1,230.0 ms]
Latency 99.9th percentile : 1830.8 ms [insert: 57.5 ms, range1: 2,029.0 ms, 
simple1: 2,063.6 ms]
Latency max : 7457.5 ms [insert: 6,358.6 ms, range1: 7,159.7 ms, simple1: 
7,457.5 ms]
Total partitions : 790,543 [insert: 378,618, range1: 33,908, simple1: 378,017]
Total errors : 0 [insert: 0, range1: 0, simple1: 0]
Total GC count : 0
Total GC memory : 0.000 KiB
Total GC time : 0.0 seconds
Avg GC time : NaN ms
StdDev GC time : 0.0 ms
Total operation time : 01:00:00|

 

 

> RangeAwareCompaction
> 
>
> Key: CASSANDRA-10540
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10540
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
>  Labels: compaction, lcs, vnodes
> Fix For: 4.x
>
>
> Broken out from CASSANDRA-6696, we should split sstables based on ranges 
> during compaction.
> Requirements;
> * dont create tiny sstables - keep them bunched together until a single vnode 
> is big enough (configurable how big that is)
> * make it possible to run existing compaction strategies on the per-range 
> sstables
> We should probably add a global compaction strategy parameter that states 
> whether this should be enabled or not.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14500) Debian package to include systemd file and conf

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-14500:
---
Status: Patch Available  (was: Awaiting Feedback)

> Debian package to include systemd file and conf
> ---
>
> Key: CASSANDRA-14500
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14500
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Packaging
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Minor
>
> I've been testing Cassandra on trunk on Debian stretch, and have been 
> creating my own systemd service files for Cassandra. My Cassandra clusters 
> would sometimes die due to too many open files. 
> As it turns out after some digging, this is because systemd ignores 
> */etc/security/limits.conf.* It relies on a configuration file in 
> .d/.conf. There's more information here: 
> [https://www.freedesktop.org/software/systemd/man/systemd-system.conf.html]. 
> So, for example, for */etc/systemd/system/cassandra.service*, the ulimits are 
> read from */etc/systemd/system/cassandra.service.d/cassandra.conf*. 
> Crosschecking with the limits of my Cassandra process, it looks like the 
> */etc/security/limits.conf* really were not respected. If I make the change 
> above, then it works as expected. */etc/security/limits.conf* is shipped in 
> Cassandra's debian package. 
> Given that there are far more distributions using Systemd (Ubuntu is now as 
> well), I was wondering if it's worth the effort to change Cassandra's debian 
> packaging to use systemd (or at least, include systemd service). I'm not 
> totally familiar with whether it's common or normal to include a service file 
> in packaging so happy to be corrected/cancelled depending on what people 
> think. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14500) Debian package to include systemd file and conf

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-14500:
---
Status: Awaiting Feedback  (was: Open)

> Debian package to include systemd file and conf
> ---
>
> Key: CASSANDRA-14500
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14500
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Packaging
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Minor
>
> I've been testing Cassandra on trunk on Debian stretch, and have been 
> creating my own systemd service files for Cassandra. My Cassandra clusters 
> would sometimes die due to too many open files. 
> As it turns out after some digging, this is because systemd ignores 
> */etc/security/limits.conf.* It relies on a configuration file in 
> .d/.conf. There's more information here: 
> [https://www.freedesktop.org/software/systemd/man/systemd-system.conf.html]. 
> So, for example, for */etc/systemd/system/cassandra.service*, the ulimits are 
> read from */etc/systemd/system/cassandra.service.d/cassandra.conf*. 
> Crosschecking with the limits of my Cassandra process, it looks like the 
> */etc/security/limits.conf* really were not respected. If I make the change 
> above, then it works as expected. */etc/security/limits.conf* is shipped in 
> Cassandra's debian package. 
> Given that there are far more distributions using Systemd (Ubuntu is now as 
> well), I was wondering if it's worth the effort to change Cassandra's debian 
> packaging to use systemd (or at least, include systemd service). I'm not 
> totally familiar with whether it's common or normal to include a service file 
> in packaging so happy to be corrected/cancelled depending on what people 
> think. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13698) Reinstate or get rid of unit tests with multiple compaction strategies

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504262#comment-16504262
 ] 

Lerh Chuan Low commented on CASSANDRA-13698:


Hi Paulo, 

It looks like there may have been a bad merge - the patch for 3.0 may have 
gotten applied to 3.11 instead. The project currently doesn't compile - the 
offending lines are here:
{code:java}
Clustering startClustering = Clustering.make(ByteBufferUtil.bytes("0"));
Clustering endClustering = Clustering.make(ByteBufferUtil.bytes("b"));
{code}
And
{code:java}
try (ReadExecutionController executionController = 
command.executionController();
 PartitionIterator iterator = command.executeInternal(executionController))
{code}
[^13698-hotfix.txt] is a patch, and here's the branch if it floats your boat 
more: 

[https://github.com/apache/cassandra/compare/cassandra-3.11...juiceblender:cassandra-13698-hotfix?expand=1]

The test runs locally and Cassandra also starts. Here's the CCI: 
[https://circleci.com/gh/juiceblender/cassandra/97]

PS: Thanks for the explanation, the ninja test works for me :) 

> Reinstate or get rid of unit tests with multiple compaction strategies
> --
>
> Key: CASSANDRA-13698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13698
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Paulo Motta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Fix For: 4.0, 3.0.17, 3.11.3
>
> Attachments: 13698-3.0.txt, 13698-3.11.txt, 13698-hotfix.txt, 
> 13698-trunk.txt
>
>
> At some point there were (anti-)compaction tests with multiple compaction 
> strategy classes, but now it's only tested with {{STCS}}:
> * 
> [AnticompactionTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/AntiCompactionTest.java#L247]
> * 
> [CompactionsTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java#L85]
> We should either reinstate these tests or decide they are not important and 
> remove the unused parameter.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13698) Reinstate or get rid of unit tests with multiple compaction strategies

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-13698:
---
Attachment: 13698-hotfix.txt

> Reinstate or get rid of unit tests with multiple compaction strategies
> --
>
> Key: CASSANDRA-13698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13698
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Paulo Motta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Fix For: 4.0, 3.0.17, 3.11.3
>
> Attachments: 13698-3.0.txt, 13698-3.11.txt, 13698-hotfix.txt, 
> 13698-trunk.txt
>
>
> At some point there were (anti-)compaction tests with multiple compaction 
> strategy classes, but now it's only tested with {{STCS}}:
> * 
> [AnticompactionTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/AntiCompactionTest.java#L247]
> * 
> [CompactionsTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java#L85]
> We should either reinstate these tests or decide they are not important and 
> remove the unused parameter.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14500) Debian package to include systemd file and conf

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16502729#comment-16502729
 ] 

Lerh Chuan Low commented on CASSANDRA-14500:


Here is a branch that currently has what I am trying to do: 
[https://github.com/juiceblender/cassandra/tree/debian-systemd.] I tested it on 
my instance - it seems to work and my Cassandra starts up fine. Though I'll 
admit I'm not good at Debian packaging and had just learned it before trying to 
implement this change.


{code:java}
admin@ip-10-0-6-20:~/cassandra$ systemctl status cassandra.service
● cassandra.service - Apache Cassandra
Loaded: loaded (/lib/systemd/system/cassandra.service; enabled; vendor preset: 
enabled)
Drop-In: /lib/systemd/system/cassandra.service.d
└─cassandra.conf
Active: active (running) since Wed 2018-06-06 00:40:18 UTC; 12s ago
Main PID: 7173 (java)
CGroup: /system.slice/cassandra.service
└─7173 java -Xloggc:/var/log/cassandra/gc.log -ea -da:net.openhft... 
-XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 
-XX:+HeapDumpOnOutOfMemoryError -Xss256k -XX:StringTableSize=103 
-XX:+AlwaysPreTouch -XX:-UseBiasedLocking -XX:+Us

Jun 06 00:40:26 ip-10-0-6-20 cassandra[7173]: INFO [main] 2018-06-06 
00:40:26,715 MigrationManager.java:238 - Create new table: system_auth.roles
Jun 06 00:40:26 ip-10-0-6-20 cassandra[7173]: INFO [main] 2018-06-06 
00:40:26,831 MigrationManager.java:238 - Create new table: 
system_auth.role_members
Jun 06 00:40:26 ip-10-0-6-20 cassandra[7173]: INFO [main] 2018-06-06 
00:40:26,932 MigrationManager.java:238 - Create new table: 
system_auth.role_permissions
Jun 06 00:40:27 ip-10-0-6-20 cassandra[7173]: INFO [main] 2018-06-06 
00:40:27,039 MigrationManager.java:238 - Create new table: 
system_auth.resource_role_permissons_index
Jun 06 00:40:27 ip-10-0-6-20 cassandra[7173]: INFO [ScheduledTasks:1] 
2018-06-06 00:40:27,084 TokenMetadata.java:489 - Updating topology for all 
endpoints that have changed
Jun 06 00:40:27 ip-10-0-6-20 cassandra[7173]: INFO [main] 2018-06-06 
00:40:27,146 MigrationManager.java:238 - Create new table: 
system_auth.network_permissions
Jun 06 00:40:27 ip-10-0-6-20 cassandra[7173]: INFO [MigrationStage:1] 
2018-06-06 00:40:27,241 Keyspace.java:369 - Creating replication strategy 
system_auth params KeyspaceParams{durable_writes=true, 
replication=ReplicationParams{class=org.apache.cass
Jun 06 00:40:27 ip-10-0-6-20 cassandra[7173]: INFO [MigrationStage:1] 
2018-06-06 00:40:27,247 ColumnFamilyStore.java:402 - Initializing 
system_auth.network_permissions
Jun 06 00:40:27 ip-10-0-6-20 cassandra[7173]: INFO [MigrationStage:1] 
2018-06-06 00:40:27,250 ViewManager.java:131 - Not submitting build tasks for 
views in keyspace system_auth as storage service is not initialized
Jun 06 00:40:27 ip-10-0-6-20 cassandra[7173]: INFO [main] 2018-06-06 
00:40:27,261 Gossiper.java:1802 - Waiting for gossip to settle...{code}

The actual commit is here: 
[https://github.com/juiceblender/cassandra/commit/54a16b83ec553dd4000907409046129abf50616b
|https://github.com/juiceblender/cassandra/commit/05c112e77e2888d3c19ec36cc40c7c6872fd42ce]

I've not removed the original init scripts because I wasn't sure if it was a 
good idea - maybe some of the users are running really old distributions? If 
people think this is a good idea, I'll cleanup some of the other things as 
well. 

> Debian package to include systemd file and conf
> ---
>
> Key: CASSANDRA-14500
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14500
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Packaging
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Minor
>
> I've been testing Cassandra on trunk on Debian stretch, and have been 
> creating my own systemd service files for Cassandra. My Cassandra clusters 
> would sometimes die due to too many open files. 
> As it turns out after some digging, this is because systemd ignores 
> */etc/security/limits.conf.* It relies on a configuration file in 
> .d/.conf. There's more information here: 
> [https://www.freedesktop.org/software/systemd/man/systemd-system.conf.html]. 
> So, for example, for */etc/systemd/system/cassandra.service*, the ulimits are 
> read from */etc/systemd/system/cassandra.service.d/cassandra.conf*. 
> Crosschecking with the limits of my Cassandra process, it looks like the 
> */etc/security/limits.conf* really were not respected. If I make the change 
> above, then it works as expected. */etc/security/limits.conf* is shipped in 
> Cassandra's debian package. 
> Given that there are far more distributions using Systemd (Ubuntu is now as 
> well), I was wondering if it's worth the effort to change Cassandra's debian 
> packaging to use systemd (or at least, include systemd service). I'm not 
> totally familiar 

[jira] [Created] (CASSANDRA-14500) Debian package to include systemd file and conf

[Lerh Chuan Low (JIRA)]

Lerh Chuan Low created CASSANDRA-14500:
--

 Summary: Debian package to include systemd file and conf
 Key: CASSANDRA-14500
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14500
 Project: Cassandra
  Issue Type: Improvement
  Components: Packaging
Reporter: Lerh Chuan Low
Assignee: Lerh Chuan Low


I've been testing Cassandra on trunk on Debian stretch, and have been creating 
my own systemd service files for Cassandra. My Cassandra clusters would 
sometimes die due to too many open files. 

As it turns out after some digging, this is because systemd ignores 
*/etc/security/limits.conf.* It relies on a configuration file in 
.d/.conf. There's more information here: 
[https://www.freedesktop.org/software/systemd/man/systemd-system.conf.html]. 

So, for example, for */etc/systemd/system/cassandra.service*, the ulimits are 
read from */etc/systemd/system/cassandra.service.d/cassandra.conf*. 

Crosschecking with the limits of my Cassandra process, it looks like the 
*/etc/security/limits.conf* really were not respected. If I make the change 
above, then it works as expected. */etc/security/limits.conf* is shipped in 
Cassandra's debian package. 

Given that there are far more distributions using Systemd (Ubuntu is now as 
well), I was wondering if it's worth the effort to change Cassandra's debian 
packaging to use systemd (or at least, include systemd service). I'm not 
totally familiar with whether it's common or normal to include a service file 
in packaging so happy to be corrected/cancelled depending on what people think. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13698) Reinstate or get rid of unit tests with multiple compaction strategies

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16502602#comment-16502602
 ] 

Lerh Chuan Low commented on CASSANDRA-13698:


Gentle prod...

> Reinstate or get rid of unit tests with multiple compaction strategies
> --
>
> Key: CASSANDRA-13698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13698
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Paulo Motta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Attachments: 13698-3.0.txt, 13698-3.11.txt, 13698-trunk.txt
>
>
> At some point there were (anti-)compaction tests with multiple compaction 
> strategy classes, but now it's only tested with {{STCS}}:
> * 
> [AnticompactionTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/AntiCompactionTest.java#L247]
> * 
> [CompactionsTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java#L85]
> We should either reinstate these tests or decide they are not important and 
> remove the unused parameter.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-13938) Default repair is broken, crashes other nodes participating in repair (in trunk)

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499721#comment-16499721
 ] 

Lerh Chuan Low edited comment on CASSANDRA-13938 at 6/4/18 4:08 AM:


Here's another stacktrace that may help - I've also been getting these while 
testing trunk in EC2. The steps I use are the same: 
 - Disable hintedhandoff
 - Take out 1 node
 - Run stress for 10 mins, then run repair

It will error out and the nodes also end up in a bizarre situation with gossip 
that I will have to stop the entire cluster and then start them up one at a 
time (in a rolling restart they still won't be able to sort themselves out). 
Sometimes it errors with {{stream can only read forward}} (as above and in the 
JIRA), but here's another stacktrace that has also showed up several times in 
some of the failed nodes:
{code:java}
May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: ERROR 
[Stream-Deserializer-35.155.140.194:39371-28daf76d] 2018-05-31 02:07:24,445 
StreamingInboundHandler.java:210 - [Stream channel: 28daf76d] stream operation 
from 35.155.140.194:39371 failed
May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: 
java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 1711542017
May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
net.jpountz.util.ByteBufferUtils.checkRange(ByteBufferUtils.java:20)
May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
net.jpountz.util.ByteBufferUtils.checkRange(ByteBufferUtils.java:14)
May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
net.jpountz.lz4.LZ4JNIFastDecompressor.decompress(LZ4JNIFastDecompressor.java:48)
May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.io.compress.LZ4Compressor.uncompress(LZ4Compressor.java:162)
May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.db.streaming.CompressedInputStream.decompress(CompressedInputStream.java:163)
May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.db.streaming.CompressedInputStream.reBuffer(CompressedInputStream.java:144)
May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.db.streaming.CompressedInputStream.reBuffer(CompressedInputStream.java:119)
May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:144)
May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.io.util.RebufferingInputStream.readPrimitiveSlowly(RebufferingInputStream.java:108)
May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.io.util.RebufferingInputStream.readShort(RebufferingInputStream.java:164)
May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.io.util.RebufferingInputStream.readUnsignedShort(RebufferingInputStream.java:170)
May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.io.util.TrackedDataInputPlus.readUnsignedShort(TrackedDataInputPlus.java:139)
May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.utils.ByteBufferUtil.readShortLength(ByteBufferUtil.java:367)
May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:377)
May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.db.streaming.CassandraStreamReader$StreamDeserializer.newPartition(CassandraStreamReader.java:199)
May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.db.streaming.CassandraStreamReader.writePartition(CassandraStreamReader.java:172){code}
I get the feeling they may be related but I'm not sure...I can open a different 
Jira for this if you like, but otherwise hope it may point out more clues as to 
what is going on :/ 


was (Author: lerh low):
Here's another stacktrace that may help - I've also been getting these while 
testing trunk in EC2. The steps I use are the same: 
 - Disable hintedhandoff
 - Take out 1 node
 - Run stress for 10 mins, then run repair

It will error out and the nodes also end up in a bizarre situation with gossip 
that I will have to stop the entire cluster and then start them up one at a 
time (in a rolling restart they still won't be able to sort themselves out). 
Sometimes it errors with {{stream can only read forward}} (as above and in the 
JIRA), but here's another stacktrace that has also showed up several times in 
some of the failed nodes:
{code:java}
May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: WARN 
[Stream-Deserializer-35.155.140.194:39371-28daf76d] 2018-05-31 02:07:24,440 
CompressedCassandraStreamReader.java:110 - [Stream 
e12b9b10-6476-11e8-936f-35a28469245e] Error while reading partition null from 
stream on May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: ERROR 
[Stream-Deserializer-35.155.140.194:39371-28daf76d] 2018-05-31 02:07:24,445 
StreamingInboundHandler.java:210 - [Stream channel: 28daf76d] stream operation 
from 

[jira] [Comment Edited] (CASSANDRA-13938) Default repair is broken, crashes other nodes participating in repair (in trunk)

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499721#comment-16499721
 ] 

Lerh Chuan Low edited comment on CASSANDRA-13938 at 6/4/18 4:07 AM:


Here's another stacktrace that may help - I've also been getting these while 
testing trunk in EC2. The steps I use are the same: 
 - Disable hintedhandoff
 - Take out 1 node
 - Run stress for 10 mins, then run repair

It will error out and the nodes also end up in a bizarre situation with gossip 
that I will have to stop the entire cluster and then start them up one at a 
time (in a rolling restart they still won't be able to sort themselves out). 
Sometimes it errors with {{stream can only read forward}} (as above and in the 
JIRA), but here's another stacktrace that has also showed up several times in 
some of the failed nodes:
{code:java}
May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: WARN 
[Stream-Deserializer-35.155.140.194:39371-28daf76d] 2018-05-31 02:07:24,440 
CompressedCassandraStreamReader.java:110 - [Stream 
e12b9b10-6476-11e8-936f-35a28469245e] Error while reading partition null from 
stream on May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: ERROR 
[Stream-Deserializer-35.155.140.194:39371-28daf76d] 2018-05-31 02:07:24,445 
StreamingInboundHandler.java:210 - [Stream channel: 28daf76d] stream operation 
from 35.155.140.194:39371 failed May 31 02:07:24 ip-10-0-18-230 
cassandra[6034]: java.lang.ArrayIndexOutOfBoundsException: Array index out of 
range: 1711542017 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
net.jpountz.util.ByteBufferUtils.checkRange(ByteBufferUtils.java:20) May 31 
02:07:24 ip-10-0-18-230 cassandra[6034]: at 
net.jpountz.util.ByteBufferUtils.checkRange(ByteBufferUtils.java:14) May 31 
02:07:24 ip-10-0-18-230 cassandra[6034]: at 
net.jpountz.lz4.LZ4JNIFastDecompressor.decompress(LZ4JNIFastDecompressor.java:48)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.io.compress.LZ4Compressor.uncompress(LZ4Compressor.java:162)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.db.streaming.CompressedInputStream.decompress(CompressedInputStream.java:163)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.db.streaming.CompressedInputStream.reBuffer(CompressedInputStream.java:144)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.db.streaming.CompressedInputStream.reBuffer(CompressedInputStream.java:119)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:144)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.io.util.RebufferingInputStream.readPrimitiveSlowly(RebufferingInputStream.java:108)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.io.util.RebufferingInputStream.readShort(RebufferingInputStream.java:164)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.io.util.RebufferingInputStream.readUnsignedShort(RebufferingInputStream.java:170)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.io.util.TrackedDataInputPlus.readUnsignedShort(TrackedDataInputPlus.java:139)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.utils.ByteBufferUtil.readShortLength(ByteBufferUtil.java:367)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:377)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.db.streaming.CassandraStreamReader$StreamDeserializer.newPartition(CassandraStreamReader.java:199)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.db.streaming.CassandraStreamReader.writePartition(CassandraStreamReader.java:172){code}
I get the feeling they may be related but I'm not sure...I can open a different 
Jira for this if you like, but otherwise hope it may point out more clues as to 
what is going on :/ 


was (Author: lerh low):
Here's another stacktrace that may help - I've also been getting these while 
testing trunk in EC2. The steps I use are the same: 

- Disable hintedhandoff
- Take out 1 node
- Run stress for 10 mins, then run repair

It will error out and the nodes also end up in a bizarre situation with gossip 
that I will have to stop the entire cluster and then start them up one at a 
time (in a rolling restart they still won't be able to sort themselves out). 
Sometimes it errors with {{stream can only read forward}}, but here's another 
stacktrace that has also showed up several times in some of the failed nodes:
{code:java}
May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: WARN 
[Stream-Deserializer-35.155.140.194:39371-28daf76d] 2018-05-31 02:07:24,440 
CompressedCassandraStreamReader.java:110 - [Stream 
e12b9b10-6476-11e8-936f-35a28469245e] Error while 

[jira] [Commented] (CASSANDRA-13938) Default repair is broken, crashes other nodes participating in repair (in trunk)

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499721#comment-16499721
 ] 

Lerh Chuan Low commented on CASSANDRA-13938:


Here's another stacktrace that may help - I've also been getting these while 
testing trunk in EC2. The steps I use are the same: 

- Disable hintedhandoff
- Take out 1 node
- Run stress for 10 mins, then run repair

It will error out and the nodes also end up in a bizarre situation with gossip 
that I will have to stop the entire cluster and then start them up one at a 
time (in a rolling restart they still won't be able to sort themselves out). 
Sometimes it errors with {{stream can only read forward}}, but here's another 
stacktrace that has also showed up several times in some of the failed nodes:
{code:java}
May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: WARN 
[Stream-Deserializer-35.155.140.194:39371-28daf76d] 2018-05-31 02:07:24,440 
CompressedCassandraStreamReader.java:110 - [Stream 
e12b9b10-6476-11e8-936f-35a28469245e] Error while reading partition null from 
stream on May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: ERROR 
[Stream-Deserializer-35.155.140.194:39371-28daf76d] 2018-05-31 02:07:24,445 
StreamingInboundHandler.java:210 - [Stream channel: 28daf76d] stream operation 
from 35.155.140.194:39371 failed May 31 02:07:24 ip-10-0-18-230 
cassandra[6034]: java.lang.ArrayIndexOutOfBoundsException: Array index out of 
range: 1711542017 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
net.jpountz.util.ByteBufferUtils.checkRange(ByteBufferUtils.java:20) May 31 
02:07:24 ip-10-0-18-230 cassandra[6034]: at 
net.jpountz.util.ByteBufferUtils.checkRange(ByteBufferUtils.java:14) May 31 
02:07:24 ip-10-0-18-230 cassandra[6034]: at 
net.jpountz.lz4.LZ4JNIFastDecompressor.decompress(LZ4JNIFastDecompressor.java:48)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.io.compress.LZ4Compressor.uncompress(LZ4Compressor.java:162)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.db.streaming.CompressedInputStream.decompress(CompressedInputStream.java:163)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.db.streaming.CompressedInputStream.reBuffer(CompressedInputStream.java:144)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.db.streaming.CompressedInputStream.reBuffer(CompressedInputStream.java:119)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:144)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.io.util.RebufferingInputStream.readPrimitiveSlowly(RebufferingInputStream.java:108)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.io.util.RebufferingInputStream.readShort(RebufferingInputStream.java:164)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.io.util.RebufferingInputStream.readUnsignedShort(RebufferingInputStream.java:170)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.io.util.TrackedDataInputPlus.readUnsignedShort(TrackedDataInputPlus.java:139)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.utils.ByteBufferUtil.readShortLength(ByteBufferUtil.java:367)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:377)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.db.streaming.CassandraStreamReader$StreamDeserializer.newPartition(CassandraStreamReader.java:199)
 May 31 02:07:24 ip-10-0-18-230 cassandra[6034]: at 
org.apache.cassandra.db.streaming.CassandraStreamReader.writePartition(CassandraStreamReader.java:172){code}
I get the feeling they may be related but I'm not sure...I can open a different 
Jira for this if you like, but otherwise hope it may point out more clues as to 
what is going on :/ 

> Default repair is broken, crashes other nodes participating in repair (in 
> trunk)
> 
>
> Key: CASSANDRA-13938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13938
> Project: Cassandra
>  Issue Type: Bug
>  Components: Repair
>Reporter: Nate McCall
>Assignee: Jason Brown
>Priority: Critical
> Attachments: 13938.yaml, test.sh
>
>
> Running through a simple scenario to test some of the new repair features, I 
> was not able to make a repair command work. Further, the exception seemed to 
> trigger a nasty failure state that basically shuts down the netty connections 
> for messaging *and* CQL on the nodes transferring back data to the node being 
> repaired. The following steps reproduce this issue consistently.
> Cassandra stress profile (probably not 

[jira] [Comment Edited] (CASSANDRA-10540) RangeAwareCompaction

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16496077#comment-16496077
 ] 

Lerh Chuan Low edited comment on CASSANDRA-10540 at 5/31/18 4:34 AM:
-

Hi [~krummas], 

Sorry for the delay, here are some initial benchmarks. I've only tried it with 
LCS, this is the Stressspec YAML, a reasonably stressful test:
{code:java}
keyspace: stresscql2

keyspace_definition: |
CREATE KEYSPACE stresscql2 WITH replication = {'class': 
'NetworkTopologyStrategy', 'Waboku': 3, 'Bokusapp': 2};

table: typestest

table_definition: |
CREATE TABLE typestest (
name text,
choice boolean,
date timestamp,
address inet,
dbl double,
lval bigint,
ival int,
uid timeuuid,
value blob,
PRIMARY KEY((name,choice), date, address, dbl, lval, ival, uid)
) WITH compaction = { 'class':'LeveledCompactionStrategy', 
'range_aware_compaction':'true', 'min_range_sstable_size_in_mb':'15' }
AND comment='A table of many types to test wide rows'

columnspec:

name: name
size: uniform(1..1000)
population: uniform(1..500M) # the range of unique values to select for the 
field (default is 100Billion)
name: date
cluster: uniform(20..1000)
name: lval
population: gaussian(1..1000)
cluster: uniform(1..4)
name: value
size: uniform(100..500)
insert:
partitions: fixed(1) # number of unique partitions to update in a single 
operation
batchtype: UNLOGGED # type of batch to use
select: uniform(1..10)/10 # uniform chance any single generated CQL row will be 
visited in a partition;

queries:
simple1:
cql: select * from typestest where name = ? and choice = ? LIMIT 1
fields: samerow 
range1:
cql: select name, choice, uid from typestest where name = ? and choice = ? and 
date >= ? LIMIT 10
fields: multirow 
simple2:
cql: select name, choice, uid from typestest where name = ? and choice = ? 
LIMIT 1
fields: samerow # samerow or multirow (select arguments from the same row, or 
randomly from all rows in the partition)
{code}
This is done over a multi DC cluster in EC2, 400GB SSD with 3 nodes in 1 and 2 
nodes in the other. Stress replicates to both DCs. 

For inserts:
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml n=15000 
cl=QUORUM ops(insert=1) -node file=nodelist.txt -rate threads=100 -log 
file=insert.log > nohup.txt &{code}
We have
|| ||RACS||NonRACS||
|Stress result|Op rate : 8,784 op/s [insert: 8,784 op/s]
 Partition rate : 8,784 pk/s [insert: 8,784 pk/s]
 Row rate : 8,784 row/s [insert: 8,784 row/s]
 Latency mean : 5.4 ms [insert: 5.4 ms]
 Latency median : 4.3 ms [insert: 4.3 ms]
 Latency 95th percentile : 8.4 ms [insert: 8.4 ms]
 Latency 99th percentile : 39.2 ms [insert: 39.2 ms]
 Latency 99.9th percentile : 63.3 ms [insert: 63.3 ms]
 Latency max : 1506.8 ms [insert: 1,506.8 ms]
 Total partitions : 150,000,000 [insert: 150,000,000]
 Total errors : 0 [insert: 0]
 Total GC count : 0
 Total GC memory : 0.000 KiB
 Total GC time : 0.0 seconds
 Avg GC time : NaN ms
 StdDev GC time : 0.0 ms
 Total operation time : 04:44:35|Op rate : 8,730 op/s [insert: 8,730 op/s]
 Partition rate : 8,730 pk/s [insert: 8,730 pk/s]
 Row rate : 8,730 row/s [insert: 8,730 row/s]
 Latency mean : 5.4 ms [insert: 5.4 ms]
 Latency median : 4.3 ms [insert: 4.3 ms]
 Latency 95th percentile : 8.5 ms [insert: 8.5 ms]
 Latency 99th percentile : 39.4 ms [insert: 39.4 ms]
 Latency 99.9th percentile : 66.1 ms [insert: 66.1 ms]
 Latency max : 944.8 ms [insert: 944.8 ms]
 Total partitions : 150,000,000 [insert: 150,000,000]
 Total errors : 0 [insert: 0]
 Total GC count : 0
 Total GC memory : 0.000 KiB
 Total GC time : 0.0 seconds
 Avg GC time : NaN ms
 StdDev GC time : 0.0 ms
 Total operation time : 04:46:22|
|SSTable count|1339
 1259
 1342
 1285
 1333|743
 750
 747
 737
 741|

For mixed workloads, which is done after the insert so reads are not just read 
off the OS page cache:
{code:java}
nohup cassandra-stress user no-warmup profile=stressspec.yaml duration=2h 
cl=QUORUM ops\(insert=10,simple1=10,range1=1\) -node file=nodelist.txt -rate 
threads=50 -log file=mixed.log > nohup.txt &
{code}
|| ||RACS||Non RACS||
|Stress result |Op rate : 415 op/s [insert: 197 op/s, range1: 20 op/s, simple1: 
198 op/s]
 Partition rate : 407 pk/s [insert: 197 pk/s, range1: 12 pk/s, simple1: 198 
pk/s]
 Row rate : 412 row/s [insert: 197 row/s, range1: 17 row/s, simple1: 198 row/s]
 Latency mean : 120.4 ms [insert: 2.3 ms, range1: 227.0 ms, simple1: 227.3 ms]
 Latency median : 38.0 ms [insert: 2.0 ms, range1: 207.0 ms, simple1: 207.4 ms]
 Latency 95th percentile : 454.6 ms [insert: 3.1 ms, range1: 541.1 ms, simple1: 
543.2 ms]
 Latency 99th percentile : 673.2 ms [insert: 5.1 ms, range1: 739.2 ms, simple1: 
741.3 ms]
 Latency 99.9th percentile : 918.0 ms [insert: 43.4 ms, range1: 985.1 ms, 
simple1: 975.2 ms]
 Latency max : 1584.4 ms [insert: 766.0 ms, range1: 1,426.1 ms, simple1: 
1,584.4 ms]
 Total partitions : 2,930,512 [insert: 

[jira] [Commented] (CASSANDRA-10540) RangeAwareCompaction

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16496077#comment-16496077
 ] 

Lerh Chuan Low commented on CASSANDRA-10540:


Hi [~krummas], 

Sorry for the delay, here are some initial benchmarks. I've only tried it with 
LCS, this is the Stressspec YAML, a reasonably stressful test:

```
keyspace: stresscql2

keyspace_definition: |
 CREATE KEYSPACE stresscql2 WITH replication = \{'class': 
'NetworkTopologyStrategy', 'Waboku': 3, 'Bokusapp': 2};

table: typestest


table_definition: |
 CREATE TABLE typestest (
 name text,
 choice boolean,
 date timestamp,
 address inet,
 dbl double,
 lval bigint,
 ival int,
 uid timeuuid,
 value blob,
 PRIMARY KEY((name,choice), date, address, dbl, lval, ival, uid)
 ) WITH compaction = \{ 'class':'LeveledCompactionStrategy', 
'range_aware_compaction':'true', 'min_range_sstable_size_in_mb':'15' }
 AND comment='A table of many types to test wide rows'

columnspec:
 - name: name
 size: uniform(1..1000)
 population: uniform(1..500M) # the range of unique values to select for the 
field (default is 100Billion)
 - name: date
 cluster: uniform(20..1000)
 - name: lval
 population: gaussian(1..1000)
 cluster: uniform(1..4)
 - name: value
 size: uniform(100..500)

insert:
 partitions: fixed(1) # number of unique partitions to update in a single 
operation
 batchtype: UNLOGGED # type of batch to use
 select: uniform(1..10)/10 # uniform chance any single generated CQL row will 
be visited in a partition;
 
queries:
 simple1:
 cql: select * from typestest where name = ? and choice = ? LIMIT 1
 fields: samerow 
 range1:
 cql: select name, choice, uid from typestest where name = ? and choice = ? and 
date >= ? LIMIT 10
 fields: multirow 
 simple2:
 cql: select name, choice, uid from typestest where name = ? and choice = ? 
LIMIT 1
 fields: samerow # samerow or multirow (select arguments from the same row, or 
randomly from all rows in the partition)
```

This is done over a multi DC cluster in EC2, 400GB SSD with 3 nodes in 1 and 2 
nodes in the other. Stress replicates to both DCs. 

For inserts:
```
nohup cassandra-stress user no-warmup profile=stressspec.yaml n=15000 
cl=QUORUM ops\(insert=1\) -node file=nodelist.txt -rate threads=100 -log 
file=insert.log > nohup.txt &
```

We have


|| ||RACS||NonRACS||
|Stress result|Op rate : 8,784 op/s [insert: 8,784 op/s]
Partition rate : 8,784 pk/s [insert: 8,784 pk/s]
Row rate : 8,784 row/s [insert: 8,784 row/s]
Latency mean : 5.4 ms [insert: 5.4 ms]
Latency median : 4.3 ms [insert: 4.3 ms]
Latency 95th percentile : 8.4 ms [insert: 8.4 ms]
Latency 99th percentile : 39.2 ms [insert: 39.2 ms]
Latency 99.9th percentile : 63.3 ms [insert: 63.3 ms]
Latency max : 1506.8 ms [insert: 1,506.8 ms]
Total partitions : 150,000,000 [insert: 150,000,000]
Total errors : 0 [insert: 0]
Total GC count : 0
Total GC memory : 0.000 KiB
Total GC time : 0.0 seconds
Avg GC time : NaN ms
StdDev GC time : 0.0 ms
Total operation time : 04:44:35|Op rate : 8,730 op/s [insert: 8,730 op/s]
Partition rate : 8,730 pk/s [insert: 8,730 pk/s]
Row rate : 8,730 row/s [insert: 8,730 row/s]
Latency mean : 5.4 ms [insert: 5.4 ms]
Latency median : 4.3 ms [insert: 4.3 ms]
Latency 95th percentile : 8.5 ms [insert: 8.5 ms]
Latency 99th percentile : 39.4 ms [insert: 39.4 ms]
Latency 99.9th percentile : 66.1 ms [insert: 66.1 ms]
Latency max : 944.8 ms [insert: 944.8 ms]
Total partitions : 150,000,000 [insert: 150,000,000]
Total errors : 0 [insert: 0]
Total GC count : 0
Total GC memory : 0.000 KiB
Total GC time : 0.0 seconds
Avg GC time : NaN ms
StdDev GC time : 0.0 ms
Total operation time : 04:46:22|
|SSTable count|1339
1259
1342
1285
1333|743
750
747
737
741|


For mixed workloads, which is done after the insert so reads are not just read 
off the OS page cache:



 
|| ||RACS||Non RACS||
|Stress result |Op rate : 415 op/s [insert: 197 op/s, range1: 20 op/s, simple1: 
198 op/s]
Partition rate : 407 pk/s [insert: 197 pk/s, range1: 12 pk/s, simple1: 198 pk/s]
Row rate : 412 row/s [insert: 197 row/s, range1: 17 row/s, simple1: 198 row/s]
Latency mean : 120.4 ms [insert: 2.3 ms, range1: 227.0 ms, simple1: 227.3 ms]
Latency median : 38.0 ms [insert: 2.0 ms, range1: 207.0 ms, simple1: 207.4 ms]
Latency 95th percentile : 454.6 ms [insert: 3.1 ms, range1: 541.1 ms, simple1: 
543.2 ms]
Latency 99th percentile : 673.2 ms [insert: 5.1 ms, range1: 739.2 ms, simple1: 
741.3 ms]
Latency 99.9th percentile : 918.0 ms [insert: 43.4 ms, range1: 985.1 ms, 
simple1: 975.2 ms]
Latency max : 1584.4 ms [insert: 766.0 ms, range1: 1,426.1 ms, simple1: 1,584.4 
ms]
Total partitions : 2,930,512 [insert: 1,419,222, range1: 86,021, simple1: 
1,425,269]
Total errors : 0 [insert: 0, range1: 0, simple1: 0]
Total GC count : 0
Total GC memory : 0.000 KiB
Total GC time : 0.0 seconds
Avg GC time : NaN ms
StdDev GC time : 0.0 ms
Total operation time : 02:00:01|Op rate : 382 op/s [insert: 182 

[jira] [Commented] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479994#comment-16479994
 ] 

Lerh Chuan Low commented on CASSANDRA-14427:


Hey [~jasobrown], 

Sorry for the late reply, was overseas and then fell sick upon returning. 
Thanks for looking at this ticket!!

Yep - the linked CVEs all reference Jackson databind, which really is 
codehaus's mapper asl. To my understanding, Jackson moved from codehaus to 
Github when Jackson 2 was released. The CVEs reference jackson databind, which 
makes it vague whether it affects codehaus or not. 

Digging a little bit deeper, there are a few good articles that describe the 
CVEs better and how they relate to Jackson:

[https://adamcaudill.com/2017/10/04/exploiting-jackson-rce-cve-2017-7525/]
[https://medium.com/@cowtowncoder/on-jackson-cves-dont-panic-here-is-what-you-need-to-know-54cd0d6e8062]

And the actual paper itself on describing the vulnerability:

[https://github.com/mbechler/marshalsec]

Reading into them, Jackson by default is not affected by the CVE. We are only 
vulnerable under certain situations as described by cowtowncoder's article in 
medium, which is accepting bad JSON to begin with (from what I can tell it 
looks like we read in JSON when altering compaction strategy via JMX/reading in 
CQL). Presumably the user wouldn't want to do it themselves so the attacker 
needs to do that first. 

Assuming that condition is met, ObjectMapper also needs to be able to be able 
to handle polymorphic types.This can be done in 2 ways: 
 * ObjectMapper.enableDefaultTyping(). We don't do this in the code base. 
 * Explicitly defining polymorphic types using annotations JsonSubType or 
JsonTypeInfo. We don't do this either.

For the record, Codehaus's ObjectMapper (Or Jackson 1's ObjectMapper) does 
allow you to do this though, if we wanted to. 
[https://github.com/codehaus/jackson/blob/master/src/mapper/java/org/codehaus/jackson/map/ObjectMapper.java#L848]
 So I would think that it is affected (being an older version of Jackson), but 
by default no and in our use case, no. 

>From a user perspective some have mentioned their worries on using Jackson 1 
>with Cassandra because it's no longer being updated or maintained from a 
>security perspective, and in general keeping dependencies up to date is a plus 
>I think. 

I'm ambivalent with regards to upgrading our previous releases, I think the 
pros is nothing more than not using out of date libraries and the cons is we(I) 
don't know if anything will break beyond the unit tests because we don't know 
what we don't know. The changes aren't very big though that said because a lot 
of the code between Jackson 1 & 2 are the same. 

With regards to trunk, if we can take this opportunity to, then bumping Jackson 
is good practice. Those are just my thoughts :) 

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: 2.1-14427.txt, 2.2-14427.txt, 3.0-14427.txt, 
> 3.X-14427.txt, trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463246#comment-16463246
 ] 

Lerh Chuan Low edited comment on CASSANDRA-14427 at 5/4/18 2:55 AM:


Updated the patch, turns out I missed a few things. 

The 2.2 CI failed, but it seems unrelated. I tried running the test locally, it 
works, so trying again:
https://circleci.com/gh/juiceblender/cassandra/84

Updated 2.1 CCI:
https://circleci.com/gh/juiceblender/cassandra/85


was (Author: lerh low):
Updated the patch, turns out I missed a few things. 



The 2.2 CI failed, but it seems unrelated. I tried running the test locally, it 
works, so trying again:
https://circleci.com/gh/juiceblender/cassandra/82

Updated 2.1 CCI:
https://circleci.com/gh/juiceblender/cassandra/81

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: 2.1-14427.txt, 2.2-14427.txt, 3.0-14427.txt, 
> 3.X-14427.txt, trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-14427:
---
Attachment: (was: 2.1-14427.txt)

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: 2.1-14427.txt, 2.2-14427.txt, 3.0-14427.txt, 
> 3.X-14427.txt, trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-14427:
---
Attachment: 2.2-14427.txt
2.1-14427.txt

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: 2.1-14427.txt, 2.2-14427.txt, 3.0-14427.txt, 
> 3.X-14427.txt, trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-14427:
---
Attachment: (was: 2.2-14427.txt)

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: 2.1-14427.txt, 2.2-14427.txt, 3.0-14427.txt, 
> 3.X-14427.txt, trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463246#comment-16463246
 ] 

Lerh Chuan Low commented on CASSANDRA-14427:


Updated the patch, turns out I missed a few things. 



The 2.2 CI failed, but it seems unrelated. I tried running the test locally, it 
works, so trying again:
https://circleci.com/gh/juiceblender/cassandra/82

Updated 2.1 CCI:
https://circleci.com/gh/juiceblender/cassandra/81

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: 2.1-14427.txt, 2.2-14427.txt, 3.0-14427.txt, 
> 3.X-14427.txt, trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-14427:
---
Attachment: 2.1-14427.txt

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: 2.1-14427.txt, 2.2-14427.txt, 3.0-14427.txt, 
> 3.X-14427.txt, trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-14427:
---
Attachment: (was: 2.1-14427.txt)

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: 2.1-14427.txt, 2.2-14427.txt, 3.0-14427.txt, 
> 3.X-14427.txt, trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463225#comment-16463225
 ] 

Lerh Chuan Low commented on CASSANDRA-14427:


Turns out the 2.1 tests didn't make it, going to check on it. 

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: 2.1-14427.txt, 2.2-14427.txt, 3.0-14427.txt, 
> 3.X-14427.txt, trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458271#comment-16458271
 ] 

Lerh Chuan Low edited comment on CASSANDRA-14427 at 4/30/18 6:59 AM:
-

Github branch if preferred:

[https://github.com/juiceblender/cassandra/tree/jackson-update] 
[https://github.com/juiceblender/cassandra/tree/jackson-update-3.X
https://github.com/juiceblender/cassandra/tree/jackson-update-3.0
https://github.com/juiceblender/cassandra/tree/jackson-update-2.2
https://github.com/juiceblender/cassandra/tree/jackson-update-2.1|https://github.com/juiceblender/cassandra/tree/jackson-update-3.X]
 
CCI:

[https://circleci.com/gh/juiceblender/cassandra/76] (trunk)
 [https://circleci.com/gh/juiceblender/cassandra/77] (3.X)
 [https://circleci.com/gh/juiceblender/cassandra/78] (3.0)
 [https://circleci.com/gh/juiceblender/cassandra/79] (2.2)
 [https://circleci.com/gh/juiceblender/cassandra/80] (2.1)

I get the feeling some of the CCIs may fail (to my knowledge they currently 
don't work on 3.X and 3.0, not sure about 2.Xs). 


was (Author: lerh low):
Github branch if preferred:

[https://github.com/juiceblender/cassandra/tree/jackson-update] 
 [https://github.com/juiceblender/cassandra/tree/jackson-update-3.X
https://github.com/juiceblender/cassandra/tree/jackson-update-3.0
|https://github.com/juiceblender/cassandra/tree/jackson-update-3.X] 
[https://github.com/juiceblender/cassandra/tree/jackson-update-2|https://github.com/juiceblender/cassandra/tree/jackson-update-3.0].2
[https://github.com/juiceblender/cassandra/tree/jackson-update-2|https://github.com/juiceblender/cassandra/tree/jackson-update-3.0].1

CCI:

[https://circleci.com/gh/juiceblender/cassandra/76] (trunk)
[https://circleci.com/gh/juiceblender/cassandra/77] (3.X)
[https://circleci.com/gh/juiceblender/cassandra/78] (3.0)
 [https://circleci.com/gh/juiceblender/cassandra/79] (2.2)
 [https://circleci.com/gh/juiceblender/cassandra/80] (2.1)

I get the feeling some of the CCIs may fail (to my knowledge they currently 
don't work on 3.X and 3.0, not sure about 2.Xs). 

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: 2.1-14427.txt, 2.2-14427.txt, 3.0-14427.txt, 
> 3.X-14427.txt, trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458271#comment-16458271
 ] 

Lerh Chuan Low edited comment on CASSANDRA-14427 at 4/30/18 6:59 AM:
-

Github branch if preferred:

[https://github.com/juiceblender/cassandra/tree/jackson-update] 
 [https://github.com/juiceblender/cassandra/tree/jackson-update-3.0]
 [https://github.com/juiceblender/cassandra/tree/jackson-update-2.2]
[https://github.com/juiceblender/cassandra/tree/jackson-update-2.1]


  
 CCI:

[https://circleci.com/gh/juiceblender/cassandra/76] (trunk)
 [https://circleci.com/gh/juiceblender/cassandra/77] (3.X)
 [https://circleci.com/gh/juiceblender/cassandra/78] (3.0)
 [https://circleci.com/gh/juiceblender/cassandra/79] (2.2)
 [https://circleci.com/gh/juiceblender/cassandra/80] (2.1)

I get the feeling some of the CCIs may fail (to my knowledge they currently 
don't work on 3.X and 3.0, not sure about 2.Xs). 


was (Author: lerh low):
Github branch if preferred:

[https://github.com/juiceblender/cassandra/tree/jackson-update] 
[https://github.com/juiceblender/cassandra/tree/jackson-update-3.X
https://github.com/juiceblender/cassandra/tree/jackson-update-3.0
https://github.com/juiceblender/cassandra/tree/jackson-update-2.2
https://github.com/juiceblender/cassandra/tree/jackson-update-2.1|https://github.com/juiceblender/cassandra/tree/jackson-update-3.X]
 
CCI:

[https://circleci.com/gh/juiceblender/cassandra/76] (trunk)
 [https://circleci.com/gh/juiceblender/cassandra/77] (3.X)
 [https://circleci.com/gh/juiceblender/cassandra/78] (3.0)
 [https://circleci.com/gh/juiceblender/cassandra/79] (2.2)
 [https://circleci.com/gh/juiceblender/cassandra/80] (2.1)

I get the feeling some of the CCIs may fail (to my knowledge they currently 
don't work on 3.X and 3.0, not sure about 2.Xs). 

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: 2.1-14427.txt, 2.2-14427.txt, 3.0-14427.txt, 
> 3.X-14427.txt, trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458271#comment-16458271
 ] 

Lerh Chuan Low edited comment on CASSANDRA-14427 at 4/30/18 7:00 AM:
-

Github branch if preferred:

[https://github.com/juiceblender/cassandra/tree/jackson-update] 
[https://github.com/juiceblender/cassandra/tree/jackson-update-3.X]
 [https://github.com/juiceblender/cassandra/tree/jackson-update-3.0]
 [https://github.com/juiceblender/cassandra/tree/jackson-update-2.2]
 [https://github.com/juiceblender/cassandra/tree/jackson-update-2.1]

 
 CCI:

[https://circleci.com/gh/juiceblender/cassandra/76] (trunk)
 [https://circleci.com/gh/juiceblender/cassandra/77] (3.X)
 [https://circleci.com/gh/juiceblender/cassandra/78] (3.0)
 [https://circleci.com/gh/juiceblender/cassandra/79] (2.2)
 [https://circleci.com/gh/juiceblender/cassandra/80] (2.1)

I get the feeling some of the CCIs may fail (to my knowledge they currently 
don't work on 3.X and 3.0, not sure about 2.Xs). 


was (Author: lerh low):
Github branch if preferred:

[https://github.com/juiceblender/cassandra/tree/jackson-update] 
 [https://github.com/juiceblender/cassandra/tree/jackson-update-3.0]
 [https://github.com/juiceblender/cassandra/tree/jackson-update-2.2]
[https://github.com/juiceblender/cassandra/tree/jackson-update-2.1]


  
 CCI:

[https://circleci.com/gh/juiceblender/cassandra/76] (trunk)
 [https://circleci.com/gh/juiceblender/cassandra/77] (3.X)
 [https://circleci.com/gh/juiceblender/cassandra/78] (3.0)
 [https://circleci.com/gh/juiceblender/cassandra/79] (2.2)
 [https://circleci.com/gh/juiceblender/cassandra/80] (2.1)

I get the feeling some of the CCIs may fail (to my knowledge they currently 
don't work on 3.X and 3.0, not sure about 2.Xs). 

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: 2.1-14427.txt, 2.2-14427.txt, 3.0-14427.txt, 
> 3.X-14427.txt, trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458271#comment-16458271
 ] 

Lerh Chuan Low edited comment on CASSANDRA-14427 at 4/30/18 6:58 AM:
-

Github branch if preferred:

[https://github.com/juiceblender/cassandra/tree/jackson-update] 
 [https://github.com/juiceblender/cassandra/tree/jackson-update-3.X
https://github.com/juiceblender/cassandra/tree/jackson-update-3.0
|https://github.com/juiceblender/cassandra/tree/jackson-update-3.X] 
[https://github.com/juiceblender/cassandra/tree/jackson-update-2|https://github.com/juiceblender/cassandra/tree/jackson-update-3.0].2
[https://github.com/juiceblender/cassandra/tree/jackson-update-2|https://github.com/juiceblender/cassandra/tree/jackson-update-3.0].1

CCI:

[https://circleci.com/gh/juiceblender/cassandra/76] (trunk)
[https://circleci.com/gh/juiceblender/cassandra/77] (3.X)
[https://circleci.com/gh/juiceblender/cassandra/78] (3.0)
 [https://circleci.com/gh/juiceblender/cassandra/79] (2.2)
 [https://circleci.com/gh/juiceblender/cassandra/80] (2.1)

I get the feeling some of the CCIs may fail (to my knowledge they currently 
don't work on 3.X and 3.0, not sure about 2.Xs). 


was (Author: lerh low):
Github branch if preferred: 

[https://github.com/juiceblender/cassandra/tree/jackson-update] [
https://github.com/juiceblender/cassandra/tree/jackson-update-3.X|https://github.com/juiceblender/cassandra/tree/jackson-update]

[https://github.com/juiceblender/cassandra/tree/jackson-update-3.0]

[https://github.com/juiceblender/cassandra/tree/jackson-update-2|https://github.com/juiceblender/cassandra/tree/jackson-update-3.0].2

[https://github.com/juiceblender/cassandra/tree/jackson-update-2|https://github.com/juiceblender/cassandra/tree/jackson-update-3.0].1

 

CCI: 

[https://circleci.com/gh/juiceblender/cassandra/76] (trunk)

[https://circleci.com/gh/juiceblender/cassandra/77] (3.X)

[https://circleci.com/gh/juiceblender/cassandra/78] (3.0)
[https://circleci.com/gh/juiceblender/cassandra/79] (2.2)
[https://circleci.com/gh/juiceblender/cassandra/80] (2.1)

I get the feeling some of the CCIs may fail (to my knowledge they currently 
don't work on 3.X and 3.0, not sure about 2.Xs). 

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: 2.1-14427.txt, 2.2-14427.txt, 3.0-14427.txt, 
> 3.X-14427.txt, trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458271#comment-16458271
 ] 

Lerh Chuan Low edited comment on CASSANDRA-14427 at 4/30/18 6:57 AM:
-

Github branch if preferred: 

[https://github.com/juiceblender/cassandra/tree/jackson-update] [
https://github.com/juiceblender/cassandra/tree/jackson-update-3.X|https://github.com/juiceblender/cassandra/tree/jackson-update]

[https://github.com/juiceblender/cassandra/tree/jackson-update-3.0]

[https://github.com/juiceblender/cassandra/tree/jackson-update-2|https://github.com/juiceblender/cassandra/tree/jackson-update-3.0].2

[https://github.com/juiceblender/cassandra/tree/jackson-update-2|https://github.com/juiceblender/cassandra/tree/jackson-update-3.0].1

 

CCI: 

[https://circleci.com/gh/juiceblender/cassandra/76] (trunk)

[https://circleci.com/gh/juiceblender/cassandra/77] (3.X)

[https://circleci.com/gh/juiceblender/cassandra/78] (3.0)
[https://circleci.com/gh/juiceblender/cassandra/79] (2.2)
[https://circleci.com/gh/juiceblender/cassandra/80] (2.1)

I get the feeling some of the CCIs may fail (to my knowledge they currently 
don't work on 3.X and 3.0, not sure about 2.Xs). 


was (Author: lerh low):
Github branch if preferred: 
[https://github.com/juiceblender/cassandra/tree/jackson-update]

CCI: [https://circleci.com/gh/juiceblender/cassandra/76]

Not sure if these should include all the previous versions (I think it should), 
let me know if I'm on the right track + if I should create patches for 
2.1/2.2/3.0/3. Thanks!

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: 2.1-14427.txt, 2.2-14427.txt, 3.0-14427.txt, 
> 3.X-14427.txt, trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-14427:
---
Attachment: 3.X-14427.txt
3.0-14427.txt
2.2-14427.txt
2.1-14427.txt

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: 2.1-14427.txt, 2.2-14427.txt, 3.0-14427.txt, 
> 3.X-14427.txt, trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13698) Reinstate or get rid of unit tests with multiple compaction strategies

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458272#comment-16458272
 ] 

Lerh Chuan Low commented on CASSANDRA-13698:


Gentle prod [~pauloricardomg] (Just a reminder, it's not urgent)

> Reinstate or get rid of unit tests with multiple compaction strategies
> --
>
> Key: CASSANDRA-13698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13698
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Paulo Motta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Attachments: 13698-3.0.txt, 13698-3.11.txt, 13698-trunk.txt
>
>
> At some point there were (anti-)compaction tests with multiple compaction 
> strategy classes, but now it's only tested with {{STCS}}:
> * 
> [AnticompactionTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/AntiCompactionTest.java#L247]
> * 
> [CompactionsTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java#L85]
> We should either reinstate these tests or decide they are not important and 
> remove the unused parameter.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458271#comment-16458271
 ] 

Lerh Chuan Low commented on CASSANDRA-14427:


Github branch if preferred: 
[https://github.com/juiceblender/cassandra/tree/jackson-update]

CCI: [https://circleci.com/gh/juiceblender/cassandra/76]

Not sure if these should include all the previous versions (I think it should), 
let me know if I'm on the right track + if I should create patches for 
2.1/2.2/3.0/3. Thanks!

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-14427:
---
Attachment: trunk-14427.txt

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

Lerh Chuan Low created CASSANDRA-14427:
--

 Summary: Bump jackson version to >= 2.9.5
 Key: CASSANDRA-14427
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
 Project: Cassandra
  Issue Type: Improvement
Reporter: Lerh Chuan Low
Assignee: Lerh Chuan Low


The Jackson being used by Cassandra is really old (1.9.2, and still references 
codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 

There have been a few jackson vulnerabilities recently (mostly around 
deserialization which allows arbitrary code execution)

[https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
 [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
 [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
 [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]

Given that Jackson in Cassandra is really old and seems to be used also for 
reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-10540) RangeAwareCompaction

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16445374#comment-16445374
 ] 

Lerh Chuan Low edited comment on CASSANDRA-10540 at 4/20/18 6:38 AM:
-

Hey Marcus, thanks for getting back so quickly :)

The big motivation for us is repairs...because when vnodes are turned on, every 
SSTable has many vnodes in them...so when a (incremental) repair happens, the 
range it is interested in gets anticompacted out. After that the next range 
gets anticompacted out and so on. RACS solves that big pain. 

Besides making the SSTables per read much nicer, it's like a LCS on steroids. I 
think there are other benefits of maintaining each SSTable to one token 
range...but I can't quite remember any more off the top of my head. 

So I am hoping it doesn't come to grouping the vnodesunless it's a last 
resort. 

Currently it looks like you create a RACS for each of the 
repaired/unrepaired/pending repairs set, and each RACS keeps track of the 
compaction strategies it is in charge of (which are all of the same class). The 
CS instances are lazily initiated (so that's a win right there) until needed. 
It seems to be that the reason why we want so many CS instances is so that each 
of them can keep track of their own SSTables (which all belong to that single 
token range). 

How about if RACS doesn't instantiate the individual CS instances? It keeps 
track of all the SSTables in the CF like other CS instances - just that the 
logic on which SSTables to involve in a compaction is done in RACS. So we can 
make RACS check L0 and if there are none, L1 would involve grouping the 
SSTables by range and then calling the next background task for the 
underlying/wrapped CS instances and submitting them. 

In this way, the downside is calculating the grouping each time we ask for the 
next background task. We could also store it in memory in the form of a 
manifest like in LCS? So an array with the SSTables in each of them - beats 
having 256 instances but we're still going to have a 256 sized array in memory, 
I guess. It just seems so starkingly similar to a LCS restricted to just L0 and 
L1. 

A final thought: Is the memory footprint actually significant enough for us to 
want to not bite the bullet and further group the vnodes because the gains from 
having each SSTable as a single range is a lot, simple is a feature and RACS is 
customizable? 

Please excuse my ignorance if none of those suggestions made sense/worked, 
still not very confident with the code base..
  
(Btw also feel free to let me know if you would like a hand with anything)


was (Author: lerh low):
Hey Marcus, thanks for getting back so quickly :)

The big motivation for us is repairs...because when vnodes are turned on, every 
SSTable has many vnodes in them...so when a (incremental) repair happens, the 
range it is interested in gets anticompacted out. After that the next range 
gets anticompacted out and so on. RACS solves that big pain. 

Besides making the SSTables per read much nicer, it's like a LCS on steroids. I 
think there are other benefits of maintaining each SSTable to one token 
range...but I can't quite remember any more off the top of my head. 

So I am hoping it doesn't come to grouping the vnodesunless it's a last 
resort. 

Currently it looks like you create a RACS for each of the 
repaired/unrepaired/pending repairs set, and each RACS keeps track of the 
compaction strategies it is in charge of (which are all of the same class). The 
CS instances are lazily initiated (so that's a win right there) until needed. 
It seems to be that the reason why we want so many CS instances is so that each 
of them can keep track of their own SSTables (which all belong to that single 
token range). 

How about if RACS doesn't instantiate the individual CS instances? It keeps 
track of all the SSTables in the CF like other CS instances - just that the 
logic on which SSTables to involve in a compaction is done in RACS. So we can 
make RACS check L0 and if there are none, L1 would involve grouping the 
SSTables by range and then calling the next background task for the 
underlying/wrapped CS instances and submitting them. 

In this way, the downside is calculating the grouping each time we ask for the 
next background task. We could also store it in memory in the form of a 
manifest like in LCS? So an array with the SSTables in each of them - beats 
having 256 instances but we're still going to have a 256 sized array in memory, 
I guess. It just seems so starkingly similar to a LCS restricted to just L0 and 
L1. 

A final thought: Is the memory footprint actually significant enough for us to 
want to not bite the bullet and further group the vnodes because the gains from 
having each SSTable as a single range is a lot, simple is a feature and RACS is 
customizable? 

Please excuse my ignorance if 

[jira] [Commented] (CASSANDRA-10540) RangeAwareCompaction

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16445374#comment-16445374
 ] 

Lerh Chuan Low commented on CASSANDRA-10540:


Hey Marcus, thanks for getting back so quickly :)

The big motivation for us is repairs...because when vnodes are turned on, every 
SSTable has many vnodes in them...so when a (incremental) repair happens, the 
range it is interested in gets anticompacted out. After that the next range 
gets anticompacted out and so on. RACS solves that big pain. 

Besides making the SSTables per read much nicer, it's like a LCS on steroids. I 
think there are other benefits of maintaining each SSTable to one token 
range...but I can't quite remember any more off the top of my head. 

So I am hoping it doesn't come to grouping the vnodesunless it's a last 
resort. 

Currently it looks like you create a RACS for each of the 
repaired/unrepaired/pending repairs set, and each RACS keeps track of the 
compaction strategies it is in charge of (which are all of the same class). The 
CS instances are lazily initiated (so that's a win right there) until needed. 
It seems to be that the reason why we want so many CS instances is so that each 
of them can keep track of their own SSTables (which all belong to that single 
token range). 

How about if RACS doesn't instantiate the individual CS instances? It keeps 
track of all the SSTables in the CF like other CS instances - just that the 
logic on which SSTables to involve in a compaction is done in RACS. So we can 
make RACS check L0 and if there are none, L1 would involve grouping the 
SSTables by range and then calling the next background task for the 
underlying/wrapped CS instances and submitting them. 

In this way, the downside is calculating the grouping each time we ask for the 
next background task. We could also store it in memory in the form of a 
manifest like in LCS? So an array with the SSTables in each of them - beats 
having 256 instances but we're still going to have a 256 sized array in memory, 
I guess. It just seems so starkingly similar to a LCS restricted to just L0 and 
L1. 

A final thought: Is the memory footprint actually significant enough for us to 
want to not bite the bullet and further group the vnodes because the gains from 
having each SSTable as a single range is a lot, simple is a feature and RACS is 
customizable? 

Please excuse my ignorance if none of those suggestions made sense/worked, 
still not very confident with the code base..
 

> RangeAwareCompaction
> 
>
> Key: CASSANDRA-10540
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10540
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
>  Labels: compaction, lcs, vnodes
> Fix For: 4.x
>
>
> Broken out from CASSANDRA-6696, we should split sstables based on ranges 
> during compaction.
> Requirements;
> * dont create tiny sstables - keep them bunched together until a single vnode 
> is big enough (configurable how big that is)
> * make it possible to run existing compaction strategies on the per-range 
> sstables
> We should probably add a global compaction strategy parameter that states 
> whether this should be enabled or not.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-10540) RangeAwareCompaction

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16445061#comment-16445061
 ] 

Lerh Chuan Low commented on CASSANDRA-10540:


[~krummas] any chance you're still working on this? (It's been a while). I 
recently updated your branch against trunk and fixed a few merge conflicts: 
[https://github.com/juiceblender/cassandra/tree/10540]

I'm also planning to write more unit tests to help get this in...which is in 
the works at the moment. Feel free to let me know if there's any other 
assistance you could use/need in getting this committed? 

> RangeAwareCompaction
> 
>
> Key: CASSANDRA-10540
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10540
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
>  Labels: compaction, lcs, vnodes
> Fix For: 4.x
>
>
> Broken out from CASSANDRA-6696, we should split sstables based on ranges 
> during compaction.
> Requirements;
> * dont create tiny sstables - keep them bunched together until a single vnode 
> is big enough (configurable how big that is)
> * make it possible to run existing compaction strategies on the per-range 
> sstables
> We should probably add a global compaction strategy parameter that states 
> whether this should be enabled or not.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16443412#comment-16443412
 ] 

Lerh Chuan Low commented on CASSANDRA-8460:
---

Hi [~rustyrazorblade],

Sorry for the delay - usually a little bit tricky to get the setup right and 
also had a few hiccups with trunk. Over the last few weeks we've done 
benchmarking on AWS using 4 different 3 node clusters (each with their own 
dedicated stress box) - a LVM setup, a HDD setup, my code setup, and a SSD 
setup. 

The details are in here: 
[https://docs.google.com/document/d/164qZ3zpG5pm_j4r9yWccmMiZh6XK4LsqnZBP7Iu3gII/edit#|https://docs.google.com/document/d/164qZ3zpG5pm_j4r9yWccmMiZh6XK4LsqnZBP7Iu3gII/edit]

It details the way I've stressed and the way I've setup LVM as also a write 
through. We've done 5 takes and in those cases it doesn't seem like LVM 
performs very well even when compared to the HDD. That said, the archiving code 
is also not as good as SSD, I think it may be related to the partition being 
spread across both the slow and the fast. LVM I know the volume is being used 
(based on cloudwatch), I guess it's kind of unintuitive for me how it may be 
this case because it did really well in the fio benchmark you ran. Would you 
(or anyone, really) like to take a look and give your thoughts? Maybe if the 
test is skewed against LVM and we can tune it better? Much appreciated :) 

> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> 
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>Priority: Major
>  Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data 
> directories where we move the sstables once they are older than 
> max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big 
> spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14362) Bind to correct local address in 4.0 streaming

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16443363#comment-16443363
 ] 

Lerh Chuan Low commented on CASSANDRA-14362:


It's not blocking me because I have that workaround, but thanks for looking 
into it so quickly, much appreciated :) 

> Bind to correct local address in 4.0 streaming
> --
>
> Key: CASSANDRA-14362
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14362
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Fix For: 4.0
>
>
> I'm not sure if anybody has tried using {{trunk}} and starting an actual EC2 
> cluster, but with a 3 node setup that works on 3.11, it fails to work on 
> {{trunk}}. I mistakenly stumbled into it while testing my own code changes. 
> My setup is as follows:
> broadcast_rpc_address: publicIP
> broadcast_address: publicIP
> listen_address: omitted. Ends up as privateIP. 
> Works on 3.11 just fine. 
> On {{trunk}} though, it never works. My node is never able to join the 
> cluster:
> {code}
> Apr 03 05:57:06 ip-10-0-47-122 cassandra[13914]: INFO  [main] 2018-04-03 
> 05:57:05,895 RangeStreamer.java:195 - Bootstrap: range 
> (-128373781239966537,-122439194129870521] exists on 52.88.241.181:7000 for 
> keyspace system_traces
> Apr 03 05:57:06 ip-10-0-47-122 cassandra[13914]: INFO  [main] 2018-04-03 
> 05:57:05,895 RangeStreamer.java:195 - Bootstrap: range 
> (6968670424536541270,6973888347502882935] exists on 52.88.241.181:7000 for 
> keyspace system_traces
> Apr 03 05:57:42 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:57:42,298 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 26215173446ns > 50ns
> Apr 03 05:57:53 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:57:53,035 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 10736485907ns > 50ns
> Apr 03 05:58:30 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:58:30,790 Gossiper.java:814 - Gossip stage has 28 pending 
> tasks; skipping status check (no nodes will be marked down)
> Apr 03 05:58:33 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:58:33,060 Gossiper.java:814 - Gossip stage has 20 pending 
> tasks; skipping status check (no nodes will be marked down)
> Apr 03 06:04:33 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 06:04:33,826 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 400790432954ns > 50ns
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 06:04:49,133 Gossiper.java:814 - Gossip stage has 2 pending tasks; 
> skipping status check (no nodes will be marked down)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: ERROR 
> [StreamConnectionEstablisher:1] 2018-04-03 06:04:49,138 
> StreamSession.java:524 - [Stream #d4cd6420-3703-11e8-a6a5-e51ddc10cfe6] 
> Streaming error occurred on session with peer 52.88.241.181:7000
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: java.io.IOException: failed 
> to connect to 52.88.241.181:7000 (STREAM) for streaming data
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:98)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:57)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.createChannel(NettyStreamingMessageSender.java:183)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.setupControlMessageChannel(NettyStreamingMessageSender.java:165)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.sendMessage(NettyStreamingMessageSender.java:222)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.initialize(NettyStreamingMessageSender.java:146)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:271)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:273)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 

[jira] [Commented] (CASSANDRA-13698) Reinstate or get rid of unit tests with multiple compaction strategies

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438984#comment-16438984
 ] 

Lerh Chuan Low commented on CASSANDRA-13698:


{quote}The patch looks good and the failures look unrelated but I just noticed 
that there are a bunch of other commented-out tests on CompactionsTest like 
testEchoedRow, testRangeTombstones, 
testUncheckedTombstoneSizeTieredCompaction,etc that are like this since 
CASSANDRA-8099. Even though this was not in the original ticket scope, I think 
we should also triage those tests and either remove or restore them. WDYT?
{quote}
Seconded :) Being relatively new to the project it unnerves me about the 
current state of tests.

I've reattached the patches and reverted {{stcs}} for 3.0 and 3.X.

Here are the branches again if they float your boat more:
 
[https://github.com/apache/cassandra/compare/trunk...juiceblender:cassandra-13698]
 
[https://github.com/apache/cassandra/compare/cassandra-3.11...juiceblender:cassandra-13698-3.11]
 
[https://github.com/apache/cassandra/compare/cassandra-3.0...juiceblender:cassandra-13698-3.0]

[https://circleci.com/gh/juiceblender/cassandra/56] (Trunk)
 [https://circleci.com/gh/juiceblender/cassandra/60] (3.0)
 [https://circleci.com/gh/juiceblender/cassandra/61] (3.11)

I get the feeling 3.0 and 3.11 are going to fail again (albeit with failures 
already happening on 3.11.2 and 3.0.16).

Just to draw your attention to a few things (which I'm not overly sure of):
 1. I got rid of {{testEchoedRow}} and {{testCompactionLog}} - it has been a 
long time since we had {{EchoedRow}} in the codebase and we no longer have 
compaction history stored in system tables. 
 2. I changed some of the {{superCFMDs}} to {{standardCFMDs}} because
 - It doesn't look like we use super columns/they have been deprecated for some 
time? The method call seems to create a nameless column "" and also expects a 
{{Map}} type, which for the purposes of the tests isn't required.
 - Basing against trunk, {{superCFMD}} calls {{standardCFMD}} so...
 3. {{testDontPurgeAccidentally}} is a little bit weird. I originally attempted 
to assert {{partition.isEmpty()}}, turns out that is false. According to my 
sstabledump it looks like that 'makes sense', because there is still a cell, 
it's just no longer live.

{code:java}
  {
"partition" : {
  "key" : [ "test1" ],
  "position" : 0
},
"rows" : [
  {
"type" : "row",
"position" : 28,
"clustering" : [ "c" ],
"liveness_info" : { "tstamp" : "1970-01-01T00:00:00.02Z" },
"cells" : [
  { "name" : "val", "deletion_info" : { "local_delete_time" : 
"2018-04-15T23:56:48Z" }
  }
]
  }
]
  }
{code}
So I attempted to check if the tombstone exists instead:
{code:java}
ImmutableBTreePartition partition = 
Util.getOnlyPartitionUnfiltered(Util.cmd(cfs, key).build());
assertTrue(!partition.isEmpty());

RowUpdateBuilder deleteRowBuilder = new RowUpdateBuilder(table, 2, key);
deleteRowBuilder.clustering("c").delete("val");
deleteRowBuilder.build().applyUnsafe();
// Remove key

partition = Util.getOnlyPartitionUnfiltered(Util.cmd(cfs, key).build());

assertTrue(partition.iterator().next().cells().iterator().next().isTombstone());

// Sleep one second so that the removal is indeed purgeable even with 
gcgrace == 0
Thread.sleep(1000);

cfs.forceBlockingFlush();

Collection sstablesAfter = cfs.getLiveSSTables();
Collection toCompact = new ArrayList();
for (SSTableReader sstable : sstablesAfter)
if (!sstablesBefore.contains(sstable))
toCompact.add(sstable);

Util.compact(cfs, toCompact);

partition = Util.getOnlyPartitionUnfiltered(Util.cmd(cfs, key).build());

assertTrue(partition.iterator().next().cells().iterator().next().isTombstone());
 <- This never works for gc_grace = 0
{code}
But it never works for the final assert for the CF with gc_grace = 0. I also 
tried using 
{{partition.getRow(Clustering.make(ByteBufferUtil.bytes("c"))).isEmpty()}}. 
Doesn't work either - it also always fails at that final assert for gc_grace = 
0. So I eventually settled for just checking {{row.size()}}. It seems that, 
when debugging, the partition really does not have any 'cells' in the sense 
that it has only an empty {{BTreeRow}} in it, and nothing else. But having that 
is not enough to be 'isEmpty', it has to have nothing in there at all 
(BTree::EMPTY_LEAF, which is just {{new Object[1];}}.

Not sure if you are able to explain why that is so, but otherwise I think the 
test (as I have it) is fair.

> Reinstate or get rid of unit tests with multiple compaction strategies
> --
>
> Key: CASSANDRA-13698
>   

[jira] [Updated] (CASSANDRA-13698) Reinstate or get rid of unit tests with multiple compaction strategies

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-13698:
---
Attachment: (was: 13698-3.0.txt)

> Reinstate or get rid of unit tests with multiple compaction strategies
> --
>
> Key: CASSANDRA-13698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13698
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Paulo Motta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Attachments: 13698-3.0.txt, 13698-3.11.txt, 13698-trunk.txt
>
>
> At some point there were (anti-)compaction tests with multiple compaction 
> strategy classes, but now it's only tested with {{STCS}}:
> * 
> [AnticompactionTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/AntiCompactionTest.java#L247]
> * 
> [CompactionsTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java#L85]
> We should either reinstate these tests or decide they are not important and 
> remove the unused parameter.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13698) Reinstate or get rid of unit tests with multiple compaction strategies

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-13698:
---
Attachment: (was: 13698-trunk.txt)

> Reinstate or get rid of unit tests with multiple compaction strategies
> --
>
> Key: CASSANDRA-13698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13698
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Paulo Motta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Attachments: 13698-3.0.txt, 13698-3.11.txt, 13698-trunk.txt
>
>
> At some point there were (anti-)compaction tests with multiple compaction 
> strategy classes, but now it's only tested with {{STCS}}:
> * 
> [AnticompactionTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/AntiCompactionTest.java#L247]
> * 
> [CompactionsTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java#L85]
> We should either reinstate these tests or decide they are not important and 
> remove the unused parameter.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13698) Reinstate or get rid of unit tests with multiple compaction strategies

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-13698:
---
Attachment: 13698-3.0.txt
13698-3.11.txt
13698-trunk.txt

> Reinstate or get rid of unit tests with multiple compaction strategies
> --
>
> Key: CASSANDRA-13698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13698
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Paulo Motta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Attachments: 13698-3.0.txt, 13698-3.11.txt, 13698-trunk.txt
>
>
> At some point there were (anti-)compaction tests with multiple compaction 
> strategy classes, but now it's only tested with {{STCS}}:
> * 
> [AnticompactionTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/AntiCompactionTest.java#L247]
> * 
> [CompactionsTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java#L85]
> We should either reinstate these tests or decide they are not important and 
> remove the unused parameter.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13698) Reinstate or get rid of unit tests with multiple compaction strategies

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-13698:
---
Attachment: (was: 13698-3.11.txt)

> Reinstate or get rid of unit tests with multiple compaction strategies
> --
>
> Key: CASSANDRA-13698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13698
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Paulo Motta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Attachments: 13698-3.0.txt, 13698-3.11.txt, 13698-trunk.txt
>
>
> At some point there were (anti-)compaction tests with multiple compaction 
> strategy classes, but now it's only tested with {{STCS}}:
> * 
> [AnticompactionTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/AntiCompactionTest.java#L247]
> * 
> [CompactionsTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java#L85]
> We should either reinstate these tests or decide they are not important and 
> remove the unused parameter.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13698) Reinstate or get rid of unit tests with multiple compaction strategies

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431589#comment-16431589
 ] 

Lerh Chuan Low commented on CASSANDRA-13698:


Yeah, I get the same failures on 3.11.2 and 3.0.16. It looks like those tests 
are already failing to begin with? I'm unsure why because those tests run on my 
local. Do you know if I'm missing something :(

https://circleci.com/gh/instaclustr/cassandra/12#tests/containers/2
https://circleci.com/gh/instaclustr/cassandra/11#tests/containers/2



> Reinstate or get rid of unit tests with multiple compaction strategies
> --
>
> Key: CASSANDRA-13698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13698
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Paulo Motta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Attachments: 13698-3.0.txt, 13698-3.11.txt, 13698-trunk.txt
>
>
> At some point there were (anti-)compaction tests with multiple compaction 
> strategy classes, but now it's only tested with {{STCS}}:
> * 
> [AnticompactionTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/AntiCompactionTest.java#L247]
> * 
> [CompactionsTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java#L85]
> We should either reinstate these tests or decide they are not important and 
> remove the unused parameter.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13698) Reinstate or get rid of unit tests with multiple compaction strategies

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430040#comment-16430040
 ] 

Lerh Chuan Low commented on CASSANDRA-13698:


Hi Paulo Motta, 

My bad. I just submitted it as patch available. If git branches float your boat 
more, here they are:

https://github.com/apache/cassandra/compare/trunk...juiceblender:cassandra-13698
https://github.com/apache/cassandra/compare/cassandra-3.11...juiceblender:cassandra-13698-3.11
https://github.com/apache/cassandra/compare/cassandra-3.0...juiceblender:cassandra-13698-3.0

Just a heads up; my CircleCI unit tests do work on trunk, but I can't for the 
life of me figure out why it doesn't work on 3.0 or 3.X. On 3.X it eventually 
times out, on 3.0 the errors doesn't seem related to my changes, e.g 
https://circleci.com/gh/instaclustr/cassandra/9#tests/containers/2 and that 
test runs perfectly fine on my local. I'm thinking of actually running tests on 
3.11.2 and 3.0.16 to see if there are actually any errors on those tags, that 
may take a while still. 

> Reinstate or get rid of unit tests with multiple compaction strategies
> --
>
> Key: CASSANDRA-13698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13698
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Paulo Motta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Attachments: 13698-3.0.txt, 13698-3.11.txt, 13698-trunk.txt
>
>
> At some point there were (anti-)compaction tests with multiple compaction 
> strategy classes, but now it's only tested with {{STCS}}:
> * 
> [AnticompactionTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/AntiCompactionTest.java#L247]
> * 
> [CompactionsTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java#L85]
> We should either reinstate these tests or decide they are not important and 
> remove the unused parameter.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13698) Reinstate or get rid of unit tests with multiple compaction strategies

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-13698:
---
Status: Patch Available  (was: Open)

> Reinstate or get rid of unit tests with multiple compaction strategies
> --
>
> Key: CASSANDRA-13698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13698
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Paulo Motta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Attachments: 13698-3.0.txt, 13698-3.11.txt, 13698-trunk.txt
>
>
> At some point there were (anti-)compaction tests with multiple compaction 
> strategy classes, but now it's only tested with {{STCS}}:
> * 
> [AnticompactionTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/AntiCompactionTest.java#L247]
> * 
> [CompactionsTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java#L85]
> We should either reinstate these tests or decide they are not important and 
> remove the unused parameter.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13698) Reinstate or get rid of unit tests with multiple compaction strategies

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-13698:
---
Attachment: 13698-3.0.txt
13698-3.11.txt
13698-trunk.txt

> Reinstate or get rid of unit tests with multiple compaction strategies
> --
>
> Key: CASSANDRA-13698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13698
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Paulo Motta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Attachments: 13698-3.0.txt, 13698-3.11.txt, 13698-trunk.txt
>
>
> At some point there were (anti-)compaction tests with multiple compaction 
> strategy classes, but now it's only tested with {{STCS}}:
> * 
> [AnticompactionTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/AntiCompactionTest.java#L247]
> * 
> [CompactionsTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java#L85]
> We should either reinstate these tests or decide they are not important and 
> remove the unused parameter.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-10540) RangeAwareCompaction

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-10540:
---
Description: 
Broken out from CASSANDRA-6696, we should split sstables based on ranges during 
compaction.

Requirements;
* dont create tiny sstables - keep them bunched together until a single vnode 
is big enough (configurable how big that is)
* make it possible to run existing compaction strategies on the per-range 
sstables

We should probably add a global compaction strategy parameter that states 
whether this should be enabled or not.

  was:
{color:red}colored text{color}Broken out from CASSANDRA-6696, we should split 
sstables based on ranges during compaction.

Requirements;
* dont create tiny sstables - keep them bunched together until a single vnode 
is big enough (configurable how big that is)
* make it possible to run existing compaction strategies on the per-range 
sstables

We should probably add a global compaction strategy parameter that states 
whether this should be enabled or not.


> RangeAwareCompaction
> 
>
> Key: CASSANDRA-10540
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10540
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
>  Labels: compaction, lcs, vnodes
> Fix For: 4.x
>
>
> Broken out from CASSANDRA-6696, we should split sstables based on ranges 
> during compaction.
> Requirements;
> * dont create tiny sstables - keep them bunched together until a single vnode 
> is big enough (configurable how big that is)
> * make it possible to run existing compaction strategies on the per-range 
> sstables
> We should probably add a global compaction strategy parameter that states 
> whether this should be enabled or not.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-10540) RangeAwareCompaction

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-10540:
---
Description: 
{color:red}colored text{color}Broken out from CASSANDRA-6696, we should split 
sstables based on ranges during compaction.

Requirements;
* dont create tiny sstables - keep them bunched together until a single vnode 
is big enough (configurable how big that is)
* make it possible to run existing compaction strategies on the per-range 
sstables

We should probably add a global compaction strategy parameter that states 
whether this should be enabled or not.

  was:
Broken out from CASSANDRA-6696, we should split sstables based on ranges during 
compaction.

Requirements;
* dont create tiny sstables - keep them bunched together until a single vnode 
is big enough (configurable how big that is)
* make it possible to run existing compaction strategies on the per-range 
sstables

We should probably add a global compaction strategy parameter that states 
whether this should be enabled or not.


> RangeAwareCompaction
> 
>
> Key: CASSANDRA-10540
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10540
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
>  Labels: compaction, lcs, vnodes
> Fix For: 4.x
>
>
> {color:red}colored text{color}Broken out from CASSANDRA-6696, we should split 
> sstables based on ranges during compaction.
> Requirements;
> * dont create tiny sstables - keep them bunched together until a single vnode 
> is big enough (configurable how big that is)
> * make it possible to run existing compaction strategies on the per-range 
> sstables
> We should probably add a global compaction strategy parameter that states 
> whether this should be enabled or not.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14362) Node unable to join cluster in trunk

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16424869#comment-16424869
 ] 

Lerh Chuan Low edited comment on CASSANDRA-14362 at 4/4/18 2:02 AM:


I'm using EC2Snitch (mostly was just using bare bones to get a cluster 
working). If the commit I made seems like the appropriate fix to you (I may be 
missing something :/ ) then I'm happy to create a proper patch for it. 

(And feel free to LMK if you need any more details)


was (Author: lerh low):
I'm using EC2Snitch (mostly was just using bare bones to get a cluster 
working). If the commit I made seems like the appropriate fix to you (I may be 
missing something :/ ) then I'm happy to create a proper patch for it. 

> Node unable to join cluster in trunk
> 
>
> Key: CASSANDRA-14362
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14362
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Lerh Chuan Low
>Priority: Major
> Fix For: 4.0
>
>
> I'm not sure if anybody has tried using {{trunk}} and starting an actual EC2 
> cluster, but with a 3 node setup that works on 3.11, it fails to work on 
> {{trunk}}. I mistakenly stumbled into it while testing my own code changes. 
> My setup is as follows:
> broadcast_rpc_address: publicIP
> broadcast_address: publicIP
> listen_address: omitted. Ends up as privateIP. 
> Works on 3.11 just fine. 
> On {{trunk}} though, it never works. My node is never able to join the 
> cluster:
> {code}
> Apr 03 05:57:06 ip-10-0-47-122 cassandra[13914]: INFO  [main] 2018-04-03 
> 05:57:05,895 RangeStreamer.java:195 - Bootstrap: range 
> (-128373781239966537,-122439194129870521] exists on 52.88.241.181:7000 for 
> keyspace system_traces
> Apr 03 05:57:06 ip-10-0-47-122 cassandra[13914]: INFO  [main] 2018-04-03 
> 05:57:05,895 RangeStreamer.java:195 - Bootstrap: range 
> (6968670424536541270,6973888347502882935] exists on 52.88.241.181:7000 for 
> keyspace system_traces
> Apr 03 05:57:42 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:57:42,298 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 26215173446ns > 50ns
> Apr 03 05:57:53 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:57:53,035 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 10736485907ns > 50ns
> Apr 03 05:58:30 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:58:30,790 Gossiper.java:814 - Gossip stage has 28 pending 
> tasks; skipping status check (no nodes will be marked down)
> Apr 03 05:58:33 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:58:33,060 Gossiper.java:814 - Gossip stage has 20 pending 
> tasks; skipping status check (no nodes will be marked down)
> Apr 03 06:04:33 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 06:04:33,826 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 400790432954ns > 50ns
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 06:04:49,133 Gossiper.java:814 - Gossip stage has 2 pending tasks; 
> skipping status check (no nodes will be marked down)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: ERROR 
> [StreamConnectionEstablisher:1] 2018-04-03 06:04:49,138 
> StreamSession.java:524 - [Stream #d4cd6420-3703-11e8-a6a5-e51ddc10cfe6] 
> Streaming error occurred on session with peer 52.88.241.181:7000
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: java.io.IOException: failed 
> to connect to 52.88.241.181:7000 (STREAM) for streaming data
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:98)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:57)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.createChannel(NettyStreamingMessageSender.java:183)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.setupControlMessageChannel(NettyStreamingMessageSender.java:165)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.sendMessage(NettyStreamingMessageSender.java:222)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.initialize(NettyStreamingMessageSender.java:146)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> 

[jira] [Commented] (CASSANDRA-14362) Node unable to join cluster in trunk

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16424869#comment-16424869
 ] 

Lerh Chuan Low commented on CASSANDRA-14362:


I'm using EC2Snitch (mostly was just using bare bones to get a cluster 
working). If the commit I made seems like the appropriate fix to you (I may be 
missing something :/ ) then I'm happy to create a proper patch for it. 

> Node unable to join cluster in trunk
> 
>
> Key: CASSANDRA-14362
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14362
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Lerh Chuan Low
>Priority: Major
> Fix For: 4.0
>
>
> I'm not sure if anybody has tried using {{trunk}} and starting an actual EC2 
> cluster, but with a 3 node setup that works on 3.11, it fails to work on 
> {{trunk}}. I mistakenly stumbled into it while testing my own code changes. 
> My setup is as follows:
> broadcast_rpc_address: publicIP
> broadcast_address: publicIP
> listen_address: omitted. Ends up as privateIP. 
> Works on 3.11 just fine. 
> On {{trunk}} though, it never works. My node is never able to join the 
> cluster:
> {code}
> Apr 03 05:57:06 ip-10-0-47-122 cassandra[13914]: INFO  [main] 2018-04-03 
> 05:57:05,895 RangeStreamer.java:195 - Bootstrap: range 
> (-128373781239966537,-122439194129870521] exists on 52.88.241.181:7000 for 
> keyspace system_traces
> Apr 03 05:57:06 ip-10-0-47-122 cassandra[13914]: INFO  [main] 2018-04-03 
> 05:57:05,895 RangeStreamer.java:195 - Bootstrap: range 
> (6968670424536541270,6973888347502882935] exists on 52.88.241.181:7000 for 
> keyspace system_traces
> Apr 03 05:57:42 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:57:42,298 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 26215173446ns > 50ns
> Apr 03 05:57:53 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:57:53,035 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 10736485907ns > 50ns
> Apr 03 05:58:30 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:58:30,790 Gossiper.java:814 - Gossip stage has 28 pending 
> tasks; skipping status check (no nodes will be marked down)
> Apr 03 05:58:33 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:58:33,060 Gossiper.java:814 - Gossip stage has 20 pending 
> tasks; skipping status check (no nodes will be marked down)
> Apr 03 06:04:33 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 06:04:33,826 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 400790432954ns > 50ns
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 06:04:49,133 Gossiper.java:814 - Gossip stage has 2 pending tasks; 
> skipping status check (no nodes will be marked down)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: ERROR 
> [StreamConnectionEstablisher:1] 2018-04-03 06:04:49,138 
> StreamSession.java:524 - [Stream #d4cd6420-3703-11e8-a6a5-e51ddc10cfe6] 
> Streaming error occurred on session with peer 52.88.241.181:7000
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: java.io.IOException: failed 
> to connect to 52.88.241.181:7000 (STREAM) for streaming data
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:98)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:57)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.createChannel(NettyStreamingMessageSender.java:183)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.setupControlMessageChannel(NettyStreamingMessageSender.java:165)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.sendMessage(NettyStreamingMessageSender.java:222)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.initialize(NettyStreamingMessageSender.java:146)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:271)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:273)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> Apr 03 

[jira] [Commented] (CASSANDRA-14362) Node unable to join cluster in trunk

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16424847#comment-16424847
 ] 

Lerh Chuan Low commented on CASSANDRA-14362:


Any thoughts on this [~jasobrown]...? :)

> Node unable to join cluster in trunk
> 
>
> Key: CASSANDRA-14362
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14362
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Lerh Chuan Low
>Priority: Major
> Fix For: 4.0
>
>
> I'm not sure if anybody has tried using {{trunk}} and starting an actual EC2 
> cluster, but with a 3 node setup that works on 3.11, it fails to work on 
> {{trunk}}. I mistakenly stumbled into it while testing my own code changes. 
> My setup is as follows:
> broadcast_rpc_address: publicIP
> broadcast_address: publicIP
> listen_address: omitted. Ends up as privateIP. 
> Works on 3.11 just fine. 
> On {{trunk}} though, it never works. My node is never able to join the 
> cluster:
> {code}
> Apr 03 05:57:06 ip-10-0-47-122 cassandra[13914]: INFO  [main] 2018-04-03 
> 05:57:05,895 RangeStreamer.java:195 - Bootstrap: range 
> (-128373781239966537,-122439194129870521] exists on 52.88.241.181:7000 for 
> keyspace system_traces
> Apr 03 05:57:06 ip-10-0-47-122 cassandra[13914]: INFO  [main] 2018-04-03 
> 05:57:05,895 RangeStreamer.java:195 - Bootstrap: range 
> (6968670424536541270,6973888347502882935] exists on 52.88.241.181:7000 for 
> keyspace system_traces
> Apr 03 05:57:42 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:57:42,298 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 26215173446ns > 50ns
> Apr 03 05:57:53 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:57:53,035 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 10736485907ns > 50ns
> Apr 03 05:58:30 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:58:30,790 Gossiper.java:814 - Gossip stage has 28 pending 
> tasks; skipping status check (no nodes will be marked down)
> Apr 03 05:58:33 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:58:33,060 Gossiper.java:814 - Gossip stage has 20 pending 
> tasks; skipping status check (no nodes will be marked down)
> Apr 03 06:04:33 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 06:04:33,826 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 400790432954ns > 50ns
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 06:04:49,133 Gossiper.java:814 - Gossip stage has 2 pending tasks; 
> skipping status check (no nodes will be marked down)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: ERROR 
> [StreamConnectionEstablisher:1] 2018-04-03 06:04:49,138 
> StreamSession.java:524 - [Stream #d4cd6420-3703-11e8-a6a5-e51ddc10cfe6] 
> Streaming error occurred on session with peer 52.88.241.181:7000
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: java.io.IOException: failed 
> to connect to 52.88.241.181:7000 (STREAM) for streaming data
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:98)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:57)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.createChannel(NettyStreamingMessageSender.java:183)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.setupControlMessageChannel(NettyStreamingMessageSender.java:165)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.sendMessage(NettyStreamingMessageSender.java:222)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.initialize(NettyStreamingMessageSender.java:146)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:271)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:273)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]:  

[jira] [Created] (CASSANDRA-14362) Node unable to join cluster in trunk

[Lerh Chuan Low (JIRA)]

Lerh Chuan Low created CASSANDRA-14362:
--

 Summary: Node unable to join cluster in trunk
 Key: CASSANDRA-14362
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14362
 Project: Cassandra
  Issue Type: Bug
Reporter: Lerh Chuan Low
 Fix For: 4.0


I'm not sure if anybody has tried using {{trunk}} and starting an actual EC2 
cluster, but with a 3 node setup that works on 3.11, it fails to work on 
{{trunk}}. I mistakenly stumbled into it while testing my own code changes. 

My setup is as follows:

broadcast_rpc_address: publicIP
broadcast_address: publicIP
listen_address: omitted. Ends up as privateIP. 

Works on 3.11 just fine. 

On {{trunk}} though, it never works. My node is never able to join the cluster:

{code}
Apr 03 05:57:06 ip-10-0-47-122 cassandra[13914]: INFO  [main] 2018-04-03 
05:57:05,895 RangeStreamer.java:195 - Bootstrap: range 
(-128373781239966537,-122439194129870521] exists on 52.88.241.181:7000 for 
keyspace system_traces
Apr 03 05:57:06 ip-10-0-47-122 cassandra[13914]: INFO  [main] 2018-04-03 
05:57:05,895 RangeStreamer.java:195 - Bootstrap: range 
(6968670424536541270,6973888347502882935] exists on 52.88.241.181:7000 for 
keyspace system_traces
Apr 03 05:57:42 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
2018-04-03 05:57:42,298 FailureDetector.java:324 - Not marking nodes down due 
to local pause of 26215173446ns > 50ns
Apr 03 05:57:53 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
2018-04-03 05:57:53,035 FailureDetector.java:324 - Not marking nodes down due 
to local pause of 10736485907ns > 50ns
Apr 03 05:58:30 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
2018-04-03 05:58:30,790 Gossiper.java:814 - Gossip stage has 28 pending tasks; 
skipping status check (no nodes will be marked down)
Apr 03 05:58:33 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
2018-04-03 05:58:33,060 Gossiper.java:814 - Gossip stage has 20 pending tasks; 
skipping status check (no nodes will be marked down)
Apr 03 06:04:33 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
2018-04-03 06:04:33,826 FailureDetector.java:324 - Not marking nodes down due 
to local pause of 400790432954ns > 50ns
Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
2018-04-03 06:04:49,133 Gossiper.java:814 - Gossip stage has 2 pending tasks; 
skipping status check (no nodes will be marked down)
Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: ERROR 
[StreamConnectionEstablisher:1] 2018-04-03 06:04:49,138 StreamSession.java:524 
- [Stream #d4cd6420-3703-11e8-a6a5-e51ddc10cfe6] Streaming error occurred on 
session with peer 52.88.241.181:7000
Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: java.io.IOException: failed to 
connect to 52.88.241.181:7000 (STREAM) for streaming data
Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:98)
Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:57)
Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
org.apache.cassandra.streaming.async.NettyStreamingMessageSender.createChannel(NettyStreamingMessageSender.java:183)
Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
org.apache.cassandra.streaming.async.NettyStreamingMessageSender.setupControlMessageChannel(NettyStreamingMessageSender.java:165)
Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
org.apache.cassandra.streaming.async.NettyStreamingMessageSender.sendMessage(NettyStreamingMessageSender.java:222)
Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
org.apache.cassandra.streaming.async.NettyStreamingMessageSender.initialize(NettyStreamingMessageSender.java:146)
Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:271)
Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:273)
Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
java.lang.Thread.run(Thread.java:748)
Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: Caused by: 
io.netty.channel.unix.Errors$NativeIoException: bind(..) failed: Cannot assign 
requested address
Apr 03 

[jira] [Commented] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413369#comment-16413369
 ] 

Lerh Chuan Low commented on CASSANDRA-8460:
---

https://github.com/apache/cassandra/compare/trunk...juiceblender:cassandra-8460-single-csm

Bump! Asking for kind souls to review/have a look at what I have at the moment 
- the branch has now been updated enough (also with tests) that should reflect 
a few of the features I have decided on. I've also tried to make archiving as 
non-invasive as possible but given the code organization (and also thinking 
about it intuitively, it sort of makes sense) some parts of archiving had to be 
known to superclasses, such as {{CompactionTask}} or {{CompactionAwareWriter}} 
being aware there really are 2 different directories; one for the hot and one 
for the cold. 

Highlights: 
- New enumeration {{DirectoryType}}. Can be either {{STANDARD}} or {{ARCHIVE}}. 
- The decision on whether or not something should be archived is made in 
{{TimeWindowCompactionStrategy}}. The decisions can be:
* If somebody turns off archiving, candidates always gets put into standard
* If candidates are already in archive, put them back into archiving
* Otherwise, do the standard check: Are their age past the archiving time? 
- CSM is aware that there is such a thing as {{DirectoryType}}. It keeps a 
running CompactionStrategy instance in every single directory; both for archive 
and standard. 
- People can turn off archiving at any time by turning off the archiving flags 
in TWCS options. If they choose to do so, any archived SSTables, if compacted, 
will be moved back to the standard directories. Otherwise, they stay in 
archive. (Maybe I could write a nodetool to move archive back to standard)
- If people turn on archiving, when SSTables are next compacted they are moved 
to archive. 

Also included comments in various places to try to say what I am trying to do. 

Finally, if you don't have time to look through the code, please at least look 
through just this file: {{ArchivingCompactionTest}}. All the methods have long 
names describing a feature of this archiving compaction, and please let me know 
if you disagree with any of them. 

There's still a lot left - dtests, Scrubber + I don't know anything about how 
it works with Repair etc, it needs some functional testing. Potentially also 
separate compaction executor and metrics and concurrent compactors. 

Thanks!

Btw, any luck, Jon? I think I may look at writing some terraform scripts to 
spin up Cassandra on Debian, which may be useful for you. 

> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> 
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>Priority: Major
>  Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data 
> directories where we move the sstables once they are older than 
> max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big 
> spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396263#comment-16396263
 ] 

Lerh Chuan Low commented on CASSANDRA-8460:
---

Read you loud and clear for stress. I favoured stress because it reports on 
operation and latency rates and I didn't want to dig through the code just yet 
on what metric exactly stress reports, rather just trusting it as it's the 
default cassandra ships with. I do have a custom Java class for doing inserts 
and reads (but it doesn't do much beyond that), let me know if you would like 
it...? I am also curious what metric you think would be an accurate measure, 
off the top of my head from the client side I can think of time from executing 
the query to when I receive the answer, but I'm not sure if I could make the 
case that the cached version is better compared to the uncached version just 
based off that (the alternative is dig through the stress code to look for 
more). 

I am not very familiar with the low level things in Linux so helping with FIO 
will be really appreciated (Doesn't help that my nodes actually run on CoreOS). 
I relied on Cloudwatch to verify that my cache is working. When I have time in 
the coming days I may write a python script to model TWCS (if you hadn't got to 
it then), I agree it should be modeled with TWCS which is why I was trying to 
make stress look like it. That said, it was interesting to see if it helped the 
other compaction strategies :) 

> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> 
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>Priority: Major
>  Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data 
> directories where we move the sstables once they are older than 
> max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big 
> spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392035#comment-16392035
 ] 

Lerh Chuan Low edited comment on CASSANDRA-8460 at 3/8/18 10:45 PM:


Sure. In the commands below {{/dev/md0}} is my RAID and {{/dev/xvdf}} is my SSD 
volume. 

{code}
sudo pvcreate /dev/md0 
sudo vgcreate VolGroupArray /dev/md0 /dev/xvdf 
sudo lvcreate -n SadOldCache -L 99900M VolGroupArray /dev/xvdf
sudo lvcreate -n SadOldCacheMeta -L 100M VolGroupArray /dev/xvdf
sudo lvconvert --type cache-pool --poolmetadata VolGroupArray/SadOldCacheMeta 
VolGroupArray/SadOldCache
sudo lvcreate -l 100%FREE -n RaidHDD VolGroupArray /dev/md0
sudo lvs -a vg
sudo lvs -a VolGroupArray
sudo lvconvert --type cache --cachepool VolGroupArray/SadOldCache 
VolGroupArray/RaidHDD
sudo vgchange -ay
{code}

Regarding using a TWCS workload, C* stress doesn't play very well with 
timestamps and TWCS in general unless I'm missing something. I've currently got 
to a point of using UDFs (minutesAgo is my UDF) because arithmetic operators 
are not supported until 4.0, and using a query spec something along those lines

{code}
queries:
  putindata:
cql: insert into twcstest (id, time, metric) VALUES (?, toTimestamp(now()), 
?)
  simple1:
cql: select * from twcstest where id = ? and time <= toTimestamp(now()) and 
time >= minutesAgo(5) LIMIT 288
fields: samerow 
{code}

I would have thought though, that using a very pointy Gaussian such as this 
{{gaussian(1..500M, 25000, 1000)}} would be a fair enough reflection of the 
right load because it will still predominantly select the same data due to an 
extremely low stdev of 1000 (Mean of 250M, stdev of 1000). I can re-run the 
benchmarks just to get more samples. 



was (Author: lerh low):
Sure. In the commands below {{/dev/md0}} is my RAID and {{/dev/xvdf}} is my SSD 
volume. 

{code}
sudo pvcreate /dev/md0 
sudo vgcreate VolGroupArray /dev/md0 /dev/xvdf 
sudo lvcreate -n SadOldCache -L 99900M VolGroupArray /dev/xvdf
sudo lvcreate -n SadOldCacheMeta -L 100M VolGroupArray /dev/xvdf
sudo lvconvert --type cache-pool --poolmetadata VolGroupArray/SadOldCacheMeta 
VolGroupArray/SadOldCache
sudo lvcreate -l 100%FREE -n RaidHDD VolGroupArray /dev/md0
sudo lvs -a vg
sudo lvs -a VolGroupArray
sudo lvconvert --type cache --cachepool VolGroupArray/SadOldCache 
VolGroupArray/RaidHDD
sudo vgchange -ay
{code}



> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> 
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>Priority: Major
>  Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data 
> directories where we move the sstables once they are older than 
> max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big 
> spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392035#comment-16392035
 ] 

Lerh Chuan Low commented on CASSANDRA-8460:
---

Sure. In the commands below {{/dev/md0}} is my RAID and {{/dev/xvdf}} is my SSD 
volume. 

{code}
sudo pvcreate /dev/md0 
sudo vgcreate VolGroupArray /dev/md0 /dev/xvdf 
sudo lvcreate -n SadOldCache -L 99900M VolGroupArray /dev/xvdf
sudo lvcreate -n SadOldCacheMeta -L 100M VolGroupArray /dev/xvdf
sudo lvconvert --type cache-pool --poolmetadata VolGroupArray/SadOldCacheMeta 
VolGroupArray/SadOldCache
sudo lvcreate -l 100%FREE -n RaidHDD VolGroupArray /dev/md0
sudo lvs -a vg
sudo lvs -a VolGroupArray
sudo lvconvert --type cache --cachepool VolGroupArray/SadOldCache 
VolGroupArray/RaidHDD
sudo vgchange -ay
{code}



> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> 
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>Priority: Major
>  Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data 
> directories where we move the sstables once they are older than 
> max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big 
> spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392030#comment-16392030
 ] 

Lerh Chuan Low commented on CASSANDRA-8460:
---

On other thoughts, maybe it isn't ideal to bundle it together with compactions 
and make a totally new {{Archiving}} Operation type.

> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> 
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>Priority: Major
>  Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data 
> directories where we move the sstables once they are older than 
> max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big 
> spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392030#comment-16392030
 ] 

Lerh Chuan Low edited comment on CASSANDRA-8460 at 3/8/18 10:35 PM:


On other thoughts, maybe it isn't ideal to bundle it together with compactions 
and make a totally new {{Archiving}} OperationType.


was (Author: lerh low):
On other thoughts, maybe it isn't ideal to bundle it together with compactions 
and make a totally new {{Archiving}} Operation type.

> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> 
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>Priority: Major
>  Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data 
> directories where we move the sstables once they are older than 
> max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big 
> spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392001#comment-16392001
 ] 

Lerh Chuan Low commented on CASSANDRA-8460:
---

[~rustyrazorblade], 

About {{disk_failure_policy}}, that I am not aware so I'll have to look into 
it. With my current patch up to where it is now, there are also other things 
like Scrubber, streaming which I have yet to get to. Thanks for the heads up! 
If such an implication is necessary then maybe we will have to enforce it in 
the code. 

About LVM Cache, I spent some time following the man page and trying it out 
with Cassandra stress. I had spun up a few EC2 clusters. They were all using a 
raid array of 800GB each; one was SSD backed, another was magnetic HDD backed, 
and the final one was magnetic HDD backed with 100GB of LVM writethrough cache. 
I inserted ~200GB of data using cassandra-stress, waited for compactions to 
finish and then attempted a mixed (random) workload...the LVM performed even 
worse than HDD. I guess this was to be expected because the cache works best 
for hot data that is frequently read. 

I did briefly attempt a mixed workload where the queries are always trying to 
select the same data as much as possible (so {{gaussian(1..500M, 25000, 
1000)}}), and there wasn't any noticeable difference between the LVM and HDD 
backed cluster. 

Not sure if you have used LVMCache with a workload before that worked out for 
you and you'd be willing to share details about it...? 

Just thinking about it further, the cache is also very slightly different than 
the original proposal. The cache duplicates the data; making Cassandra 
understand archiving does not. There's also a slight bonus at least from the 
scenario for AWS, the cache consumes the IOPS of the volumes due to the 
duplication (or amplifying) of read and writes back and from the cache.

Any thoughts? (And thank you for your input once again :)) My clusters are 
still running so happy to try a few configurations if you have any to suggest, 
for now I'm just going to refresh myself on the code and look into getting it 
more presentable if someone else swings by and is willing to give their 
thoughts. 

> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> 
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>Priority: Major
>  Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data 
> directories where we move the sstables once they are older than 
> max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big 
> spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373922#comment-16373922
 ] 

Lerh Chuan Low commented on CASSANDRA-8460:
---

Hi Jon, 

Thanks for pitching in :)

Maybe you have stumbled upon the case where data has been resurrected in JBOD 
configuration in your experiences...? In theory since splitting by token range 
there should be no more such cases. It is safe. 

That said, I have not heard of lvmcache so I'll go and have a look at it. I do 
agree that this as it is introduces a lot of code branches and complexity and 
simple is a feature, which is why I was seeking feedback and becoming 
wary...this sounds good - readily available, works for every CS and doesn't 
introduce all that complexity. I'll test it. 

> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> 
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>Priority: Major
>  Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data 
> directories where we move the sstables once they are older than 
> max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big 
> spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357936#comment-16357936
 ] 

Lerh Chuan Low commented on CASSANDRA-8460:
---

Bump! 

The branch has the latest updates I have (I've decided to go with a single CSM: 
https://github.com/apache/cassandra/compare/trunk...juiceblender:cassandra-8460-single-csm).
 I'm currently working my way through the unit tests, the weird thing is when 
run in isolation they work, when run together they fail, as if it's not being 
cleaned up properly. 

So far all the tests should work (as far as I can tell compared to 3.11) and I 
still have yet to add some tests for the archiving compaction. There are 
definitely a lot more things that require checking and I also haven't gotten 
round to checking what happens when you turn it off, etc. Just trying to get a 
lot of the compaction infrastructure to be aware that an archive directory 
exists and there's existing logic to actually perform the archiving compaction. 
I've also yet to test whether it's able to pick up compactions in the archive 
directory when there legitly exists compactions to be done in that directory. 

As before, comments welcome on this is going down the right path or not/is 
there a better way to do it.

> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> 
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>Priority: Major
>  Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data 
> directories where we move the sstables once they are older than 
> max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big 
> spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346255#comment-16346255
 ] 

Lerh Chuan Low edited comment on CASSANDRA-8460 at 2/6/18 2:07 AM:
---

I've tentatively started work on this, and it's turning out to be a relatively 
bigger code change than I was originally expecting, so would really love to get 
some feedback from the community who knows more (and review my initial 
patches). 

{{CompactionAwareWriter}}, {{DiskBoundaryManager}}, {{Directories}} and 
{{CompactionStrategyManager}} needs to know about archives. I've gone ahead and 
created a new Enumeration for `DirectoryType` that can be either ARCHIVE or 
STANDARD. 

{{CompactionAwareWriter}} always calls {{maybeSwitchWriter(Decorated Key)}} 
before calling {{realAppend}}. This is to handle the JBOD case, 
{{maybeSwitchWriter}} helps the writer write to the right location depending on 
the key to make sure keys do not overlap across directories. So it needs to 
have knowledge on which {{diskBoundaries}} it is actually using so as not to 
get into the situation where it can't differentiate between an actual archive 
disk and an actual JBOD disk. 

It would be wise to re-use the logic in {{diskBoundaries}} to also handle the 
case when the archive directory has been configured as JBOD, so 
{{DiskBoundaryManager}} now also needs to know about archive directories. When 
it tries to {{getWriteableLocations}} or generate disk boundaries, it should be 
able to differentiate between archive and non-archive. 

The same goes for {{CompactionStrategyManager}}. We still need to be able to 
run separate compaction strategy instances in the archive directory to handle 
the case of repairs and streaming (so archived SSTables don't just accumulate 
indefinitely). Here's where I am not sure which way to proceed forward. 

Option 1: 
Have it so that {{ColumnFamilyStore}} still only maintains one CSM and DBM and 
one {{Directories}}. CSM, DBM and {{Directories}} all start knowing about the 
existence of an archive directory; this can either be an extra field, or an 
EnumMap:

{code}
new EnumMap(Directories.DirectoryType.class){{
put(Directories.DirectoryType.STANDARD, 
cfs.getDiskBoundaries(Directories.DirectoryType.STANDARD));
put(Directories.DirectoryType.ARCHIVE, 
cfs.getDiskBoundaries(Directories.DirectoryType.ARCHIVE));
}}
{code}

The worry here for me is that some things may subtly break even as I fix up 
everything else that gets logged as errors...The CSM's own internal fields of 
{{repaired}}, {{unrepaired}} and {{pendingRepaired}} will also need to become 
maps, otherwise the individual instances will again become confused, being 
unable to differentiate between an actual JBOD disk or an archive disk. Some of 
the APIs, e.g reload, shutdown, enable etc will all need some smarts on which 
directory type is needed (in some cases it won't matter). Every consumer of 
these APIs will also need to be updated. 

Here's how it looks like in an initial go: 
https://github.com/apache/cassandra/compare/trunk...juiceblender:cassandra-8460-single-csm?expand=1

Option 2:
Have it so that {{ColumnFamilyStore}} keeps 2 CSMs and 2 DBMs, of which the 
archiving equivalents are {{null}} if not applicable/reloaded. In this case 
there's a reasonable level of confidence that each CSM and BDM will just 'do 
the right thing', regardless whether it's an archive or not. In this case then 
every call to getting DBM or CSM (and there are a lot for getting CSM) will 
need to be evaluated and checked. 

Here's how it looks like in an initial go: 
https://github.com/apache/cassandra/compare/trunk...juiceblender:cassandra-8460?expand=1

Both still have work on them (Scrubber, relocate SSTables, what happens when 
the archiving is turned off etc), but before I continue down the track, just 
wondering if anyone can point out which way is better/this is all misguided and 
, in the event this are the changes that need to happen (I can't seem to find a 
way for just TWCS to be aware that there's an archive directory, CFS needs to 
know as well), is this still worth the complexity introduced? 

[~pavel.trukhanov] Re "Why can't we simply allow a CS instance to spread across 
two disks - SSD
and corresponding archival HDD" -> I think in this case you're back in the 
situation where you can have data resurrected. You can have other replicas 
compact away tombstones (because the CS can see both directories) and then have 
your last remaining replica, before it manages to, get its SSD with the 
tombstone corrupted. Upon replacing the SSD with a new one and issuing repair, 
the tombstone is resurrected. Of course, this can be mitigated by making it 
clear to operators that every time there's a corrupt disk, every single disk 
needs to be replaced. 

Even if we did so, there 

[jira] [Commented] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346255#comment-16346255
 ] 

Lerh Chuan Low commented on CASSANDRA-8460:
---

I've tentatively started work on this, and it's turning out to be a relatively 
bigger code change than I was originally expecting, so would really love to get 
some feedback from the community who knows more (and review my initial 
patches). 

{{CompactionAwareWriter}}, {{DiskBoundaryManager}}, {{Directories}} and 
{{CompactionStrategyManager}} needs to know about archives. I've gone ahead and 
created a new Enumeration for `DirectoryType` that can be either ARCHIVE or 
STANDARD. 

{{CompactionAwareWriter}} always calls {{maybeSwitchWriter(Decorated Key)}} 
before calling {{realAppend}}. This is to handle the JBOD case, 
{{maybeSwitchWriter}} helps the writer write to the right location depending on 
the key to make sure keys do not overlap across directories. So it needs to 
have knowledge on which {{diskBoundaries}} it is actually using so as not to 
get into the situation where it can't differentiate between an actual archive 
disk and an actual JBOD disk. 

It would be wise to re-use the logic in {{diskBoundaries}} to also handle the 
case when the archive directory has been configured as JBOD, so 
{{DiskBoundaryManager}} now also needs to know about archive directories. When 
it tries to {{getWriteableLocations}} or generate disk boundaries, it should be 
able to differentiate between archive and non-archive. 

The same goes for {{CompactionStrategyManager}}. We still need to be able to 
run separate compaction strategy instances in the archive directory to handle 
the case of repairs and streaming (so archived SSTables don't just accumulate 
indefinitely). Here's where I am not sure which way to proceed forward. 

Option 1: 
Have it so that {{ColumnFamilyStore}} still only maintains one CSM and DBM and 
one {{Directories}}. CSM, DBM and {{Directories}} all start knowing about the 
existence of an archive directory; this can either be an extra field, or an 
EnumMap:

{code}
new EnumMap(Directories.DirectoryType.class){{
put(Directories.DirectoryType.STANDARD, 
cfs.getDiskBoundaries(Directories.DirectoryType.STANDARD));
put(Directories.DirectoryType.ARCHIVE, 
cfs.getDiskBoundaries(Directories.DirectoryType.ARCHIVE));
}}
{code}

The worry here for me is that some things may subtly break even as I fix up 
everything else that gets logged as errors...The CSM's own internal fields of 
{{repaired}}, {{unrepaired}} and {{pendingRepaired}} will also need to become 
maps, otherwise the individual instances will again become confused, being 
unable to differentiate between an actual JBOD disk or an archive disk. Some of 
the APIs, e.g reload, shutdown, enable etc will all need some smarts on which 
directory type is needed (in some cases it won't matter). Every consumer of 
these APIs will also need to be updated. 

Here's how it looks like in an initial go: 
https://github.com/apache/cassandra/compare/trunk...juiceblender:cassandra-8460?expand=1

Option 2:
Have it so that {{ColumnFamilyStore}} keeps 2 CSMs and 2 DBMs, of which the 
archiving equivalents are {{null}} if not applicable/reloaded. In this case 
there's a reasonable level of confidence that each CSM and BDM will just 'do 
the right thing', regardless whether it's an archive or not. In this case then 
every call to getting DBM or CSM (and there are a lot for getting CSM) will 
need to be evaluated and checked. 

Here's how it looks like in an initial go: 
https://github.com/apache/cassandra/compare/trunk...juiceblender:cassandra-8460-single-csm?expand=1

Both still have work on them (Scrubber, relocate SSTables, what happens when 
the archiving is turned off etc), but before I continue down the track, just 
wondering if anyone can point out which way is better/this is all misguided and 
, in the event this are the changes that need to happen (I can't seem to find a 
way for just TWCS to be aware that there's an archive directory, CFS needs to 
know as well), is this still worth the complexity introduced? 

[~pavel.trukhanov] Re "Why can't we simply allow a CS instance to spread across 
two disks - SSD
and corresponding archival HDD" -> I think in this case you're back in the 
situation where you can have data resurrected. You can have other replicas 
compact away tombstones (because the CS can see both directories) and then have 
your last remaining replica, before it manages to, get its SSD with the 
tombstone corrupted. Upon replacing the SSD with a new one and issuing repair, 
the tombstone is resurrected. Of course, this can be mitigated by making it 
clear to operators that every time there's a corrupt disk, every single disk 
needs to be replaced. 

Even if we did so, there will still be large code changes to make CSM 

[jira] [Commented] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16336502#comment-16336502
 ] 

Lerh Chuan Low commented on CASSANDRA-8460:
---

[~pavel.trukhanov] That's a really good question. I can't think of any reason 
why other than it just being a relic of my thoughts from JBOD/making sure 
unrepaired/repaired/pending repaired SSTables stay in different disks...so if 
the user wanted to replace just the cold archive disk they could do so. Though 
I'm not sure if having a separate CS actually allows that. Hmm...

I guess it may become clearer to me as I dive into the code, but thank you for 
pointing it out :)

> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> 
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Priority: Major
>  Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data 
> directories where we move the sstables once they are older than 
> max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big 
> spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333894#comment-16333894
 ] 

Lerh Chuan Low commented on CASSANDRA-8460:
---

Thinking about this further, looks like this will be (reasonably) complex. 

The main issue is that by introducing an archival directory, we now have 
multiple data directories, which is like a JBOD setup. 
https://issues.apache.org/jira/browse/CASSANDRA-6696 (Partition SSTables by 
token range) seeks to prevent resurrected tombstones - the scenario where you 
can have resurrected tombstones is described here: 
https://www.datastax.com/dev/blog/improving-jbod. 

However, with an archiving directory, we can no longer guarantee that a single 
token range (or vnode) will live in one directory (unless I'm missing 
something. Archiving is based on SSTable age; it doesn't know anything about 
tokens)

High levelly, the situation goes like this: 

1. You have a SSD and a HDD. 
2. Key x is written into the SSD. 
3. After some time, x passes the archive days, and ends up in the HDD. 
4. For some reason not quite clear, the user decides to write a tombstone for x 
(They shouldn't for TWCS). So we now have tomb(x) in the SSD. 

At this point, we must keep in mind that there are 3 separate 
{{CompactionStrategy}} (CS) instances running in both the SSD and HDD, each 
managing repaired, unrepaired and pending repair SSTables. So there are 3 in 
the SSD and 3 in the HDD. These CS instances cannot see each other's 
candidates; when considering candidates for compaction, they see only the 
SSTables in their own directories. 

5. It passes gc_grace_second and tomb(x) is compacted away. So now x is 
resurrected. In an actual JBOD setup, this can't happen because a single token 
range or vnode can only live in one directory. This can't be guaranteed with an 
archiving setup. 

We can solve this issue by introducing a new flag. This flag will make it so 
that a tombstone is only dropped if it lives in the archiving directory. 
Enforcing {{gc_grace > archive_days}} is not sufficient because the node can 
always be taken offline or compactions disabled or similar. 

Consider the case where: 

6. The SSD is corrupted and needs to be replaced. In this case, the fix would 
be to replace the entire node, not just the SSD. This is to prevent tombstone 
resurrection but also that the system tables are gone (system tables live in 
the SSD), so a full replace is needed. 

This is the high level design we came up with: 
* In typical TTL use case TTL should always be greater than archive days 
* Introduce a new YAML setting; call it cold_data_directories possibly. This is 
to signal that 'archive' doesn't mean we can just forget it there; compactions 
still need to happen in that directory, for joining nodes, streaming nodes, and 
keeping the disk usage low. 
* An option on TWCS to specify to use cold directory after a certain amount of 
days. 
* Need a new flag to handle the situation described - cannot drop tombstones 
unless it’s in the cold directory. This also has the implication that we can’t 
drop data using tombstones on the non-archived data. Pretty much means we can’t 
use manual deletions on the table and we should only use this when TTLing 
everything, writing once, and we should turn off read repair.
* Need a separate compaction throughput and concurrent compactors setting for 
the cold directory

Caveats with changes to flags/properties:
* Removing cold flag from the yaml means we've lost the data in those 
directories.
* Removing cold flag from table only means data will no longer be archived to 
cold. Existing SSTables in the cold directory should be loaded in; however if 
compacted moved back to hot storage.
* Reducing the archive time on the table will just cause more data to be moved 
to the cold directory.
* Increasing the archive time means existing data that should no longer be 
archived could go back to the live set if compacted, however will stay in cold 
data with no negative impact.
* When promoting data to cold directory need to check that there’s not an 
overlapping SSTable with a max timestamp greater than minimum timestamp, same 
as TWCS expiry.

There will still be significant I/O when it comes to 
compacting/repairing/streaming the SSTables in the cold directory, and it adds 
reasonable complexity to the code base. It's not trivial to reason about 
either, it took 3 hours between me and my colleagues. The only leftover 
question we had was when changing the table level property will Cassandra need 
to be restarted to take effect? Or is there a hook/property checked constantly?

Anybody notice anything we missed or have any thoughts on it so far on the 
feature itself and the value it adds for the complexity introduced (if you have 
time)? Before we go ahead with it. Will be really appreciated! [~krummas] 
[~bdeggleston] [~jjirsa] [~stone] 




> Make it 

[jira] [Comment Edited] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331469#comment-16331469
 ] 

Lerh Chuan Low edited comment on CASSANDRA-8460 at 1/19/18 12:22 AM:
-

Also just bumping this, wondering if you still had plans with it [~jjirsa] or 
[~bdeggleston]? Looks like with the patch you had previously 
(https://github.com/jeffjirsa/cassandra/commit/cc0ab8f733eef63ed0eaea30cc6f471b467c3ec5#diff-f628011a74763c0d0abc369bc8f5762bR126)
 most of the code changes are still applicable. I am willing to give it a go. 

It sounds like we may still be uncertain on how to go about implementing this. 
My original thoughts are with Jeff's, where the archive directories also keep 
an instance of {{XCompactionStrategy}} running for a repaired, unrepaired and 
pending repair set. It will still have to be read and used eventually when 
doing repairs or streaming when adding a new node...so it increasingly looks 
like it will not be ideal to put it into archiving directory and just never 
touch it again, though I'm happy to implement it however people think is better 
because there may be things that are not obvious to me. Flushing won't be aware 
that an archiving directory exists in this case...and will keep flushing to the 
actual {{data_directories}}. Eventually compaction will pick it up and toss it 
into {{archive_data_directories}}, if applicable. 

Just on that though, one thing I am unable to wrap my head around so far is 
whether the archive directory will need to have the same guarantee as a 
multiple data directories setting...so whether a single vnode/token range 
cannot span across it and another directory, and we have to include it when 
distributing token ranges across the multiple directories. 

[~stone] does raise an interesting point though on making it uncoupled from CS 
and using a background periodic task that archives SSTables. I'm guessing in 
this case you would archive based on...SSTable metadata min/max timestamp? Or 
just the last modified of the SSTable files? It will be a YAML property and if 
there is an SSTable with max timestamp behind X days, archive the SSTable? 



was (Author: lerh low):
Also just bumping this, wondering if you still had plans with it [~jjirsa] or 
[~bdeggleston]? Looks like with the patch you had previously 
(https://github.com/jeffjirsa/cassandra/commit/cc0ab8f733eef63ed0eaea30cc6f471b467c3ec5#diff-f628011a74763c0d0abc369bc8f5762bR126)
 most of the code changes are still applicable. I am willing to give it a go. 

It sounds like we may still be uncertain on how to go about implementing this. 
My original thoughts are with Jeff's, where the archive directories also keep 
an instance of {{XCompactionStrategy}} running for a repaired, unrepaired and 
pending repair set. It will still have to be read and used eventually when 
doing repairs or streaming when adding a new node...so it increasingly looks 
like it will not be ideal to put it into archiving directory and just never 
touch it again, though I'm happy to implement it however people think is better 
because there may be things that are not obvious to me. Flushing won't be aware 
that an archiving directory exists in this case...and will keep flushing to the 
actual {{data_directories}}. Eventually compaction will pick it up and toss it 
into {{archive_data_directories}}, if applicable. 

Just on that though, one thing I am unable to wrap my head around so far is 
whether the archive directory will need to have the same guarantee as a 
multiple data directories setting...so whether a single vnode/token range 
cannot span across it and another directory. 

[~stone] does raise an interesting point though on making it uncoupled from CS 
and using a background periodic task that archives SSTables. I'm guessing in 
this case you would archive based on...SSTable metadata min/max timestamp? Or 
just the last modified of the SSTable files? It will be a YAML property and if 
there is an SSTable with max timestamp behind X days, archive the SSTable? 


> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> 
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Priority: Major
>  Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data 
> directories where we move the sstables once they are older than 
> max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big 
> spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA

[jira] [Comment Edited] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331469#comment-16331469
 ] 

Lerh Chuan Low edited comment on CASSANDRA-8460 at 1/19/18 12:21 AM:
-

Also just bumping this, wondering if you still had plans with it [~jjirsa] or 
[~bdeggleston]? Looks like with the patch you had previously 
(https://github.com/jeffjirsa/cassandra/commit/cc0ab8f733eef63ed0eaea30cc6f471b467c3ec5#diff-f628011a74763c0d0abc369bc8f5762bR126)
 most of the code changes are still applicable. I am willing to give it a go. 

It sounds like we may still be uncertain on how to go about implementing this. 
My original thoughts are with Jeff's, where the archive directories also keep 
an instance of {{XCompactionStrategy}} running for a repaired, unrepaired and 
pending repair set. It will still have to be read and used eventually when 
doing repairs or streaming when adding a new node...so it increasingly looks 
like it will not be ideal to put it into archiving directory and just never 
touch it again, though I'm happy to implement it however people think is better 
because there may be things that are not obvious to me. Flushing won't be aware 
that an archiving directory exists in this case...and will keep flushing to the 
actual {{data_directories}}. Eventually compaction will pick it up and toss it 
into {{archive_data_directories}}, if applicable. 

Just on that though, one thing I am unable to wrap my head around so far is 
whether the archive directory will need to have the same guarantee as a 
multiple data directories setting...so whether a single vnode/token range 
cannot span across it and another directory. 

[~stone] does raise an interesting point though on making it uncoupled from CS 
and using a background periodic task that archives SSTables. I'm guessing in 
this case you would archive based on...SSTable metadata min/max timestamp? Or 
just the last modified of the SSTable files? It will be a YAML property and if 
there is an SSTable with max timestamp behind X days, archive the SSTable? 



was (Author: lerh low):
Also just bumping this, wondering if you still had plans with it [~jjirsa] or 
[~bdeggleston]? Looks like with the patch you had previously 
(https://github.com/jeffjirsa/cassandra/commit/cc0ab8f733eef63ed0eaea30cc6f471b467c3ec5#diff-f628011a74763c0d0abc369bc8f5762bR126)
 most of the code changes are still applicable. I am willing to give it a go. 

It sounds like we may still be uncertain on how to go about implementing this. 
My original thoughts are with Jeff's, where the archive directories also keep 
an instance of {{XCompactionStrategy}} running for a repaired, unrepaired and 
pending repair set. It will still have to be read and used eventually when 
doing repairs or streaming when adding a new node...so it increasingly looks 
like it will not be ideal to put it into archiving directory and just never 
touch it again, though I'm happy to implement it however people think is better 
because there may be things that are not obvious to me. Flushing won't be aware 
that an archiving directory exists in this case...and will keep flushing to the 
actual {{data_directories}}. Eventually compaction will pick it up and toss it 
into {{archive_data_directories}}, if applicable. 

[~stone] does raise an interesting point though on making it uncoupled from CS 
and using a background periodic task that archives SSTables. I'm guessing in 
this case you would archive based on...SSTable metadata min/max timestamp? Or 
just the last modified of the SSTable files? It will be a YAML property and if 
there is an SSTable with max timestamp behind X days, archive the SSTable? 


> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> 
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Priority: Major
>  Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data 
> directories where we move the sstables once they are older than 
> max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big 
> spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331469#comment-16331469
 ] 

Lerh Chuan Low commented on CASSANDRA-8460:
---

Also just bumping this, wondering if you still had plans with it [~jjirsa] or 
[~bdeggleston]? Looks like with the patch you had previously 
(https://github.com/jeffjirsa/cassandra/commit/cc0ab8f733eef63ed0eaea30cc6f471b467c3ec5#diff-f628011a74763c0d0abc369bc8f5762bR126)
 most of the code changes are still applicable. I am willing to give it a go. 

It sounds like we may still be uncertain on how to go about implementing this. 
My original thoughts are with Jeff's, where the archive directories also keep 
an instance of {{XCompactionStrategy}} running for a repaired, unrepaired and 
pending repair set. It will still have to be read and used eventually when 
doing repairs or streaming when adding a new node...so it increasingly looks 
like it will not be ideal to put it into archiving directory and just never 
touch it again, though I'm happy to implement it however people think is better 
because there may be things that are not obvious to me. Flushing won't be aware 
that an archiving directory exists in this case...and will keep flushing to the 
actual {{data_directories}}. Eventually compaction will pick it up and toss it 
into {{archive_data_directories}}, if applicable. 

[~stone] does raise an interesting point though on making it uncoupled from CS 
and using a background periodic task that archives SSTables. I'm guessing in 
this case you would archive based on...SSTable metadata min/max timestamp? Or 
just the last modified of the SSTable files? It will be a YAML property and if 
there is an SSTable with max timestamp behind X days, archive the SSTable? 


> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> 
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Priority: Major
>  Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data 
> directories where we move the sstables once they are older than 
> max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big 
> spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13528) nodetool describeclusters shows different snitch info as to what is configured.

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-13528:
---
Attachment: 13528-trunk.txt

> nodetool describeclusters shows different snitch info as to what is 
> configured.
> ---
>
> Key: CASSANDRA-13528
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13528
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Paul Villacorta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Attachments: 13528-trunk.txt, Screen Shot 2017-05-12 at 14.15.04.png
>
>
> I couldn't find any similar issue as this one so I'm creating one.
> I noticed that doing nodetool describecluster shows a different Snitch 
> Information as to what is being set in the configuration file.
> My setup is hosted in AWS and I am using Ec2Snitch.
> cassandra@cassandra3$ nodetool describecluster
> Cluster Information:
>   Name: testv3
>   Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
>   Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
>   Schema versions:
>   fc6e8656-ee7a-341b-9782-b569d1fd1a51: 
> [10.0.3.61,10.0.3.62,10.0.3.63]
> I checked via MX4J and it shows the same, I haven't verified tho using a 
> different Snitch and I am using 2.2.6 above and 3.0.X 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13528) nodetool describeclusters shows different snitch info as to what is configured.

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-13528:
---
Attachment: (was: 13528-trunk.txt)

> nodetool describeclusters shows different snitch info as to what is 
> configured.
> ---
>
> Key: CASSANDRA-13528
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13528
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Paul Villacorta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Attachments: Screen Shot 2017-05-12 at 14.15.04.png
>
>
> I couldn't find any similar issue as this one so I'm creating one.
> I noticed that doing nodetool describecluster shows a different Snitch 
> Information as to what is being set in the configuration file.
> My setup is hosted in AWS and I am using Ec2Snitch.
> cassandra@cassandra3$ nodetool describecluster
> Cluster Information:
>   Name: testv3
>   Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
>   Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
>   Schema versions:
>   fc6e8656-ee7a-341b-9782-b569d1fd1a51: 
> [10.0.3.61,10.0.3.62,10.0.3.63]
> I checked via MX4J and it shows the same, I haven't verified tho using a 
> different Snitch and I am using 2.2.6 above and 3.0.X 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13528) nodetool describeclusters shows different snitch info as to what is configured.

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329643#comment-16329643
 ] 

Lerh Chuan Low commented on CASSANDRA-13528:


Thanks for reviewing Jason :)

Agreed. For some reason back then I was thinking the code would be less elegant 
to support that, but now revisiting it it's still relatively straightforward 
and doesn't need too many hacks. 

So it's now back to: 
{code:java}
Cluster Information:
Name: Test Cluster
Snitch: org.apache.cassandra.locator.SimpleSnitch
DynamicEndPointSnitch: enabled
Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
Schema versions:
59a1610b-0384-337c-a2c5-9c8efaba12be: [127.0.0.1]{code}

And 
{code:java}
Cluster Information:
Name: Test Cluster
Snitch: org.apache.cassandra.locator.SimpleSnitch
DynamicEndPointSnitch: disabled
Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
Schema versions:
59a1610b-0384-337c-a2c5-9c8efaba12be: [127.0.0.1]{code}

Just finished testing this on trunk and 3.11 (patch applies cleanly). Here's 
the Github link: 
[https://github.com/juiceblender/cassandra/commit/e93c219f86469357e01595f3adcea81efd0dec84]
 if that floats your boat more, otherwise the attached patch has also been 
updated. 

I believe it actually applies alright all the way down to 2.1 (so we can just 
apply it to 3.0 if we do wish). But it's up to you, just let me know. 

> nodetool describeclusters shows different snitch info as to what is 
> configured.
> ---
>
> Key: CASSANDRA-13528
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13528
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Paul Villacorta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Attachments: Screen Shot 2017-05-12 at 14.15.04.png
>
>
> I couldn't find any similar issue as this one so I'm creating one.
> I noticed that doing nodetool describecluster shows a different Snitch 
> Information as to what is being set in the configuration file.
> My setup is hosted in AWS and I am using Ec2Snitch.
> cassandra@cassandra3$ nodetool describecluster
> Cluster Information:
>   Name: testv3
>   Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
>   Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
>   Schema versions:
>   fc6e8656-ee7a-341b-9782-b569d1fd1a51: 
> [10.0.3.61,10.0.3.62,10.0.3.63]
> I checked via MX4J and it shows the same, I haven't verified tho using a 
> different Snitch and I am using 2.2.6 above and 3.0.X 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13698) Reinstate or get rid of unit tests with multiple compaction strategies

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323398#comment-16323398
 ] 

Lerh Chuan Low commented on CASSANDRA-13698:


https://github.com/juiceblender/cassandra/commit/35260ed0c49cb886b71ef0b8ffcd2c8d2e82a73d

I've made a tentative commit that works on trunk. 

For {{CompactionsTest}}, what the test method really is testing is that if 
there's a single SSTable with droppable tombstones whether it would get 
compacted away. Regardless of which compaction strategy one chooses, it will 
always do that (all the compaction strategies account for it). It feels like it 
was meant to test that {{worthDroppingTombstones}} really worked in a 
functional test so I don't think we need to parameterize it for each compaction 
strategy, but I'm happy to change it if people think otherwise. 

For {{AntiCompactionTest}} then that code path as far as I can tell pays no 
attention to what compaction strategy the CFS is actually using, so can be 
removed. I did also make one of the tests marked to be ignored 
({{antiCompactionSizeTest}} work again - it looks like all that was missing is 
that the repair session had to be registered with {{ActiveRepairService}} 
first. 

Also did a brief refactor rename of {{scts}} to {{stcs}}. 

Let me know any thoughts - if you're happy I can look at porting these changes 
to branches other than {{trunk}}. 

> Reinstate or get rid of unit tests with multiple compaction strategies
> --
>
> Key: CASSANDRA-13698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13698
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Paulo Motta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
>
> At some point there were (anti-)compaction tests with multiple compaction 
> strategy classes, but now it's only tested with {{STCS}}:
> * 
> [AnticompactionTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/AntiCompactionTest.java#L247]
> * 
> [CompactionsTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java#L85]
> We should either reinstate these tests or decide they are not important and 
> remove the unused parameter.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13528) nodetool describeclusters shows different snitch info as to what is configured.

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323267#comment-16323267
 ] 

Lerh Chuan Low commented on CASSANDRA-13528:


Just wondering if anyone would like to review the patch? :)

> nodetool describeclusters shows different snitch info as to what is 
> configured.
> ---
>
> Key: CASSANDRA-13528
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13528
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Paul Villacorta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Attachments: 13528-trunk.txt, Screen Shot 2017-05-12 at 14.15.04.png
>
>
> I couldn't find any similar issue as this one so I'm creating one.
> I noticed that doing nodetool describecluster shows a different Snitch 
> Information as to what is being set in the configuration file.
> My setup is hosted in AWS and I am using Ec2Snitch.
> cassandra@cassandra3$ nodetool describecluster
> Cluster Information:
>   Name: testv3
>   Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
>   Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
>   Schema versions:
>   fc6e8656-ee7a-341b-9782-b569d1fd1a51: 
> [10.0.3.61,10.0.3.62,10.0.3.63]
> I checked via MX4J and it shows the same, I haven't verified tho using a 
> different Snitch and I am using 2.2.6 above and 3.0.X 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13698) Reinstate or get rid of unit tests with multiple compaction strategies

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281136#comment-16281136
 ] 

Lerh Chuan Low commented on CASSANDRA-13698:


I'll take a look and see. 

> Reinstate or get rid of unit tests with multiple compaction strategies
> --
>
> Key: CASSANDRA-13698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13698
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Paulo Motta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
>
> At some point there were (anti-)compaction tests with multiple compaction 
> strategy classes, but now it's only tested with {{STCS}}:
> * 
> [AnticompactionTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/AntiCompactionTest.java#L247]
> * 
> [CompactionsTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java#L85]
> We should either reinstate these tests or decide they are not important and 
> remove the unused parameter.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-13698) Reinstate or get rid of unit tests with multiple compaction strategies

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low reassigned CASSANDRA-13698:
--

Assignee: Lerh Chuan Low

> Reinstate or get rid of unit tests with multiple compaction strategies
> --
>
> Key: CASSANDRA-13698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13698
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Paulo Motta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
>
> At some point there were (anti-)compaction tests with multiple compaction 
> strategy classes, but now it's only tested with {{STCS}}:
> * 
> [AnticompactionTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/AntiCompactionTest.java#L247]
> * 
> [CompactionsTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java#L85]
> We should either reinstate these tests or decide they are not important and 
> remove the unused parameter.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13528) nodetool describeclusters shows different snitch info as to what is configured.

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278044#comment-16278044
 ] 

Lerh Chuan Low commented on CASSANDRA-13528:


Hello, sorry for taking so long on this, it's ancient by now. I've reuploaded a 
new patch without changing the MBean interfaces, though the UI is slightly 
different now (less code changes required): 

{code}
Cluster Information:
Name: Test Cluster
Snitch: org.apache.cassandra.locator.SimpleSnitch
Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
Schema versions:
59a1610b-0384-337c-a2c5-9c8efaba12be: [127.0.0.1]
{code}

{code}
Cluster Information:
Name: Test Cluster
Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
SubSnitch: org.apache.cassandra.locator.SimpleSnitch
Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
Schema versions:
59a1610b-0384-337c-a2c5-9c8efaba12be: [127.0.0.1]
{code}

Applies cleanly to 2.1/2.2/3.0.X/3.X, and tested as well. 

> nodetool describeclusters shows different snitch info as to what is 
> configured.
> ---
>
> Key: CASSANDRA-13528
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13528
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Paul Villacorta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Attachments: 13528-trunk.txt, Screen Shot 2017-05-12 at 14.15.04.png
>
>
> I couldn't find any similar issue as this one so I'm creating one.
> I noticed that doing nodetool describecluster shows a different Snitch 
> Information as to what is being set in the configuration file.
> My setup is hosted in AWS and I am using Ec2Snitch.
> cassandra@cassandra3$ nodetool describecluster
> Cluster Information:
>   Name: testv3
>   Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
>   Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
>   Schema versions:
>   fc6e8656-ee7a-341b-9782-b569d1fd1a51: 
> [10.0.3.61,10.0.3.62,10.0.3.63]
> I checked via MX4J and it shows the same, I haven't verified tho using a 
> different Snitch and I am using 2.2.6 above and 3.0.X 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13528) nodetool describeclusters shows different snitch info as to what is configured.

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-13528:
---
Attachment: 13528-trunk.txt

> nodetool describeclusters shows different snitch info as to what is 
> configured.
> ---
>
> Key: CASSANDRA-13528
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13528
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Paul Villacorta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Attachments: 13528-trunk.txt, Screen Shot 2017-05-12 at 14.15.04.png
>
>
> I couldn't find any similar issue as this one so I'm creating one.
> I noticed that doing nodetool describecluster shows a different Snitch 
> Information as to what is being set in the configuration file.
> My setup is hosted in AWS and I am using Ec2Snitch.
> cassandra@cassandra3$ nodetool describecluster
> Cluster Information:
>   Name: testv3
>   Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
>   Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
>   Schema versions:
>   fc6e8656-ee7a-341b-9782-b569d1fd1a51: 
> [10.0.3.61,10.0.3.62,10.0.3.63]
> I checked via MX4J and it shows the same, I haven't verified tho using a 
> different Snitch and I am using 2.2.6 above and 3.0.X 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13528) nodetool describeclusters shows different snitch info as to what is configured.

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-13528:
---
Attachment: (was: 13528-trunk.txt)

> nodetool describeclusters shows different snitch info as to what is 
> configured.
> ---
>
> Key: CASSANDRA-13528
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13528
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Paul Villacorta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Attachments: Screen Shot 2017-05-12 at 14.15.04.png
>
>
> I couldn't find any similar issue as this one so I'm creating one.
> I noticed that doing nodetool describecluster shows a different Snitch 
> Information as to what is being set in the configuration file.
> My setup is hosted in AWS and I am using Ec2Snitch.
> cassandra@cassandra3$ nodetool describecluster
> Cluster Information:
>   Name: testv3
>   Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
>   Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
>   Schema versions:
>   fc6e8656-ee7a-341b-9782-b569d1fd1a51: 
> [10.0.3.61,10.0.3.62,10.0.3.63]
> I checked via MX4J and it shows the same, I haven't verified tho using a 
> different Snitch and I am using 2.2.6 above and 3.0.X 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13528) nodetool describeclusters shows different snitch info as to what is configured.

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-13528:
---
Attachment: (was: 13528-2.1.txt)

> nodetool describeclusters shows different snitch info as to what is 
> configured.
> ---
>
> Key: CASSANDRA-13528
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13528
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Paul Villacorta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Attachments: Screen Shot 2017-05-12 at 14.15.04.png
>
>
> I couldn't find any similar issue as this one so I'm creating one.
> I noticed that doing nodetool describecluster shows a different Snitch 
> Information as to what is being set in the configuration file.
> My setup is hosted in AWS and I am using Ec2Snitch.
> cassandra@cassandra3$ nodetool describecluster
> Cluster Information:
>   Name: testv3
>   Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
>   Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
>   Schema versions:
>   fc6e8656-ee7a-341b-9782-b569d1fd1a51: 
> [10.0.3.61,10.0.3.62,10.0.3.63]
> I checked via MX4J and it shows the same, I haven't verified tho using a 
> different Snitch and I am using 2.2.6 above and 3.0.X 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13528) nodetool describeclusters shows different snitch info as to what is configured.

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16051236#comment-16051236
 ] 

Lerh Chuan Low commented on CASSANDRA-13528:


Hell. My bad, I didn't know that existed. I'll look into it, if that already 
exists then definitely we should use it. Thanks for your suggestion. 

> nodetool describeclusters shows different snitch info as to what is 
> configured.
> ---
>
> Key: CASSANDRA-13528
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13528
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Paul Villacorta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Attachments: 13528-2.1.txt, 13528-trunk.txt, Screen Shot 2017-05-12 
> at 14.15.04.png
>
>
> I couldn't find any similar issue as this one so I'm creating one.
> I noticed that doing nodetool describecluster shows a different Snitch 
> Information as to what is being set in the configuration file.
> My setup is hosted in AWS and I am using Ec2Snitch.
> cassandra@cassandra3$ nodetool describecluster
> Cluster Information:
>   Name: testv3
>   Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
>   Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
>   Schema versions:
>   fc6e8656-ee7a-341b-9782-b569d1fd1a51: 
> [10.0.3.61,10.0.3.62,10.0.3.63]
> I checked via MX4J and it shows the same, I haven't verified tho using a 
> different Snitch and I am using 2.2.6 above and 3.0.X 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13346) Failed unregistering mbean during drop keyspace

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16037831#comment-16037831
 ] 

Lerh Chuan Low commented on CASSANDRA-13346:


[~jjirsa] The ones attached to the JIRA :) 

> Failed unregistering mbean during drop keyspace
> ---
>
> Key: CASSANDRA-13346
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13346
> Project: Cassandra
>  Issue Type: Bug
>  Components: Materialized Views
> Environment: Cassandra 3.9
>Reporter: Gábor Auth
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Fix For: 3.10, 3.0.x
>
> Attachments: 13346-3.0.X.txt, 13346-3.X.txt, 13346-trunk.txt
>
>
> All node throw exceptions about materialized views during drop keyspace:
> {code}
> WARN  [MigrationStage:1] 2017-03-16 16:54:25,016 ColumnFamilyStore.java:535 - 
> Failed unregistering mbean: 
> org.apache.cassandra.db:type=Tables,keyspace=test20160810,table=unit_by_account
> java.lang.NullPointerException: null
> at 
> java.util.concurrent.ConcurrentHashMap.replaceNode(ConcurrentHashMap.java:1106)
>  ~[na:1.8.0_121]
> at 
> java.util.concurrent.ConcurrentHashMap.remove(ConcurrentHashMap.java:1097) 
> ~[na:1.8.0_121]
> at 
> java.util.concurrent.ConcurrentHashMap$KeySetView.remove(ConcurrentHashMap.java:4569)
>  ~[na:1.8.0_121]
> at 
> org.apache.cassandra.metrics.TableMetrics.release(TableMetrics.java:712) 
> ~[apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.db.ColumnFamilyStore.unregisterMBean(ColumnFamilyStore.java:570)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.db.ColumnFamilyStore.invalidate(ColumnFamilyStore.java:527)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.db.ColumnFamilyStore.invalidate(ColumnFamilyStore.java:517)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at org.apache.cassandra.db.Keyspace.unloadCf(Keyspace.java:365) 
> [apache-cassandra-3.9.0.jar:3.9.0]
> at org.apache.cassandra.db.Keyspace.dropCf(Keyspace.java:358) 
> [apache-cassandra-3.9.0.jar:3.9.0]
> at org.apache.cassandra.config.Schema.dropView(Schema.java:744) 
> [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.lambda$mergeSchema$373(SchemaKeyspace.java:1287)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_121]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.mergeSchema(SchemaKeyspace.java:1287)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.mergeSchemaAndAnnounceVersion(SchemaKeyspace.java:1256)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:51)
>  ~[apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[apache-cassandra-3.9.0.jar:3.9.0]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_121]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_121]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_121]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  ~[na:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_121]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-13547) Filtered materialized views missing data

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low reassigned CASSANDRA-13547:
--

Assignee: Krishna Dattu Koneru

> Filtered materialized views missing data
> 
>
> Key: CASSANDRA-13547
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13547
> Project: Cassandra
>  Issue Type: Bug
>  Components: Materialized Views
> Environment: Official Cassandra 3.10 Docker image (ID 154b919bf8ce).
>Reporter: Craig Nicholson
>Assignee: Krishna Dattu Koneru
>Priority: Blocker
>  Labels: materializedviews
> Fix For: 3.11.x
>
>
> When creating a materialized view against a base table the materialized view 
> does not always reflect the correct data.
> Using the following test schema:
> {code:title=Schema|language=sql}
> DROP KEYSPACE IF EXISTS test;
> CREATE KEYSPACE test
>   WITH REPLICATION = { 
>'class' : 'SimpleStrategy', 
>'replication_factor' : 1 
>   };
> CREATE TABLE test.table1 (
> id int,
> name text,
> enabled boolean,
> foo text,
> PRIMARY KEY (id, name));
> CREATE MATERIALIZED VIEW test.table1_mv1 AS SELECT id, name, foo
> FROM test.table1
> WHERE id IS NOT NULL 
> AND name IS NOT NULL 
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> CREATE MATERIALIZED VIEW test.table1_mv2 AS SELECT id, name, foo, enabled
> FROM test.table1
> WHERE id IS NOT NULL 
> AND name IS NOT NULL 
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> {code}
> When I insert a row into the base table the materialized views are updated 
> appropriately. (+)
> {code:title=Insert row|language=sql}
> cqlsh> INSERT INTO test.table1 (id, name, enabled, foo) VALUES (1, 'One', 
> TRUE, 'Bar');
> cqlsh> SELECT * FROM test.table1;
>  id | name | enabled | foo
> +--+-+-
>   1 |  One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
>  name | id | foo
> --++-
>   One |  1 | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
>  name | id | enabled | foo
> --++-+-
>   One |  1 |True | Bar
> (1 rows)
> {code}
> Updating the record in the base table and setting enabled to FALSE will 
> filter the record from both materialized views. (+)
> {code:title=Disable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = FALSE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
>  id | name | enabled | foo
> +--+-+-
>   1 |  One |   False | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
>  name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
>  name | id | enabled | foo
> --++-+-
> (0 rows)
> {code}
> However a further update to the base table setting enabled to TRUE should 
> include the record in both materialzed views, however only one view 
> (table1_mv2) gets updated. (-)
> It appears that only the view (table1_mv2) that returns the filtered column 
> (enabled) is updated. (-)
> Additionally columns that are not part of the partiion or clustering key are 
> not updated. You can see that the foo column has a null value in table1_mv2. 
> (-)
> {code:title=Enable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = TRUE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
>  id | name | enabled | foo
> +--+-+-
>   1 |  One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
>  name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
>  name | id | enabled | foo
> --++-+--
>   One |  1 |True | null
> (1 rows)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13068) Fully expired sstable not dropped when running out of disk space

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028817#comment-16028817
 ] 

Lerh Chuan Low commented on CASSANDRA-13068:


Hi [~krummas],

Here's the new branch: 
https://github.com/apache/cassandra/compare/trunk...juiceblender:cassandra-13068

I've added some 3 different tests using the example, but I've also added an 
extra check 
{code}
final Set fullyExpiredSSTables = 
controller.getFullyExpiredSSTables();
final Set actuallyCompact = 
Sets.difference(transaction.originals(), fullyExpiredSSTables);

// note that we need to do a rough estimate early if we can fit the 
compaction on disk - this is pessimistic, but
// since we might remove sstables from the compaction in 
checkAvailableDiskSpace it needs to be done here
// If there are no fully expired SSTables, check available disk 
space. Otherwise, we want to try and always compact
// fully expired SSTables
if (fullyExpiredSSTables.isEmpty())
checkAvailableDiskSpace(actuallyCompact);
{code}

This is to make the test I wrote 
{{testExpiredSSTablesStillGetDroppedWithNoDiskSpace()}} work. I am not sure if 
this is the right way to do it or if it was intentionally left that way - 
basically if you have expired SSTables and SSTables that are too big, we will 
throw the run time exception instead of try and compact the expired SSTables 
away. I don't know if there's ordering of compactions at this point though, so 
I don't know if this is a safe operation to do. 

Feel free to let me know about any feedback. There seem to be other ways to go 
about this, such as {{checkAvailableDiskSpace}} should just estimate fully 
expired SSTables to have compacted file size 0 instead...? 

> Fully expired sstable not dropped when running out of disk space
> 
>
> Key: CASSANDRA-13068
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13068
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>  Labels: lhf
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> If a fully expired sstable is larger than the remaining disk space we won't 
> run the compaction that can drop the sstable (ie, in our disk space check 
> should not include the fully expired sstables)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13068) Fully expired sstable not dropped when running out of disk space

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028709#comment-16028709
 ] 

Lerh Chuan Low commented on CASSANDRA-13068:


Thanks for the quick reply. I'll look into writing some tests (I was initially 
wondering how that would be done without actually filling up the disk of the 
machine). I didn't know about Byteman, so going to do some digging into it and 
also thanks for providing an example. 

Re: {{Sets.difference}} - you're right, just checked the java doc. Wasn't aware 
it returned an unmodifiable view of a set. 

> Fully expired sstable not dropped when running out of disk space
> 
>
> Key: CASSANDRA-13068
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13068
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>  Labels: lhf
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> If a fully expired sstable is larger than the remaining disk space we won't 
> run the compaction that can drop the sstable (ie, in our disk space check 
> should not include the fully expired sstables)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13505) dtest failure in user_functions_test.TestUserFunctions.test_migration

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16027991#comment-16027991
 ] 

Lerh Chuan Low commented on CASSANDRA-13505:


Hi Ariel,

I tried running this 20 times on a test box and they all passed. Looking at the 
history of the jobs there hasn't been any failures recently:

CassCI:
http://cassci.datastax.com/job/trunk_dtest/1575/testReport/user_functions_test/TestUserFunctions/test_migration/history/

Apache Jenkins: (These are the only 2 failures recently and they failed due to 
"Address already in use; a cluster may already be running or you may need to 
add the loopback alias". There were 800-1000 failures in each of the jobs so 
there may have been an infrastructure issue. 
https://builds.apache.org/job/Cassandra-trunk-dtest/142/testReport/user_functions_test/
https://builds.apache.org/job/Cassandra-trunk-dtest/143/testReport/user_functions_test/

I think we can close this, what do you think? 


> dtest failure in user_functions_test.TestUserFunctions.test_migration
> -
>
> Key: CASSANDRA-13505
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13505
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Ariel Weisberg
>  Labels: dtest, test-failure
>
> {noformat}
> Failed 1 times in the last 10 runs. Flakiness: 11%, Stability: 90%
> Error Message
>  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> java.lang.RuntimeException: java.nio.file.NoSuchFileException: 
> /tmp/dtest-c0Kk_e/test/node3/data2/system_schema/keyspaces-abac5682dea631c5b535b3d6cffd0fb6/na_txn_flush_1304bca0-2b13-11e7-9307-c95a627b1fe3.log">
>  >> begin captured logging << 
> dtest: DEBUG: cluster ccm directory: /tmp/dtest-c0Kk_e
> dtest: DEBUG: Done setting configuration options:
> {   'enable_scripted_user_defined_functions': 'true',
> 'enable_user_defined_functions': 'true',
> 'initial_token': None,
> 'num_tokens': '32',
> 'phi_convict_threshold': 5}
> cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, 
> scheduling retry in 600.0 seconds: [Errno 111] Tried connecting to 
> [('127.0.0.3', 9042)]. Last error: Connection refused
> cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, 
> scheduling retry in 600.0 seconds: [Errno 111] Tried connecting to 
> [('127.0.0.3', 9042)]. Last error: Connection refused
> cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, 
> scheduling retry in 600.0 seconds: [Errno 111] Tried connecting to 
> [('127.0.0.3', 9042)]. Last error: Connection refused
> cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, 
> scheduling retry in 600.0 seconds: [Errno 111] Tried connecting to 
> [('127.0.0.3', 9042)]. Last error: Connection refused
> cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, 
> scheduling retry in 600.0 seconds: [Errno 111] Tried connecting to 
> [('127.0.0.3', 9042)]. Last error: Connection refused
> cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, 
> scheduling retry in 600.0 seconds: [Errno 111] Tried connecting to 
> [('127.0.0.3', 9042)]. Last error: Connection refused
> cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, 
> scheduling retry in 600.0 seconds: [Errno 111] Tried connecting to 
> [('127.0.0.3', 9042)]. Last error: Connection refused
> cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, 
> scheduling retry in 600.0 seconds: [Errno 111] Tried connecting to 
> [('127.0.0.3', 9042)]. Last error: Connection refused
> cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, 
> scheduling retry in 600.0 seconds: [Errno 111] Tried connecting to 
> [('127.0.0.3', 9042)]. Last error: Connection refused
> cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, 
> scheduling retry in 600.0 seconds: [Errno 111] Tried connecting to 
> [('127.0.0.3', 9042)]. Last error: Connection refused
> cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, 
> scheduling retry in 600.0 seconds: [Errno 111] Tried connecting to 
> [('127.0.0.3', 9042)]. Last error: Connection refused
> cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, 
> scheduling retry in 600.0 seconds: [Errno 111] Tried connecting to 
> [('127.0.0.3', 9042)]. Last error: Connection refused
> cassandra.policies: INFO: Using datacenter 'datacenter1' for 
> DCAwareRoundRobinPolicy (via host '127.0.0.1'); if incorrect, please specify 
> a local_dc to the constructor, or limit contact points to local cluster nodes
> cassandra.cluster: INFO: New Cassandra host  
> discovered
> 

[jira] [Comment Edited] (CASSANDRA-13182) test failure in sstableutil_test.SSTableUtilTest.compaction_test

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16025586#comment-16025586
 ] 

Lerh Chuan Low edited comment on CASSANDRA-13182 at 5/29/17 12:19 AM:
--

Thanks for your feedback Ariel, makes sense :)

The ticket for marking those methods as deprecated is here: 
https://issues.apache.org/jira/browse/CASSANDRA-13541


was (Author: lerh low):
The ticket for marking those methods as deprecated is here: 
https://issues.apache.org/jira/browse/CASSANDRA-13541

> test failure in sstableutil_test.SSTableUtilTest.compaction_test
> 
>
> Key: CASSANDRA-13182
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13182
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Shuler
>Assignee: Lerh Chuan Low
>  Labels: dtest, test-failure, test-failure-fresh
> Attachments: node1_debug.log, node1_gc.log, node1.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_novnode_dtest/506/testReport/sstableutil_test/SSTableUtilTest/compaction_test
> {noformat}
> Error Message
> Lists differ: ['/tmp/dtest-Rk_3Cs/test/node1... != 
> ['/tmp/dtest-Rk_3Cs/test/node1...
> First differing element 8:
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-2-big-CRC.db'
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-6-big-CRC.db'
> First list contains 7 additional elements.
> First extra element 24:
> '/tmp/dtest-Rk_3Cs/test/node1/data2/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-5-big-Data.db'
>   
> ['/tmp/dtest-Rk_3Cs/test/node1/data0/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-4-big-CRC.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data0/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-4-big-Data.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data0/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-4-big-Digest.crc32',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data0/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-4-big-Filter.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data0/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-4-big-Index.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data0/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-4-big-Statistics.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data0/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-4-big-Summary.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data0/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-4-big-TOC.txt',
> -  
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-2-big-CRC.db',
> -  
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-2-big-Digest.crc32',
> -  
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-2-big-Filter.db',
> -  
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-2-big-Index.db',
> -  
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-2-big-Statistics.db',
> -  
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-2-big-Summary.db',
> -  
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-2-big-TOC.txt',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-6-big-CRC.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-6-big-Data.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-6-big-Digest.crc32',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-6-big-Filter.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-6-big-Index.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-6-big-Statistics.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-6-big-Summary.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-6-big-TOC.txt',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data2/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-5-big-CRC.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data2/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-5-big-Data.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data2/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-5-big-Digest.crc32',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data2/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-5-big-Filter.db',
>   

[jira] [Commented] (CASSANDRA-13068) Fully expired sstable not dropped when running out of disk space

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16027975#comment-16027975
 ] 

Lerh Chuan Low commented on CASSANDRA-13068:


Hi [~krummas], 

I've pushed up a patch for this for trunk here: 
https://github.com/apache/cassandra/compare/trunk...juiceblender:cassandra-13068,
 any thoughts? If it seems ok I'll make one for the other branches (it doesn't 
apply cleanly). I've manually tested it locally on my machine to verify that 
the disk space check no longer includes fully expired SSTables, and also re-ran 
the LongCompactionsTest. 

Ty! 

> Fully expired sstable not dropped when running out of disk space
> 
>
> Key: CASSANDRA-13068
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13068
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Marcus Eriksson
>  Labels: lhf
>
> If a fully expired sstable is larger than the remaining disk space we won't 
> run the compaction that can drop the sstable (ie, in our disk space check 
> should not include the fully expired sstables)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13182) test failure in sstableutil_test.SSTableUtilTest.compaction_test

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16025586#comment-16025586
 ] 

Lerh Chuan Low commented on CASSANDRA-13182:


The ticket for marking those methods as deprecated is here: 
https://issues.apache.org/jira/browse/CASSANDRA-13541

> test failure in sstableutil_test.SSTableUtilTest.compaction_test
> 
>
> Key: CASSANDRA-13182
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13182
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Shuler
>Assignee: Lerh Chuan Low
>  Labels: dtest, test-failure, test-failure-fresh
> Attachments: node1_debug.log, node1_gc.log, node1.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_novnode_dtest/506/testReport/sstableutil_test/SSTableUtilTest/compaction_test
> {noformat}
> Error Message
> Lists differ: ['/tmp/dtest-Rk_3Cs/test/node1... != 
> ['/tmp/dtest-Rk_3Cs/test/node1...
> First differing element 8:
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-2-big-CRC.db'
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-6-big-CRC.db'
> First list contains 7 additional elements.
> First extra element 24:
> '/tmp/dtest-Rk_3Cs/test/node1/data2/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-5-big-Data.db'
>   
> ['/tmp/dtest-Rk_3Cs/test/node1/data0/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-4-big-CRC.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data0/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-4-big-Data.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data0/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-4-big-Digest.crc32',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data0/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-4-big-Filter.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data0/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-4-big-Index.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data0/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-4-big-Statistics.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data0/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-4-big-Summary.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data0/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-4-big-TOC.txt',
> -  
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-2-big-CRC.db',
> -  
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-2-big-Digest.crc32',
> -  
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-2-big-Filter.db',
> -  
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-2-big-Index.db',
> -  
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-2-big-Statistics.db',
> -  
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-2-big-Summary.db',
> -  
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-2-big-TOC.txt',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-6-big-CRC.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-6-big-Data.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-6-big-Digest.crc32',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-6-big-Filter.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-6-big-Index.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-6-big-Statistics.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-6-big-Summary.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data1/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-6-big-TOC.txt',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data2/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-5-big-CRC.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data2/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-5-big-Data.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data2/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-5-big-Digest.crc32',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data2/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-5-big-Filter.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data2/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-5-big-Index.db',
>
> '/tmp/dtest-Rk_3Cs/test/node1/data2/keyspace1/standard1-11ee2450e8ab11e6b5a68de39eb517c4/mc-5-big-Statistics.db',
>

[jira] [Commented] (CASSANDRA-13541) Mark couple of API methods for Compaction as Deprecated

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16019080#comment-16019080
 ] 

Lerh Chuan Low commented on CASSANDRA-13541:


Feel free to let me know if there's anything you would like changed :)

https://github.com/apache/cassandra/pull/114

> Mark couple of API methods for Compaction as Deprecated
> ---
>
> Key: CASSANDRA-13541
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13541
> Project: Cassandra
>  Issue Type: Task
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Trivial
>
> A follow up from 
> https://issues.apache.org/jira/browse/CASSANDRA-13182?focusedCommentId=16013347=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16013347.
>  
> Enabling and disabling Compaction is done via {{CompactionStrategyManager}} 
> so these methods are no longer used. We shouldn't totally remove them as 
> people may have written plugins for CompactionStrategy. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-13541) Mark couple of API methods for Compaction as Deprecated

[Lerh Chuan Low (JIRA)]

Lerh Chuan Low created CASSANDRA-13541:
--

 Summary: Mark couple of API methods for Compaction as Deprecated
 Key: CASSANDRA-13541
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13541
 Project: Cassandra
  Issue Type: Task
Reporter: Lerh Chuan Low
Assignee: Lerh Chuan Low
Priority: Trivial


A follow up from 
https://issues.apache.org/jira/browse/CASSANDRA-13182?focusedCommentId=16013347=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16013347.
 

Enabling and disabling Compaction is done via {{CompactionStrategyManager}} so 
these methods are no longer used. We shouldn't totally remove them as people 
may have written plugins for CompactionStrategy. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13528) nodetool describeclusters shows different snitch info as to what is configured.

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16016585#comment-16016585
 ] 

Lerh Chuan Low commented on CASSANDRA-13528:


I like that approach, sounds reasonable. I've attached some patches, one for 
2.1 and one for 2.2, 3.0, 3.X, trunk (it applies cleanly). Any thoughts? 

{code}
Cluster Information:
Name: Test Cluster
Snitch: org.apache.cassandra.locator.SimpleSnitch
DynamicEndPointSnitch: enabled
Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
Schema versions:
9cf2fd0e-ab63-348e-b9af-6db0a3da5a29: [127.0.0.1]
{code}

{code}
Cluster Information:
Name: Test Cluster
Snitch: org.apache.cassandra.locator.SimpleSnitch
DynamicEndPointSnitch: disabled
Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
Schema versions:
9cf2fd0e-ab63-348e-b9af-6db0a3da5a29: [127.0.0.1]
{code}

> nodetool describeclusters shows different snitch info as to what is 
> configured.
> ---
>
> Key: CASSANDRA-13528
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13528
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Paul Villacorta
>Priority: Minor
>  Labels: lhf
> Attachments: 13528-2.1.txt, 13528-trunk.txt, Screen Shot 2017-05-12 
> at 14.15.04.png
>
>
> I couldn't find any similar issue as this one so I'm creating one.
> I noticed that doing nodetool describecluster shows a different Snitch 
> Information as to what is being set in the configuration file.
> My setup is hosted in AWS and I am using Ec2Snitch.
> cassandra@cassandra3$ nodetool describecluster
> Cluster Information:
>   Name: testv3
>   Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
>   Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
>   Schema versions:
>   fc6e8656-ee7a-341b-9782-b569d1fd1a51: 
> [10.0.3.61,10.0.3.62,10.0.3.63]
> I checked via MX4J and it shows the same, I haven't verified tho using a 
> different Snitch and I am using 2.2.6 above and 3.0.X 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13528) nodetool describeclusters shows different snitch info as to what is configured.

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-13528:
---
Attachment: 13528-trunk.txt
13528-2.1.txt

> nodetool describeclusters shows different snitch info as to what is 
> configured.
> ---
>
> Key: CASSANDRA-13528
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13528
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Paul Villacorta
>Priority: Minor
>  Labels: lhf
> Attachments: 13528-2.1.txt, 13528-trunk.txt, Screen Shot 2017-05-12 
> at 14.15.04.png
>
>
> I couldn't find any similar issue as this one so I'm creating one.
> I noticed that doing nodetool describecluster shows a different Snitch 
> Information as to what is being set in the configuration file.
> My setup is hosted in AWS and I am using Ec2Snitch.
> cassandra@cassandra3$ nodetool describecluster
> Cluster Information:
>   Name: testv3
>   Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
>   Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
>   Schema versions:
>   fc6e8656-ee7a-341b-9782-b569d1fd1a51: 
> [10.0.3.61,10.0.3.62,10.0.3.63]
> I checked via MX4J and it shows the same, I haven't verified tho using a 
> different Snitch and I am using 2.2.6 above and 3.0.X 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13528) nodetool describeclusters shows different snitch info as to what is configured.

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-13528:
---
Assignee: Lerh Chuan Low
  Status: Patch Available  (was: Open)

> nodetool describeclusters shows different snitch info as to what is 
> configured.
> ---
>
> Key: CASSANDRA-13528
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13528
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Paul Villacorta
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Attachments: 13528-2.1.txt, 13528-trunk.txt, Screen Shot 2017-05-12 
> at 14.15.04.png
>
>
> I couldn't find any similar issue as this one so I'm creating one.
> I noticed that doing nodetool describecluster shows a different Snitch 
> Information as to what is being set in the configuration file.
> My setup is hosted in AWS and I am using Ec2Snitch.
> cassandra@cassandra3$ nodetool describecluster
> Cluster Information:
>   Name: testv3
>   Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
>   Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
>   Schema versions:
>   fc6e8656-ee7a-341b-9782-b569d1fd1a51: 
> [10.0.3.61,10.0.3.62,10.0.3.63]
> I checked via MX4J and it shows the same, I haven't verified tho using a 
> different Snitch and I am using 2.2.6 above and 3.0.X 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13346) Failed unregistering mbean during drop keyspace

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16016460#comment-16016460
 ] 

Lerh Chuan Low commented on CASSANDRA-13346:


Ahhh I see, thanks for going through all that with me though :)

> Failed unregistering mbean during drop keyspace
> ---
>
> Key: CASSANDRA-13346
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13346
> Project: Cassandra
>  Issue Type: Bug
>  Components: Materialized Views
> Environment: Cassandra 3.9
>Reporter: Gábor Auth
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Fix For: 3.10, 3.0.x
>
> Attachments: 13346-3.0.X.txt, 13346-3.X.txt, 13346-trunk.txt
>
>
> All node throw exceptions about materialized views during drop keyspace:
> {code}
> WARN  [MigrationStage:1] 2017-03-16 16:54:25,016 ColumnFamilyStore.java:535 - 
> Failed unregistering mbean: 
> org.apache.cassandra.db:type=Tables,keyspace=test20160810,table=unit_by_account
> java.lang.NullPointerException: null
> at 
> java.util.concurrent.ConcurrentHashMap.replaceNode(ConcurrentHashMap.java:1106)
>  ~[na:1.8.0_121]
> at 
> java.util.concurrent.ConcurrentHashMap.remove(ConcurrentHashMap.java:1097) 
> ~[na:1.8.0_121]
> at 
> java.util.concurrent.ConcurrentHashMap$KeySetView.remove(ConcurrentHashMap.java:4569)
>  ~[na:1.8.0_121]
> at 
> org.apache.cassandra.metrics.TableMetrics.release(TableMetrics.java:712) 
> ~[apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.db.ColumnFamilyStore.unregisterMBean(ColumnFamilyStore.java:570)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.db.ColumnFamilyStore.invalidate(ColumnFamilyStore.java:527)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.db.ColumnFamilyStore.invalidate(ColumnFamilyStore.java:517)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at org.apache.cassandra.db.Keyspace.unloadCf(Keyspace.java:365) 
> [apache-cassandra-3.9.0.jar:3.9.0]
> at org.apache.cassandra.db.Keyspace.dropCf(Keyspace.java:358) 
> [apache-cassandra-3.9.0.jar:3.9.0]
> at org.apache.cassandra.config.Schema.dropView(Schema.java:744) 
> [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.lambda$mergeSchema$373(SchemaKeyspace.java:1287)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_121]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.mergeSchema(SchemaKeyspace.java:1287)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.mergeSchemaAndAnnounceVersion(SchemaKeyspace.java:1256)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:51)
>  ~[apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[apache-cassandra-3.9.0.jar:3.9.0]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_121]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_121]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_121]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  ~[na:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_121]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13346) Failed unregistering mbean during drop keyspace

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-13346:
---
Fix Version/s: (was: 3.11.x)
   3.10

> Failed unregistering mbean during drop keyspace
> ---
>
> Key: CASSANDRA-13346
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13346
> Project: Cassandra
>  Issue Type: Bug
>  Components: Materialized Views
> Environment: Cassandra 3.9
>Reporter: Gábor Auth
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Fix For: 3.10, 3.0.x
>
> Attachments: 13346-3.0.X.txt, 13346-3.X.txt, 13346-trunk.txt
>
>
> All node throw exceptions about materialized views during drop keyspace:
> {code}
> WARN  [MigrationStage:1] 2017-03-16 16:54:25,016 ColumnFamilyStore.java:535 - 
> Failed unregistering mbean: 
> org.apache.cassandra.db:type=Tables,keyspace=test20160810,table=unit_by_account
> java.lang.NullPointerException: null
> at 
> java.util.concurrent.ConcurrentHashMap.replaceNode(ConcurrentHashMap.java:1106)
>  ~[na:1.8.0_121]
> at 
> java.util.concurrent.ConcurrentHashMap.remove(ConcurrentHashMap.java:1097) 
> ~[na:1.8.0_121]
> at 
> java.util.concurrent.ConcurrentHashMap$KeySetView.remove(ConcurrentHashMap.java:4569)
>  ~[na:1.8.0_121]
> at 
> org.apache.cassandra.metrics.TableMetrics.release(TableMetrics.java:712) 
> ~[apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.db.ColumnFamilyStore.unregisterMBean(ColumnFamilyStore.java:570)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.db.ColumnFamilyStore.invalidate(ColumnFamilyStore.java:527)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.db.ColumnFamilyStore.invalidate(ColumnFamilyStore.java:517)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at org.apache.cassandra.db.Keyspace.unloadCf(Keyspace.java:365) 
> [apache-cassandra-3.9.0.jar:3.9.0]
> at org.apache.cassandra.db.Keyspace.dropCf(Keyspace.java:358) 
> [apache-cassandra-3.9.0.jar:3.9.0]
> at org.apache.cassandra.config.Schema.dropView(Schema.java:744) 
> [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.lambda$mergeSchema$373(SchemaKeyspace.java:1287)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_121]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.mergeSchema(SchemaKeyspace.java:1287)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.mergeSchemaAndAnnounceVersion(SchemaKeyspace.java:1256)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:51)
>  ~[apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[apache-cassandra-3.9.0.jar:3.9.0]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_121]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_121]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_121]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  ~[na:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_121]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13346) Failed unregistering mbean during drop keyspace

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16015203#comment-16015203
 ] 

Lerh Chuan Low commented on CASSANDRA-13346:


Attached, trunk is the same as 3.10.

> Failed unregistering mbean during drop keyspace
> ---
>
> Key: CASSANDRA-13346
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13346
> Project: Cassandra
>  Issue Type: Bug
>  Components: Materialized Views
> Environment: Cassandra 3.9
>Reporter: Gábor Auth
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Fix For: 3.0.x, 3.11.x
>
> Attachments: 13346-3.0.X.txt, 13346-3.X.txt, 13346-trunk.txt
>
>
> All node throw exceptions about materialized views during drop keyspace:
> {code}
> WARN  [MigrationStage:1] 2017-03-16 16:54:25,016 ColumnFamilyStore.java:535 - 
> Failed unregistering mbean: 
> org.apache.cassandra.db:type=Tables,keyspace=test20160810,table=unit_by_account
> java.lang.NullPointerException: null
> at 
> java.util.concurrent.ConcurrentHashMap.replaceNode(ConcurrentHashMap.java:1106)
>  ~[na:1.8.0_121]
> at 
> java.util.concurrent.ConcurrentHashMap.remove(ConcurrentHashMap.java:1097) 
> ~[na:1.8.0_121]
> at 
> java.util.concurrent.ConcurrentHashMap$KeySetView.remove(ConcurrentHashMap.java:4569)
>  ~[na:1.8.0_121]
> at 
> org.apache.cassandra.metrics.TableMetrics.release(TableMetrics.java:712) 
> ~[apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.db.ColumnFamilyStore.unregisterMBean(ColumnFamilyStore.java:570)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.db.ColumnFamilyStore.invalidate(ColumnFamilyStore.java:527)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.db.ColumnFamilyStore.invalidate(ColumnFamilyStore.java:517)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at org.apache.cassandra.db.Keyspace.unloadCf(Keyspace.java:365) 
> [apache-cassandra-3.9.0.jar:3.9.0]
> at org.apache.cassandra.db.Keyspace.dropCf(Keyspace.java:358) 
> [apache-cassandra-3.9.0.jar:3.9.0]
> at org.apache.cassandra.config.Schema.dropView(Schema.java:744) 
> [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.lambda$mergeSchema$373(SchemaKeyspace.java:1287)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_121]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.mergeSchema(SchemaKeyspace.java:1287)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.mergeSchemaAndAnnounceVersion(SchemaKeyspace.java:1256)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:51)
>  ~[apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[apache-cassandra-3.9.0.jar:3.9.0]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_121]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_121]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_121]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  ~[na:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_121]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13346) Failed unregistering mbean during drop keyspace

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-13346:
---
Attachment: 13346-trunk.txt

> Failed unregistering mbean during drop keyspace
> ---
>
> Key: CASSANDRA-13346
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13346
> Project: Cassandra
>  Issue Type: Bug
>  Components: Materialized Views
> Environment: Cassandra 3.9
>Reporter: Gábor Auth
>Assignee: Lerh Chuan Low
>Priority: Minor
>  Labels: lhf
> Fix For: 3.0.x, 3.11.x
>
> Attachments: 13346-3.0.X.txt, 13346-3.X.txt, 13346-trunk.txt
>
>
> All node throw exceptions about materialized views during drop keyspace:
> {code}
> WARN  [MigrationStage:1] 2017-03-16 16:54:25,016 ColumnFamilyStore.java:535 - 
> Failed unregistering mbean: 
> org.apache.cassandra.db:type=Tables,keyspace=test20160810,table=unit_by_account
> java.lang.NullPointerException: null
> at 
> java.util.concurrent.ConcurrentHashMap.replaceNode(ConcurrentHashMap.java:1106)
>  ~[na:1.8.0_121]
> at 
> java.util.concurrent.ConcurrentHashMap.remove(ConcurrentHashMap.java:1097) 
> ~[na:1.8.0_121]
> at 
> java.util.concurrent.ConcurrentHashMap$KeySetView.remove(ConcurrentHashMap.java:4569)
>  ~[na:1.8.0_121]
> at 
> org.apache.cassandra.metrics.TableMetrics.release(TableMetrics.java:712) 
> ~[apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.db.ColumnFamilyStore.unregisterMBean(ColumnFamilyStore.java:570)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.db.ColumnFamilyStore.invalidate(ColumnFamilyStore.java:527)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.db.ColumnFamilyStore.invalidate(ColumnFamilyStore.java:517)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at org.apache.cassandra.db.Keyspace.unloadCf(Keyspace.java:365) 
> [apache-cassandra-3.9.0.jar:3.9.0]
> at org.apache.cassandra.db.Keyspace.dropCf(Keyspace.java:358) 
> [apache-cassandra-3.9.0.jar:3.9.0]
> at org.apache.cassandra.config.Schema.dropView(Schema.java:744) 
> [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.lambda$mergeSchema$373(SchemaKeyspace.java:1287)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_121]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.mergeSchema(SchemaKeyspace.java:1287)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.mergeSchemaAndAnnounceVersion(SchemaKeyspace.java:1256)
>  [apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:51)
>  ~[apache-cassandra-3.9.0.jar:3.9.0]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[apache-cassandra-3.9.0.jar:3.9.0]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_121]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_121]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_121]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  ~[na:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_121]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org