[jira] [Created] (CASSANDRA-15932) CommitLogSegmentManager#sync is doing sync one by one need to be improved
Yadong Chen created CASSANDRA-15932: --- Summary: CommitLogSegmentManager#sync is doing sync one by one need to be improved Key: CASSANDRA-15932 URL: https://issues.apache.org/jira/browse/CASSANDRA-15932 Project: Cassandra Issue Type: Improvement Components: Local/Commit Log Reporter: Yadong Chen Assignee: Yadong Chen Attachments: image-2020-07-08-12-05-07-746.png, image-2020-07-08-12-21-10-369.png test machine settings: # 24 cores # 45G memory # maximum disk thoughput can over 500MB/s I was running PeriodicCommitLogStressTest#testFixedSize in that test machine and notice both cpu and io can't be full utilized PeriodicCommitLogStressTest#testFixedSize was running with CommitLogSyncPeriod set to 1 (I changed it to match production environment setting) and no compression and no encryption !image-2020-07-08-12-05-07-746.png! After some digging I find out it's the sync process can't keep up and when lastSyncedAt is far behind all thread doing CommitLog#add will wait until it catch up. Once catch up, all threads go back continue writing and so on. The overall IO utilization I guess is about 60% more or less. After introduce a thread pool to do sync parallel instead of doing it one by one. The IO utilization is improved !image-2020-07-08-12-21-10-369.png! the IO throughput is more steady (almost always above 400MB) and need less time to reach that throughput -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15931) USING_G1 is incorrectly set in cassandra-env.sh if G1 is explicitly disabled
[
https://issues.apache.org/jira/browse/CASSANDRA-15931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-15931:
--
Test and Documentation Plan: deploy cluster in same environment which first
hit the issue
Status: Patch Available (was: Open)
> USING_G1 is incorrectly set in cassandra-env.sh if G1 is explicitly disabled
>
>
> Key: CASSANDRA-15931
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15931
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Startup and Shutdown
>Reporter: Jeremy Hanna
>Assignee: Jeremy Hanna
>Priority: Normal
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> echo $JVM_OPTS | grep -q UseG1GC
> USING_G1=$?
> {code}
> This code will set {{USING_G1}} to {{0}} if G1 is explicitly enabled
> ({{+UseG1GC}}) *or* explicitly disabled ({{-UseG1GC}}), as found on
> CASSANDRA-15839.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15931) USING_G1 is incorrectly set in cassandra-env.sh if G1 is explicitly disabled
[
https://issues.apache.org/jira/browse/CASSANDRA-15931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-15931:
--
Reviewers: David Capwell, David Capwell (was: David Capwell)
David Capwell, David Capwell
Status: Review In Progress (was: Patch Available)
Patch LGTM, to make sure it is working I am deploying it to a cluster using the
same setup that detected the issue.
> USING_G1 is incorrectly set in cassandra-env.sh if G1 is explicitly disabled
>
>
> Key: CASSANDRA-15931
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15931
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Startup and Shutdown
>Reporter: Jeremy Hanna
>Assignee: Jeremy Hanna
>Priority: Normal
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> echo $JVM_OPTS | grep -q UseG1GC
> USING_G1=$?
> {code}
> This code will set {{USING_G1}} to {{0}} if G1 is explicitly enabled
> ({{+UseG1GC}}) *or* explicitly disabled ({{-UseG1GC}}), as found on
> CASSANDRA-15839.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15931) USING_G1 is incorrectly set in cassandra-env.sh if G1 is explicitly disabled
[
https://issues.apache.org/jira/browse/CASSANDRA-15931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-15931:
--
Bug Category: Parent values: Correctness(12982)Level 1 values: API /
Semantic Implementation(12988)
Complexity: Low Hanging Fruit
Discovered By: Adhoc Test
Fix Version/s: 4.0-alpha
Severity: Normal
Status: Open (was: Triage Needed)
> USING_G1 is incorrectly set in cassandra-env.sh if G1 is explicitly disabled
>
>
> Key: CASSANDRA-15931
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15931
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Startup and Shutdown
>Reporter: Jeremy Hanna
>Assignee: Jeremy Hanna
>Priority: Normal
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> echo $JVM_OPTS | grep -q UseG1GC
> USING_G1=$?
> {code}
> This code will set {{USING_G1}} to {{0}} if G1 is explicitly enabled
> ({{+UseG1GC}}) *or* explicitly disabled ({{-UseG1GC}}), as found on
> CASSANDRA-15839.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15931) USING_G1 is incorrectly set in cassandra-env.sh if G1 is explicitly disabled
[
https://issues.apache.org/jira/browse/CASSANDRA-15931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeremy Hanna updated CASSANDRA-15931:
-
Component/s: Local/Startup and Shutdown
> USING_G1 is incorrectly set in cassandra-env.sh if G1 is explicitly disabled
>
>
> Key: CASSANDRA-15931
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15931
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Startup and Shutdown
>Reporter: Jeremy Hanna
>Assignee: Jeremy Hanna
>Priority: Normal
>
> {code}
> echo $JVM_OPTS | grep -q UseG1GC
> USING_G1=$?
> {code}
> This code will set {{USING_G1}} to {{0}} if G1 is explicitly enabled
> ({{+UseG1GC}}) *or* explicitly disabled ({{-UseG1GC}}), as found on
> CASSANDRA-15839.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15931) USING_G1 is incorrectly set in cassandra-env.sh if G1 is explicitly disabled
[
https://issues.apache.org/jira/browse/CASSANDRA-15931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeremy Hanna updated CASSANDRA-15931:
-
Status: Triage Needed (was: Awaiting Feedback)
> USING_G1 is incorrectly set in cassandra-env.sh if G1 is explicitly disabled
>
>
> Key: CASSANDRA-15931
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15931
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jeremy Hanna
>Assignee: Jeremy Hanna
>Priority: Normal
>
> {code}
> echo $JVM_OPTS | grep -q UseG1GC
> USING_G1=$?
> {code}
> This code will set {{USING_G1}} to {{0}} if G1 is explicitly enabled
> ({{+UseG1GC}}) *or* explicitly disabled ({{-UseG1GC}}), as found on
> CASSANDRA-15839.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15931) USING_G1 is incorrectly set in cassandra-env.sh if G1 is explicitly disabled
[
https://issues.apache.org/jira/browse/CASSANDRA-15931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeremy Hanna reassigned CASSANDRA-15931:
Assignee: Jeremy Hanna
> USING_G1 is incorrectly set in cassandra-env.sh if G1 is explicitly disabled
>
>
> Key: CASSANDRA-15931
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15931
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jeremy Hanna
>Assignee: Jeremy Hanna
>Priority: Normal
>
> {code}
> echo $JVM_OPTS | grep -q UseG1GC
> USING_G1=$?
> {code}
> This code will set {{USING_G1}} to {{0}} if G1 is explicitly enabled
> ({{+UseG1GC}}) *or* explicitly disabled ({{-UseG1GC}}), as found on
> CASSANDRA-15839.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15931) USING_G1 is incorrectly set in cassandra-env.sh if G1 is explicitly disabled
[
https://issues.apache.org/jira/browse/CASSANDRA-15931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeremy Hanna updated CASSANDRA-15931:
-
Status: Awaiting Feedback (was: Triage Needed)
> USING_G1 is incorrectly set in cassandra-env.sh if G1 is explicitly disabled
>
>
> Key: CASSANDRA-15931
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15931
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jeremy Hanna
>Assignee: Jeremy Hanna
>Priority: Normal
>
> {code}
> echo $JVM_OPTS | grep -q UseG1GC
> USING_G1=$?
> {code}
> This code will set {{USING_G1}} to {{0}} if G1 is explicitly enabled
> ({{+UseG1GC}}) *or* explicitly disabled ({{-UseG1GC}}), as found on
> CASSANDRA-15839.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15931) USING_G1 is incorrectly set in cassandra-env.sh if G1 is explicitly disabled
[
https://issues.apache.org/jira/browse/CASSANDRA-15931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153201#comment-17153201
]
Jeremy Hanna commented on CASSANDRA-15931:
--
PR after testing that it matched {{+UseG1GC}} but not {{-UseG1GC}}:
https://github.com/apache/cassandra/pull/667
> USING_G1 is incorrectly set in cassandra-env.sh if G1 is explicitly disabled
>
>
> Key: CASSANDRA-15931
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15931
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Startup and Shutdown
>Reporter: Jeremy Hanna
>Assignee: Jeremy Hanna
>Priority: Normal
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> echo $JVM_OPTS | grep -q UseG1GC
> USING_G1=$?
> {code}
> This code will set {{USING_G1}} to {{0}} if G1 is explicitly enabled
> ({{+UseG1GC}}) *or* explicitly disabled ({{-UseG1GC}}), as found on
> CASSANDRA-15839.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15931) USING_G1 is incorrectly set in cassandra-env.sh if G1 is explicitly disabled
[
https://issues.apache.org/jira/browse/CASSANDRA-15931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeremy Hanna updated CASSANDRA-15931:
-
Description:
{code}
echo $JVM_OPTS | grep -q UseG1GC
USING_G1=$?
{code}
This code will set {{USING_G1}} to {{0}} if G1 is explicitly enabled
({{+UseG1GC}}) *or* explicitly disabled ({{-UseG1GC}}), as found on
CASSANDRA-15839.
was:
{code}
echo $JVM_OPTS | grep -q UseG1GC
USING_G1=$?
{code}
This code will set {{USING_G1}} to {{0}} if G1 is explicitly enabled *or*
explicitly disabled, as found on CASSANDRA-15839.
> USING_G1 is incorrectly set in cassandra-env.sh if G1 is explicitly disabled
>
>
> Key: CASSANDRA-15931
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15931
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jeremy Hanna
>Priority: Normal
>
> {code}
> echo $JVM_OPTS | grep -q UseG1GC
> USING_G1=$?
> {code}
> This code will set {{USING_G1}} to {{0}} if G1 is explicitly enabled
> ({{+UseG1GC}}) *or* explicitly disabled ({{-UseG1GC}}), as found on
> CASSANDRA-15839.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15839) Warn or fail to start server when G1 is used and Xmn is set
[
https://issues.apache.org/jira/browse/CASSANDRA-15839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153193#comment-17153193
]
Jeremy Hanna edited comment on CASSANDRA-15839 at 7/8/20, 2:49 AM:
---
This is the code that *should* detect whether or not G1 is enabled. However
you can either enable or disable it with {{+UseG1GC}} or {{-UseG1GC}} which are
both matched by that {{grep}}. I have to admit I've never seen anyone in
practice explicitly disable G1.
{code:sh}
echo $JVM_OPTS | grep -q UseG1GC
USING_G1=$?
{code}
That also affects whether we calculate heap sizes automatically or whether we
fail out when we don't set the heap and new size in pairs:
{code:sh}
# only calculate the size if it's not set manually
if [ "x$MAX_HEAP_SIZE" = "x" ] && [ "x$HEAP_NEWSIZE" = "x" -o $USING_G1 -eq 0
]; then
calculate_heap_sizes
elif [ "x$MAX_HEAP_SIZE" = "x" ] || [ "x$HEAP_NEWSIZE" = "x" -a $USING_G1 -ne
0 ]; then
echo "please set or unset MAX_HEAP_SIZE and HEAP_NEWSIZE in pairs when
using CMS GC (see cassandra-env.sh)"
exit 1
fi
{code}
Created CASSANDRA-15931 to address this.
was (Author: jeromatron):
This is the code that *should* detect whether or not G1 is enabled. However
since you can either enable or disable it with that string {{+UseG1GC}} or
{{-UseG1GC}}. I have to admit I've never seen anyone in practice explicitly
disable G1.
{code:sh}
echo $JVM_OPTS | grep -q UseG1GC
USING_G1=$?
{code}
That also affects whether we calculate heap sizes automatically or whether we
fail out when we don't set the heap and new size in pairs:
{code:sh}
# only calculate the size if it's not set manually
if [ "x$MAX_HEAP_SIZE" = "x" ] && [ "x$HEAP_NEWSIZE" = "x" -o $USING_G1 -eq 0
]; then
calculate_heap_sizes
elif [ "x$MAX_HEAP_SIZE" = "x" ] || [ "x$HEAP_NEWSIZE" = "x" -a $USING_G1 -ne
0 ]; then
echo "please set or unset MAX_HEAP_SIZE and HEAP_NEWSIZE in pairs when
using CMS GC (see cassandra-env.sh)"
exit 1
fi
{code}
Created CASSANDRA-15931 to address this.
> Warn or fail to start server when G1 is used and Xmn is set
> ---
>
> Key: CASSANDRA-15839
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15839
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Startup and Shutdown
>Reporter: Jeremy Hanna
>Assignee: Anthony Grasso
>Priority: Normal
> Fix For: 4.0, 4.0-alpha5
>
>
> In jvm.options, we currently have a comment above where Xmn is set that says
> that you shouldn't set Xmn with G1 GC. That isn't enough - we should either
> warn in the logs or fail startup when they are set together.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15839) Warn or fail to start server when G1 is used and Xmn is set
[
https://issues.apache.org/jira/browse/CASSANDRA-15839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153193#comment-17153193
]
Jeremy Hanna commented on CASSANDRA-15839:
--
This is the code that *should* detect whether or not G1 is enabled. However
since you can either enable or disable it with that string {{+UseG1GC}} or
{{-UseG1GC}}. I have to admit I've never seen anyone in practice explicitly
disable G1.
{code:sh}
echo $JVM_OPTS | grep -q UseG1GC
USING_G1=$?
{code}
That also affects whether we calculate heap sizes automatically or whether we
fail out when we don't set the heap and new size in pairs:
{code:sh}
# only calculate the size if it's not set manually
if [ "x$MAX_HEAP_SIZE" = "x" ] && [ "x$HEAP_NEWSIZE" = "x" -o $USING_G1 -eq 0
]; then
calculate_heap_sizes
elif [ "x$MAX_HEAP_SIZE" = "x" ] || [ "x$HEAP_NEWSIZE" = "x" -a $USING_G1 -ne
0 ]; then
echo "please set or unset MAX_HEAP_SIZE and HEAP_NEWSIZE in pairs when
using CMS GC (see cassandra-env.sh)"
exit 1
fi
{code}
Created CASSANDRA-15931 to address this.
> Warn or fail to start server when G1 is used and Xmn is set
> ---
>
> Key: CASSANDRA-15839
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15839
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Startup and Shutdown
>Reporter: Jeremy Hanna
>Assignee: Anthony Grasso
>Priority: Normal
> Fix For: 4.0, 4.0-alpha5
>
>
> In jvm.options, we currently have a comment above where Xmn is set that says
> that you shouldn't set Xmn with G1 GC. That isn't enough - we should either
> warn in the logs or fail startup when they are set together.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15931) USING_G1 is incorrectly set in cassandra-env.sh if G1 is explicitly disabled
Jeremy Hanna created CASSANDRA-15931:
Summary: USING_G1 is incorrectly set in cassandra-env.sh if G1 is
explicitly disabled
Key: CASSANDRA-15931
URL: https://issues.apache.org/jira/browse/CASSANDRA-15931
Project: Cassandra
Issue Type: Bug
Reporter: Jeremy Hanna
{code}
echo $JVM_OPTS | grep -q UseG1GC
USING_G1=$?
{code}
This code will set {{USING_G1}} to {{0}} if G1 is explicitly enabled *or*
explicitly disabled, as found on CASSANDRA-15839.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15685) flaky testWithMismatchingPending - org.apache.cassandra.distributed.test.PreviewRepairTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153154#comment-17153154
]
Ekaterina Dimitrova commented on CASSANDRA-15685:
-
Thanks [~marcuse], I raised CASSANDRA-15930 for handling of the mentioned edge
case.
> flaky testWithMismatchingPending -
> org.apache.cassandra.distributed.test.PreviewRepairTest
> --
>
> Key: CASSANDRA-15685
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15685
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Kevin Gallardo
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-beta
>
> Attachments: log-CASSANDRA-15685.txt, output
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Observed in:
> https://app.circleci.com/pipelines/github/newkek/cassandra/34/workflows/1c6b157d-13c3-48a9-85fb-9fe8c153256b/jobs/191/tests
> Failure:
> {noformat}
> testWithMismatchingPending -
> org.apache.cassandra.distributed.test.PreviewRepairTest
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.distributed.test.PreviewRepairTest.testWithMismatchingPending(PreviewRepairTest.java:97)
> {noformat}
> [Circle
> CI|https://circleci.com/gh/dcapwell/cassandra/tree/bug%2FCASSANDRA-15685]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15930) Follow up CASSANDRA-15685
Ekaterina Dimitrova created CASSANDRA-15930:
---
Summary: Follow up CASSANDRA-15685
Key: CASSANDRA-15930
URL: https://issues.apache.org/jira/browse/CASSANDRA-15930
Project: Cassandra
Issue Type: Improvement
Reporter: Ekaterina Dimitrova
While working on CASSANDRA-15685, we acknowledged that there is an edge case
not handled. If we try to run
Preview repair isn't allowed to be run concurrently with IR. If we run it right
after IR and IR completes before the completion message is handled on the
participant; the below is impacted by this race condition.
{code:java}
nodetool repair foo bar
# does not look to be running concurrently with the above IR, since it was
notified of success
# but the commit message may not have been seen yet which cause this to be
concurrent with the above
nodetool repair foo bar --validate
{code}
See here
[https://github.com/apache/cassandra/blob/ec1808a34f9aa5ae0b956c1527828566f4ba2be5/src/java/org/apache/cassandra/repair/consistent/CoordinatorSession.java#L234-L244].
We send the message but don't make sure its ack on.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15929) Startup fails with NoClassDefFoundError after CASSANDRA-15884
[
https://issues.apache.org/jira/browse/CASSANDRA-15929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-15929:
--
Description:
CASSANDRA-15884 added logic to detect if the native lib of lz4 was properly
loaded and to log a warning if this was not the case, it seems that the catch
for this didn’t handle the class loading issue that also happens in this case.
{code}
ERROR [main] 2020-07-07 22:49:32,560 CassandraDaemon.java:800 - Exception
encountered during startup
java.lang.NoClassDefFoundError: Could not initialize class
net.jpountz.lz4.LZ4JNI
at net.jpountz.lz4.LZ4JNICompressor.compress(LZ4JNICompressor.java:36)
at net.jpountz.lz4.LZ4Factory.(LZ4Factory.java:207)
at net.jpountz.lz4.LZ4Factory.instance(LZ4Factory.java:51)
at net.jpountz.lz4.LZ4Factory.nativeInstance(LZ4Factory.java:91)
at
org.apache.cassandra.service.StartupChecks.lambda$static$0(StartupChecks.java:149)
at
org.apache.cassandra.service.StartupChecks.verify(StartupChecks.java:127)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:211)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:661)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:778)
{code}
After patching, the following is the behavior
WARN [main] 2020-07-07 22:50:45,018 StartupChecks.java:153 - lz4-java was
unable to load native libraries; this will lower the performance of lz4
(network/sstables/etc.): Could not initialize class net.jpountz.lz4.LZ4JNI
was:
CASSANDRA-15884 added logic to detect if the native lib of lz4 was properly
loaded and to log a warning if this was not the case, it seems that the catch
for this didn’t handle the class loading issue that also happens in this case.
ERROR [main] 2020-07-07 22:49:32,560 CassandraDaemon.java:800 - Exception
encountered during startup
java.lang.NoClassDefFoundError: Could not initialize class
net.jpountz.lz4.LZ4JNI
at net.jpountz.lz4.LZ4JNICompressor.compress(LZ4JNICompressor.java:36)
at net.jpountz.lz4.LZ4Factory.(LZ4Factory.java:207)
at net.jpountz.lz4.LZ4Factory.instance(LZ4Factory.java:51)
at net.jpountz.lz4.LZ4Factory.nativeInstance(LZ4Factory.java:91)
at
org.apache.cassandra.service.StartupChecks.lambda$static$0(StartupChecks.java:149)
at
org.apache.cassandra.service.StartupChecks.verify(StartupChecks.java:127)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:211)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:661)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:778)
After patching, the following is the behavior
WARN [main] 2020-07-07 22:50:45,018 StartupChecks.java:153 - lz4-java was
unable to load native libraries; this will lower the performance of lz4
(network/sstables/etc.): Could not initialize class net.jpountz.lz4.LZ4JNI
> Startup fails with NoClassDefFoundError after CASSANDRA-15884
> -
>
> Key: CASSANDRA-15929
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15929
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Startup and Shutdown
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> CASSANDRA-15884 added logic to detect if the native lib of lz4 was properly
> loaded and to log a warning if this was not the case, it seems that the catch
> for this didn’t handle the class loading issue that also happens in this case.
> {code}
> ERROR [main] 2020-07-07 22:49:32,560 CassandraDaemon.java:800 - Exception
> encountered during startup
> java.lang.NoClassDefFoundError: Could not initialize class
> net.jpountz.lz4.LZ4JNI
>at net.jpountz.lz4.LZ4JNICompressor.compress(LZ4JNICompressor.java:36)
>at net.jpountz.lz4.LZ4Factory.(LZ4Factory.java:207)
>at net.jpountz.lz4.LZ4Factory.instance(LZ4Factory.java:51)
>at net.jpountz.lz4.LZ4Factory.nativeInstance(LZ4Factory.java:91)
>at
> org.apache.cassandra.service.StartupChecks.lambda$static$0(StartupChecks.java:149)
>at
> org.apache.cassandra.service.StartupChecks.verify(StartupChecks.java:127)
>at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:211)
>at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:661)
>at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:778)
> {code}
> After patching, the following is the behavior
> WARN [main] 2020-07-07 22:50:45,018 StartupChecks.java:153 - lz4-java was
> unable to load native libraries; this will lower the performance of lz4
>
[jira] [Updated] (CASSANDRA-15929) Startup fails with NoClassDefFoundError after CASSANDRA-15884
[ https://issues.apache.org/jira/browse/CASSANDRA-15929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-15929: -- Test and Documentation Plan: manual testing using glibc 2.12 Status: Patch Available (was: Open) > Startup fails with NoClassDefFoundError after CASSANDRA-15884 > - > > Key: CASSANDRA-15929 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15929 > Project: Cassandra > Issue Type: Bug > Components: Local/Startup and Shutdown >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-alpha > > > CASSANDRA-15884 added logic to detect if the native lib of lz4 was properly > loaded and to log a warning if this was not the case, it seems that the catch > for this didn’t handle the class loading issue that also happens in this case. > ERROR [main] 2020-07-07 22:49:32,560 CassandraDaemon.java:800 - Exception > encountered during startup > java.lang.NoClassDefFoundError: Could not initialize class > net.jpountz.lz4.LZ4JNI >at net.jpountz.lz4.LZ4JNICompressor.compress(LZ4JNICompressor.java:36) >at net.jpountz.lz4.LZ4Factory.(LZ4Factory.java:207) >at net.jpountz.lz4.LZ4Factory.instance(LZ4Factory.java:51) >at net.jpountz.lz4.LZ4Factory.nativeInstance(LZ4Factory.java:91) >at > org.apache.cassandra.service.StartupChecks.lambda$static$0(StartupChecks.java:149) >at > org.apache.cassandra.service.StartupChecks.verify(StartupChecks.java:127) >at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:211) >at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:661) >at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:778) > After patching, the following is the behavior > WARN [main] 2020-07-07 22:50:45,018 StartupChecks.java:153 - lz4-java was > unable to load native libraries; this will lower the performance of lz4 > (network/sstables/etc.): Could not initialize class net.jpountz.lz4.LZ4JNI -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15929) Startup fails with NoClassDefFoundError after CASSANDRA-15884
[ https://issues.apache.org/jira/browse/CASSANDRA-15929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-15929: -- Bug Category: Parent values: Availability(12983)Level 1 values: Process Crash(12992) Complexity: Low Hanging Fruit Discovered By: Adhoc Test Fix Version/s: 4.0-alpha Severity: Normal Status: Open (was: Triage Needed) > Startup fails with NoClassDefFoundError after CASSANDRA-15884 > - > > Key: CASSANDRA-15929 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15929 > Project: Cassandra > Issue Type: Bug > Components: Local/Startup and Shutdown >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-alpha > > > CASSANDRA-15884 added logic to detect if the native lib of lz4 was properly > loaded and to log a warning if this was not the case, it seems that the catch > for this didn’t handle the class loading issue that also happens in this case. > ERROR [main] 2020-07-07 22:49:32,560 CassandraDaemon.java:800 - Exception > encountered during startup > java.lang.NoClassDefFoundError: Could not initialize class > net.jpountz.lz4.LZ4JNI >at net.jpountz.lz4.LZ4JNICompressor.compress(LZ4JNICompressor.java:36) >at net.jpountz.lz4.LZ4Factory.(LZ4Factory.java:207) >at net.jpountz.lz4.LZ4Factory.instance(LZ4Factory.java:51) >at net.jpountz.lz4.LZ4Factory.nativeInstance(LZ4Factory.java:91) >at > org.apache.cassandra.service.StartupChecks.lambda$static$0(StartupChecks.java:149) >at > org.apache.cassandra.service.StartupChecks.verify(StartupChecks.java:127) >at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:211) >at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:661) >at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:778) > After patching, the following is the behavior > WARN [main] 2020-07-07 22:50:45,018 StartupChecks.java:153 - lz4-java was > unable to load native libraries; this will lower the performance of lz4 > (network/sstables/etc.): Could not initialize class net.jpountz.lz4.LZ4JNI -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15929) Startup fails with NoClassDefFoundError after CASSANDRA-15884
David Capwell created CASSANDRA-15929: - Summary: Startup fails with NoClassDefFoundError after CASSANDRA-15884 Key: CASSANDRA-15929 URL: https://issues.apache.org/jira/browse/CASSANDRA-15929 Project: Cassandra Issue Type: Bug Components: Local/Startup and Shutdown Reporter: David Capwell Assignee: David Capwell CASSANDRA-15884 added logic to detect if the native lib of lz4 was properly loaded and to log a warning if this was not the case, it seems that the catch for this didn’t handle the class loading issue that also happens in this case. ERROR [main] 2020-07-07 22:49:32,560 CassandraDaemon.java:800 - Exception encountered during startup java.lang.NoClassDefFoundError: Could not initialize class net.jpountz.lz4.LZ4JNI at net.jpountz.lz4.LZ4JNICompressor.compress(LZ4JNICompressor.java:36) at net.jpountz.lz4.LZ4Factory.(LZ4Factory.java:207) at net.jpountz.lz4.LZ4Factory.instance(LZ4Factory.java:51) at net.jpountz.lz4.LZ4Factory.nativeInstance(LZ4Factory.java:91) at org.apache.cassandra.service.StartupChecks.lambda$static$0(StartupChecks.java:149) at org.apache.cassandra.service.StartupChecks.verify(StartupChecks.java:127) at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:211) at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:661) at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:778) After patching, the following is the behavior WARN [main] 2020-07-07 22:50:45,018 StartupChecks.java:153 - lz4-java was unable to load native libraries; this will lower the performance of lz4 (network/sstables/etc.): Could not initialize class net.jpountz.lz4.LZ4JNI -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15920) SimpleQueryResult should contain client warnings
[
https://issues.apache.org/jira/browse/CASSANDRA-15920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-15920:
--
Reviewers: David Capwell, David Capwell (was: David Capwell)
David Capwell, David Capwell
Status: Review In Progress (was: Patch Available)
Left comments in PR.
My main question is on the paging setup. I don't mind if this is added later,
but the `QueryResult` that is used for paging doesn't have changes to add
warning.
Overall I am +1 to this. Please add the builder warning support, and will
leave up to you on pager in scope or not for this JIRA.
> SimpleQueryResult should contain client warnings
>
>
> Key: CASSANDRA-15920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15920
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-beta
>
>
> There isn't a way to get query warnings generated via \{{ClientWarn}} using
> the existing \{{ICoordinator}} interface. The simplest possible change here
> would be to...
> 1.) Make sure we activate warning collection before queries start.
> 2.) Rip warnings out of \{{ClientWarn}} when queries return.
> 3.) Include those warnings when we create \{{SimpleQueryResult}}.
> {\{QueryResult}} might not need to expose the warnings, given not all its
> implementations would be expected to have them, but we could just return an
> empty list of warnings by default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15928) Throw FSWriteError upon write failures in order to apply DiskFailurePolicy
Yifan Cai created CASSANDRA-15928: - Summary: Throw FSWriteError upon write failures in order to apply DiskFailurePolicy Key: CASSANDRA-15928 URL: https://issues.apache.org/jira/browse/CASSANDRA-15928 Project: Cassandra Issue Type: Bug Components: Local/Other Reporter: Yifan Cai Assignee: Yifan Cai Active handling of FSError in AbstractLocalAwareExecutorService was introduced in CASSANDRA-14993. However, in the current code base, there are places that catch the IOException but not wrap it with FSError, hence not triggering the DiskFailurePolicy. Especially when the bad disk no longer permits write operations, it could leads to the mutation stage backup. Therefore I propose to fix the IOException handling in those cases. >From the code inspection, those 6 places are current not re-throwing an >IOException with FSWriteError. # org.apache.cassandra.triggers.CustomClassLoader#addClassPath throws IOError. Invoked in TriggerExecutor's constructor # org.apache.cassandra.io.util.FileUtils#renameWithConfirm throws RuntimeException # org.apache.cassandra.io.util.FileUtils#truncate throws RuntimeException # org.apache.cassandra.io.util.FileUtils#write throws RuntimeException # org.apache.cassandra.db.compaction.LeveledManifest#sendBackToL0 throws RuntimeException. Invokes rewriteSSTableMetadata # org.apache.cassandra.io.sstable.SSTableHeaderFix#writeNewMetadata throws RuntimeException. Invokes rewriteSSTableMetadata -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15920) SimpleQueryResult should contain client warnings
[
https://issues.apache.org/jira/browse/CASSANDRA-15920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-15920:
Test and Documentation Plan: Additional cases in {{QueryResultTest}} to
ensure that warnings behave as expected, even w/ delegation.
Status: Patch Available (was: Open)
Pull request
[here|https://github.com/apache/cassandra-in-jvm-dtest-api/pull/12].
> SimpleQueryResult should contain client warnings
>
>
> Key: CASSANDRA-15920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15920
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-beta
>
>
> There isn't a way to get query warnings generated via \{{ClientWarn}} using
> the existing \{{ICoordinator}} interface. The simplest possible change here
> would be to...
> 1.) Make sure we activate warning collection before queries start.
> 2.) Rip warnings out of \{{ClientWarn}} when queries return.
> 3.) Include those warnings when we create \{{SimpleQueryResult}}.
> {\{QueryResult}} might not need to expose the warnings, given not all its
> implementations would be expected to have them, but we could just return an
> empty list of warnings by default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15920) SimpleQueryResult should contain client warnings
[
https://issues.apache.org/jira/browse/CASSANDRA-15920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CASSANDRA-15920:
---
Labels: pull-request-available (was: )
> SimpleQueryResult should contain client warnings
>
>
> Key: CASSANDRA-15920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15920
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-beta
>
>
> There isn't a way to get query warnings generated via \{{ClientWarn}} using
> the existing \{{ICoordinator}} interface. The simplest possible change here
> would be to...
> 1.) Make sure we activate warning collection before queries start.
> 2.) Rip warnings out of \{{ClientWarn}} when queries return.
> 3.) Include those warnings when we create \{{SimpleQueryResult}}.
> {\{QueryResult}} might not need to expose the warnings, given not all its
> implementations would be expected to have them, but we could just return an
> empty list of warnings by default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15920) SimpleQueryResult should contain client warnings
[
https://issues.apache.org/jira/browse/CASSANDRA-15920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-15920:
Change Category: Quality Assurance
Complexity: Normal
Fix Version/s: 4.0-beta
Status: Open (was: Triage Needed)
> SimpleQueryResult should contain client warnings
>
>
> Key: CASSANDRA-15920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15920
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 4.0-beta
>
>
> There isn't a way to get query warnings generated via \{{ClientWarn}} using
> the existing \{{ICoordinator}} interface. The simplest possible change here
> would be to...
> 1.) Make sure we activate warning collection before queries start.
> 2.) Rip warnings out of \{{ClientWarn}} when queries return.
> 3.) Include those warnings when we create \{{SimpleQueryResult}}.
> {\{QueryResult}} might not need to expose the warnings, given not all its
> implementations would be expected to have them, but we could just return an
> empty list of warnings by default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15920) SimpleQueryResult should contain client warnings
[
https://issues.apache.org/jira/browse/CASSANDRA-15920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe reassigned CASSANDRA-15920:
---
Assignee: Caleb Rackliffe
> SimpleQueryResult should contain client warnings
>
>
> Key: CASSANDRA-15920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15920
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
>
> There isn't a way to get query warnings generated via \{{ClientWarn}} using
> the existing \{{ICoordinator}} interface. The simplest possible change here
> would be to...
> 1.) Make sure we activate warning collection before queries start.
> 2.) Rip warnings out of \{{ClientWarn}} when queries return.
> 3.) Include those warnings when we create \{{SimpleQueryResult}}.
> {\{QueryResult}} might not need to expose the warnings, given not all its
> implementations would be expected to have them, but we could just return an
> empty list of warnings by default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15881) Flaky unit test: SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex
[
https://issues.apache.org/jira/browse/CASSANDRA-15881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-15881:
Fix Version/s: 4.0-rc
> Flaky unit test: SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex
> ---
>
> Key: CASSANDRA-15881
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15881
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Andres de la Peña
>Priority: Normal
> Fix For: 3.11.7, 4.0-rc
>
>
> The unit test {{SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex}}
> seems to be flaky in 3.11, as it can be seen in
> [cassandra-ci|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testInsertingIncorrectValuesIntoAgeIndex/]
> and also locally. Trunk doesn't seem to be affected.
> {{SASIIndexTest.testIndexMemtableSwitching}} is [also
> flaky|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testIndexMemtableSwitching/],
> although I haven't been able to reproduce it running it separately, but only
> when running the entire {{SASIIndexTest}}.
> These could have been introduced when merging up CASSANDRA-15778, since they
> failed for their [CircleCI
> run|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra/17/workflows/0442eebe-c764-41c5-ba06-6617bcb9fc5f/jobs/2213],
> and they didn't seem to fail before that, or at least I cannot reproduce
> them locally with the previous commit.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13791) unable to install apache-cassandra-3.11.0 in linux box
[ https://issues.apache.org/jira/browse/CASSANDRA-13791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153083#comment-17153083 ] david cherepov edited comment on CASSANDRA-13791 at 7/7/20, 8:41 PM: - I'm getting this same error with 3.11.6 on centos 7. I'm using adoptopenjdk's 1.8 jdk. I'll try another jdk implementation but i can't use sun jdk for this. was (Author: dcherpov): I'm getting this same error with 3.11.6 on centos 8. I'm using adoptopenjdk's 1.8 jdk. I'll try another jdk implementation but i can't use sun jdk for this. > unable to install apache-cassandra-3.11.0 in linux box > --- > > Key: CASSANDRA-13791 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13791 > Project: Cassandra > Issue Type: Bug > Components: Local/Config > Environment: Linux >Reporter: rajasekhar >Priority: Urgent > Fix For: 3.11.0 > > Attachments: log_cassdb.txt > > > While installing the Cassandra in linux serverr, I am getting below error . > could you please look int it and provide suggestions. PFA atttached log for > more information. > [cassdb@alsc_dev_db bin]$ sh > /u01/Cassandra_home/apache-cassandra-3.11.0/bin/cassandra > Error: > ERROR [main] 2017-08-23 09:48:21,467 NativeLibraryLinux.java:62 - Failed to > link the C library against JNA. Native methods will be unavailable. > java.lang.UnsatisfiedLinkError: > /tmp/jna--1367560132/jna4859101025087222330.tmp: /lib64/libc.so.6: version > `GLIBC_2.14' not found (required by > /tmp/jna--1367560132/jna4859101025087222330.tmp) > at java.lang.ClassLoader$NativeLibrary.load(Native Method) > ~[na:1.8.0_71] > at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1938) > ~[na:1.8.0_71] >at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1821) > ~[na:1.8.0_71] > at java.lang.Runtime.load0(Runtime.java:809) ~[na:1.8.0_71] > at java.lang.System.load(System.java:1086) ~[na:1.8.0_71] > at > com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:947) > ~[jna-4.4.0.jar:4.4.0 (b0)] > at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:922) > ~[jna-4.4.0.jar:4.4.0 (b0)] > at com.sun.jna.Native.(Native.java:190) ~[jna-4.4.0.jar:4.4.0 > (b0)] > at > org.apache.cassandra.utils.NativeLibraryLinux.(NativeLibraryLinux.java:53) > ~[apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.utils.NativeLibrary.(NativeLibrary.java:93) > [apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:196) > [apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:600) > [apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:689) > [apache-cassandra-3.11.0.jar:3.11.0] > WARN [main] 2017-08-23 09:48:21,468 StartupChecks.java:127 - jemalloc shared > library could not be preloaded to speed up memory allocations > WARN [main] 2017-08-23 09:48:21,469 StartupChecks.java:160 - JMX is not > enabled to receive remote connections. Please see cassandra-env.sh for more > info. > ERROR [main] 2017-08-23 09:48:21,470 CassandraDaemon.java:706 - The native > library could not be initialized properly -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13791) unable to install apache-cassandra-3.11.0 in linux box
[ https://issues.apache.org/jira/browse/CASSANDRA-13791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] david cherepov updated CASSANDRA-13791: --- Status: Open (was: Resolved) I'm getting this same error with 3.11.6 on centos 8. I'm using adoptopenjdk's 1.8 jdk. I'll try another jdk implementation but i can't use sun jdk for this. > unable to install apache-cassandra-3.11.0 in linux box > --- > > Key: CASSANDRA-13791 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13791 > Project: Cassandra > Issue Type: Bug > Components: Local/Config > Environment: Linux >Reporter: rajasekhar >Priority: Urgent > Fix For: 3.11.0 > > Attachments: log_cassdb.txt > > > While installing the Cassandra in linux serverr, I am getting below error . > could you please look int it and provide suggestions. PFA atttached log for > more information. > [cassdb@alsc_dev_db bin]$ sh > /u01/Cassandra_home/apache-cassandra-3.11.0/bin/cassandra > Error: > ERROR [main] 2017-08-23 09:48:21,467 NativeLibraryLinux.java:62 - Failed to > link the C library against JNA. Native methods will be unavailable. > java.lang.UnsatisfiedLinkError: > /tmp/jna--1367560132/jna4859101025087222330.tmp: /lib64/libc.so.6: version > `GLIBC_2.14' not found (required by > /tmp/jna--1367560132/jna4859101025087222330.tmp) > at java.lang.ClassLoader$NativeLibrary.load(Native Method) > ~[na:1.8.0_71] > at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1938) > ~[na:1.8.0_71] >at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1821) > ~[na:1.8.0_71] > at java.lang.Runtime.load0(Runtime.java:809) ~[na:1.8.0_71] > at java.lang.System.load(System.java:1086) ~[na:1.8.0_71] > at > com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:947) > ~[jna-4.4.0.jar:4.4.0 (b0)] > at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:922) > ~[jna-4.4.0.jar:4.4.0 (b0)] > at com.sun.jna.Native.(Native.java:190) ~[jna-4.4.0.jar:4.4.0 > (b0)] > at > org.apache.cassandra.utils.NativeLibraryLinux.(NativeLibraryLinux.java:53) > ~[apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.utils.NativeLibrary.(NativeLibrary.java:93) > [apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:196) > [apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:600) > [apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:689) > [apache-cassandra-3.11.0.jar:3.11.0] > WARN [main] 2017-08-23 09:48:21,468 StartupChecks.java:127 - jemalloc shared > library could not be preloaded to speed up memory allocations > WARN [main] 2017-08-23 09:48:21,469 StartupChecks.java:160 - JMX is not > enabled to receive remote connections. Please see cassandra-env.sh for more > info. > ERROR [main] 2017-08-23 09:48:21,470 CassandraDaemon.java:706 - The native > library could not be initialized properly -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15881) Flaky unit test: SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex
[
https://issues.apache.org/jira/browse/CASSANDRA-15881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153078#comment-17153078
]
Andres de la Peña commented on CASSANDRA-15881:
---
[~maedhroz] I'm not aware of any test using mixed sstable versions in
{{SASIIndexTest}}, but I might be wrong, I haven't looked carefully into it.
> Flaky unit test: SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex
> ---
>
> Key: CASSANDRA-15881
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15881
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Andres de la Peña
>Priority: Normal
> Fix For: 3.11.7
>
>
> The unit test {{SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex}}
> seems to be flaky in 3.11, as it can be seen in
> [cassandra-ci|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testInsertingIncorrectValuesIntoAgeIndex/]
> and also locally. Trunk doesn't seem to be affected.
> {{SASIIndexTest.testIndexMemtableSwitching}} is [also
> flaky|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testIndexMemtableSwitching/],
> although I haven't been able to reproduce it running it separately, but only
> when running the entire {{SASIIndexTest}}.
> These could have been introduced when merging up CASSANDRA-15778, since they
> failed for their [CircleCI
> run|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra/17/workflows/0442eebe-c764-41c5-ba06-6617bcb9fc5f/jobs/2213],
> and they didn't seem to fail before that, or at least I cannot reproduce
> them locally with the previous commit.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15925) Jenkins pipeline can copy wrong test report artefacts from stage builds
[ https://issues.apache.org/jira/browse/CASSANDRA-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15925: --- Summary: Jenkins pipeline can copy wrong test report artefacts from stage builds (was: Jenkins pipeline can copy wrong test report artefacts from dtest stage builds) > Jenkins pipeline can copy wrong test report artefacts from stage builds > --- > > Key: CASSANDRA-15925 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15925 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 4.0-rc > > > Spotted in > https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/196/console > Looks like copyArtifact will need to be specific to a build. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15925) Jenkins pipeline can copy wrong test report artefacts from dtest stage builds
[ https://issues.apache.org/jira/browse/CASSANDRA-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153077#comment-17153077 ] Michael Semb Wever commented on CASSANDRA-15925: patches - https://github.com/apache/cassandra-builds/compare/master...thelastpickle:mck/jenkins_copy_artefacts_with_build_number - https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_15925 > Jenkins pipeline can copy wrong test report artefacts from dtest stage builds > - > > Key: CASSANDRA-15925 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15925 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 4.0-rc > > > Spotted in > https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/196/console > Looks like copyArtifact will need to be specific to a build. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15927) Add option to disable compaction at startup
[ https://issues.apache.org/jira/browse/CASSANDRA-15927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-15927: - Fix Version/s: 4.0-alpha5 Source Control Link: https://github.com/apache/cassandra/commit/c21cde119d12d0c5bec67c554f298874b3a70b8e Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed, thanks! We've been needing this for a while. > Add option to disable compaction at startup > --- > > Key: CASSANDRA-15927 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15927 > Project: Cassandra > Issue Type: Improvement > Components: Local/Compaction >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-alpha5 > > > There are cases in which its desirable to avoid compaction starting up on > startup and to be enabled after. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15927) Add option to disable compaction at startup
[ https://issues.apache.org/jira/browse/CASSANDRA-15927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-15927: - Status: Ready to Commit (was: Review In Progress) > Add option to disable compaction at startup > --- > > Key: CASSANDRA-15927 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15927 > Project: Cassandra > Issue Type: Improvement > Components: Local/Compaction >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > > There are cases in which its desirable to avoid compaction starting up on > startup and to be enabled after. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15927) Add option to disable compaction at startup
[ https://issues.apache.org/jira/browse/CASSANDRA-15927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-15927: - Reviewers: Brandon Williams, Brandon Williams (was: Brandon Williams) Brandon Williams, Brandon Williams Status: Review In Progress (was: Patch Available) > Add option to disable compaction at startup > --- > > Key: CASSANDRA-15927 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15927 > Project: Cassandra > Issue Type: Improvement > Components: Local/Compaction >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > > There are cases in which its desirable to avoid compaction starting up on > startup and to be enabled after. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Add option to disable compaction at startup
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new c21cde1 Add option to disable compaction at startup
c21cde1 is described below
commit c21cde119d12d0c5bec67c554f298874b3a70b8e
Author: David Capwell
AuthorDate: Tue Jul 7 10:25:06 2020 -0700
Add option to disable compaction at startup
Patch by David Capwell, reviewed by brandonwilliams for CASSANDRA-15927
---
CHANGES.txt | 1 +
src/java/org/apache/cassandra/config/Config.java | 2 ++
src/java/org/apache/cassandra/config/DatabaseDescriptor.java | 5 +
src/java/org/apache/cassandra/service/CassandraDaemon.java | 9 -
4 files changed, 16 insertions(+), 1 deletion(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index d382650..9bca849 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0-alpha5
+ * Add option to disable compaction at startup (CASSANDRA-15927)
* FBUtilities.getJustLocalAddress falls back to lo ip on misconfigured nodes
(CASSANDRA-15901)
* Close channel and reduce buffer allocation during entire sstable streaming
with SSL (CASSANDRA-15900)
* Prune expired messages less frequently in internode messaging
(CASSANDRA-15700)
diff --git a/src/java/org/apache/cassandra/config/Config.java
b/src/java/org/apache/cassandra/config/Config.java
index 34ac049..63cfff4 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -521,6 +521,8 @@ public class Config
public volatile boolean check_for_duplicate_rows_during_reads = true;
public volatile boolean check_for_duplicate_rows_during_compaction = true;
+public boolean autocompaction_on_startup_enabled =
Boolean.parseBoolean(System.getProperty("cassandra.autocompaction_on_startup_enabled",
"true"));
+
/**
* Client mode means that the process is a pure client, that uses C* code
base but does
* not read or write local C* database files.
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index 9cc2a3b..5f94710 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -3154,4 +3154,9 @@ public class DatabaseDescriptor
{
conf.range_tombstone_list_growth_factor = resizeFactor;
}
+
+public static boolean getAutocompactionOnStartupEnabled()
+{
+return conf.autocompaction_on_startup_enabled;
+}
}
diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java
b/src/java/org/apache/cassandra/service/CassandraDaemon.java
index c7591d5..10d9cea 100644
--- a/src/java/org/apache/cassandra/service/CassandraDaemon.java
+++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java
@@ -404,7 +404,14 @@ public class CassandraDaemon
store.reload(); //reload CFs in case there was a change of
disk boundaries
if (store.getCompactionStrategyManager().shouldBeEnabled())
{
-store.enableAutoCompaction();
+if
(DatabaseDescriptor.getAutocompactionOnStartupEnabled())
+{
+store.enableAutoCompaction();
+}
+else
+{
+logger.info("Not enabling compaction for {}.{};
autocompaction_on_startup_enabled is set to false", store.keyspace.getName(),
store.name);
+}
}
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15889) Debian package fails to download on Arm-based hosts
[
https://issues.apache.org/jira/browse/CASSANDRA-15889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152964#comment-17152964
]
Matt Davis commented on CASSANDRA-15889:
Thanks [~mck], that's very helpful. Does {{dpkg-buildpackage}} [need to be
called
here|https://github.com/apache/cassandra-builds/blob/master/docker/build-debs.sh#L120]
with -A? See
https://man7.org/linux/man-pages/man1/dpkg-buildpackage.1.html#OPTIONS
You would think that's not necessary given that the control file specifies
{{all}}, but I wonder if it would solve the issue.
> Debian package fails to download on Arm-based hosts
> ---
>
> Key: CASSANDRA-15889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15889
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Matt Davis
>Priority: Normal
> Labels: ARM, debian
>
> Following the first three steps of the [Debian install
> process|https://cassandra.apache.org/download/], after an apt-get update
> you'll see this line:
> {code:bash}
> $ sudo apt-get update
> ...
> N: Skipping acquire of configured file 'main/binary-arm64/Packages' as
> repository 'https://downloads.apache.org/cassandra/debian 311x InRelease'
> doesn't support architecture 'arm64'
> {code}
> Checking the [Debian
> repo|https://dl.bintray.com/apache/cassandra/dists/311x/main/] confirms there
> is no aarch64 variant available.
> Should you then attempt to install Cassandra:
> {code:bash}
> $ sudo apt-get install cassandra
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> Package cassandra is not available, but is referred to by another package.
> This may mean that the package is missing, has been obsoleted, or
> is only available from another source
> E: Package 'cassandra' has no installation candidate
> {code}
> The Redhat RPM contains a "noarch" arch type, so it will download on any
> host. (Cassandra does not use separate binaries/releases for different
> architectures, so this seems to be the correct approach, but adding an
> aarch64 variant would also suffice.)
> Note that there is a workaround available: if you specify "amd64" as the arch
> for the source, it downloads and runs on Arm without issue:
> {code:bash}
> deb [arch=amd64] https://downloads.apache.org/cassandra/debian 311x main
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15907) Operational Improvements & Hardening for Replica Filtering Protection
[
https://issues.apache.org/jira/browse/CASSANDRA-15907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152248#comment-17152248
]
Caleb Rackliffe edited comment on CASSANDRA-15907 at 7/7/20, 6:10 PM:
--
[~jwest] I've hopefully addressed the points from [~adelapena]'s first round of
review, so I think this is officially ready for a second reviewer.
3.0: [patch|https://github.com/apache/cassandra/pull/659],
[CircleCI|https://app.circleci.com/pipelines/github/maedhroz/cassandra/22/workflows/d272c9e6-1db6-472f-93d9-f2715a25ef97]
If we're happy with the implementation, the next step will be to do some basic
stress testing.
Note: CASSANDRA-15881 is visible in the unit tests results.
was (Author: maedhroz):
[~jwest] I've hopefully addressed the points from [~adelapena]'s first round of
review, so I think this is officially ready for a second reviewer.
3.0: [patch|https://github.com/apache/cassandra/pull/659],
[CircleCI|https://app.circleci.com/pipelines/github/maedhroz/cassandra/22/workflows/d272c9e6-1db6-472f-93d9-f2715a25ef97]
If we're happy with the implementation, the next step will be to do some basic
stress testing.
> Operational Improvements & Hardening for Replica Filtering Protection
> -
>
> Key: CASSANDRA-15907
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15907
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Coordination, Feature/2i Index
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: 2i, memory
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> CASSANDRA-8272 uses additional space on the heap to ensure correctness for 2i
> and filtering queries at consistency levels above ONE/LOCAL_ONE. There are a
> few things we should follow up on, however, to make life a bit easier for
> operators and generally de-risk usage:
> (Note: Line numbers are based on {{trunk}} as of
> {{3cfe3c9f0dcf8ca8b25ad111800a21725bf152cb}}.)
> *Minor Optimizations*
> * {{ReplicaFilteringProtection:114}} - Given we size them up-front, we may be
> able to use simple arrays instead of lists for {{rowsToFetch}} and
> {{originalPartitions}}. Alternatively (or also), we may be able to null out
> references in these two collections more aggressively. (ex. Using
> {{ArrayList#set()}} instead of {{get()}} in {{queryProtectedPartitions()}},
> assuming we pass {{toFetch}} as an argument to {{querySourceOnKey()}}.)
> * {{ReplicaFilteringProtection:323}} - We may be able to use
> {{EncodingStats.merge()}} and remove the custom {{stats()}} method.
> * {{DataResolver:111 & 228}} - Cache an instance of
> {{UnaryOperator#identity()}} instead of creating one on the fly.
> * {{ReplicaFilteringProtection:217}} - We may be able to scatter/gather
> rather than serially querying every row that needs to be completed. This
> isn't a clear win perhaps, given it targets the latency of single queries and
> adds some complexity. (Certainly a decent candidate to kick even out of this
> issue.)
> *Documentation and Intelligibility*
> * There are a few places (CHANGES.txt, tracing output in
> {{ReplicaFilteringProtection}}, etc.) where we mention "replica-side
> filtering protection" (which makes it seem like the coordinator doesn't
> filter) rather than "replica filtering protection" (which sounds more like
> what we actually do, which is protect ourselves against incorrect replica
> filtering results). It's a minor fix, but would avoid confusion.
> * The method call chain in {{DataResolver}} might be a bit simpler if we put
> the {{repairedDataTracker}} in {{ResolveContext}}.
> *Testing*
> * I want to bite the bullet and get some basic tests for RFP (including any
> guardrails we might add here) onto the in-JVM dtest framework.
> *Guardrails*
> * As it stands, we don't have a way to enforce an upper bound on the memory
> usage of {{ReplicaFilteringProtection}} which caches row responses from the
> first round of requests. (Remember, these are later used to merged with the
> second round of results to complete the data for filtering.) Operators will
> likely need a way to protect themselves, i.e. simply fail queries if they hit
> a particular threshold rather than GC nodes into oblivion. (Having control
> over limits and page sizes doesn't quite get us there, because stale results
> _expand_ the number of incomplete results we must cache.) The fun question is
> how we do this, with the primary axes being scope (per-query, global, etc.)
> and granularity (per-partition, per-row, per-cell, actual heap usage, etc.).
> My starting disposition on the right trade-off between
> performance/complexity and accuracy is
[jira] [Commented] (CASSANDRA-15881) Flaky unit test: SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex
[
https://issues.apache.org/jira/browse/CASSANDRA-15881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152962#comment-17152962
]
Caleb Rackliffe commented on CASSANDRA-15881:
-
[~adelapena] RE CASSANDRA-15778, {{SASIIndexTest}} doesn't actually deal with
any tables with mixed SSTable versions, right?
> Flaky unit test: SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex
> ---
>
> Key: CASSANDRA-15881
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15881
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Andres de la Peña
>Priority: Normal
> Fix For: 3.11.7
>
>
> The unit test {{SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex}}
> seems to be flaky in 3.11, as it can be seen in
> [cassandra-ci|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testInsertingIncorrectValuesIntoAgeIndex/]
> and also locally. Trunk doesn't seem to be affected.
> {{SASIIndexTest.testIndexMemtableSwitching}} is [also
> flaky|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testIndexMemtableSwitching/],
> although I haven't been able to reproduce it running it separately, but only
> when running the entire {{SASIIndexTest}}.
> These could have been introduced when merging up CASSANDRA-15778, since they
> failed for their [CircleCI
> run|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra/17/workflows/0442eebe-c764-41c5-ba06-6617bcb9fc5f/jobs/2213],
> and they didn't seem to fail before that, or at least I cannot reproduce
> them locally with the previous commit.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15839) Warn or fail to start server when G1 is used and Xmn is set
[
https://issues.apache.org/jira/browse/CASSANDRA-15839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152954#comment-17152954
]
David Capwell commented on CASSANDRA-15839:
---
BTW the check for USING_G1 doesn't handle the case where you disable g1gc. In
a test with -Xmn defined and -XX:-UseG1GC, the following error stops C* from
starting
{code}
+++ echo 'It is not recommended to set -Xmn with the G1 garbage collector. See
comments for -Xmn in jvm-server.options for details.'
It is not recommended to set -Xmn with the G1 garbage collector. See comments
for -Xmn in jvm-server.options for details.
+++ exit 1
{code}
> Warn or fail to start server when G1 is used and Xmn is set
> ---
>
> Key: CASSANDRA-15839
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15839
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Startup and Shutdown
>Reporter: Jeremy Hanna
>Assignee: Anthony Grasso
>Priority: Normal
> Fix For: 4.0, 4.0-alpha5
>
>
> In jvm.options, we currently have a comment above where Xmn is set that says
> that you shouldn't set Xmn with G1 GC. That isn't enough - we should either
> warn in the logs or fail startup when they are set together.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15889) Debian package fails to download on Arm-based hosts
[
https://issues.apache.org/jira/browse/CASSANDRA-15889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152951#comment-17152951
]
Michael Semb Wever commented on CASSANDRA-15889:
[~mattsplat], see
-
https://github.com/apache/cassandra-builds/blob/master/build-scripts/cassandra-deb-packaging.sh
-
https://github.com/apache/cassandra-builds/blob/master/build-scripts/cassandra-rpm-packaging.sh
- https://github.com/apache/cassandra-builds/blob/master/docker/build-debs.sh
- https://github.com/apache/cassandra-builds/blob/master/docker/build-rpms.sh
The former two are what are run from the artefact builds in Jenkins.
The latter two are the scripts called inside the docker container the former
two, correspondingly, create.
> Debian package fails to download on Arm-based hosts
> ---
>
> Key: CASSANDRA-15889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15889
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Matt Davis
>Priority: Normal
> Labels: ARM, debian
>
> Following the first three steps of the [Debian install
> process|https://cassandra.apache.org/download/], after an apt-get update
> you'll see this line:
> {code:bash}
> $ sudo apt-get update
> ...
> N: Skipping acquire of configured file 'main/binary-arm64/Packages' as
> repository 'https://downloads.apache.org/cassandra/debian 311x InRelease'
> doesn't support architecture 'arm64'
> {code}
> Checking the [Debian
> repo|https://dl.bintray.com/apache/cassandra/dists/311x/main/] confirms there
> is no aarch64 variant available.
> Should you then attempt to install Cassandra:
> {code:bash}
> $ sudo apt-get install cassandra
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> Package cassandra is not available, but is referred to by another package.
> This may mean that the package is missing, has been obsoleted, or
> is only available from another source
> E: Package 'cassandra' has no installation candidate
> {code}
> The Redhat RPM contains a "noarch" arch type, so it will download on any
> host. (Cassandra does not use separate binaries/releases for different
> architectures, so this seems to be the correct approach, but adding an
> aarch64 variant would also suffice.)
> Note that there is a workaround available: if you specify "amd64" as the arch
> for the source, it downloads and runs on Arm without issue:
> {code:bash}
> deb [arch=amd64] https://downloads.apache.org/cassandra/debian 311x main
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15889) Debian package fails to download on Arm-based hosts
[
https://issues.apache.org/jira/browse/CASSANDRA-15889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152945#comment-17152945
]
Matt Davis edited comment on CASSANDRA-15889 at 7/7/20, 5:47 PM:
-
{quote}
Specifying only all indicates that the source package will only build
architecture-independent packages.
{quote}
That sounds correct to me - a single 'noarch' package. What process builds the
package from this source?
was (Author: mattsplat):
{quote}
Specifying only all indicates that the source package will only build
architecture-independent packages.
{quote}
That sounds correct to me. What process builds the package from this source?
> Debian package fails to download on Arm-based hosts
> ---
>
> Key: CASSANDRA-15889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15889
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Matt Davis
>Priority: Normal
> Labels: ARM, debian
>
> Following the first three steps of the [Debian install
> process|https://cassandra.apache.org/download/], after an apt-get update
> you'll see this line:
> {code:bash}
> $ sudo apt-get update
> ...
> N: Skipping acquire of configured file 'main/binary-arm64/Packages' as
> repository 'https://downloads.apache.org/cassandra/debian 311x InRelease'
> doesn't support architecture 'arm64'
> {code}
> Checking the [Debian
> repo|https://dl.bintray.com/apache/cassandra/dists/311x/main/] confirms there
> is no aarch64 variant available.
> Should you then attempt to install Cassandra:
> {code:bash}
> $ sudo apt-get install cassandra
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> Package cassandra is not available, but is referred to by another package.
> This may mean that the package is missing, has been obsoleted, or
> is only available from another source
> E: Package 'cassandra' has no installation candidate
> {code}
> The Redhat RPM contains a "noarch" arch type, so it will download on any
> host. (Cassandra does not use separate binaries/releases for different
> architectures, so this seems to be the correct approach, but adding an
> aarch64 variant would also suffice.)
> Note that there is a workaround available: if you specify "amd64" as the arch
> for the source, it downloads and runs on Arm without issue:
> {code:bash}
> deb [arch=amd64] https://downloads.apache.org/cassandra/debian 311x main
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15851) Add bytebuddy support for in-jvm dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152943#comment-17152943 ] David Capwell commented on CASSANDRA-15851: --- Patches LGTM, +1 to them The only failing test I see is trunk jvm 8: https://app.circleci.com/pipelines/github/krummas/cassandra/451/workflows/bb1125f6-f455-4a0b-863b-e50dc8a03036/jobs/3428/tests. testRestartNode[1: provision strategy=OneNetworkInterface] - org.apache.cassandra.distributed.test.TopologyChangeTest. I don't see any JIRA for TopologyChangeTest, but saw the test past on jvm 11 so feel its unrelated to this patch. > Add bytebuddy support for in-jvm dtests > --- > > Key: CASSANDRA-15851 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15851 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Labels: pull-request-available > Fix For: 4.0 > > > Old python dtests support byteman, but that is quite horrible to work with, > [bytebuddy|https://bytebuddy.net/#/] is much better, so we should add support > for that in the in-jvm dtests. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15889) Debian package fails to download on Arm-based hosts
[
https://issues.apache.org/jira/browse/CASSANDRA-15889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152945#comment-17152945
]
Matt Davis commented on CASSANDRA-15889:
{quote}
Specifying only all indicates that the source package will only build
architecture-independent packages.
{quote}
That sounds correct to me. What process builds the package from this source?
> Debian package fails to download on Arm-based hosts
> ---
>
> Key: CASSANDRA-15889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15889
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Matt Davis
>Priority: Normal
> Labels: ARM, debian
>
> Following the first three steps of the [Debian install
> process|https://cassandra.apache.org/download/], after an apt-get update
> you'll see this line:
> {code:bash}
> $ sudo apt-get update
> ...
> N: Skipping acquire of configured file 'main/binary-arm64/Packages' as
> repository 'https://downloads.apache.org/cassandra/debian 311x InRelease'
> doesn't support architecture 'arm64'
> {code}
> Checking the [Debian
> repo|https://dl.bintray.com/apache/cassandra/dists/311x/main/] confirms there
> is no aarch64 variant available.
> Should you then attempt to install Cassandra:
> {code:bash}
> $ sudo apt-get install cassandra
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> Package cassandra is not available, but is referred to by another package.
> This may mean that the package is missing, has been obsoleted, or
> is only available from another source
> E: Package 'cassandra' has no installation candidate
> {code}
> The Redhat RPM contains a "noarch" arch type, so it will download on any
> host. (Cassandra does not use separate binaries/releases for different
> architectures, so this seems to be the correct approach, but adding an
> aarch64 variant would also suffice.)
> Note that there is a workaround available: if you specify "amd64" as the arch
> for the source, it downloads and runs on Arm without issue:
> {code:bash}
> deb [arch=amd64] https://downloads.apache.org/cassandra/debian 311x main
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15927) Add option to disable compaction at startup
[ https://issues.apache.org/jira/browse/CASSANDRA-15927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-15927: -- Test and Documentation Plan: manual testing Status: Patch Available (was: Open) > Add option to disable compaction at startup > --- > > Key: CASSANDRA-15927 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15927 > Project: Cassandra > Issue Type: Improvement > Components: Local/Compaction >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > > There are cases in which its desirable to avoid compaction starting up on > startup and to be enabled after. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15927) Add option to disable compaction at startup
[ https://issues.apache.org/jira/browse/CASSANDRA-15927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-15927: -- Change Category: Operability Complexity: Low Hanging Fruit Status: Open (was: Triage Needed) > Add option to disable compaction at startup > --- > > Key: CASSANDRA-15927 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15927 > Project: Cassandra > Issue Type: Improvement > Components: Local/Compaction >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > > There are cases in which its desirable to avoid compaction starting up on > startup and to be enabled after. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15927) Add option to disable compaction at startup
David Capwell created CASSANDRA-15927: - Summary: Add option to disable compaction at startup Key: CASSANDRA-15927 URL: https://issues.apache.org/jira/browse/CASSANDRA-15927 Project: Cassandra Issue Type: Improvement Components: Local/Compaction Reporter: David Capwell Assignee: David Capwell There are cases in which its desirable to avoid compaction starting up on startup and to be enabled after. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15894) JAVA 8: test_multiple_repair - repair_tests.incremental_repair_test.TestIncRepair
[ https://issues.apache.org/jira/browse/CASSANDRA-15894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-15894: Resolution: Not A Problem Status: Resolved (was: Open) > JAVA 8: test_multiple_repair - > repair_tests.incremental_repair_test.TestIncRepair > - > > Key: CASSANDRA-15894 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15894 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-rc > > Attachments: test_multiple_repair_log.txt > > > JAVA 8: test_multiple_repair - > repair_tests.incremental_repair_test.TestIncRepair > Fails locally and in CircleCI: > https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/222/workflows/46515d14-9be4-4edb-8db4-5930312d2bfb/jobs/1329 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15894) JAVA 8: test_multiple_repair - repair_tests.incremental_repair_test.TestIncRepair
[ https://issues.apache.org/jira/browse/CASSANDRA-15894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152910#comment-17152910 ] Ekaterina Dimitrova commented on CASSANDRA-15894: - Thank you! Closing > JAVA 8: test_multiple_repair - > repair_tests.incremental_repair_test.TestIncRepair > - > > Key: CASSANDRA-15894 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15894 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-rc > > Attachments: test_multiple_repair_log.txt > > > JAVA 8: test_multiple_repair - > repair_tests.incremental_repair_test.TestIncRepair > Fails locally and in CircleCI: > https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/222/workflows/46515d14-9be4-4edb-8db4-5930312d2bfb/jobs/1329 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15894) JAVA 8: test_multiple_repair - repair_tests.incremental_repair_test.TestIncRepair
[ https://issues.apache.org/jira/browse/CASSANDRA-15894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-15894: Reviewers: Benjamin Lerer, Berenguer Blasi (was: Ekaterina Dimitrova) > JAVA 8: test_multiple_repair - > repair_tests.incremental_repair_test.TestIncRepair > - > > Key: CASSANDRA-15894 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15894 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-rc > > Attachments: test_multiple_repair_log.txt > > > JAVA 8: test_multiple_repair - > repair_tests.incremental_repair_test.TestIncRepair > Fails locally and in CircleCI: > https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/222/workflows/46515d14-9be4-4edb-8db4-5930312d2bfb/jobs/1329 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15926) When schema does not match sstable serialization header can fail when merging rows
[
https://issues.apache.org/jira/browse/CASSANDRA-15926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-15926:
--
Change Category: Semantic
Complexity: Normal
Status: Open (was: Triage Needed)
> When schema does not match sstable serialization header can fail when merging
> rows
> --
>
> Key: CASSANDRA-15926
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15926
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema, Local/SSTable
>Reporter: David Capwell
>Priority: Normal
>
> If the table schema had timeuuid and then an alter is done to change the
> column to uuid, then the on-disk serialization header won’t match the table
> schema; this is problematic as there isn’t logic to migrate the column from
> the old type to the new type.
> The ALTER TABLE command does not directly allow this type of schema change
> (though possible before 3.0.11, see CASSANDRA-12443), but this is possible if
> the following is done
> {code}
> INSERT INTO system_schema.columns (keyspace_name, table_name, column_name,
> clustering_order, column_name_bytes, kind, position, type) VALUES (‘ks',
> ’table', ‘col', 'none', bytes, 'regular', -1, 'uuid’);
> ALTER TABLE ks.table WITH comment = 'something’; — this will trigger the
> schema to migrate across the cluster
> {code}
> The below is a test to show this
> {code}
> package org.apache.cassandra.distributed.test;
> import java.util.UUID;
> import org.junit.Test;
> import org.apache.cassandra.distributed.Cluster;
> import org.apache.cassandra.distributed.impl.IInvokableInstance;
> import org.apache.cassandra.utils.UUIDGen;
> public class AlterWithOnDiskMismatch extends DistributedTestBase
> {
> @Test
> public void alterTypeMixedSSTablesNoConflict() throws Throwable
> {
> try (Cluster cluster = init(Cluster.create(1)))
> {
> IInvokableInstance node = cluster.get(1);
> cluster.schemaChange("CREATE TABLE " + KEYSPACE + ".mismatch (key
> text primary key, value timeuuid)");
> // write valid timeuuid then flush
> node.executeInternal("INSERT INTO " + KEYSPACE + ".mismatch (key,
> value) VALUES (?, ?)", "k1", UUIDGen.getTimeUUID());
> node.flush(KEYSPACE);
> // alter schema
> Object[] columns = node.executeInternal("SELECT keyspace_name,
> table_name, column_name, clustering_order, column_name_bytes, kind, position,
> type " +
> "FROM system_schema.columns WHERE
> keyspace_name=? AND table_name=? AND column_name=?", KEYSPACE, "mismatch",
> "value")[0];
> columns[columns.length - 1] = "uuid";
> node.executeInternal("INSERT INTO system_schema.columns
> (keyspace_name, table_name, column_name, clustering_order, column_name_bytes,
> kind, position, type) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", columns);
> cluster.schemaChange("ALTER TABLE " + KEYSPACE + ".mismatch WITH
> comment='upgrade'");
> node.executeInternal("INSERT INTO " + KEYSPACE + ".mismatch (key,
> value) VALUES (?, ?)", "k2", UUID.randomUUID());
> node.flush(KEYSPACE);
> node.forceCompact(KEYSPACE, "mismatch");
> }
> }
> @Test
> public void alterTypeMixedSSTablesConflict() throws Throwable
> {
> try (Cluster cluster = init(Cluster.create(1)))
> {
> IInvokableInstance node = cluster.get(1);
> cluster.schemaChange("CREATE TABLE " + KEYSPACE + ".mismatch (key
> text primary key, value timeuuid)");
> // write valid timeuuid then flush
> node.executeInternal("INSERT INTO " + KEYSPACE + ".mismatch (key,
> value) VALUES (?, ?)", "k1", UUIDGen.getTimeUUID());
> node.flush(KEYSPACE);
> // alter schema
> Object[] columns = node.executeInternal("SELECT keyspace_name,
> table_name, column_name, clustering_order, column_name_bytes, kind, position,
> type " +
> "FROM
> system_schema.columns WHERE keyspace_name=? AND table_name=? AND
> column_name=?", KEYSPACE, "mismatch", "value")[0];
> columns[columns.length - 1] = "uuid";
> node.executeInternal("INSERT INTO system_schema.columns
> (keyspace_name, table_name, column_name, clustering_order, column_name_bytes,
> kind, position, type) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", columns);
> cluster.schemaChange("ALTER TABLE " + KEYSPACE + ".mismatch WITH
> comment='upgrade'");
> node.executeInternal("INSERT INTO " + KEYSPACE + ".mismatch (key,
> value) VALUES (?, ?)", "k1", UUID.randomUUID());
> node.flush(KEYSPACE);
>
[jira] [Updated] (CASSANDRA-15926) When schema does not match sstable serialization header can fail when merging rows
[
https://issues.apache.org/jira/browse/CASSANDRA-15926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-15926:
--
Description:
If the table schema had timeuuid and then an alter is done to change the column
to uuid, then the on-disk serialization header won’t match the table schema;
this is problematic as there isn’t logic to migrate the column from the old
type to the new type.
The ALTER TABLE command does not directly allow this type of schema change
(though possible before 3.0.11, see CASSANDRA-12443), but this is possible if
the following is done
{code}
INSERT INTO system_schema.columns (keyspace_name, table_name, column_name,
clustering_order, column_name_bytes, kind, position, type) VALUES (‘ks',
’table', ‘col', 'none', bytes, 'regular', -1, 'uuid’);
ALTER TABLE ks.table WITH comment = 'something’; — this will trigger the schema
to migrate across the cluster
{code}
The below is a test to show this
{code}
package org.apache.cassandra.distributed.test;
import java.util.UUID;
import org.junit.Test;
import org.apache.cassandra.distributed.Cluster;
import org.apache.cassandra.distributed.impl.IInvokableInstance;
import org.apache.cassandra.utils.UUIDGen;
public class AlterWithOnDiskMismatch extends DistributedTestBase
{
@Test
public void alterTypeMixedSSTablesNoConflict() throws Throwable
{
try (Cluster cluster = init(Cluster.create(1)))
{
IInvokableInstance node = cluster.get(1);
cluster.schemaChange("CREATE TABLE " + KEYSPACE + ".mismatch (key
text primary key, value timeuuid)");
// write valid timeuuid then flush
node.executeInternal("INSERT INTO " + KEYSPACE + ".mismatch (key,
value) VALUES (?, ?)", "k1", UUIDGen.getTimeUUID());
node.flush(KEYSPACE);
// alter schema
Object[] columns = node.executeInternal("SELECT keyspace_name,
table_name, column_name, clustering_order, column_name_bytes, kind, position,
type " +
"FROM system_schema.columns WHERE
keyspace_name=? AND table_name=? AND column_name=?", KEYSPACE, "mismatch",
"value")[0];
columns[columns.length - 1] = "uuid";
node.executeInternal("INSERT INTO system_schema.columns
(keyspace_name, table_name, column_name, clustering_order, column_name_bytes,
kind, position, type) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", columns);
cluster.schemaChange("ALTER TABLE " + KEYSPACE + ".mismatch WITH
comment='upgrade'");
node.executeInternal("INSERT INTO " + KEYSPACE + ".mismatch (key,
value) VALUES (?, ?)", "k2", UUID.randomUUID());
node.flush(KEYSPACE);
node.forceCompact(KEYSPACE, "mismatch");
}
}
@Test
public void alterTypeMixedSSTablesConflict() throws Throwable
{
try (Cluster cluster = init(Cluster.create(1)))
{
IInvokableInstance node = cluster.get(1);
cluster.schemaChange("CREATE TABLE " + KEYSPACE + ".mismatch (key
text primary key, value timeuuid)");
// write valid timeuuid then flush
node.executeInternal("INSERT INTO " + KEYSPACE + ".mismatch (key,
value) VALUES (?, ?)", "k1", UUIDGen.getTimeUUID());
node.flush(KEYSPACE);
// alter schema
Object[] columns = node.executeInternal("SELECT keyspace_name,
table_name, column_name, clustering_order, column_name_bytes, kind, position,
type " +
"FROM system_schema.columns
WHERE keyspace_name=? AND table_name=? AND column_name=?", KEYSPACE,
"mismatch", "value")[0];
columns[columns.length - 1] = "uuid";
node.executeInternal("INSERT INTO system_schema.columns
(keyspace_name, table_name, column_name, clustering_order, column_name_bytes,
kind, position, type) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", columns);
cluster.schemaChange("ALTER TABLE " + KEYSPACE + ".mismatch WITH
comment='upgrade'");
node.executeInternal("INSERT INTO " + KEYSPACE + ".mismatch (key,
value) VALUES (?, ?)", "k1", UUID.randomUUID());
node.flush(KEYSPACE);
node.forceCompact(KEYSPACE, "mismatch");
}
}
}
{code}
The following is the exception
{code}
java.lang.IllegalArgumentException: Trying to compare 2 different types:
org.apache.cassandra.db.marshal.TimeUUIDType and
org.apache.cassandra.db.marshal.UUIDType
at
org.apache.cassandra.db.rows.AbstractTypeVersionComparator.compare(AbstractTypeVersionComparator.java:42)
at
org.apache.cassandra.db.rows.AbstractTypeVersionComparator.compare(AbstractTypeVersionComparator.java:30)
at
org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.useColumnDefinition(Row.java:650)
at
org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.reduce(Row.java:634)
at
[jira] [Updated] (CASSANDRA-15926) When schema does not match sstable serialization header can fail when merging rows
[
https://issues.apache.org/jira/browse/CASSANDRA-15926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-15926:
--
Summary: When schema does not match sstable serialization header can fail
when merging rows (was: When schema does not match sstable serialization
header compaction may fail)
> When schema does not match sstable serialization header can fail when merging
> rows
> --
>
> Key: CASSANDRA-15926
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15926
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema, Local/SSTable
>Reporter: David Capwell
>Priority: Normal
>
> If the table schema had timeuuid and then an alter is done to change the
> column to uuid, then the on-disk serialization header won’t match the table
> schema; this is problematic as there isn’t logic to migrate the column from
> the old type to the new type.
> The ALTER TABLE command does not directly allow this type of schema change
> (though possible before 3.0.11, see CASSANDRA-12443), but this is possible if
> the following is done
> {code}
> INSERT INTO system_schema.columns (keyspace_name, table_name, column_name,
> clustering_order, column_name_bytes, kind, position, type) VALUES (‘ks',
> ’table', ‘col', 'none', bytes, 'regular', -1, 'uuid’);
> ALTER TABLE ks.table WITH comment = 'something’; — this will trigger the
> schema to migrate across the cluster
> {code}
> The below is a test to show this
> {code}
> package org.apache.cassandra.distributed.test;
> import java.util.UUID;
> import org.junit.Test;
> import org.apache.cassandra.distributed.Cluster;
> import org.apache.cassandra.distributed.impl.IInvokableInstance;
> import org.apache.cassandra.utils.UUIDGen;
> public class AlterWithOnDiskMismatch extends DistributedTestBase
> {
> @Test
> public void alterTypeMixedSSTablesNoConflict() throws Throwable
> {
> try (Cluster cluster = init(Cluster.create(1)))
> {
> IInvokableInstance node = cluster.get(1);
> cluster.schemaChange("CREATE TABLE " + KEYSPACE + ".mismatch (key
> text primary key, value timeuuid)");
> // write valid timeuuid then flush
> node.executeInternal("INSERT INTO " + KEYSPACE + ".mismatch (key,
> value) VALUES (?, ?)", "k1", UUIDGen.getTimeUUID());
> node.flush(KEYSPACE);
> // alter schema
> Object[] columns = node.executeInternal("SELECT keyspace_name,
> table_name, column_name, clustering_order, column_name_bytes, kind, position,
> type " +
> "FROM system_schema.columns WHERE
> keyspace_name=? AND table_name=? AND column_name=?", KEYSPACE, "mismatch",
> "value")[0];
> columns[columns.length - 1] = "uuid";
> node.executeInternal("INSERT INTO system_schema.columns
> (keyspace_name, table_name, column_name, clustering_order, column_name_bytes,
> kind, position, type) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", columns);
> cluster.schemaChange("ALTER TABLE " + KEYSPACE + ".mismatch WITH
> comment='upgrade'");
> node.executeInternal("INSERT INTO " + KEYSPACE + ".mismatch (key,
> value) VALUES (?, ?)", "k2", UUID.randomUUID());
> node.flush(KEYSPACE);
> node.forceCompact(KEYSPACE, "mismatch");
> }
> }
> @Test
> public void alterTypeMixedSSTablesConflict() throws Throwable
> {
> try (Cluster cluster = init(Cluster.create(1)))
> {
> IInvokableInstance node = cluster.get(1);
> cluster.schemaChange("CREATE TABLE " + KEYSPACE + ".mismatch (key
> text primary key, value timeuuid)");
> // write valid timeuuid then flush
> node.executeInternal("INSERT INTO " + KEYSPACE + ".mismatch (key,
> value) VALUES (?, ?)", "k1", UUIDGen.getTimeUUID());
> node.flush(KEYSPACE);
> // alter schema
> Object[] columns = node.executeInternal("SELECT keyspace_name,
> table_name, column_name, clustering_order, column_name_bytes, kind, position,
> type " +
> "FROM
> system_schema.columns WHERE keyspace_name=? AND table_name=? AND
> column_name=?", KEYSPACE, "mismatch", "value")[0];
> columns[columns.length - 1] = "uuid";
> node.executeInternal("INSERT INTO system_schema.columns
> (keyspace_name, table_name, column_name, clustering_order, column_name_bytes,
> kind, position, type) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", columns);
> cluster.schemaChange("ALTER TABLE " + KEYSPACE + ".mismatch WITH
> comment='upgrade'");
> node.executeInternal("INSERT INTO " + KEYSPACE + ".mismatch (key,
> value) VALUES (?,
[jira] [Created] (CASSANDRA-15926) When schema does not match sstable serialization header compaction may fail
David Capwell created CASSANDRA-15926:
-
Summary: When schema does not match sstable serialization header
compaction may fail
Key: CASSANDRA-15926
URL: https://issues.apache.org/jira/browse/CASSANDRA-15926
Project: Cassandra
Issue Type: Improvement
Components: Cluster/Schema, Local/SSTable
Reporter: David Capwell
If the table schema had timeuuid and then an alter is done to change the column
to uuid, then the on-disk serialization header won’t match the table schema;
this is problematic as there isn’t logic to migrate the column from the old
type to the new type.
The ALTER TABLE command does not directly allow this type of schema change
(though possible before 3.0.11, see CASSANDRA-12443), but this is possible if
the following is done
{code}
INSERT INTO system_schema.columns (keyspace_name, table_name, column_name,
clustering_order, column_name_bytes, kind, position, type) VALUES (‘ks',
’table', ‘col', 'none', bytes, 'regular', -1, 'uuid’);
ALTER TABLE ks.table WITH comment = 'something’; — this will trigger the schema
to migrate across the cluster
{code}
The below is a test to show this
{code}
package org.apache.cassandra.distributed.test;
import java.util.UUID;
import org.junit.Test;
import org.apache.cassandra.distributed.Cluster;
import org.apache.cassandra.distributed.impl.IInvokableInstance;
import org.apache.cassandra.utils.UUIDGen;
public class AlterWithOnDiskMismatch extends DistributedTestBase
{
@Test
public void alterTypeMixedSSTablesNoConflict() throws Throwable
{
try (Cluster cluster = init(Cluster.create(1)))
{
IInvokableInstance node = cluster.get(1);
cluster.schemaChange("CREATE TABLE " + KEYSPACE + ".mismatch (key
text primary key, value timeuuid)");
// write valid timeuuid then flush
node.executeInternal("INSERT INTO " + KEYSPACE + ".mismatch (key,
value) VALUES (?, ?)", "k1", UUIDGen.getTimeUUID());
node.flush(KEYSPACE);
// alter schema
Object[] columns = node.executeInternal("SELECT keyspace_name,
table_name, column_name, clustering_order, column_name_bytes, kind, position,
type " +
"FROM system_schema.columns WHERE
keyspace_name=? AND table_name=? AND column_name=?", KEYSPACE, "mismatch",
"value")[0];
columns[columns.length - 1] = "uuid";
node.executeInternal("INSERT INTO system_schema.columns
(keyspace_name, table_name, column_name, clustering_order, column_name_bytes,
kind, position, type) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", columns);
cluster.schemaChange("ALTER TABLE " + KEYSPACE + ".mismatch WITH
comment='upgrade'");
node.executeInternal("INSERT INTO " + KEYSPACE + ".mismatch (key,
value) VALUES (?, ?)", "k2", UUID.randomUUID());
node.flush(KEYSPACE);
node.forceCompact(KEYSPACE, "mismatch");
}
}
@Test
public void alterTypeMixedSSTablesConflict() throws Throwable
{
try (Cluster cluster = init(Cluster.create(1)))
{
IInvokableInstance node = cluster.get(1);
cluster.schemaChange("CREATE TABLE " + KEYSPACE + ".mismatch (key
text primary key, value timeuuid)");
// write valid timeuuid then flush
node.executeInternal("INSERT INTO " + KEYSPACE + ".mismatch (key,
value) VALUES (?, ?)", "k1", UUIDGen.getTimeUUID());
node.flush(KEYSPACE);
// alter schema
Object[] columns = node.executeInternal("SELECT keyspace_name,
table_name, column_name, clustering_order, column_name_bytes, kind, position,
type " +
"FROM system_schema.columns
WHERE keyspace_name=? AND table_name=? AND column_name=?", KEYSPACE,
"mismatch", "value")[0];
columns[columns.length - 1] = "uuid";
node.executeInternal("INSERT INTO system_schema.columns
(keyspace_name, table_name, column_name, clustering_order, column_name_bytes,
kind, position, type) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", columns);
cluster.schemaChange("ALTER TABLE " + KEYSPACE + ".mismatch WITH
comment='upgrade'");
node.executeInternal("INSERT INTO " + KEYSPACE + ".mismatch (key,
value) VALUES (?, ?)", "k1", UUID.randomUUID());
node.flush(KEYSPACE);
node.forceCompact(KEYSPACE, "mismatch");
}
}
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15894) JAVA 8: test_multiple_repair - repair_tests.incremental_repair_test.TestIncRepair
[ https://issues.apache.org/jira/browse/CASSANDRA-15894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152902#comment-17152902 ] Benjamin Lerer commented on CASSANDRA-15894: I believe that it is ok to ignore it for now. If the problem start showing up more often we will have to figure out some better timeout. > JAVA 8: test_multiple_repair - > repair_tests.incremental_repair_test.TestIncRepair > - > > Key: CASSANDRA-15894 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15894 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-rc > > Attachments: test_multiple_repair_log.txt > > > JAVA 8: test_multiple_repair - > repair_tests.incremental_repair_test.TestIncRepair > Fails locally and in CircleCI: > https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/222/workflows/46515d14-9be4-4edb-8db4-5930312d2bfb/jobs/1329 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15894) JAVA 8: test_multiple_repair - repair_tests.incremental_repair_test.TestIncRepair
[ https://issues.apache.org/jira/browse/CASSANDRA-15894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152880#comment-17152880 ] Ekaterina Dimitrova edited comment on CASSANDRA-15894 at 7/7/20, 4:37 PM: -- Thank you for your clarification [~blerer] . I figured it out from the logs that it is a driver, client connection time out. I am tempted not to touch it for now while it is stable enough, wdyt? was (Author: e.dimitrova): Thank you for your clarification @Benjamin. I figured it out from the logs that it is a driver, client connection time out. I am tempted not to touch it for now while it is stable enough, wdyt? > JAVA 8: test_multiple_repair - > repair_tests.incremental_repair_test.TestIncRepair > - > > Key: CASSANDRA-15894 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15894 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-rc > > Attachments: test_multiple_repair_log.txt > > > JAVA 8: test_multiple_repair - > repair_tests.incremental_repair_test.TestIncRepair > Fails locally and in CircleCI: > https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/222/workflows/46515d14-9be4-4edb-8db4-5930312d2bfb/jobs/1329 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15894) JAVA 8: test_multiple_repair - repair_tests.incremental_repair_test.TestIncRepair
[ https://issues.apache.org/jira/browse/CASSANDRA-15894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152880#comment-17152880 ] Ekaterina Dimitrova commented on CASSANDRA-15894: - Thank you for your clarification @Benjamin. I figured it out from the logs that it is a driver, client connection time out. I am tempted not to touch it for now while it is stable enough, wdyt? > JAVA 8: test_multiple_repair - > repair_tests.incremental_repair_test.TestIncRepair > - > > Key: CASSANDRA-15894 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15894 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-rc > > Attachments: test_multiple_repair_log.txt > > > JAVA 8: test_multiple_repair - > repair_tests.incremental_repair_test.TestIncRepair > Fails locally and in CircleCI: > https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/222/workflows/46515d14-9be4-4edb-8db4-5930312d2bfb/jobs/1329 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15894) JAVA 8: test_multiple_repair - repair_tests.incremental_repair_test.TestIncRepair
[ https://issues.apache.org/jira/browse/CASSANDRA-15894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova reassigned CASSANDRA-15894: --- Assignee: Ekaterina Dimitrova (was: Berenguer Blasi) > JAVA 8: test_multiple_repair - > repair_tests.incremental_repair_test.TestIncRepair > - > > Key: CASSANDRA-15894 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15894 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-rc > > Attachments: test_multiple_repair_log.txt > > > JAVA 8: test_multiple_repair - > repair_tests.incremental_repair_test.TestIncRepair > Fails locally and in CircleCI: > https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/222/workflows/46515d14-9be4-4edb-8db4-5930312d2bfb/jobs/1329 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15889) Debian package fails to download on Arm-based hosts
[
https://issues.apache.org/jira/browse/CASSANDRA-15889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152859#comment-17152859
]
Brandon Williams commented on CASSANDRA-15889:
--
That is the source from which the packages are built. In the control file we
specify the architecture as 'all', but according to
[https://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-architecture]
maybe we should be using 'any'? /cc [~mshuler]
> Debian package fails to download on Arm-based hosts
> ---
>
> Key: CASSANDRA-15889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15889
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Matt Davis
>Priority: Normal
> Labels: ARM, debian
>
> Following the first three steps of the [Debian install
> process|https://cassandra.apache.org/download/], after an apt-get update
> you'll see this line:
> {code:bash}
> $ sudo apt-get update
> ...
> N: Skipping acquire of configured file 'main/binary-arm64/Packages' as
> repository 'https://downloads.apache.org/cassandra/debian 311x InRelease'
> doesn't support architecture 'arm64'
> {code}
> Checking the [Debian
> repo|https://dl.bintray.com/apache/cassandra/dists/311x/main/] confirms there
> is no aarch64 variant available.
> Should you then attempt to install Cassandra:
> {code:bash}
> $ sudo apt-get install cassandra
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> Package cassandra is not available, but is referred to by another package.
> This may mean that the package is missing, has been obsoleted, or
> is only available from another source
> E: Package 'cassandra' has no installation candidate
> {code}
> The Redhat RPM contains a "noarch" arch type, so it will download on any
> host. (Cassandra does not use separate binaries/releases for different
> architectures, so this seems to be the correct approach, but adding an
> aarch64 variant would also suffice.)
> Note that there is a workaround available: if you specify "amd64" as the arch
> for the source, it downloads and runs on Arm without issue:
> {code:bash}
> deb [arch=amd64] https://downloads.apache.org/cassandra/debian 311x main
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15889) Debian package fails to download on Arm-based hosts
[
https://issues.apache.org/jira/browse/CASSANDRA-15889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152850#comment-17152850
]
Matt Davis commented on CASSANDRA-15889:
Thanks Michael! Be glad to submit a patch if you can give me some guidance. I
took a look at https://github.com/apache/cassandra/tree/trunk/debian but didn't
see any references to the currently supported arches, so it seemed to me this
wasn't where the deb package was built. I got the impression this was being
done with some non-public CI/CD pipelines.
> Debian package fails to download on Arm-based hosts
> ---
>
> Key: CASSANDRA-15889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15889
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Matt Davis
>Priority: Normal
> Labels: ARM, debian
>
> Following the first three steps of the [Debian install
> process|https://cassandra.apache.org/download/], after an apt-get update
> you'll see this line:
> {code:bash}
> $ sudo apt-get update
> ...
> N: Skipping acquire of configured file 'main/binary-arm64/Packages' as
> repository 'https://downloads.apache.org/cassandra/debian 311x InRelease'
> doesn't support architecture 'arm64'
> {code}
> Checking the [Debian
> repo|https://dl.bintray.com/apache/cassandra/dists/311x/main/] confirms there
> is no aarch64 variant available.
> Should you then attempt to install Cassandra:
> {code:bash}
> $ sudo apt-get install cassandra
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> Package cassandra is not available, but is referred to by another package.
> This may mean that the package is missing, has been obsoleted, or
> is only available from another source
> E: Package 'cassandra' has no installation candidate
> {code}
> The Redhat RPM contains a "noarch" arch type, so it will download on any
> host. (Cassandra does not use separate binaries/releases for different
> architectures, so this seems to be the correct approach, but adding an
> aarch64 variant would also suffice.)
> Note that there is a workaround available: if you specify "amd64" as the arch
> for the source, it downloads and runs on Arm without issue:
> {code:bash}
> deb [arch=amd64] https://downloads.apache.org/cassandra/debian 311x main
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15889) Debian package fails to download on Arm-based hosts
[
https://issues.apache.org/jira/browse/CASSANDRA-15889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152844#comment-17152844
]
Michael Semb Wever commented on CASSANDRA-15889:
We're looking for help/contributors on the deb/rpm packaging front.
I suspect the fastest solution here is if you can jump in and offer up a patch
[~mattsplat]. I'd be happy to help along the way.
> Debian package fails to download on Arm-based hosts
> ---
>
> Key: CASSANDRA-15889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15889
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Matt Davis
>Priority: Normal
> Labels: ARM, debian
>
> Following the first three steps of the [Debian install
> process|https://cassandra.apache.org/download/], after an apt-get update
> you'll see this line:
> {code:bash}
> $ sudo apt-get update
> ...
> N: Skipping acquire of configured file 'main/binary-arm64/Packages' as
> repository 'https://downloads.apache.org/cassandra/debian 311x InRelease'
> doesn't support architecture 'arm64'
> {code}
> Checking the [Debian
> repo|https://dl.bintray.com/apache/cassandra/dists/311x/main/] confirms there
> is no aarch64 variant available.
> Should you then attempt to install Cassandra:
> {code:bash}
> $ sudo apt-get install cassandra
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> Package cassandra is not available, but is referred to by another package.
> This may mean that the package is missing, has been obsoleted, or
> is only available from another source
> E: Package 'cassandra' has no installation candidate
> {code}
> The Redhat RPM contains a "noarch" arch type, so it will download on any
> host. (Cassandra does not use separate binaries/releases for different
> architectures, so this seems to be the correct approach, but adding an
> aarch64 variant would also suffice.)
> Note that there is a workaround available: if you specify "amd64" as the arch
> for the source, it downloads and runs on Arm without issue:
> {code:bash}
> deb [arch=amd64] https://downloads.apache.org/cassandra/debian 311x main
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15889) Debian package fails to download on Arm-based hosts
[
https://issues.apache.org/jira/browse/CASSANDRA-15889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15889:
---
Component/s: Build
> Debian package fails to download on Arm-based hosts
> ---
>
> Key: CASSANDRA-15889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15889
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Matt Davis
>Priority: Normal
>
> Following the first three steps of the [Debian install
> process|https://cassandra.apache.org/download/], after an apt-get update
> you'll see this line:
> {code:bash}
> $ sudo apt-get update
> ...
> N: Skipping acquire of configured file 'main/binary-arm64/Packages' as
> repository 'https://downloads.apache.org/cassandra/debian 311x InRelease'
> doesn't support architecture 'arm64'
> {code}
> Checking the [Debian
> repo|https://dl.bintray.com/apache/cassandra/dists/311x/main/] confirms there
> is no aarch64 variant available.
> Should you then attempt to install Cassandra:
> {code:bash}
> $ sudo apt-get install cassandra
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> Package cassandra is not available, but is referred to by another package.
> This may mean that the package is missing, has been obsoleted, or
> is only available from another source
> E: Package 'cassandra' has no installation candidate
> {code}
> The Redhat RPM contains a "noarch" arch type, so it will download on any
> host. (Cassandra does not use separate binaries/releases for different
> architectures, so this seems to be the correct approach, but adding an
> aarch64 variant would also suffice.)
> Note that there is a workaround available: if you specify "amd64" as the arch
> for the source, it downloads and runs on Arm without issue:
> {code:bash}
> deb [arch=amd64] https://downloads.apache.org/cassandra/debian 311x main
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15889) Debian package fails to download on Arm-based hosts
[
https://issues.apache.org/jira/browse/CASSANDRA-15889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15889:
---
Labels: ARM debian (was: )
> Debian package fails to download on Arm-based hosts
> ---
>
> Key: CASSANDRA-15889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15889
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Matt Davis
>Priority: Normal
> Labels: ARM, debian
>
> Following the first three steps of the [Debian install
> process|https://cassandra.apache.org/download/], after an apt-get update
> you'll see this line:
> {code:bash}
> $ sudo apt-get update
> ...
> N: Skipping acquire of configured file 'main/binary-arm64/Packages' as
> repository 'https://downloads.apache.org/cassandra/debian 311x InRelease'
> doesn't support architecture 'arm64'
> {code}
> Checking the [Debian
> repo|https://dl.bintray.com/apache/cassandra/dists/311x/main/] confirms there
> is no aarch64 variant available.
> Should you then attempt to install Cassandra:
> {code:bash}
> $ sudo apt-get install cassandra
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> Package cassandra is not available, but is referred to by another package.
> This may mean that the package is missing, has been obsoleted, or
> is only available from another source
> E: Package 'cassandra' has no installation candidate
> {code}
> The Redhat RPM contains a "noarch" arch type, so it will download on any
> host. (Cassandra does not use separate binaries/releases for different
> architectures, so this seems to be the correct approach, but adding an
> aarch64 variant would also suffice.)
> Note that there is a workaround available: if you specify "amd64" as the arch
> for the source, it downloads and runs on Arm without issue:
> {code:bash}
> deb [arch=amd64] https://downloads.apache.org/cassandra/debian 311x main
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15925) Jenkins pipeline can copy wrong test report artefacts from dtest stage builds
[ https://issues.apache.org/jira/browse/CASSANDRA-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152835#comment-17152835 ] Michael Semb Wever commented on CASSANDRA-15925: nice tip on how to do this [here|https://stackoverflow.com/questions/39170144/jenkins-groovy-pipeline-retrieve-build-number-of-built-job] > Jenkins pipeline can copy wrong test report artefacts from dtest stage builds > - > > Key: CASSANDRA-15925 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15925 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 4.0-rc > > > Spotted in > https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/196/console > Looks like copyArtifact will need to be specific to a build. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15912) Occasional NEW_NODE event sent instead of NODE_UP in dTest
[ https://issues.apache.org/jira/browse/CASSANDRA-15912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-15912: - Reviewers: Brandon Williams > Occasional NEW_NODE event sent instead of NODE_UP in dTest > -- > > Key: CASSANDRA-15912 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15912 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Client >Reporter: Bryn Cooke >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > > A new test for topology events in CASSANDRA-15677 is occasionally failing due > to a NEW_NODE event sent instead of NODE_UP. > For instance: > [https://app.circleci.com/pipelines/github/jasonstack/cassandra/231/workflows/cdf55335-c876-450b-8bf9-1d778a2df806/jobs/2240] > > Previously topology events were not been sent at all, so it is possible that > this is a long standing bug. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13701) Lower default num_tokens
[ https://issues.apache.org/jira/browse/CASSANDRA-13701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152819#comment-17152819 ] Brandon Williams commented on CASSANDRA-13701: -- Changed the tests and circle configs here: https://github.com/driftx/cassandra/tree/CASSANDRA-13701 CI: https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/198/ https://app.circleci.com/pipelines/github/driftx/cassandra/48/workflows/885af1d3-10a7-4fd3-aa2a-54370a102ba9 > Lower default num_tokens > > > Key: CASSANDRA-13701 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13701 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Chris Lohfink >Assignee: Jeremy Hanna >Priority: Low > > For reasons highlighted in CASSANDRA-7032, the high number of vnodes is not > necessary. It is very expensive for operations processes and scanning. Its > come up a lot and its pretty standard and known now to always reduce the > num_tokens within the community. We should just lower the defaults. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15592) IllegalStateException in gossip after removing node
[
https://issues.apache.org/jira/browse/CASSANDRA-15592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152816#comment-17152816
]
Brandon Williams commented on CASSANDRA-15592:
--
The node would never get evicted. In practice I don't think this is a huge
problem, since it would just hang around in gossip uselessly.
> IllegalStateException in gossip after removing node
> ---
>
> Key: CASSANDRA-15592
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15592
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Marcus Olsson
>Assignee: Marcus Olsson
>Priority: Normal
> Fix For: 3.0.21, 3.11.7, 4.0, 4.0-alpha4
>
>
> In one of our test environments we encountered the following exception:
> {noformat}
> 2020-02-02T10:50:13.276+0100 [GossipTasks:1] ERROR
> o.a.c.u.NoSpamLogger$NoSpamLogStatement:97 log
> java.lang.IllegalStateException: Attempting gossip state mutation from
> illegal thread: GossipTasks:1
> at
> org.apache.cassandra.gms.Gossiper.checkProperThreadForStateMutation(Gossiper.java:178)
> at org.apache.cassandra.gms.Gossiper.evictFromMembership(Gossiper.java:465)
> at org.apache.cassandra.gms.Gossiper.doStatusCheck(Gossiper.java:895)
> at org.apache.cassandra.gms.Gossiper.access$700(Gossiper.java:78)
> at org.apache.cassandra.gms.Gossiper$GossipTask.run(Gossiper.java:240)
> at
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:84)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)
> java.lang.IllegalStateException: Attempting gossip state mutation from
> illegal thread: GossipTasks:1
> at
> org.apache.cassandra.gms.Gossiper.checkProperThreadForStateMutation(Gossiper.java:178)
> [apache-cassandra-3.11.5.jar:3.11.5]
> at org.apache.cassandra.gms.Gossiper.evictFromMembership(Gossiper.java:465)
> [apache-cassandra-3.11.5.jar:3.11.5]
> at org.apache.cassandra.gms.Gossiper.doStatusCheck(Gossiper.java:895)
> [apache-cassandra-3.11.5.jar:3.11.5]
> at org.apache.cassandra.gms.Gossiper.access$700(Gossiper.java:78)
> [apache-cassandra-3.11.5.jar:3.11.5]
> at org.apache.cassandra.gms.Gossiper$GossipTask.run(Gossiper.java:240)
> [apache-cassandra-3.11.5.jar:3.11.5]
> at
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
> [apache-cassandra-3.11.5.jar:3.11.5]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_231]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_231]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [na:1.8.0_231]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [na:1.8.0_231]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [na:1.8.0_231]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [na:1.8.0_231]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:84)
> [apache-cassandra-3.11.5.jar:3.11.5]
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> ~[netty-all-4.1.42.Final.jar:4.1.42.Final]
> at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_231]
> {noformat}
> Since CASSANDRA-15059 we check that all state changes are performed in the
> GossipStage but it seems like it was still performed in the "current" thread
> [here|https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/gms/Gossiper.java#L895].
> It should be as simple as adding a
> {code:java}
> runInGossipStageBlocking(() ->)
> {code}
> for it.
> I'll upload patches for 3.0, 3.11 and 4.0.
>
--
This message was sent by Atlassian Jira
[jira] [Commented] (CASSANDRA-15900) Close channel and reduce buffer allocation during entire sstable streaming with SSL
[ https://issues.apache.org/jira/browse/CASSANDRA-15900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152771#comment-17152771 ] ZhaoYang commented on CASSANDRA-15900: -- thanks for the review > Close channel and reduce buffer allocation during entire sstable streaming > with SSL > --- > > Key: CASSANDRA-15900 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15900 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Streaming and Messaging >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Normal > Fix For: 4.0-beta > > > CASSANDRA-15740 added the ability to stream entire sstable by loading on-disk > file into user-space off-heap buffer when SSL is enabled, because netty > doesn't support zero-copy with SSL. > But there are two issues: > # file channel is not closed. > # 1mb batch size is used. 1mb exceeds buffer pool's max allocation size, > thus it's all allocated outside the pool and will cause large amount of > allocations. > [Patch|https://github.com/apache/cassandra/pull/651]: > # close file channel when the last batch is loaded into off-heap bytebuffer. > I don't think we need to wait until buffer is flushed by netty. > # reduce the batch to 64kb which is more buffer pool friendly when streaming > entire sstable with SSL. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15912) Occasional NEW_NODE event sent instead of NODE_UP in dTest
[ https://issues.apache.org/jira/browse/CASSANDRA-15912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi reassigned CASSANDRA-15912: --- Assignee: Berenguer Blasi > Occasional NEW_NODE event sent instead of NODE_UP in dTest > -- > > Key: CASSANDRA-15912 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15912 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Client >Reporter: Bryn Cooke >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > > A new test for topology events in CASSANDRA-15677 is occasionally failing due > to a NEW_NODE event sent instead of NODE_UP. > For instance: > [https://app.circleci.com/pipelines/github/jasonstack/cassandra/231/workflows/cdf55335-c876-450b-8bf9-1d778a2df806/jobs/2240] > > Previously topology events were not been sent at all, so it is possible that > this is a long standing bug. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15912) Occasional NEW_NODE event sent instead of NODE_UP in dTest
[ https://issues.apache.org/jira/browse/CASSANDRA-15912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-15912: Bug Category: Parent values: Correctness(12982)Level 1 values: Test Failure(12990) Complexity: Normal Discovered By: Unit Test Severity: Normal Status: Open (was: Triage Needed) > Occasional NEW_NODE event sent instead of NODE_UP in dTest > -- > > Key: CASSANDRA-15912 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15912 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Client >Reporter: Bryn Cooke >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > > A new test for topology events in CASSANDRA-15677 is occasionally failing due > to a NEW_NODE event sent instead of NODE_UP. > For instance: > [https://app.circleci.com/pipelines/github/jasonstack/cassandra/231/workflows/cdf55335-c876-450b-8bf9-1d778a2df806/jobs/2240] > > Previously topology events were not been sent at all, so it is possible that > this is a long standing bug. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10968) When taking snapshot, manifest.json contains incorrect or no files when column family has secondary indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-10968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152682#comment-17152682 ] Andres de la Peña commented on CASSANDRA-10968: --- Good catch. Rerunning: ||branch||utest||dtest|| |2.1 |[173|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-test/173/]|[207|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/207/]| |2.2 |[174|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-test/174/]|[208|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/208/]| > When taking snapshot, manifest.json contains incorrect or no files when > column family has secondary indexes > --- > > Key: CASSANDRA-10968 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10968 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index >Reporter: Fred A >Assignee: Aleksandr Sorokoumov >Priority: Normal > Labels: lhf > Fix For: 2.2.x, 3.0.x, 3.11.x > > Time Spent: 1h 40m > Remaining Estimate: 0h > > xNoticed indeterminate behaviour when taking snapshot on column families that > has secondary indexes setup. The created manifest.json created when doing > snapshot, sometimes contains no file names at all and sometimes some file > names. > I don't know if this post is related but that was the only thing I could find: > http://www.mail-archive.com/user%40cassandra.apache.org/msg42019.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10968) When taking snapshot, manifest.json contains incorrect or no files when column family has secondary indexes
[
https://issues.apache.org/jira/browse/CASSANDRA-10968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152659#comment-17152659
]
Aleksandr Sorokoumov commented on CASSANDRA-10968:
--
Thanks for the review [~adelapena]!
I double-checked
[TestSnapshot.test_basic_snapshot_and_restore|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/202/testReport/junit/dtest.snapshot_test/TestSnapshot/test_basic_snapshot_and_restore/]
failure and it was actually caused by the introduction of
{{Directories#isSecondaryIndexFolder}}. DatabaseDescriptor was crashing during
static initialization in client mode. I updated the patch - [added default
value for data file
directories|https://github.com/apache/cassandra/pull/624/files#diff-b66584c9ce7b64019b5db5a531deeda1R180].
This change is actually present in 3.0+.
Can you please rerun the CI for 2.1?
> When taking snapshot, manifest.json contains incorrect or no files when
> column family has secondary indexes
> ---
>
> Key: CASSANDRA-10968
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10968
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Fred A
>Assignee: Aleksandr Sorokoumov
>Priority: Normal
> Labels: lhf
> Fix For: 2.2.x, 3.0.x, 3.11.x
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> xNoticed indeterminate behaviour when taking snapshot on column families that
> has secondary indexes setup. The created manifest.json created when doing
> snapshot, sometimes contains no file names at all and sometimes some file
> names.
> I don't know if this post is related but that was the only thing I could find:
> http://www.mail-archive.com/user%40cassandra.apache.org/msg42019.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15922) High CAS failures in NativeAllocator.Region.allocate(..)
[
https://issues.apache.org/jira/browse/CASSANDRA-15922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15922:
---
Fix Version/s: (was: 3.11.x)
(was: 4.0)
(was: 3.0.x)
4.0-alpha5
3.11.7
3.0.21
Since Version: 1.0.0
Source Control Link:
https://github.com/apache/cassandra/commit/5a6d52b26191ab1f5df6e9cf941e0f964dd95a28
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed as [5a6d52b26191ab1f5df6e9cf941e0f964dd95a28
|https://github.com/apache/cassandra/commit/5a6d52b26191ab1f5df6e9cf941e0f964dd95a28].
> High CAS failures in NativeAllocator.Region.allocate(..)
> -
>
> Key: CASSANDRA-15922
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15922
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Memtable
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 3.0.21, 3.11.7, 4.0-alpha5
>
> Attachments: NativeAllocatorRegion2Test.java,
> NativeAllocatorRegionTest.java, Screen Shot 2020-07-05 at 13.16.10.png,
> Screen Shot 2020-07-05 at 13.26.17.png, Screen Shot 2020-07-05 at
> 13.35.55.png, Screen Shot 2020-07-05 at 13.37.01.png, Screen Shot 2020-07-05
> at 13.48.16.png, Screen Shot 2020-07-06 at 11.35.35.png, Screen Shot
> 2020-07-06 at 11.36.44.png, Screen Shot 2020-07-06 at 13.26.10.png,
> profile_pbdpc23zafsrh_20200702.svg
>
>
> h4. Problem
> The method {{NativeAllocator.Region.allocate(..)}} uses an {{AtomicInteger}}
> for the current offset in the region. Allocations depends on a
> {{.compareAndSet(..)}} call.
> In highly contended environments the CAS failures can be high, starving
> writes in a running Cassandra node.
> h4. Example
> It has been witnessed up to 33% of CPU time stuck in the
> {{NativeAllocator.Region.allocate(..)}} loop (due to the CAS failures) during
> a heavy spark analytics write load.
> These nodes: 40 CPU cores and 256GB ram; have relevant settings
> - {{memtable_allocation_type: offheap_objects}}
> - {{memtable_offheap_space_in_mb: 5120}}
> - {{concurrent_writes: 160}}
> Numerous flamegraphs demonstrate the problem. See attached
> [^profile_pbdpc23zafsrh_20200702.svg].
> h4. Suggestion: ThreadLocal Regions
> One possible solution is to have separate Regions per thread.
> Code wise this is relatively easy to do, for example replacing
> NativeAllocator:59
> {code}private final AtomicReference currentRegion = new
> AtomicReference<>();{code}
> with
> {code}private final ThreadLocal> currentRegion = new
> ThreadLocal<>() {...};{code}
> But this approach substantially changes the allocation behaviour, with more
> than concurrent_writes number of Regions in use at any one time. For example
> with {{concurrent_writes: 160}} that's 160+ regions, each of 1MB.
> h4. Suggestion: Simple Contention Management Algorithm (Constant Backoff)
> Another possible solution is to introduce a contention management algorithm
> that a) reduces CAS failures in high contention environments, b) doesn't
> impact normal environments, and c) keeps the allocation strategy of using one
> region at a time.
> The research paper [arXiv:1305.5800|https://arxiv.org/abs/1305.5800]
> describes this contention CAS problem and demonstrates a number of algorithms
> to apply. The simplest of these algorithms is the Constant Backoff CAS
> Algorithm.
> Applying the Constant Backoff CAS Algorithm involves adding one line of code
> to {{NativeAllocator.Region.allocate(..)}} to sleep for one (or some constant
> number) nanoseconds after a CAS failure occurs.
> That is...
> {code}
> // we raced and lost alloc, try again
> LockSupport.parkNanos(1);
> {code}
> h4. Constant Backoff CAS Algorithm Experiments
> Using the code attached in NativeAllocatorRegionTest.java the concurrency and
> CAS failures of {{NativeAllocator.Region.allocate(..)}} can be demonstrated.
> In the attached [^NativeAllocatorRegionTest.java] class, which can be run
> standalone, the {{Region}} class: copied from {{NativeAllocator.Region}}; has
> also the {{casFailures}} field added. The following two screenshots are from
> data collected from this class on a 6 CPU (12 core) MBP, running the
> {{NativeAllocatorRegionTest.testRegionCAS}} method.
> This attached screenshot shows the number of CAS failures during the life of
> a Region (over ~215 million allocations), using different threads and park
> times. This illustrates the improvement (reduction) of CAS failures from zero
> park time, through orders of magnitude, up to 1000ns (10ms). The biggest
> improvement is from no
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 6912d32b7dfccba81118e12b920f010e060c2941 Merge: 4654ef0 9b165c9 Author: Mick Semb Wever AuthorDate: Tue Jul 7 12:34:49 2020 +0200 Merge branch 'cassandra-3.11' into trunk CHANGES.txt| 1 + .../cassandra/utils/memory/NativeAllocator.java| 33 ++ .../cassandra/utils/memory/SlabAllocator.java | 31 ++-- 3 files changed, 18 insertions(+), 47 deletions(-) diff --cc CHANGES.txt index 76cd271,72c05ff..d382650 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -55,7 -2,9 +55,8 @@@ Merged from 3.11 * Fix cqlsh output when fetching all rows in batch mode (CASSANDRA-15905) * Upgrade Jackson to 2.9.10 (CASSANDRA-15867) * Fix CQL formatting of read command restrictions for slow query log (CASSANDRA-15503) - * Allow sstableloader to use SSL on the native port (CASSANDRA-14904) Merged from 3.0: + * Avoid thread starvation, and improve compare-and-swap performance, in the slab allocators (CASSANDRA-15922) * Add token to tombstone warning and error messages (CASSANDRA-15890) * Fixed range read concurrency factor computation and capped as 10 times tpc cores (CASSANDRA-15752) * Catch exception on bootstrap resume and init native transport (CASSANDRA-15863) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (4654ef0 -> 6912d32)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 4654ef0 FBUtilities.getJustLocalAddress falls back to lo ip on misconfigured nodes new 5a6d52b Remove CAS failures and competition in Region.allocate(..) methods with use of getAndAdd(..) updates instead of compareAndSet(..) loops. new 9b165c9 Merge branch 'cassandra-3.0' into cassandra-3.11 new 6912d32 Merge branch 'cassandra-3.11' into trunk The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + .../cassandra/utils/memory/NativeAllocator.java| 33 ++ .../cassandra/utils/memory/SlabAllocator.java | 31 ++-- 3 files changed, 18 insertions(+), 47 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (c5b2a37 -> 9b165c9)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from c5b2a37 Merge commit '64c80f4ef89f0cc88c15febed8c01eb07ae0a84e' into cassandra-3.11 new 5a6d52b Remove CAS failures and competition in Region.allocate(..) methods with use of getAndAdd(..) updates instead of compareAndSet(..) loops. new 9b165c9 Merge branch 'cassandra-3.0' into cassandra-3.11 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + .../cassandra/utils/memory/NativeAllocator.java| 33 ++ .../cassandra/utils/memory/SlabAllocator.java | 31 ++-- 3 files changed, 18 insertions(+), 47 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 9b165c91bc4c49226b1118f103d3c75c90da7c6b Merge: c5b2a37 5a6d52b Author: Mick Semb Wever AuthorDate: Tue Jul 7 12:33:11 2020 +0200 Merge branch 'cassandra-3.0' into cassandra-3.11 CHANGES.txt| 1 + .../cassandra/utils/memory/NativeAllocator.java| 33 ++ .../cassandra/utils/memory/SlabAllocator.java | 31 ++-- 3 files changed, 18 insertions(+), 47 deletions(-) diff --cc CHANGES.txt index d89d22b,a363005..72c05ff --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,9 -1,6 +1,10 @@@ -3.0.21 +3.11.7 + * Fix cqlsh output when fetching all rows in batch mode (CASSANDRA-15905) + * Upgrade Jackson to 2.9.10 (CASSANDRA-15867) + * Fix CQL formatting of read command restrictions for slow query log (CASSANDRA-15503) + * Allow sstableloader to use SSL on the native port (CASSANDRA-14904) +Merged from 3.0: + * Avoid thread starvation, and improve compare-and-swap performance, in the slab allocators (CASSANDRA-15922) - * Fix broken KEYS 2i queries after DROP COMPACT STORAGE (CASSANDRA-15906) * Add token to tombstone warning and error messages (CASSANDRA-15890) * Fixed range read concurrency factor computation and capped as 10 times tpc cores (CASSANDRA-15752) * Catch exception on bootstrap resume and init native transport (CASSANDRA-15863) diff --cc src/java/org/apache/cassandra/utils/memory/NativeAllocator.java index 61e8407,3d4ec16..af8b750 --- a/src/java/org/apache/cassandra/utils/memory/NativeAllocator.java +++ b/src/java/org/apache/cassandra/utils/memory/NativeAllocator.java @@@ -215,14 -184,9 +215,9 @@@ public class NativeAllocator extends Me * Offset for the next allocation, or the sentinel value -1 * which implies that the region is still uninitialized. */ -private AtomicInteger nextFreeOffset = new AtomicInteger(0); +private final AtomicInteger nextFreeOffset = new AtomicInteger(0); /** - * Total number of allocations satisfied from this buffer - */ - private final AtomicInteger allocCount = new AtomicInteger(); - - /** * Create an uninitialized region. Note that memory is not allocated yet, so * this is cheap. * diff --cc src/java/org/apache/cassandra/utils/memory/SlabAllocator.java index 1db4b7f,5a8ec18..f72a2c3 --- a/src/java/org/apache/cassandra/utils/memory/SlabAllocator.java +++ b/src/java/org/apache/cassandra/utils/memory/SlabAllocator.java @@@ -176,14 -169,9 +176,9 @@@ public class SlabAllocator extends Memt * Offset for the next allocation, or the sentinel value -1 * which implies that the region is still uninitialized. */ -private AtomicInteger nextFreeOffset = new AtomicInteger(0); +private final AtomicInteger nextFreeOffset = new AtomicInteger(0); /** - * Total number of allocations satisfied from this buffer - */ - private final AtomicInteger allocCount = new AtomicInteger(); - - /** * Create an uninitialized region. Note that memory is not allocated yet, so * this is cheap. * - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated: Remove CAS failures and competition in Region.allocate(..) methods with use of getAndAdd(..) updates instead of compareAndSet(..) loops.
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-3.0 by this push:
new 5a6d52b Remove CAS failures and competition in Region.allocate(..)
methods with use of getAndAdd(..) updates instead of compareAndSet(..) loops.
5a6d52b is described below
commit 5a6d52b26191ab1f5df6e9cf941e0f964dd95a28
Author: Mick Semb Wever
AuthorDate: Sun Jul 5 14:06:25 2020 +0200
Remove CAS failures and competition in Region.allocate(..) methods with use
of getAndAdd(..) updates instead of compareAndSet(..) loops.
patch by Mick Semb Wever; reviewed by Robert Stupp, Benedict Elliott Smith
for CASSANDRA-15922
---
CHANGES.txt| 1 +
.../cassandra/utils/memory/NativeAllocator.java| 34 ++
.../cassandra/utils/memory/SlabAllocator.java | 31 +---
3 files changed, 18 insertions(+), 48 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index d94041b..a363005 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.21
+ * Avoid thread starvation, and improve compare-and-swap performance, in the
slab allocators (CASSANDRA-15922)
* Fix broken KEYS 2i queries after DROP COMPACT STORAGE (CASSANDRA-15906)
* Add token to tombstone warning and error messages (CASSANDRA-15890)
* Fixed range read concurrency factor computation and capped as 10 times tpc
cores (CASSANDRA-15752)
diff --git a/src/java/org/apache/cassandra/utils/memory/NativeAllocator.java
b/src/java/org/apache/cassandra/utils/memory/NativeAllocator.java
index 67f2a36..3d4ec16 100644
--- a/src/java/org/apache/cassandra/utils/memory/NativeAllocator.java
+++ b/src/java/org/apache/cassandra/utils/memory/NativeAllocator.java
@@ -24,7 +24,6 @@ import java.util.concurrent.Semaphore;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicReference;
-import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.db.DecoratedKey;
import org.apache.cassandra.db.NativeDecoratedKey;
import org.apache.cassandra.db.rows.Row;
@@ -188,11 +187,6 @@ public class NativeAllocator extends MemtableAllocator
private AtomicInteger nextFreeOffset = new AtomicInteger(0);
/**
- * Total number of allocations satisfied from this buffer
- */
-private AtomicInteger allocCount = new AtomicInteger();
-
-/**
* Create an uninitialized region. Note that memory is not allocated
yet, so
* this is cheap.
*
@@ -207,34 +201,24 @@ public class NativeAllocator extends MemtableAllocator
/**
* Try to allocate size bytes from the region.
*
- * @return the successful allocation, or null to indicate
not-enough-space
+ * @return the successful allocation, or -1 to indicate
not-enough-space
*/
long allocate(int size)
{
-while (true)
-{
-int oldOffset = nextFreeOffset.get();
-
-if (oldOffset + size > capacity) // capacity == remaining
-return -1;
-
-// Try to atomically claim this region
-if (nextFreeOffset.compareAndSet(oldOffset, oldOffset + size))
-{
-// we got the alloc
-allocCount.incrementAndGet();
-return peer + oldOffset;
-}
-// we raced and lost alloc, try again
-}
+int newOffset = nextFreeOffset.getAndAdd(size);
+
+if (newOffset + size > capacity)
+// this region is full
+return -1;
+
+return peer + newOffset;
}
@Override
public String toString()
{
return "Region@" + System.identityHashCode(this) +
-" allocs=" + allocCount.get() + "waste=" +
-(capacity - nextFreeOffset.get());
+"waste=" + Math.max(0, capacity - nextFreeOffset.get());
}
}
diff --git a/src/java/org/apache/cassandra/utils/memory/SlabAllocator.java
b/src/java/org/apache/cassandra/utils/memory/SlabAllocator.java
index e797575..5a8ec18 100644
--- a/src/java/org/apache/cassandra/utils/memory/SlabAllocator.java
+++ b/src/java/org/apache/cassandra/utils/memory/SlabAllocator.java
@@ -172,11 +172,6 @@ public class SlabAllocator extends MemtableBufferAllocator
private AtomicInteger nextFreeOffset = new AtomicInteger(0);
/**
- * Total number of allocations satisfied from this buffer
- */
-private AtomicInteger allocCount = new AtomicInteger();
-
-/**
* Create an uninitialized region. Note that memory is not allocated
yet, so
* this is cheap.
*
@@
[jira] [Updated] (CASSANDRA-15922) High CAS failures in NativeAllocator.Region.allocate(..)
[
https://issues.apache.org/jira/browse/CASSANDRA-15922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15922:
---
Status: Ready to Commit (was: Review In Progress)
> High CAS failures in NativeAllocator.Region.allocate(..)
> -
>
> Key: CASSANDRA-15922
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15922
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Memtable
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 4.0, 3.0.x, 3.11.x
>
> Attachments: NativeAllocatorRegion2Test.java,
> NativeAllocatorRegionTest.java, Screen Shot 2020-07-05 at 13.16.10.png,
> Screen Shot 2020-07-05 at 13.26.17.png, Screen Shot 2020-07-05 at
> 13.35.55.png, Screen Shot 2020-07-05 at 13.37.01.png, Screen Shot 2020-07-05
> at 13.48.16.png, Screen Shot 2020-07-06 at 11.35.35.png, Screen Shot
> 2020-07-06 at 11.36.44.png, Screen Shot 2020-07-06 at 13.26.10.png,
> profile_pbdpc23zafsrh_20200702.svg
>
>
> h4. Problem
> The method {{NativeAllocator.Region.allocate(..)}} uses an {{AtomicInteger}}
> for the current offset in the region. Allocations depends on a
> {{.compareAndSet(..)}} call.
> In highly contended environments the CAS failures can be high, starving
> writes in a running Cassandra node.
> h4. Example
> It has been witnessed up to 33% of CPU time stuck in the
> {{NativeAllocator.Region.allocate(..)}} loop (due to the CAS failures) during
> a heavy spark analytics write load.
> These nodes: 40 CPU cores and 256GB ram; have relevant settings
> - {{memtable_allocation_type: offheap_objects}}
> - {{memtable_offheap_space_in_mb: 5120}}
> - {{concurrent_writes: 160}}
> Numerous flamegraphs demonstrate the problem. See attached
> [^profile_pbdpc23zafsrh_20200702.svg].
> h4. Suggestion: ThreadLocal Regions
> One possible solution is to have separate Regions per thread.
> Code wise this is relatively easy to do, for example replacing
> NativeAllocator:59
> {code}private final AtomicReference currentRegion = new
> AtomicReference<>();{code}
> with
> {code}private final ThreadLocal> currentRegion = new
> ThreadLocal<>() {...};{code}
> But this approach substantially changes the allocation behaviour, with more
> than concurrent_writes number of Regions in use at any one time. For example
> with {{concurrent_writes: 160}} that's 160+ regions, each of 1MB.
> h4. Suggestion: Simple Contention Management Algorithm (Constant Backoff)
> Another possible solution is to introduce a contention management algorithm
> that a) reduces CAS failures in high contention environments, b) doesn't
> impact normal environments, and c) keeps the allocation strategy of using one
> region at a time.
> The research paper [arXiv:1305.5800|https://arxiv.org/abs/1305.5800]
> describes this contention CAS problem and demonstrates a number of algorithms
> to apply. The simplest of these algorithms is the Constant Backoff CAS
> Algorithm.
> Applying the Constant Backoff CAS Algorithm involves adding one line of code
> to {{NativeAllocator.Region.allocate(..)}} to sleep for one (or some constant
> number) nanoseconds after a CAS failure occurs.
> That is...
> {code}
> // we raced and lost alloc, try again
> LockSupport.parkNanos(1);
> {code}
> h4. Constant Backoff CAS Algorithm Experiments
> Using the code attached in NativeAllocatorRegionTest.java the concurrency and
> CAS failures of {{NativeAllocator.Region.allocate(..)}} can be demonstrated.
> In the attached [^NativeAllocatorRegionTest.java] class, which can be run
> standalone, the {{Region}} class: copied from {{NativeAllocator.Region}}; has
> also the {{casFailures}} field added. The following two screenshots are from
> data collected from this class on a 6 CPU (12 core) MBP, running the
> {{NativeAllocatorRegionTest.testRegionCAS}} method.
> This attached screenshot shows the number of CAS failures during the life of
> a Region (over ~215 million allocations), using different threads and park
> times. This illustrates the improvement (reduction) of CAS failures from zero
> park time, through orders of magnitude, up to 1000ns (10ms). The biggest
> improvement is from no algorithm to a park time of 1ns where CAS failures are
> ~two orders of magnitude lower. From a park time 10μs and higher there is a
> significant drop also at low contention rates.
> !Screen Shot 2020-07-05 at 13.16.10.png|width=500px!
> This attached screenshot shows the time it takes to fill a Region (~215
> million allocations), using different threads and park times. The biggest
> improvement is from no algorithm to a park time of 1ns where performance is
> one order of magnitude faster. From a park time of 100μs and higher there is
> a
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152607#comment-17152607
]
Benedict Elliott Smith commented on CASSANDRA-15234:
bq. would you mind starting the discussion on the dev list?
Happy to get more directly involved to move this along, if you like. In this
case I will put together a specific proposal to bring to the dev list, to avoid
it being an aimless discussion.
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15925) Jenkins pipeline can copy wrong test report artefacts from dtest stage builds
[ https://issues.apache.org/jira/browse/CASSANDRA-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever reassigned CASSANDRA-15925: -- Assignee: Michael Semb Wever > Jenkins pipeline can copy wrong test report artefacts from dtest stage builds > - > > Key: CASSANDRA-15925 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15925 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > > Spotted in > https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/196/console > Looks like copyArtifact will need to be specific to a build. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15925) Jenkins pipeline can copy wrong test report artefacts from dtest stage builds
Michael Semb Wever created CASSANDRA-15925: -- Summary: Jenkins pipeline can copy wrong test report artefacts from dtest stage builds Key: CASSANDRA-15925 URL: https://issues.apache.org/jira/browse/CASSANDRA-15925 Project: Cassandra Issue Type: Bug Components: CI Reporter: Michael Semb Wever Spotted in https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/196/console Looks like copyArtifact will need to be specific to a build. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15925) Jenkins pipeline can copy wrong test report artefacts from dtest stage builds
[ https://issues.apache.org/jira/browse/CASSANDRA-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15925: --- Bug Category: Parent values: Correctness(12982)Level 1 values: Test Failure(12990) Complexity: Low Hanging Fruit Discovered By: Unit Test Fix Version/s: 4.0-rc Severity: Low Status: Open (was: Triage Needed) > Jenkins pipeline can copy wrong test report artefacts from dtest stage builds > - > > Key: CASSANDRA-15925 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15925 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 4.0-rc > > > Spotted in > https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/196/console > Looks like copyArtifact will need to be specific to a build. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15901) Fix unit tests to load test/conf/cassandra.yaml (so to listen on a valid ip)
[
https://issues.apache.org/jira/browse/CASSANDRA-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152600#comment-17152600
]
Berenguer Blasi commented on CASSANDRA-15901:
-
Ok after some investigation [~mck] discovered the final jenkins report is
collecting results from other runs! If you check the stages one by one and make
sure commit id matches we see none of those failures. This is jenkins being
broken.
> Fix unit tests to load test/conf/cassandra.yaml (so to listen on a valid ip)
>
>
> Key: CASSANDRA-15901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15901
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0, 4.0-alpha5
>
>
> Many of the ci-cassandra jenkins runs fail on {{ip-10-0-5-5: Name or service
> not known}}. CASSANDRA-15622 addressed some of these but many still remain.
> Currently test C* nodes are either failing or listening on a public ip
> depending on which agent they end up.
> The idea behind this ticket is to make ant force the private VPC ip in the
> cassandra yaml when building, this will force the nodes to listen on the
> correct ip.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15901) Fix unit tests to load test/conf/cassandra.yaml (so to listen on a valid ip)
[
https://issues.apache.org/jira/browse/CASSANDRA-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152599#comment-17152599
]
Michael Semb Wever commented on CASSANDRA-15901:
bq. the full pipeline failed on dtest replica_side_filtering_test which passes
locally and passes on circle-ci. Is there a chance to re-run that?
this was a first-time-witnessed bug where the Jenkins pipeline copied the test
report artefact from the wrong build. investigating this separately.
> Fix unit tests to load test/conf/cassandra.yaml (so to listen on a valid ip)
>
>
> Key: CASSANDRA-15901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15901
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0, 4.0-alpha5
>
>
> Many of the ci-cassandra jenkins runs fail on {{ip-10-0-5-5: Name or service
> not known}}. CASSANDRA-15622 addressed some of these but many still remain.
> Currently test C* nodes are either failing or listening on a public ip
> depending on which agent they end up.
> The idea behind this ticket is to make ant force the private VPC ip in the
> cassandra yaml when building, this will force the nodes to listen on the
> correct ip.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15901) Fix unit tests to load test/conf/cassandra.yaml (so to listen on a valid ip)
[
https://issues.apache.org/jira/browse/CASSANDRA-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15901:
---
Status: Ready to Commit (was: Review In Progress)
> Fix unit tests to load test/conf/cassandra.yaml (so to listen on a valid ip)
>
>
> Key: CASSANDRA-15901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15901
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Many of the ci-cassandra jenkins runs fail on {{ip-10-0-5-5: Name or service
> not known}}. CASSANDRA-15622 addressed some of these but many still remain.
> Currently test C* nodes are either failing or listening on a public ip
> depending on which agent they end up.
> The idea behind this ticket is to make ant force the private VPC ip in the
> cassandra yaml when building, this will force the nodes to listen on the
> correct ip.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15901) Fix unit tests to load test/conf/cassandra.yaml (so to listen on a valid ip)
[
https://issues.apache.org/jira/browse/CASSANDRA-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15901:
---
Fix Version/s: (was: 4.0-rc)
4.0-alpha5
4.0
Since Version: 0.3
Source Control Link:
https://github.com/apache/cassandra/commit/4654ef09c1d3736e0b50e8d5756664cbf9e4ca84
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed as [4654ef09c1d3736e0b50e8d5756664cbf9e4ca84
|https://github.com/apache/cassandra/commit/4654ef09c1d3736e0b50e8d5756664cbf9e4ca84].
> Fix unit tests to load test/conf/cassandra.yaml (so to listen on a valid ip)
>
>
> Key: CASSANDRA-15901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15901
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0, 4.0-alpha5
>
>
> Many of the ci-cassandra jenkins runs fail on {{ip-10-0-5-5: Name or service
> not known}}. CASSANDRA-15622 addressed some of these but many still remain.
> Currently test C* nodes are either failing or listening on a public ip
> depending on which agent they end up.
> The idea behind this ticket is to make ant force the private VPC ip in the
> cassandra yaml when building, this will force the nodes to listen on the
> correct ip.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: FBUtilities.getJustLocalAddress falls back to lo ip on misconfigured nodes
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 4654ef0 FBUtilities.getJustLocalAddress falls back to lo ip on
misconfigured nodes
4654ef0 is described below
commit 4654ef09c1d3736e0b50e8d5756664cbf9e4ca84
Author: Bereng
AuthorDate: Thu Jul 2 17:44:22 2020 +0200
FBUtilities.getJustLocalAddress falls back to lo ip on misconfigured nodes
patch by Berenguer Blasi; reviewed by Robert Stupp, Mick Semb Wever for
CASSANDRA-15901
---
CHANGES.txt| 1 +
conf/cassandra.yaml| 3 ++-
.../org/apache/cassandra/utils/FBUtilities.java| 29 --
.../org/apache/cassandra/utils/JMXServerUtils.java | 12 +++--
.../org/apache/cassandra/auth/jmx/JMXAuthTest.java | 2 +-
5 files changed, 35 insertions(+), 12 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index c3fdf4f..76cd271 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0-alpha5
+ * FBUtilities.getJustLocalAddress falls back to lo ip on misconfigured nodes
(CASSANDRA-15901)
* Close channel and reduce buffer allocation during entire sstable streaming
with SSL (CASSANDRA-15900)
* Prune expired messages less frequently in internode messaging
(CASSANDRA-15700)
* Fix Ec2Snitch handling of legacy mode for dc names matching both formats,
eg "us-west-2" (CASSANDRA-15878)
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index 0778628..80dbf38 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -630,7 +630,8 @@ ssl_storage_port: 7001
# Leaving it blank leaves it up to InetAddress.getLocalHost(). This
# will always do the Right Thing _if_ the node is properly configured
# (hostname, name resolution, etc), and the Right Thing is to use the
-# address associated with the hostname (it might not be).
+# address associated with the hostname (it might not be). If unresolvable
+# it will fall back to InetAddress.getLoopbackAddress(), which is wrong for
production systems.
#
# Setting listen_address to 0.0.0.0 is always wrong.
#
diff --git a/src/java/org/apache/cassandra/utils/FBUtilities.java
b/src/java/org/apache/cassandra/utils/FBUtilities.java
index 6db50a4..8e000d5 100644
--- a/src/java/org/apache/cassandra/utils/FBUtilities.java
+++ b/src/java/org/apache/cassandra/utils/FBUtilities.java
@@ -130,16 +130,29 @@ public class FBUtilities
public static InetAddress getJustLocalAddress()
{
if (localInetAddress == null)
-try
-{
-localInetAddress = DatabaseDescriptor.getListenAddress() ==
null
-? InetAddress.getLocalHost()
-: DatabaseDescriptor.getListenAddress();
-}
-catch (UnknownHostException e)
+{
+if (DatabaseDescriptor.getListenAddress() == null)
{
-throw new RuntimeException(e);
+try
+{
+localInetAddress = InetAddress.getLocalHost();
+logger.info("InetAddress.getLocalHost() was used to
resolve listen_address to {}, double check this is "
++ "correct. Please check your node's config
and set the listen_address in cassandra.yaml accordingly if applicable.",
+localInetAddress);
+}
+catch(UnknownHostException e)
+{
+logger.info("InetAddress.getLocalHost() could not resolve
the address for the hostname ({}), please "
++ "check your node's config and set the
listen_address in cassandra.yaml. Falling back to {}",
+e,
+InetAddress.getLoopbackAddress());
+// CASSANDRA-15901 fallback for misconfigured nodes
+localInetAddress = InetAddress.getLoopbackAddress();
+}
}
+else
+localInetAddress = DatabaseDescriptor.getListenAddress();
+}
return localInetAddress;
}
diff --git a/src/java/org/apache/cassandra/utils/JMXServerUtils.java
b/src/java/org/apache/cassandra/utils/JMXServerUtils.java
index 12036f9..1f79a33 100644
--- a/src/java/org/apache/cassandra/utils/JMXServerUtils.java
+++ b/src/java/org/apache/cassandra/utils/JMXServerUtils.java
@@ -47,6 +47,7 @@ import javax.rmi.ssl.SslRMIClientSocketFactory;
import javax.rmi.ssl.SslRMIServerSocketFactory;
import javax.security.auth.Subject;
+import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableMap;
import org.apache.commons.lang3.StringUtils;
import org.slf4j.Logger;
@@ -63,7 +64,8 @@ public class JMXServerUtils
*
[jira] [Updated] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15234:
---
Fix Version/s: (was: 4.0-alpha)
4.0-beta
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15234:
---
Status: Patch Available (was: In Progress)
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15234:
---
Status: Patch Available (was: Review In Progress)
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15234:
---
Status: In Progress (was: Patch Available)
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152593#comment-17152593
]
Benjamin Lerer commented on CASSANDRA-15234:
{quote}Can we mark this ticket and the grouping one as non-blocking for
beta{quote}
I will do that.
{quote}Can we start the conversation and start asking operators of Cassandra
clusters on their thoughts on grouping vs not grouping? Grouping could be nice
for humans but could be horrid for some automation (I am neither pro or against
grouping, I defer to operators preference here).{quote}
[~benedict], [~maedhroz] as you are the ones that suggested the change, would
you mind starting the discussion on the dev list?
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152590#comment-17152590
]
Benedict Elliott Smith commented on CASSANDRA-15234:
bq. We can ... not deprecat[e] the old API and not announc[e] ... the new API
This would be fine by me. My -1 as expressed applied only to the public API
changes. All of the other changes in this ticket are absolutely good to go
(once review is completed, of course).
Also to reiterate again (since things easily get lost in such a contentious
exchange):
* I think this work is great, and am really glad [~e.dimitrova] undertook it.
* I don't personally mind too strongly _which_ API we settle on, just expect
that we settle on one upfront before committing the userbase to it.
* I think it's a shame that this has seemingly turned into a heightened and
acrimonious exchange.
* It's worth remembering this situation is not unique; we have all experienced
having to revisit work (or abandon it, even - my GitHub is a graveyard of good
intentions) to our frustration and disappointment; it's a part of OSS
development.
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15901) Fix unit tests to load test/conf/cassandra.yaml (so to listen on a valid ip)
[
https://issues.apache.org/jira/browse/CASSANDRA-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152581#comment-17152581
]
Berenguer Blasi commented on CASSANDRA-15901:
-
[~mck] the full pipeline failed on dtest
[replica_side_filtering_test|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/196/testReport/]
which passes locally and passes on circle-ci. Is there a chance to re-run
that? I know this is an extra day of waiting before commit, but it'll be better
than merging sthg broken, despite I have the intuition this is probably a legit
flaky.
> Fix unit tests to load test/conf/cassandra.yaml (so to listen on a valid ip)
>
>
> Key: CASSANDRA-15901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15901
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Many of the ci-cassandra jenkins runs fail on {{ip-10-0-5-5: Name or service
> not known}}. CASSANDRA-15622 addressed some of these but many still remain.
> Currently test C* nodes are either failing or listening on a public ip
> depending on which agent they end up.
> The idea behind this ticket is to make ant force the private VPC ip in the
> cassandra yaml when building, this will force the nodes to listen on the
> correct ip.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15924) Avoid emitting empty range tombstones from RangeTombstoneList
[
https://issues.apache.org/jira/browse/CASSANDRA-15924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152563#comment-17152563
]
Sylvain Lebresne commented on CASSANDRA-15924:
--
Left a minor comment on github for making operators priority explicit in the
2nd commit, but lgtm otherwise, +1.
> Avoid emitting empty range tombstones from RangeTombstoneList
> -
>
> Key: CASSANDRA-15924
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15924
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> In {{RangeTombstoneList#iterator}} there is a chance we emit empty range
> tombstones depending on the slice passed in. This can happen during read
> repair with either an empty slice or with paging and the final page being
> empty.
> This creates problems in RTL if we try to insert a new range tombstone which
> covers the empty ones;
> {code}
> Caused by: java.lang.AssertionError
> at
> org.apache.cassandra.db.RangeTombstoneList.insertFrom(RangeTombstoneList.java:541)
> at
> org.apache.cassandra.db.RangeTombstoneList.addAll(RangeTombstoneList.java:217)
> at
> org.apache.cassandra.db.MutableDeletionInfo.add(MutableDeletionInfo.java:141)
> at
> org.apache.cassandra.db.partitions.AtomicBTreePartition.addAllWithSizeDelta(AtomicBTreePartition.java:137)
> at org.apache.cassandra.db.Memtable.put(Memtable.java:254)
> at
> org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:1210)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:573)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:421)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:210)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:215)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:224)
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.executeInternalWithoutCondition(ModificationStatement.java:582)
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:572)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15894) JAVA 8: test_multiple_repair - repair_tests.incremental_repair_test.TestIncRepair
[ https://issues.apache.org/jira/browse/CASSANDRA-15894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152558#comment-17152558 ] Benjamin Lerer edited comment on CASSANDRA-15894 at 7/7/20, 8:15 AM: - My guess is that some tests were border line in term of duration and that the team in charge of the tests at that time discovered that if you run them twice you reduce the number of failures (the dtest project had different owners over time). For that test I would simply try to see if we can increase reasonably the timeout at the point of failure. was (Author: blerer): My guess is that some tests were border line in term of duration and that the team in charge of the tests at that time discovered that if you run them twice you reduce the number of failures (the dtest project had different owner over time). For that test I would simply try to see if we can increase reasonably the timeout at the point of failure. > JAVA 8: test_multiple_repair - > repair_tests.incremental_repair_test.TestIncRepair > - > > Key: CASSANDRA-15894 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15894 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Ekaterina Dimitrova >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-rc > > Attachments: test_multiple_repair_log.txt > > > JAVA 8: test_multiple_repair - > repair_tests.incremental_repair_test.TestIncRepair > Fails locally and in CircleCI: > https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/222/workflows/46515d14-9be4-4edb-8db4-5930312d2bfb/jobs/1329 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15894) JAVA 8: test_multiple_repair - repair_tests.incremental_repair_test.TestIncRepair
[ https://issues.apache.org/jira/browse/CASSANDRA-15894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152558#comment-17152558 ] Benjamin Lerer commented on CASSANDRA-15894: My guess is that some tests were border line in term of duration and that the team in charge of the tests at that time discovered that if you run them twice you reduce the number of failures (the dtest project had different owner over time). For that test I would simply try to see if we can increase reasonably the timeout at the point of failure. > JAVA 8: test_multiple_repair - > repair_tests.incremental_repair_test.TestIncRepair > - > > Key: CASSANDRA-15894 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15894 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Ekaterina Dimitrova >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-rc > > Attachments: test_multiple_repair_log.txt > > > JAVA 8: test_multiple_repair - > repair_tests.incremental_repair_test.TestIncRepair > Fails locally and in CircleCI: > https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/222/workflows/46515d14-9be4-4edb-8db4-5930312d2bfb/jobs/1329 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15685) flaky testWithMismatchingPending - org.apache.cassandra.distributed.test.PreviewRepairTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152542#comment-17152542
]
Marcus Eriksson commented on CASSANDRA-15685:
-
lets fix the test and open a new jira for any other improvements?
> flaky testWithMismatchingPending -
> org.apache.cassandra.distributed.test.PreviewRepairTest
> --
>
> Key: CASSANDRA-15685
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15685
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Kevin Gallardo
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-beta
>
> Attachments: log-CASSANDRA-15685.txt, output
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Observed in:
> https://app.circleci.com/pipelines/github/newkek/cassandra/34/workflows/1c6b157d-13c3-48a9-85fb-9fe8c153256b/jobs/191/tests
> Failure:
> {noformat}
> testWithMismatchingPending -
> org.apache.cassandra.distributed.test.PreviewRepairTest
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.distributed.test.PreviewRepairTest.testWithMismatchingPending(PreviewRepairTest.java:97)
> {noformat}
> [Circle
> CI|https://circleci.com/gh/dcapwell/cassandra/tree/bug%2FCASSANDRA-15685]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15851) Add bytebuddy support for in-jvm dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152536#comment-17152536 ] Marcus Eriksson commented on CASSANDRA-15851: - trunk: [patch|https://github.com/krummas/cassandra/commits/marcuse/15851] [unit tests|https://circleci.com/gh/krummas/cassandra/3429] [jvm dtests|https://circleci.com/gh/krummas/cassandra/3428] [jvm upgrade dtests|https://circleci.com/gh/krummas/cassandra/3452] (only running them on trunk because it is a pita updating the cci conf for custom branches) [java11 jvm dtests|https://circleci.com/gh/krummas/cassandra/3451] 3.11: [patch|https://github.com/krummas/cassandra/commits/marcuse/15851-3.11] [unit tests|https://circleci.com/gh/krummas/cassandra/3424] [jvm dtests|https://circleci.com/gh/krummas/cassandra/3423] 3.0: [patch|https://github.com/krummas/cassandra/commits/marcuse/15851-3.0] [unit tests|https://circleci.com/gh/krummas/cassandra/3427] [jvm dtests|https://circleci.com/gh/krummas/cassandra/3426] 2.2: [patch|https://github.com/krummas/cassandra/commits/marcuse/15851-2.2] [unit tests|https://circleci.com/gh/krummas/cassandra/3421] [jvm dtests|https://circleci.com/gh/krummas/cassandra/3422] > Add bytebuddy support for in-jvm dtests > --- > > Key: CASSANDRA-15851 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15851 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Labels: pull-request-available > Fix For: 4.0 > > > Old python dtests support byteman, but that is quite horrible to work with, > [bytebuddy|https://bytebuddy.net/#/] is much better, so we should add support > for that in the in-jvm dtests. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15859) Avoid per-host hinted-handoff throttle being rounded to 0 in large cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-15859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15859:
-
Reviewers: Robert Stupp, Robert Stupp (was: Robert Stupp)
Robert Stupp, Robert Stupp
Status: Review In Progress (was: Patch Available)
+1
Mind backporting it to 3.11 and 3.0?
> Avoid per-host hinted-handoff throttle being rounded to 0 in large cluster
> --
>
> Key: CASSANDRA-15859
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15859
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Hints
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> When "hinted_handoff_throttle_in_kb" is sufficiently small or num of nodes in
> the cluster is sufficiently large, the per-host throttle will be rounded to
> 0, aka. unthrottled.
>
> {code:java|title=HintsDispatchExecutor.java}
> int throttleInKB = DatabaseDescriptor.getHintedHandoffThrottleInKB() /
> nodesCount;
> this.rateLimiter = RateLimiter.create(throttleInKB == 0 ? Double.MAX_VALUE :
> throttleInKB * 1024);
> {code}
> [trunk-patch|https://github.com/apache/cassandra/pull/616]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
