[jira] [Comment Edited] (CASSANDRA-15797) Fix flaky BinLogTest - org.apache.cassandra.utils.binlog.BinLogTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110858#comment-17110858
]
Vinay Chella edited comment on CASSANDRA-15797 at 5/19/20, 5:23 AM:
Changes LGTM, +1. [~yifanc].
was (Author: vinaykumarcse):
Changes LGTM [~yifanc].
> Fix flaky BinLogTest - org.apache.cassandra.utils.binlog.BinLogTest
> ---
>
> Key: CASSANDRA-15797
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15797
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Jon Meredith
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> An internal CI system is failing BinLogTest somewhat frequently under JDK11.
> Configuration was recently changed to reduce the number of cores the tests
> run with, however it is reproducible on an 8 core laptop.
> {code}
> [junit-timeout] OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC
> was deprecated in version 9.0 and will likely be removed in a future release.
> [junit-timeout] Testsuite: org.apache.cassandra.utils.binlog.BinLogTest
> [Junit-timeout] WARNING: An illegal reflective access operation has occurred
> [junit-timeout] WARNING: Illegal reflective access by
> net.openhft.chronicle.core.Jvm (file:/.../lib/chronicle-core-1.16.4.jar) to
> field java.nio.Bits.RESERVED_MEMORY
> [junit-timeout] WARNING: Please consider reporting this to the maintainers of
> net.openhft.chronicle.core.Jvm
> [junit-timeout] WARNING: Use --illegal-access=warn to enable warnings of
> further illegal reflective access operations
> [junit-timeout] WARNING: All illegal access operations will be denied in a
> future release
> [junit-timeout] Testsuite: org.apache.cassandra.utils.binlog.BinLogTest Tests
> run: 13, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 13.895 sec
> [junit-timeout]
> [junit-timeout] Testcase:
> testPutAfterStop(org.apache.cassandra.utils.binlog.BinLogTest): FAILED
> [junit-timeout] expected: but
> was:
> [junit-timeout] junit.framework.AssertionFailedError: expected: but
> was:
> [junit-timeout] at
> org.apache.cassandra.utils.binlog.BinLogTest.testPutAfterStop(BinLogTest.java:431)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit-timeout]
> [junit-timeout]
> [junit-timeout] Test org.apache.cassandra.utils.binlog.BinLogTest FAILED
> {code}
> There's also a different failure under JDK8
> {code}
> junit-timeout] Testsuite: org.apache.cassandra.utils.binlog.BinLogTest
> [junit-timeout] Testsuite: org.apache.cassandra.utils.binlog.BinLogTest Tests
> run: 13, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 15.273 sec
> [junit-timeout]
> [junit-timeout] Testcase:
> testBinLogStartStop(org.apache.cassandra.utils.binlog.BinLogTest): FAILED
> [junit-timeout] expected:<2> but was:<0>
> [junit-timeout] junit.framework.AssertionFailedError: expected:<2> but was:<0>
> [junit-timeout] at
> org.apache.cassandra.utils.binlog.BinLogTest.testBinLogStartStop(BinLogTest.java:172)
> [junit-timeout]
> [junit-timeout]
> [junit-timeout] Test org.apache.cassandra.utils.binlog.BinLogTest FAILED
> {code}
> Reproducer
> {code}
> PASSED=0; time { while ant testclasslist -Dtest.classlistprefix=unit
> -Dtest.classlistfile=<(echo
> org/apache/cassandra/utils/binlog/BinLogTest.java); do PASSED=$((PASSED+1));
> echo PASSED $PASSED; done }; echo FAILED after $PASSED runs.
> {code}
> In the last four attempts it has taken 31, 38, 27 and 10 rounds respectively
> under JDK11 and took 51 under JDK8 (about 15 minutes).
> I have not tried running in a cpu-limited container or anything like that yet.
> Additionally, this went past in the logs a few times (under JDK11). No idea
> if it's just an artifact of weird test setup, or something more serious.
> {code}
> [junit-timeout] WARNING: Please consider reporting this to the maintainers of
> net.openhft.chronicle.core.Jvm
> [junit-timeout] WARNING: Use --illegal-access=warn to enable warnings of
> further illegal reflective access operations
> [junit-timeout] WARNING: All illegal access operations will be denied in a
> future release
> [junit-timeout] Testsuite: org.apache.cassandra.utils.binlog.BinLogTest Tests
> run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.839 sec
> [junit-timeout]
> [junit-timeout] java.lang.Throwable: 1e53135d-main creation ref-count=1
> [junit-timeout] at
>
[jira] [Comment Edited] (CASSANDRA-15748) Provide ability to configure IAuditLogger
[
https://issues.apache.org/jira/browse/CASSANDRA-15748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110859#comment-17110859
]
Vinay Chella edited comment on CASSANDRA-15748 at 5/19/20, 5:23 AM:
Latest changes LGTM, +1.
{quote}yes, this was done on purpose, I think I had this discussion with Marcus
privately, the thing is that I dont know how to propagate map
from nodetool's enableauditlogging - there is not any way to pass map from the
command line (at least the straightforward one). If you look closely into
StorageServiceMBean, that patch contains another enableAuditLog method which
can accept a map. It is there for cases somebody really wants to pass some
parameters via map there via JMX. There is as well the original method without
that map, because in tooling like JConsole, that method is not invokable
anymore if it contains a map, so there is not any way to invoke it via JMX in
JConsole, hence the original method is left there for this purpose{quote}
This sounds fair to me.
{quote}
It is there for cases somebody really wants to pass some parameters via map
there via JMX.
{quote}
Yes, let's add this to the documentation.
was (Author: vinaykumarcse):
{quote}yes, this was done on purpose, I think I had this discussion with Marcus
privately, the thing is that I dont know how to propagate map
from nodetool's enableauditlogging - there is not any way to pass map from the
command line (at least the straightforward one). If you look closely into
StorageServiceMBean, that patch contains another enableAuditLog method which
can accept a map. It is there for cases somebody really wants to pass some
parameters via map there via JMX. There is as well the original method without
that map, because in tooling like JConsole, that method is not invokable
anymore if it contains a map, so there is not any way to invoke it via JMX in
JConsole, hence the original method is left there for this purpose{quote}
This sounds fair to me.
{quote}
It is there for cases somebody really wants to pass some parameters via map
there via JMX.
{quote}
Yes, let's add this to the documentation.
> Provide ability to configure IAuditLogger
> -
>
> Key: CASSANDRA-15748
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15748
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Other
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Currently there is a parameterless constructor expected for IAuditLogger
> instances. There is not any way how to "inject" configuration properties from
> cassandra.yaml into these concrete classes. This would be very handy for
> custom IAuditLogger implementations.
>
> PR: [https://github.com/apache/cassandra/pull/555]
> [|https://github.com/smiklosovic/cassandra/tree/CASSANDRA-15748]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15748) Provide ability to configure IAuditLogger
[
https://issues.apache.org/jira/browse/CASSANDRA-15748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110859#comment-17110859
]
Vinay Chella commented on CASSANDRA-15748:
--
{quote}yes, this was done on purpose, I think I had this discussion with Marcus
privately, the thing is that I dont know how to propagate map
from nodetool's enableauditlogging - there is not any way to pass map from the
command line (at least the straightforward one). If you look closely into
StorageServiceMBean, that patch contains another enableAuditLog method which
can accept a map. It is there for cases somebody really wants to pass some
parameters via map there via JMX. There is as well the original method without
that map, because in tooling like JConsole, that method is not invokable
anymore if it contains a map, so there is not any way to invoke it via JMX in
JConsole, hence the original method is left there for this purpose{quote}
This sounds fair to me.
{quote}
It is there for cases somebody really wants to pass some parameters via map
there via JMX.
{quote}
Yes, let's add this to the documentation.
> Provide ability to configure IAuditLogger
> -
>
> Key: CASSANDRA-15748
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15748
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Other
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Currently there is a parameterless constructor expected for IAuditLogger
> instances. There is not any way how to "inject" configuration properties from
> cassandra.yaml into these concrete classes. This would be very handy for
> custom IAuditLogger implementations.
>
> PR: [https://github.com/apache/cassandra/pull/555]
> [|https://github.com/smiklosovic/cassandra/tree/CASSANDRA-15748]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15797) Fix flaky BinLogTest - org.apache.cassandra.utils.binlog.BinLogTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110858#comment-17110858
]
Vinay Chella commented on CASSANDRA-15797:
--
Changes LGTM [~yifanc].
> Fix flaky BinLogTest - org.apache.cassandra.utils.binlog.BinLogTest
> ---
>
> Key: CASSANDRA-15797
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15797
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Jon Meredith
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> An internal CI system is failing BinLogTest somewhat frequently under JDK11.
> Configuration was recently changed to reduce the number of cores the tests
> run with, however it is reproducible on an 8 core laptop.
> {code}
> [junit-timeout] OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC
> was deprecated in version 9.0 and will likely be removed in a future release.
> [junit-timeout] Testsuite: org.apache.cassandra.utils.binlog.BinLogTest
> [Junit-timeout] WARNING: An illegal reflective access operation has occurred
> [junit-timeout] WARNING: Illegal reflective access by
> net.openhft.chronicle.core.Jvm (file:/.../lib/chronicle-core-1.16.4.jar) to
> field java.nio.Bits.RESERVED_MEMORY
> [junit-timeout] WARNING: Please consider reporting this to the maintainers of
> net.openhft.chronicle.core.Jvm
> [junit-timeout] WARNING: Use --illegal-access=warn to enable warnings of
> further illegal reflective access operations
> [junit-timeout] WARNING: All illegal access operations will be denied in a
> future release
> [junit-timeout] Testsuite: org.apache.cassandra.utils.binlog.BinLogTest Tests
> run: 13, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 13.895 sec
> [junit-timeout]
> [junit-timeout] Testcase:
> testPutAfterStop(org.apache.cassandra.utils.binlog.BinLogTest): FAILED
> [junit-timeout] expected: but
> was:
> [junit-timeout] junit.framework.AssertionFailedError: expected: but
> was:
> [junit-timeout] at
> org.apache.cassandra.utils.binlog.BinLogTest.testPutAfterStop(BinLogTest.java:431)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit-timeout]
> [junit-timeout]
> [junit-timeout] Test org.apache.cassandra.utils.binlog.BinLogTest FAILED
> {code}
> There's also a different failure under JDK8
> {code}
> junit-timeout] Testsuite: org.apache.cassandra.utils.binlog.BinLogTest
> [junit-timeout] Testsuite: org.apache.cassandra.utils.binlog.BinLogTest Tests
> run: 13, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 15.273 sec
> [junit-timeout]
> [junit-timeout] Testcase:
> testBinLogStartStop(org.apache.cassandra.utils.binlog.BinLogTest): FAILED
> [junit-timeout] expected:<2> but was:<0>
> [junit-timeout] junit.framework.AssertionFailedError: expected:<2> but was:<0>
> [junit-timeout] at
> org.apache.cassandra.utils.binlog.BinLogTest.testBinLogStartStop(BinLogTest.java:172)
> [junit-timeout]
> [junit-timeout]
> [junit-timeout] Test org.apache.cassandra.utils.binlog.BinLogTest FAILED
> {code}
> Reproducer
> {code}
> PASSED=0; time { while ant testclasslist -Dtest.classlistprefix=unit
> -Dtest.classlistfile=<(echo
> org/apache/cassandra/utils/binlog/BinLogTest.java); do PASSED=$((PASSED+1));
> echo PASSED $PASSED; done }; echo FAILED after $PASSED runs.
> {code}
> In the last four attempts it has taken 31, 38, 27 and 10 rounds respectively
> under JDK11 and took 51 under JDK8 (about 15 minutes).
> I have not tried running in a cpu-limited container or anything like that yet.
> Additionally, this went past in the logs a few times (under JDK11). No idea
> if it's just an artifact of weird test setup, or something more serious.
> {code}
> [junit-timeout] WARNING: Please consider reporting this to the maintainers of
> net.openhft.chronicle.core.Jvm
> [junit-timeout] WARNING: Use --illegal-access=warn to enable warnings of
> further illegal reflective access operations
> [junit-timeout] WARNING: All illegal access operations will be denied in a
> future release
> [junit-timeout] Testsuite: org.apache.cassandra.utils.binlog.BinLogTest Tests
> run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.839 sec
> [junit-timeout]
> [junit-timeout] java.lang.Throwable: 1e53135d-main creation ref-count=1
> [junit-timeout] at
> net.openhft.chronicle.core.ReferenceCounter.newRefCountHistory(ReferenceCounter.java:45)
> [junit-timeout] at
>
[jira] [Updated] (CASSANDRA-15797) Fix flaky BinLogTest - org.apache.cassandra.utils.binlog.BinLogTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinay Chella updated CASSANDRA-15797:
-
Status: Ready to Commit (was: Review In Progress)
> Fix flaky BinLogTest - org.apache.cassandra.utils.binlog.BinLogTest
> ---
>
> Key: CASSANDRA-15797
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15797
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Jon Meredith
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> An internal CI system is failing BinLogTest somewhat frequently under JDK11.
> Configuration was recently changed to reduce the number of cores the tests
> run with, however it is reproducible on an 8 core laptop.
> {code}
> [junit-timeout] OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC
> was deprecated in version 9.0 and will likely be removed in a future release.
> [junit-timeout] Testsuite: org.apache.cassandra.utils.binlog.BinLogTest
> [Junit-timeout] WARNING: An illegal reflective access operation has occurred
> [junit-timeout] WARNING: Illegal reflective access by
> net.openhft.chronicle.core.Jvm (file:/.../lib/chronicle-core-1.16.4.jar) to
> field java.nio.Bits.RESERVED_MEMORY
> [junit-timeout] WARNING: Please consider reporting this to the maintainers of
> net.openhft.chronicle.core.Jvm
> [junit-timeout] WARNING: Use --illegal-access=warn to enable warnings of
> further illegal reflective access operations
> [junit-timeout] WARNING: All illegal access operations will be denied in a
> future release
> [junit-timeout] Testsuite: org.apache.cassandra.utils.binlog.BinLogTest Tests
> run: 13, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 13.895 sec
> [junit-timeout]
> [junit-timeout] Testcase:
> testPutAfterStop(org.apache.cassandra.utils.binlog.BinLogTest): FAILED
> [junit-timeout] expected: but
> was:
> [junit-timeout] junit.framework.AssertionFailedError: expected: but
> was:
> [junit-timeout] at
> org.apache.cassandra.utils.binlog.BinLogTest.testPutAfterStop(BinLogTest.java:431)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit-timeout]
> [junit-timeout]
> [junit-timeout] Test org.apache.cassandra.utils.binlog.BinLogTest FAILED
> {code}
> There's also a different failure under JDK8
> {code}
> junit-timeout] Testsuite: org.apache.cassandra.utils.binlog.BinLogTest
> [junit-timeout] Testsuite: org.apache.cassandra.utils.binlog.BinLogTest Tests
> run: 13, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 15.273 sec
> [junit-timeout]
> [junit-timeout] Testcase:
> testBinLogStartStop(org.apache.cassandra.utils.binlog.BinLogTest): FAILED
> [junit-timeout] expected:<2> but was:<0>
> [junit-timeout] junit.framework.AssertionFailedError: expected:<2> but was:<0>
> [junit-timeout] at
> org.apache.cassandra.utils.binlog.BinLogTest.testBinLogStartStop(BinLogTest.java:172)
> [junit-timeout]
> [junit-timeout]
> [junit-timeout] Test org.apache.cassandra.utils.binlog.BinLogTest FAILED
> {code}
> Reproducer
> {code}
> PASSED=0; time { while ant testclasslist -Dtest.classlistprefix=unit
> -Dtest.classlistfile=<(echo
> org/apache/cassandra/utils/binlog/BinLogTest.java); do PASSED=$((PASSED+1));
> echo PASSED $PASSED; done }; echo FAILED after $PASSED runs.
> {code}
> In the last four attempts it has taken 31, 38, 27 and 10 rounds respectively
> under JDK11 and took 51 under JDK8 (about 15 minutes).
> I have not tried running in a cpu-limited container or anything like that yet.
> Additionally, this went past in the logs a few times (under JDK11). No idea
> if it's just an artifact of weird test setup, or something more serious.
> {code}
> [junit-timeout] WARNING: Please consider reporting this to the maintainers of
> net.openhft.chronicle.core.Jvm
> [junit-timeout] WARNING: Use --illegal-access=warn to enable warnings of
> further illegal reflective access operations
> [junit-timeout] WARNING: All illegal access operations will be denied in a
> future release
> [junit-timeout] Testsuite: org.apache.cassandra.utils.binlog.BinLogTest Tests
> run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.839 sec
> [junit-timeout]
> [junit-timeout] java.lang.Throwable: 1e53135d-main creation ref-count=1
> [junit-timeout] at
> net.openhft.chronicle.core.ReferenceCounter.newRefCountHistory(ReferenceCounter.java:45)
> [junit-timeout] at
>
[jira] [Updated] (CASSANDRA-15820) tools/bin/fqltool doesn't work on all distributions
[
https://issues.apache.org/jira/browse/CASSANDRA-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15820:
-
Test and Documentation Plan: Local dtest run
Status: Patch Available (was: Open)
> tools/bin/fqltool doesn't work on all distributions
> ---
>
> Key: CASSANDRA-15820
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15820
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/fql
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> The line
> {code}
> if [ ! $1 ]; then break; fi
> {code}
> doesn't work on all OSes/Linux distributions (e.g. a bare Ubuntu 18.04) with
> {{#!/bin/sh}} causing {{fqltool}} to fail. The fix is quite simple
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15820) tools/bin/fqltool doesn't work on all distributions
[
https://issues.apache.org/jira/browse/CASSANDRA-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15820:
-
Bug Category: Parent values: Code(13163)Level 1 values: Bug - Unclear
Impact(13164)
Complexity: Low Hanging Fruit
Discovered By: Adhoc Test
Severity: Normal
Status: Open (was: Triage Needed)
> tools/bin/fqltool doesn't work on all distributions
> ---
>
> Key: CASSANDRA-15820
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15820
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/fql
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> The line
> {code}
> if [ ! $1 ]; then break; fi
> {code}
> doesn't work on all OSes/Linux distributions (e.g. a bare Ubuntu 18.04) with
> {{#!/bin/sh}} causing {{fqltool}} to fail. The fix is quite simple
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15820) tools/bin/fqltool doesn't work on all distributions
[
https://issues.apache.org/jira/browse/CASSANDRA-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110855#comment-17110855
]
Robert Stupp commented on CASSANDRA-15820:
--
PR: https://github.com/apache/cassandra/pull/595
> tools/bin/fqltool doesn't work on all distributions
> ---
>
> Key: CASSANDRA-15820
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15820
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/fql
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> The line
> {code}
> if [ ! $1 ]; then break; fi
> {code}
> doesn't work on all OSes/Linux distributions (e.g. a bare Ubuntu 18.04) with
> {{#!/bin/sh}} causing {{fqltool}} to fail. The fix is quite simple
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15820) tools/bin/fqltool doesn't work on all distributions
[
https://issues.apache.org/jira/browse/CASSANDRA-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15820:
-
Fix Version/s: 4.0-alpha
> tools/bin/fqltool doesn't work on all distributions
> ---
>
> Key: CASSANDRA-15820
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15820
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/fql
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> The line
> {code}
> if [ ! $1 ]; then break; fi
> {code}
> doesn't work on all OSes/Linux distributions (e.g. a bare Ubuntu 18.04) with
> {{#!/bin/sh}} causing {{fqltool}} to fail. The fix is quite simple
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15820) tools/bin/fqltool doesn't work on all distributions
Robert Stupp created CASSANDRA-15820:
Summary: tools/bin/fqltool doesn't work on all distributions
Key: CASSANDRA-15820
URL: https://issues.apache.org/jira/browse/CASSANDRA-15820
Project: Cassandra
Issue Type: Bug
Components: Tool/fql
Reporter: Robert Stupp
Assignee: Robert Stupp
The line
{code}
if [ ! $1 ]; then break; fi
{code}
doesn't work on all OSes/Linux distributions (e.g. a bare Ubuntu 18.04) with
{{#!/bin/sh}} causing {{fqltool}} to fail. The fix is quite simple
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15819) nodetool enablefullquerylog doesn't allow caller to make non-blocking
[
https://issues.apache.org/jira/browse/CASSANDRA-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110743#comment-17110743
]
David Capwell commented on CASSANDRA-15819:
---
Its possible the defaults for the cluster are set to false, so absence of
--blocking would be non-blocking, but in the cases where the configs are true
and you should still be able to mark non-blocking.
> nodetool enablefullquerylog doesn't allow caller to make non-blocking
> -
>
> Key: CASSANDRA-15819
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15819
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> {code}
> $ ./bin/nodetool enablefullquerylog --path /tmp/deleteme --blocking false
> Picked up _JAVA_OPTIONS: -Djava.net.preferIPv4Stack=true
> nodetool: Found unexpected parameters: [false]
> See 'nodetool help' or 'nodetool help '.
> {code}
> The root cause is boolean is special cased in airlift, so any time —blocking
> is set it gets turned on.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15819) nodetool enablefullquerylog doesn't allow caller to make non-blocking
[
https://issues.apache.org/jira/browse/CASSANDRA-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110735#comment-17110735
]
David Capwell commented on CASSANDRA-15819:
---
Seeing how this feature was added to 4.0, we should not release with this bug;
so marked it for beta.
> nodetool enablefullquerylog doesn't allow caller to make non-blocking
> -
>
> Key: CASSANDRA-15819
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15819
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> {code}
> $ ./bin/nodetool enablefullquerylog --path /tmp/deleteme --blocking false
> Picked up _JAVA_OPTIONS: -Djava.net.preferIPv4Stack=true
> nodetool: Found unexpected parameters: [false]
> See 'nodetool help' or 'nodetool help '.
> {code}
> The root cause is boolean is special cased in airlift, so any time —blocking
> is set it gets turned on.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15819) nodetool enablefullquerylog doesn't allow caller to make non-blocking
[
https://issues.apache.org/jira/browse/CASSANDRA-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-15819:
--
Fix Version/s: 4.0-beta
> nodetool enablefullquerylog doesn't allow caller to make non-blocking
> -
>
> Key: CASSANDRA-15819
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15819
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> {code}
> $ ./bin/nodetool enablefullquerylog --path /tmp/deleteme --blocking false
> Picked up _JAVA_OPTIONS: -Djava.net.preferIPv4Stack=true
> nodetool: Found unexpected parameters: [false]
> See 'nodetool help' or 'nodetool help '.
> {code}
> The root cause is boolean is special cased in airlift, so any time —blocking
> is set it gets turned on.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15819) nodetool enablefullquerylog doesn't allow caller to make non-blocking
David Capwell created CASSANDRA-15819:
-
Summary: nodetool enablefullquerylog doesn't allow caller to make
non-blocking
Key: CASSANDRA-15819
URL: https://issues.apache.org/jira/browse/CASSANDRA-15819
Project: Cassandra
Issue Type: Bug
Components: Tool/nodetool
Reporter: David Capwell
{code}
$ ./bin/nodetool enablefullquerylog --path /tmp/deleteme --blocking false
Picked up _JAVA_OPTIONS: -Djava.net.preferIPv4Stack=true
nodetool: Found unexpected parameters: [false]
See 'nodetool help' or 'nodetool help '.
{code}
The root cause is boolean is special cased in airlift, so any time —blocking is
set it gets turned on.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15819) nodetool enablefullquerylog doesn't allow caller to make non-blocking
[
https://issues.apache.org/jira/browse/CASSANDRA-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-15819:
--
Bug Category: Parent values: Correctness(12982)Level 1 values: API /
Semantic Implementation(12988)
Complexity: Low Hanging Fruit
Discovered By: User Report
Severity: Low
Status: Open (was: Triage Needed)
> nodetool enablefullquerylog doesn't allow caller to make non-blocking
> -
>
> Key: CASSANDRA-15819
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15819
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: David Capwell
>Priority: Normal
>
> {code}
> $ ./bin/nodetool enablefullquerylog --path /tmp/deleteme --blocking false
> Picked up _JAVA_OPTIONS: -Djava.net.preferIPv4Stack=true
> nodetool: Found unexpected parameters: [false]
> See 'nodetool help' or 'nodetool help '.
> {code}
> The root cause is boolean is special cased in airlift, so any time —blocking
> is set it gets turned on.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14608) Confirm correctness of windows scripts post-9608
[ https://issues.apache.org/jira/browse/CASSANDRA-14608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hazem reassigned CASSANDRA-14608: - Authors: (was: Yuki Morishita) Component/s: (was: Packaging) Fix Version/s: (was: 4.0) Assignee: (was: Yuki Morishita) Labels: (was: Java11 Windows) > Confirm correctness of windows scripts post-9608 > > > Key: CASSANDRA-14608 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14608 > Project: Cassandra > Issue Type: Task > Environment: Windows >Reporter: Jason Brown >Priority: Urgent > > In CASSANDRA-9608, we chose to defer making all the changes to Windows > scripts. This ticket is to ensure that we do that work. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15619) Counter tables should use a 4KB chunk length by default
[ https://issues.apache.org/jira/browse/CASSANDRA-15619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swen Moczarski reassigned CASSANDRA-15619: -- Assignee: Swen Moczarski > Counter tables should use a 4KB chunk length by default > --- > > Key: CASSANDRA-15619 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15619 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Counters >Reporter: Jon Haddad >Assignee: Swen Moczarski >Priority: Normal > > One of the issues I found when using counters is throughput gets > significantly limited due to the default chunk length. When tuning from 64 > to 4, I saw a massive improvement in both throughput and latency. > Our default chunk length is 16, which is fine for general purpose tables, but > since tables with counters are special purpose we should give users this > optimization automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15636) Refactor protocol related code to eliminate magic numbers
[ https://issues.apache.org/jira/browse/CASSANDRA-15636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110558#comment-17110558 ] Dinesh Joshi commented on CASSANDRA-15636: -- hi [~Dean Z], sorry about the delay in responding. I did a quick scan of your patch and I think this ticket should go a lot further. However, if you're looking for LHF with a bit more clarity, please take a look at the flakey test jiras that are open. This ticket is a bit nebulous so I'd avoid it for now. > Refactor protocol related code to eliminate magic numbers > - > > Key: CASSANDRA-15636 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15636 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Client >Reporter: Dinesh Joshi >Priority: Low > Attachments: patch.diff > > > Parts of Cassandra's codebase has protocol related magic numbers. It makes > the code hard to understand. We should refactor the code to remove magic > numbers and instead replace them with versioned constants. This will help > make the code clearer. Example: > https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/transport/messages/ErrorMessage.java#L288 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15636) Refactor protocol related code to eliminate magic numbers
[ https://issues.apache.org/jira/browse/CASSANDRA-15636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi reassigned CASSANDRA-15636: Assignee: Dinesh Joshi > Refactor protocol related code to eliminate magic numbers > - > > Key: CASSANDRA-15636 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15636 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Client >Reporter: Dinesh Joshi >Assignee: Dinesh Joshi >Priority: Low > Attachments: patch.diff > > > Parts of Cassandra's codebase has protocol related magic numbers. It makes > the code hard to understand. We should refactor the code to remove magic > numbers and instead replace them with versioned constants. This will help > make the code clearer. Example: > https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/transport/messages/ErrorMessage.java#L288 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14557) Consider adding default and required keyspace replication options
[
https://issues.apache.org/jira/browse/CASSANDRA-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110462#comment-17110462
]
Sumanth Pasupuleti edited comment on CASSANDRA-14557 at 5/18/20, 6:57 PM:
--
Based on the suggestion in the dev list, I've updated the patch to include a
note about impact on system keyspaces in cassandra.yaml.
Updated github branch:
https://github.com/sumanth-pasupuleti/cassandra/tree/14557-trunk
Added patch file: [^14557-4.0.txt]
was (Author: sumanth.pasupuleti):
Based on the suggestion in the dev list, I've updated the patch to include a
note about impact on system keyspaces in cassandra.yaml.
Updated github branch:
https://github.com/sumanth-pasupuleti/cassandra/tree/14557-trunk
Updated patch file: [^14557-4.0.txt]
> Consider adding default and required keyspace replication options
> -
>
> Key: CASSANDRA-14557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14557
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
> Attachments: 14557-4.0.txt, 14557-trunk.patch
>
>
> Ending up with a keyspace of RF=1 is unfortunately pretty easy in C* right
> now - the system_auth table for example is created with RF=1 (to take into
> account single node setups afaict from CASSANDRA-5112), and a user can
> further create a keyspace with RF=1 posing availability and streaming risks
> (e.g. rebuild).
> I propose we add two configuration options in cassandra.yaml:
> # {{default_keyspace_rf}} (default: 1) - If replication factors are not
> specified, use this number.
> # {{required_minimum_keyspace_rf}} (default: unset) - Prevent users from
> creating a keyspace with an RF less than what is configured
> These settings could further be re-used to:
> * Provide defaults for new keyspaces created with SimpleStrategy or
> NetworkTopologyStrategy (CASSANDRA-14303)
> * Make the automatic token [allocation
> algorithm|https://issues.apache.org/jira/browse/CASSANDRA-13701?focusedCommentId=16095662=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16095662]
> interface more intuitive allowing easy use of the new token allocation
> algorithm.
> At the end of the day, if someone really wants to allow RF=1, they simply
> don’t set the setting. For backwards compatibility the default remains 1 and
> C* would create with RF=1, and would default to current behavior of allowing
> any RF on keyspaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15805) Potential duplicate rows on 2.X->3.X upgrade when multi-rows range tombstones interacts with collection tombstones
[
https://issues.apache.org/jira/browse/CASSANDRA-15805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110514#comment-17110514
]
Aleksey Yeschenko commented on CASSANDRA-15805:
---
Nit, feel free to ignore: in {{UnfilteredDeserializer}} on line 701, redundant
{{this}}. Otherwise pretty straightforward and LGTM.
> Potential duplicate rows on 2.X->3.X upgrade when multi-rows range tombstones
> interacts with collection tombstones
> --
>
> Key: CASSANDRA-15805
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15805
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination, Local/SSTable
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
> The legacy reading code ({{LegacyLayout}} and
> {{UnfilteredDeserializer.OldFormatDeserializer}}) does not handle correctly
> the case where a range tombstone covering multiple rows interacts with a
> collection tombstone.
> A simple example of this problem is if one runs on 2.X:
> {noformat}
> CREATE TABLE t (
> k int,
> c1 text,
> c2 text,
> a text,
> b set,
> c text,
> PRIMARY KEY((k), c1, c2)
> );
> // Delete all rows where c1 is 'A'
> DELETE FROM t USING TIMESTAMP 1 WHERE k = 0 AND c1 = 'A';
> // Inserts a row covered by that previous range tombstone
> INSERT INTO t(k, c1, c2, a, b, c) VALUES (0, 'A', 'X', 'foo', {'whatever'},
> 'bar') USING TIMESTAMP 2;
> // Delete the collection of that previously inserted row
> DELETE b FROM t USING TIMESTAMP 3 WHERE k = 0 AND c1 = 'A' and c2 = 'X';
> {noformat}
> If the following is ran on 2.X (with everything either flushed in the same
> table or compacted together), then this will result in the inserted row being
> duplicated (one part containing the {{a}} column, the other the {{c}} one).
> I will note that this is _not_ a duplicate of CASSANDRA-15789 and this
> reproduce even with the fix to {{LegacyLayout}} of this ticket. That said,
> the additional code added to CASSANDRA-15789 to force merging duplicated rows
> if they are produced _will_ end up fixing this as a consequence (assuming
> there is no variation of this problem that leads to other visible issues than
> duplicated rows). That said, I "think" we'd still rather fix the source of
> the issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15805) Potential duplicate rows on 2.X->3.X upgrade when multi-rows range tombstones interacts with collection tombstones
[
https://issues.apache.org/jira/browse/CASSANDRA-15805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-15805:
--
Status: Ready to Commit (was: Review In Progress)
> Potential duplicate rows on 2.X->3.X upgrade when multi-rows range tombstones
> interacts with collection tombstones
> --
>
> Key: CASSANDRA-15805
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15805
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination, Local/SSTable
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
> The legacy reading code ({{LegacyLayout}} and
> {{UnfilteredDeserializer.OldFormatDeserializer}}) does not handle correctly
> the case where a range tombstone covering multiple rows interacts with a
> collection tombstone.
> A simple example of this problem is if one runs on 2.X:
> {noformat}
> CREATE TABLE t (
> k int,
> c1 text,
> c2 text,
> a text,
> b set,
> c text,
> PRIMARY KEY((k), c1, c2)
> );
> // Delete all rows where c1 is 'A'
> DELETE FROM t USING TIMESTAMP 1 WHERE k = 0 AND c1 = 'A';
> // Inserts a row covered by that previous range tombstone
> INSERT INTO t(k, c1, c2, a, b, c) VALUES (0, 'A', 'X', 'foo', {'whatever'},
> 'bar') USING TIMESTAMP 2;
> // Delete the collection of that previously inserted row
> DELETE b FROM t USING TIMESTAMP 3 WHERE k = 0 AND c1 = 'A' and c2 = 'X';
> {noformat}
> If the following is ran on 2.X (with everything either flushed in the same
> table or compacted together), then this will result in the inserted row being
> duplicated (one part containing the {{a}} column, the other the {{c}} one).
> I will note that this is _not_ a duplicate of CASSANDRA-15789 and this
> reproduce even with the fix to {{LegacyLayout}} of this ticket. That said,
> the additional code added to CASSANDRA-15789 to force merging duplicated rows
> if they are produced _will_ end up fixing this as a consequence (assuming
> there is no variation of this problem that leads to other visible issues than
> duplicated rows). That said, I "think" we'd still rather fix the source of
> the issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15807) Support multiple keyspaces in Cassandra-Diff
[
https://issues.apache.org/jira/browse/CASSANDRA-15807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110486#comment-17110486
]
Yifan Cai commented on CASSANDRA-15807:
---
Updated the PR with the comments in there.
* Removed todos.
* Renamed the column in various tables from {{keyspace_table_name*}} to
{{qualified_table_name*}}.
* Removed unused imports.
Attaching the [^local_e2e_log.txt] from the local spark run.
> Support multiple keyspaces in Cassandra-Diff
>
>
> Key: CASSANDRA-15807
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15807
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/diff
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Labels: pull-request-available
> Attachments: local_e2e_log.txt
>
>
> Adding the support to run diff comparison of tables across multiple keyspaces
> to avoid bringing up multiple spark jobs and give a better view of data diffs
> in the whole cluster.
> Cassandra-diff currently only support compare multiple tables under one
> single keyspace.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15807) Support multiple keyspaces in Cassandra-Diff
[ https://issues.apache.org/jira/browse/CASSANDRA-15807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRA-15807: -- Attachment: local_e2e_log.txt > Support multiple keyspaces in Cassandra-Diff > > > Key: CASSANDRA-15807 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15807 > Project: Cassandra > Issue Type: Improvement > Components: Tool/diff >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Labels: pull-request-available > Attachments: local_e2e_log.txt > > > Adding the support to run diff comparison of tables across multiple keyspaces > to avoid bringing up multiple spark jobs and give a better view of data diffs > in the whole cluster. > Cassandra-diff currently only support compare multiple tables under one > single keyspace. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15778) CorruptSSTableException after a 2.1 SSTable is upgraded to 3.0, failing reads
[
https://issues.apache.org/jira/browse/CASSANDRA-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-15778:
Status: Patch Available (was: In Progress)
I think I have a patch I think solves the problem without introducing breaking
changes:
|[circleci|https://circleci.com/gh/ifesdjeen/cassandra/tree/15778-3.0]|[3.0
patch|https://github.com/apache/cassandra/compare/trunk...ifesdjeen:15778-3.0?expand=1]|
It does the following things:
* makes {{EmptyType}} restrictions slightly stricter withouot changing
semantics. Just make sure we can not write values with empty type.
* changes {{LegacySchemaMigrator}}, allowing to start a 3.0 node during
upgrade with {{-D cassandra.init_dense_table_compact_value_as_bytes=true}},
which would initialize a dense table compact value as {{ByteType}}. It will be
initialized as {{EmptyType}} by default.
* improves an error message in {{EmptySerializer}} for anyone who have faced
written data in the column that was supposed to be empty, referring them here.
* since we initialize with {{EmptyType}} by default, in case someone has
proceeded to upgrade their nodes withouot even testing an upgrade out (and
being able to just start upgrade with an aforementioned flag), allow {{ALTER
TABLE ALTER TYPE}} _only_ from {{EmptyType}} to {{BytesType}} and _only_ for a
dense table.
We should not change compact storage behaviour to suddenly reveal a
{{BytesType}} column for a huge userbase. Instead, we help out anyone who may
have written values with Thrift: either to _prevent_ seeing this issue (which
is ideal), or to _quickly fix_ it after upgrade. Schema changes won't propagate
accross versions, but they only need to propagate to 3.0 nodes only anyways.
> CorruptSSTableException after a 2.1 SSTable is upgraded to 3.0, failing reads
> -
>
> Key: CASSANDRA-15778
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15778
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction, Local/SSTable
>Reporter: Sumanth Pasupuleti
>Assignee: Alex Petrov
>Priority: Normal
> Fix For: 3.0.x
>
>
> Below is the exception with stack trace. This issue is consistently
> reproduce-able.
> {code:java}
> ERROR [SharedPool-Worker-1] 2020-05-01 14:57:57,661
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[SharedPool-Worker-1,5,main]ERROR [SharedPool-Worker-1] 2020-05-01
> 14:57:57,661 AbstractLocalAwareExecutorService.java:169 - Uncaught exception
> on thread
> Thread[SharedPool-Worker-1,5,main]org.apache.cassandra.io.sstable.CorruptSSTableException:
> Corrupted:
> /mnt/data/cassandra/data// at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:349)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:220)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.columniterator.SSTableIterator.hasNext(SSTableIterator.java:33)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:131)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
>
[jira] [Commented] (CASSANDRA-14557) Consider adding default and required keyspace replication options
[
https://issues.apache.org/jira/browse/CASSANDRA-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110462#comment-17110462
]
Sumanth Pasupuleti commented on CASSANDRA-14557:
Based on the suggestion in the dev list, I've updated the patch to include a
note about impact on system keyspaces in cassandra.yaml.
Updated github branch:
https://github.com/sumanth-pasupuleti/cassandra/tree/14557-trunk
Updated patch file: [^14557-4.0.txt]
> Consider adding default and required keyspace replication options
> -
>
> Key: CASSANDRA-14557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14557
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
> Attachments: 14557-4.0.txt, 14557-trunk.patch
>
>
> Ending up with a keyspace of RF=1 is unfortunately pretty easy in C* right
> now - the system_auth table for example is created with RF=1 (to take into
> account single node setups afaict from CASSANDRA-5112), and a user can
> further create a keyspace with RF=1 posing availability and streaming risks
> (e.g. rebuild).
> I propose we add two configuration options in cassandra.yaml:
> # {{default_keyspace_rf}} (default: 1) - If replication factors are not
> specified, use this number.
> # {{required_minimum_keyspace_rf}} (default: unset) - Prevent users from
> creating a keyspace with an RF less than what is configured
> These settings could further be re-used to:
> * Provide defaults for new keyspaces created with SimpleStrategy or
> NetworkTopologyStrategy (CASSANDRA-14303)
> * Make the automatic token [allocation
> algorithm|https://issues.apache.org/jira/browse/CASSANDRA-13701?focusedCommentId=16095662=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16095662]
> interface more intuitive allowing easy use of the new token allocation
> algorithm.
> At the end of the day, if someone really wants to allow RF=1, they simply
> don’t set the setting. For backwards compatibility the default remains 1 and
> C* would create with RF=1, and would default to current behavior of allowing
> any RF on keyspaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14557) Consider adding default and required keyspace replication options
[
https://issues.apache.org/jira/browse/CASSANDRA-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumanth Pasupuleti updated CASSANDRA-14557:
---
Attachment: 14557-4.0.txt
> Consider adding default and required keyspace replication options
> -
>
> Key: CASSANDRA-14557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14557
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
> Attachments: 14557-4.0.txt, 14557-trunk.patch
>
>
> Ending up with a keyspace of RF=1 is unfortunately pretty easy in C* right
> now - the system_auth table for example is created with RF=1 (to take into
> account single node setups afaict from CASSANDRA-5112), and a user can
> further create a keyspace with RF=1 posing availability and streaming risks
> (e.g. rebuild).
> I propose we add two configuration options in cassandra.yaml:
> # {{default_keyspace_rf}} (default: 1) - If replication factors are not
> specified, use this number.
> # {{required_minimum_keyspace_rf}} (default: unset) - Prevent users from
> creating a keyspace with an RF less than what is configured
> These settings could further be re-used to:
> * Provide defaults for new keyspaces created with SimpleStrategy or
> NetworkTopologyStrategy (CASSANDRA-14303)
> * Make the automatic token [allocation
> algorithm|https://issues.apache.org/jira/browse/CASSANDRA-13701?focusedCommentId=16095662=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16095662]
> interface more intuitive allowing easy use of the new token allocation
> algorithm.
> At the end of the day, if someone really wants to allow RF=1, they simply
> don’t set the setting. For backwards compatibility the default remains 1 and
> C* would create with RF=1, and would default to current behavior of allowing
> any RF on keyspaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15778) CorruptSSTableException after a 2.1 SSTable is upgraded to 3.0, failing reads
[
https://issues.apache.org/jira/browse/CASSANDRA-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-15778:
Status: In Progress (was: Changes Suggested)
> CorruptSSTableException after a 2.1 SSTable is upgraded to 3.0, failing reads
> -
>
> Key: CASSANDRA-15778
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15778
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction, Local/SSTable
>Reporter: Sumanth Pasupuleti
>Assignee: Alex Petrov
>Priority: Normal
> Fix For: 3.0.x
>
>
> Below is the exception with stack trace. This issue is consistently
> reproduce-able.
> {code:java}
> ERROR [SharedPool-Worker-1] 2020-05-01 14:57:57,661
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[SharedPool-Worker-1,5,main]ERROR [SharedPool-Worker-1] 2020-05-01
> 14:57:57,661 AbstractLocalAwareExecutorService.java:169 - Uncaught exception
> on thread
> Thread[SharedPool-Worker-1,5,main]org.apache.cassandra.io.sstable.CorruptSSTableException:
> Corrupted:
> /mnt/data/cassandra/data// at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:349)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:220)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.columniterator.SSTableIterator.hasNext(SSTableIterator.java:33)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:131)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:77)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:294)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:187)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:180)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:176)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:76)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:341)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:47)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_231] at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:165)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
>
[jira] [Commented] (CASSANDRA-15794) Upgraded C* (4.x) fail to start because of Compact Tables & dropping compact tables in downgraded C* (3.11.4) introduces non-existent columns
[
https://issues.apache.org/jira/browse/CASSANDRA-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110435#comment-17110435
]
Zhuqi Jin commented on CASSANDRA-15794:
---
Thanks for you prompt reply.
I still think that users who have already used compact storage should be
provided with a solution (even assuming CASSANDRA-15811 is fixed). Otherwise,
users may be confused, especially when they roll it back but cannot start 3.11.
For example,
1) Instead of directly block cassandra startup, is it possible to correctly
drop existing compact storage in 4.x (with the fix from CASSANDRA-15811)?
2) Or, if we stick to ask users to downgrade to 3.x and drop compact storage,
can we not generate new commit log before we hit the error in 4.x, so that we
avoid blocking 3.x from starting?
3) Or, block or warn users when they try to introduce compact storage in 3.x?
Throw a warning telling them that compact storage is deprecated in 3.x and will
cause problem upgrading to 4.x, or simply block them from doing that, or at
least modify the manual of 3.X, indicating that you should not use compact
storage.
If you think any of these is reasonable, I can provide help.
> Upgraded C* (4.x) fail to start because of Compact Tables & dropping compact
> tables in downgraded C* (3.11.4) introduces non-existent columns
> -
>
> Key: CASSANDRA-15794
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15794
> Project: Cassandra
> Issue Type: Bug
>Reporter: Zhuqi Jin
>Priority: Normal
>
> We tried to test upgrading a 3.11.4 C* cluster to 4.x and run into the
> following problems.
> * We started a single 3.11.4 C* node.
> * We ran cassandra-stress like this
> {code:java}
> ./cassandra-stress write n = 30 -rate threads = 10 -node 172.17.0.2 {code}
> * We stopped this node, and started a C* node running C* compiled from trunk
> (git commit: e394dc0bb32f612a476269010930c617dd1ed3cb)
> * New C* failed to start with the following error message
> {code:java}
> ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 - Error while
> loading schema: ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245
> - Error while loading schema: java.lang.IllegalArgumentException: Compact
> Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ...
> DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to
> migrate off Compact Storage. at
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
> at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at
> org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770)Exception
> (java.lang.IllegalArgumentException) encountered during startup: Compact
> Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ...
> DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to
> migrate off Compact Storage.ERROR [main] 2020-05-07 00:58:18,520
> CassandraDaemon.java:792 - Exception encountered during
> startupjava.lang.IllegalArgumentException: Compact Tables are not allowed in
> Cassandra starting with 4.0 version. Use `ALTER ... DROP COMPACT STORAGE`
> command supplied in 3.x/3.11 Cassandra in order to migrate off Compact
> Storage. at
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
> at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at
> org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653)
> at
>
[jira] [Updated] (CASSANDRA-15807) Support multiple keyspaces in Cassandra-Diff
[ https://issues.apache.org/jira/browse/CASSANDRA-15807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CASSANDRA-15807: --- Labels: pull-request-available (was: ) > Support multiple keyspaces in Cassandra-Diff > > > Key: CASSANDRA-15807 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15807 > Project: Cassandra > Issue Type: Improvement > Components: Tool/diff >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Labels: pull-request-available > > Adding the support to run diff comparison of tables across multiple keyspaces > to avoid bringing up multiple spark jobs and give a better view of data diffs > in the whole cluster. > Cassandra-diff currently only support compare multiple tables under one > single keyspace. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15685) flaky testWithMismatchingPending - org.apache.cassandra.distributed.test.PreviewRepairTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15685:
Status: In Progress (was: Changes Suggested)
> flaky testWithMismatchingPending -
> org.apache.cassandra.distributed.test.PreviewRepairTest
> --
>
> Key: CASSANDRA-15685
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15685
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Kevin Gallardo
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Attachments: log-CASSANDRA-15685.txt, output
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Observed in:
> https://app.circleci.com/pipelines/github/newkek/cassandra/34/workflows/1c6b157d-13c3-48a9-85fb-9fe8c153256b/jobs/191/tests
> Failure:
> {noformat}
> testWithMismatchingPending -
> org.apache.cassandra.distributed.test.PreviewRepairTest
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.distributed.test.PreviewRepairTest.testWithMismatchingPending(PreviewRepairTest.java:97)
> {noformat}
> [Circle
> CI|https://circleci.com/gh/dcapwell/cassandra/tree/bug%2FCASSANDRA-15685]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15789) Rows can get duplicated in mixed major-version clusters and after full upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-15789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110270#comment-17110270
]
Sylvain Lebresne commented on CASSANDRA-15789:
--
+1 from me.
> Rows can get duplicated in mixed major-version clusters and after full upgrade
> --
>
> Key: CASSANDRA-15789
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15789
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination, Local/Memtable, Local/SSTable
>Reporter: Aleksey Yeschenko
>Assignee: Marcus Eriksson
>Priority: Normal
>
> In a mixed 2.X/3.X major version cluster a sequence of row deletes,
> collection overwrites, paging, and read repair can cause 3.X nodes to split
> individual rows into several rows with identical clustering. This happens due
> to 2.X paging and RT semantics, and a 3.X {{LegacyLayout}} deficiency.
> To reproduce, set up a 2-node mixed major version cluster with the following
> table:
> {code}
> CREATE TABLE distributed_test_keyspace.tlb (
> pk int,
> ck int,
> v map,
> PRIMARY KEY (pk, ck)
> );
> {code}
> 1. Using either node as the coordinator, delete the row with ck=2 using
> timestamp 1
> {code}
> DELETE FROM tbl USING TIMESTAMP 1 WHERE pk = 1 AND ck = 2;
> {code}
> 2. Using either node as the coordinator, insert the following 3 rows:
> {code}
> INSERT INTO tbl (pk, ck, v) VALUES (1, 1, {'e':'f'}) USING TIMESTAMP 3;
> INSERT INTO tbl (pk, ck, v) VALUES (1, 2, {'g':'h'}) USING TIMESTAMP 3;
> INSERT INTO tbl (pk, ck, v) VALUES (1, 3, {'i':'j'}) USING TIMESTAMP 3;
> {code}
> 3. Flush the table on both nodes
> 4. Using the 2.2 node as the coordinator, force read repar by querying the
> table with page size = 2:
>
> {code}
> SELECT * FROM tbl;
> {code}
> 5. Overwrite the row with ck=2 using timestamp 5:
> {code}
> INSERT INTO tbl (pk, ck, v) VALUES (1, 2, {'g':'h'}) USING TIMESTAMP 5;}}
> {code}
> 6. Query the 3.0 node and observe the split row:
> {code}
> cqlsh> select * from distributed_test_keyspace.tlb ;
> pk | ck | v
> ++
> 1 | 1 | {'e': 'f'}
> 1 | 2 | {'g': 'h'}
> 1 | 2 | {'k': 'l'}
> 1 | 3 | {'i': 'j'}
> {code}
> This happens because the read to query the second page ends up generating the
> following mutation for the 3.0 node:
> {code}
> ColumnFamily(tbl -{deletedAt=-9223372036854775808, localDeletion=2147483647,
> ranges=[2:v:_-2:v:!, deletedAt=2, localDeletion=1588588821]
> [2:v:!-2:!, deletedAt=1, localDeletion=1588588821]
> [3:v:_-3:v:!, deletedAt=2, localDeletion=1588588821]}-
> [2:v:63:false:1@3,])
> {code}
> Which on 3.0 side gets incorrectly deserialized as
> {code}
> Mutation(keyspace='distributed_test_keyspace', key='0001', modifications=[
> [distributed_test_keyspace.tbl] key=1
> partition_deletion=deletedAt=-9223372036854775808, localDeletion=2147483647
> columns=[[] | [v]]
> Row[info=[ts=-9223372036854775808] ]: ck=2 | del(v)=deletedAt=2,
> localDeletion=1588588821, [v[c]=d ts=3]
> Row[info=[ts=-9223372036854775808] del=deletedAt=1,
> localDeletion=1588588821 ]: ck=2 |
> Row[info=[ts=-9223372036854775808] ]: ck=3 | del(v)=deletedAt=2,
> localDeletion=1588588821
> ])
> {code}
> {{LegacyLayout}} correctly interprets a range tombstone whose start and
> finish {{collectionName}} values don't match as a wrapping fragment of a
> legacy row deletion that's being interrupted by a collection deletion
> (correctly) - see
> [code|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/LegacyLayout.java#L1874-L1889].
> Quoting the comment inline:
> {code}
> // Because of the way RangeTombstoneList work, we can have a tombstone where
> only one of
> // the bound has a collectionName. That happens if we have a big tombstone A
> (spanning one
> // or multiple rows) and a collection tombstone B. In that case,
> RangeTombstoneList will
> // split this into 3 RTs: the first one from the beginning of A to the
> beginning of B,
> // then B, then a third one from the end of B to the end of A. To make this
> simpler, if
> // we detect that case we transform the 1st and 3rd tombstone so they don't
> end in the middle
> // of a row (which is still correct).
> {code}
> {{LegacyLayout#addRowTombstone()}} method then chokes when it encounters such
> a tombstone in the middle of an existing row - having seen {{v[c]=d}} first,
> and mistakenly starts a new row, while in the middle of an existing one: (see
> [code|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/LegacyLayout.java#L1500-L1501]).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CASSANDRA-8643) merkle tree creation fails with NoSuchElementException
[
https://issues.apache.org/jira/browse/CASSANDRA-8643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sanjay Karnam updated CASSANDRA-8643:
-
Description:
We have a problem that we encountered during testing over the weekend.
During the tests we noticed that repairs started to fail. This error has
occured on multiple non-coordinator nodes during repair. It also ran at least
once without producing this error.
We run repair -pr on all nodes on different days. CPU values were around 40%
and disk was 50% full.
>From what I understand, the coordinator asked for merkle trees from the other
>two nodes. However one of the nodes fails to create his merkle tree.
Unfortunately we do not have a way to reproduce this problem.
The coordinator receives:
{noformat}
2015-01-09T17:55:57.091+0100 INFO [RepairJobTask:4] RepairJob.java:145 [repair
#59455950-9820-11e4-b5c1-7797064e1316] requesting merkle trees for censored (to
[/xx.90, /xx.98, /xx.82])
2015-01-09T17:55:58.516+0100 INFO [AntiEntropyStage:1] RepairSession.java:171
[repair #59455950-9820-11e4-b5c1-7797064e1316] Received merkle tree for
censored from /xx.90
2015-01-09T17:55:59.581+0100 ERROR [AntiEntropySessions:76]
RepairSession.java:303 [repair #59455950-9820-11e4-b5c1-7797064e1316] session
completed with the following error
org.apache.cassandra.exceptions.RepairException: [repair
#59455950-9820-11e4-b5c1-7797064e1316 on censored/censored,
(-6476420463551243930,-6471459119674373580]] Validation failed in /xx.98
at
org.apache.cassandra.repair.RepairSession.validationComplete(RepairSession.java:166)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:384)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:126)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_51]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_51]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
2015-01-09T17:55:59.582+0100 ERROR [AntiEntropySessions:76]
CassandraDaemon.java:153 Exception in thread
Thread[AntiEntropySessions:76,5,RMI Runtime]
java.lang.RuntimeException: org.apache.cassandra.exceptions.RepairException:
[repair #59455950-9820-11e4-b5c1-7797064e1316 on censored/censored,
(-6476420463551243930,-6471459119674373580]] Validation failed in /xx.98
at com.google.common.base.Throwables.propagate(Throwables.java:160)
~[guava-16.0.jar:na]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_51]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
~[na:1.7.0_51]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_51]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_51]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51] Caused by:
org.apache.cassandra.exceptions.RepairException: [repair
#59455950-9820-11e4-b5c1-7797064e1316 on censored/censored,
(-6476420463551243930,-6471459119674373580]] Validation failed in /xx.98
at
org.apache.cassandra.repair.RepairSession.validationComplete(RepairSession.java:166)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:384)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:126)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62)
~[apache-cassandra-2.1.1.jar:2.1.1]
... 3 common frames omitted
{noformat}
While one of the other nodes produces this error:
{noformat}
2015-01-09T17:55:59.574+0100 ERROR [ValidationExecutor:16] Validator.java:232
Failed creating a merkle tree for [repair #59455950-9820-11e4-b5c1-7797064e1316
on censored/censored, (-6476420463551243930,-6471459119674373580]], /xx.82 (see
log for details)
2015-01-09T17:55:59.578+0100 ERROR [ValidationExecutor:16]
CassandraDaemon.java:153 Exception in thread
Thread[ValidationExecutor:16,1,main]
java.util.NoSuchElementException: null
at
com.google.common.collect.AbstractIterator.next(AbstractIterator.java:154)
~[guava-16.0.jar:na]
at org.apache.cassandra.repair.Validator.add(Validator.java:137)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
[jira] [Commented] (CASSANDRA-15789) Rows can get duplicated in mixed major-version clusters and after full upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-15789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110126#comment-17110126
]
Marcus Eriksson commented on CASSANDRA-15789:
-
Pushed a few commits to the branches above addressing the comments;
* Made {{PartitionUpdate.fromPre30Iterator}} log whenever it [merges
rows|https://github.com/krummas/cassandra/commit/7f88a2d60e0fa8da1d328c84d07da3dee6b78b18]
*
[Revert|https://github.com/krummas/cassandra/commit/aedbfc761fa798c8118409cfc07ece47990578d8]
to debug logging for in-jvm dtests
*
[Make|https://github.com/krummas/cassandra/commit/210da67d6cae3b8c01df315d28e89c455bac488e]
it possible to disable read/compaction time duplicate detection
* Various
[minor|https://github.com/krummas/cassandra/commit/7a18bbea6bb5c1203014396d85fc8a3822c7969a]
[fixes|https://github.com/krummas/cassandra/commit/c4373b3b75aa234b4ebd67ffa03ad5553d73e357]
> Rows can get duplicated in mixed major-version clusters and after full upgrade
> --
>
> Key: CASSANDRA-15789
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15789
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination, Local/Memtable, Local/SSTable
>Reporter: Aleksey Yeschenko
>Assignee: Marcus Eriksson
>Priority: Normal
>
> In a mixed 2.X/3.X major version cluster a sequence of row deletes,
> collection overwrites, paging, and read repair can cause 3.X nodes to split
> individual rows into several rows with identical clustering. This happens due
> to 2.X paging and RT semantics, and a 3.X {{LegacyLayout}} deficiency.
> To reproduce, set up a 2-node mixed major version cluster with the following
> table:
> {code}
> CREATE TABLE distributed_test_keyspace.tlb (
> pk int,
> ck int,
> v map,
> PRIMARY KEY (pk, ck)
> );
> {code}
> 1. Using either node as the coordinator, delete the row with ck=2 using
> timestamp 1
> {code}
> DELETE FROM tbl USING TIMESTAMP 1 WHERE pk = 1 AND ck = 2;
> {code}
> 2. Using either node as the coordinator, insert the following 3 rows:
> {code}
> INSERT INTO tbl (pk, ck, v) VALUES (1, 1, {'e':'f'}) USING TIMESTAMP 3;
> INSERT INTO tbl (pk, ck, v) VALUES (1, 2, {'g':'h'}) USING TIMESTAMP 3;
> INSERT INTO tbl (pk, ck, v) VALUES (1, 3, {'i':'j'}) USING TIMESTAMP 3;
> {code}
> 3. Flush the table on both nodes
> 4. Using the 2.2 node as the coordinator, force read repar by querying the
> table with page size = 2:
>
> {code}
> SELECT * FROM tbl;
> {code}
> 5. Overwrite the row with ck=2 using timestamp 5:
> {code}
> INSERT INTO tbl (pk, ck, v) VALUES (1, 2, {'g':'h'}) USING TIMESTAMP 5;}}
> {code}
> 6. Query the 3.0 node and observe the split row:
> {code}
> cqlsh> select * from distributed_test_keyspace.tlb ;
> pk | ck | v
> ++
> 1 | 1 | {'e': 'f'}
> 1 | 2 | {'g': 'h'}
> 1 | 2 | {'k': 'l'}
> 1 | 3 | {'i': 'j'}
> {code}
> This happens because the read to query the second page ends up generating the
> following mutation for the 3.0 node:
> {code}
> ColumnFamily(tbl -{deletedAt=-9223372036854775808, localDeletion=2147483647,
> ranges=[2:v:_-2:v:!, deletedAt=2, localDeletion=1588588821]
> [2:v:!-2:!, deletedAt=1, localDeletion=1588588821]
> [3:v:_-3:v:!, deletedAt=2, localDeletion=1588588821]}-
> [2:v:63:false:1@3,])
> {code}
> Which on 3.0 side gets incorrectly deserialized as
> {code}
> Mutation(keyspace='distributed_test_keyspace', key='0001', modifications=[
> [distributed_test_keyspace.tbl] key=1
> partition_deletion=deletedAt=-9223372036854775808, localDeletion=2147483647
> columns=[[] | [v]]
> Row[info=[ts=-9223372036854775808] ]: ck=2 | del(v)=deletedAt=2,
> localDeletion=1588588821, [v[c]=d ts=3]
> Row[info=[ts=-9223372036854775808] del=deletedAt=1,
> localDeletion=1588588821 ]: ck=2 |
> Row[info=[ts=-9223372036854775808] ]: ck=3 | del(v)=deletedAt=2,
> localDeletion=1588588821
> ])
> {code}
> {{LegacyLayout}} correctly interprets a range tombstone whose start and
> finish {{collectionName}} values don't match as a wrapping fragment of a
> legacy row deletion that's being interrupted by a collection deletion
> (correctly) - see
> [code|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/LegacyLayout.java#L1874-L1889].
> Quoting the comment inline:
> {code}
> // Because of the way RangeTombstoneList work, we can have a tombstone where
> only one of
> // the bound has a collectionName. That happens if we have a big tombstone A
> (spanning one
> // or multiple rows) and a collection tombstone B. In that case,
> RangeTombstoneList will
> // split this into 3 RTs: the first one from the beginning of A to the
> beginning of B,
> // then
[jira] [Commented] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110101#comment-17110101
]
Sam Tunnicliffe commented on CASSANDRA-15299:
-
bq. length fields are just big enough to accommodate the 128 KiB limit.
Wouldn't it be more cautious to keep a bit of margin?
bq.
The layout of the header fields were chosen (in CASSANDRA-15066) for efficiency
and correctness. Specifically regarding correctness, the lengths of both the
header and the frame body are tightly coupled to the CRCs. That is, the
specific lengths are chosen to ensure that the CRC 24/32 provides adequate
protection.
bq. maybe the header should reserve more space for flags
bq.
The framing format is intentionally extremely lightweight and we should be
really cautious about introducing features _at this level of the protocol_
which require flags or any other configuration. I agree wholeheartedly with
your point about previous issues that been made more difficult because of a
lack of protocol extensibility. My counter argument is that those all relate to
the CQL protocol itself, which remains unchanged by this. Changes to the "inner
frame"/CQL messages themselves are decoupled from the "outer frame" format.
bq. why store the uncompressed length in the header? In the legacy frame
format, it's encoded in the compressed payload
bq.
As you mention in the LZ4 case, the uncompressed length isn't part of the
compressed payload, it's prepended (and necessary to allocate a correctly sized
buffer for decompression). The only real difference between having this in the
header and nominally being part of the body is that the integrity of the header
is protected, which saves us from potentially OOMing if the size is corrupted.
> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Labels: protocolv5
> Fix For: 4.0-alpha
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it
> introduced checksumming/CRC32 to request and response bodies. It’s an
> important step forward, but it doesn’t cover the entire stream. In
> particular, the message header is not covered by a checksum or a crc, which
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of
> computing the CRC32 on the bytes written on the wire - losing the properties
> of the CRC32. In some cases, due to this sequencing, attempting to decompress
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order,
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression
> operate on individual message bodies rather than frames of multiple complete
> messages. In reality, this has several important additional downsides. To
> name a couple:
> # For compression, we are getting poor compression ratios for smaller
> messages - when operating on tiny sequences of bytes. In reality, for most
> small requests and responses we are discarding the compressed value as it’d
> be smaller than the uncompressed one - incurring both redundant allocations
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages.
> 4 bytes extra is *a lot* for an empty write response, for example.
> To address the correctness issue of {{streamId}} not being covered by the
> checksum/CRC32 and the inefficiency in compression and checksumming/CRC32, we
> should switch to a framing protocol with multiple messages in a single frame.
> I suggest we reuse the framing protocol recently implemented for internode
> messaging in CASSANDRA-15066 to the extent that its logic can be borrowed,
> and that we do it before native protocol v5 graduates from beta. See
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderCrc.java
> and
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderLZ4.java.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CASSANDRA-15794) Upgraded C* (4.x) fail to start because of Compact Tables & dropping compact tables in downgraded C* (3.11.4) introduces non-existent columns
[
https://issues.apache.org/jira/browse/CASSANDRA-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-15794:
Resolution: Information Provided
Status: Resolved (was: Triage Needed)
> Upgraded C* (4.x) fail to start because of Compact Tables & dropping compact
> tables in downgraded C* (3.11.4) introduces non-existent columns
> -
>
> Key: CASSANDRA-15794
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15794
> Project: Cassandra
> Issue Type: Bug
>Reporter: Zhuqi Jin
>Priority: Normal
>
> We tried to test upgrading a 3.11.4 C* cluster to 4.x and run into the
> following problems.
> * We started a single 3.11.4 C* node.
> * We ran cassandra-stress like this
> {code:java}
> ./cassandra-stress write n = 30 -rate threads = 10 -node 172.17.0.2 {code}
> * We stopped this node, and started a C* node running C* compiled from trunk
> (git commit: e394dc0bb32f612a476269010930c617dd1ed3cb)
> * New C* failed to start with the following error message
> {code:java}
> ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 - Error while
> loading schema: ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245
> - Error while loading schema: java.lang.IllegalArgumentException: Compact
> Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ...
> DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to
> migrate off Compact Storage. at
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
> at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at
> org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770)Exception
> (java.lang.IllegalArgumentException) encountered during startup: Compact
> Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ...
> DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to
> migrate off Compact Storage.ERROR [main] 2020-05-07 00:58:18,520
> CassandraDaemon.java:792 - Exception encountered during
> startupjava.lang.IllegalArgumentException: Compact Tables are not allowed in
> Cassandra starting with 4.0 version. Use `ALTER ... DROP COMPACT STORAGE`
> command supplied in 3.x/3.11 Cassandra in order to migrate off Compact
> Storage. at
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
> at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at
> org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770){code}
> * We stopped the trunk version C* and started the 3.11.4 version C*.
> * 3.11.4 C* failed to start with the following error messages:
> {code:java}
> Exception (java.lang.IllegalStateException) encountered during startup:
> Unknown commitlog version 7Exception (java.lang.IllegalStateException)
> encountered during startup: Unknown commitlog version 7ERROR [main]
> 2020-05-07 01:13:26,798 CassandraDaemon.java:749 - Exception encountered
> during startupjava.lang.IllegalStateException: Unknown commitlog version 7 at
> org.apache.cassandra.db.commitlog.CommitLogDescriptor.getMessagingVersion(CommitLogDescriptor.java:227)
> ~[main/:na] at
> org.apache.cassandra.db.commitlog.CommitLogReader.shouldSkipSegmentId(CommitLogReader.java:276)
> ~[main/:na] at
> org.apache.cassandra.db.commitlog.CommitLogReader.readCommitLogSegment(CommitLogReader.java:220)
> ~[main/:na] at
>
[jira] [Commented] (CASSANDRA-15794) Upgraded C* (4.x) fail to start because of Compact Tables & dropping compact tables in downgraded C* (3.11.4) introduces non-existent columns
[
https://issues.apache.org/jira/browse/CASSANDRA-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110098#comment-17110098
]
Alex Petrov commented on CASSANDRA-15794:
-
This behaviour is correct and is to be expected. I would recommend not using
compact storage when creating 3.x tables, since there's no benefit from using
it after 2.x. There's [CASSANDRA-15811] that will add new syntax and allow to
drop compact storage without having to manually drop shown columns. Meanwhile,
you can just drop the revealed {{value}} ({{column1}} won't be droppable
because it's a clustering column) colummn manually (after running {{DROP
COMPACT STORAGE}}) to unblock yourself and changing schema of cassandra-stress
to actually recognise the column. The best way to proceed would really be not
to use compact storage in 3.0 in the first place.
> Upgraded C* (4.x) fail to start because of Compact Tables & dropping compact
> tables in downgraded C* (3.11.4) introduces non-existent columns
> -
>
> Key: CASSANDRA-15794
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15794
> Project: Cassandra
> Issue Type: Bug
>Reporter: Zhuqi Jin
>Priority: Normal
>
> We tried to test upgrading a 3.11.4 C* cluster to 4.x and run into the
> following problems.
> * We started a single 3.11.4 C* node.
> * We ran cassandra-stress like this
> {code:java}
> ./cassandra-stress write n = 30 -rate threads = 10 -node 172.17.0.2 {code}
> * We stopped this node, and started a C* node running C* compiled from trunk
> (git commit: e394dc0bb32f612a476269010930c617dd1ed3cb)
> * New C* failed to start with the following error message
> {code:java}
> ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 - Error while
> loading schema: ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245
> - Error while loading schema: java.lang.IllegalArgumentException: Compact
> Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ...
> DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to
> migrate off Compact Storage. at
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
> at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at
> org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770)Exception
> (java.lang.IllegalArgumentException) encountered during startup: Compact
> Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ...
> DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to
> migrate off Compact Storage.ERROR [main] 2020-05-07 00:58:18,520
> CassandraDaemon.java:792 - Exception encountered during
> startupjava.lang.IllegalArgumentException: Compact Tables are not allowed in
> Cassandra starting with 4.0 version. Use `ALTER ... DROP COMPACT STORAGE`
> command supplied in 3.x/3.11 Cassandra in order to migrate off Compact
> Storage. at
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
> at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at
> org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770){code}
> * We stopped the trunk version C* and started the 3.11.4 version C*.
> * 3.11.4 C* failed to start with the following error messages:
> {code:java}
> Exception (java.lang.IllegalStateException) encountered during startup:
> Unknown commitlog version 7Exception
[jira] [Updated] (CASSANDRA-15789) Rows can get duplicated in mixed major-version clusters and after full upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-15789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-15789:
Reviewers: Alex Petrov, Sylvain Lebresne (was: Alex Petrov)
> Rows can get duplicated in mixed major-version clusters and after full upgrade
> --
>
> Key: CASSANDRA-15789
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15789
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination, Local/Memtable, Local/SSTable
>Reporter: Aleksey Yeschenko
>Assignee: Marcus Eriksson
>Priority: Normal
>
> In a mixed 2.X/3.X major version cluster a sequence of row deletes,
> collection overwrites, paging, and read repair can cause 3.X nodes to split
> individual rows into several rows with identical clustering. This happens due
> to 2.X paging and RT semantics, and a 3.X {{LegacyLayout}} deficiency.
> To reproduce, set up a 2-node mixed major version cluster with the following
> table:
> {code}
> CREATE TABLE distributed_test_keyspace.tlb (
> pk int,
> ck int,
> v map,
> PRIMARY KEY (pk, ck)
> );
> {code}
> 1. Using either node as the coordinator, delete the row with ck=2 using
> timestamp 1
> {code}
> DELETE FROM tbl USING TIMESTAMP 1 WHERE pk = 1 AND ck = 2;
> {code}
> 2. Using either node as the coordinator, insert the following 3 rows:
> {code}
> INSERT INTO tbl (pk, ck, v) VALUES (1, 1, {'e':'f'}) USING TIMESTAMP 3;
> INSERT INTO tbl (pk, ck, v) VALUES (1, 2, {'g':'h'}) USING TIMESTAMP 3;
> INSERT INTO tbl (pk, ck, v) VALUES (1, 3, {'i':'j'}) USING TIMESTAMP 3;
> {code}
> 3. Flush the table on both nodes
> 4. Using the 2.2 node as the coordinator, force read repar by querying the
> table with page size = 2:
>
> {code}
> SELECT * FROM tbl;
> {code}
> 5. Overwrite the row with ck=2 using timestamp 5:
> {code}
> INSERT INTO tbl (pk, ck, v) VALUES (1, 2, {'g':'h'}) USING TIMESTAMP 5;}}
> {code}
> 6. Query the 3.0 node and observe the split row:
> {code}
> cqlsh> select * from distributed_test_keyspace.tlb ;
> pk | ck | v
> ++
> 1 | 1 | {'e': 'f'}
> 1 | 2 | {'g': 'h'}
> 1 | 2 | {'k': 'l'}
> 1 | 3 | {'i': 'j'}
> {code}
> This happens because the read to query the second page ends up generating the
> following mutation for the 3.0 node:
> {code}
> ColumnFamily(tbl -{deletedAt=-9223372036854775808, localDeletion=2147483647,
> ranges=[2:v:_-2:v:!, deletedAt=2, localDeletion=1588588821]
> [2:v:!-2:!, deletedAt=1, localDeletion=1588588821]
> [3:v:_-3:v:!, deletedAt=2, localDeletion=1588588821]}-
> [2:v:63:false:1@3,])
> {code}
> Which on 3.0 side gets incorrectly deserialized as
> {code}
> Mutation(keyspace='distributed_test_keyspace', key='0001', modifications=[
> [distributed_test_keyspace.tbl] key=1
> partition_deletion=deletedAt=-9223372036854775808, localDeletion=2147483647
> columns=[[] | [v]]
> Row[info=[ts=-9223372036854775808] ]: ck=2 | del(v)=deletedAt=2,
> localDeletion=1588588821, [v[c]=d ts=3]
> Row[info=[ts=-9223372036854775808] del=deletedAt=1,
> localDeletion=1588588821 ]: ck=2 |
> Row[info=[ts=-9223372036854775808] ]: ck=3 | del(v)=deletedAt=2,
> localDeletion=1588588821
> ])
> {code}
> {{LegacyLayout}} correctly interprets a range tombstone whose start and
> finish {{collectionName}} values don't match as a wrapping fragment of a
> legacy row deletion that's being interrupted by a collection deletion
> (correctly) - see
> [code|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/LegacyLayout.java#L1874-L1889].
> Quoting the comment inline:
> {code}
> // Because of the way RangeTombstoneList work, we can have a tombstone where
> only one of
> // the bound has a collectionName. That happens if we have a big tombstone A
> (spanning one
> // or multiple rows) and a collection tombstone B. In that case,
> RangeTombstoneList will
> // split this into 3 RTs: the first one from the beginning of A to the
> beginning of B,
> // then B, then a third one from the end of B to the end of A. To make this
> simpler, if
> // we detect that case we transform the 1st and 3rd tombstone so they don't
> end in the middle
> // of a row (which is still correct).
> {code}
> {{LegacyLayout#addRowTombstone()}} method then chokes when it encounters such
> a tombstone in the middle of an existing row - having seen {{v[c]=d}} first,
> and mistakenly starts a new row, while in the middle of an existing one: (see
> [code|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/LegacyLayout.java#L1500-L1501]).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CASSANDRA-15807) Support multiple keyspaces in Cassandra-Diff
[ https://issues.apache.org/jira/browse/CASSANDRA-15807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-15807: Reviewers: Marcus Eriksson > Support multiple keyspaces in Cassandra-Diff > > > Key: CASSANDRA-15807 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15807 > Project: Cassandra > Issue Type: Improvement > Components: Tool/diff >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > > Adding the support to run diff comparison of tables across multiple keyspaces > to avoid bringing up multiple spark jobs and give a better view of data diffs > in the whole cluster. > Cassandra-diff currently only support compare multiple tables under one > single keyspace. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-diff] branch master updated: Add .asf.yml fixing where MLs notifications are sent to.
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/cassandra-diff.git The following commit(s) were added to refs/heads/master by this push: new 3b24036 Add .asf.yml fixing where MLs notifications are sent to. 3b24036 is described below commit 3b240361651b9341f939e6a5db8bc5157a46adf5 Author: mck AuthorDate: Sat May 16 10:17:13 2020 +0200 Add .asf.yml fixing where MLs notifications are sent to. ref: - https://cwiki.apache.org/confluence/display/INFRA/.asf.yaml+features+for+git+repositories - https://lists.apache.org/thread.html/r2c5a4158218dab3b84c97877d38f0df1cbd54da9b0c337bca63a99ca%40%3Cdev.cassandra.apache.org%3E patch by Mick Semb Wever; reviewed by Yifan Cai --- .asf.yaml | 10 ++ 1 file changed, 10 insertions(+) diff --git a/.asf.yaml b/.asf.yaml new file mode 100644 index 000..1d14ccd --- /dev/null +++ b/.asf.yaml @@ -0,0 +1,10 @@ +notifications: + commits: commits@cassandra.apache.org + issues: commits@cassandra.apache.org + pullrequests: p...@cassandra.apache.org + +github: + enabled_merge_buttons: +squash: false +merge: false +rebase: true - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
