[jira] [Commented] (CASSANDRA-15583) 4.0 quality testing: Tooling, Bundled and First Party
[
https://issues.apache.org/jira/browse/CASSANDRA-15583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17210016#comment-17210016
]
Berenguer Blasi commented on CASSANDRA-15583:
-
So this is trickier than expected as some commands need a node to run properly.
So I managed to make {{ToolRunner}} work against jvm-dtests nodetool commands.
I also looked at the sstable utils you mentioned and it's going to need some
hacking as well. You'll need sstables to upgrade from that the CQLTester
instance is aware of etc..
For starters I got nodetool ring test up so far. I will update here as I make
progress:
*nodetool
** ring CASSANDRA-16200
** tablestats
** tpstats
** netstats
** disablebinary
** enablebinary
** gossipinfo
** sstableupgrade
** sstablescrub
> 4.0 quality testing: Tooling, Bundled and First Party
> -
>
> Key: CASSANDRA-15583
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15583
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/python, Test/unit
>Reporter: Josh McKenzie
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Reference [doc from
> NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
> for context.
> *Shepherd: Sam Tunnicliffe*
> Test plans should cover bundled first-party tooling and CLIs such as
> nodetool, cqlsh, and new tools supporting full query and audit logging
> (CASSANDRA-13983, CASSANDRA-12151).
> *Progress as of Aug 2020*
> {{ToolRunner}} has been added enabling us to test tools in java unit tests.
> This includes capturing their stdout/err and stdin i.e. Most tools have a
> starting unit test testing their cmd line args happy path. Tickets have been
> created to improve coverage of those and flagged LHF. Also for those tools
> big enough they can't be addressed in a simple ticket such as nodetool, a
> placeholder ticket for future improvements has been created as well. Tickets
> and status are:
> ||Tool||UX test||UT coverage||dtest coverage||Comments||
> |Nodetool|(x)|(x) CASSANDRA-16026|(!)|Not all the sub commands are tested.
> Dtest also test nodetool as a side effect|
> |Cqlsh|(x)|(x) CASSANDRA-16025|(!)| |
> |Cassandra-stress|(x)|(x) CASSANDRA-16024|(x)| |
> |debug-cql|(x)|(x) CASSANDRA-16023|(x)| |
> |fqltool|(x)|(/) CASSANDRA-16022|(!)| |
> |auditlogviewer|(/) CASSANDRA-15991|(!) CASSANDRA-16021|(!)| |
> |*Sstable utilities*| | | | |
> |sstabledump|(/) CASSANDRA-15991|(/) CASSANDRA-16020|(!)| |
> |sstableexpiredblockers|(/) CASSANDRA-15991|(x) CASSANDRA-16019|(!)| |
> |sstablelevelreset|(/) CASSANDRA-15991|(x) CASSANDRA-16018|(!)| |
> |sstableloader|(x)|(x) CASSANDRA-16017|(!)| |
> |sstablemetadata|(/) CASSANDRA-15991|(x) CASSANDRA-16016|(x)|Ran in dtests,
> no dedicated test|
> |sstableofflinerelevel|(/) CASSANDRA-15991|(x) CASSANDRA-16015|(!)| |
> |sstablerepairedset|(/) CASSANDRA-15991|(x) CASSANDRA-16014|(x)|Ran in
> dtests, no dedicated test|
> |sstablescrub|(/) CASSANDRA-15991|(x) CASSANDRA-16013|(!)| |
> |sstablesplit|(/) CASSANDRA-15991|(x) CASSANDRA-16012|(!)| |
> |sstableupgrade|(/) CASSANDRA-15991|(x) CASSANDRA-16011|(!)| |
> |sstableutil|(/) CASSANDRA-15991|(x) CASSANDRA-16010|(!)| |
> |sstableverify|(/) CASSANDRA-15991|(x) CASSANDRA-16009|(!)| |
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16200) Nodetool ring unit testing
[ https://issues.apache.org/jira/browse/CASSANDRA-16200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-16200: Fix Version/s: 4.0-beta > Nodetool ring unit testing > -- > > Key: CASSANDRA-16200 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16200 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > > Add nodetool ring testing -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16200) Nodetool ring unit testing
[ https://issues.apache.org/jira/browse/CASSANDRA-16200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-16200: Change Category: Quality Assurance Complexity: Normal Status: Open (was: Triage Needed) > Nodetool ring unit testing > -- > > Key: CASSANDRA-16200 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16200 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > > Add nodetool ring testing -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16200) Nodetool ring unit testing
Berenguer Blasi created CASSANDRA-16200: --- Summary: Nodetool ring unit testing Key: CASSANDRA-16200 URL: https://issues.apache.org/jira/browse/CASSANDRA-16200 Project: Cassandra Issue Type: Improvement Components: Test/dtest/java Reporter: Berenguer Blasi Assignee: Berenguer Blasi Add nodetool ring testing -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16063) Fix user experience when upgrading to 4.0 with compact tables
[
https://issues.apache.org/jira/browse/CASSANDRA-16063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209988#comment-17209988
]
Ekaterina Dimitrova commented on CASSANDRA-16063:
-
Branches rebased and wrapped for final review.
PRs as follow:
[trunk|https://github.com/ekaterinadimitrova2/cassandra/pull/54]|
[3.0|https://github.com/ekaterinadimitrova2/cassandra/pull/56] |
[dtetsts|https://github.com/ekaterinadimitrova2/cassandra-dtest/pull/4]
No PR for 3.11 as it is a merge from 3.0
CI:
[JAVA8
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/388/workflows/c8714c4c-7b68-4d3a-a8b9-e0708d69b13b]
| [JAVA
11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/388/workflows/885828f4-9f47-4f3e-91d0-65c110c0db7e]
_testHaveVersion3Nodes - org.apache.cassandra.gms.GossiperTest_ -
CASSANDRA-16148
_test_alter_rf_and_run_read_repair - read_repair_test.TestReadRepair -_
CASSANDRA-16148
_test_atomic_writes[blocking] - read_repair_test.TestReadRepairGuarantees -_
CASSANDRA-16148
test_compression_cql_options - compression_test.TestCompression - not related
but have to find/file a ticket; I will do it tomorrow
test_multiple_ranges_repair - repair_tests.repair_test.TestRepair - not related
but have to find/file a ticket; I will do it tomorrow
corruptionCausesFailure -
org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest -
CASSANDRA-15313
> Fix user experience when upgrading to 4.0 with compact tables
> -
>
> Key: CASSANDRA-16063
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16063
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Sylvain Lebresne
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: Compact_storage_upgrade_tests.txt
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The code to handle compact tables has been removed from 4.0, and the intended
> upgrade path to 4.0 for users having compact tables on 3.x is that they must
> execute {{ALTER ... DROP COMPACT STORAGE}} on all of their compact tables
> *before* attempting the upgrade.
> Obviously, some users won't read the upgrade instructions (or miss a table)
> and may try upgrading despite still having compact tables. If they do so, the
> intent is that the node will _not_ start, with a message clearly indicating
> the pre-upgrade step the user has missed. The user will then downgrade back
> the node(s) to 3.x, run the proper {{ALTER ... DROP COMPACT STORAGE}}, and
> then upgrade again.
> But while 4.0 does currently fail startup when finding any compact tables
> with a decent message, I believe the check is done too late during startup.
> Namely, that check is done as we read the tables schema, so within
> [{{Schema.instance.loadFromDisk()}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/CassandraDaemon.java#L241].
> But by then, we've _at least_ called
> {{SystemKeyspace.persistLocalMetadata()}}} and
> {{SystemKeyspaceMigrator40.migrate()}}, which will get into the commit log,
> and even possibly flush new {{na}} format sstables. As a results, a user
> might not be able to seemlessly restart the node on 3.x (to drop compact
> storage on the appropriate tables).
> Basically, we should make sure the check for compact tables done at 4.0
> startup is done as a {{StartupCheck}}, before the node does anything.
> We should also add a test for this (checking that if you try upgrading to 4.0
> with compact storage, you can downgrade back with no intervention whatsoever).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16199) cassandra.logdir undefined when CASSANDRA_LOG_DIR
[
https://issues.apache.org/jira/browse/CASSANDRA-16199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cyril Scetbon updated CASSANDRA-16199:
--
Description:
When ${cassandra.logdir} is used in logback.xml nodetool doesn’t use the env
variable CASSANDRA_LOG_DIR or the default value. and complains
{noformat}
03:07:27,387 |-ERROR in
ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] - Failed to create
parent directories for [/cassandra.logdir_IS_UNDEFINED/debug.log]03:07:27,387
|-ERROR in ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] - Failed
to create parent directories for
[/cassandra.logdir_IS_UNDEFINED/debug.log]03:07:27,388 |-ERROR in
ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] -
openFile(cassandra.logdir_IS_UNDEFINED/debug.log,true) call failed.
java.io.FileNotFoundException: cassandra.logdir_IS_UNDEFINED/debug.log (No such
file or directory) at java.io.FileNotFoundException:
cassandra.logdir_IS_UNDEFINED/debug.log (No such file or directory)
...{noformat}
It’s different for cassandra for instance
[https://github.com/apache/cassandra/blob/324267b3c0676ad31bd4f2fac0e2e673a9257a37/bin/cassandra#L186].
I feel like it should be added to
[https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/bin/nodetool],
or that it should call cassandra-env.sh
Seen on 3.11 and 4.0-beta1
was:
When ${cassandra.logdir} is used in logback.xml nodetool doesn’t use the env
variable CASSANDRA_LOG_DIR. and complains
{noformat}
03:07:27,387 |-ERROR in
ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] - Failed to create
parent directories for [/cassandra.logdir_IS_UNDEFINED/debug.log]03:07:27,387
|-ERROR in ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] - Failed
to create parent directories for
[/cassandra.logdir_IS_UNDEFINED/debug.log]03:07:27,388 |-ERROR in
ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] -
openFile(cassandra.logdir_IS_UNDEFINED/debug.log,true) call failed.
java.io.FileNotFoundException: cassandra.logdir_IS_UNDEFINED/debug.log (No such
file or directory) at java.io.FileNotFoundException:
cassandra.logdir_IS_UNDEFINED/debug.log (No such file or directory)
...{noformat}
It’s different for cassandra for instance
[https://github.com/apache/cassandra/blob/324267b3c0676ad31bd4f2fac0e2e673a9257a37/bin/cassandra#L186].
I feel like it should be added to
[https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/bin/nodetool],
or that it should call cassandra-env.sh
Seen on 3.11 and 4.0-beta1
> cassandra.logdir undefined when CASSANDRA_LOG_DIR
> -
>
> Key: CASSANDRA-16199
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16199
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Cyril Scetbon
>Priority: Normal
>
> When ${cassandra.logdir} is used in logback.xml nodetool doesn’t use the env
> variable CASSANDRA_LOG_DIR or the default value. and complains
> {noformat}
> 03:07:27,387 |-ERROR in
> ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] - Failed to create
> parent directories for [/cassandra.logdir_IS_UNDEFINED/debug.log]03:07:27,387
> |-ERROR in ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] - Failed
> to create parent directories for
> [/cassandra.logdir_IS_UNDEFINED/debug.log]03:07:27,388 |-ERROR in
> ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] -
> openFile(cassandra.logdir_IS_UNDEFINED/debug.log,true) call failed.
> java.io.FileNotFoundException: cassandra.logdir_IS_UNDEFINED/debug.log (No
> such file or directory) at java.io.FileNotFoundException:
> cassandra.logdir_IS_UNDEFINED/debug.log (No such file or directory)
> ...{noformat}
> It’s different for cassandra for instance
> [https://github.com/apache/cassandra/blob/324267b3c0676ad31bd4f2fac0e2e673a9257a37/bin/cassandra#L186].
> I feel like it should be added to
> [https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/bin/nodetool],
> or that it should call cassandra-env.sh
>
> Seen on 3.11 and 4.0-beta1
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16199) cassandra.logdir undefined when CASSANDRA_LOG_DIR
[
https://issues.apache.org/jira/browse/CASSANDRA-16199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cyril Scetbon updated CASSANDRA-16199:
--
Description:
When ${cassandra.logdir} is used in logback.xml nodetool doesn’t use the env
variable CASSANDRA_LOG_DIR. and complains
{noformat}
03:07:27,387 |-ERROR in
ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] - Failed to create
parent directories for [/cassandra.logdir_IS_UNDEFINED/debug.log]03:07:27,387
|-ERROR in ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] - Failed
to create parent directories for
[/cassandra.logdir_IS_UNDEFINED/debug.log]03:07:27,388 |-ERROR in
ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] -
openFile(cassandra.logdir_IS_UNDEFINED/debug.log,true) call failed.
java.io.FileNotFoundException: cassandra.logdir_IS_UNDEFINED/debug.log (No such
file or directory) at java.io.FileNotFoundException:
cassandra.logdir_IS_UNDEFINED/debug.log (No such file or directory)
...{noformat}
It’s different for cassandra for instance
[https://github.com/apache/cassandra/blob/324267b3c0676ad31bd4f2fac0e2e673a9257a37/bin/cassandra#L186].
I feel like it should be added to
[https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/bin/nodetool],
or that it should call cassandra-env.sh
Seen on 3.11 and 4.0-beta1
was:
When ${cassandra.logdir} is used in logback.xml nodetool doesn’t use the env
variable CASSANDRA_LOG_DIR. and complains
{noformat}
03:07:27,387 |-ERROR in
ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] - Failed to create
parent directories for [/cassandra.logdir_IS_UNDEFINED/debug.log]03:07:27,387
|-ERROR in ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] - Failed
to create parent directories for
[/cassandra.logdir_IS_UNDEFINED/debug.log]03:07:27,388 |-ERROR in
ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] -
openFile(cassandra.logdir_IS_UNDEFINED/debug.log,true) call failed.
java.io.FileNotFoundException: cassandra.logdir_IS_UNDEFINED/debug.log (No such
file or directory) at java.io.FileNotFoundException:
cassandra.logdir_IS_UNDEFINED/debug.log (No such file or directory)
...{noformat}
It’s different for cassandra for instance
[https://github.com/apache/cassandra/blob/324267b3c0676ad31bd4f2fac0e2e673a9257a37/bin/cassandra#L186].
I feel like it should be added to
[https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/bin/nodetool],
or that it should call cassandra-env.sh.
> cassandra.logdir undefined when CASSANDRA_LOG_DIR
> -
>
> Key: CASSANDRA-16199
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16199
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Cyril Scetbon
>Priority: Normal
>
> When ${cassandra.logdir} is used in logback.xml nodetool doesn’t use the env
> variable CASSANDRA_LOG_DIR. and complains
> {noformat}
> 03:07:27,387 |-ERROR in
> ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] - Failed to create
> parent directories for [/cassandra.logdir_IS_UNDEFINED/debug.log]03:07:27,387
> |-ERROR in ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] - Failed
> to create parent directories for
> [/cassandra.logdir_IS_UNDEFINED/debug.log]03:07:27,388 |-ERROR in
> ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] -
> openFile(cassandra.logdir_IS_UNDEFINED/debug.log,true) call failed.
> java.io.FileNotFoundException: cassandra.logdir_IS_UNDEFINED/debug.log (No
> such file or directory) at java.io.FileNotFoundException:
> cassandra.logdir_IS_UNDEFINED/debug.log (No such file or directory)
> ...{noformat}
> It’s different for cassandra for instance
> [https://github.com/apache/cassandra/blob/324267b3c0676ad31bd4f2fac0e2e673a9257a37/bin/cassandra#L186].
> I feel like it should be added to
> [https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/bin/nodetool],
> or that it should call cassandra-env.sh
>
> Seen on 3.11 and 4.0-beta1
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16199) cassandra.logdir undefined when CASSANDRA_LOG_DIR
Cyril Scetbon created CASSANDRA-16199:
-
Summary: cassandra.logdir undefined when CASSANDRA_LOG_DIR
Key: CASSANDRA-16199
URL: https://issues.apache.org/jira/browse/CASSANDRA-16199
Project: Cassandra
Issue Type: Bug
Components: Local/Config
Reporter: Cyril Scetbon
When ${cassandra.logdir} is used in logback.xml nodetool doesn’t use the env
variable CASSANDRA_LOG_DIR. and complains
{noformat}
03:07:27,387 |-ERROR in
ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] - Failed to create
parent directories for [/cassandra.logdir_IS_UNDEFINED/debug.log]03:07:27,387
|-ERROR in ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] - Failed
to create parent directories for
[/cassandra.logdir_IS_UNDEFINED/debug.log]03:07:27,388 |-ERROR in
ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] -
openFile(cassandra.logdir_IS_UNDEFINED/debug.log,true) call failed.
java.io.FileNotFoundException: cassandra.logdir_IS_UNDEFINED/debug.log (No such
file or directory) at java.io.FileNotFoundException:
cassandra.logdir_IS_UNDEFINED/debug.log (No such file or directory)
...{noformat}
It’s different for cassandra for instance
[https://github.com/apache/cassandra/blob/324267b3c0676ad31bd4f2fac0e2e673a9257a37/bin/cassandra#L186].
I feel like it should be added to
[https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/bin/nodetool],
or that it should call cassandra-env.sh.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16199) cassandra.logdir undefined when CASSANDRA_LOG_DIR
[
https://issues.apache.org/jira/browse/CASSANDRA-16199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cyril Scetbon updated CASSANDRA-16199:
--
Impacts: (was: None)
> cassandra.logdir undefined when CASSANDRA_LOG_DIR
> -
>
> Key: CASSANDRA-16199
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16199
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Cyril Scetbon
>Priority: Normal
>
> When ${cassandra.logdir} is used in logback.xml nodetool doesn’t use the env
> variable CASSANDRA_LOG_DIR. and complains
> {noformat}
> 03:07:27,387 |-ERROR in
> ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] - Failed to create
> parent directories for [/cassandra.logdir_IS_UNDEFINED/debug.log]03:07:27,387
> |-ERROR in ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] - Failed
> to create parent directories for
> [/cassandra.logdir_IS_UNDEFINED/debug.log]03:07:27,388 |-ERROR in
> ch.qos.logback.core.rolling.RollingFileAppender[DEBUGLOG] -
> openFile(cassandra.logdir_IS_UNDEFINED/debug.log,true) call failed.
> java.io.FileNotFoundException: cassandra.logdir_IS_UNDEFINED/debug.log (No
> such file or directory) at java.io.FileNotFoundException:
> cassandra.logdir_IS_UNDEFINED/debug.log (No such file or directory)
> ...{noformat}
> It’s different for cassandra for instance
> [https://github.com/apache/cassandra/blob/324267b3c0676ad31bd4f2fac0e2e673a9257a37/bin/cassandra#L186].
> I feel like it should be added to
> [https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/bin/nodetool],
> or that it should call cassandra-env.sh.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16120) Add ability for jvm-dtest to grep instance logs
[
https://issues.apache.org/jira/browse/CASSANDRA-16120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-16120:
--
Status: Resolved (was: Open)
> Add ability for jvm-dtest to grep instance logs
> ---
>
> Key: CASSANDRA-16120
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16120
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Fix For: NA
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> One of the main gaps between python dtest and jvm dtest is python dtest
> supports the ability to grep the logs of an instance; we need this capability
> as some tests require validating logs were triggered.
> Pydocs for common log methods
> {code}
> | grep_log(self, expr, filename='system.log', from_mark=None)
> | Returns a list of lines matching the regular expression in parameter
> | in the Cassandra log of this node
> |
> | grep_log_for_errors(self, filename='system.log')
> | Returns a list of errors with stack traces
> | in the Cassandra log of this node
> |
> | grep_log_for_errors_from(self, filename='system.log', seek_start=0)
> {code}
> {code}
> | watch_log_for(self, exprs, from_mark=None, timeout=600, process=None,
> verbose=False, filename='system.log')
> | Watch the log until one or more (regular) expression are found.
> | This methods when all the expressions have been found or the method
> | timeouts (a TimeoutError is then raised). On successful completion,
> | a list of pair (line matched, match object) is returned.
> {code}
> Below is a POC showing a way to do such logic
> {code}
> package org.apache.cassandra.distributed.test;
> import java.io.BufferedReader;
> import java.io.FileInputStream;
> import java.io.IOException;
> import java.io.InputStreamReader;
> import java.io.UncheckedIOException;
> import java.nio.charset.StandardCharsets;
> import java.util.Iterator;
> import java.util.Spliterator;
> import java.util.Spliterators;
> import java.util.regex.Matcher;
> import java.util.regex.Pattern;
> import java.util.stream.Stream;
> import java.util.stream.StreamSupport;
> import com.google.common.io.Closeables;
> import org.junit.Test;
> import org.apache.cassandra.distributed.Cluster;
> import org.apache.cassandra.utils.AbstractIterator;
> public class AllTheLogs extends TestBaseImpl
> {
>@Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = System.getProperty("cassandra.testtag",
> "cassandra.testtag_IS_UNDEFINED");
>String suite = System.getProperty("suitename",
> "suitename_IS_UNDEFINED");
>String log = String.format("build/test/logs/%s/TEST-%s.log", tag,
> suite);
>grep(log, "Enqueuing flush of tables").forEach(l ->
> System.out.println("I found the thing: " + l));
>}
>}
>private static Stream grep(String file, String regex) throws
> IOException
>{
>return grep(file, Pattern.compile(regex));
>}
>private static Stream grep(String file, Pattern regex) throws
> IOException
>{
>BufferedReader reader = new BufferedReader(new InputStreamReader(new
> FileInputStream(file), StandardCharsets.UTF_8));
>Iterator it = new AbstractIterator()
>{
>protected String computeNext()
>{
>try
>{
>String s;
>while ((s = reader.readLine()) != null)
>{
>Matcher m = regex.matcher(s);
>if (m.find())
>return s;
>}
>reader.close();
>return endOfData();
>}
>catch (IOException e)
>{
>Closeables.closeQuietly(reader);
>throw new UncheckedIOException(e);
>}
>}
>};
>return StreamSupport.stream(Spliterators.spliteratorUnknownSize(it,
> Spliterator.ORDERED), false);
>}
> }
> {code}
> And
> {code}
> @Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = System.getProperty("cassandra.testtag",
> "cassandra.testtag_IS_UNDEFINED");
>String suite = System.getProperty("suitename",
> "suitename_IS_UNDEFINED");
>//TODO missing way to get node id
> //cluster.get(1);
>String log =
>
[cassandra] branch cassandra-3.11 updated (6c8c791 -> 925ad35)
This is an automated email from the ASF dual-hosted git repository.
dcapwell pushed a change to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git.
from 6c8c791 Merge branch 'cassandra-3.0' into cassandra-3.11
new 0eb8cec Backport changes from CASSANDRA-16120 to other branches
new 5ef77d2 Merge branch 'cassandra-2.2' into cassandra-3.0
new 925ad35 Merge branch 'cassandra-3.0' into cassandra-3.11
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
test/conf/logback-dtest.xml| 27 +
.../distributed/impl/AbstractCluster.java | 22 +++-
...nstanceIDDefiner.java => ClusterIDDefiner.java} | 22 ++--
.../cassandra/distributed/impl/FileLogAction.java | 133 +
.../cassandra/distributed/impl/Instance.java | 24 +++-
.../distributed/impl/InstanceIDDefiner.java| 12 +-
.../cassandra/distributed/test/JVMDTestTest.java | 28 +
7 files changed, 229 insertions(+), 39 deletions(-)
copy
test/distributed/org/apache/cassandra/distributed/impl/{InstanceIDDefiner.java
=> ClusterIDDefiner.java} (72%)
create mode 100644
test/distributed/org/apache/cassandra/distributed/impl/FileLogAction.java
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository.
dcapwell pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit f6f46fe1a2f432ef2fc00af47855e16e149cab07
Merge: 16da961 925ad35
Author: David Capwell
AuthorDate: Wed Oct 7 17:06:30 2020 -0700
Merge branch 'cassandra-3.11' into trunk
.../org/apache/cassandra/distributed/impl/AbstractCluster.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --cc
test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
index bd3f338,6fae4f0..3fee754
---
a/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
+++
b/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
@@@ -186,11 -163,11 +186,11 @@@ public abstract class AbstractCluster<
private IInvokableInstance newInstance(int generation)
{
- InstanceClassLoader classLoader = new
InstanceClassLoader(generation, config.num(), version.classpath,
sharedClassLoader);
+ ClassLoader classLoader = new InstanceClassLoader(generation,
config.num(), version.classpath, sharedClassLoader);
if (instanceInitializer != null)
instanceInitializer.accept(classLoader, config.num());
-return
Instance.transferAdhoc((SerializableBiFunction)Instance::new, classLoader)
-.apply(config, classLoader);
+return
Instance.transferAdhoc((SerializableBiFunction)Instance::new, classLoader)
+
.apply(config.forVersion(version.major), classLoader);
}
public IInstanceConfig config()
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16120) Add ability for jvm-dtest to grep instance logs
[
https://issues.apache.org/jira/browse/CASSANDRA-16120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209886#comment-17209886
]
David Capwell edited comment on CASSANDRA-16120 at 10/8/20, 12:08 AM:
--
Committed:
https://github.com/apache/cassandra/commit/b155991818087fb9fdb26033df5fd059b299b0da
Ci results: normal
2.2
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16120-cassandra-2.2-FE41DC4E-74EA-4ED6-9A2C-1B1ADB6E40E9
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/79/
3.0
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16120-cassandra-3.0-FE41DC4E-74EA-4ED6-9A2C-1B1ADB6E40E9
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/80/
3.11
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16120-cassandra-3.11-FE41DC4E-74EA-4ED6-9A2C-1B1ADB6E40E9
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/81/
trunk
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16120-trunk-FE41DC4E-74EA-4ED6-9A2C-1B1ADB6E40E9
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/82/
was (Author: dcapwell):
Starting commit
Ci results (pending):
2.2
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16120-cassandra-2.2-FE41DC4E-74EA-4ED6-9A2C-1B1ADB6E40E9
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/79/
3.0
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16120-cassandra-3.0-FE41DC4E-74EA-4ED6-9A2C-1B1ADB6E40E9
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/80/
3.11
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16120-cassandra-3.11-FE41DC4E-74EA-4ED6-9A2C-1B1ADB6E40E9
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/81/
trunk
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16120-trunk-FE41DC4E-74EA-4ED6-9A2C-1B1ADB6E40E9
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/82/
> Add ability for jvm-dtest to grep instance logs
> ---
>
> Key: CASSANDRA-16120
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16120
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Fix For: NA
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> One of the main gaps between python dtest and jvm dtest is python dtest
> supports the ability to grep the logs of an instance; we need this capability
> as some tests require validating logs were triggered.
> Pydocs for common log methods
> {code}
> | grep_log(self, expr, filename='system.log', from_mark=None)
> | Returns a list of lines matching the regular expression in parameter
> | in the Cassandra log of this node
> |
> | grep_log_for_errors(self, filename='system.log')
> | Returns a list of errors with stack traces
> | in the Cassandra log of this node
> |
> | grep_log_for_errors_from(self, filename='system.log', seek_start=0)
> {code}
> {code}
> | watch_log_for(self, exprs, from_mark=None, timeout=600, process=None,
> verbose=False, filename='system.log')
> | Watch the log until one or more (regular) expression are found.
> | This methods when all the expressions have been found or the method
> | timeouts (a TimeoutError is then raised). On successful completion,
> | a list of pair (line matched, match object) is returned.
> {code}
> Below is a POC showing a way to do such logic
> {code}
> package org.apache.cassandra.distributed.test;
> import java.io.BufferedReader;
> import java.io.FileInputStream;
> import java.io.IOException;
> import java.io.InputStreamReader;
> import java.io.UncheckedIOException;
> import java.nio.charset.StandardCharsets;
> import java.util.Iterator;
> import java.util.Spliterator;
> import java.util.Spliterators;
> import java.util.regex.Matcher;
> import java.util.regex.Pattern;
> import java.util.stream.Stream;
> import java.util.stream.StreamSupport;
> import com.google.common.io.Closeables;
> import org.junit.Test;
> import org.apache.cassandra.distributed.Cluster;
> import org.apache.cassandra.utils.AbstractIterator;
> public class AllTheLogs extends TestBaseImpl
> {
>@Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
[cassandra] branch cassandra-2.2 updated: Backport changes from CASSANDRA-16120 to other branches
This is an automated email from the ASF dual-hosted git repository.
dcapwell pushed a commit to branch cassandra-2.2
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-2.2 by this push:
new 0eb8cec Backport changes from CASSANDRA-16120 to other branches
0eb8cec is described below
commit 0eb8cec5c75a7dc9503f59505bed67da65c49503
Author: David Capwell
AuthorDate: Wed Oct 7 14:46:03 2020 -0700
Backport changes from CASSANDRA-16120 to other branches
patch by David Capwell; reviewed by Alex Petrov, Jordan West, Yifan Cai for
CASSANDRA-16120
---
test/conf/logback-dtest.xml| 27 +
.../distributed/impl/AbstractCluster.java | 22 +++-
...nstanceIDDefiner.java => ClusterIDDefiner.java} | 22 ++--
.../cassandra/distributed/impl/FileLogAction.java | 133 +
.../cassandra/distributed/impl/Instance.java | 23 +++-
.../distributed/impl/InstanceIDDefiner.java| 12 +-
.../cassandra/distributed/test/JVMDTestTest.java | 28 +
7 files changed, 229 insertions(+), 38 deletions(-)
diff --git a/test/conf/logback-dtest.xml b/test/conf/logback-dtest.xml
index 4282fee..9b3216d 100644
--- a/test/conf/logback-dtest.xml
+++ b/test/conf/logback-dtest.xml
@@ -18,35 +18,18 @@
-->
+
-
-
-./build/test/logs/${cassandra.testtag}/TEST-${suitename}.log
-
-
./build/test/logs/${cassandra.testtag}/TEST-${suitename}.log.%i.gz
- 1
- 20
-
-
-
- 20MB
-
-
+
+
./build/test/logs/${cassandra.testtag}/${suitename}/${cluster_id}/${instance_id}/system.log
%-5level [%thread] ${instance_id} %date{ISO8601}
%msg%n
-false
-
-
-
-0
-0
-1024
-
+true
@@ -79,7 +62,7 @@
-
+
diff --git
a/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
b/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
index 9793add..f74078e 100644
---
a/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
+++
b/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
@@ -22,12 +22,12 @@ import java.io.File;
import java.net.InetSocketAddress;
import java.util.ArrayList;
import java.util.Arrays;
-import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
+import java.util.UUID;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
@@ -35,7 +35,6 @@ import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.BiConsumer;
import java.util.function.BiPredicate;
import java.util.function.Consumer;
-import java.util.function.Predicate;
import java.util.stream.Collectors;
import java.util.stream.Stream;
@@ -45,11 +44,11 @@ import org.slf4j.LoggerFactory;
import org.apache.cassandra.db.ColumnFamilyStore;
import org.apache.cassandra.db.Keyspace;
+import org.apache.cassandra.dht.IPartitioner;
+import org.apache.cassandra.dht.Token;
import org.apache.cassandra.distributed.api.ConsistencyLevel;
import org.apache.cassandra.distributed.api.Feature;
import org.apache.cassandra.distributed.api.ICluster;
-import org.apache.cassandra.dht.IPartitioner;
-import org.apache.cassandra.dht.Token;
import org.apache.cassandra.distributed.api.ICoordinator;
import org.apache.cassandra.distributed.api.IInstance;
import org.apache.cassandra.distributed.api.IInstanceConfig;
@@ -59,6 +58,7 @@ import org.apache.cassandra.distributed.api.IListen;
import org.apache.cassandra.distributed.api.IMessage;
import org.apache.cassandra.distributed.api.IMessageFilters;
import org.apache.cassandra.distributed.api.IUpgradeableInstance;
+import org.apache.cassandra.distributed.api.LogAction;
import org.apache.cassandra.distributed.api.NodeToolResult;
import org.apache.cassandra.distributed.api.TokenSupplier;
import org.apache.cassandra.distributed.shared.AbstractBuilder;
@@ -106,6 +106,7 @@ public abstract class AbstractCluster
implements ICluster
implements ICluster
implements IClusterhttp://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.distributed.impl;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.io.UncheckedIOException;
+import java.util.Objects;
+import java.util.function.Predicate;
+
+import com.google.common.collect.AbstractIterator;
+import
[cassandra] branch trunk updated (16da961 -> f6f46fe)
This is an automated email from the ASF dual-hosted git repository. dcapwell pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 16da961 Merge branch 'cassandra-3.11' into trunk new 0eb8cec Backport changes from CASSANDRA-16120 to other branches new 5ef77d2 Merge branch 'cassandra-2.2' into cassandra-3.0 new 925ad35 Merge branch 'cassandra-3.0' into cassandra-3.11 new f6f46fe Merge branch 'cassandra-3.11' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../org/apache/cassandra/distributed/impl/AbstractCluster.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository.
dcapwell pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 5ef77d234f26e985811db4ff5fc2f01f6fe5a29c
Merge: 8d2556f 0eb8cec
Author: David Capwell
AuthorDate: Wed Oct 7 17:03:07 2020 -0700
Merge branch 'cassandra-2.2' into cassandra-3.0
test/conf/logback-dtest.xml| 27 +
.../distributed/impl/AbstractCluster.java | 22 +++-
...nstanceIDDefiner.java => ClusterIDDefiner.java} | 22 ++--
.../cassandra/distributed/impl/FileLogAction.java | 133 +
.../cassandra/distributed/impl/Instance.java | 24 +++-
.../distributed/impl/InstanceIDDefiner.java| 12 +-
.../cassandra/distributed/test/JVMDTestTest.java | 28 +
7 files changed, 229 insertions(+), 39 deletions(-)
diff --cc
test/distributed/org/apache/cassandra/distributed/impl/FileLogAction.java
index 000,1d65fe1..0f48a23
mode 00,100644..100644
--- a/test/distributed/org/apache/cassandra/distributed/impl/FileLogAction.java
+++ b/test/distributed/org/apache/cassandra/distributed/impl/FileLogAction.java
@@@ -1,0 -1,133 +1,133 @@@
+ /*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+ package org.apache.cassandra.distributed.impl;
+
+ import java.io.File;
+ import java.io.FileNotFoundException;
+ import java.io.IOException;
+ import java.io.RandomAccessFile;
+ import java.io.UncheckedIOException;
+ import java.util.Objects;
+ import java.util.function.Predicate;
+
-import com.google.common.collect.AbstractIterator;
+ import com.google.common.io.Closeables;
+
++import org.apache.cassandra.utils.AbstractIterator;
+ import org.apache.cassandra.distributed.api.LogAction;
+ import org.apache.cassandra.distributed.api.LineIterator;
+
+ public class FileLogAction implements LogAction
+ {
+ private final File file;
+
+ public FileLogAction(File file)
+ {
+ this.file = Objects.requireNonNull(file);
+ }
+
+ @Override
+ public long mark()
+ {
+ return file.length();
+ }
+
+ @Override
+ public LineIterator match(long startPosition, Predicate fn)
+ {
+ RandomAccessFile reader;
+ try
+ {
+ reader = new RandomAccessFile(file, "r");
+ }
+ catch (FileNotFoundException e)
+ {
+ // if file isn't present, don't return an empty stream as it
looks the same as no log lines matched
+ throw new UncheckedIOException(e);
+ }
+ if (startPosition > 0) // -1 used to disable, so ignore any negative
values or 0 (default offset)
+ {
+ try
+ {
+ reader.seek(startPosition);
+ }
+ catch (IOException e)
+ {
+ throw new UncheckedIOException("Unable to seek to " +
startPosition, e);
+ }
+ }
+ return new FileLineIterator(reader, fn);
+ }
+
+ private static final class FileLineIterator extends
AbstractIterator implements LineIterator
+ {
+ private final RandomAccessFile reader;
+ private final Predicate fn;
+
+ private FileLineIterator(RandomAccessFile reader, Predicate
fn)
+ {
+ this.reader = reader;
+ this.fn = fn;
+ }
+
+ @Override
+ public long mark()
+ {
+ try
+ {
+ return reader.getFilePointer();
+ }
+ catch (IOException e)
+ {
+ throw new UncheckedIOException(e);
+ }
+ }
+
+ @Override
+ protected String computeNext()
+ {
+ try
+ {
+ String s;
+ while ((s = reader.readLine()) != null)
+ {
+ if (fn.test(s))
+ return s;
+ }
+ return endOfData();
+ }
+ catch (IOException e)
+ {
+ throw new UncheckedIOException(e);
+ }
+ }
+
+ @Override
+
[cassandra] branch cassandra-3.0 updated (8d2556f -> 5ef77d2)
This is an automated email from the ASF dual-hosted git repository.
dcapwell pushed a change to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git.
from 8d2556f Merge branch 'cassandra-2.2' into cassandra-3.0
new 0eb8cec Backport changes from CASSANDRA-16120 to other branches
new 5ef77d2 Merge branch 'cassandra-2.2' into cassandra-3.0
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
test/conf/logback-dtest.xml| 27 +
.../distributed/impl/AbstractCluster.java | 22 +++-
...nstanceIDDefiner.java => ClusterIDDefiner.java} | 22 ++--
.../cassandra/distributed/impl/FileLogAction.java | 133 +
.../cassandra/distributed/impl/Instance.java | 24 +++-
.../distributed/impl/InstanceIDDefiner.java| 12 +-
.../cassandra/distributed/test/JVMDTestTest.java | 28 +
7 files changed, 229 insertions(+), 39 deletions(-)
copy
test/distributed/org/apache/cassandra/distributed/impl/{InstanceIDDefiner.java
=> ClusterIDDefiner.java} (72%)
create mode 100644
test/distributed/org/apache/cassandra/distributed/impl/FileLogAction.java
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository. dcapwell pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 925ad35a7d80a92346ed76f941ae3873433a9ad1 Merge: 6c8c791 5ef77d2 Author: David Capwell AuthorDate: Wed Oct 7 17:04:51 2020 -0700 Merge branch 'cassandra-3.0' into cassandra-3.11 test/conf/logback-dtest.xml| 27 + .../distributed/impl/AbstractCluster.java | 22 +++- ...nstanceIDDefiner.java => ClusterIDDefiner.java} | 22 ++-- .../cassandra/distributed/impl/FileLogAction.java | 133 + .../cassandra/distributed/impl/Instance.java | 24 +++- .../distributed/impl/InstanceIDDefiner.java| 12 +- .../cassandra/distributed/test/JVMDTestTest.java | 28 + 7 files changed, 229 insertions(+), 39 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16127) NullPointerException when calling nodetool enablethrift
[
https://issues.apache.org/jira/browse/CASSANDRA-16127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209918#comment-17209918
]
David Capwell commented on CASSANDRA-16127:
---
[~e.dimitrova] I plan to remove the python dtest in favor of the jvm-dtest,
will do that after I get a clean run (waiting on
https://issues.apache.org/jira/browse/CASSANDRA-16120).
> NullPointerException when calling nodetool enablethrift
> ---
>
> Key: CASSANDRA-16127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16127
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Thrift
>Reporter: Tibor Repasi
>Assignee: David Capwell
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x
>
>
> Having thrift disabled, it's impossible to enable it again without restarting
> the node:
> {code}
> $ nodetool statusthrift
> not running
> $ nodetool enablethrift
> error: null
> -- StackTrace --
> java.lang.NullPointerException
> at
> org.apache.cassandra.service.StorageService.startRPCServer(StorageService.java:392)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
> at
> javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
> at
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
> at
> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
> at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
> at sun.rmi.transport.Transport$1.run(Transport.java:200)
> at sun.rmi.transport.Transport$1.run(Transport.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
> at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16057) Should update in-jvm dtest to expose stdout and stderr for nodetool
[ https://issues.apache.org/jira/browse/CASSANDRA-16057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRA-16057: -- Status: Open (was: Resolved) Reopen for the backports > Should update in-jvm dtest to expose stdout and stderr for nodetool > --- > > Key: CASSANDRA-16057 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16057 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: David Capwell >Assignee: Yifan Cai >Priority: Normal > Fix For: NA > > Time Spent: 2h 20m > Remaining Estimate: 0h > > Many nodetool commands output to stdout or stderr so running nodetool using > in-jvm dtest should expose that to tests. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16057) Should update in-jvm dtest to expose stdout and stderr for nodetool
[ https://issues.apache.org/jira/browse/CASSANDRA-16057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRA-16057: -- Status: Patch Available (was: Open) > Should update in-jvm dtest to expose stdout and stderr for nodetool > --- > > Key: CASSANDRA-16057 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16057 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: David Capwell >Assignee: Yifan Cai >Priority: Normal > Fix For: NA > > Time Spent: 2h 20m > Remaining Estimate: 0h > > Many nodetool commands output to stdout or stderr so running nodetool using > in-jvm dtest should expose that to tests. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16057) Should update in-jvm dtest to expose stdout and stderr for nodetool
[ https://issues.apache.org/jira/browse/CASSANDRA-16057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209912#comment-17209912 ] Yifan Cai edited comment on CASSANDRA-16057 at 10/7/20, 10:49 PM: -- Backports: 2.2: [https://github.com/yifan-c/cassandra/tree/b/CASSANDRA-16057-2.2] CI: [https://app.circleci.com/pipelines/github/yifan-c/cassandra/121/workflows/6ed104ea-788e-4c3e-89e5-fd516bae7b51] 3.0: [https://github.com/yifan-c/cassandra/tree/b/CASSANDRA-16057-3.0] CI: [https://app.circleci.com/pipelines/github/yifan-c/cassandra/120/workflows/fa570014-6078-47df-a401-794f8d163ac5] 3.11: [https://github.com/yifan-c/cassandra/tree/b/CASSANDRA-16057-3.11] CI: [https://app.circleci.com/pipelines/github/yifan-c/cassandra/119/workflows/945130ca-274e-4311-8b30-fd72a405d9f9] cc: [~dcapwell] was (Author: yifanc): Backports: 2.0: [https://github.com/yifan-c/cassandra/tree/b/CASSANDRA-16057-2.2] CI: [https://app.circleci.com/pipelines/github/yifan-c/cassandra/121/workflows/6ed104ea-788e-4c3e-89e5-fd516bae7b51] 3.0: [https://github.com/yifan-c/cassandra/tree/b/CASSANDRA-16057-3.0] CI: [https://app.circleci.com/pipelines/github/yifan-c/cassandra/120/workflows/fa570014-6078-47df-a401-794f8d163ac5] 3.11: [https://github.com/yifan-c/cassandra/tree/b/CASSANDRA-16057-3.11] CI: [https://app.circleci.com/pipelines/github/yifan-c/cassandra/119/workflows/945130ca-274e-4311-8b30-fd72a405d9f9] cc: [~dcapwell] > Should update in-jvm dtest to expose stdout and stderr for nodetool > --- > > Key: CASSANDRA-16057 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16057 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: David Capwell >Assignee: Yifan Cai >Priority: Normal > Fix For: NA > > Time Spent: 2h 20m > Remaining Estimate: 0h > > Many nodetool commands output to stdout or stderr so running nodetool using > in-jvm dtest should expose that to tests. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16057) Should update in-jvm dtest to expose stdout and stderr for nodetool
[ https://issues.apache.org/jira/browse/CASSANDRA-16057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209912#comment-17209912 ] Yifan Cai commented on CASSANDRA-16057: --- Backports: 2.0: [https://github.com/yifan-c/cassandra/tree/b/CASSANDRA-16057-2.2] CI: [https://app.circleci.com/pipelines/github/yifan-c/cassandra/121/workflows/6ed104ea-788e-4c3e-89e5-fd516bae7b51] 3.0: [https://github.com/yifan-c/cassandra/tree/b/CASSANDRA-16057-3.0] CI: [https://app.circleci.com/pipelines/github/yifan-c/cassandra/120/workflows/fa570014-6078-47df-a401-794f8d163ac5] 3.11: [https://github.com/yifan-c/cassandra/tree/b/CASSANDRA-16057-3.11] CI: [https://app.circleci.com/pipelines/github/yifan-c/cassandra/119/workflows/945130ca-274e-4311-8b30-fd72a405d9f9] cc: [~dcapwell] > Should update in-jvm dtest to expose stdout and stderr for nodetool > --- > > Key: CASSANDRA-16057 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16057 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: David Capwell >Assignee: Yifan Cai >Priority: Normal > Fix For: NA > > Time Spent: 2h 20m > Remaining Estimate: 0h > > Many nodetool commands output to stdout or stderr so running nodetool using > in-jvm dtest should expose that to tests. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16120) Add ability for jvm-dtest to grep instance logs
[
https://issues.apache.org/jira/browse/CASSANDRA-16120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209886#comment-17209886
]
David Capwell commented on CASSANDRA-16120:
---
Starting commit
Ci results (pending):
2.2
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16120-cassandra-2.2-FE41DC4E-74EA-4ED6-9A2C-1B1ADB6E40E9
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/79/
3.0
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16120-cassandra-3.0-FE41DC4E-74EA-4ED6-9A2C-1B1ADB6E40E9
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/80/
3.11
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16120-cassandra-3.11-FE41DC4E-74EA-4ED6-9A2C-1B1ADB6E40E9
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/81/
trunk
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16120-trunk-FE41DC4E-74EA-4ED6-9A2C-1B1ADB6E40E9
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/82/
> Add ability for jvm-dtest to grep instance logs
> ---
>
> Key: CASSANDRA-16120
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16120
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Fix For: NA
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> One of the main gaps between python dtest and jvm dtest is python dtest
> supports the ability to grep the logs of an instance; we need this capability
> as some tests require validating logs were triggered.
> Pydocs for common log methods
> {code}
> | grep_log(self, expr, filename='system.log', from_mark=None)
> | Returns a list of lines matching the regular expression in parameter
> | in the Cassandra log of this node
> |
> | grep_log_for_errors(self, filename='system.log')
> | Returns a list of errors with stack traces
> | in the Cassandra log of this node
> |
> | grep_log_for_errors_from(self, filename='system.log', seek_start=0)
> {code}
> {code}
> | watch_log_for(self, exprs, from_mark=None, timeout=600, process=None,
> verbose=False, filename='system.log')
> | Watch the log until one or more (regular) expression are found.
> | This methods when all the expressions have been found or the method
> | timeouts (a TimeoutError is then raised). On successful completion,
> | a list of pair (line matched, match object) is returned.
> {code}
> Below is a POC showing a way to do such logic
> {code}
> package org.apache.cassandra.distributed.test;
> import java.io.BufferedReader;
> import java.io.FileInputStream;
> import java.io.IOException;
> import java.io.InputStreamReader;
> import java.io.UncheckedIOException;
> import java.nio.charset.StandardCharsets;
> import java.util.Iterator;
> import java.util.Spliterator;
> import java.util.Spliterators;
> import java.util.regex.Matcher;
> import java.util.regex.Pattern;
> import java.util.stream.Stream;
> import java.util.stream.StreamSupport;
> import com.google.common.io.Closeables;
> import org.junit.Test;
> import org.apache.cassandra.distributed.Cluster;
> import org.apache.cassandra.utils.AbstractIterator;
> public class AllTheLogs extends TestBaseImpl
> {
>@Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = System.getProperty("cassandra.testtag",
> "cassandra.testtag_IS_UNDEFINED");
>String suite = System.getProperty("suitename",
> "suitename_IS_UNDEFINED");
>String log = String.format("build/test/logs/%s/TEST-%s.log", tag,
> suite);
>grep(log, "Enqueuing flush of tables").forEach(l ->
> System.out.println("I found the thing: " + l));
>}
>}
>private static Stream grep(String file, String regex) throws
> IOException
>{
>return grep(file, Pattern.compile(regex));
>}
>private static Stream grep(String file, Pattern regex) throws
> IOException
>{
>BufferedReader reader = new BufferedReader(new InputStreamReader(new
> FileInputStream(file), StandardCharsets.UTF_8));
>Iterator it = new AbstractIterator()
>{
>protected String computeNext()
>{
>try
>{
>String s;
>while ((s = reader.readLine()) != null)
>{
>Matcher m = regex.matcher(s);
>if (m.find())
>
[jira] [Updated] (CASSANDRA-16120) Add ability for jvm-dtest to grep instance logs
[
https://issues.apache.org/jira/browse/CASSANDRA-16120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-16120:
--
Reviewers: Alex Petrov, Jordan West, Yifan Cai (was: Alex Petrov, Yifan
Cai)
> Add ability for jvm-dtest to grep instance logs
> ---
>
> Key: CASSANDRA-16120
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16120
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Fix For: NA
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> One of the main gaps between python dtest and jvm dtest is python dtest
> supports the ability to grep the logs of an instance; we need this capability
> as some tests require validating logs were triggered.
> Pydocs for common log methods
> {code}
> | grep_log(self, expr, filename='system.log', from_mark=None)
> | Returns a list of lines matching the regular expression in parameter
> | in the Cassandra log of this node
> |
> | grep_log_for_errors(self, filename='system.log')
> | Returns a list of errors with stack traces
> | in the Cassandra log of this node
> |
> | grep_log_for_errors_from(self, filename='system.log', seek_start=0)
> {code}
> {code}
> | watch_log_for(self, exprs, from_mark=None, timeout=600, process=None,
> verbose=False, filename='system.log')
> | Watch the log until one or more (regular) expression are found.
> | This methods when all the expressions have been found or the method
> | timeouts (a TimeoutError is then raised). On successful completion,
> | a list of pair (line matched, match object) is returned.
> {code}
> Below is a POC showing a way to do such logic
> {code}
> package org.apache.cassandra.distributed.test;
> import java.io.BufferedReader;
> import java.io.FileInputStream;
> import java.io.IOException;
> import java.io.InputStreamReader;
> import java.io.UncheckedIOException;
> import java.nio.charset.StandardCharsets;
> import java.util.Iterator;
> import java.util.Spliterator;
> import java.util.Spliterators;
> import java.util.regex.Matcher;
> import java.util.regex.Pattern;
> import java.util.stream.Stream;
> import java.util.stream.StreamSupport;
> import com.google.common.io.Closeables;
> import org.junit.Test;
> import org.apache.cassandra.distributed.Cluster;
> import org.apache.cassandra.utils.AbstractIterator;
> public class AllTheLogs extends TestBaseImpl
> {
>@Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = System.getProperty("cassandra.testtag",
> "cassandra.testtag_IS_UNDEFINED");
>String suite = System.getProperty("suitename",
> "suitename_IS_UNDEFINED");
>String log = String.format("build/test/logs/%s/TEST-%s.log", tag,
> suite);
>grep(log, "Enqueuing flush of tables").forEach(l ->
> System.out.println("I found the thing: " + l));
>}
>}
>private static Stream grep(String file, String regex) throws
> IOException
>{
>return grep(file, Pattern.compile(regex));
>}
>private static Stream grep(String file, Pattern regex) throws
> IOException
>{
>BufferedReader reader = new BufferedReader(new InputStreamReader(new
> FileInputStream(file), StandardCharsets.UTF_8));
>Iterator it = new AbstractIterator()
>{
>protected String computeNext()
>{
>try
>{
>String s;
>while ((s = reader.readLine()) != null)
>{
>Matcher m = regex.matcher(s);
>if (m.find())
>return s;
>}
>reader.close();
>return endOfData();
>}
>catch (IOException e)
>{
>Closeables.closeQuietly(reader);
>throw new UncheckedIOException(e);
>}
>}
>};
>return StreamSupport.stream(Spliterators.spliteratorUnknownSize(it,
> Spliterator.ORDERED), false);
>}
> }
> {code}
> And
> {code}
> @Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = System.getProperty("cassandra.testtag",
> "cassandra.testtag_IS_UNDEFINED");
>String suite = System.getProperty("suitename",
> "suitename_IS_UNDEFINED");
>//TODO missing way to get node id
> //cluster.get(1);
>String log =
>
[jira] [Updated] (CASSANDRA-16061) transient_replication_ring_test.py::TestTransientReplicationRing::test_move_forwards_and_cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-16061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-16061: -- Status: Open (was: Patch Available) > transient_replication_ring_test.py::TestTransientReplicationRing::test_move_forwards_and_cleanup > > > Key: CASSANDRA-16061 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16061 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-beta > > > Failing here, also locally: > [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/312/workflows/da4ce69c-e778-467e-b9f3-27ab166a8321/jobs/1945] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16061) transient_replication_ring_test.py::TestTransientReplicationRing::test_move_forwards_and_cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-16061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-16061: -- Test and Documentation Plan: asdf Status: Patch Available (was: In Progress) > transient_replication_ring_test.py::TestTransientReplicationRing::test_move_forwards_and_cleanup > > > Key: CASSANDRA-16061 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16061 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-beta > > > Failing here, also locally: > [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/312/workflows/da4ce69c-e778-467e-b9f3-27ab166a8321/jobs/1945] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16198) Add ant task to check license and fix all files missing a licence
[ https://issues.apache.org/jira/browse/CASSANDRA-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209856#comment-17209856 ] David Capwell edited comment on CASSANDRA-16198 at 10/7/20, 9:37 PM: - CI results: Yellow, expected stuff. I stopped the build once I made sure upgrades were working, so all paths which triggered builds was touched. 2.2: Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16198-cassandra-2.2-E8B175C8-38F7-4128-8C54-A3A59B304844 Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/75/ 3.0: Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16198-cassandra-3.0-E8B175C8-38F7-4128-8C54-A3A59B304844 Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/76/ 3.11: Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16198-cassandra-3.11-E8B175C8-38F7-4128-8C54-A3A59B304844 Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/77/ trunk: Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16198-trunk-E8B175C8-38F7-4128-8C54-A3A59B304844 Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/78/ was (Author: dcapwell): Starting commit CI results (pending): 2.2: Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16198-cassandra-2.2-E8B175C8-38F7-4128-8C54-A3A59B304844 Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/75/ 3.0: Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16198-cassandra-3.0-E8B175C8-38F7-4128-8C54-A3A59B304844 Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/76/ 3.11: Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16198-cassandra-3.11-E8B175C8-38F7-4128-8C54-A3A59B304844 Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/77/ trunk: Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16198-trunk-E8B175C8-38F7-4128-8C54-A3A59B304844 Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/78/ > Add ant task to check license and fix all files missing a licence > - > > Key: CASSANDRA-16198 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16198 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: NA > > > We should block the build on license to avoid missing licenses in the build. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (b1efb8e -> 6c8c791)
This is an automated email from the ASF dual-hosted git repository. dcapwell pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from b1efb8e Merge branch 'cassandra-3.0' into cassandra-3.11 new 844d19d Add ant task to check license and fix all files missing a licence new 8d2556f Merge branch 'cassandra-2.2' into cassandra-3.0 new 6c8c791 Merge branch 'cassandra-3.0' into cassandra-3.11 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: build.xml | 22 +- .../apache/cassandra/db/commitlog/IntervalSet.java | 18 ++ .../org/apache/cassandra/hints/InputPosition.java | 18 ++ .../org/apache/cassandra/metrics/AuthMetrics.java | 18 ++ .../apache/cassandra/utils/IntegerInterval.java| 18 ++ .../utils/logging/LogbackLoggingSupport.java | 18 ++ .../cassandra/utils/logging/LoggingSupport.java| 18 ++ .../utils/logging/LoggingSupportFactory.java | 18 ++ .../utils/logging/NoOpFallbackLoggingSupport.java | 18 ++ .../distributed/test/SimpleReadWriteTest.java | 18 ++ .../distributed/test/ThriftClientTest.java | 18 ++ .../distributed/test/ThriftClientUtils.java| 18 ++ .../distributed/upgrade/MigrateDropColumns.java| 18 ++ .../upgrade/MigrateDropColumns22To30To311Test.java | 18 ++ .../upgrade/MigrateDropColumns22To311Test.java | 18 ++ .../upgrade/MigrateDropColumns30To311Test.java | 18 ++ .../org/apache/cassandra/cql3/ViewComplexTest.java | 18 ++ .../cql3/validation/operations/TTLTest.java| 18 ++ .../cassandra/db/commitlog/CommitLogCQLTest.java | 18 ++ .../apache/cassandra/db/marshal/EmptyTypeTest.java | 18 ++ .../apache/cassandra/io/DiskSpaceMetricsTest.java | 18 ++ .../cassandra/utils/IntegerIntervalsTest.java | 18 ++ 22 files changed, 399 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.2 updated: Add ant task to check license and fix all files missing a licence
This is an automated email from the ASF dual-hosted git repository. dcapwell pushed a commit to branch cassandra-2.2 in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/cassandra-2.2 by this push: new 844d19d Add ant task to check license and fix all files missing a licence 844d19d is described below commit 844d19d4e6d2cb52e5d5de9aad22fcc4a4da3e8d Author: David Capwell AuthorDate: Wed Oct 7 13:45:26 2020 -0700 Add ant task to check license and fix all files missing a licence patch by David Capwell; reviewed by Jordan West for CASSANDRA-16198 --- build.xml | 22 +- .../distributed/test/SimpleReadWritePathTest.java | 18 ++ .../distributed/test/ThriftClientTest.java | 18 ++ .../distributed/test/ThriftClientUtils.java| 18 ++ .../cql3/validation/operations/TTLTest.java| 18 ++ 5 files changed, 93 insertions(+), 1 deletion(-) diff --git a/build.xml b/build.xml index d003edf..657cbbb 100644 --- a/build.xml +++ b/build.xml @@ -833,7 +833,7 @@ depends="maven-ant-tasks-retrieve-build,build-project" description="Compile Cassandra classes"/> - @@ -1214,6 +1214,24 @@ + + + + + + + + + + + + + + + + + + + diff --git a/test/distributed/org/apache/cassandra/distributed/test/SimpleReadWritePathTest.java b/test/distributed/org/apache/cassandra/distributed/test/SimpleReadWritePathTest.java index 8c9e8af..9efab66 100644 --- a/test/distributed/org/apache/cassandra/distributed/test/SimpleReadWritePathTest.java +++ b/test/distributed/org/apache/cassandra/distributed/test/SimpleReadWritePathTest.java @@ -1,3 +1,21 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + package org.apache.cassandra.distributed.test; import java.io.IOException; diff --git a/test/distributed/org/apache/cassandra/distributed/test/ThriftClientTest.java b/test/distributed/org/apache/cassandra/distributed/test/ThriftClientTest.java index c5d5b9b..0548e47 100644 --- a/test/distributed/org/apache/cassandra/distributed/test/ThriftClientTest.java +++ b/test/distributed/org/apache/cassandra/distributed/test/ThriftClientTest.java @@ -1,3 +1,21 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + package org.apache.cassandra.distributed.test; import java.io.IOException; diff --git a/test/distributed/org/apache/cassandra/distributed/test/ThriftClientUtils.java b/test/distributed/org/apache/cassandra/distributed/test/ThriftClientUtils.java index 51b153c..7863525 100644 --- a/test/distributed/org/apache/cassandra/distributed/test/ThriftClientUtils.java +++ b/test/distributed/org/apache/cassandra/distributed/test/ThriftClientUtils.java @@ -1,3 +1,21 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
dcapwell pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 6c8c791f19be13f4d1f02fe9ffc0d2d532e6c932
Merge: b1efb8e 8d2556f
Author: David Capwell
AuthorDate: Wed Oct 7 14:33:08 2020 -0700
Merge branch 'cassandra-3.0' into cassandra-3.11
build.xml | 22 +-
.../apache/cassandra/db/commitlog/IntervalSet.java | 18 ++
.../org/apache/cassandra/hints/InputPosition.java | 18 ++
.../org/apache/cassandra/metrics/AuthMetrics.java | 18 ++
.../apache/cassandra/utils/IntegerInterval.java| 18 ++
.../utils/logging/LogbackLoggingSupport.java | 18 ++
.../cassandra/utils/logging/LoggingSupport.java| 18 ++
.../utils/logging/LoggingSupportFactory.java | 18 ++
.../utils/logging/NoOpFallbackLoggingSupport.java | 18 ++
.../distributed/test/SimpleReadWriteTest.java | 18 ++
.../distributed/test/ThriftClientTest.java | 18 ++
.../distributed/test/ThriftClientUtils.java| 18 ++
.../distributed/upgrade/MigrateDropColumns.java| 18 ++
.../upgrade/MigrateDropColumns22To30To311Test.java | 18 ++
.../upgrade/MigrateDropColumns22To311Test.java | 18 ++
.../upgrade/MigrateDropColumns30To311Test.java | 18 ++
.../org/apache/cassandra/cql3/ViewComplexTest.java | 18 ++
.../cql3/validation/operations/TTLTest.java| 18 ++
.../cassandra/db/commitlog/CommitLogCQLTest.java | 18 ++
.../apache/cassandra/db/marshal/EmptyTypeTest.java | 18 ++
.../apache/cassandra/io/DiskSpaceMetricsTest.java | 18 ++
.../cassandra/utils/IntegerIntervalsTest.java | 18 ++
22 files changed, 399 insertions(+), 1 deletion(-)
diff --cc build.xml
index 82c35e9,e3270d8..f078d34
--- a/build.xml
+++ b/build.xml
@@@ -843,7 -787,7 +843,7 @@@
depends="maven-ant-tasks-retrieve-build,build-project"
description="Compile Cassandra classes"/>
-
@@@ -1199,28 -1165,26 +1199,46 @@@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
++
package org.apache.cassandra.hints;
/**
diff --cc src/java/org/apache/cassandra/metrics/AuthMetrics.java
index 57d08ef,000..8f0f97e
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/metrics/AuthMetrics.java
+++ b/src/java/org/apache/cassandra/metrics/AuthMetrics.java
@@@ -1,40 -1,0 +1,58 @@@
++/*
++ * Licensed to the Apache Software Foundation (ASF) under one
++ * or more contributor license agreements. See the NOTICE file
++ * distributed with this work for additional information
++ * regarding copyright ownership. The ASF licenses this file
++ * to you under the Apache License, Version 2.0 (the
++ * "License"); you may not use this file except in compliance
++ * with the License. You may obtain a copy of the License at
++ *
++ * http://www.apache.org/licenses/LICENSE-2.0
++ *
++ * Unless required by applicable law or agreed to in writing, software
++ * distributed under the License is distributed on an "AS IS" BASIS,
++ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
++ * See the License for the specific language governing permissions and
++ * limitations under the License.
++ */
++
+package org.apache.cassandra.metrics;
+
+import com.codahale.metrics.Meter;
+
+/**
+ * Metrics about authentication
+ */
+public class AuthMetrics
+{
+
+public static final AuthMetrics instance = new AuthMetrics();
+
+public static void init()
+{
+// no-op, just used to force instance creation
+}
+
+/** Number and rate of successful logins */
+protected final Meter success;
+
+/** Number and rate of login failures */
+protected final Meter failure;
+
+private AuthMetrics()
+{
+
+success =
[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository.
dcapwell pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 8d2556fd59995737418e550c0ac5b04e45c29225
Merge: 5a2b5a4 844d19d
Author: David Capwell
AuthorDate: Wed Oct 7 14:32:14 2020 -0700
Merge branch 'cassandra-2.2' into cassandra-3.0
build.xml | 22 +-
.../apache/cassandra/db/commitlog/IntervalSet.java | 18 ++
.../apache/cassandra/utils/IntegerInterval.java| 18 ++
.../distributed/test/SimpleReadWriteTest.java | 18 ++
.../distributed/test/ThriftClientTest.java | 18 ++
.../distributed/test/ThriftClientUtils.java| 18 ++
.../upgrade/MigrateDropColumnsTest.java| 18 ++
.../org/apache/cassandra/cql3/ViewComplexTest.java | 18 ++
.../cql3/validation/operations/TTLTest.java| 18 ++
.../cassandra/db/commitlog/BatchCommitLogTest.java | 18 ++
.../cassandra/db/commitlog/CommitLogCQLTest.java | 18 ++
.../apache/cassandra/db/marshal/EmptyTypeTest.java | 18 ++
.../apache/cassandra/io/DiskSpaceMetricsTest.java | 18 ++
.../cassandra/utils/IntegerIntervalsTest.java | 18 ++
14 files changed, 255 insertions(+), 1 deletion(-)
diff --cc src/java/org/apache/cassandra/db/commitlog/IntervalSet.java
index bd0ea22,000..a25dbfd
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/db/commitlog/IntervalSet.java
+++ b/src/java/org/apache/cassandra/db/commitlog/IntervalSet.java
@@@ -1,192 -1,0 +1,210 @@@
++/*
++ * Licensed to the Apache Software Foundation (ASF) under one
++ * or more contributor license agreements. See the NOTICE file
++ * distributed with this work for additional information
++ * regarding copyright ownership. The ASF licenses this file
++ * to you under the Apache License, Version 2.0 (the
++ * "License"); you may not use this file except in compliance
++ * with the License. You may obtain a copy of the License at
++ *
++ * http://www.apache.org/licenses/LICENSE-2.0
++ *
++ * Unless required by applicable law or agreed to in writing, software
++ * distributed under the License is distributed on an "AS IS" BASIS,
++ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
++ * See the License for the specific language governing permissions and
++ * limitations under the License.
++ */
++
+package org.apache.cassandra.db.commitlog;
+
+import java.io.IOException;
+import java.util.*;
+
+import com.google.common.collect.ImmutableSortedMap;
+
+import org.apache.cassandra.db.TypeSizes;
+import org.apache.cassandra.io.ISerializer;
+import org.apache.cassandra.io.util.DataInputPlus;
+import org.apache.cassandra.io.util.DataOutputPlus;
+
+/**
+ * An immutable set of closed intervals, stored in normalized form (i.e.
where overlapping intervals are converted
+ * to a single interval covering both).
+ *
+ * The set is stored as a sorted map from interval starts to the
corresponding end. The map satisfies
+ * curr().getKey() <= curr().getValue() < next().getKey()
+ */
+public class IntervalSet>
+{
+@SuppressWarnings({ "rawtypes", "unchecked" })
+private static final IntervalSet EMPTY = new
IntervalSet(ImmutableSortedMap.of());
+
+final private NavigableMap ranges;
+
+private IntervalSet(ImmutableSortedMap ranges)
+{
+this.ranges = ranges;
+}
+
+/**
+ * Construct new set containing the interval with the given start and end
position.
+ */
+public IntervalSet(T start, T end)
+{
+this(ImmutableSortedMap.of(start, end));
+}
+
+@SuppressWarnings("unchecked")
+public static > IntervalSet empty()
+{
+return (IntervalSet) EMPTY;
+}
+
+public boolean contains(T position)
+{
+// closed (i.e. inclusive) intervals
+Map.Entry range = ranges.floorEntry(position);
+return range != null && position.compareTo(range.getValue()) <= 0;
+}
+
+public boolean isEmpty()
+{
+return ranges.isEmpty();
+}
+
+public Optional lowerBound()
+{
+return isEmpty() ? Optional.empty() : Optional.of(ranges.firstKey());
+}
+
+public Optional upperBound()
+{
+return isEmpty() ? Optional.empty() :
Optional.of(ranges.lastEntry().getValue());
+}
+
+public Collection starts()
+{
+return ranges.keySet();
+}
+
+public Collection ends()
+{
+return ranges.values();
+}
+
+public String toString()
+{
+return ranges.toString();
+}
+
+@Override
+public int hashCode()
+{
+return ranges.hashCode();
+}
+
+@Override
+
[cassandra] branch cassandra-3.0 updated (5a2b5a4 -> 8d2556f)
This is an automated email from the ASF dual-hosted git repository. dcapwell pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 5a2b5a4 Merge branch 'cassandra-2.2' into cassandra-3.0 new 844d19d Add ant task to check license and fix all files missing a licence new 8d2556f Merge branch 'cassandra-2.2' into cassandra-3.0 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: build.xml | 22 +- .../apache/cassandra/db/commitlog/IntervalSet.java | 18 ++ .../apache/cassandra/utils/IntegerInterval.java| 18 ++ .../distributed/test/SimpleReadWriteTest.java | 18 ++ .../distributed/test/ThriftClientTest.java | 18 ++ .../distributed/test/ThriftClientUtils.java| 18 ++ .../upgrade/MigrateDropColumnsTest.java| 18 ++ .../org/apache/cassandra/cql3/ViewComplexTest.java | 18 ++ .../cql3/validation/operations/TTLTest.java| 18 ++ .../cassandra/db/commitlog/BatchCommitLogTest.java | 18 ++ .../cassandra/db/commitlog/CommitLogCQLTest.java | 18 ++ .../apache/cassandra/db/marshal/EmptyTypeTest.java | 18 ++ .../apache/cassandra/io/DiskSpaceMetricsTest.java | 18 ++ .../cassandra/utils/IntegerIntervalsTest.java | 18 ++ 14 files changed, 255 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16198) Add ant task to check license and fix all files missing a licence
[ https://issues.apache.org/jira/browse/CASSANDRA-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16198: -- Source Control Link: https://github.com/apache/cassandra/commit/1f68540272e900077dc5775e70a3df65c491836a Resolution: Fixed Status: Resolved (was: Ready to Commit) > Add ant task to check license and fix all files missing a licence > - > > Key: CASSANDRA-16198 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16198 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: NA > > > We should block the build on license to avoid missing licenses in the build. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (83e1e9e -> 16da961)
This is an automated email from the ASF dual-hosted git repository. dcapwell pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 83e1e9e in-jvm dtest now exposes stdout and stderr for nodetool new 844d19d Add ant task to check license and fix all files missing a licence new 8d2556f Merge branch 'cassandra-2.2' into cassandra-3.0 new 6c8c791 Merge branch 'cassandra-3.0' into cassandra-3.11 new 16da961 Merge branch 'cassandra-3.11' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: build.xml | 22 +- .../diag/LastEventIdBroadcasterMBean.java | 18 ++ .../cassandra/hadoop/cql3/CqlClientHelper.java | 18 ++ .../org/apache/cassandra/hints/InputPosition.java | 18 ++ .../apache/cassandra/schema/SchemaProvider.java| 18 ++ src/java/org/apache/cassandra/tools/JMXTool.java | 18 ++ .../utils/logging/LogbackLoggingSupport.java | 18 ++ .../cassandra/utils/logging/LoggingSupport.java| 18 ++ .../utils/logging/LoggingSupportFactory.java | 18 ++ .../utils/logging/NoOpFallbackLoggingSupport.java | 18 ++ .../cassandra/distributed/impl/FileLogAction.java | 18 ++ .../test/FullRepairCoordinatorTimeoutTest.java | 18 ++ .../IncrementalRepairCoordinatorTimeoutTest.java | 18 ++ .../test/PreviewRepairCoordinatorTimeoutTest.java | 18 ++ .../distributed/test/RepairCoordinatorTimeout.java | 18 ++ .../distributed/test/TableEstimatesTest.java | 18 ++ .../org/apache/cassandra/cql3/ViewComplexTest.java | 18 ++ .../cql3/validation/operations/TTLTest.java| 18 ++ .../apache/cassandra/db/marshal/EmptyTypeTest.java | 18 ++ .../apache/cassandra/io/DiskSpaceMetricsTest.java | 18 ++ .../reads/repair/AbstractReadRepairTest.java | 18 ++ .../cassandra/tools/JMXCompatabilityTest.java | 18 ++ .../org/apache/cassandra/tools/JMXToolTest.java| 18 ++ .../apache/cassandra/tools/TopPartitionsTest.java | 18 ++ .../org/apache/cassandra/utils/AssertUtil.java | 18 ++ .../apache/cassandra/utils/ByteArrayUtilTest.java | 18 ++ .../org/apache/cassandra/utils/GeneratorsTest.java | 18 ++ 27 files changed, 489 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. dcapwell pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 16da961f2bbe9a63d97844ecbc54c55838a5964a Merge: 83e1e9e 6c8c791 Author: David Capwell AuthorDate: Wed Oct 7 14:34:11 2020 -0700 Merge branch 'cassandra-3.11' into trunk build.xml | 22 +- .../diag/LastEventIdBroadcasterMBean.java | 18 ++ .../cassandra/hadoop/cql3/CqlClientHelper.java | 18 ++ .../org/apache/cassandra/hints/InputPosition.java | 18 ++ .../apache/cassandra/schema/SchemaProvider.java| 18 ++ src/java/org/apache/cassandra/tools/JMXTool.java | 18 ++ .../utils/logging/LogbackLoggingSupport.java | 18 ++ .../cassandra/utils/logging/LoggingSupport.java| 18 ++ .../utils/logging/LoggingSupportFactory.java | 18 ++ .../utils/logging/NoOpFallbackLoggingSupport.java | 18 ++ .../cassandra/distributed/impl/FileLogAction.java | 18 ++ .../test/FullRepairCoordinatorTimeoutTest.java | 18 ++ .../IncrementalRepairCoordinatorTimeoutTest.java | 18 ++ .../test/PreviewRepairCoordinatorTimeoutTest.java | 18 ++ .../distributed/test/RepairCoordinatorTimeout.java | 18 ++ .../distributed/test/TableEstimatesTest.java | 18 ++ .../org/apache/cassandra/cql3/ViewComplexTest.java | 18 ++ .../cql3/validation/operations/TTLTest.java| 18 ++ .../apache/cassandra/db/marshal/EmptyTypeTest.java | 18 ++ .../apache/cassandra/io/DiskSpaceMetricsTest.java | 18 ++ .../reads/repair/AbstractReadRepairTest.java | 18 ++ .../cassandra/tools/JMXCompatabilityTest.java | 18 ++ .../org/apache/cassandra/tools/JMXToolTest.java| 18 ++ .../apache/cassandra/tools/TopPartitionsTest.java | 18 ++ .../org/apache/cassandra/utils/AssertUtil.java | 18 ++ .../apache/cassandra/utils/ByteArrayUtilTest.java | 18 ++ .../org/apache/cassandra/utils/GeneratorsTest.java | 18 ++ 27 files changed, 489 insertions(+), 1 deletion(-) diff --cc build.xml index 6961cf9,f078d34..6a3eb1e --- a/build.xml +++ b/build.xml @@@ -999,18 -861,9 +999,18 @@@ - - + + + + + + - + + + @@@ -1345,7 -1199,25 +1345,25 @@@ + + + + + + + + + + + + + + + + + - + diff --cc src/java/org/apache/cassandra/diag/LastEventIdBroadcasterMBean.java index 03f05dc,000..01c7bbf mode 100644,00..100644 --- a/src/java/org/apache/cassandra/diag/LastEventIdBroadcasterMBean.java +++ b/src/java/org/apache/cassandra/diag/LastEventIdBroadcasterMBean.java @@@ -1,41 -1,0 +1,59 @@@ ++/* ++ * Licensed to the Apache Software Foundation (ASF) under one ++ * or more contributor license agreements. See the NOTICE file ++ * distributed with this work for additional information ++ * regarding copyright ownership. The ASF licenses this file ++ * to you under the Apache License, Version 2.0 (the ++ * "License"); you may not use this file except in compliance ++ * with the License. You may obtain a copy of the License at ++ * ++ * http://www.apache.org/licenses/LICENSE-2.0 ++ * ++ * Unless required by applicable law or agreed to in writing, software ++ * distributed under the License is distributed on an "AS IS" BASIS, ++ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ++ * See the License for the specific language governing permissions and ++ * limitations under the License. ++ */ ++ +package org.apache.cassandra.diag; + +import java.util.Map; + +/** + * Provides a list of event types and the corresponding highest event IDs. Consumers may these IDs to determine + * if new data is available. + * + * Example result + * + * + * + * Event + * Last ID + * + * + * BootstrapEvent + * 312 + * + * + * CompactionEvent + * a53f9338-5f24-11e8-9c2d-fa7ae01bbebc + * + * + * + * Clients may either retrieve the current list of all events IDs, or make conditional requests for event IDs + * based on the timestamp of the last update (much in the sense of e.g. HTTP's If-Modified-Since semantics). + */ +public interface
[jira] [Commented] (CASSANDRA-15585) 4.0 quality testing: Test Frameworks, Tooling, Infra / Automation
[
https://issues.apache.org/jira/browse/CASSANDRA-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209878#comment-17209878
]
Josh McKenzie commented on CASSANDRA-15585:
---
{quote}We as a project could come together and define 4.0.1 as "8 weeks after
4.0 GA's, the goal is to 1) fix anything operators surface in the release, and
2) have a suite of Harry tests running 24/7 on ASF hardware, and 3) have a
fallout server for ASF C* up and running with adverse scenario tests running".
{quote}
[~jwest] - any chance this resonates with you or are you thinking we block 4.0
GA on wiring up the longer adversarial / fuzz testing in CI/CD prior to GA?
> 4.0 quality testing: Test Frameworks, Tooling, Infra / Automation
> -
>
> Key: CASSANDRA-15585
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15585
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/python
>Reporter: Josh McKenzie
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Reference [doc from
> NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
> for context.
> *Shepherd: Jordan West*
> This area refers to contributions to test frameworks/tooling (e.g., dtests,
> QuickTheories, CASSANDRA-14821), and automation enabling those tools to be
> applied at scale (e.g., replay testing via Spark-based replay of captured FQL
> logs).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16196) Fix flaky test test_disk_balance_after_boundary_change_lcs - disk_balance_test.TestDiskBalance
[
https://issues.apache.org/jira/browse/CASSANDRA-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg reassigned CASSANDRA-16196:
-
Assignee: Adam Holmberg
> Fix flaky test test_disk_balance_after_boundary_change_lcs -
> disk_balance_test.TestDiskBalance
> --
>
> Key: CASSANDRA-16196
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16196
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: David Capwell
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/622/workflows/adcd463c-156a-43c7-a9bc-7f3e4938dbe8/jobs/3514
> {code}
> error_message = '' if 'error_message' not in kwargs else
> kwargs['error_message']
> assert vmin > vmax * (1.0 - error) or vmin == vmax, \
> > "values not within {:.2f}% of the max: {} ({})".format(error *
> > 100, args, error_message)
> E AssertionError: values not within 10.00% of the max: (8022760,
> 9192165, 4575645, 9235566, 9091014) (node2)
> tools/assertions.py:206: AssertionError
> {code}
> Marking as distinct issue after chat in CASSANDRA-14030
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16198) Add ant task to check license and fix all files missing a licence
[
https://issues.apache.org/jira/browse/CASSANDRA-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209867#comment-17209867
]
David Capwell edited comment on CASSANDRA-16198 at 10/7/20, 9:07 PM:
-
[~mck]
bq. What was the purpose with the existing "rat" target?
No idea! It requires we build the tarball then checks both tarballs
(binary/source). I tried running it and gave up after 5 minutes (2.2).
bq. When and were was it (intended to be) run?
Not sure, git says you added it!
{code}
commit 382bcad07dc84b58eb2a0fbec8eab91d8517932b
Author: Mick Semb Wever
Date: Fri Aug 28 10:57:16 2020 +0200
Backport CASSANDRA-14970 to 2.1 - New releases must supply SHA-256 and/or
SHA-512 checksums
patch by Mick Semb Wever; reviewed by Benjamin Lerer for CASSANDRA-16081
{code}
CASSANDRA-14970 doesn't really tell me much though, best I see is "I have moved
the checksumming into the `artifacts` tasks (alongside the generation of the
original artefacts), and renamed the `release` task to `rat`."
First commit I see with rat report is
{code}
commit 0f8b6097559815a3f770fda01b1ec5c19cdad547
Author: Eric Evans
Date: Wed Feb 24 19:54:14 2010 +
merge w/ 0.6 branch
git-svn-id:
https://svn.apache.org/repos/asf/incubator/cassandra/trunk@915953
13f79535-47bb-0310-9956-ffa450edef68
{code}
bq. What shall we do with it now?
Not sure what uses it so I left as is. It was too slow that I don't intend to
use it, not sure if anyone else is...
was (Author: dcapwell):
[~mck]
bq. What was the purpose with the existing "rat" target?
No idea! It requires we build the tarball then checks both tarballs
(binary/source). I tried running it and gave up after 5 minutes (2.2).
bq. When and were was it (intended to be) run?
Not sure, git says you added it!
{code}
commit 382bcad07dc84b58eb2a0fbec8eab91d8517932b
Author: Mick Semb Wever
Date: Fri Aug 28 10:57:16 2020 +0200
Backport CASSANDRA-14970 to 2.1 - New releases must supply SHA-256 and/or
SHA-512 checksums
patch by Mick Semb Wever; reviewed by Benjamin Lerer for CASSANDRA-16081
{code}
CASSANDRA-14970 doesn't really tell me much though, best I see is "I have moved
the checksumming into the `artifacts` tasks (alongside the generation of the
original artefacts), and renamed the `release` task to `rat`."
bq. What shall we do with it now?
Not sure what uses it so I left as is. It was too slow that I don't intend to
use it, not sure if anyone else is...
> Add ant task to check license and fix all files missing a licence
> -
>
> Key: CASSANDRA-16198
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16198
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Low
> Fix For: NA
>
>
> We should block the build on license to avoid missing licenses in the build.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16198) Add ant task to check license and fix all files missing a licence
[
https://issues.apache.org/jira/browse/CASSANDRA-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209867#comment-17209867
]
David Capwell commented on CASSANDRA-16198:
---
[~mck]
bq. What was the purpose with the existing "rat" target?
No idea! It requires we build the tarball then checks both tarballs
(binary/source). I tried running it and gave up after 5 minutes (2.2).
bq. When and were was it (intended to be) run?
Not sure, git says you added it!
{code}
commit 382bcad07dc84b58eb2a0fbec8eab91d8517932b
Author: Mick Semb Wever
Date: Fri Aug 28 10:57:16 2020 +0200
Backport CASSANDRA-14970 to 2.1 - New releases must supply SHA-256 and/or
SHA-512 checksums
patch by Mick Semb Wever; reviewed by Benjamin Lerer for CASSANDRA-16081
{code}
CASSANDRA-14970 doesn't really tell me much though, best I see is "I have moved
the checksumming into the `artifacts` tasks (alongside the generation of the
original artefacts), and renamed the `release` task to `rat`."
bq. What shall we do with it now?
Not sure what uses it so I left as is. It was too slow that I don't intend to
use it, not sure if anyone else is...
> Add ant task to check license and fix all files missing a licence
> -
>
> Key: CASSANDRA-16198
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16198
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Low
> Fix For: NA
>
>
> We should block the build on license to avoid missing licenses in the build.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16113) Consolidate dead nodes check in force repair
[ https://issues.apache.org/jira/browse/CASSANDRA-16113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209863#comment-17209863 ] Yifan Cai commented on CASSANDRA-16113: --- Thank you for the feedback, Blake. I have pushed a new commit to address your comments. `force` was changed to `shouldExcludeDeadParticipants`, which I feel it sounds better. Beside it, all the other suggestions were applied. Kicking off the CI: https://app.circleci.com/pipelines/github/yifan-c/cassandra/116/workflows/503bef2e-0d16-4ce0-9eb0-90fc0752c6d7 > Consolidate dead nodes check in force repair > > > Key: CASSANDRA-16113 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16113 > Project: Cassandra > Issue Type: Improvement > Components: Local/Other >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 4h 10m > Remaining Estimate: 0h > > The check for dead nodes during force repair is duplicated in the normal and > incremental repair. We could consolidate those 2 checks to make the code more > dry. > The check should throw a more meaningful error message to indicate that all > neighbor nodes are down, instead of "java.lang.IllegalArgumentException: > Endpoints can not be empty" -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16127) NullPointerException when calling nodetool enablethrift
[
https://issues.apache.org/jira/browse/CASSANDRA-16127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209860#comment-17209860
]
Jordan West commented on CASSANDRA-16127:
-
LGTM. Changes look good, ran the tests locally, and also manually checked
{{enable/disablethrift}} on earlier branches (and enable/disable binary on
trunk).
> NullPointerException when calling nodetool enablethrift
> ---
>
> Key: CASSANDRA-16127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16127
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Thrift
>Reporter: Tibor Repasi
>Assignee: David Capwell
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x
>
>
> Having thrift disabled, it's impossible to enable it again without restarting
> the node:
> {code}
> $ nodetool statusthrift
> not running
> $ nodetool enablethrift
> error: null
> -- StackTrace --
> java.lang.NullPointerException
> at
> org.apache.cassandra.service.StorageService.startRPCServer(StorageService.java:392)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
> at
> javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
> at
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
> at
> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
> at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
> at sun.rmi.transport.Transport$1.run(Transport.java:200)
> at sun.rmi.transport.Transport$1.run(Transport.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
> at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16198) Add ant task to check license and fix all files missing a licence
[ https://issues.apache.org/jira/browse/CASSANDRA-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209856#comment-17209856 ] David Capwell commented on CASSANDRA-16198: --- Starting commit CI results (pending): 2.2: Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16198-cassandra-2.2-E8B175C8-38F7-4128-8C54-A3A59B304844 Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/75/ 3.0: Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16198-cassandra-3.0-E8B175C8-38F7-4128-8C54-A3A59B304844 Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/76/ 3.11: Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16198-cassandra-3.11-E8B175C8-38F7-4128-8C54-A3A59B304844 Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/77/ trunk: Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16198-trunk-E8B175C8-38F7-4128-8C54-A3A59B304844 Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/78/ > Add ant task to check license and fix all files missing a licence > - > > Key: CASSANDRA-16198 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16198 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: NA > > > We should block the build on license to avoid missing licenses in the build. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16198) Add ant task to check license and fix all files missing a licence
[ https://issues.apache.org/jira/browse/CASSANDRA-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209854#comment-17209854 ] Michael Semb Wever commented on CASSANDRA-16198: Nice spot [~dcapwell]! What was the purpose with the existing "rat" target? When and were was it (intended to be) run? What shall we do with it now? > Add ant task to check license and fix all files missing a licence > - > > Key: CASSANDRA-16198 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16198 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: NA > > > We should block the build on license to avoid missing licenses in the build. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16198) Add ant task to check license and fix all files missing a licence
[ https://issues.apache.org/jira/browse/CASSANDRA-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16198: -- Status: Ready to Commit (was: Review In Progress) > Add ant task to check license and fix all files missing a licence > - > > Key: CASSANDRA-16198 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16198 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: NA > > > We should block the build on license to avoid missing licenses in the build. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16198) Add ant task to check license and fix all files missing a licence
[ https://issues.apache.org/jira/browse/CASSANDRA-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16198: -- Reviewers: Jordan West, David Capwell (was: David Capwell, Jordan West) Jordan West, David Capwell Status: Review In Progress (was: Patch Available) > Add ant task to check license and fix all files missing a licence > - > > Key: CASSANDRA-16198 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16198 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: NA > > > We should block the build on license to avoid missing licenses in the build. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16198) Add ant task to check license and fix all files missing a licence
[ https://issues.apache.org/jira/browse/CASSANDRA-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16198: -- Reviewers: Jordan West (was: David Capwell, Jordan West) > Add ant task to check license and fix all files missing a licence > - > > Key: CASSANDRA-16198 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16198 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: NA > > > We should block the build on license to avoid missing licenses in the build. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16198) Add ant task to check license and fix all files missing a licence
[ https://issues.apache.org/jira/browse/CASSANDRA-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209853#comment-17209853 ] Jordan West commented on CASSANDRA-16198: - +1. Great check to have automated as part of CI. > Add ant task to check license and fix all files missing a licence > - > > Key: CASSANDRA-16198 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16198 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: NA > > > We should block the build on license to avoid missing licenses in the build. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14157) [DTEST] [TRUNK] test_tracing_does_not_interfere_with_digest_calculation - cql_tracing_test.TestCqlTracing failed once : AssertionError: assert 0 == 1
[
https://issues.apache.org/jira/browse/CASSANDRA-14157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-14157:
--
Authors: Adam Holmberg
Test and Documentation Plan:
Test relocated.
No docs required.
Status: Patch Available (was: In Progress)
> [DTEST] [TRUNK] test_tracing_does_not_interfere_with_digest_calculation -
> cql_tracing_test.TestCqlTracing failed once : AssertionError: assert 0 == 1
> -
>
> Key: CASSANDRA-14157
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14157
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Michael Kjellman
>Assignee: Adam Holmberg
>Priority: Normal
> Labels: dtest
> Fix For: 4.0-beta3
>
>
> test_tracing_does_not_interfere_with_digest_calculation -
> cql_tracing_test.TestCqlTracing failed it's assertion once today in a
> circleci run. the dtests were running against trunk.
> Although it has failed once so far, a quick read of the comments in the test
> seems to indicate that the assertion failing this way might mean that
> CASSANDRA-13964 didn't fully fix the issue.
> {code:python}
> if jmx.has_mbean(rr_count):
> # expect 0 digest mismatches
> > assert 0 == jmx.read_attribute(rr_count, 'Count')
> E AssertionError: assert 0 == 1
> E+ where 1 = 0x7f62d4156898>>('org.apache.cassandra.metrics:type=ReadRepair,name=RepairedBlocking',
> 'Count')
> E+where > =
> .read_attribute
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16120) Add ability for jvm-dtest to grep instance logs
[
https://issues.apache.org/jira/browse/CASSANDRA-16120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209851#comment-17209851
]
Jordan West commented on CASSANDRA-16120:
-
Follow-up changes LGTM
> Add ability for jvm-dtest to grep instance logs
> ---
>
> Key: CASSANDRA-16120
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16120
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Fix For: NA
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> One of the main gaps between python dtest and jvm dtest is python dtest
> supports the ability to grep the logs of an instance; we need this capability
> as some tests require validating logs were triggered.
> Pydocs for common log methods
> {code}
> | grep_log(self, expr, filename='system.log', from_mark=None)
> | Returns a list of lines matching the regular expression in parameter
> | in the Cassandra log of this node
> |
> | grep_log_for_errors(self, filename='system.log')
> | Returns a list of errors with stack traces
> | in the Cassandra log of this node
> |
> | grep_log_for_errors_from(self, filename='system.log', seek_start=0)
> {code}
> {code}
> | watch_log_for(self, exprs, from_mark=None, timeout=600, process=None,
> verbose=False, filename='system.log')
> | Watch the log until one or more (regular) expression are found.
> | This methods when all the expressions have been found or the method
> | timeouts (a TimeoutError is then raised). On successful completion,
> | a list of pair (line matched, match object) is returned.
> {code}
> Below is a POC showing a way to do such logic
> {code}
> package org.apache.cassandra.distributed.test;
> import java.io.BufferedReader;
> import java.io.FileInputStream;
> import java.io.IOException;
> import java.io.InputStreamReader;
> import java.io.UncheckedIOException;
> import java.nio.charset.StandardCharsets;
> import java.util.Iterator;
> import java.util.Spliterator;
> import java.util.Spliterators;
> import java.util.regex.Matcher;
> import java.util.regex.Pattern;
> import java.util.stream.Stream;
> import java.util.stream.StreamSupport;
> import com.google.common.io.Closeables;
> import org.junit.Test;
> import org.apache.cassandra.distributed.Cluster;
> import org.apache.cassandra.utils.AbstractIterator;
> public class AllTheLogs extends TestBaseImpl
> {
>@Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = System.getProperty("cassandra.testtag",
> "cassandra.testtag_IS_UNDEFINED");
>String suite = System.getProperty("suitename",
> "suitename_IS_UNDEFINED");
>String log = String.format("build/test/logs/%s/TEST-%s.log", tag,
> suite);
>grep(log, "Enqueuing flush of tables").forEach(l ->
> System.out.println("I found the thing: " + l));
>}
>}
>private static Stream grep(String file, String regex) throws
> IOException
>{
>return grep(file, Pattern.compile(regex));
>}
>private static Stream grep(String file, Pattern regex) throws
> IOException
>{
>BufferedReader reader = new BufferedReader(new InputStreamReader(new
> FileInputStream(file), StandardCharsets.UTF_8));
>Iterator it = new AbstractIterator()
>{
>protected String computeNext()
>{
>try
>{
>String s;
>while ((s = reader.readLine()) != null)
>{
>Matcher m = regex.matcher(s);
>if (m.find())
>return s;
>}
>reader.close();
>return endOfData();
>}
>catch (IOException e)
>{
>Closeables.closeQuietly(reader);
>throw new UncheckedIOException(e);
>}
>}
>};
>return StreamSupport.stream(Spliterators.spliteratorUnknownSize(it,
> Spliterator.ORDERED), false);
>}
> }
> {code}
> And
> {code}
> @Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = System.getProperty("cassandra.testtag",
> "cassandra.testtag_IS_UNDEFINED");
>String suite = System.getProperty("suitename",
> "suitename_IS_UNDEFINED");
>//TODO missing way to get node id
> //cluster.get(1);
>String log =
>
[jira] [Commented] (CASSANDRA-16198) Add ant task to check license and fix all files missing a licence
[ https://issues.apache.org/jira/browse/CASSANDRA-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209850#comment-17209850 ] David Capwell commented on CASSANDRA-16198: --- CI passing now. > Add ant task to check license and fix all files missing a licence > - > > Key: CASSANDRA-16198 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16198 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: NA > > > We should block the build on license to avoid missing licenses in the build. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14157) [DTEST] [TRUNK] test_tracing_does_not_interfere_with_digest_calculation - cql_tracing_test.TestCqlTracing failed once : AssertionError: assert 0 == 1
[
https://issues.apache.org/jira/browse/CASSANDRA-14157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209835#comment-17209835
]
Adam Holmberg commented on CASSANDRA-14157:
---
Running on a resource-constrained VM reproduces almost 100% for me.
I believe this is just a problem with the test. It looks like the read repairs
are valid, occasionally being triggered by queries "under the hood" in cqlsh.
The test rightly expects the [insert and subsequent
select|https://github.com/apache/cassandra-dtest/blob/b117565b8f0096a3ed2af05fdec6e014a05788a1/cql_tracing_test.py#L80-L98]
at CL.ALL to produce no read repairs. However, running with trace in cqlsh
[causes additional queries getting the trace
data|https://github.com/apache/cassandra/blob/83e1e9e45193322f18f57aa7cc4ad31d9d5a152d/bin/cqlsh.py#L1044].
They will be run at the session level of CL.ALL, but the data is written at
CL.ONE. Therefore these queries can cause RR.
Rather than try to work around this in-place, my suggested change relocates the
test out of cql_tracing_tests and uses the driver directly (cqlsh is not the
thing we're testing here).
[patch|https://github.com/apache/cassandra-dtest/compare/master...aholmberg:CASSANDRA-14157?expand=1]
[ci|https://app.circleci.com/pipelines/github/aholmberg/cassandra?branch=CASSANDRA-14157]
(note that another read_repair_test is failing on trunk for a different reason
CASSANDRA-16148)
> [DTEST] [TRUNK] test_tracing_does_not_interfere_with_digest_calculation -
> cql_tracing_test.TestCqlTracing failed once : AssertionError: assert 0 == 1
> -
>
> Key: CASSANDRA-14157
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14157
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Michael Kjellman
>Assignee: Adam Holmberg
>Priority: Normal
> Labels: dtest
> Fix For: 4.0-beta3
>
>
> test_tracing_does_not_interfere_with_digest_calculation -
> cql_tracing_test.TestCqlTracing failed it's assertion once today in a
> circleci run. the dtests were running against trunk.
> Although it has failed once so far, a quick read of the comments in the test
> seems to indicate that the assertion failing this way might mean that
> CASSANDRA-13964 didn't fully fix the issue.
> {code:python}
> if jmx.has_mbean(rr_count):
> # expect 0 digest mismatches
> > assert 0 == jmx.read_attribute(rr_count, 'Count')
> E AssertionError: assert 0 == 1
> E+ where 1 = 0x7f62d4156898>>('org.apache.cassandra.metrics:type=ReadRepair,name=RepairedBlocking',
> 'Count')
> E+where > =
> .read_attribute
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16198) Add ant task to check license and fix all files missing a licence
[
https://issues.apache.org/jira/browse/CASSANDRA-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209832#comment-17209832
]
David Capwell commented on CASSANDRA-16198:
---
CI missing jar, fixing now
{code}
BUILD FAILED
/home/cassandra/cassandra/build.xml:1684: Unable to resolve artifact: Missing:
--
1) org.apache.rat:apache-rat-tasks:jar:0.6
Try downloading the file manually from the project website.
Then, install it using the command:
mvn install:install-file -DgroupId=org.apache.rat
-DartifactId=apache-rat-tasks -Dversion=0.6 -Dpackaging=jar -Dfile=/path/to/file
Alternatively, if you host your own repository you can deploy the file there:
mvn deploy:deploy-file -DgroupId=org.apache.rat
-DartifactId=apache-rat-tasks -Dversion=0.6 -Dpackaging=jar
-Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
Path to dependency:
1) org.apache.maven:super-pom:pom:2.0
2) org.apache.rat:apache-rat-tasks:jar:0.6
{code}
> Add ant task to check license and fix all files missing a licence
> -
>
> Key: CASSANDRA-16198
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16198
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Low
> Fix For: NA
>
>
> We should block the build on license to avoid missing licenses in the build.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16120) Add ability for jvm-dtest to grep instance logs
[
https://issues.apache.org/jira/browse/CASSANDRA-16120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209827#comment-17209827
]
David Capwell commented on CASSANDRA-16120:
---
simpler for me to fix the license in
https://issues.apache.org/jira/browse/CASSANDRA-16198 as this isn't the only
file.
> Add ability for jvm-dtest to grep instance logs
> ---
>
> Key: CASSANDRA-16120
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16120
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Fix For: NA
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> One of the main gaps between python dtest and jvm dtest is python dtest
> supports the ability to grep the logs of an instance; we need this capability
> as some tests require validating logs were triggered.
> Pydocs for common log methods
> {code}
> | grep_log(self, expr, filename='system.log', from_mark=None)
> | Returns a list of lines matching the regular expression in parameter
> | in the Cassandra log of this node
> |
> | grep_log_for_errors(self, filename='system.log')
> | Returns a list of errors with stack traces
> | in the Cassandra log of this node
> |
> | grep_log_for_errors_from(self, filename='system.log', seek_start=0)
> {code}
> {code}
> | watch_log_for(self, exprs, from_mark=None, timeout=600, process=None,
> verbose=False, filename='system.log')
> | Watch the log until one or more (regular) expression are found.
> | This methods when all the expressions have been found or the method
> | timeouts (a TimeoutError is then raised). On successful completion,
> | a list of pair (line matched, match object) is returned.
> {code}
> Below is a POC showing a way to do such logic
> {code}
> package org.apache.cassandra.distributed.test;
> import java.io.BufferedReader;
> import java.io.FileInputStream;
> import java.io.IOException;
> import java.io.InputStreamReader;
> import java.io.UncheckedIOException;
> import java.nio.charset.StandardCharsets;
> import java.util.Iterator;
> import java.util.Spliterator;
> import java.util.Spliterators;
> import java.util.regex.Matcher;
> import java.util.regex.Pattern;
> import java.util.stream.Stream;
> import java.util.stream.StreamSupport;
> import com.google.common.io.Closeables;
> import org.junit.Test;
> import org.apache.cassandra.distributed.Cluster;
> import org.apache.cassandra.utils.AbstractIterator;
> public class AllTheLogs extends TestBaseImpl
> {
>@Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = System.getProperty("cassandra.testtag",
> "cassandra.testtag_IS_UNDEFINED");
>String suite = System.getProperty("suitename",
> "suitename_IS_UNDEFINED");
>String log = String.format("build/test/logs/%s/TEST-%s.log", tag,
> suite);
>grep(log, "Enqueuing flush of tables").forEach(l ->
> System.out.println("I found the thing: " + l));
>}
>}
>private static Stream grep(String file, String regex) throws
> IOException
>{
>return grep(file, Pattern.compile(regex));
>}
>private static Stream grep(String file, Pattern regex) throws
> IOException
>{
>BufferedReader reader = new BufferedReader(new InputStreamReader(new
> FileInputStream(file), StandardCharsets.UTF_8));
>Iterator it = new AbstractIterator()
>{
>protected String computeNext()
>{
>try
>{
>String s;
>while ((s = reader.readLine()) != null)
>{
>Matcher m = regex.matcher(s);
>if (m.find())
>return s;
>}
>reader.close();
>return endOfData();
>}
>catch (IOException e)
>{
>Closeables.closeQuietly(reader);
>throw new UncheckedIOException(e);
>}
>}
>};
>return StreamSupport.stream(Spliterators.spliteratorUnknownSize(it,
> Spliterator.ORDERED), false);
>}
> }
> {code}
> And
> {code}
> @Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = System.getProperty("cassandra.testtag",
> "cassandra.testtag_IS_UNDEFINED");
>String suite = System.getProperty("suitename",
> "suitename_IS_UNDEFINED");
>//TODO missing way
[jira] [Commented] (CASSANDRA-16012) sstablesplit unit test hardening
[
https://issues.apache.org/jira/browse/CASSANDRA-16012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209821#comment-17209821
]
Yifan Cai commented on CASSANDRA-16012:
---
Thanks for the changes. The code LGTM.
Regarding the test coverage, can you add 1 more case that splits multiple
sstables? e.g.
https://cassandra.apache.org/doc/latest/tools/sstable/sstablesplit.html#split-multiple-files
> sstablesplit unit test hardening
>
>
> Key: CASSANDRA-16012
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16012
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/sstable
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Labels: low-hanging-fruit
> Fix For: 4.0-beta
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
>
> During CASSANDRA-15883 / CASSANDRA-15991 it was detected unit test coverage
> for this tool is minimal. There is a unit test to enhance upon under
> {{test/unit/org/apache/cassandra/tools}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16113) Consolidate dead nodes check in force repair
[ https://issues.apache.org/jira/browse/CASSANDRA-16113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Blake Eggleston updated CASSANDRA-16113: Status: Changes Suggested (was: Review In Progress) > Consolidate dead nodes check in force repair > > > Key: CASSANDRA-16113 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16113 > Project: Cassandra > Issue Type: Improvement > Components: Local/Other >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 4h 10m > Remaining Estimate: 0h > > The check for dead nodes during force repair is duplicated in the normal and > incremental repair. We could consolidate those 2 checks to make the code more > dry. > The check should throw a more meaningful error message to indicate that all > neighbor nodes are down, instead of "java.lang.IllegalArgumentException: > Endpoints can not be empty" -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16113) Consolidate dead nodes check in force repair
[ https://issues.apache.org/jira/browse/CASSANDRA-16113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209819#comment-17209819 ] Blake Eggleston commented on CASSANDRA-16113: - Just a few minor naming type stuff: * RepairRunnable.java ** LiveNeighorsAndRanges *** I think naming would make more sense if we changed the naming around for this a bit Rename class back to `NeighorsAndRanges`, since it can contain dead neighbors if the force flag wasn't set. Rename `allNeighbors` to `participants`, so we're not implying liveness or completeness Rename `force` to `deadParticipantsExcluded` or something, to indicate the contents of `participants` WDYT? ** `filterCommonRanges` *** I'd reword the start of the new log message a bit. Something like "Skipping forced repair for ...", which is a bit clearer about what's happening and why *** you only need `Sets.*difference*(commonRange.endpoints, endpoints)` when computing skipped replicas, since trans endpoints are a subset of all endpoints * DistributedRepairUtils.java ** let's add a null check for `moreSuccessCritera` at the top of `assertParentRepairSuccess` so this fails quickly in the case of a null arg > Consolidate dead nodes check in force repair > > > Key: CASSANDRA-16113 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16113 > Project: Cassandra > Issue Type: Improvement > Components: Local/Other >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 4h 10m > Remaining Estimate: 0h > > The check for dead nodes during force repair is duplicated in the normal and > incremental repair. We could consolidate those 2 checks to make the code more > dry. > The check should throw a more meaningful error message to indicate that all > neighbor nodes are down, instead of "java.lang.IllegalArgumentException: > Endpoints can not be empty" -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16198) Add ant task to check license and fix all files missing a licence
[ https://issues.apache.org/jira/browse/CASSANDRA-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16198: -- Test and Documentation Plan: CI build Status: Patch Available (was: Open) > Add ant task to check license and fix all files missing a licence > - > > Key: CASSANDRA-16198 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16198 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: NA > > > We should block the build on license to avoid missing licenses in the build. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15313) Fix flaky - ChecksummingTransformerTest - org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209811#comment-17209811
]
Caleb Rackliffe commented on CASSANDRA-15313:
-
Just for visibility for whoever is watching this,
{{ChecksummingTransformerTest}} will be removed in CASSANDRA-15299. We can keep
this open until that merges though...
> Fix flaky - ChecksummingTransformerTest -
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
> ---
>
> Key: CASSANDRA-15313
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15313
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Vinay Chella
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: CASSANDRA-15313-hack.patch
>
>
> During the recent runs, this test appears to be flaky.
> Example failure:
> [https://circleci.com/gh/vinaykumarchella/cassandra/459#tests/containers/94]
> corruptionCausesFailure-compression -
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
> {code:java}
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
> at org.quicktheories.impl.Precursor.(Precursor.java:17)
> at
> org.quicktheories.impl.ConcreteDetachedSource.(ConcreteDetachedSource.java:8)
> at
> org.quicktheories.impl.ConcreteDetachedSource.detach(ConcreteDetachedSource.java:23)
> at org.quicktheories.generators.Retry.generate(CodePoints.java:51)
> at
> org.quicktheories.generators.Generate.lambda$intArrays$10(Generate.java:190)
> at
> org.quicktheories.generators.Generate$$Lambda$17/1847008471.generate(Unknown
> Source)
> at org.quicktheories.core.DescribingGenerator.generate(Gen.java:255)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$mix$10(Gen.java:184)
> at org.quicktheories.core.Gen$$Lambda$45/802243390.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$flatMap$5(Gen.java:93)
> at org.quicktheories.core.Gen$$Lambda$48/363509958.generate(Unknown
> Source)
> at
> org.quicktheories.dsl.TheoryBuilder4.lambda$prgnToTuple$12(TheoryBuilder4.java:188)
> at
> org.quicktheories.dsl.TheoryBuilder4$$Lambda$40/2003496028.generate(Unknown
> Source)
> at org.quicktheories.core.DescribingGenerator.generate(Gen.java:255)
> at org.quicktheories.core.FilteredGenerator.generate(Gen.java:225)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.impl.Core.generate(Core.java:150)
> at org.quicktheories.impl.Core.shrink(Core.java:103)
> at org.quicktheories.impl.Core.run(Core.java:39)
> at org.quicktheories.impl.TheoryRunner.check(TheoryRunner.java:35)
> at org.quicktheories.dsl.TheoryBuilder4.check(TheoryBuilder4.java:150)
> at
> org.quicktheories.dsl.TheoryBuilder4.checkAssert(TheoryBuilder4.java:162)
> at
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest.corruptionCausesFailure(ChecksummingTransformerTest.java:87)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16198) Add ant task to check license and fix all files missing a licence
David Capwell created CASSANDRA-16198: - Summary: Add ant task to check license and fix all files missing a licence Key: CASSANDRA-16198 URL: https://issues.apache.org/jira/browse/CASSANDRA-16198 Project: Cassandra Issue Type: Improvement Components: Build Reporter: David Capwell Assignee: David Capwell We should block the build on license to avoid missing licenses in the build. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16198) Add ant task to check license and fix all files missing a licence
[ https://issues.apache.org/jira/browse/CASSANDRA-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16198: -- Change Category: Code Clarity Complexity: Low Hanging Fruit Fix Version/s: NA Priority: Low (was: Normal) Status: Open (was: Triage Needed) > Add ant task to check license and fix all files missing a licence > - > > Key: CASSANDRA-16198 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16198 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: NA > > > We should block the build on license to avoid missing licenses in the build. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15181) Ensure Nodes can Start and Stop
[ https://issues.apache.org/jira/browse/CASSANDRA-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinay Chella updated CASSANDRA-15181: - Fix Version/s: (was: 4.0-triage) > Ensure Nodes can Start and Stop > --- > > Key: CASSANDRA-15181 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15181 > Project: Cassandra > Issue Type: Sub-task > Components: Legacy/Streaming and Messaging, Test/benchmark >Reporter: Joey Lynch >Assignee: Vinay Chella >Priority: High > Fix For: 4.0-beta > > > Let's load a cluster up with data and start killing nodes. We can do hard > failures (node terminations) and soft failures (process kills) We plan to > observe the following: > * Can nodes successfully bootstrap? > * How long does it take to bootstrap > * What are the effects of TLS on and off (e.g. on stream time) > * Are hints properly played after a node restart > * Do nodes properly shutdown and start back up. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-12126) CAS Reads Inconsistencies
[ https://issues.apache.org/jira/browse/CASSANDRA-12126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-12126: --- Fix Version/s: (was: 4.0-triage) > CAS Reads Inconsistencies > -- > > Key: CASSANDRA-12126 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12126 > Project: Cassandra > Issue Type: Bug > Components: Feature/Lightweight Transactions, Legacy/Coordination >Reporter: Sankalp Kohli >Assignee: Sylvain Lebresne >Priority: Normal > Labels: LWT, pull-request-available > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Time Spent: 10m > Remaining Estimate: 0h > > While looking at the CAS code in Cassandra, I found a potential issue with > CAS Reads. Here is how it can happen with RF=3 > 1) You issue a CAS Write and it fails in the propose phase. A machine replies > true to a propose and saves the commit in accepted filed. The other two > machines B and C does not get to the accept phase. > Current state is that machine A has this commit in paxos table as accepted > but not committed and B and C does not. > 2) Issue a CAS Read and it goes to only B and C. You wont be able to read the > value written in step 1. This step is as if nothing is inflight. > 3) Issue another CAS Read and it goes to A and B. Now we will discover that > there is something inflight from A and will propose and commit it with the > current ballot. Now we can read the value written in step 1 as part of this > CAS read. > If we skip step 3 and instead run step 4, we will never learn about value > written in step 1. > 4. Issue a CAS Write and it involves only B and C. This will succeed and > commit a different value than step 1. Step 1 value will never be seen again > and was never seen before. > If you read the Lamport “paxos made simple” paper and read section 2.3. It > talks about this issue which is how learners can find out if majority of the > acceptors have accepted the proposal. > In step 3, it is correct that we propose the value again since we dont know > if it was accepted by majority of acceptors. When we ask majority of > acceptors, and more than one acceptors but not majority has something in > flight, we have no way of knowing if it is accepted by majority of acceptors. > So this behavior is correct. > However we need to fix step 2, since it caused reads to not be linearizable > with respect to writes and other reads. In this case, we know that majority > of acceptors have no inflight commit which means we have majority that > nothing was accepted by majority. I think we should run a propose step here > with empty commit and that will cause write written in step 1 to not be > visible ever after. > With this fix, we will either see data written in step 1 on next serial read > or will never see it which is what we want. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16152) In-JVM dtest - modify schema with stopped nodes and use yaml fragments for config
[ https://issues.apache.org/jira/browse/CASSANDRA-16152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209799#comment-17209799 ] Jon Meredith commented on CASSANDRA-16152: -- I knew it would throw intentionally, but we'll only write this code once and use it countless times so will improve the exception. Good call. > In-JVM dtest - modify schema with stopped nodes and use yaml fragments for > config > - > > Key: CASSANDRA-16152 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16152 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > > Some convenience improvements to in-JVM dtest that are useful across versions > that I needed while working on CASSANDRA-16144 > * Add support for changing schema with stopped nodes. > * Make it simpler to modify nested configuration items by specifying Yaml > fragments -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16120) Add ability for jvm-dtest to grep instance logs
[
https://issues.apache.org/jira/browse/CASSANDRA-16120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209792#comment-17209792
]
David Capwell commented on CASSANDRA-16120:
---
[~yifanc] pushed fix to license to all branches.
> Add ability for jvm-dtest to grep instance logs
> ---
>
> Key: CASSANDRA-16120
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16120
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Fix For: NA
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> One of the main gaps between python dtest and jvm dtest is python dtest
> supports the ability to grep the logs of an instance; we need this capability
> as some tests require validating logs were triggered.
> Pydocs for common log methods
> {code}
> | grep_log(self, expr, filename='system.log', from_mark=None)
> | Returns a list of lines matching the regular expression in parameter
> | in the Cassandra log of this node
> |
> | grep_log_for_errors(self, filename='system.log')
> | Returns a list of errors with stack traces
> | in the Cassandra log of this node
> |
> | grep_log_for_errors_from(self, filename='system.log', seek_start=0)
> {code}
> {code}
> | watch_log_for(self, exprs, from_mark=None, timeout=600, process=None,
> verbose=False, filename='system.log')
> | Watch the log until one or more (regular) expression are found.
> | This methods when all the expressions have been found or the method
> | timeouts (a TimeoutError is then raised). On successful completion,
> | a list of pair (line matched, match object) is returned.
> {code}
> Below is a POC showing a way to do such logic
> {code}
> package org.apache.cassandra.distributed.test;
> import java.io.BufferedReader;
> import java.io.FileInputStream;
> import java.io.IOException;
> import java.io.InputStreamReader;
> import java.io.UncheckedIOException;
> import java.nio.charset.StandardCharsets;
> import java.util.Iterator;
> import java.util.Spliterator;
> import java.util.Spliterators;
> import java.util.regex.Matcher;
> import java.util.regex.Pattern;
> import java.util.stream.Stream;
> import java.util.stream.StreamSupport;
> import com.google.common.io.Closeables;
> import org.junit.Test;
> import org.apache.cassandra.distributed.Cluster;
> import org.apache.cassandra.utils.AbstractIterator;
> public class AllTheLogs extends TestBaseImpl
> {
>@Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = System.getProperty("cassandra.testtag",
> "cassandra.testtag_IS_UNDEFINED");
>String suite = System.getProperty("suitename",
> "suitename_IS_UNDEFINED");
>String log = String.format("build/test/logs/%s/TEST-%s.log", tag,
> suite);
>grep(log, "Enqueuing flush of tables").forEach(l ->
> System.out.println("I found the thing: " + l));
>}
>}
>private static Stream grep(String file, String regex) throws
> IOException
>{
>return grep(file, Pattern.compile(regex));
>}
>private static Stream grep(String file, Pattern regex) throws
> IOException
>{
>BufferedReader reader = new BufferedReader(new InputStreamReader(new
> FileInputStream(file), StandardCharsets.UTF_8));
>Iterator it = new AbstractIterator()
>{
>protected String computeNext()
>{
>try
>{
>String s;
>while ((s = reader.readLine()) != null)
>{
>Matcher m = regex.matcher(s);
>if (m.find())
>return s;
>}
>reader.close();
>return endOfData();
>}
>catch (IOException e)
>{
>Closeables.closeQuietly(reader);
>throw new UncheckedIOException(e);
>}
>}
>};
>return StreamSupport.stream(Spliterators.spliteratorUnknownSize(it,
> Spliterator.ORDERED), false);
>}
> }
> {code}
> And
> {code}
> @Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = System.getProperty("cassandra.testtag",
> "cassandra.testtag_IS_UNDEFINED");
>String suite = System.getProperty("suitename",
> "suitename_IS_UNDEFINED");
>//TODO missing way to get node id
> //cluster.get(1);
>String log =
[jira] [Comment Edited] (CASSANDRA-16120) Add ability for jvm-dtest to grep instance logs
[
https://issues.apache.org/jira/browse/CASSANDRA-16120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209261#comment-17209261
]
David Capwell edited comment on CASSANDRA-16120 at 10/7/20, 7:14 PM:
-
Backports:
trunk:
https://github.com/dcapwell/cassandra/tree/backport/CASSANDRA-16120-trunk (fix
license)
3.11: https://github.com/dcapwell/cassandra/tree/backport/CASSANDRA-16120
3.0: https://github.com/dcapwell/cassandra/tree/backport/CASSANDRA-16120-3.0
2.2: https://github.com/dcapwell/cassandra/tree/backport/CASSANDRA-16120-2.2
was (Author: dcapwell):
Backports:
3.11: https://github.com/dcapwell/cassandra/tree/backport/CASSANDRA-16120
3.0: https://github.com/dcapwell/cassandra/tree/backport/CASSANDRA-16120-3.0
2.2: https://github.com/dcapwell/cassandra/tree/backport/CASSANDRA-16120-2.2
> Add ability for jvm-dtest to grep instance logs
> ---
>
> Key: CASSANDRA-16120
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16120
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Fix For: NA
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> One of the main gaps between python dtest and jvm dtest is python dtest
> supports the ability to grep the logs of an instance; we need this capability
> as some tests require validating logs were triggered.
> Pydocs for common log methods
> {code}
> | grep_log(self, expr, filename='system.log', from_mark=None)
> | Returns a list of lines matching the regular expression in parameter
> | in the Cassandra log of this node
> |
> | grep_log_for_errors(self, filename='system.log')
> | Returns a list of errors with stack traces
> | in the Cassandra log of this node
> |
> | grep_log_for_errors_from(self, filename='system.log', seek_start=0)
> {code}
> {code}
> | watch_log_for(self, exprs, from_mark=None, timeout=600, process=None,
> verbose=False, filename='system.log')
> | Watch the log until one or more (regular) expression are found.
> | This methods when all the expressions have been found or the method
> | timeouts (a TimeoutError is then raised). On successful completion,
> | a list of pair (line matched, match object) is returned.
> {code}
> Below is a POC showing a way to do such logic
> {code}
> package org.apache.cassandra.distributed.test;
> import java.io.BufferedReader;
> import java.io.FileInputStream;
> import java.io.IOException;
> import java.io.InputStreamReader;
> import java.io.UncheckedIOException;
> import java.nio.charset.StandardCharsets;
> import java.util.Iterator;
> import java.util.Spliterator;
> import java.util.Spliterators;
> import java.util.regex.Matcher;
> import java.util.regex.Pattern;
> import java.util.stream.Stream;
> import java.util.stream.StreamSupport;
> import com.google.common.io.Closeables;
> import org.junit.Test;
> import org.apache.cassandra.distributed.Cluster;
> import org.apache.cassandra.utils.AbstractIterator;
> public class AllTheLogs extends TestBaseImpl
> {
>@Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = System.getProperty("cassandra.testtag",
> "cassandra.testtag_IS_UNDEFINED");
>String suite = System.getProperty("suitename",
> "suitename_IS_UNDEFINED");
>String log = String.format("build/test/logs/%s/TEST-%s.log", tag,
> suite);
>grep(log, "Enqueuing flush of tables").forEach(l ->
> System.out.println("I found the thing: " + l));
>}
>}
>private static Stream grep(String file, String regex) throws
> IOException
>{
>return grep(file, Pattern.compile(regex));
>}
>private static Stream grep(String file, Pattern regex) throws
> IOException
>{
>BufferedReader reader = new BufferedReader(new InputStreamReader(new
> FileInputStream(file), StandardCharsets.UTF_8));
>Iterator it = new AbstractIterator()
>{
>protected String computeNext()
>{
>try
>{
>String s;
>while ((s = reader.readLine()) != null)
>{
>Matcher m = regex.matcher(s);
>if (m.find())
>return s;
>}
>reader.close();
>return endOfData();
>}
>catch (IOException e)
>{
>Closeables.closeQuietly(reader);
>throw new UncheckedIOException(e);
>}
>
[jira] [Assigned] (CASSANDRA-15865) Flaky dtest hintedhandoff_test.py::TestHintedHandoffConfig::test_hintedhandoff_setmaxwindow
[ https://issues.apache.org/jira/browse/CASSANDRA-15865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams reassigned CASSANDRA-15865: Assignee: Brandon Williams > Flaky dtest > hintedhandoff_test.py::TestHintedHandoffConfig::test_hintedhandoff_setmaxwindow > --- > > Key: CASSANDRA-15865 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15865 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Sam Tunnicliffe >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.0-beta > > > I've seen this fail a couple of times under JDK11, when it doesn't appear to > be related to the changes under test. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15865) Flaky dtest hintedhandoff_test.py::TestHintedHandoffConfig::test_hintedhandoff_setmaxwindow
[ https://issues.apache.org/jira/browse/CASSANDRA-15865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams reassigned CASSANDRA-15865: Assignee: (was: Charles Attwood Thomas) > Flaky dtest > hintedhandoff_test.py::TestHintedHandoffConfig::test_hintedhandoff_setmaxwindow > --- > > Key: CASSANDRA-15865 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15865 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Sam Tunnicliffe >Priority: Normal > Fix For: 4.0-beta > > > I've seen this fail a couple of times under JDK11, when it doesn't appear to > be related to the changes under test. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16182) A replacement node, although completed bootstrap and joined ring according to itself, stuck in Joining state as per the peers
[
https://issues.apache.org/jira/browse/CASSANDRA-16182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209772#comment-17209772
]
Sumanth Pasupuleti commented on CASSANDRA-16182:
{quote}C' would not complete replacement if it detects C presence before
then.{quote}
I agree it will be nice if C' would halt its replacement, however, this depends
on the timing/chance that C' should hear about C before it completes it
replacement.
{quote}nodes should apply the C' state once C is taken offline, after
confirming it is still valid{quote}
There would be a window of time between C' announcing itself to be available
and returning inconsistent results for the LOCAL_ONE requests coming directly
to it and when C is marked DOWN by other peers. But +1 to this approach since
this would address the issue deterministically imo.
{quote}But that's the contract accepted at ONE anyway{quote}
I agree with this w.r.t. legit data inconsistency that would eventually be
fixed either through hints / repairs. However, in this scenario, until the
operator takes an action, there is no bound on the potential inconsistency.
> A replacement node, although completed bootstrap and joined ring according to
> itself, stuck in Joining state as per the peers
> -
>
> Key: CASSANDRA-16182
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16182
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 3.0.x
>
>
> This issue occurred in a production 3.0.21 cluster.
> Here is what happened
> # We had, say, a three node Cassandra cluster with nodes A, B and C
> # C got "terminated by cloud provider" due to health check failure and a
> replacement node C' got launched.
> # C' started bootstrapping data from its neighbors
> # Network flaw: Nodes A,B were still able to communicate with terminated node
> C and consequently still have C as alive.
> # The replacement node C' learnt about C through gossip but was unable to
> communicate with C and marked C as DOWN.
> # C' completed bootstrapping successfully and itself and its peers logged
> this statement "Node C' will complete replacement of C for tokens
> [-7686143363672898397]"
> # C' logged the statement "Nodes C' and C have the same token
> -7686143363672898397. C' is the new owner"
> # C' started listening for thrift and cql clients
> # Peer nodes A and B logged "Node C' cannot complete replacement of alive
> node C "
> # A few seconds later, A and B marked C as DOWN
> C' continued to log below lines in an endless fashion
> {code:java}
> Node C is now part of the cluster
> Nodes () and C' have the same token C. Ignoring -7686143363672898397 (Needs
> a log statement fix)
> FatClient C has been silent for 3ms, removing from gossip
> {code}
> My reasoning of what happened:
> By the time replacement node (C') finished bootstrapping and announced it's
> state to Normal, A and B were still able to communicate with the replacing
> node C (while C' was not able to with C), and hence rejected C' replacing C.
> C' does not know this and does not attempt to recommunicate its "Normal"
> state to rest of the cluster. (Worth noting that A and B marked C as down
> soon after)
> Gossip keeps telling C' to add C to its metadata, and C' keeps kicking C out
> eventually based on FailureDetector.
> Proposed fix:
> When C' is notified through gossip about C, and given both own the same token
> and given C' has finished bootstrapping, C' can emit its Normal state again
> which should fix this in my opinion (so long as A and B have marked C as
> DOWN, which they did eventually)
> I ended up manually fixing this by restarting Cassandra on C', which forced
> it to announce its "Normal" state via
> StorageService.initServer --> joinTokenRing() --> finishJoiningRing() -->
> setTokens() --> setGossipTokens()
> Alternately, I could have possibly achieved the same behavior if I disabled
> and enabled gossip via jmx/nodetool.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14030) disk_balance_bootstrap_test - disk_balance_test.TestDiskBalance fails: Missing: ['127.0.0.5.* now UP']:
[
https://issues.apache.org/jira/browse/CASSANDRA-14030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-14030:
--
Fix Version/s: (was: 4.0-beta)
> disk_balance_bootstrap_test - disk_balance_test.TestDiskBalance fails:
> Missing: ['127.0.0.5.* now UP']:
> ---
>
> Key: CASSANDRA-14030
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14030
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Testing
>Reporter: Michael Kjellman
>Assignee: Adam Holmberg
>Priority: Normal
>
> disk_balance_bootstrap_test - disk_balance_test.TestDiskBalance fails:
> Missing: ['127.0.0.5.* now UP']:
> {code}
> 15 Nov 2017 11:28:03 [node4] Missing: ['127.0.0.5.* now UP']:
> .
> See system.log for remainder
> >> begin captured logging <<
> dtest: DEBUG: cluster ccm directory: /tmp/dtest-NZzhNb
> dtest: DEBUG: Done setting configuration options:
> { 'initial_token': None,
> 'num_tokens': '32',
> 'phi_convict_threshold': 5,
> 'range_request_timeout_in_ms': 1,
> 'read_request_timeout_in_ms': 1,
> 'request_timeout_in_ms': 1,
> 'truncate_request_timeout_in_ms': 1,
> 'write_request_timeout_in_ms': 1}
> - >> end captured logging << -
> File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
> File "/home/cassandra/cassandra-dtest/disk_balance_test.py", line 44, in
> disk_balance_bootstrap_test
> node5.start(wait_for_binary_proto=True)
> File
> "/home/cassandra/env/local/lib/python2.7/site-packages/ccmlib/node.py", line
> 706, in start
> node.watch_log_for_alive(self, from_mark=mark)
> File
> "/home/cassandra/env/local/lib/python2.7/site-packages/ccmlib/node.py", line
> 520, in watch_log_for_alive
> self.watch_log_for(tofind, from_mark=from_mark, timeout=timeout,
> filename=filename)
> File
> "/home/cassandra/env/local/lib/python2.7/site-packages/ccmlib/node.py", line
> 488, in watch_log_for
> raise TimeoutError(time.strftime("%d %b %Y %H:%M:%S", time.gmtime()) + "
> [" + self.name + "] Missing: " + str([e.pattern for e in tofind]) + ":\n" +
> reads[:50] + ".\nSee {} for remainder".format(filename))
> "15 Nov 2017 11:28:03 [node4] Missing: ['127.0.0.5.* now UP']:\n.\nSee
> system.log for remainder\n >> begin captured logging <<
> \ndtest: DEBUG: cluster ccm directory:
> /tmp/dtest-NZzhNb\ndtest: DEBUG: Done setting configuration options:\n{
> 'initial_token': None,\n'num_tokens': '32',\n'phi_convict_threshold':
> 5,\n'range_request_timeout_in_ms': 1,\n
> 'read_request_timeout_in_ms': 1,\n'request_timeout_in_ms': 1,\n
> 'truncate_request_timeout_in_ms': 1,\n'write_request_timeout_in_ms':
> 1}\n- >> end captured logging <<
> -"
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14030) disk_balance_bootstrap_test - disk_balance_test.TestDiskBalance fails: Missing: ['127.0.0.5.* now UP']:
[
https://issues.apache.org/jira/browse/CASSANDRA-14030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-14030:
--
Resolution: Cannot Reproduce
Status: Resolved (was: Open)
> disk_balance_bootstrap_test - disk_balance_test.TestDiskBalance fails:
> Missing: ['127.0.0.5.* now UP']:
> ---
>
> Key: CASSANDRA-14030
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14030
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Testing
>Reporter: Michael Kjellman
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> disk_balance_bootstrap_test - disk_balance_test.TestDiskBalance fails:
> Missing: ['127.0.0.5.* now UP']:
> {code}
> 15 Nov 2017 11:28:03 [node4] Missing: ['127.0.0.5.* now UP']:
> .
> See system.log for remainder
> >> begin captured logging <<
> dtest: DEBUG: cluster ccm directory: /tmp/dtest-NZzhNb
> dtest: DEBUG: Done setting configuration options:
> { 'initial_token': None,
> 'num_tokens': '32',
> 'phi_convict_threshold': 5,
> 'range_request_timeout_in_ms': 1,
> 'read_request_timeout_in_ms': 1,
> 'request_timeout_in_ms': 1,
> 'truncate_request_timeout_in_ms': 1,
> 'write_request_timeout_in_ms': 1}
> - >> end captured logging << -
> File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
> File "/home/cassandra/cassandra-dtest/disk_balance_test.py", line 44, in
> disk_balance_bootstrap_test
> node5.start(wait_for_binary_proto=True)
> File
> "/home/cassandra/env/local/lib/python2.7/site-packages/ccmlib/node.py", line
> 706, in start
> node.watch_log_for_alive(self, from_mark=mark)
> File
> "/home/cassandra/env/local/lib/python2.7/site-packages/ccmlib/node.py", line
> 520, in watch_log_for_alive
> self.watch_log_for(tofind, from_mark=from_mark, timeout=timeout,
> filename=filename)
> File
> "/home/cassandra/env/local/lib/python2.7/site-packages/ccmlib/node.py", line
> 488, in watch_log_for
> raise TimeoutError(time.strftime("%d %b %Y %H:%M:%S", time.gmtime()) + "
> [" + self.name + "] Missing: " + str([e.pattern for e in tofind]) + ":\n" +
> reads[:50] + ".\nSee {} for remainder".format(filename))
> "15 Nov 2017 11:28:03 [node4] Missing: ['127.0.0.5.* now UP']:\n.\nSee
> system.log for remainder\n >> begin captured logging <<
> \ndtest: DEBUG: cluster ccm directory:
> /tmp/dtest-NZzhNb\ndtest: DEBUG: Done setting configuration options:\n{
> 'initial_token': None,\n'num_tokens': '32',\n'phi_convict_threshold':
> 5,\n'range_request_timeout_in_ms': 1,\n
> 'read_request_timeout_in_ms': 1,\n'request_timeout_in_ms': 1,\n
> 'truncate_request_timeout_in_ms': 1,\n'write_request_timeout_in_ms':
> 1}\n- >> end captured logging <<
> -"
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16152) In-JVM dtest - modify schema with stopped nodes and use yaml fragments for config
[
https://issues.apache.org/jira/browse/CASSANDRA-16152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209753#comment-17209753
]
Yifan Cai commented on CASSANDRA-16152:
---
Glad it helps.
I took a look at the newer commits. It seems that we are taking advantage of
{{Optional.get()}} in {{AbstractCluster#schemaChange(String, boolean)}}
intentionally. Wondering if it gives better exception by throwing an
{{IllegalStateException}} with the message saying "all nodes are down".
Otherwise, I am +1 on the trunk patch.
> In-JVM dtest - modify schema with stopped nodes and use yaml fragments for
> config
> -
>
> Key: CASSANDRA-16152
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16152
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest/java
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
>
> Some convenience improvements to in-JVM dtest that are useful across versions
> that I needed while working on CASSANDRA-16144
> * Add support for changing schema with stopped nodes.
> * Make it simpler to modify nested configuration items by specifying Yaml
> fragments
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
[ https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209723#comment-17209723 ] David Capwell commented on CASSANDRA-16150: --- tests look mostly clean, so I am +1 (I am ok with 1.27 or 1.26) [~ifesdjeen] can you review as well? just want to make sure the harry side of things is also fine with this change. > Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix > --- > > Key: CASSANDRA-16150 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16150 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Rahul Nandi >Assignee: Rahul Nandi >Priority: Normal > Fix For: 4.x > > > There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml > version earlier to 1.26. This has been patched into snakeyaml version 1.26. > Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640] > This card is expected to upgrade the snakeyaml version to 1.26. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
[ https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16150: -- Reviewers: Alex Petrov, David Capwell (was: David Capwell) > Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix > --- > > Key: CASSANDRA-16150 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16150 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Rahul Nandi >Assignee: Rahul Nandi >Priority: Normal > Fix For: 4.x > > > There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml > version earlier to 1.26. This has been patched into snakeyaml version 1.26. > Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640] > This card is expected to upgrade the snakeyaml version to 1.26. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16120) Add ability for jvm-dtest to grep instance logs
[
https://issues.apache.org/jira/browse/CASSANDRA-16120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209710#comment-17209710
]
David Capwell commented on CASSANDRA-16120:
---
good catch, ill fix that in this then.
> Add ability for jvm-dtest to grep instance logs
> ---
>
> Key: CASSANDRA-16120
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16120
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Fix For: NA
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> One of the main gaps between python dtest and jvm dtest is python dtest
> supports the ability to grep the logs of an instance; we need this capability
> as some tests require validating logs were triggered.
> Pydocs for common log methods
> {code}
> | grep_log(self, expr, filename='system.log', from_mark=None)
> | Returns a list of lines matching the regular expression in parameter
> | in the Cassandra log of this node
> |
> | grep_log_for_errors(self, filename='system.log')
> | Returns a list of errors with stack traces
> | in the Cassandra log of this node
> |
> | grep_log_for_errors_from(self, filename='system.log', seek_start=0)
> {code}
> {code}
> | watch_log_for(self, exprs, from_mark=None, timeout=600, process=None,
> verbose=False, filename='system.log')
> | Watch the log until one or more (regular) expression are found.
> | This methods when all the expressions have been found or the method
> | timeouts (a TimeoutError is then raised). On successful completion,
> | a list of pair (line matched, match object) is returned.
> {code}
> Below is a POC showing a way to do such logic
> {code}
> package org.apache.cassandra.distributed.test;
> import java.io.BufferedReader;
> import java.io.FileInputStream;
> import java.io.IOException;
> import java.io.InputStreamReader;
> import java.io.UncheckedIOException;
> import java.nio.charset.StandardCharsets;
> import java.util.Iterator;
> import java.util.Spliterator;
> import java.util.Spliterators;
> import java.util.regex.Matcher;
> import java.util.regex.Pattern;
> import java.util.stream.Stream;
> import java.util.stream.StreamSupport;
> import com.google.common.io.Closeables;
> import org.junit.Test;
> import org.apache.cassandra.distributed.Cluster;
> import org.apache.cassandra.utils.AbstractIterator;
> public class AllTheLogs extends TestBaseImpl
> {
>@Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = System.getProperty("cassandra.testtag",
> "cassandra.testtag_IS_UNDEFINED");
>String suite = System.getProperty("suitename",
> "suitename_IS_UNDEFINED");
>String log = String.format("build/test/logs/%s/TEST-%s.log", tag,
> suite);
>grep(log, "Enqueuing flush of tables").forEach(l ->
> System.out.println("I found the thing: " + l));
>}
>}
>private static Stream grep(String file, String regex) throws
> IOException
>{
>return grep(file, Pattern.compile(regex));
>}
>private static Stream grep(String file, Pattern regex) throws
> IOException
>{
>BufferedReader reader = new BufferedReader(new InputStreamReader(new
> FileInputStream(file), StandardCharsets.UTF_8));
>Iterator it = new AbstractIterator()
>{
>protected String computeNext()
>{
>try
>{
>String s;
>while ((s = reader.readLine()) != null)
>{
>Matcher m = regex.matcher(s);
>if (m.find())
>return s;
>}
>reader.close();
>return endOfData();
>}
>catch (IOException e)
>{
>Closeables.closeQuietly(reader);
>throw new UncheckedIOException(e);
>}
>}
>};
>return StreamSupport.stream(Spliterators.spliteratorUnknownSize(it,
> Spliterator.ORDERED), false);
>}
> }
> {code}
> And
> {code}
> @Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = System.getProperty("cassandra.testtag",
> "cassandra.testtag_IS_UNDEFINED");
>String suite = System.getProperty("suitename",
> "suitename_IS_UNDEFINED");
>//TODO missing way to get node id
> //cluster.get(1);
>String log =
>
[jira] [Commented] (CASSANDRA-16127) NullPointerException when calling nodetool enablethrift
[
https://issues.apache.org/jira/browse/CASSANDRA-16127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209706#comment-17209706
]
David Capwell commented on CASSANDRA-16127:
---
Update, I found out what was failing the test and it was a review comment to
remove the validation; this is important for survey mode.
I ported the test to a jvm dtest and found a few gaps in which the test acts
differently so also fixed them to be in-line with CassandraDaemon.
I will wait for CASSANDRA-16120 to merge as it is needed for the bootstrap test.
> NullPointerException when calling nodetool enablethrift
> ---
>
> Key: CASSANDRA-16127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16127
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Thrift
>Reporter: Tibor Repasi
>Assignee: David Capwell
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x
>
>
> Having thrift disabled, it's impossible to enable it again without restarting
> the node:
> {code}
> $ nodetool statusthrift
> not running
> $ nodetool enablethrift
> error: null
> -- StackTrace --
> java.lang.NullPointerException
> at
> org.apache.cassandra.service.StorageService.startRPCServer(StorageService.java:392)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
> at
> javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
> at
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
> at
> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
> at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
> at sun.rmi.transport.Transport$1.run(Transport.java:200)
> at sun.rmi.transport.Transport$1.run(Transport.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
> at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16152) In-JVM dtest - modify schema with stopped nodes and use yaml fragments for config
[ https://issues.apache.org/jira/browse/CASSANDRA-16152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209700#comment-17209700 ] Jon Meredith commented on CASSANDRA-16152: -- Thanks for the feedback, I liked your commit and think it makes things better. I've applied it to trunk only as I wanted to limit maintenance of older releases to the minimum. I've added the requested test and found an issue with running while node1 is down. Thanks for asking for the test. I also added checks to the message filters to make sure we don't get delegate test-related errors/noise sending messages to down nodes. I'll re-check CircleCI and try and round up a committer to look over it and... commit. > In-JVM dtest - modify schema with stopped nodes and use yaml fragments for > config > - > > Key: CASSANDRA-16152 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16152 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > > Some convenience improvements to in-JVM dtest that are useful across versions > that I needed while working on CASSANDRA-16144 > * Add support for changing schema with stopped nodes. > * Make it simpler to modify nested configuration items by specifying Yaml > fragments -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16197) Upgrade the metrics version
[ https://issues.apache.org/jira/browse/CASSANDRA-16197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-16197: --- Change Category: Quality Assurance Complexity: Low Hanging Fruit Status: Open (was: Triage Needed) > Upgrade the metrics version > --- > > Key: CASSANDRA-16197 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16197 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies >Reporter: Benjamin Lerer >Assignee: Benjamin Lerer >Priority: Normal > > The current metrics version used by Cassandra is 3.1.5 which was not compiled > and targeted for the JDK 8 > (https://metrics.dropwizard.io/4.1.2/about/release-notes.html). > There are several bug fixes that would also be interesting to get. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16197) Upgrade the metrics version
Benjamin Lerer created CASSANDRA-16197: -- Summary: Upgrade the metrics version Key: CASSANDRA-16197 URL: https://issues.apache.org/jira/browse/CASSANDRA-16197 Project: Cassandra Issue Type: Improvement Components: Dependencies Reporter: Benjamin Lerer Assignee: Benjamin Lerer The current metrics version used by Cassandra is 3.1.5 which was not compiled and targeted for the JDK 8 (https://metrics.dropwizard.io/4.1.2/about/release-notes.html). There are several bug fixes that would also be interesting to get. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16063) Fix user experience when upgrading to 4.0 with compact tables
[
https://issues.apache.org/jira/browse/CASSANDRA-16063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209615#comment-17209615
]
Brandon Williams commented on CASSANDRA-16063:
--
bq. Gossip-based at the time, so Brandon Williams is not going to be happy
It can definitely work, so that's fine. You're just going to leave a scar in
AppicationState to forever remind of us of your transgression. :)
Really though, it's fine, it's certainly a big enough hammer to get the job
done.
> Fix user experience when upgrading to 4.0 with compact tables
> -
>
> Key: CASSANDRA-16063
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16063
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Sylvain Lebresne
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: Compact_storage_upgrade_tests.txt
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The code to handle compact tables has been removed from 4.0, and the intended
> upgrade path to 4.0 for users having compact tables on 3.x is that they must
> execute {{ALTER ... DROP COMPACT STORAGE}} on all of their compact tables
> *before* attempting the upgrade.
> Obviously, some users won't read the upgrade instructions (or miss a table)
> and may try upgrading despite still having compact tables. If they do so, the
> intent is that the node will _not_ start, with a message clearly indicating
> the pre-upgrade step the user has missed. The user will then downgrade back
> the node(s) to 3.x, run the proper {{ALTER ... DROP COMPACT STORAGE}}, and
> then upgrade again.
> But while 4.0 does currently fail startup when finding any compact tables
> with a decent message, I believe the check is done too late during startup.
> Namely, that check is done as we read the tables schema, so within
> [{{Schema.instance.loadFromDisk()}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/CassandraDaemon.java#L241].
> But by then, we've _at least_ called
> {{SystemKeyspace.persistLocalMetadata()}}} and
> {{SystemKeyspaceMigrator40.migrate()}}, which will get into the commit log,
> and even possibly flush new {{na}} format sstables. As a results, a user
> might not be able to seemlessly restart the node on 3.x (to drop compact
> storage on the appropriate tables).
> Basically, we should make sure the check for compact tables done at 4.0
> startup is done as a {{StartupCheck}}, before the node does anything.
> We should also add a test for this (checking that if you try upgrading to 4.0
> with compact storage, you can downgrade back with no intervention whatsoever).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-15897: Fix Version/s: 4.0-beta3 > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus Eriksson >Assignee: Sylvain Lebresne >Priority: Normal > Fix For: 4.0-beta3 > > > Test reproducing: > https://github.com/krummas/cassandra/commits/marcuse/dropcompactstorage -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16063) Fix user experience when upgrading to 4.0 with compact tables
[
https://issues.apache.org/jira/browse/CASSANDRA-16063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209602#comment-17209602
]
Ekaterina Dimitrova commented on CASSANDRA-16063:
-
[~slebresne], please, ping me on the other ticket when you push the branch you
mentioned and I will be happy to take a look and finish whatever is left there.
> Fix user experience when upgrading to 4.0 with compact tables
> -
>
> Key: CASSANDRA-16063
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16063
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Sylvain Lebresne
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: Compact_storage_upgrade_tests.txt
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The code to handle compact tables has been removed from 4.0, and the intended
> upgrade path to 4.0 for users having compact tables on 3.x is that they must
> execute {{ALTER ... DROP COMPACT STORAGE}} on all of their compact tables
> *before* attempting the upgrade.
> Obviously, some users won't read the upgrade instructions (or miss a table)
> and may try upgrading despite still having compact tables. If they do so, the
> intent is that the node will _not_ start, with a message clearly indicating
> the pre-upgrade step the user has missed. The user will then downgrade back
> the node(s) to 3.x, run the proper {{ALTER ... DROP COMPACT STORAGE}}, and
> then upgrade again.
> But while 4.0 does currently fail startup when finding any compact tables
> with a decent message, I believe the check is done too late during startup.
> Namely, that check is done as we read the tables schema, so within
> [{{Schema.instance.loadFromDisk()}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/CassandraDaemon.java#L241].
> But by then, we've _at least_ called
> {{SystemKeyspace.persistLocalMetadata()}}} and
> {{SystemKeyspaceMigrator40.migrate()}}, which will get into the commit log,
> and even possibly flush new {{na}} format sstables. As a results, a user
> might not be able to seemlessly restart the node on 3.x (to drop compact
> storage on the appropriate tables).
> Basically, we should make sure the check for compact tables done at 4.0
> startup is done as a {{StartupCheck}}, before the node does anything.
> We should also add a test for this (checking that if you try upgrading to 4.0
> with compact storage, you can downgrade back with no intervention whatsoever).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16063) Fix user experience when upgrading to 4.0 with compact tables
[
https://issues.apache.org/jira/browse/CASSANDRA-16063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209600#comment-17209600
]
Ekaterina Dimitrova commented on CASSANDRA-16063:
-
[~slebresne], thanks a lot for reminding me of that one! Looks like I even
commented on it but as I looked at that more than a month ago I totally forgot
about it and that it will handle this... :(
But on the bright side, these are good news that this is already on its way. :)
I will rebase and wrap up the code here for commit and submit a final CI run.
Thanks both [~brandon.williams] and [~slebresne]!
> Fix user experience when upgrading to 4.0 with compact tables
> -
>
> Key: CASSANDRA-16063
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16063
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Sylvain Lebresne
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: Compact_storage_upgrade_tests.txt
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The code to handle compact tables has been removed from 4.0, and the intended
> upgrade path to 4.0 for users having compact tables on 3.x is that they must
> execute {{ALTER ... DROP COMPACT STORAGE}} on all of their compact tables
> *before* attempting the upgrade.
> Obviously, some users won't read the upgrade instructions (or miss a table)
> and may try upgrading despite still having compact tables. If they do so, the
> intent is that the node will _not_ start, with a message clearly indicating
> the pre-upgrade step the user has missed. The user will then downgrade back
> the node(s) to 3.x, run the proper {{ALTER ... DROP COMPACT STORAGE}}, and
> then upgrade again.
> But while 4.0 does currently fail startup when finding any compact tables
> with a decent message, I believe the check is done too late during startup.
> Namely, that check is done as we read the tables schema, so within
> [{{Schema.instance.loadFromDisk()}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/CassandraDaemon.java#L241].
> But by then, we've _at least_ called
> {{SystemKeyspace.persistLocalMetadata()}}} and
> {{SystemKeyspaceMigrator40.migrate()}}, which will get into the commit log,
> and even possibly flush new {{na}} format sstables. As a results, a user
> might not be able to seemlessly restart the node on 3.x (to drop compact
> storage on the appropriate tables).
> Basically, we should make sure the check for compact tables done at 4.0
> startup is done as a {{StartupCheck}}, before the node does anything.
> We should also add a test for this (checking that if you try upgrading to 4.0
> with compact storage, you can downgrade back with no intervention whatsoever).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16193) Add tests for Messaging/Internode metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-16193: --- Change Category: Quality Assurance Complexity: Low Hanging Fruit Status: Open (was: Triage Needed) > Add tests for Messaging/Internode metrics > - > > Key: CASSANDRA-16193 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16193 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Benjamin Lerer >Priority: Normal > Fix For: 4.0-beta > > > We do not seems to have tests for the Messaging/Internode metrics. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16191) Add tests for Repair metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-16191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-16191:
---
Change Category: Quality Assurance
Complexity: Low Hanging Fruit
Status: Open (was: Triage Needed)
> Add tests for Repair metrics
>
>
> Key: CASSANDRA-16191
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16191
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest/java
>Reporter: Benjamin Lerer
>Priority: Normal
> Fix For: 4.0-beta
>
>
> We do not seems to have some tests for the {{RepairMetrics.previewFailures}}
> counter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16192) Add more tests to cover compation metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-16192: --- Change Category: Quality Assurance Complexity: Low Hanging Fruit Status: Open (was: Triage Needed) > Add more tests to cover compation metrics > - > > Key: CASSANDRA-16192 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16192 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Benjamin Lerer >Priority: Normal > Fix For: 4.0-beta > > > Some compaction metrics do not seems to be tested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16189) Add tests for the Hint service metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-16189: --- Change Category: Quality Assurance Complexity: Low Hanging Fruit Status: Open (was: Triage Needed) > Add tests for the Hint service metrics > -- > > Key: CASSANDRA-16189 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16189 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/python >Reporter: Benjamin Lerer >Priority: Normal > Fix For: 4.0-beta > > > There are currently no tests for the hint metrics -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16190) Add tests for streaming metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-16190: --- Summary: Add tests for streaming metrics (was: Add tests for streaming mettrics) > Add tests for streaming metrics > --- > > Key: CASSANDRA-16190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16190 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/python >Reporter: Benjamin Lerer >Priority: Normal > Fix For: 4.0-beta > > > We currently have no tests to checks the streaming metrics -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16190) Add tests for streaming metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-16190: --- Change Category: Quality Assurance Complexity: Low Hanging Fruit Status: Open (was: Triage Needed) > Add tests for streaming metrics > --- > > Key: CASSANDRA-16190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16190 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/python >Reporter: Benjamin Lerer >Priority: Normal > Fix For: 4.0-beta > > > We currently have no tests to checks the streaming metrics -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16188) Add more tests to cover Keyspace and Table metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-16188: --- Change Category: Quality Assurance Complexity: Low Hanging Fruit Status: Open (was: Triage Needed) > Add more tests to cover Keyspace and Table metrics > --- > > Key: CASSANDRA-16188 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16188 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java, Test/unit >Reporter: Benjamin Lerer >Priority: Normal > Fix For: 4.0-beta > > > Several Keyspace and Table related metrics are currently not tested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16187) Add test to check the ReconcileRead ReadRepair metric
[
https://issues.apache.org/jira/browse/CASSANDRA-16187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-16187:
---
Change Category: Quality Assurance
Complexity: Low Hanging Fruit
Status: Open (was: Triage Needed)
> Add test to check the ReconcileRead ReadRepair metric
> -
>
> Key: CASSANDRA-16187
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16187
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest/java
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 4.0-beta
>
>
> While going through the metrics for {{ReadRepair}} in CASSANDRA-15582. I
> discovered that we are not testing the {{ReconcileRead}} metric.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16185) Add tests to cover CommitLog metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-16185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-16185:
---
Change Category: Quality Assurance
Complexity: Low Hanging Fruit
Status: Open (was: Triage Needed)
> Add tests to cover CommitLog metrics
>
>
> Key: CASSANDRA-16185
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16185
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Benjamin Lerer
>Priority: Normal
> Fix For: 4.0-beta
>
>
> The only metrics that seems to be covered by unit test for the CommitLog
> metrics is {{oversizedMutations}}. We should add testing the other ones.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16186) Add tests for ThreadPoolMetrics
[
https://issues.apache.org/jira/browse/CASSANDRA-16186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-16186:
---
Change Category: Quality Assurance
Complexity: Low Hanging Fruit
Status: Open (was: Triage Needed)
> Add tests for ThreadPoolMetrics
> ---
>
> Key: CASSANDRA-16186
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16186
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Benjamin Lerer
>Priority: Normal
> Fix For: 4.0-beta
>
>
> We should add some tests to ensure that metrics are correctly published for
> {{SEPExecutor}} and {{JMXEnabledThreadPoolExecutor}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16183) Add tests to cover ClientRequest metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-16183: --- Change Category: Quality Assurance Complexity: Low Hanging Fruit Status: Open (was: Triage Needed) > Add tests to cover ClientRequest metrics > - > > Key: CASSANDRA-16183 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16183 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Benjamin Lerer >Priority: Normal > Fix For: 4.0-beta > > > We do not have test that covers the ClientRequest metrics. > * ClientRequestMetrics > * CASClientRequestMetrics > * CASClientWriteRequestMetrics > * ClientWriteRequestMetrics > * ViewWriteMetrics -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16063) Fix user experience when upgrading to 4.0 with compact tables
[
https://issues.apache.org/jira/browse/CASSANDRA-16063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209591#comment-17209591
]
Sylvain Lebresne commented on CASSANDRA-16063:
--
bq. Before I start any implementation, I decided first to update the ticket and
confirm the approach.
CASSANDRA-15897 is open to implement _exactly_ that problem. I suggest we
commit the fix on this ticket as is and leave the issue of cluster-wide
detection to CASSANDRA-15897. We did discuss options there some time ago, and
kind of settled on Gossip-based at the time, so [~brandon.williams] is not
going to be happy. I do have an almost ready branch for that Gossip approach
btw (which, I won't deny, is a bit involved), and while I don't have time to
get this to the finish line right now, I can share my branch (tomorrow most
likely) and you can decide whether to use that or not.
bq. how do we handle nodes that are down?
Fwiw, my existing branch for CASSANDRA-15897 make nodes share the sstables
version they have in use. If a node is down, other nodes simply rely on the
last information they got from that node, which should work pretty well in
practice.
> Fix user experience when upgrading to 4.0 with compact tables
> -
>
> Key: CASSANDRA-16063
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16063
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Sylvain Lebresne
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: Compact_storage_upgrade_tests.txt
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The code to handle compact tables has been removed from 4.0, and the intended
> upgrade path to 4.0 for users having compact tables on 3.x is that they must
> execute {{ALTER ... DROP COMPACT STORAGE}} on all of their compact tables
> *before* attempting the upgrade.
> Obviously, some users won't read the upgrade instructions (or miss a table)
> and may try upgrading despite still having compact tables. If they do so, the
> intent is that the node will _not_ start, with a message clearly indicating
> the pre-upgrade step the user has missed. The user will then downgrade back
> the node(s) to 3.x, run the proper {{ALTER ... DROP COMPACT STORAGE}}, and
> then upgrade again.
> But while 4.0 does currently fail startup when finding any compact tables
> with a decent message, I believe the check is done too late during startup.
> Namely, that check is done as we read the tables schema, so within
> [{{Schema.instance.loadFromDisk()}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/CassandraDaemon.java#L241].
> But by then, we've _at least_ called
> {{SystemKeyspace.persistLocalMetadata()}}} and
> {{SystemKeyspaceMigrator40.migrate()}}, which will get into the commit log,
> and even possibly flush new {{na}} format sstables. As a results, a user
> might not be able to seemlessly restart the node on 3.x (to drop compact
> storage on the appropriate tables).
> Basically, we should make sure the check for compact tables done at 4.0
> startup is done as a {{StartupCheck}}, before the node does anything.
> We should also add a test for this (checking that if you try upgrading to 4.0
> with compact storage, you can downgrade back with no intervention whatsoever).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16063) Fix user experience when upgrading to 4.0 with compact tables
[
https://issues.apache.org/jira/browse/CASSANDRA-16063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209574#comment-17209574
]
Brandon Williams commented on CASSANDRA-16063:
--
Gossip is a really big hammer for this nail, and in general not a good fit.
Also, how do we handle nodes that are down? Since a schema change is
two-pronged, first trying an rpc call to every machine, and then using gossip
as a fallback, we should probably do the same here: make an rpc call to every
node to determine if the change is ok, and if any node is not up or the check
doesn't succeed, everything fails. If the operator wants to really drop these,
they need to satisfy the same conditions as say, truncate, and have all nodes
available.
> Fix user experience when upgrading to 4.0 with compact tables
> -
>
> Key: CASSANDRA-16063
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16063
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Sylvain Lebresne
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: Compact_storage_upgrade_tests.txt
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The code to handle compact tables has been removed from 4.0, and the intended
> upgrade path to 4.0 for users having compact tables on 3.x is that they must
> execute {{ALTER ... DROP COMPACT STORAGE}} on all of their compact tables
> *before* attempting the upgrade.
> Obviously, some users won't read the upgrade instructions (or miss a table)
> and may try upgrading despite still having compact tables. If they do so, the
> intent is that the node will _not_ start, with a message clearly indicating
> the pre-upgrade step the user has missed. The user will then downgrade back
> the node(s) to 3.x, run the proper {{ALTER ... DROP COMPACT STORAGE}}, and
> then upgrade again.
> But while 4.0 does currently fail startup when finding any compact tables
> with a decent message, I believe the check is done too late during startup.
> Namely, that check is done as we read the tables schema, so within
> [{{Schema.instance.loadFromDisk()}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/CassandraDaemon.java#L241].
> But by then, we've _at least_ called
> {{SystemKeyspace.persistLocalMetadata()}}} and
> {{SystemKeyspaceMigrator40.migrate()}}, which will get into the commit log,
> and even possibly flush new {{na}} format sstables. As a results, a user
> might not be able to seemlessly restart the node on 3.x (to drop compact
> storage on the appropriate tables).
> Basically, we should make sure the check for compact tables done at 4.0
> startup is done as a {{StartupCheck}}, before the node does anything.
> We should also add a test for this (checking that if you try upgrading to 4.0
> with compact storage, you can downgrade back with no intervention whatsoever).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209560#comment-17209560
]
Ekaterina Dimitrova commented on CASSANDRA-15234:
-
Thanks [~benedict]. I think maybe it is a matter of Jira permissions. My only
option is to select only from a list of already created versions.
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 5.0
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16063) Fix user experience when upgrading to 4.0 with compact tables
[
https://issues.apache.org/jira/browse/CASSANDRA-16063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209556#comment-17209556
]
Ekaterina Dimitrova commented on CASSANDRA-16063:
-
Thanks for the review [~adelapena] and [~slebresne]
[~adelapena]'s point is correct. I was checking yesterday for a way to collect
the global information but I didn't find an implemented already way. Also,
additional test was added to handle this topic.
>From what we talked with [~adelapena] and other engineers, probably our best
>bet will be to use Gossip?
Before I start any implementation, I decided first to update the ticket and
confirm the approach.
[~adelapena] , [~slebresne] , [~marcuse], [~ifesdjeen], any thoughts here? Or
anyone else?
Maybe [~brandon.williams] or [~jwest] had some idea as they raised the upgrade
of SSTables topic as part of this ticket :)
To summarize, in order to enforce users to upgrade their SSTables before using
ALTER DROP COMPACT STORAGE, we need to be able to check globally which nodes we
need to update. It seems there is no current implementation to collect this
knowledge. So Gossip?
> Fix user experience when upgrading to 4.0 with compact tables
> -
>
> Key: CASSANDRA-16063
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16063
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Sylvain Lebresne
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: Compact_storage_upgrade_tests.txt
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The code to handle compact tables has been removed from 4.0, and the intended
> upgrade path to 4.0 for users having compact tables on 3.x is that they must
> execute {{ALTER ... DROP COMPACT STORAGE}} on all of their compact tables
> *before* attempting the upgrade.
> Obviously, some users won't read the upgrade instructions (or miss a table)
> and may try upgrading despite still having compact tables. If they do so, the
> intent is that the node will _not_ start, with a message clearly indicating
> the pre-upgrade step the user has missed. The user will then downgrade back
> the node(s) to 3.x, run the proper {{ALTER ... DROP COMPACT STORAGE}}, and
> then upgrade again.
> But while 4.0 does currently fail startup when finding any compact tables
> with a decent message, I believe the check is done too late during startup.
> Namely, that check is done as we read the tables schema, so within
> [{{Schema.instance.loadFromDisk()}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/CassandraDaemon.java#L241].
> But by then, we've _at least_ called
> {{SystemKeyspace.persistLocalMetadata()}}} and
> {{SystemKeyspaceMigrator40.migrate()}}, which will get into the commit log,
> and even possibly flush new {{na}} format sstables. As a results, a user
> might not be able to seemlessly restart the node on 3.x (to drop compact
> storage on the appropriate tables).
> Basically, we should make sure the check for compact tables done at 4.0
> startup is done as a {{StartupCheck}}, before the node does anything.
> We should also add a test for this (checking that if you try upgrading to 4.0
> with compact storage, you can downgrade back with no intervention whatsoever).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209537#comment-17209537
]
Benedict Elliott Smith edited comment on CASSANDRA-15234 at 10/7/20, 1:24 PM:
--
Not sure if you need special permissions, but you can just create a new version.
Sorry for my having not had time to finish my proposal - but I agree having a
bit more time to decide and consult on the best layout is probably a good thing.
was (Author: benedict):
Not sure if you need special permissions, but you can just create a new version.
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 5.0
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15889) Debian package fails to download on Arm-based hosts
[
https://issues.apache.org/jira/browse/CASSANDRA-15889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Davis updated CASSANDRA-15889:
---
Description:
Following the first three steps of the [Debian install
process|https://cassandra.apache.org/download/], after an apt-get update you'll
see this line:
{code:bash}
$ sudo apt-get update
...
N: Skipping acquire of configured file 'main/binary-arm64/Packages' as
repository 'https://downloads.apache.org/cassandra/debian 311x InRelease'
doesn't support architecture 'arm64'
{code}
Checking the [Debian
repo|https://dl.bintray.com/apache/cassandra/dists/311x/main/] confirms there
is no aarch64 variant available.
Should you then attempt to install Cassandra:
{code:bash}
$ sudo apt-get install cassandra
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package cassandra is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package 'cassandra' has no installation candidate
{code}
Note that there is a workaround available: if you specify "amd64" as the arch
for the source, it downloads and runs on Arm without issue:
{code:bash}
echo "deb [arch=amd64] https://downloads.apache.org/cassandra/debian 311x main"
| sudo tee -a /etc/apt/sources.list.d/cassandra.sources.list
{code}
The Redhat RPM contains a "noarch" arch type, so it will download on any host.
(Cassandra does not use separate binaries/releases for different architectures,
so this seems to be the correct approach, but adding an aarch64 variant would
also suffice.)
was:
Following the first three steps of the [Debian install
process|https://cassandra.apache.org/download/], after an apt-get update you'll
see this line:
{code:bash}
$ sudo apt-get update
...
N: Skipping acquire of configured file 'main/binary-arm64/Packages' as
repository 'https://downloads.apache.org/cassandra/debian 311x InRelease'
doesn't support architecture 'arm64'
{code}
Checking the [Debian
repo|https://dl.bintray.com/apache/cassandra/dists/311x/main/] confirms there
is no aarch64 variant available.
Should you then attempt to install Cassandra:
{code:bash}
$ sudo apt-get install cassandra
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package cassandra is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package 'cassandra' has no installation candidate
{code}
Note that there is a workaround available: if you specify "amd64" as the arch
for the source, it downloads and runs on Arm without issue:
{code:bash}
deb [arch=amd64] https://downloads.apache.org/cassandra/debian 311x main
{code}
The Redhat RPM contains a "noarch" arch type, so it will download on any host.
(Cassandra does not use separate binaries/releases for different architectures,
so this seems to be the correct approach, but adding an aarch64 variant would
also suffice.)
> Debian package fails to download on Arm-based hosts
> ---
>
> Key: CASSANDRA-15889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15889
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Matt Davis
>Priority: Normal
> Labels: ARM, debian
>
> Following the first three steps of the [Debian install
> process|https://cassandra.apache.org/download/], after an apt-get update
> you'll see this line:
> {code:bash}
> $ sudo apt-get update
> ...
> N: Skipping acquire of configured file 'main/binary-arm64/Packages' as
> repository 'https://downloads.apache.org/cassandra/debian 311x InRelease'
> doesn't support architecture 'arm64'
> {code}
> Checking the [Debian
> repo|https://dl.bintray.com/apache/cassandra/dists/311x/main/] confirms there
> is no aarch64 variant available.
> Should you then attempt to install Cassandra:
> {code:bash}
> $ sudo apt-get install cassandra
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> Package cassandra is not available, but is referred to by another package.
> This may mean that the package is missing, has been obsoleted, or
> is only available from another source
> E: Package 'cassandra' has no installation candidate
> {code}
> Note that there is a workaround available: if you specify "amd64" as the arch
> for the source, it downloads and runs on Arm without issue:
> {code:bash}
> echo "deb [arch=amd64] https://downloads.apache.org/cassandra/debian 311x
> main" | sudo tee -a /etc/apt/sources.list.d/cassandra.sources.list
> {code}
> The Redhat RPM contains a "noarch" arch type, so it will download on any
> host. (Cassandra does not use separate binaries/releases
