[jira] [Commented] (CASSANDRA-13994) Remove COMPACT STORAGE internals before 4.0 release
[
https://issues.apache.org/jira/browse/CASSANDRA-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057504#comment-17057504
]
Ekaterina Dimitrova commented on CASSANDRA-13994:
-
Thanks, I put it on my list!
> Remove COMPACT STORAGE internals before 4.0 release
> ---
>
> Key: CASSANDRA-13994
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13994
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Local Write-Read Paths
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Low
> Fix For: 4.0, 4.0-rc
>
>
> 4.0 comes without thrift (after [CASSANDRA-5]) and COMPACT STORAGE (after
> [CASSANDRA-10857]), and since Compact Storage flags are now disabled, all of
> the related functionality is useless.
> There are still some things to consider:
> 1. One of the system tables (built indexes) was compact. For now, we just
> added {{value}} column to it to make sure it's backwards-compatible, but we
> might want to make sure it's just a "normal" table and doesn't have redundant
> columns.
> 2. Compact Tables were building indexes in {{KEYS}} mode. Removing it is
> trivial, but this would mean that all built indexes will be defunct. We could
> log a warning for now and ask users to migrate off those for now and
> completely remove it from future releases. It's just a couple of classes
> though.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15637) CqlInputFormat regression going from 2.1 to 3.x caused by semantic difference between thrift and the new system.size_estimates table when dealing with multiple dc de
[
https://issues.apache.org/jira/browse/CASSANDRA-15637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-15637:
--
Description:
In 3.0 CqlInputFormat switched away from thrift in favor of a new
system.size_estimates table, but the semantics changed when dealing with
multiple DCs or when Cassandra is not collocated with Hadoop.
The core issues are:
* system.size_estimates uses the primary range, in a multi-dc setup this could
lead to uneven ranges
example:
{code}
DC1: [0, 10, 20, 30]
DC2: [1, 11, 21, 31]
DC3: [2, 12, 22, 32]
{code}
Using NetworkTopologyStrategy the primary ranges are: [0, 1), [1, 2), [2, 10),
[10, 11), [11, 12), [12, 20), [20, 21), [21, 22), [22, 30), [30, 31), [31, 32),
[32, 0).
Given this the only ranges that are more than one token are: [2, 10), [12, 20),
[22, 30).
* system.size_estimates is not replicated so need to hit every node in the
cluster to get estimates, if nodes are down in the DC with non-size-1 ranges
there is no way to get a estimate.
* CqlInputFormat used to call describe_local_ring so all interactions were with
a single DC, the java driver doesn't filter the DC so looks to allow cross DC
traffic and includes nodes from other DCs in the replica set; in the example
above, the amount of splits went from 4 to 12.
* CqlInputFormat used to call describe_splits_ex to dynamically calculate the
estimates, this was on the "local primary range" and was able to hit replicas
to create estimates if the primary was down. With system.size_estimates we no
longer have backup and no longer expose the "local primary range" in multi-dc.
* CqlInputFormat had a config cassandra.input.keyRange which let you define
your own range. If the range doesn't perfectly match the local range then the
intersectWith calls will produce ranges with no estimates. Example: [0, 10,
20], cassandra.input.keyRange=5,15. This won't find any estimates so will
produce 2 splits with 128 estimate (default when not found).
* CqlInputFormat special cases Cassandra being collocated with Hadoop and
assumes this when querying system.size_estimates as it doesn't filter to the
specific host, this means that non-collocated deployments randomly select the
nodes and create splits with ranges the hosts do not have locally.
The problems are deterministic to replicate, the following test will show it
1) deploy a 3 DC cluster with 3 nodes each
2) create DC2 tokens are +1 of DC1 and DC3 are +1 of DC2
3) CREATE KEYSPACE simpleuniform0 WITH replication = {‘class’:
‘NetworkTopologyStrategy’, ‘DC1’: 3, ‘DC2’: 3, ‘DC3’: 3};
4) CREATE TABLE simpletable0 (pk bigint, ck bigint, value blob, PRIMARY KEY
(pk, ck))
5) insert 500k partitions uniformly: [0, 500,000)
6) wait until estimates catch up to writes
7) for all nodes, SELECT * FROM system.size_estimates
You will get the following
{code}
keyspace_name | table_name | range_start | range_end|
mean_partition_size | partitions_count
+--+--+--+-+--
simpleuniform0 | simpletable0 | -9223372036854775808 | -6148914691236517206 |
87 | 122240
simpleuniform0 | simpletable0 | 6148914691236517207 | -9223372036854775808 |
87 | 121472
(2 rows)
keyspace_name | table_name | range_start | range_end |
mean_partition_size | partitions_count
+--+-+-+-+--
simpleuniform0 | simpletable0 | 2 | 6148914691236517205 |
87 | 243072
(1 rows)
keyspace_name | table_name | range_start | range_end|
mean_partition_size | partitions_count
+--+--+--+-+--
simpleuniform0 | simpletable0 | -6148914691236517206 | -6148914691236517205 |
87 |1
(1 rows)
keyspace_name | table_name | range_start | range_end | mean_partition_size
| partitions_count
+--+-+---+-+--
simpleuniform0 | simpletable0 | 0 | 1 | 87
|1
(1 rows)
keyspace_name | table_name | range_start | range_end |
mean_partition_size | partitions_count
+--+-+-+-+--
simpleuniform0 | simpletable0 | 6148914691236517205 | 6148914691236517206 |
87 |1
(1 rows)
keyspace_name | table_name | range_start | range_end|
mean_partition_size | partitions_count
[jira] [Updated] (CASSANDRA-15313) Fix flaky - ChecksummingTransformerTest - org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-15313:
--
Fix Version/s: (was: 4.0-alpha)
4.0-beta
> Fix flaky - ChecksummingTransformerTest -
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
> ---
>
> Key: CASSANDRA-15313
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15313
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Vinay Chella
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: CASSANDRA-15313-hack.patch
>
>
> During the recent runs, this test appears to be flaky.
> Example failure:
> [https://circleci.com/gh/vinaykumarchella/cassandra/459#tests/containers/94]
> corruptionCausesFailure-compression -
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
> {code:java}
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
> at org.quicktheories.impl.Precursor.(Precursor.java:17)
> at
> org.quicktheories.impl.ConcreteDetachedSource.(ConcreteDetachedSource.java:8)
> at
> org.quicktheories.impl.ConcreteDetachedSource.detach(ConcreteDetachedSource.java:23)
> at org.quicktheories.generators.Retry.generate(CodePoints.java:51)
> at
> org.quicktheories.generators.Generate.lambda$intArrays$10(Generate.java:190)
> at
> org.quicktheories.generators.Generate$$Lambda$17/1847008471.generate(Unknown
> Source)
> at org.quicktheories.core.DescribingGenerator.generate(Gen.java:255)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$mix$10(Gen.java:184)
> at org.quicktheories.core.Gen$$Lambda$45/802243390.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$flatMap$5(Gen.java:93)
> at org.quicktheories.core.Gen$$Lambda$48/363509958.generate(Unknown
> Source)
> at
> org.quicktheories.dsl.TheoryBuilder4.lambda$prgnToTuple$12(TheoryBuilder4.java:188)
> at
> org.quicktheories.dsl.TheoryBuilder4$$Lambda$40/2003496028.generate(Unknown
> Source)
> at org.quicktheories.core.DescribingGenerator.generate(Gen.java:255)
> at org.quicktheories.core.FilteredGenerator.generate(Gen.java:225)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.impl.Core.generate(Core.java:150)
> at org.quicktheories.impl.Core.shrink(Core.java:103)
> at org.quicktheories.impl.Core.run(Core.java:39)
> at org.quicktheories.impl.TheoryRunner.check(TheoryRunner.java:35)
> at org.quicktheories.dsl.TheoryBuilder4.check(TheoryBuilder4.java:150)
> at
> org.quicktheories.dsl.TheoryBuilder4.checkAssert(TheoryBuilder4.java:162)
> at
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest.corruptionCausesFailure(ChecksummingTransformerTest.java:87)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15313) Fix flaky - ChecksummingTransformerTest - org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh McKenzie updated CASSANDRA-15313:
--
Status: Open (was: Patch Available)
> Fix flaky - ChecksummingTransformerTest -
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
> ---
>
> Key: CASSANDRA-15313
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15313
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Vinay Chella
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15313-hack.patch
>
>
> During the recent runs, this test appears to be flaky.
> Example failure:
> [https://circleci.com/gh/vinaykumarchella/cassandra/459#tests/containers/94]
> corruptionCausesFailure-compression -
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
> {code:java}
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
> at org.quicktheories.impl.Precursor.(Precursor.java:17)
> at
> org.quicktheories.impl.ConcreteDetachedSource.(ConcreteDetachedSource.java:8)
> at
> org.quicktheories.impl.ConcreteDetachedSource.detach(ConcreteDetachedSource.java:23)
> at org.quicktheories.generators.Retry.generate(CodePoints.java:51)
> at
> org.quicktheories.generators.Generate.lambda$intArrays$10(Generate.java:190)
> at
> org.quicktheories.generators.Generate$$Lambda$17/1847008471.generate(Unknown
> Source)
> at org.quicktheories.core.DescribingGenerator.generate(Gen.java:255)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$mix$10(Gen.java:184)
> at org.quicktheories.core.Gen$$Lambda$45/802243390.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$flatMap$5(Gen.java:93)
> at org.quicktheories.core.Gen$$Lambda$48/363509958.generate(Unknown
> Source)
> at
> org.quicktheories.dsl.TheoryBuilder4.lambda$prgnToTuple$12(TheoryBuilder4.java:188)
> at
> org.quicktheories.dsl.TheoryBuilder4$$Lambda$40/2003496028.generate(Unknown
> Source)
> at org.quicktheories.core.DescribingGenerator.generate(Gen.java:255)
> at org.quicktheories.core.FilteredGenerator.generate(Gen.java:225)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.impl.Core.generate(Core.java:150)
> at org.quicktheories.impl.Core.shrink(Core.java:103)
> at org.quicktheories.impl.Core.run(Core.java:39)
> at org.quicktheories.impl.TheoryRunner.check(TheoryRunner.java:35)
> at org.quicktheories.dsl.TheoryBuilder4.check(TheoryBuilder4.java:150)
> at
> org.quicktheories.dsl.TheoryBuilder4.checkAssert(TheoryBuilder4.java:162)
> at
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest.corruptionCausesFailure(ChecksummingTransformerTest.java:87)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15313) Fix flaky - ChecksummingTransformerTest - org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh McKenzie updated CASSANDRA-15313:
--
Test and Documentation Plan: Trying to get this thing to Blocked. That's
the plan.
Status: Patch Available (was: In Progress)
> Fix flaky - ChecksummingTransformerTest -
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
> ---
>
> Key: CASSANDRA-15313
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15313
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Vinay Chella
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15313-hack.patch
>
>
> During the recent runs, this test appears to be flaky.
> Example failure:
> [https://circleci.com/gh/vinaykumarchella/cassandra/459#tests/containers/94]
> corruptionCausesFailure-compression -
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
> {code:java}
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
> at org.quicktheories.impl.Precursor.(Precursor.java:17)
> at
> org.quicktheories.impl.ConcreteDetachedSource.(ConcreteDetachedSource.java:8)
> at
> org.quicktheories.impl.ConcreteDetachedSource.detach(ConcreteDetachedSource.java:23)
> at org.quicktheories.generators.Retry.generate(CodePoints.java:51)
> at
> org.quicktheories.generators.Generate.lambda$intArrays$10(Generate.java:190)
> at
> org.quicktheories.generators.Generate$$Lambda$17/1847008471.generate(Unknown
> Source)
> at org.quicktheories.core.DescribingGenerator.generate(Gen.java:255)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$mix$10(Gen.java:184)
> at org.quicktheories.core.Gen$$Lambda$45/802243390.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$flatMap$5(Gen.java:93)
> at org.quicktheories.core.Gen$$Lambda$48/363509958.generate(Unknown
> Source)
> at
> org.quicktheories.dsl.TheoryBuilder4.lambda$prgnToTuple$12(TheoryBuilder4.java:188)
> at
> org.quicktheories.dsl.TheoryBuilder4$$Lambda$40/2003496028.generate(Unknown
> Source)
> at org.quicktheories.core.DescribingGenerator.generate(Gen.java:255)
> at org.quicktheories.core.FilteredGenerator.generate(Gen.java:225)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.impl.Core.generate(Core.java:150)
> at org.quicktheories.impl.Core.shrink(Core.java:103)
> at org.quicktheories.impl.Core.run(Core.java:39)
> at org.quicktheories.impl.TheoryRunner.check(TheoryRunner.java:35)
> at org.quicktheories.dsl.TheoryBuilder4.check(TheoryBuilder4.java:150)
> at
> org.quicktheories.dsl.TheoryBuilder4.checkAssert(TheoryBuilder4.java:162)
> at
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest.corruptionCausesFailure(ChecksummingTransformerTest.java:87)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15637) CqlInputFormat regression going from 2.1 to 3.x caused by semantic difference between thrift and the new system.size_estimates table when dealing with multiple dc de
[
https://issues.apache.org/jira/browse/CASSANDRA-15637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-15637:
--
Description:
In 3.0 CqlInputFormat switched away from thrift in favor of a new
system.size_estimates table, but the semantics changed when dealing with
multiple DCs or when Cassandra is not collocated with Hadoop.
The core issues are:
* system.size_estimates uses the primary range, in a multi-dc setup this could
lead to uneven ranges
example:
{code}
DC1: [0, 10, 20, 30]
DC2: [1, 11, 21, 31]
DC3: [2, 12, 22, 32]
{code}
Using NetworkTopologyStrategy the primary ranges are: [0, 1), [1, 2), [2, 10),
[10, 11), [11, 12), [12, 20), [20, 21), [21, 22), [22, 30), [30, 31), [31, 32),
[32, 0).
Given this the only ranges that are more than one token are: [2, 10), [12, 20),
[22, 30).
* system.size_estimates is not replicated so need to hit every node in the
cluster to get estimates, if nodes are down in the DC with non-size-1 ranges
there is no way to get a estimate.
* CqlInputFormat used to call describe_local_ring so all interactions were with
a single DC, the java driver doesn't filter the DC so looks to allow cross DC
traffic and includes nodes from other DCs in the replica set; in the example
above, the amount of splits went from 4 to 12.
* CqlInputFormat used to call describe_splits_ex to dynamically calculate the
estimates, this was on the "local primary range" and was able to hit replicas
to create estimates if the primary was down. With system.size_estimates we no
longer have backup and no longer expose the "local primary range" in multi-dc.
* CqlInputFormat special cases Cassandra being collocated with Hadoop and
assumes this when querying system.size_estimates as it doesn't filter to the
specific host, this means that non-collocated deployments randomly select the
nodes and create splits with ranges the hosts do not have locally.
The problems are deterministic to replicate, the following test will show it
1) deploy a 3 DC cluster with 3 nodes each
2) create DC2 tokens are +1 of DC1 and DC3 are +1 of DC2
3) CREATE KEYSPACE simpleuniform0 WITH replication = {‘class’:
‘NetworkTopologyStrategy’, ‘DC1’: 3, ‘DC2’: 3, ‘DC3’: 3};
4) CREATE TABLE simpletable0 (pk bigint, ck bigint, value blob, PRIMARY KEY
(pk, ck))
5) insert 500k partitions uniformly: [0, 500,000)
6) wait until estimates catch up to writes
7) for all nodes, SELECT * FROM system.size_estimates
You will get the following
{code}
keyspace_name | table_name | range_start | range_end|
mean_partition_size | partitions_count
+--+--+--+-+--
simpleuniform0 | simpletable0 | -9223372036854775808 | -6148914691236517206 |
87 | 122240
simpleuniform0 | simpletable0 | 6148914691236517207 | -9223372036854775808 |
87 | 121472
(2 rows)
keyspace_name | table_name | range_start | range_end |
mean_partition_size | partitions_count
+--+-+-+-+--
simpleuniform0 | simpletable0 | 2 | 6148914691236517205 |
87 | 243072
(1 rows)
keyspace_name | table_name | range_start | range_end|
mean_partition_size | partitions_count
+--+--+--+-+--
simpleuniform0 | simpletable0 | -6148914691236517206 | -6148914691236517205 |
87 |1
(1 rows)
keyspace_name | table_name | range_start | range_end | mean_partition_size
| partitions_count
+--+-+---+-+--
simpleuniform0 | simpletable0 | 0 | 1 | 87
|1
(1 rows)
keyspace_name | table_name | range_start | range_end |
mean_partition_size | partitions_count
+--+-+-+-+--
simpleuniform0 | simpletable0 | 6148914691236517205 | 6148914691236517206 |
87 |1
(1 rows)
keyspace_name | table_name | range_start | range_end|
mean_partition_size | partitions_count
+--+--+--+-+--
simpleuniform0 | simpletable0 | -6148914691236517205 | -6148914691236517204 |
87 |1
(1 rows)
keyspace_name | table_name | range_start | range_end | mean_partition_size
| partitions_count
+--+-+---+-+--
[jira] [Commented] (CASSANDRA-15637) CqlInputFormat regression going from 2.1 to 3.x caused by semantic difference between thrift and the new system.size_estimates table when dealing with multiple dc
[
https://issues.apache.org/jira/browse/CASSANDRA-15637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057354#comment-17057354
]
David Capwell commented on CASSANDRA-15637:
---
I have a patch which switches the estimate to the "local primary range" and
updated the client to work with this. The patch passes my test above but there
is concern that this may regress others, so not sure if the patch is
appropriate or not.
> CqlInputFormat regression going from 2.1 to 3.x caused by semantic difference
> between thrift and the new system.size_estimates table when dealing with
> multiple dc deployments
> --
>
> Key: CASSANDRA-15637
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15637
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Tools
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
>
> In 3.0 CqlInputFormat switched away from thrift in favor of a new
> system.size_estimates table, but the semantics changed when dealing with
> multiple DCs or when Cassandra is not collocated with Hadoop.
> The core issues are:
> * system.size_estimates uses the primary range, in a multi-dc setup this
> could lead to uneven ranges
> example:
> DC1: [0, 10, 20, 30]
> DC2: [1, 11, 21, 31]
> DC3: [2, 12, 22, 32]
> Using NetworkTopologyStrategy the primary ranges are: [0, 1), [1, 2), [2,
> 10), [10, 11), [11, 12), [12, 20), [20, 21), [21, 22), [22, 30), [30, 31),
> [31, 32), [32, 0).
> Given this the only ranges that are more than one token are: [2, 10), [12,
> 20), [22, 30).
> * system.size_estimates is not replicated so need to hit every node in the
> cluster to get estimates, if nodes are down in the DC with non-size-1 ranges
> there is no way to get a estimate.
> * CqlInputFormat used to call describe_local_ring so all interactions were
> with a single DC, the java driver doesn't filter the DC so looks to allow
> cross DC traffic and includes nodes from other DCs in the replica set; in the
> example above, the amount of splits went from 4 to 12.
> * CqlInputFormat used to call describe_splits_ex to dynamically calculate the
> estimates, this was on the "local primary range" and was able to hit replicas
> to create estimates if the primary was down. With system.size_estimates we no
> longer have backup and no longer expose the "local primary range" in multi-dc.
> * CqlInputFormat special cases Cassandra being collocated with Hadoop and
> assumes this when querying system.size_estimates as it doesn't filter to the
> specific host, this means that non-collocated deployments randomly select the
> nodes and create splits with ranges the hosts do not have locally.
> The problems are deterministic to replicate, the following test will show it
> 1) deploy a 3 DC cluster with 3 nodes each
> 2) create DC2 tokens are +1 of DC1 and DC3 are +1 of DC2
> 3) CREATE KEYSPACE simpleuniform0 WITH replication = {‘class’:
> ‘NetworkTopologyStrategy’, ‘DC1’: 3, ‘DC2’: 3, ‘DC3’: 3};
> 4) CREATE TABLE simpletable0 (pk bigint, ck bigint, value blob, PRIMARY KEY
> (pk, ck))
> 5) insert 500k partitions uniformly: [0, 500,000)
> 6) wait until estimates catch up to writes
> 7) for all nodes, SELECT * FROM system.size_estimates
> You will get the following
> keyspace_name | table_name | range_start | range_end
> | mean_partition_size | partitions_count
> +--+--+--+-+--
> simpleuniform0 | simpletable0 | -9223372036854775808 | -6148914691236517206
> | 87 | 122240
> simpleuniform0 | simpletable0 | 6148914691236517207 | -9223372036854775808
> | 87 | 121472
> (2 rows)
> keyspace_name | table_name | range_start | range_end |
> mean_partition_size | partitions_count
> +--+-+-+-+--
> simpleuniform0 | simpletable0 | 2 | 6148914691236517205 |
> 87 | 243072
> (1 rows)
> keyspace_name | table_name | range_start | range_end
> | mean_partition_size | partitions_count
> +--+--+--+-+--
> simpleuniform0 | simpletable0 | -6148914691236517206 | -6148914691236517205
> | 87 |1
> (1 rows)
> keyspace_name | table_name | range_start | range_end |
> mean_partition_size | partitions_count
>
[jira] [Updated] (CASSANDRA-15637) CqlInputFormat regression going from 2.1 to 3.x caused by semantic difference between thrift and the new system.size_estimates table when dealing with multiple dc de
[
https://issues.apache.org/jira/browse/CASSANDRA-15637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-15637:
--
Bug Category: Parent values: Correctness(12982)Level 1 values: API /
Semantic Implementation(12988)
Complexity: Challenging
Discovered By: User Report
Severity: Critical
Status: Open (was: Triage Needed)
Marked this as "challenging" as getting this to work for people upgrading from
2.1 and those already integrated with 3.0 (presto came up) is the issue.
> CqlInputFormat regression going from 2.1 to 3.x caused by semantic difference
> between thrift and the new system.size_estimates table when dealing with
> multiple dc deployments
> --
>
> Key: CASSANDRA-15637
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15637
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Tools
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
>
> In 3.0 CqlInputFormat switched away from thrift in favor of a new
> system.size_estimates table, but the semantics changed when dealing with
> multiple DCs or when Cassandra is not collocated with Hadoop.
> The core issues are:
> * system.size_estimates uses the primary range, in a multi-dc setup this
> could lead to uneven ranges
> example:
> DC1: [0, 10, 20, 30]
> DC2: [1, 11, 21, 31]
> DC3: [2, 12, 22, 32]
> Using NetworkTopologyStrategy the primary ranges are: [0, 1), [1, 2), [2,
> 10), [10, 11), [11, 12), [12, 20), [20, 21), [21, 22), [22, 30), [30, 31),
> [31, 32), [32, 0).
> Given this the only ranges that are more than one token are: [2, 10), [12,
> 20), [22, 30).
> * system.size_estimates is not replicated so need to hit every node in the
> cluster to get estimates, if nodes are down in the DC with non-size-1 ranges
> there is no way to get a estimate.
> * CqlInputFormat used to call describe_local_ring so all interactions were
> with a single DC, the java driver doesn't filter the DC so looks to allow
> cross DC traffic and includes nodes from other DCs in the replica set; in the
> example above, the amount of splits went from 4 to 12.
> * CqlInputFormat used to call describe_splits_ex to dynamically calculate the
> estimates, this was on the "local primary range" and was able to hit replicas
> to create estimates if the primary was down. With system.size_estimates we no
> longer have backup and no longer expose the "local primary range" in multi-dc.
> * CqlInputFormat special cases Cassandra being collocated with Hadoop and
> assumes this when querying system.size_estimates as it doesn't filter to the
> specific host, this means that non-collocated deployments randomly select the
> nodes and create splits with ranges the hosts do not have locally.
> The problems are deterministic to replicate, the following test will show it
> 1) deploy a 3 DC cluster with 3 nodes each
> 2) create DC2 tokens are +1 of DC1 and DC3 are +1 of DC2
> 3) CREATE KEYSPACE simpleuniform0 WITH replication = {‘class’:
> ‘NetworkTopologyStrategy’, ‘DC1’: 3, ‘DC2’: 3, ‘DC3’: 3};
> 4) CREATE TABLE simpletable0 (pk bigint, ck bigint, value blob, PRIMARY KEY
> (pk, ck))
> 5) insert 500k partitions uniformly: [0, 500,000)
> 6) wait until estimates catch up to writes
> 7) for all nodes, SELECT * FROM system.size_estimates
> You will get the following
> keyspace_name | table_name | range_start | range_end
> | mean_partition_size | partitions_count
> +--+--+--+-+--
> simpleuniform0 | simpletable0 | -9223372036854775808 | -6148914691236517206
> | 87 | 122240
> simpleuniform0 | simpletable0 | 6148914691236517207 | -9223372036854775808
> | 87 | 121472
> (2 rows)
> keyspace_name | table_name | range_start | range_end |
> mean_partition_size | partitions_count
> +--+-+-+-+--
> simpleuniform0 | simpletable0 | 2 | 6148914691236517205 |
> 87 | 243072
> (1 rows)
> keyspace_name | table_name | range_start | range_end
> | mean_partition_size | partitions_count
> +--+--+--+-+--
> simpleuniform0 | simpletable0 | -6148914691236517206 | -6148914691236517205
> | 87 |1
> (1 rows)
> keyspace_name | table_name | range_start | range_end |
> mean_partition_size |
[jira] [Created] (CASSANDRA-15637) CqlInputFormat regression going from 2.1 to 3.x caused by semantic difference between thrift and the new system.size_estimates table when dealing with multiple dc de
David Capwell created CASSANDRA-15637:
-
Summary: CqlInputFormat regression going from 2.1 to 3.x caused by
semantic difference between thrift and the new system.size_estimates table when
dealing with multiple dc deployments
Key: CASSANDRA-15637
URL: https://issues.apache.org/jira/browse/CASSANDRA-15637
Project: Cassandra
Issue Type: Bug
Components: Legacy/Tools
Reporter: David Capwell
Assignee: David Capwell
In 3.0 CqlInputFormat switched away from thrift in favor of a new
system.size_estimates table, but the semantics changed when dealing with
multiple DCs or when Cassandra is not collocated with Hadoop.
The core issues are:
* system.size_estimates uses the primary range, in a multi-dc setup this could
lead to uneven ranges
example:
DC1: [0, 10, 20, 30]
DC2: [1, 11, 21, 31]
DC3: [2, 12, 22, 32]
Using NetworkTopologyStrategy the primary ranges are: [0, 1), [1, 2), [2, 10),
[10, 11), [11, 12), [12, 20), [20, 21), [21, 22), [22, 30), [30, 31), [31, 32),
[32, 0).
Given this the only ranges that are more than one token are: [2, 10), [12, 20),
[22, 30).
* system.size_estimates is not replicated so need to hit every node in the
cluster to get estimates, if nodes are down in the DC with non-size-1 ranges
there is no way to get a estimate.
* CqlInputFormat used to call describe_local_ring so all interactions were with
a single DC, the java driver doesn't filter the DC so looks to allow cross DC
traffic and includes nodes from other DCs in the replica set; in the example
above, the amount of splits went from 4 to 12.
* CqlInputFormat used to call describe_splits_ex to dynamically calculate the
estimates, this was on the "local primary range" and was able to hit replicas
to create estimates if the primary was down. With system.size_estimates we no
longer have backup and no longer expose the "local primary range" in multi-dc.
* CqlInputFormat special cases Cassandra being collocated with Hadoop and
assumes this when querying system.size_estimates as it doesn't filter to the
specific host, this means that non-collocated deployments randomly select the
nodes and create splits with ranges the hosts do not have locally.
The problems are deterministic to replicate, the following test will show it
1) deploy a 3 DC cluster with 3 nodes each
2) create DC2 tokens are +1 of DC1 and DC3 are +1 of DC2
3) CREATE KEYSPACE simpleuniform0 WITH replication = {‘class’:
‘NetworkTopologyStrategy’, ‘DC1’: 3, ‘DC2’: 3, ‘DC3’: 3};
4) CREATE TABLE simpletable0 (pk bigint, ck bigint, value blob, PRIMARY KEY
(pk, ck))
5) insert 500k partitions uniformly: [0, 500,000)
6) wait until estimates catch up to writes
7) for all nodes, SELECT * FROM system.size_estimates
You will get the following
keyspace_name | table_name | range_start | range_end|
mean_partition_size | partitions_count
+--+--+--+-+--
simpleuniform0 | simpletable0 | -9223372036854775808 | -6148914691236517206 |
87 | 122240
simpleuniform0 | simpletable0 | 6148914691236517207 | -9223372036854775808 |
87 | 121472
(2 rows)
keyspace_name | table_name | range_start | range_end |
mean_partition_size | partitions_count
+--+-+-+-+--
simpleuniform0 | simpletable0 | 2 | 6148914691236517205 |
87 | 243072
(1 rows)
keyspace_name | table_name | range_start | range_end|
mean_partition_size | partitions_count
+--+--+--+-+--
simpleuniform0 | simpletable0 | -6148914691236517206 | -6148914691236517205 |
87 |1
(1 rows)
keyspace_name | table_name | range_start | range_end | mean_partition_size
| partitions_count
+--+-+---+-+--
simpleuniform0 | simpletable0 | 0 | 1 | 87
|1
(1 rows)
keyspace_name | table_name | range_start | range_end |
mean_partition_size | partitions_count
+--+-+-+-+--
simpleuniform0 | simpletable0 | 6148914691236517205 | 6148914691236517206 |
87 |1
(1 rows)
keyspace_name | table_name | range_start | range_end|
mean_partition_size | partitions_count
+--+--+--+-+--
simpleuniform0 |
[jira] [Updated] (CASSANDRA-15388) Add compaction allocation measurement test to support compaction gc optimization.
[
https://issues.apache.org/jira/browse/CASSANDRA-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Blake Eggleston updated CASSANDRA-15388:
Source Control Link:
https://github.com/apache/cassandra/commit/c9e40277f0389a773b21d4310a85a8c761932d61
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
thanks, committed to trunk as
[c9e40277f0389a773b21d4310a85a8c761932d61|https://github.com/apache/cassandra/commit/c9e40277f0389a773b21d4310a85a8c761932d61]
> Add compaction allocation measurement test to support compaction gc
> optimization.
> --
>
> Key: CASSANDRA-15388
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15388
> Project: Cassandra
> Issue Type: Sub-task
> Components: Local/Compaction
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 4.0
>
>
> This adds a test that is able to quickly and accurately measure the effect of
> potential gc optimizations against a wide range of (synthetic) compaction
> workloads. This test accurately measures allocation rates from 16 workloads
> in less that 2 minutes.
> This test uses google’s {{java-allocation-instrumenter}} agent to measure the
> workloads. Measurements using this agent are very accurate and pretty
> repeatable from run to run, with most variance being negligible (1-2 bytes
> per partition), although workloads with larger but fewer partitions vary a
> bit more (still less that 0.03%).
> The thinking behind this patch is that with compaction, we’re generally
> interested in the memory allocated per partition, since garbage scales more
> or less linearly with the number of partitions compacted. So measuring
> allocation from a small number of partitions that otherwise represent real
> world use cases is a good enough approximation.
> In addition to helping with compaction optimizations, this test could be used
> as a template for future optimization work. This pattern could also be used
> to set allocation limits on workloads/operations and fail CI if the
> allocation behavior changes past some threshold.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Add compaction allocation measurement test
This is an automated email from the ASF dual-hosted git repository.
bdeggleston pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new c9e4027 Add compaction allocation measurement test
c9e4027 is described below
commit c9e40277f0389a773b21d4310a85a8c761932d61
Author: Blake Eggleston
AuthorDate: Tue Oct 15 13:04:26 2019 -0700
Add compaction allocation measurement test
Patch by Blake Eggleston, reviewed by Benedict Elliott Smith and David
Capwell for CASSANDRA-15388
---
CHANGES.txt| 1 +
build.xml | 13 +
ide/idea-iml-file.xml | 1 +
.../db/compaction/CompactionAllocationTest.java| 767 +
4 files changed, 782 insertions(+)
diff --git a/CHANGES.txt b/CHANGES.txt
index d863df2..c6a2eac 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0-alpha4
+ * Add compaction allocation measurement test (CASSANDRA-15388)
* Added UnleveledSSTables global and table level metric (CASSANDRA-15620)
* Added Virtual Table exposing Cassandra relevant system properties
(CASSANDRA-15616)
* Add data modeling introduction (CASSANDRA-15481)
diff --git a/build.xml b/build.xml
index 9c4f5b0..bd1f557 100644
--- a/build.xml
+++ b/build.xml
@@ -60,6 +60,7 @@
+
@@ -96,6 +97,7 @@
+
@@ -118,6 +120,7 @@
+
@@ -539,6 +542,7 @@
+
@@ -672,6 +676,7 @@
version="${version}"/>
+
@@ -1324,6 +1329,7 @@
+
@@ -1620,6 +1626,13 @@
+
+
+
+
+
+
diff --git a/ide/idea-iml-file.xml b/ide/idea-iml-file.xml
index 17d4c5b..86827c3 100644
--- a/ide/idea-iml-file.xml
+++ b/ide/idea-iml-file.xml
@@ -32,6 +32,7 @@
+
diff --git
a/test/memory/org/apache/cassandra/db/compaction/CompactionAllocationTest.java
b/test/memory/org/apache/cassandra/db/compaction/CompactionAllocationTest.java
new file mode 100644
index 000..a58303d
--- /dev/null
+++
b/test/memory/org/apache/cassandra/db/compaction/CompactionAllocationTest.java
@@ -0,0 +1,767 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.db.compaction;
+
+import java.io.IOException;
+import java.lang.management.ManagementFactory;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Random;
+import java.util.Set;
+
+import com.google.common.base.Joiner;
+import com.google.common.collect.Iterables;
+import com.google.common.collect.Lists;
+import com.google.common.primitives.Ints;
+import org.junit.AfterClass;
+import org.junit.Assert;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.monitoring.runtime.instrumentation.AllocationRecorder;
+import com.google.monitoring.runtime.instrumentation.Sampler;
+import com.sun.management.ThreadMXBean;
+import org.apache.cassandra.OrderedJUnit4ClassRunner;
+import org.apache.cassandra.SchemaLoader;
+import org.apache.cassandra.cql3.QueryOptions;
+import org.apache.cassandra.cql3.QueryProcessor;
+import org.apache.cassandra.cql3.statements.SelectStatement;
+import org.apache.cassandra.cql3.statements.schema.CreateTableStatement;
+import org.apache.cassandra.db.ColumnFamilyStore;
+import org.apache.cassandra.db.ReadExecutionController;
+import org.apache.cassandra.db.ReadQuery;
+import org.apache.cassandra.db.filter.ColumnFilter;
+import org.apache.cassandra.db.partitions.UnfilteredPartitionIterator;
+import org.apache.cassandra.db.partitions.UnfilteredPartitionIterators;
+import
[jira] [Updated] (CASSANDRA-15388) Add compaction allocation measurement test to support compaction gc optimization.
[
https://issues.apache.org/jira/browse/CASSANDRA-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Blake Eggleston updated CASSANDRA-15388:
Status: Ready to Commit (was: Changes Suggested)
> Add compaction allocation measurement test to support compaction gc
> optimization.
> --
>
> Key: CASSANDRA-15388
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15388
> Project: Cassandra
> Issue Type: Sub-task
> Components: Local/Compaction
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 4.0
>
>
> This adds a test that is able to quickly and accurately measure the effect of
> potential gc optimizations against a wide range of (synthetic) compaction
> workloads. This test accurately measures allocation rates from 16 workloads
> in less that 2 minutes.
> This test uses google’s {{java-allocation-instrumenter}} agent to measure the
> workloads. Measurements using this agent are very accurate and pretty
> repeatable from run to run, with most variance being negligible (1-2 bytes
> per partition), although workloads with larger but fewer partitions vary a
> bit more (still less that 0.03%).
> The thinking behind this patch is that with compaction, we’re generally
> interested in the memory allocated per partition, since garbage scales more
> or less linearly with the number of partitions compacted. So measuring
> allocation from a small number of partitions that otherwise represent real
> world use cases is a good enough approximation.
> In addition to helping with compaction optimizations, this test could be used
> as a template for future optimization work. This pattern could also be used
> to set allocation limits on workloads/operations and fail CI if the
> allocation behavior changes past some threshold.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15262) server_encryption_options is not backwards compatible with 3.11
[
https://issues.apache.org/jira/browse/CASSANDRA-15262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057290#comment-17057290
]
Adam Holmberg commented on CASSANDRA-15262:
---
[~jolynch] you mentioned having WIP that you might be able to hand off to
someone. Will you be able to push that soon?
> server_encryption_options is not backwards compatible with 3.11
> ---
>
> Key: CASSANDRA-15262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15262
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Joey Lynch
>Assignee: Joey Lynch
>Priority: Normal
> Fix For: 4.0, 4.0-alpha
>
>
> The current `server_encryption_options` configuration options are as follows:
> {noformat}
> server_encryption_options:
> # set to true for allowing secure incoming connections
> enabled: false
> # If enabled and optional are both set to true, encrypted and unencrypted
> connections are handled on the storage_port
> optional: false
> # if enabled, will open up an encrypted listening socket on
> ssl_storage_port. Should be used
> # during upgrade to 4.0; otherwise, set to false.
> enable_legacy_ssl_storage_port: false
> # on outbound connections, determine which type of peers to securely
> connect to. 'enabled' must be set to true.
> internode_encryption: none
> keystore: conf/.keystore
> keystore_password: cassandra
> truststore: conf/.truststore
> truststore_password: cassandra
> # More advanced defaults below:
> # protocol: TLS
> # store_type: JKS
> # cipher_suites:
> [TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA]
> # require_client_auth: false
> # require_endpoint_verification: false
> {noformat}
> A couple of issues here:
> 1. optional defaults to false, which will break existing TLS configurations
> for (from what I can tell) no particularly good reason
> 2. The provided protocol and cipher suites are not good ideas (in particular
> encouraging anyone to use CBC ciphers is a bad plan
> I propose that before the 4.0 cut we fixup server_encryption_options and even
> client_encryption_options :
> # Change the default {{optional}} setting to true. As the new Netty code
> intelligently decides to open a TLS connection or not this is the more
> sensible default (saves operators a step while transitioning to TLS as well)
> # Update the defaults to what netty actually defaults to
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15314) Fix failing test - test_rolling_upgrade_with_internode_ssl - upgrade_tests.upgrade_through_versions_test.TestProtoV4Upgrade_AllVersions_EndsAt_Trunk_HEAD
[
https://issues.apache.org/jira/browse/CASSANDRA-15314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-15314:
--
Bug Category: Parent values: Correctness(12982)Level 1 values: Test
Failure(12990)
Complexity: Normal
Discovered By: Unit Test
Severity: Normal
Status: Open (was: Triage Needed)
> Fix failing test - test_rolling_upgrade_with_internode_ssl -
> upgrade_tests.upgrade_through_versions_test.TestProtoV4Upgrade_AllVersions_EndsAt_Trunk_HEAD
> -
>
> Key: CASSANDRA-15314
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15314
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Vinay Chella
>Assignee: Brandon Williams
>Priority: Normal
> Labels: dtest
> Fix For: 4.0-alpha
>
>
> Example failure:
> [https://circleci.com/gh/vinaykumarchella/cassandra/468#tests/containers/11]
>
> {code:java}
> ccmlib.node.TimeoutError: 06 Sep 2019 20:23:57 [node2] Missing: ['127.0.0.1.*
> now UP']: INFO [HANDSHAKE-/127.0.0.1] 2019-09-06 20:20:01,7. See
> system.log for remainder
> self =
> object at 0x7f6d90d43b38>
> @pytest.mark.timeout(3000)
> def test_rolling_upgrade_with_internode_ssl(self):
> """
> Rolling upgrade test using internode ssl.
> """
> > self.upgrade_scenario(rolling=True, internode_ssl=True)
> upgrade_tests/upgrade_through_versions_test.py:296:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> upgrade_tests/upgrade_through_versions_test.py:352: in upgrade_scenario
> self.upgrade_to_version(version_meta, partial=True, nodes=(node,),
> internode_ssl=internode_ssl)
> upgrade_tests/upgrade_through_versions_test.py:456: in upgrade_to_version
> node.start(wait_other_notice=240, wait_for_binary_proto=True)
> ../env/src/ccm/ccmlib/node.py:751: in start
> node.watch_log_for_alive(self, from_mark=mark, timeout=wait_other_notice)
> ../env/src/ccm/ccmlib/node.py:568: in watch_log_for_alive
> self.watch_log_for(tofind, from_mark=from_mark, timeout=timeout,
> filename=filename)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> self =
> exprs = ['127.0.0.1.* now UP'], from_mark = 150742, timeout = 240
> process = None, verbose = False, filename = 'system.log'
> def watch_log_for(self, exprs, from_mark=None, timeout=600, process=None,
> verbose=False, filename='system.log'):
> """
> Watch the log until one or more (regular) expression are found.
> This methods when all the expressions have been found or the
> method
> timeouts (a TimeoutError is then raised). On successful
> completion,
> a list of pair (line matched, match object) is returned.
> """
> start = time.time()
> tofind = [exprs] if isinstance(exprs, string_types) else exprs
> tofind = [re.compile(e) for e in tofind]
> matchings = []
> reads = ""
> if len(tofind) == 0:
> return None
>
> log_file = os.path.join(self.get_path(), 'logs', filename)
> output_read = False
> while not os.path.exists(log_file):
> time.sleep(.5)
> if start + timeout < time.time():
> raise TimeoutError(time.strftime("%d %b %Y %H:%M:%S",
> time.gmtime()) + " [" + self.name + "] Timed out waiting for {} to be
> created.".format(log_file))
> if process and not output_read:
> process.poll()
> if process.returncode is not None:
> self.print_process_output(self.name, process, verbose)
> output_read = True
> if process.returncode != 0:
> raise RuntimeError() # Shouldn't reuse RuntimeError
> but I'm lazy
>
> with open(log_file) as f:
> if from_mark:
> f.seek(from_mark)
>
> while True:
> # First, if we have a process to check, then check it.
> # Skip on Windows - stdout/stderr is cassandra.bat
> if not common.is_win() and not output_read:
> if process:
> process.poll()
> if process.returncode is not None:
> self.print_process_output(self.name, process,
> verbose)
> output_read = True
> if process.returncode != 0:
> raise RuntimeError() # Shouldn't reuse
> RuntimeError but I'm lazy
>
>
[jira] [Updated] (CASSANDRA-15315) Fix failing test - test_rolling_upgrade_with_internode_ssl - upgrade_tests.upgrade_through_versions_test.TestProtoV4Upgrade_AllVersions_RandomPartitioner_EndsAt_Trun
[
https://issues.apache.org/jira/browse/CASSANDRA-15315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-15315:
--
Bug Category: Parent values: Correctness(12982)Level 1 values: Test
Failure(12990)
Complexity: Normal
Discovered By: Unit Test
Severity: Normal
Status: Open (was: Triage Needed)
> Fix failing test - test_rolling_upgrade_with_internode_ssl -
> upgrade_tests.upgrade_through_versions_test.TestProtoV4Upgrade_AllVersions_RandomPartitioner_EndsAt_Trunk_HEAD
> ---
>
> Key: CASSANDRA-15315
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15315
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Vinay Chella
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> Example failure:
> [https://circleci.com/gh/vinaykumarchella/cassandra/468#tests/containers/11]
> [https://circleci.com/gh/vinaykumarchella/cassandra/451#tests/containers/11]
> {code:java}
> ccmlib.node.TimeoutError: 06 Sep 2019 20:21:39 [node2] Missing: ['127.0.0.1.*
> now UP']: INFO [HANDSHAKE-/127.0.0.1] 2019-09-06 20:17:43,8. See
> system.log for remainder
> self =
> object at 0x7fbb75245a90>
> @pytest.mark.timeout(3000)
> def test_rolling_upgrade_with_internode_ssl(self):
> """
> Rolling upgrade test using internode ssl.
> """
> > self.upgrade_scenario(rolling=True, internode_ssl=True)
> upgrade_tests/upgrade_through_versions_test.py:296:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> upgrade_tests/upgrade_through_versions_test.py:352: in upgrade_scenario
> self.upgrade_to_version(version_meta, partial=True, nodes=(node,),
> internode_ssl=internode_ssl)
> upgrade_tests/upgrade_through_versions_test.py:456: in upgrade_to_version
> node.start(wait_other_notice=240, wait_for_binary_proto=True)
> ../env/src/ccm/ccmlib/node.py:751: in start
> node.watch_log_for_alive(self, from_mark=mark, timeout=wait_other_notice)
> ../env/src/ccm/ccmlib/node.py:568: in watch_log_for_alive
> self.watch_log_for(tofind, from_mark=from_mark, timeout=timeout,
> filename=filename)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> self =
> exprs = ['127.0.0.1.* now UP'], from_mark = 151813, timeout = 240
> process = None, verbose = False, filename = 'system.log'
> def watch_log_for(self, exprs, from_mark=None, timeout=600, process=None,
> verbose=False, filename='system.log'):
> """
> Watch the log until one or more (regular) expression are found.
> This methods when all the expressions have been found or the
> method
> timeouts (a TimeoutError is then raised). On successful
> completion,
> a list of pair (line matched, match object) is returned.
> """
> start = time.time()
> tofind = [exprs] if isinstance(exprs, string_types) else exprs
> tofind = [re.compile(e) for e in tofind]
> matchings = []
> reads = ""
> if len(tofind) == 0:
> return None
>
> log_file = os.path.join(self.get_path(), 'logs', filename)
> output_read = False
> while not os.path.exists(log_file):
> time.sleep(.5)
> if start + timeout < time.time():
> raise TimeoutError(time.strftime("%d %b %Y %H:%M:%S",
> time.gmtime()) + " [" + self.name + "] Timed out waiting for {} to be
> created.".format(log_file))
> if process and not output_read:
> process.poll()
> if process.returncode is not None:
> self.print_process_output(self.name, process, verbose)
> output_read = True
> if process.returncode != 0:
> raise RuntimeError() # Shouldn't reuse RuntimeError
> but I'm lazy
>
> with open(log_file) as f:
> if from_mark:
> f.seek(from_mark)
>
> while True:
> # First, if we have a process to check, then check it.
> # Skip on Windows - stdout/stderr is cassandra.bat
> if not common.is_win() and not output_read:
> if process:
> process.poll()
> if process.returncode is not None:
> self.print_process_output(self.name, process,
> verbose)
> output_read = True
> if process.returncode != 0:
>
[jira] [Created] (CASSANDRA-15636) Refactor protocol related code to eliminate magic numbers
Dinesh Joshi created CASSANDRA-15636: Summary: Refactor protocol related code to eliminate magic numbers Key: CASSANDRA-15636 URL: https://issues.apache.org/jira/browse/CASSANDRA-15636 Project: Cassandra Issue Type: Improvement Components: Messaging/Client Reporter: Dinesh Joshi Parts of Cassandra's codebase has protocol related magic numbers. It makes the code hard to understand. We should refactor the code to remove magic numbers and instead replace them with versioned constants. This will help make the code clearer. Example: https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/transport/messages/ErrorMessage.java#L288 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15605) Broken dtest replication_test.py::TestSnitchConfigurationUpdate
[
https://issues.apache.org/jira/browse/CASSANDRA-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057238#comment-17057238
]
Ryan Svihla edited comment on CASSANDRA-15605 at 3/11/20, 5:25 PM:
---
Looks like the racks were assumed to be in the same order, adding a sort gets
the tests to pass. I'm not sure why the order is changing sometimes, but in all
cases the topology had changed and the comparison was just missing that fact.
fails
{{Status=Up/Down}}
{{|/ State=Normal/Leaving/Joining/Moving}}
{{-- Address Load Tokens Owns (effective) Host ID Rack}}
{{UN 127.0.0.3 105.28 KiB 1 ? 41df89ae-0631-447c-9def-ff1fbff3b8a7 rack2}}
{{UN 127.0.0.2 164.25 KiB 1 ? 47a49e4e-6964-4b00-b353-d3ab345f4aba rack1}}
{{UN 127.0.0.1 128.46 KiB 1 ? b63ad12d-1d12-4e51-95ba-88ff61b252ea rack0}}
works
{\{16:05:35,630 replication_test DEBUG Datacenter: dc1 }}
{{===}}
{{Status=Up/Down}}
{{|/ State=Normal/Leaving/Joining/Moving}}
{{-- Address Load Tokens Owns (effective) Host ID Rack}}
{{UN 127.0.0.1 132.51 KiB 1 100.0% 8b244cb0-135d-4745-b498-86f46187348f rack0}}
{{UN 127.0.0.2 164.36 KiB 1 100.0% 36f42f34-fe8c-4543-b6c9-6db12b93b660 rack1}}
{{UN 127.0.0.3 192.66 KiB 1 100.0% 1224e768-db1f-4d56-a1c8-db9b764a46d5 rack2}}
[pr|https://github.com/apache/cassandra-dtest/pull/58]
was (Author: rssvihla):
https://github.com/apache/cassandra-dtest/pull/58
> Broken dtest replication_test.py::TestSnitchConfigurationUpdate
> ---
>
> Key: CASSANDRA-15605
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15605
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Sam Tunnicliffe
>Assignee: Ryan Svihla
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> Noticed this failing on a couple of CI runs and repros when running trunk
> locally and on CircleCI
> 2 or 3 tests are consistently failing:
> * {{test_rf_expand_gossiping_property_file_snitch}}
> * {{test_rf_expand_property_file_snitch}}
> * {{test_move_forwards_between_and_cleanup}}
> [https://circleci.com/workflow-run/f23f13a9-bbdc-4764-8336-109517e137f1]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Issue Comment Deleted] (CASSANDRA-15605) Broken dtest replication_test.py::TestSnitchConfigurationUpdate
[
https://issues.apache.org/jira/browse/CASSANDRA-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Svihla updated CASSANDRA-15605:
Comment: was deleted
(was: Looks like the racks were assumed to be in the same order, adding a sort
gets the tests to pass. I'm not sure why the order is changing sometimes, but
in all cases the topology had changed and the comparison was just missing that
fact.
fails
{{Status=Up/Down}}
{{|/ State=Normal/Leaving/Joining/Moving}}
{{-- Address Load Tokens Owns (effective) Host ID Rack}}
{{UN 127.0.0.3 105.28 KiB 1 ? 41df89ae-0631-447c-9def-ff1fbff3b8a7 rack2}}
{{UN 127.0.0.2 164.25 KiB 1 ? 47a49e4e-6964-4b00-b353-d3ab345f4aba rack1}}
{{UN 127.0.0.1 128.46 KiB 1 ? b63ad12d-1d12-4e51-95ba-88ff61b252ea rack0}}
works
{{16:05:35,630 replication_test DEBUG Datacenter: dc1 }}
{{===}}
{{Status=Up/Down}}
{{|/ State=Normal/Leaving/Joining/Moving}}
{{-- Address Load Tokens Owns (effective) Host ID Rack}}
{{UN 127.0.0.1 132.51 KiB 1 100.0% 8b244cb0-135d-4745-b498-86f46187348f rack0}}
{{UN 127.0.0.2 164.36 KiB 1 100.0% 36f42f34-fe8c-4543-b6c9-6db12b93b660 rack1}}
{{UN 127.0.0.3 192.66 KiB 1 100.0% 1224e768-db1f-4d56-a1c8-db9b764a46d5 rack2}}
[pr|https://github.com/apache/cassandra-dtest/pull/58])
> Broken dtest replication_test.py::TestSnitchConfigurationUpdate
> ---
>
> Key: CASSANDRA-15605
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15605
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Sam Tunnicliffe
>Assignee: Ryan Svihla
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> Noticed this failing on a couple of CI runs and repros when running trunk
> locally and on CircleCI
> 2 or 3 tests are consistently failing:
> * {{test_rf_expand_gossiping_property_file_snitch}}
> * {{test_rf_expand_property_file_snitch}}
> * {{test_move_forwards_between_and_cleanup}}
> [https://circleci.com/workflow-run/f23f13a9-bbdc-4764-8336-109517e137f1]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15605) Broken dtest replication_test.py::TestSnitchConfigurationUpdate
[
https://issues.apache.org/jira/browse/CASSANDRA-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Svihla updated CASSANDRA-15605:
Test and Documentation Plan: not needed
Status: Patch Available (was: In Progress)
https://github.com/apache/cassandra-dtest/pull/58
> Broken dtest replication_test.py::TestSnitchConfigurationUpdate
> ---
>
> Key: CASSANDRA-15605
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15605
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Sam Tunnicliffe
>Assignee: Ryan Svihla
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> Noticed this failing on a couple of CI runs and repros when running trunk
> locally and on CircleCI
> 2 or 3 tests are consistently failing:
> * {{test_rf_expand_gossiping_property_file_snitch}}
> * {{test_rf_expand_property_file_snitch}}
> * {{test_move_forwards_between_and_cleanup}}
> [https://circleci.com/workflow-run/f23f13a9-bbdc-4764-8336-109517e137f1]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15605) Broken dtest replication_test.py::TestSnitchConfigurationUpdate
[
https://issues.apache.org/jira/browse/CASSANDRA-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057232#comment-17057232
]
Ryan Svihla edited comment on CASSANDRA-15605 at 3/11/20, 5:17 PM:
---
Looks like the racks were assumed to be in the same order, adding a sort gets
the tests to pass. I'm not sure why the order is changing sometimes, but in all
cases the topology had changed and the comparison was just missing that fact.
fails
{{Status=Up/Down}}
{{|/ State=Normal/Leaving/Joining/Moving}}
{{-- Address Load Tokens Owns (effective) Host ID Rack}}
{{UN 127.0.0.3 105.28 KiB 1 ? 41df89ae-0631-447c-9def-ff1fbff3b8a7 rack2}}
{{UN 127.0.0.2 164.25 KiB 1 ? 47a49e4e-6964-4b00-b353-d3ab345f4aba rack1}}
{{UN 127.0.0.1 128.46 KiB 1 ? b63ad12d-1d12-4e51-95ba-88ff61b252ea rack0}}
works
{{16:05:35,630 replication_test DEBUG Datacenter: dc1 }}
{{===}}
{{Status=Up/Down}}
{{|/ State=Normal/Leaving/Joining/Moving}}
{{-- Address Load Tokens Owns (effective) Host ID Rack}}
{{UN 127.0.0.1 132.51 KiB 1 100.0% 8b244cb0-135d-4745-b498-86f46187348f rack0}}
{{UN 127.0.0.2 164.36 KiB 1 100.0% 36f42f34-fe8c-4543-b6c9-6db12b93b660 rack1}}
{{UN 127.0.0.3 192.66 KiB 1 100.0% 1224e768-db1f-4d56-a1c8-db9b764a46d5 rack2}}
[pr|https://github.com/apache/cassandra-dtest/pull/58]
was (Author: rssvihla):
Looks like the racks were assumed to be in the same order, adding a sort gets
the tests to pass. I'm not sure why the order is changing sometimes, but in all
cases the topology had changed and the comparison was just missing that fact:
[pr|https://github.com/apache/cassandra-dtest/pull/58]
> Broken dtest replication_test.py::TestSnitchConfigurationUpdate
> ---
>
> Key: CASSANDRA-15605
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15605
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Sam Tunnicliffe
>Assignee: Ryan Svihla
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> Noticed this failing on a couple of CI runs and repros when running trunk
> locally and on CircleCI
> 2 or 3 tests are consistently failing:
> * {{test_rf_expand_gossiping_property_file_snitch}}
> * {{test_rf_expand_property_file_snitch}}
> * {{test_move_forwards_between_and_cleanup}}
> [https://circleci.com/workflow-run/f23f13a9-bbdc-4764-8336-109517e137f1]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15635) Script to autogenerate cassandra.yaml
Dinesh Joshi created CASSANDRA-15635: Summary: Script to autogenerate cassandra.yaml Key: CASSANDRA-15635 URL: https://issues.apache.org/jira/browse/CASSANDRA-15635 Project: Cassandra Issue Type: Improvement Components: Local/Config, Local/Scripts Reporter: Dinesh Joshi It would be useful to have a script that can ask the user a few questions and generate a recommended cassandra.yaml based on their answers. This will help solve issues like selecting num_tokens. It can also be integrated into OS specific packaging tools such as debconf[1]. Rather than just documenting on the website, it is best to provide a simple script to auto-generate configuration based on common use-cases. [1] https://wiki.debian.org/debconf -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15605) Broken dtest replication_test.py::TestSnitchConfigurationUpdate
[
https://issues.apache.org/jira/browse/CASSANDRA-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057232#comment-17057232
]
Ryan Svihla commented on CASSANDRA-15605:
-
Looks like the racks were assumed to be in the same order, adding a sort gets
the tests to pass. I'm not sure why the order is changing sometimes, but in all
cases the topology had changed and the comparison was just missing that fact:
[pr|https://github.com/apache/cassandra-dtest/pull/58]
> Broken dtest replication_test.py::TestSnitchConfigurationUpdate
> ---
>
> Key: CASSANDRA-15605
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15605
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Sam Tunnicliffe
>Assignee: Ryan Svihla
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> Noticed this failing on a couple of CI runs and repros when running trunk
> locally and on CircleCI
> 2 or 3 tests are consistently failing:
> * {{test_rf_expand_gossiping_property_file_snitch}}
> * {{test_rf_expand_property_file_snitch}}
> * {{test_move_forwards_between_and_cleanup}}
> [https://circleci.com/workflow-run/f23f13a9-bbdc-4764-8336-109517e137f1]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15564) Refactor repair coordinator so errors are consistent
[ https://issues.apache.org/jira/browse/CASSANDRA-15564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057203#comment-17057203 ] Blake Eggleston commented on CASSANDRA-15564: - +1, response/service time thing seems reasonable > Refactor repair coordinator so errors are consistent > > > Key: CASSANDRA-15564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15564 > Project: Cassandra > Issue Type: Sub-task > Components: Consistency/Repair >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Time Spent: 17h > Remaining Estimate: 0h > > This is to split the change in CASSANDRA-15399 so the refactor is isolated > out. > Currently the repair coordinator special cases the exit cases at each call > site; this makes it so that errors can be inconsistent and there are cases > where proper complete isn't done (proper notifications, and forgetting to > update ActiveRepairService). > [Circle > CI|https://circleci.com/gh/dcapwell/cassandra/tree/bug%2FrepairCoordinatorJmxConsistency] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15633) Cassandra repair failing
[ https://issues.apache.org/jira/browse/CASSANDRA-15633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057196#comment-17057196 ] Brandon Williams commented on CASSANDRA-15633: -- bq. you need to do as it suggests, and check the server log. > Cassandra repair failing > - > > Key: CASSANDRA-15633 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15633 > Project: Cassandra > Issue Type: Task >Reporter: Chirantan >Priority: Normal > > [2020-03-10 18:09:51,533] Repair session 7be9f3c0-631e-11ea-8e47-ef25c77f7899 > for range [(6301399909211397355,6302095577836606571], > (6301291625775958686,6301399909211397355], > (6302095577836606571,6313970392023331777], > (6297705134651154643,6301291625775958686]] finished (progress: 44%) > Exception occurred during clean-up. > java.lang.reflect.UndeclaredThrowableException > Cassandra has shutdown. > error: [2020-03-10 18:09:56,868] JMX connection closed. You should check > server log for repair status of keyspace cortex(Subsequent keyspaces are not > going to be repaired). > -- StackTrace -- > java.io.IOException: [2020-03-10 18:09:56,868] JMX connection closed. You > should check server log for repair status of keyspace cortex(Subsequent > keyspaces are not going to be repaired). > at > org.apache.cassandra.tools.RepairRunner.handleConnectionFailed(RepairRunner.java:98) > at > org.apache.cassandra.utils.progress.jmx.JMXNotificationProgressListener.handleNotification(JMXNotificationProgressListener.java:86) > at > javax.management.NotificationBroadcasterSupport.handleNotification(NotificationBroadcasterSupport.java:286) > at > javax.management.NotificationBroadcasterSupport$SendNotifJob.run(NotificationBroadcasterSupport.java:363) > at > javax.management.NotificationBroadcasterSupport$1.execute(NotificationBroadcasterSupport.java:348) > at > javax.management.NotificationBroadcasterSupport.sendNotification(NotificationBroadcasterSupport.java:259) > at > javax.management.remote.rmi.RMIConnector.sendNotification(RMIConnector.java:452) > at > javax.management.remote.rmi.RMIConnector.access$1200(RMIConnector.java:132) > at > javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.gotIOException(RMIConnector.java:1542) > at > javax.management.remote.rmi.RMIConnector$RMINotifClient.fetchNotifs(RMIConnector.java:1363) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchOneNotif(ClientNotifForwarder.java:667) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchNotifs(ClientNotifForwarder.java:619) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.doRun(ClientNotifForwarder.java:483) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.run(ClientNotifForwarder.java:464) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$LinearExecutor$1.run(ClientNotifForwarder.java:120) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15622) Unit tests throw UnknownHostException trying to use `InetAddress.getLocalHost()` instead of `FBUtilities.getLocalAddress()`
[
https://issues.apache.org/jira/browse/CASSANDRA-15622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15622:
---
Fix Version/s: (was: 3.11.x)
(was: 4.x)
(was: 3.0.x)
(was: 2.2.x)
4.0-alpha
3.11.7
3.0.21
2.2.17
Since Version: 0.3
Source Control Link:
https://github.com/apache/cassandra/commit/a8ef9c9ad1495f3eb9dc1ef09c0995926561fd8c
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed as a8ef9c9ad1495f3eb9dc1ef09c0995926561fd8c
> Unit tests throw UnknownHostException trying to use
> `InetAddress.getLocalHost()` instead of `FBUtilities.getLocalAddress()`
> ---
>
> Key: CASSANDRA-15622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15622
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.17, 3.0.21, 3.11.7, 4.0-alpha
>
>
> Many of the unit tests in Jenkins fail because of the use of
> {{`InetAddress.getLocalHost()`}} in the test classes.
> The Jenkins agents need a public ipaddress (and a hostname associated to it)
> so the Jenkins master can connect to them (agents can be hosted externally,
> by donating third-parties).
> The call to {{`InetAddress.getLocalHost()`}} can resolve to a hostname that
> can't be looked up.
> Not only can it not be listed in {{`/etc/hosts`}}, but we don't want it to be
> either (in case of accidental external port opening if the hostname points to
> the public ipaddress). (Which is also ASF policy on this infrastructure.)
> The unit test code needs to replace these code occurrences with the call to
> {{`FBUtilities.getLocalAddress()`}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15633) Cassandra repair failing
[ https://issues.apache.org/jira/browse/CASSANDRA-15633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-15633: - Summary: Cassandra repair failing (was: Cassandra rebuild failing ) > Cassandra repair failing > - > > Key: CASSANDRA-15633 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15633 > Project: Cassandra > Issue Type: Task >Reporter: Chirantan >Priority: Normal > > [2020-03-10 18:09:51,533] Repair session 7be9f3c0-631e-11ea-8e47-ef25c77f7899 > for range [(6301399909211397355,6302095577836606571], > (6301291625775958686,6301399909211397355], > (6302095577836606571,6313970392023331777], > (6297705134651154643,6301291625775958686]] finished (progress: 44%) > Exception occurred during clean-up. > java.lang.reflect.UndeclaredThrowableException > Cassandra has shutdown. > error: [2020-03-10 18:09:56,868] JMX connection closed. You should check > server log for repair status of keyspace cortex(Subsequent keyspaces are not > going to be repaired). > -- StackTrace -- > java.io.IOException: [2020-03-10 18:09:56,868] JMX connection closed. You > should check server log for repair status of keyspace cortex(Subsequent > keyspaces are not going to be repaired). > at > org.apache.cassandra.tools.RepairRunner.handleConnectionFailed(RepairRunner.java:98) > at > org.apache.cassandra.utils.progress.jmx.JMXNotificationProgressListener.handleNotification(JMXNotificationProgressListener.java:86) > at > javax.management.NotificationBroadcasterSupport.handleNotification(NotificationBroadcasterSupport.java:286) > at > javax.management.NotificationBroadcasterSupport$SendNotifJob.run(NotificationBroadcasterSupport.java:363) > at > javax.management.NotificationBroadcasterSupport$1.execute(NotificationBroadcasterSupport.java:348) > at > javax.management.NotificationBroadcasterSupport.sendNotification(NotificationBroadcasterSupport.java:259) > at > javax.management.remote.rmi.RMIConnector.sendNotification(RMIConnector.java:452) > at > javax.management.remote.rmi.RMIConnector.access$1200(RMIConnector.java:132) > at > javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.gotIOException(RMIConnector.java:1542) > at > javax.management.remote.rmi.RMIConnector$RMINotifClient.fetchNotifs(RMIConnector.java:1363) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchOneNotif(ClientNotifForwarder.java:667) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchNotifs(ClientNotifForwarder.java:619) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.doRun(ClientNotifForwarder.java:483) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.run(ClientNotifForwarder.java:464) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$LinearExecutor$1.run(ClientNotifForwarder.java:120) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14842) SSL connection problems when upgrading to 4.0 when upgrading from 3.0.x
[
https://issues.apache.org/jira/browse/CASSANDRA-14842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-14842:
-
Resolution: Fixed
Status: Resolved (was: Open)
> SSL connection problems when upgrading to 4.0 when upgrading from 3.0.x
> ---
>
> Key: CASSANDRA-14842
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14842
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Tommy Stendahl
>Assignee: Tommy Stendahl
>Priority: Normal
> Fix For: 4.0
>
>
> While testing to upgrade from 3.0.15 to 4.0 the old nodes fails to connect to
> the 4.0 node, I get this exception on the 4.0 node:
>
> {noformat}
> 2018-10-22T11:57:44.366+0200 ERROR [MessagingService-NettyInbound-Thread-3-8]
> InboundHandshakeHandler.java:300 Failed to properly handshake with peer
> /10.216.193.246:58296. Closing the channel.
> io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException:
> SSLv2Hello is disabled
> at
> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459)
> at
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
> at
> io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> at
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965)
> at
> io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:808)
> at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:417)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:317)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: javax.net.ssl.SSLHandshakeException: SSLv2Hello is disabled
> at sun.security.ssl.InputRecord.handleUnknownRecord(InputRecord.java:637)
> at sun.security.ssl.InputRecord.read(InputRecord.java:527)
> at sun.security.ssl.EngineInputRecord.read(EngineInputRecord.java:382)
> at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:962)
> at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:907)
> at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781)
> at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
> at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:294)
> at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1275)
> at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1177)
> at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1221)
> at
> io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489)
> at
> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428)
> ... 14 common frames omitted{noformat}
> In the server encryption options on the 4.0 node I have both "enabled and
> "enable_legacy_ssl_storage_port" set to true so it should accept incoming
> connections on the "ssl_storage_port".
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (8a5b127 -> 9eab263)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 8a5b127 Merge branch 'cassandra-3.0' into cassandra-3.11 new a8ef9c9 Fix unit tests to listen on FBUtilities.getLocalAddress() instead of InetAddress.getLocalHost() new 0cd3b68 Merge branch 'cassandra-2.2' into cassandra-3.0 new 9eab263 Merge branch 'cassandra-3.0' into cassandra-3.11 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../org/apache/cassandra/utils/GuidGenerator.java | 10 +++--- .../org/apache/cassandra/auth/jmx/JMXAuthTest.java | 1 + .../cassandra/concurrent/SEPExecutorTest.java | 9 + .../validation/operations/ThriftCQLTester.java | 10 +++--- .../cassandra/db/OldFormatDeserializerTest.java| 8 .../db/commitlog/CommitLogDescriptorTest.java | 3 +++ .../apache/cassandra/gms/ArrivalWindowTest.java| 22 +++--- .../org/apache/cassandra/gms/GossipDigestTest.java | 9 + .../cassandra/io/sstable/SSTableHeaderFixTest.java | 1 + .../io/sstable/format/ClientModeSSTableTest.java | 3 ++- .../apache/cassandra/net/MessagingServiceTest.java | 3 ++- .../messages/RepairMessageSerializationsTest.java | 9 + .../schema/TupleTypesRepresentationTest.java | 1 + .../apache/cassandra/security/SSLFactoryTest.java | 11 +-- .../cassandra/streaming/SessionInfoTest.java | 8 .../cassandra/tools/CompactionStressTest.java | 7 +++ .../org/apache/cassandra/tools/ToolsTester.java| 5 + .../apache/cassandra/triggers/TriggersTest.java| 13 ++--- .../org/apache/cassandra/utils/MerkleTreeTest.java | 7 +++ .../apache/cassandra/utils/TopKSamplerTest.java| 8 20 files changed, 108 insertions(+), 40 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 0cd3b689597604982e6bb0c822ad300549638d17
Merge: c81d7dc a8ef9c9
Author: Mick Semb Wever
AuthorDate: Wed Mar 11 17:34:20 2020 +0100
Merge branch 'cassandra-2.2' into cassandra-3.0
src/java/org/apache/cassandra/utils/GuidGenerator.java | 6 ++
.../cql3/validation/operations/ThriftCQLTester.java| 10 +++---
test/unit/org/apache/cassandra/gms/ArrivalWindowTest.java | 14 +++---
.../unit/org/apache/cassandra/security/SSLFactoryTest.java | 4 ++--
test/unit/org/apache/cassandra/triggers/TriggersTest.java | 13 ++---
5 files changed, 16 insertions(+), 31 deletions(-)
diff --cc
test/unit/org/apache/cassandra/cql3/validation/operations/ThriftCQLTester.java
index a77e861,7947317..5d4d1a0
---
a/test/unit/org/apache/cassandra/cql3/validation/operations/ThriftCQLTester.java
+++
b/test/unit/org/apache/cassandra/cql3/validation/operations/ThriftCQLTester.java
@@@ -77,10 -74,10 +73,10 @@@ public class ThriftCQLTester extends CQ
public Cassandra.Client getClient() throws Throwable
{
- return getClient(InetAddress.getLocalHost().getHostName(),
thriftPort);
+ return getClient(FBUtilities.getLocalAddress().getHostName(),
thriftPort);
}
- public Cassandra.Client getClient(String hostname, int thriftPort)
throws Throwable
+public Cassandra.Client getClient(String hostname, int thriftPort) throws
Throwable
{
if (client == null)
client = new Cassandra.Client(new TBinaryProtocol(new
TFramedTransportFactory().openTransport(hostname, thriftPort)));
diff --cc test/unit/org/apache/cassandra/triggers/TriggersTest.java
index 67619dc,3cd0a2c..70e040b
--- a/test/unit/org/apache/cassandra/triggers/TriggersTest.java
+++ b/test/unit/org/apache/cassandra/triggers/TriggersTest.java
@@@ -66,10 -61,9 +65,10 @@@ public class TriggersTes
@Before
public void setup() throws Exception
{
+StorageService.instance.initServer(0);
if (thriftServer == null || ! thriftServer.isRunning())
{
- thriftServer = new ThriftServer(InetAddress.getLocalHost(), 9170,
50);
+ thriftServer = new ThriftServer(FBUtilities.getLocalAddress(),
9170, 50);
thriftServer.start();
}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 9eab2633cc194af7a6b04977c0f148a65a735189
Merge: 8a5b127 0cd3b68
Author: Mick Semb Wever
AuthorDate: Wed Mar 11 17:35:26 2020 +0100
Merge branch 'cassandra-3.0' into cassandra-3.11
.../org/apache/cassandra/utils/GuidGenerator.java | 10 +++---
.../org/apache/cassandra/auth/jmx/JMXAuthTest.java | 1 +
.../cassandra/concurrent/SEPExecutorTest.java | 9 +
.../validation/operations/ThriftCQLTester.java | 10 +++---
.../cassandra/db/OldFormatDeserializerTest.java| 8
.../db/commitlog/CommitLogDescriptorTest.java | 3 +++
.../apache/cassandra/gms/ArrivalWindowTest.java| 22 +++---
.../org/apache/cassandra/gms/GossipDigestTest.java | 9 +
.../cassandra/io/sstable/SSTableHeaderFixTest.java | 1 +
.../io/sstable/format/ClientModeSSTableTest.java | 3 ++-
.../apache/cassandra/net/MessagingServiceTest.java | 3 ++-
.../messages/RepairMessageSerializationsTest.java | 9 +
.../schema/TupleTypesRepresentationTest.java | 1 +
.../apache/cassandra/security/SSLFactoryTest.java | 11 +--
.../cassandra/streaming/SessionInfoTest.java | 8
.../cassandra/tools/CompactionStressTest.java | 7 +++
.../org/apache/cassandra/tools/ToolsTester.java| 5 +
.../apache/cassandra/triggers/TriggersTest.java| 13 ++---
.../org/apache/cassandra/utils/MerkleTreeTest.java | 7 +++
.../apache/cassandra/utils/TopKSamplerTest.java| 8
20 files changed, 108 insertions(+), 40 deletions(-)
diff --cc test/unit/org/apache/cassandra/auth/jmx/JMXAuthTest.java
index 10c871b,000..08a89df
mode 100644,00..100644
--- a/test/unit/org/apache/cassandra/auth/jmx/JMXAuthTest.java
+++ b/test/unit/org/apache/cassandra/auth/jmx/JMXAuthTest.java
@@@ -1,279 -1,0 +1,280 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.auth.jmx;
+
+import java.lang.reflect.Field;
+import java.nio.file.Paths;
+import java.rmi.server.RMISocketFactory;
+import java.util.HashMap;
+import java.util.Map;
+import javax.management.JMX;
+import javax.management.MBeanServerConnection;
+import javax.management.ObjectName;
+import javax.management.remote.*;
+import javax.security.auth.Subject;
+import javax.security.auth.callback.CallbackHandler;
+import javax.security.auth.login.LoginException;
+import javax.security.auth.spi.LoginModule;
+
+import com.google.common.collect.ImmutableSet;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import org.apache.cassandra.auth.*;
+import org.apache.cassandra.config.DatabaseDescriptor;
+import org.apache.cassandra.cql3.CQLTester;
+import org.apache.cassandra.db.ColumnFamilyStoreMBean;
+import org.apache.cassandra.utils.JMXServerUtils;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.fail;
+
+public class JMXAuthTest extends CQLTester
+{
+private static JMXConnectorServer jmxServer;
+private static MBeanServerConnection connection;
+private RoleResource role;
+private String tableName;
+private JMXResource tableMBean;
+
+@FunctionalInterface
+private interface MBeanAction
+{
+void execute();
+}
+
+@BeforeClass
+public static void setupClass() throws Exception
+{
++DatabaseDescriptor.daemonInitialization();
+setupAuthorizer();
+setupJMXServer();
+}
+
+private static void setupAuthorizer()
+{
+try
+{
+IAuthorizer authorizer = new StubAuthorizer();
+Field authorizerField =
DatabaseDescriptor.class.getDeclaredField("authorizer");
+authorizerField.setAccessible(true);
+authorizerField.set(null, authorizer);
+DatabaseDescriptor.setPermissionsValidity(0);
+}
+catch (IllegalAccessException | NoSuchFieldException e)
+{
+throw new RuntimeException(e);
+
[cassandra] branch trunk updated (95224b7 -> 8006ec6)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 95224b7 Wait for async abort to finish in StreamTransferTaskTest new a8ef9c9 Fix unit tests to listen on FBUtilities.getLocalAddress() instead of InetAddress.getLocalHost() new 0cd3b68 Merge branch 'cassandra-2.2' into cassandra-3.0 new 9eab263 Merge branch 'cassandra-3.0' into cassandra-3.11 new 8006ec6 Merge branch 'cassandra-3.11' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: src/java/org/apache/cassandra/utils/GuidGenerator.java | 10 +++--- 1 file changed, 3 insertions(+), 7 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 8006ec6f6629e7ea178ad21f80fc70328172bdb2
Merge: 95224b7 9eab263
Author: Mick Semb Wever
AuthorDate: Wed Mar 11 17:37:15 2020 +0100
Merge branch 'cassandra-3.11' into trunk
src/java/org/apache/cassandra/utils/GuidGenerator.java | 10 +++---
1 file changed, 3 insertions(+), 7 deletions(-)
diff --cc src/java/org/apache/cassandra/utils/GuidGenerator.java
index b8c3732,9742353..2b3af01
--- a/src/java/org/apache/cassandra/utils/GuidGenerator.java
+++ b/src/java/org/apache/cassandra/utils/GuidGenerator.java
@@@ -38,12 -36,10 +36,10 @@@ public class GuidGenerato
mySecureRand = new SecureRandom();
long secureInitializer = mySecureRand.nextLong();
myRand = new Random(secureInitializer);
- try
- {
- s_id = InetAddress.getLocalHost().toString();
+ try {
-s_id = FBUtilities.getLocalAddress().toString();
++s_id = FBUtilities.getLocalAddressAndPort().toString();
}
- catch (UnknownHostException e)
- {
+ catch (RuntimeException e) {
throw new AssertionError(e);
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.2 updated: Fix unit tests to listen on FBUtilities.getLocalAddress() instead of InetAddress.getLocalHost()
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch cassandra-2.2
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-2.2 by this push:
new a8ef9c9 Fix unit tests to listen on FBUtilities.getLocalAddress()
instead of InetAddress.getLocalHost()
a8ef9c9 is described below
commit a8ef9c9ad1495f3eb9dc1ef09c0995926561fd8c
Author: Mick Semb Wever
AuthorDate: Wed Mar 4 23:05:11 2020 +0100
Fix unit tests to listen on FBUtilities.getLocalAddress() instead of
InetAddress.getLocalHost()
patch by Mick Semb Wever; reviewed by Benjamin Lerer for CASSANDRA-15622
---
src/java/org/apache/cassandra/utils/GuidGenerator.java | 6 ++
.../cql3/validation/operations/ThriftCQLTester.java| 10 +++---
test/unit/org/apache/cassandra/gms/ArrivalWindowTest.java | 14 +++---
.../unit/org/apache/cassandra/security/SSLFactoryTest.java | 4 ++--
test/unit/org/apache/cassandra/triggers/TriggersTest.java | 14 +++---
5 files changed, 17 insertions(+), 31 deletions(-)
diff --git a/src/java/org/apache/cassandra/utils/GuidGenerator.java
b/src/java/org/apache/cassandra/utils/GuidGenerator.java
index 33f7330..09c7f3e 100644
--- a/src/java/org/apache/cassandra/utils/GuidGenerator.java
+++ b/src/java/org/apache/cassandra/utils/GuidGenerator.java
@@ -17,8 +17,6 @@
*/
package org.apache.cassandra.utils;
-import java.net.InetAddress;
-import java.net.UnknownHostException;
import java.nio.ByteBuffer;
import java.security.SecureRandom;
import java.util.Random;
@@ -36,9 +34,9 @@ public class GuidGenerator {
long secureInitializer = mySecureRand.nextLong();
myRand = new Random(secureInitializer);
try {
-s_id = InetAddress.getLocalHost().toString();
+s_id = FBUtilities.getLocalAddress().toString();
}
-catch (UnknownHostException e) {
+catch (RuntimeException e) {
throw new AssertionError(e);
}
}
diff --git
a/test/unit/org/apache/cassandra/cql3/validation/operations/ThriftCQLTester.java
b/test/unit/org/apache/cassandra/cql3/validation/operations/ThriftCQLTester.java
index 13d6901..7947317 100644
---
a/test/unit/org/apache/cassandra/cql3/validation/operations/ThriftCQLTester.java
+++
b/test/unit/org/apache/cassandra/cql3/validation/operations/ThriftCQLTester.java
@@ -19,9 +19,7 @@
package org.apache.cassandra.cql3.validation.operations;
import java.io.IOException;
-import java.net.InetAddress;
import java.net.ServerSocket;
-import java.net.UnknownHostException;
import org.junit.AfterClass;
import org.junit.BeforeClass;
@@ -29,12 +27,10 @@ import org.junit.BeforeClass;
import org.apache.cassandra.config.DatabaseDescriptor;
import org.apache.cassandra.cql3.*;
import org.apache.cassandra.dht.ByteOrderedPartitioner;
-import org.apache.cassandra.dht.Murmur3Partitioner;
import org.apache.cassandra.service.*;
import org.apache.cassandra.thrift.*;
+import org.apache.cassandra.utils.FBUtilities;
import org.apache.thrift.protocol.TBinaryProtocol;
-import org.apache.thrift.transport.TFramedTransport;
-import org.apache.thrift.transport.TSocket;
public class ThriftCQLTester extends CQLTester
{
@@ -62,7 +58,7 @@ public class ThriftCQLTester extends CQLTester
if (thriftServer == null || ! thriftServer.isRunning())
{
-thriftServer = new ThriftServer(InetAddress.getLocalHost(),
thriftPort, 50);
+thriftServer = new ThriftServer(FBUtilities.getLocalAddress(),
thriftPort, 50);
thriftServer.start();
}
}
@@ -78,7 +74,7 @@ public class ThriftCQLTester extends CQLTester
public Cassandra.Client getClient() throws Throwable
{
-return getClient(InetAddress.getLocalHost().getHostName(), thriftPort);
+return getClient(FBUtilities.getLocalAddress().getHostName(),
thriftPort);
}
public Cassandra.Client getClient(String hostname, int thriftPort)
throws Throwable
diff --git a/test/unit/org/apache/cassandra/gms/ArrivalWindowTest.java
b/test/unit/org/apache/cassandra/gms/ArrivalWindowTest.java
index 511511b..459956b 100644
--- a/test/unit/org/apache/cassandra/gms/ArrivalWindowTest.java
+++ b/test/unit/org/apache/cassandra/gms/ArrivalWindowTest.java
@@ -25,9 +25,9 @@ import static org.junit.Assert.*;
import org.junit.Test;
-import java.lang.RuntimeException;
import java.net.InetAddress;
-import java.net.UnknownHostException;
+
+import org.apache.cassandra.utils.FBUtilities;
public class ArrivalWindowTest
{
@@ -36,15 +36,7 @@ public class ArrivalWindowTest
{
final ArrivalWindow windowWithNano = new ArrivalWindow(4);
final long toNano = 100L;
-InetAddress ep;
-try
-{
-ep = InetAddress.getLocalHost();
-}
-catch (UnknownHostException e)
-{
-throw new
[cassandra] branch cassandra-3.0 updated (c81d7dc -> 0cd3b68)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from c81d7dc Use result size for built index test new a8ef9c9 Fix unit tests to listen on FBUtilities.getLocalAddress() instead of InetAddress.getLocalHost() new 0cd3b68 Merge branch 'cassandra-2.2' into cassandra-3.0 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: src/java/org/apache/cassandra/utils/GuidGenerator.java | 6 ++ .../cql3/validation/operations/ThriftCQLTester.java| 10 +++--- test/unit/org/apache/cassandra/gms/ArrivalWindowTest.java | 14 +++--- .../unit/org/apache/cassandra/security/SSLFactoryTest.java | 4 ++-- test/unit/org/apache/cassandra/triggers/TriggersTest.java | 13 ++--- 5 files changed, 16 insertions(+), 31 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15633) Cassandra rebuild failing
[ https://issues.apache.org/jira/browse/CASSANDRA-15633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057189#comment-17057189 ] Chirantan commented on CASSANDRA-15633: --- sorry for type issue is with repair when i try to bring up node it came up but nodetool command is not working that node getting below message nodetool: Failed to connect to '127.0.0.1:7199' - FailedLoginException: 'Cannot login as server authentication setup is not yet completed'. for other node its showing as down asd in describe cluster that node is showing as unavailable and for other node > Cassandra rebuild failing > -- > > Key: CASSANDRA-15633 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15633 > Project: Cassandra > Issue Type: Task >Reporter: Chirantan >Priority: Normal > > [2020-03-10 18:09:51,533] Repair session 7be9f3c0-631e-11ea-8e47-ef25c77f7899 > for range [(6301399909211397355,6302095577836606571], > (6301291625775958686,6301399909211397355], > (6302095577836606571,6313970392023331777], > (6297705134651154643,6301291625775958686]] finished (progress: 44%) > Exception occurred during clean-up. > java.lang.reflect.UndeclaredThrowableException > Cassandra has shutdown. > error: [2020-03-10 18:09:56,868] JMX connection closed. You should check > server log for repair status of keyspace cortex(Subsequent keyspaces are not > going to be repaired). > -- StackTrace -- > java.io.IOException: [2020-03-10 18:09:56,868] JMX connection closed. You > should check server log for repair status of keyspace cortex(Subsequent > keyspaces are not going to be repaired). > at > org.apache.cassandra.tools.RepairRunner.handleConnectionFailed(RepairRunner.java:98) > at > org.apache.cassandra.utils.progress.jmx.JMXNotificationProgressListener.handleNotification(JMXNotificationProgressListener.java:86) > at > javax.management.NotificationBroadcasterSupport.handleNotification(NotificationBroadcasterSupport.java:286) > at > javax.management.NotificationBroadcasterSupport$SendNotifJob.run(NotificationBroadcasterSupport.java:363) > at > javax.management.NotificationBroadcasterSupport$1.execute(NotificationBroadcasterSupport.java:348) > at > javax.management.NotificationBroadcasterSupport.sendNotification(NotificationBroadcasterSupport.java:259) > at > javax.management.remote.rmi.RMIConnector.sendNotification(RMIConnector.java:452) > at > javax.management.remote.rmi.RMIConnector.access$1200(RMIConnector.java:132) > at > javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.gotIOException(RMIConnector.java:1542) > at > javax.management.remote.rmi.RMIConnector$RMINotifClient.fetchNotifs(RMIConnector.java:1363) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchOneNotif(ClientNotifForwarder.java:667) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchNotifs(ClientNotifForwarder.java:619) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.doRun(ClientNotifForwarder.java:483) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.run(ClientNotifForwarder.java:464) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$LinearExecutor$1.run(ClientNotifForwarder.java:120) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15622) Unit tests throw UnknownHostException trying to use `InetAddress.getLocalHost()` instead of `FBUtilities.getLocalAddress()`
[
https://issues.apache.org/jira/browse/CASSANDRA-15622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15622:
---
Description:
Many of the unit tests in Jenkins fail because of the use of
{{`InetAddress.getLocalHost()`}} in the test classes.
The Jenkins agents need a public ipaddress (and a hostname associated to it) so
the Jenkins master can connect to them (agents can be hosted externally, by
donating third-parties).
The call to {{`InetAddress.getLocalHost()`}} can resolve to a hostname that
can't be looked up.
Not only can it not be listed in {{`/etc/hosts`}}, but we don't want it to be
either (in case of accidental external port opening if the hostname points to
the public ipaddress). (Which is also ASF policy on this infrastructure.)
The unit test code needs to replace these code occurrences with the call to
{{`FBUtilities.getLocalAddress()`}}
was:
Many of the unit tests in Jenkins fail because of the use of
{{`InetAddress.getLocalHost()`}} in the test classes.
The Jenkins agents need a public ipaddress (and a hostname associated to it) so
the Jenkins master can connect to them (agents can be hosted externally, by
donating third-parties).
The call to {{`InetAddress.getLocalHost()`}} can resolve to a hostname that
can't be looked up.
Not only it likely not listed in {{`/etc/hosts`}}, but we don't want it to be
either (in case of accidental external port opening if the hostname points to
the public ipaddress). (This is also ASF policy on this infrastructure.)
The unit test code needs to replace these code occurrences with the call to
{{`FBUtilities.getLocalAddress()`}}
> Unit tests throw UnknownHostException trying to use
> `InetAddress.getLocalHost()` instead of `FBUtilities.getLocalAddress()`
> ---
>
> Key: CASSANDRA-15622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15622
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
>
> Many of the unit tests in Jenkins fail because of the use of
> {{`InetAddress.getLocalHost()`}} in the test classes.
> The Jenkins agents need a public ipaddress (and a hostname associated to it)
> so the Jenkins master can connect to them (agents can be hosted externally,
> by donating third-parties).
> The call to {{`InetAddress.getLocalHost()`}} can resolve to a hostname that
> can't be looked up.
> Not only can it not be listed in {{`/etc/hosts`}}, but we don't want it to be
> either (in case of accidental external port opening if the hostname points to
> the public ipaddress). (Which is also ASF policy on this infrastructure.)
> The unit test code needs to replace these code occurrences with the call to
> {{`FBUtilities.getLocalAddress()`}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15634) one node up but its showing as down is showing as down for other node
[ https://issues.apache.org/jira/browse/CASSANDRA-15634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-15634: - Resolution: Invalid Status: Resolved (was: Triage Needed) Jira is not a support system, the best way to get help with your issue is to email the user list, or join the #cassandra slack channel. http://cassandra.apache.org/community/ > one node up but its showing as down is showing as down for other node > --- > > Key: CASSANDRA-15634 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15634 > Project: Cassandra > Issue Type: Task >Reporter: Chirantan >Priority: Normal > > > one node up but its showing as down is showing as down for other node > and in describecluster its showing as unreachable > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15634) one node up but its showing as down is showing as down for other node
Chirantan created CASSANDRA-15634: - Summary: one node up but its showing as down is showing as down for other node Key: CASSANDRA-15634 URL: https://issues.apache.org/jira/browse/CASSANDRA-15634 Project: Cassandra Issue Type: Task Reporter: Chirantan one node up but its showing as down is showing as down for other node and in describecluster its showing as unreachable -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15633) Cassandra rebuild failing
[ https://issues.apache.org/jira/browse/CASSANDRA-15633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057151#comment-17057151 ] Brandon Williams edited comment on CASSANDRA-15633 at 3/11/20, 4:03 PM: Your description is from repair, not rebuild. You only have the (jmx) client side error, you need to do as it suggests, and check the server log. was (Author: brandon.williams): Your description is from repair, not rebuild. You only have the (jmx) client side error, you need to do as it suggest, and check the server log. > Cassandra rebuild failing > -- > > Key: CASSANDRA-15633 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15633 > Project: Cassandra > Issue Type: Task >Reporter: Chirantan >Priority: Normal > > [2020-03-10 18:09:51,533] Repair session 7be9f3c0-631e-11ea-8e47-ef25c77f7899 > for range [(6301399909211397355,6302095577836606571], > (6301291625775958686,6301399909211397355], > (6302095577836606571,6313970392023331777], > (6297705134651154643,6301291625775958686]] finished (progress: 44%) > Exception occurred during clean-up. > java.lang.reflect.UndeclaredThrowableException > Cassandra has shutdown. > error: [2020-03-10 18:09:56,868] JMX connection closed. You should check > server log for repair status of keyspace cortex(Subsequent keyspaces are not > going to be repaired). > -- StackTrace -- > java.io.IOException: [2020-03-10 18:09:56,868] JMX connection closed. You > should check server log for repair status of keyspace cortex(Subsequent > keyspaces are not going to be repaired). > at > org.apache.cassandra.tools.RepairRunner.handleConnectionFailed(RepairRunner.java:98) > at > org.apache.cassandra.utils.progress.jmx.JMXNotificationProgressListener.handleNotification(JMXNotificationProgressListener.java:86) > at > javax.management.NotificationBroadcasterSupport.handleNotification(NotificationBroadcasterSupport.java:286) > at > javax.management.NotificationBroadcasterSupport$SendNotifJob.run(NotificationBroadcasterSupport.java:363) > at > javax.management.NotificationBroadcasterSupport$1.execute(NotificationBroadcasterSupport.java:348) > at > javax.management.NotificationBroadcasterSupport.sendNotification(NotificationBroadcasterSupport.java:259) > at > javax.management.remote.rmi.RMIConnector.sendNotification(RMIConnector.java:452) > at > javax.management.remote.rmi.RMIConnector.access$1200(RMIConnector.java:132) > at > javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.gotIOException(RMIConnector.java:1542) > at > javax.management.remote.rmi.RMIConnector$RMINotifClient.fetchNotifs(RMIConnector.java:1363) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchOneNotif(ClientNotifForwarder.java:667) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchNotifs(ClientNotifForwarder.java:619) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.doRun(ClientNotifForwarder.java:483) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.run(ClientNotifForwarder.java:464) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$LinearExecutor$1.run(ClientNotifForwarder.java:120) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15633) Cassandra rebuild failing
[ https://issues.apache.org/jira/browse/CASSANDRA-15633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057151#comment-17057151 ] Brandon Williams commented on CASSANDRA-15633: -- Your description is from repair, not build. You only have the (jmx) client side error, you need to do as it suggest, and check the server log. > Cassandra rebuild failing > -- > > Key: CASSANDRA-15633 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15633 > Project: Cassandra > Issue Type: Task >Reporter: Chirantan >Priority: Normal > > [2020-03-10 18:09:51,533] Repair session 7be9f3c0-631e-11ea-8e47-ef25c77f7899 > for range [(6301399909211397355,6302095577836606571], > (6301291625775958686,6301399909211397355], > (6302095577836606571,6313970392023331777], > (6297705134651154643,6301291625775958686]] finished (progress: 44%) > Exception occurred during clean-up. > java.lang.reflect.UndeclaredThrowableException > Cassandra has shutdown. > error: [2020-03-10 18:09:56,868] JMX connection closed. You should check > server log for repair status of keyspace cortex(Subsequent keyspaces are not > going to be repaired). > -- StackTrace -- > java.io.IOException: [2020-03-10 18:09:56,868] JMX connection closed. You > should check server log for repair status of keyspace cortex(Subsequent > keyspaces are not going to be repaired). > at > org.apache.cassandra.tools.RepairRunner.handleConnectionFailed(RepairRunner.java:98) > at > org.apache.cassandra.utils.progress.jmx.JMXNotificationProgressListener.handleNotification(JMXNotificationProgressListener.java:86) > at > javax.management.NotificationBroadcasterSupport.handleNotification(NotificationBroadcasterSupport.java:286) > at > javax.management.NotificationBroadcasterSupport$SendNotifJob.run(NotificationBroadcasterSupport.java:363) > at > javax.management.NotificationBroadcasterSupport$1.execute(NotificationBroadcasterSupport.java:348) > at > javax.management.NotificationBroadcasterSupport.sendNotification(NotificationBroadcasterSupport.java:259) > at > javax.management.remote.rmi.RMIConnector.sendNotification(RMIConnector.java:452) > at > javax.management.remote.rmi.RMIConnector.access$1200(RMIConnector.java:132) > at > javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.gotIOException(RMIConnector.java:1542) > at > javax.management.remote.rmi.RMIConnector$RMINotifClient.fetchNotifs(RMIConnector.java:1363) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchOneNotif(ClientNotifForwarder.java:667) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchNotifs(ClientNotifForwarder.java:619) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.doRun(ClientNotifForwarder.java:483) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.run(ClientNotifForwarder.java:464) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$LinearExecutor$1.run(ClientNotifForwarder.java:120) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15633) Cassandra rebuild failing
[ https://issues.apache.org/jira/browse/CASSANDRA-15633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057151#comment-17057151 ] Brandon Williams edited comment on CASSANDRA-15633 at 3/11/20, 4:03 PM: Your description is from repair, not rebuild. You only have the (jmx) client side error, you need to do as it suggest, and check the server log. was (Author: brandon.williams): Your description is from repair, not build. You only have the (jmx) client side error, you need to do as it suggest, and check the server log. > Cassandra rebuild failing > -- > > Key: CASSANDRA-15633 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15633 > Project: Cassandra > Issue Type: Task >Reporter: Chirantan >Priority: Normal > > [2020-03-10 18:09:51,533] Repair session 7be9f3c0-631e-11ea-8e47-ef25c77f7899 > for range [(6301399909211397355,6302095577836606571], > (6301291625775958686,6301399909211397355], > (6302095577836606571,6313970392023331777], > (6297705134651154643,6301291625775958686]] finished (progress: 44%) > Exception occurred during clean-up. > java.lang.reflect.UndeclaredThrowableException > Cassandra has shutdown. > error: [2020-03-10 18:09:56,868] JMX connection closed. You should check > server log for repair status of keyspace cortex(Subsequent keyspaces are not > going to be repaired). > -- StackTrace -- > java.io.IOException: [2020-03-10 18:09:56,868] JMX connection closed. You > should check server log for repair status of keyspace cortex(Subsequent > keyspaces are not going to be repaired). > at > org.apache.cassandra.tools.RepairRunner.handleConnectionFailed(RepairRunner.java:98) > at > org.apache.cassandra.utils.progress.jmx.JMXNotificationProgressListener.handleNotification(JMXNotificationProgressListener.java:86) > at > javax.management.NotificationBroadcasterSupport.handleNotification(NotificationBroadcasterSupport.java:286) > at > javax.management.NotificationBroadcasterSupport$SendNotifJob.run(NotificationBroadcasterSupport.java:363) > at > javax.management.NotificationBroadcasterSupport$1.execute(NotificationBroadcasterSupport.java:348) > at > javax.management.NotificationBroadcasterSupport.sendNotification(NotificationBroadcasterSupport.java:259) > at > javax.management.remote.rmi.RMIConnector.sendNotification(RMIConnector.java:452) > at > javax.management.remote.rmi.RMIConnector.access$1200(RMIConnector.java:132) > at > javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.gotIOException(RMIConnector.java:1542) > at > javax.management.remote.rmi.RMIConnector$RMINotifClient.fetchNotifs(RMIConnector.java:1363) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchOneNotif(ClientNotifForwarder.java:667) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchNotifs(ClientNotifForwarder.java:619) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.doRun(ClientNotifForwarder.java:483) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.run(ClientNotifForwarder.java:464) > at > com.sun.jmx.remote.internal.ClientNotifForwarder$LinearExecutor$1.run(ClientNotifForwarder.java:120) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15633) Cassandra rebuild failing
Chirantan created CASSANDRA-15633: - Summary: Cassandra rebuild failing Key: CASSANDRA-15633 URL: https://issues.apache.org/jira/browse/CASSANDRA-15633 Project: Cassandra Issue Type: Task Reporter: Chirantan [2020-03-10 18:09:51,533] Repair session 7be9f3c0-631e-11ea-8e47-ef25c77f7899 for range [(6301399909211397355,6302095577836606571], (6301291625775958686,6301399909211397355], (6302095577836606571,6313970392023331777], (6297705134651154643,6301291625775958686]] finished (progress: 44%) Exception occurred during clean-up. java.lang.reflect.UndeclaredThrowableException Cassandra has shutdown. error: [2020-03-10 18:09:56,868] JMX connection closed. You should check server log for repair status of keyspace cortex(Subsequent keyspaces are not going to be repaired). -- StackTrace -- java.io.IOException: [2020-03-10 18:09:56,868] JMX connection closed. You should check server log for repair status of keyspace cortex(Subsequent keyspaces are not going to be repaired). at org.apache.cassandra.tools.RepairRunner.handleConnectionFailed(RepairRunner.java:98) at org.apache.cassandra.utils.progress.jmx.JMXNotificationProgressListener.handleNotification(JMXNotificationProgressListener.java:86) at javax.management.NotificationBroadcasterSupport.handleNotification(NotificationBroadcasterSupport.java:286) at javax.management.NotificationBroadcasterSupport$SendNotifJob.run(NotificationBroadcasterSupport.java:363) at javax.management.NotificationBroadcasterSupport$1.execute(NotificationBroadcasterSupport.java:348) at javax.management.NotificationBroadcasterSupport.sendNotification(NotificationBroadcasterSupport.java:259) at javax.management.remote.rmi.RMIConnector.sendNotification(RMIConnector.java:452) at javax.management.remote.rmi.RMIConnector.access$1200(RMIConnector.java:132) at javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.gotIOException(RMIConnector.java:1542) at javax.management.remote.rmi.RMIConnector$RMINotifClient.fetchNotifs(RMIConnector.java:1363) at com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchOneNotif(ClientNotifForwarder.java:667) at com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchNotifs(ClientNotifForwarder.java:619) at com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.doRun(ClientNotifForwarder.java:483) at com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.run(ClientNotifForwarder.java:464) at com.sun.jmx.remote.internal.ClientNotifForwarder$LinearExecutor$1.run(ClientNotifForwarder.java:120) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15614) Flakey test - TestBootstrap.test_resumable_bootstrap
[
https://issues.apache.org/jira/browse/CASSANDRA-15614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057141#comment-17057141
]
Ekaterina Dimitrova commented on CASSANDRA-15614:
-
As this was closed I created ticket for improvement CASSANDRA-15632
> Flakey test - TestBootstrap.test_resumable_bootstrap
> -
>
> Key: CASSANDRA-15614
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15614
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: test_resumable_bootstrap-CASSANDRA-15614.txt
>
>
> Fails 2 out of 10 times on Mac/Java 8
> {noformat}
> ==
> FAILURES
> ==
> *___
> TestBootstrap.test_resumable_bootstrap
> ___*
>
> self =
>
> *@since('2.2')*
> *def test_resumable_bootstrap(self):*
> *"""*
> *Test resuming bootstrap after data streaming failure*
> *"""*
> *cluster = self.cluster*
> *cluster.populate(2)*
> **
> *node1 = cluster.nodes['node1']*
> *# set up byteman*
> *node1.byteman_port = '8100'*
> *node1.import_config_files()*
> **
> *cluster.start(wait_other_notice=True)*
> *# kill stream to node3 in the middle of streaming to let it fail*
> *if cluster.version() < '4.0':*
> *node1.byteman_submit([self.byteman_submit_path_pre_4_0])*
> *else:*
> *node1.byteman_submit([self.byteman_submit_path_4_0])*
> *node1.stress(['write', 'n=1K', 'no-warmup', 'cl=TWO', '-schema',
> 'replication(factor=2)', '-rate', 'threads=50'])*
> *cluster.flush()*
> **
> *# start bootstrapping node3 and wait for streaming*
> *node3 = new_node(cluster)*
> *node3.start(wait_other_notice=False)*
> **
> *# let streaming fail as we expect*
> *> node3.watch_log_for('Some data streaming failed')*
>
> *bootstrap_test.py*:365:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _
>
> self = , exprs = 'Some data streaming
> failed', from_mark = None, timeout = 600, process = None, verbose = False,
> filename = 'system.log'
>
> *def watch_log_for(self, exprs, from_mark=None, timeout=600,
> process=None, verbose=False, filename='system.log'):*
> *"""*
> *Watch the log until one or more (regular) expression are found.*
> *This methods when all the expressions have been found or the
> method*
> *timeouts (a TimeoutError is then raised). On successful
> completion,*
> *a list of pair (line matched, match object) is returned.*
> *"""*
> *start = time.time()*
> *tofind = [exprs] if isinstance(exprs, string_types) else exprs*
> *tofind = [re.compile(e) for e in tofind]*
> *matchings = []*
> *reads = ""*
> *if len(tofind) == 0:*
> *return None*
> **
> *log_file = os.path.join(self.get_path(), 'logs', filename)*
> *output_read = False*
> *while not os.path.exists(log_file):*
> *time.sleep(.5)*
> *if start + timeout < time.time():*
> *raise TimeoutError(time.strftime("%d %b %Y %H:%M:%S",
> time.gmtime()) + " [" + self.name + "] Timed out waiting for {} to be
> created.".format(log_file))*
> *if process and not output_read:*
> *process.poll()*
> *if process.returncode is not None:*
> *self.print_process_output(self.name, process, verbose)*
> *output_read = True*
> *if process.returncode != 0:*
> *raise RuntimeError() # Shouldn't reuse RuntimeError
> but I'm lazy*
> **
> *with open(log_file) as f:*
> *if from_mark:*
> *f.seek(from_mark)*
> **
> *while True:*
> *# First, if we have a process to check, then check it.*
> *# Skip on Windows - stdout/stderr is cassandra.bat*
> *if not
[jira] [Created] (CASSANDRA-15632) Create a deterministic in-jvm equivalent of dtest TestBootstrap.test_resumable_bootstrap
Ekaterina Dimitrova created CASSANDRA-15632: --- Summary: Create a deterministic in-jvm equivalent of dtest TestBootstrap.test_resumable_bootstrap Key: CASSANDRA-15632 URL: https://issues.apache.org/jira/browse/CASSANDRA-15632 Project: Cassandra Issue Type: Improvement Reporter: Ekaterina Dimitrova This ticket is a followup on CASSANDRA-15614 - fix flakey test TestBootstrap.test_resumable_bootstrap. The dtest is not really fully deterministic. In-jvm test might give a way for better control on the streaming failures. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14633) Update trunk's netty to 4.1.28
[ https://issues.apache.org/jira/browse/CASSANDRA-14633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Konstantinov updated CASSANDRA-14633: Summary: Update trunk's netty to 4.1.28 (was: Update trunk's netty to 4.1.128) > Update trunk's netty to 4.1.28 > -- > > Key: CASSANDRA-14633 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14633 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Streaming and Messaging >Reporter: Jason Brown >Assignee: Jason Brown >Priority: Low > Fix For: 4.0 > > > Upgrade the version of netty in trunk (4.0) to 4.1.28. We currently have > 4.1.14, and there's been lots of bug fixes since we last updated this for > trunk. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15550) Fix flaky test org.apache.cassandra.streaming.StreamTransferTaskTest testFailSessionDuringTransferShouldNotReleaseReferences
[
https://issues.apache.org/jira/browse/CASSANDRA-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057136#comment-17057136
]
Brandon Williams commented on CASSANDRA-15550:
--
Commited with retries limited to 10, since recently I've seen a trend where
tests fail on weaker machines, and 10 retries is still only 100ms total.
> Fix flaky test org.apache.cassandra.streaming.StreamTransferTaskTest
> testFailSessionDuringTransferShouldNotReleaseReferences
>
>
> Key: CASSANDRA-15550
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15550
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> {code}
> junit.framework.AssertionFailedError: expected:<0> but was:<1>
> at
> org.apache.cassandra.streaming.StreamTransferTaskTest.testFailSessionDuringTransferShouldNotReleaseReferences(StreamTransferTaskTest.java:181)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> Failure was on java 11
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15550) Fix flaky test org.apache.cassandra.streaming.StreamTransferTaskTest testFailSessionDuringTransferShouldNotReleaseReferences
[
https://issues.apache.org/jira/browse/CASSANDRA-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-15550:
-
Since Version: 2.2.8
Source Control Link:
https://github.com/apache/cassandra/commit/95224b71e19e2b7f18596da070937291f040f695
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Fix flaky test org.apache.cassandra.streaming.StreamTransferTaskTest
> testFailSessionDuringTransferShouldNotReleaseReferences
>
>
> Key: CASSANDRA-15550
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15550
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> {code}
> junit.framework.AssertionFailedError: expected:<0> but was:<1>
> at
> org.apache.cassandra.streaming.StreamTransferTaskTest.testFailSessionDuringTransferShouldNotReleaseReferences(StreamTransferTaskTest.java:181)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> Failure was on java 11
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15543) flaky test org.apache.cassandra.distributed.test.SimpleReadWriteTest.readWithSchemaDisagreement
[
https://issues.apache.org/jira/browse/CASSANDRA-15543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057135#comment-17057135
]
Kevin Gallardo commented on CASSANDRA-15543:
bq. Also, RequestFailureException is not used only here in the codebase...
Right, I was planning on making the immutable copy changes for calls to
WriteFailureException too
> flaky test
> org.apache.cassandra.distributed.test.SimpleReadWriteTest.readWithSchemaDisagreement
> ---
>
> Key: CASSANDRA-15543
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15543
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: David Capwell
>Assignee: Kevin Gallardo
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> This fails infrequently, last seen failure was on java 8
> {code}
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.distributed.test.DistributedReadWritePathTest.readWithSchemaDisagreement(DistributedReadWritePathTest.java:276)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Wait for async abort to finish in StreamTransferTaskTest
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 95224b7 Wait for async abort to finish in StreamTransferTaskTest
95224b7 is described below
commit 95224b71e19e2b7f18596da070937291f040f695
Author: Brandon Williams
AuthorDate: Wed Feb 12 17:47:53 2020 -0600
Wait for async abort to finish in StreamTransferTaskTest
Patch by brandonwilliams, reviewed by Dinesh Joshi for CASSANDRA-15550
---
.../apache/cassandra/streaming/StreamTransferTaskTest.java| 11 +++
1 file changed, 11 insertions(+)
diff --git
a/test/unit/org/apache/cassandra/streaming/StreamTransferTaskTest.java
b/test/unit/org/apache/cassandra/streaming/StreamTransferTaskTest.java
index 80607c5..2361125 100644
--- a/test/unit/org/apache/cassandra/streaming/StreamTransferTaskTest.java
+++ b/test/unit/org/apache/cassandra/streaming/StreamTransferTaskTest.java
@@ -34,6 +34,7 @@ import org.junit.Test;
import io.netty.channel.embedded.EmbeddedChannel;
import org.junit.Assert;
import org.apache.cassandra.SchemaLoader;
+import org.apache.cassandra.concurrent.ScheduledExecutors;
import org.apache.cassandra.db.ColumnFamilyStore;
import org.apache.cassandra.db.Keyspace;
import org.apache.cassandra.db.streaming.CassandraOutgoingFile;
@@ -169,6 +170,16 @@ public class StreamTransferTaskTest
assertEquals(1, ref.globalCount());
}
+//wait for stream to abort asynchronously
+int tries = 10;
+while (ScheduledExecutors.nonPeriodicTasks.getActiveCount() > 0)
+{
+if(tries < 1)
+throw new RuntimeException("test did not complete in time");
+Thread.sleep(10);
+tries--;
+}
+
//simulate finish transfer
for (OutgoingStreamMessage file : files)
{
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15550) Fix flaky test org.apache.cassandra.streaming.StreamTransferTaskTest testFailSessionDuringTransferShouldNotReleaseReferences
[
https://issues.apache.org/jira/browse/CASSANDRA-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-15550:
-
Status: Ready to Commit (was: Changes Suggested)
> Fix flaky test org.apache.cassandra.streaming.StreamTransferTaskTest
> testFailSessionDuringTransferShouldNotReleaseReferences
>
>
> Key: CASSANDRA-15550
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15550
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> {code}
> junit.framework.AssertionFailedError: expected:<0> but was:<1>
> at
> org.apache.cassandra.streaming.StreamTransferTaskTest.testFailSessionDuringTransferShouldNotReleaseReferences(StreamTransferTaskTest.java:181)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> Failure was on java 11
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15543) flaky test org.apache.cassandra.distributed.test.SimpleReadWriteTest.readWithSchemaDisagreement
[
https://issues.apache.org/jira/browse/CASSANDRA-15543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057127#comment-17057127
]
Ekaterina Dimitrova commented on CASSANDRA-15543:
-
Also, RequestFailureException is not used only here in the codebase...
> flaky test
> org.apache.cassandra.distributed.test.SimpleReadWriteTest.readWithSchemaDisagreement
> ---
>
> Key: CASSANDRA-15543
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15543
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: David Capwell
>Assignee: Kevin Gallardo
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> This fails infrequently, last seen failure was on java 8
> {code}
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.distributed.test.DistributedReadWritePathTest.readWithSchemaDisagreement(DistributedReadWritePathTest.java:276)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15543) flaky test org.apache.cassandra.distributed.test.SimpleReadWriteTest.readWithSchemaDisagreement
[
https://issues.apache.org/jira/browse/CASSANDRA-15543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057125#comment-17057125
]
Ekaterina Dimitrova commented on CASSANDRA-15543:
-
Yes, I would also separate them. This is test fix. The other one would be to
improve the failure reporting for the end users and try to mitigate any
potential confusion.
> flaky test
> org.apache.cassandra.distributed.test.SimpleReadWriteTest.readWithSchemaDisagreement
> ---
>
> Key: CASSANDRA-15543
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15543
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: David Capwell
>Assignee: Kevin Gallardo
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> This fails infrequently, last seen failure was on java 8
> {code}
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.distributed.test.DistributedReadWritePathTest.readWithSchemaDisagreement(DistributedReadWritePathTest.java:276)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15543) flaky test org.apache.cassandra.distributed.test.SimpleReadWriteTest.readWithSchemaDisagreement
[
https://issues.apache.org/jira/browse/CASSANDRA-15543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057091#comment-17057091
]
Kevin Gallardo edited comment on CASSANDRA-15543 at 3/11/20, 3:23 PM:
--
Thanks for the reply, Ekaterina, and for confirming the behavior described.
Got some more time to think about this, and it seems the best way to approach
it, since what I suggest would be a larger behavioral change, would be to
create a separate ticket for it.
I think in this ticket I would rather: Fix the discrepancies between the
failures number and the failure messages with the immutable map copy + fix the
test that expects 2 failures to only expect 1 to match current behavior.
Then, in the other ticket we can properly advocate/discuss pros and cons of
returning cohesive error messages VS fail-fast & incohesive error message.
was (Author: newkek):
Thanks for the reply, Ekaterina.
Got some more time to think about this, and I think the best way to approach
it, since what I suggest would be a larger behavioral change, would be to
create a separate ticket for it.
I think in this ticket I would rather: Fix the discrepancies between the
failures number and the failure messages with the immutable map copy + fix the
test that expects 2 failures to only expect 1 to match current behavior.
Then, in the other ticket we can discuss pros and cons of returning cohesive
error messages VS fail-fast & incohesive error message.
> flaky test
> org.apache.cassandra.distributed.test.SimpleReadWriteTest.readWithSchemaDisagreement
> ---
>
> Key: CASSANDRA-15543
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15543
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: David Capwell
>Assignee: Kevin Gallardo
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> This fails infrequently, last seen failure was on java 8
> {code}
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.distributed.test.DistributedReadWritePathTest.readWithSchemaDisagreement(DistributedReadWritePathTest.java:276)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15543) flaky test org.apache.cassandra.distributed.test.SimpleReadWriteTest.readWithSchemaDisagreement
[
https://issues.apache.org/jira/browse/CASSANDRA-15543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057091#comment-17057091
]
Kevin Gallardo edited comment on CASSANDRA-15543 at 3/11/20, 3:23 PM:
--
Thanks for the reply, Ekaterina, and for confirming the behavior described.
Got some more time to think about this, and it seems the best way to approach
it, since what I suggest would be a larger behavioral change, would be to
create a separate ticket for it.
I think in this ticket I would rather: Fix the discrepancies between the
failures number and the failure messages with the immutable map copy + fix the
test that expects 2 failures to only expect 1 to match current behavior.
Then, in the other ticket we can properly advocate/discuss pros and cons of
returning cohesive error messages VS fail-fast & incohesive error message.
Does that make sense?
was (Author: newkek):
Thanks for the reply, Ekaterina, and for confirming the behavior described.
Got some more time to think about this, and it seems the best way to approach
it, since what I suggest would be a larger behavioral change, would be to
create a separate ticket for it.
I think in this ticket I would rather: Fix the discrepancies between the
failures number and the failure messages with the immutable map copy + fix the
test that expects 2 failures to only expect 1 to match current behavior.
Then, in the other ticket we can properly advocate/discuss pros and cons of
returning cohesive error messages VS fail-fast & incohesive error message.
> flaky test
> org.apache.cassandra.distributed.test.SimpleReadWriteTest.readWithSchemaDisagreement
> ---
>
> Key: CASSANDRA-15543
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15543
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: David Capwell
>Assignee: Kevin Gallardo
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> This fails infrequently, last seen failure was on java 8
> {code}
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.distributed.test.DistributedReadWritePathTest.readWithSchemaDisagreement(DistributedReadWritePathTest.java:276)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13994) Remove COMPACT STORAGE internals before 4.0 release
[
https://issues.apache.org/jira/browse/CASSANDRA-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057098#comment-17057098
]
Alex Petrov commented on CASSANDRA-13994:
-
[~e.dimitrova] sure!
In short, we've recently deprecated thrift, but some of the parts that were
used for thrift compatibility in the new storage engine are still present. For
example, {{isDense}}, {{isCompact}}, {{isSuper}}
[methods|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/schema/TableMetadata.java#L238-L276]
and
[flags|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/schema/TableMetadata.java#L70].
We need to remove usages and mentions of those, only leave enough to make sure
we have reasonable error messages. You can find a bit more information on
compact storage in [CASSANDRA-10857].
> Remove COMPACT STORAGE internals before 4.0 release
> ---
>
> Key: CASSANDRA-13994
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13994
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Local Write-Read Paths
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Low
> Fix For: 4.0, 4.0-rc
>
>
> 4.0 comes without thrift (after [CASSANDRA-5]) and COMPACT STORAGE (after
> [CASSANDRA-10857]), and since Compact Storage flags are now disabled, all of
> the related functionality is useless.
> There are still some things to consider:
> 1. One of the system tables (built indexes) was compact. For now, we just
> added {{value}} column to it to make sure it's backwards-compatible, but we
> might want to make sure it's just a "normal" table and doesn't have redundant
> columns.
> 2. Compact Tables were building indexes in {{KEYS}} mode. Removing it is
> trivial, but this would mean that all built indexes will be defunct. We could
> log a warning for now and ask users to migrate off those for now and
> completely remove it from future releases. It's just a couple of classes
> though.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15543) flaky test org.apache.cassandra.distributed.test.SimpleReadWriteTest.readWithSchemaDisagreement
[
https://issues.apache.org/jira/browse/CASSANDRA-15543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057091#comment-17057091
]
Kevin Gallardo commented on CASSANDRA-15543:
Thanks for the reply, Ekaterina.
Got some more time to think about this, and I think the best way to approach
it, since what I suggest would be a larger behavioral change, would be to
create a separate ticket for it.
I think in this ticket I would rather: Fix the discrepancies between the
failures number and the failure messages with the immutable map copy + fix the
test that expects 2 failures to only expect 1 to match current behavior.
Then, in the other ticket we can discuss pros and cons of returning cohesive
error messages VS fail-fast & incohesive error message.
> flaky test
> org.apache.cassandra.distributed.test.SimpleReadWriteTest.readWithSchemaDisagreement
> ---
>
> Key: CASSANDRA-15543
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15543
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: David Capwell
>Assignee: Kevin Gallardo
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> This fails infrequently, last seen failure was on java 8
> {code}
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.distributed.test.DistributedReadWritePathTest.readWithSchemaDisagreement(DistributedReadWritePathTest.java:276)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15152) Batch Log - Mutation too large while bootstrapping a newly added node
[ https://issues.apache.org/jira/browse/CASSANDRA-15152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057070#comment-17057070 ] Bjorn Carlsson commented on CASSANDRA-15152: Hi [~via.vokal], we are experiencing the same issue when joining new nodes to our cluster. Did you manage to solve this? We have the same questions regarding which keyspace table the mutations belongs too. Kind regards, Bjorn > Batch Log - Mutation too large while bootstrapping a newly added node > - > > Key: CASSANDRA-15152 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15152 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Batch Log >Reporter: Avraham Kalvo >Priority: Normal > > Scaling our six nodes cluster by three more nodes, we came upon behavior in > which bootstrap appears hung under `UJ` (two previously added were joined > within approximately 2.5 hours). > Examining the logs the following became apparent shortly after the bootstrap > process has commenced for this node: > ``` > ERROR [BatchlogTasks:1] 2019-06-05 14:43:46,508 CassandraDaemon.java:207 - > Exception in thread Thread[BatchlogTasks:1,5,main] > java.lang.IllegalArgumentException: Mutation of 108035175 bytes is too large > for the maximum size of 16777216 > at > org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:256) > ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:520) > ~[apache-cassandra-3.0.10.jar:3.0.10] > at > org.apache.cassandra.db.Keyspace.applyNotDeferrable(Keyspace.java:399) > ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.db.Mutation.apply(Mutation.java:213) > ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.db.Mutation.apply(Mutation.java:227) > ~[apache-cassandra-3.0.10.jar:3.0.10] > at > org.apache.cassandra.batchlog.BatchlogManager$ReplayingBatch.sendSingleReplayMutation(BatchlogManager.java:427) > ~[apache-cassandra-3.0.10.jar:3.0.10] > at > org.apache.cassandra.batchlog.BatchlogManager$ReplayingBatch.sendReplays(BatchlogManager.java:402) > ~[apache-cassandra-3.0.10.jar:3.0.10] > at > org.apache.cassandra.batchlog.BatchlogManager$ReplayingBatch.replay(BatchlogManager.java:318) > ~[apache-cassandra-3.0.10.jar:3.0.10] > at > org.apache.cassandra.batchlog.BatchlogManager.processBatchlogEntries(BatchlogManager.java:238) > ~[apache-cassandra-3.0.10.jar:3.0.10] > at > org.apache.cassandra.batchlog.BatchlogManager.replayFailedBatches(BatchlogManager.java:207) > ~[apache-cassandra-3.0.10.jar:3.0.10] > at > org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118) > ~[apache-cassandra-3.0.10.jar:3.0.10] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_201] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > [na:1.8.0_201] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > [na:1.8.0_201] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > [na:1.8.0_201] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [na:1.8.0_201] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [na:1.8.0_201] > at java.lang.Thread.run(Thread.java:748) [na:1.8.0_201] > ``` > And since then, repeating itself in the logs. > We decided to discard the newly added apparently still joining node by doing > the following: > 1. at first - simply restarting it, which resulted in it starting up > apparently normally > 2. then - decommission it by issuing `nodetool decommission`, this took long > (over 2.5 hours) and eventually was terminated by issuing `nodetool > removenode` > 3. node removal was hung on a specific token, which led us to complete it by > force. > 4. forcing the node removal has generated a corruption with one of the > `system.batches` table SSTables, which was removed (backed up) from its > underlying data dir as mitigation (78MB worth) > 5. cluster-wide repair was run > 6. `Mutation too large` error is now repeating itself in three different > permutations (alerted sizes) under three different nodes (our standard > replication factor is of three) > We're not sure whether we're hitting > https://issues.apache.org/jira/browse/CASSANDRA-11670 or not, as it's said to > be resolved in our current version of 3.0.10. > Still would like to verify what's the root cause for this? as we
[jira] [Created] (CASSANDRA-15631) Add AssertJ test dependency
Kevin Gallardo created CASSANDRA-15631: -- Summary: Add AssertJ test dependency Key: CASSANDRA-15631 URL: https://issues.apache.org/jira/browse/CASSANDRA-15631 Project: Cassandra Issue Type: Improvement Components: Test/dtest, Test/unit Reporter: Kevin Gallardo Assignee: Kevin Gallardo See [proposal|https://lists.apache.org/thread.html/rc562ec47578d0ae6f346ba9e3d7469c1cd3f8b521a72ddcb2accc47b%40%3Cdev.cassandra.apache.org%3E]. The goal is to add [AssertJ|https://assertj.github.io/doc/] to the test framework to allow for more comprehensible and easier to write tests. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15622) Unit tests throw UnknownHostException trying to use `InetAddress.getLocalHost()` instead of `FBUtilities.getLocalAddress()`
[
https://issues.apache.org/jira/browse/CASSANDRA-15622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15622:
---
Reviewers: Benjamin Lerer, Benjamin Lerer (was: Benjamin Lerer)
Benjamin Lerer, Benjamin Lerer (was: Benjamin Lerer)
Status: Review In Progress (was: Patch Available)
> Unit tests throw UnknownHostException trying to use
> `InetAddress.getLocalHost()` instead of `FBUtilities.getLocalAddress()`
> ---
>
> Key: CASSANDRA-15622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15622
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
>
> Many of the unit tests in Jenkins fail because of the use of
> {{`InetAddress.getLocalHost()`}} in the test classes.
> The Jenkins agents need a public ipaddress (and a hostname associated to it)
> so the Jenkins master can connect to them (agents can be hosted externally,
> by donating third-parties).
> The call to {{`InetAddress.getLocalHost()`}} can resolve to a hostname that
> can't be looked up.
> Not only it likely not listed in {{`/etc/hosts`}}, but we don't want it to be
> either (in case of accidental external port opening if the hostname points to
> the public ipaddress). (This is also ASF policy on this infrastructure.)
> The unit test code needs to replace these code occurrences with the call to
> {{`FBUtilities.getLocalAddress()`}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15622) Unit tests throw UnknownHostException trying to use `InetAddress.getLocalHost()` instead of `FBUtilities.getLocalAddress()`
[
https://issues.apache.org/jira/browse/CASSANDRA-15622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15622:
---
Status: Ready to Commit (was: Review In Progress)
> Unit tests throw UnknownHostException trying to use
> `InetAddress.getLocalHost()` instead of `FBUtilities.getLocalAddress()`
> ---
>
> Key: CASSANDRA-15622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15622
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
>
> Many of the unit tests in Jenkins fail because of the use of
> {{`InetAddress.getLocalHost()`}} in the test classes.
> The Jenkins agents need a public ipaddress (and a hostname associated to it)
> so the Jenkins master can connect to them (agents can be hosted externally,
> by donating third-parties).
> The call to {{`InetAddress.getLocalHost()`}} can resolve to a hostname that
> can't be looked up.
> Not only it likely not listed in {{`/etc/hosts`}}, but we don't want it to be
> either (in case of accidental external port opening if the hostname points to
> the public ipaddress). (This is also ASF policy on this infrastructure.)
> The unit test code needs to replace these code occurrences with the call to
> {{`FBUtilities.getLocalAddress()`}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15622) Unit tests throw UnknownHostException trying to use `InetAddress.getLocalHost()` instead of `FBUtilities.getLocalAddress()`
[
https://issues.apache.org/jira/browse/CASSANDRA-15622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17056934#comment-17056934
]
Benjamin Lerer commented on CASSANDRA-15622:
Sorry, for the delay. The patches look good.
> Unit tests throw UnknownHostException trying to use
> `InetAddress.getLocalHost()` instead of `FBUtilities.getLocalAddress()`
> ---
>
> Key: CASSANDRA-15622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15622
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
>
> Many of the unit tests in Jenkins fail because of the use of
> {{`InetAddress.getLocalHost()`}} in the test classes.
> The Jenkins agents need a public ipaddress (and a hostname associated to it)
> so the Jenkins master can connect to them (agents can be hosted externally,
> by donating third-parties).
> The call to {{`InetAddress.getLocalHost()`}} can resolve to a hostname that
> can't be looked up.
> Not only it likely not listed in {{`/etc/hosts`}}, but we don't want it to be
> either (in case of accidental external port opening if the hostname points to
> the public ipaddress). (This is also ASF policy on this infrastructure.)
> The unit test code needs to replace these code occurrences with the call to
> {{`FBUtilities.getLocalAddress()`}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15605) Broken dtest replication_test.py::TestSnitchConfigurationUpdate
[
https://issues.apache.org/jira/browse/CASSANDRA-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17056886#comment-17056886
]
Ryan Svihla commented on CASSANDRA-15605:
-
{{So I got them to finally fail, I had to get down to a 2 core VM before they
would:}}
{{test_rf_collapse_gossiping_property_file_snitch passed 1 out of the required
1 times. Success!}}
{{test_rf_expand_gossiping_property_file_snitch failed and was not selected for
rerun.}}
{{ }}
{{ Ran out of time waiting for topology to change on node 0}}
{{ [,
, ] }}
{{test_rf_collapse_property_file_snitch passed 1 out of the required 1 times.
Success!}}
{{test_rf_expand_property_file_snitch failed and was not selected for rerun.}}
{{ }}
{{ Ran out of time waiting for topology to change on node 0}}
{{ [,
, ] }}
{{test_cannot_restart_with_different_rack passed 1 out of the required 1 times.
Success!}}
{{test_failed_snitch_update_gossiping_property_file_snitch passed 1 out of the
required 1 times. Success! }}
{{test_failed_snitch_update_property_file_snitch passed 1 out of the required 1
times. Success!}}
{{test_switch_data_center_startup_fails passed 1 out of the required 1 times.
Success!}}
> Broken dtest replication_test.py::TestSnitchConfigurationUpdate
> ---
>
> Key: CASSANDRA-15605
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15605
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Sam Tunnicliffe
>Assignee: Ryan Svihla
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> Noticed this failing on a couple of CI runs and repros when running trunk
> locally and on CircleCI
> 2 or 3 tests are consistently failing:
> * {{test_rf_expand_gossiping_property_file_snitch}}
> * {{test_rf_expand_property_file_snitch}}
> * {{test_move_forwards_between_and_cleanup}}
> [https://circleci.com/workflow-run/f23f13a9-bbdc-4764-8336-109517e137f1]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15605) Broken dtest replication_test.py::TestSnitchConfigurationUpdate
[
https://issues.apache.org/jira/browse/CASSANDRA-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Svihla reassigned CASSANDRA-15605:
---
Assignee: Ryan Svihla
> Broken dtest replication_test.py::TestSnitchConfigurationUpdate
> ---
>
> Key: CASSANDRA-15605
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15605
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Sam Tunnicliffe
>Assignee: Ryan Svihla
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> Noticed this failing on a couple of CI runs and repros when running trunk
> locally and on CircleCI
> 2 or 3 tests are consistently failing:
> * {{test_rf_expand_gossiping_property_file_snitch}}
> * {{test_rf_expand_property_file_snitch}}
> * {{test_move_forwards_between_and_cleanup}}
> [https://circleci.com/workflow-run/f23f13a9-bbdc-4764-8336-109517e137f1]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15565) Fix flaky test org.apache.cassandra.index.internal.CassandraIndexTest indexCorrectlyMarkedAsBuildAndRemoved
[
https://issues.apache.org/jira/browse/CASSANDRA-15565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-15565:
Fix Version/s: 3.11.7
3.0.21
Since Version: 3.0.0
Source Control Link:
https://github.com/apache/cassandra/commit/c81d7dcc09fa1c35f190f0baad764fbe8a024679
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Thanks, looks good to me. I've made a small addition to your patch to verify
that when dropped, the index is removed from the built indexes table. From the
name, it seemed like that should've been part of the test all along. I've
committed to 3.0 and merged to 3.11 & trunk.
> Fix flaky test org.apache.cassandra.index.internal.CassandraIndexTest
> indexCorrectlyMarkedAsBuildAndRemoved
> ---
>
> Key: CASSANDRA-15565
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15565
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Jorge Bay
>Priority: Normal
> Labels: pull-request-available
> Fix For: 3.0.21, 3.11.7, 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> junit.framework.AssertionFailedError: Got more rows than expected. Expected 1
> but got 4.
> at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1098)
> at
> org.apache.cassandra.index.internal.CassandraIndexTest.indexCorrectlyMarkedAsBuildAndRemoved(CassandraIndexTest.java:499)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> The failure was seen on java 11.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (1916e5d -> 8ebe72d)
This is an automated email from the ASF dual-hosted git repository. samt pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 1916e5d Add UnleveledSSTables table level metric new c81d7dc Use result size for built index test new 8a5b127 Merge branch 'cassandra-3.0' into cassandra-3.11 new 8ebe72d Merge branch 'cassandra-3.11' into trunk The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../index/internal/CassandraIndexTest.java | 23 ++ 1 file changed, 19 insertions(+), 4 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 8a5b1275ff5ffb8e867a8ffa1558f12923c3f956
Merge: b714667 c81d7dc
Author: Sam Tunnicliffe
AuthorDate: Wed Mar 11 10:26:36 2020 +
Merge branch 'cassandra-3.0' into cassandra-3.11
.../index/internal/CassandraIndexTest.java | 23 ++
1 file changed, 19 insertions(+), 4 deletions(-)
diff --cc test/unit/org/apache/cassandra/index/internal/CassandraIndexTest.java
index 23b27b8,7cea9da..b573e6f
--- a/test/unit/org/apache/cassandra/index/internal/CassandraIndexTest.java
+++ b/test/unit/org/apache/cassandra/index/internal/CassandraIndexTest.java
@@@ -515,6 -488,12 +515,12 @@@ public class CassandraIndexTest extend
@Test
public void indexCorrectlyMarkedAsBuildAndRemoved() throws Throwable
{
+ String selectBuiltIndexesQuery = String.format("SELECT * FROM
%s.\"%s\"",
- SystemKeyspace.NAME,
++
SchemaConstants.SYSTEM_KEYSPACE_NAME,
+
SystemKeyspace.BUILT_INDEXES);
+ UntypedResultSet rs = execute(selectBuiltIndexesQuery);
+ int initialSize = rs.size();
+
String indexName = "build_remove_test_idx";
String tableName = createTable("CREATE TABLE %s (a int, b int, c int,
PRIMARY KEY (a, b))");
createIndex(String.format("CREATE INDEX %s ON %%s(c)", indexName));
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 8ebe72d853c18d62f3e50f39736330b2bf141518
Merge: 1916e5d 8a5b127
Author: Sam Tunnicliffe
AuthorDate: Wed Mar 11 10:29:33 2020 +
Merge branch 'cassandra-3.11' into trunk
.../index/internal/CassandraIndexTest.java | 23 ++
1 file changed, 19 insertions(+), 4 deletions(-)
diff --cc test/unit/org/apache/cassandra/index/internal/CassandraIndexTest.java
index 65c2417,b573e6f..bab2eb2
--- a/test/unit/org/apache/cassandra/index/internal/CassandraIndexTest.java
+++ b/test/unit/org/apache/cassandra/index/internal/CassandraIndexTest.java
@@@ -496,24 -526,32 +502,33 @@@ public class CassandraIndexTest extend
createIndex(String.format("CREATE INDEX %s ON %%s(c)", indexName));
waitForIndex(KEYSPACE, tableName, indexName);
// check that there are no other rows in the built indexes table
- assertRows(execute(String.format("SELECT * FROM %s.\"%s\"",
SchemaConstants.SYSTEM_KEYSPACE_NAME, SystemKeyspace.BUILT_INDEXES)),
-row(KEYSPACE, indexName, null));
+ rs = execute(selectBuiltIndexesQuery);
+ int sizeAfterBuild = rs.size();
-assertRowsIgnoringOrderAndExtra(rs, row(KEYSPACE, indexName));
++assertRowsIgnoringOrderAndExtra(rs, row(KEYSPACE, indexName, null));
// rebuild the index and verify the built status table
getCurrentColumnFamilyStore().rebuildSecondaryIndex(indexName);
waitForIndex(KEYSPACE, tableName, indexName);
// check that there are no other rows in the built indexes table
- assertRows(execute(String.format("SELECT * FROM %s.\"%s\"",
SchemaConstants.SYSTEM_KEYSPACE_NAME, SystemKeyspace.BUILT_INDEXES)),
-row(KEYSPACE, indexName, null ));
+ rs = execute(selectBuiltIndexesQuery);
+ assertEquals(sizeAfterBuild, rs.size());
-assertRowsIgnoringOrderAndExtra(rs, row(KEYSPACE, indexName));
++assertRowsIgnoringOrderAndExtra(rs, row(KEYSPACE, indexName, null));
+
+ // check that dropping the index removes it from the built indexes
table
+ dropIndex("DROP INDEX %s." + indexName);
+ rs = execute(selectBuiltIndexesQuery);
+ assertEquals(initialSize, rs.size());
+ rs.forEach(row ->
assertFalse(row.getString("table_name").equals(KEYSPACE) // table_name is
actually keyspace
+ &&
row.getString("index_name").equals(indexName)));
}
+
// this is slightly annoying, but we cannot read rows from the methods in
Util as
-// ReadCommand#executeInternal uses metadata retrieved via the cfId,
which the index
+// ReadCommand#executeLocally uses metadata retrieved via the tableId,
which the index
// CFS inherits from the base CFS. This has the 'wrong' partitioner (the
index table
// uses LocalPartition, the base table a real one, so we cannot read from
the index
-// table with executeInternal
+// table with executeLocally
private void assertIndexRowTtl(ColumnFamilyStore indexCfs, int
indexedValue, int ttl) throws Throwable
{
DecoratedKey indexKey =
indexCfs.decorateKey(ByteBufferUtil.bytes(indexedValue));
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15565) Fix flaky test org.apache.cassandra.index.internal.CassandraIndexTest indexCorrectlyMarkedAsBuildAndRemoved
[
https://issues.apache.org/jira/browse/CASSANDRA-15565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-15565:
Status: Ready to Commit (was: Review In Progress)
> Fix flaky test org.apache.cassandra.index.internal.CassandraIndexTest
> indexCorrectlyMarkedAsBuildAndRemoved
> ---
>
> Key: CASSANDRA-15565
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15565
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Jorge Bay
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> junit.framework.AssertionFailedError: Got more rows than expected. Expected 1
> but got 4.
> at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1098)
> at
> org.apache.cassandra.index.internal.CassandraIndexTest.indexCorrectlyMarkedAsBuildAndRemoved(CassandraIndexTest.java:499)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> The failure was seen on java 11.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated: Use result size for built index test
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-3.0 by this push:
new c81d7dc Use result size for built index test
c81d7dc is described below
commit c81d7dcc09fa1c35f190f0baad764fbe8a024679
Author: Jorge Bay Gondra
AuthorDate: Wed Mar 11 10:19:28 2020 +
Use result size for built index test
Patch by Jorge Bay Gondra; reviewed by Sam Tunnicliffe for CASSANDRA-15565
---
.../index/internal/CassandraIndexTest.java | 23 ++
1 file changed, 19 insertions(+), 4 deletions(-)
diff --git
a/test/unit/org/apache/cassandra/index/internal/CassandraIndexTest.java
b/test/unit/org/apache/cassandra/index/internal/CassandraIndexTest.java
index 286c418..7cea9da 100644
--- a/test/unit/org/apache/cassandra/index/internal/CassandraIndexTest.java
+++ b/test/unit/org/apache/cassandra/index/internal/CassandraIndexTest.java
@@ -488,21 +488,36 @@ public class CassandraIndexTest extends CQLTester
@Test
public void indexCorrectlyMarkedAsBuildAndRemoved() throws Throwable
{
+String selectBuiltIndexesQuery = String.format("SELECT * FROM
%s.\"%s\"",
+ SystemKeyspace.NAME,
+
SystemKeyspace.BUILT_INDEXES);
+UntypedResultSet rs = execute(selectBuiltIndexesQuery);
+int initialSize = rs.size();
+
String indexName = "build_remove_test_idx";
String tableName = createTable("CREATE TABLE %s (a int, b int, c int,
PRIMARY KEY (a, b))");
createIndex(String.format("CREATE INDEX %s ON %%s(c)", indexName));
waitForIndex(KEYSPACE, tableName, indexName);
// check that there are no other rows in the built indexes table
-assertRows(execute(String.format("SELECT * FROM %s.\"%s\"",
SystemKeyspace.NAME, SystemKeyspace.BUILT_INDEXES)),
- row(KEYSPACE, indexName));
+rs = execute(selectBuiltIndexesQuery);
+int sizeAfterBuild = rs.size();
+assertRowsIgnoringOrderAndExtra(rs, row(KEYSPACE, indexName));
// rebuild the index and verify the built status table
getCurrentColumnFamilyStore().rebuildSecondaryIndex(indexName);
waitForIndex(KEYSPACE, tableName, indexName);
// check that there are no other rows in the built indexes table
-assertRows(execute(String.format("SELECT * FROM %s.\"%s\"",
SystemKeyspace.NAME, SystemKeyspace.BUILT_INDEXES)),
- row(KEYSPACE, indexName));
+rs = execute(selectBuiltIndexesQuery);
+assertEquals(sizeAfterBuild, rs.size());
+assertRowsIgnoringOrderAndExtra(rs, row(KEYSPACE, indexName));
+
+// check that dropping the index removes it from the built indexes
table
+dropIndex("DROP INDEX %s." + indexName);
+rs = execute(selectBuiltIndexesQuery);
+assertEquals(initialSize, rs.size());
+rs.forEach(row ->
assertFalse(row.getString("table_name").equals(KEYSPACE) // table_name is
actually keyspace
+ &&
row.getString("index_name").equals(indexName)));
}
// this is slightly annoying, but we cannot read rows from the methods in
Util as
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (b714667 -> 8a5b127)
This is an automated email from the ASF dual-hosted git repository. samt pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from b714667 Merge branch 'cassandra-3.0' into cassandra-3.11 new c81d7dc Use result size for built index test new 8a5b127 Merge branch 'cassandra-3.0' into cassandra-3.11 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../index/internal/CassandraIndexTest.java | 23 ++ 1 file changed, 19 insertions(+), 4 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15565) Fix flaky test org.apache.cassandra.index.internal.CassandraIndexTest indexCorrectlyMarkedAsBuildAndRemoved
[
https://issues.apache.org/jira/browse/CASSANDRA-15565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-15565:
Reviewers: Sam Tunnicliffe, Sam Tunnicliffe (was: Sam Tunnicliffe)
Sam Tunnicliffe, Sam Tunnicliffe
Status: Review In Progress (was: Patch Available)
> Fix flaky test org.apache.cassandra.index.internal.CassandraIndexTest
> indexCorrectlyMarkedAsBuildAndRemoved
> ---
>
> Key: CASSANDRA-15565
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15565
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Jorge Bay
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> junit.framework.AssertionFailedError: Got more rows than expected. Expected 1
> but got 4.
> at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1098)
> at
> org.apache.cassandra.index.internal.CassandraIndexTest.indexCorrectlyMarkedAsBuildAndRemoved(CassandraIndexTest.java:499)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> The failure was seen on java 11.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15620) Add "unleveled sstables" table metric
[ https://issues.apache.org/jira/browse/CASSANDRA-15620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Podkowinski updated CASSANDRA-15620: --- Fix Version/s: 4.0-alpha Source Control Link: https://github.com/apache/cassandra/commit/1916e5d1ca6ed526a65c4fef7ff933de70006496 Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed as 1916e5d1ca6ed526a65c4fef7ff933de70006496 > Add "unleveled sstables" table metric > - > > Key: CASSANDRA-15620 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15620 > Project: Cassandra > Issue Type: Improvement > Components: Observability/Metrics >Reporter: Stefan Podkowinski >Assignee: Stefan Podkowinski >Priority: Normal > Fix For: 4.0-alpha > > > The number of unleveled sstables is an important indicator that deserves to > be a dedicated table metric on its own. This will also add a global gauge > that is convenient to query. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Add UnleveledSSTables table level metric
This is an automated email from the ASF dual-hosted git repository.
spod pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 1916e5d Add UnleveledSSTables table level metric
1916e5d is described below
commit 1916e5d1ca6ed526a65c4fef7ff933de70006496
Author: Stefan Podkowinski
AuthorDate: Tue Mar 3 13:09:43 2020 +0100
Add UnleveledSSTables table level metric
patch by Stefan Podkowinski; reviewed by Chris Lohfink for CASSANDRA-15620
---
CHANGES.txt | 1 +
src/java/org/apache/cassandra/metrics/TableMetrics.java | 12
2 files changed, 13 insertions(+)
diff --git a/CHANGES.txt b/CHANGES.txt
index e418db7..d863df2 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0-alpha4
+ * Added UnleveledSSTables global and table level metric (CASSANDRA-15620)
* Added Virtual Table exposing Cassandra relevant system properties
(CASSANDRA-15616)
* Add data modeling introduction (CASSANDRA-15481)
* Improve the algorithmic token allocation in case racks = RF
(CASSANDRA-15600)
diff --git a/src/java/org/apache/cassandra/metrics/TableMetrics.java
b/src/java/org/apache/cassandra/metrics/TableMetrics.java
index d8330cc..6095f50 100644
--- a/src/java/org/apache/cassandra/metrics/TableMetrics.java
+++ b/src/java/org/apache/cassandra/metrics/TableMetrics.java
@@ -218,6 +218,8 @@ public class TableMetrics
public final Counter additionalWrites;
public final Gauge additionalWriteLatencyNanos;
+public final Gauge unleveledSSTables;
+
/**
* Metrics for inconsistencies detected between repaired data sets across
replicas. These
* are tracked on the coordinator.
@@ -943,6 +945,16 @@ public class TableMetrics
confirmedRepairedInconsistencies =
createTableMeter("RepairedDataInconsistenciesConfirmed",
cfs.keyspace.metric.confirmedRepairedInconsistencies);
unconfirmedRepairedInconsistencies =
createTableMeter("RepairedDataInconsistenciesUnconfirmed",
cfs.keyspace.metric.unconfirmedRepairedInconsistencies);
+
+unleveledSSTables = createTableGauge("UnleveledSSTables",
cfs::getUnleveledSSTables, () -> {
+// global gauge
+int cnt = 0;
+for (Metric cfGauge : allTableMetrics.get("UnleveledSSTables"))
+{
+cnt += ((Gauge)
cfGauge).getValue().intValue();
+}
+return cnt;
+});
}
public void updateSSTableIterated(int count)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
