[jira] [Commented] (CASSANDRA-16061) transient_replication_ring_test.py::TestTransientReplicationRing::test_move_forwards_and_cleanup

[Berenguer Blasi (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281598#comment-17281598
 ] 

Berenguer Blasi commented on CASSANDRA-16061:
-

Anybody has seen this recently?

> transient_replication_ring_test.py::TestTransientReplicationRing::test_move_forwards_and_cleanup
> 
>
> Key: CASSANDRA-16061
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16061
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/python
>Reporter: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.x
>
>
> Failing here, also locally:
> [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/312/workflows/da4ce69c-e778-467e-b9f3-27ab166a8321/jobs/1945]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-16433) Version family is probably broken for Cassandra 2 and 3.11 in dtests

[Tomasz Lasica (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281592#comment-17281592
 ] 

Tomasz Lasica edited comment on CASSANDRA-16433 at 2/9/21, 7:45 AM:


I am not a big fun of making things complex, and rather to abstract reality as 
simple as it is.

So to me family should reflect  family... means:
 * 2.0.15 and 2.0.2 will be 2.0 family.
 * 3.11.8 and 3.11.9 will be 3.11 family.
 * 3.10.1 and 3.10.20 will be 3.10 family.
 * And 4.0.0  or 4.0.1. will be 4.0 family. 

Explicit is better than implicit. Versions are are added not often, so no need 
to be smart in the way upgrade tests will automatically switch to latest, 
because we will have to make sure we still have enough tests for both 
pre-latest and latest in such case.


was (Author: tomasz.lasica):
I am not a big fun of making things complex, and rather to abstract reality as 
simple as it is.

So to me family should reflect  family... means:
 * 2.0.15 and 2.0.2 will be 2.0 family.
 * 3.11.8 and 3.11.9 will be 3.11 family.
 * 3.10.1 and 3.10.20 will be 3.10 family.
 * And 4.0.0  or 4.0.1. will be 4.0 family.

 

Explicit is better than implicit. Versions are are added not often, so no need 
to be smart in the way upgrade tests will automatically switch to latest, 
because we will have to make sure we still have enough tests for both 
pre-latest and latest in such case.

> Version family is probably broken for Cassandra 2 and 3.11 in dtests
> 
>
> Key: CASSANDRA-16433
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16433
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/python
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> It looks like the version families are a bit confused - when we determine 
> version family for the current build, we do:
> {code:python}
> if current_version.vstring.startswith('2.0'):
> version_family = '2.0.x'
> elif current_version.vstring.startswith('2.1'):
> version_family = '2.1.x'
> elif current_version.vstring.startswith('2.2'):
> version_family = '2.2.x'
> elif current_version.vstring.startswith('3.0'):
> version_family = '3.0.x'
> elif '3.1' <= current_version < '4.0':
> version_family = '3.x'
> elif '4.0' <= current_version < '4.1':
> version_family = 'trunk'
> else:
> # when this occurs, it's time to update this manifest a bit!
> raise RuntimeError("4.1+ not yet supported on upgrade tests!")
> {code}
> but later, in the upgrade manifest we have:
> {code:python}
> indev_2_1_x = VersionMeta(name='indev_2_1_x', family='2.1', variant='indev', 
> version='github:apache/cassandra-2.1', min_proto_v=1, max_proto_v=3, 
> java_versions=(7, 8))
> current_2_1_x = VersionMeta(name='current_2_1_x', family='2.1', 
> variant='current', version='2.1.20', min_proto_v=1, max_proto_v=3, 
> java_versions=(7, 8))
> indev_2_2_x = VersionMeta(name='indev_2_2_x', family='2.2', variant='indev', 
> version='github:apache/cassandra-2.2', min_proto_v=1, max_proto_v=4, 
> java_versions=(7, 8))
> current_2_2_x = VersionMeta(name='current_2_2_x', family='2.2', 
> variant='current', version='2.2.13', min_proto_v=1, max_proto_v=4, 
> java_versions=(7, 8))
> indev_3_0_x = VersionMeta(name='indev_3_0_x', family='3.0', variant='indev', 
> version='github:apache/cassandra-3.0', min_proto_v=3, max_proto_v=4, 
> java_versions=(8,))
> current_3_0_x = VersionMeta(name='current_3_0_x', family='3.0', 
> variant='current', version='3.0.23', min_proto_v=3, max_proto_v=4, 
> java_versions=(8,))
> indev_3_11_x = VersionMeta(name='indev_3_11_x', family='3.11', 
> variant='indev', version='github:apache/cassandra-3.11', min_proto_v=3, 
> max_proto_v=4, java_versions=(8,))
> current_3_11_x = VersionMeta(name='current_3_11_x', family='3.11', 
> variant='current', version='3.11.9', min_proto_v=3, max_proto_v=4, 
> java_versions=(8,))
> {code}
> later on in the code we have some manifest filtering:
> {code:python}
> if filter_for_current_family and not 
> origin_meta.matches_current_env_version_family and not 
> destination_meta.matches_current_env_version_family:
> logger.debug("skipping class creation, origin version {} and 
> destination version {} do not match target version {}, and 
> --upgrade-target-version-only was set".format(origin_meta.name, 
> destination_meta.name, VERSION_FAMILY))
> continue
> {code}
> This does not cause any problems for {{trunk}}, but when I tried to run some 
> upgrade tests on 3.11 build, I could not do anything. 
> Therefore we need to either change families in manifest as follows:

[jira] [Comment Edited] (CASSANDRA-16433) Version family is probably broken for Cassandra 2 and 3.11 in dtests

[Tomasz Lasica (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281592#comment-17281592
 ] 

Tomasz Lasica edited comment on CASSANDRA-16433 at 2/9/21, 7:45 AM:


I am not a big fun of making things complex, and rather to abstract reality as 
simple as it is.

So to me family should reflect  family... means:
 * 2.0.15 and 2.0.2 will be 2.0 family.
 * 3.11.8 and 3.11.9 will be 3.11 family.
 * 3.10.1 and 3.10.20 will be 3.10 family.
 * And 4.0.0  or 4.0.1. will be 4.0 family. 

Explicit is better than implicit. Versions are are added not often, so no need 
to be smart in the way upgrade tests will automatically switch to latest, 
because we will have to make sure we still have enough tests for both 
pre-latest and latest in such case.

 

But I understand that unfortunate 3.0-4.0 -> 3.x was done some time ago.


was (Author: tomasz.lasica):
I am not a big fun of making things complex, and rather to abstract reality as 
simple as it is.

So to me family should reflect  family... means:
 * 2.0.15 and 2.0.2 will be 2.0 family.
 * 3.11.8 and 3.11.9 will be 3.11 family.
 * 3.10.1 and 3.10.20 will be 3.10 family.
 * And 4.0.0  or 4.0.1. will be 4.0 family. 

Explicit is better than implicit. Versions are are added not often, so no need 
to be smart in the way upgrade tests will automatically switch to latest, 
because we will have to make sure we still have enough tests for both 
pre-latest and latest in such case.

> Version family is probably broken for Cassandra 2 and 3.11 in dtests
> 
>
> Key: CASSANDRA-16433
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16433
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/python
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> It looks like the version families are a bit confused - when we determine 
> version family for the current build, we do:
> {code:python}
> if current_version.vstring.startswith('2.0'):
> version_family = '2.0.x'
> elif current_version.vstring.startswith('2.1'):
> version_family = '2.1.x'
> elif current_version.vstring.startswith('2.2'):
> version_family = '2.2.x'
> elif current_version.vstring.startswith('3.0'):
> version_family = '3.0.x'
> elif '3.1' <= current_version < '4.0':
> version_family = '3.x'
> elif '4.0' <= current_version < '4.1':
> version_family = 'trunk'
> else:
> # when this occurs, it's time to update this manifest a bit!
> raise RuntimeError("4.1+ not yet supported on upgrade tests!")
> {code}
> but later, in the upgrade manifest we have:
> {code:python}
> indev_2_1_x = VersionMeta(name='indev_2_1_x', family='2.1', variant='indev', 
> version='github:apache/cassandra-2.1', min_proto_v=1, max_proto_v=3, 
> java_versions=(7, 8))
> current_2_1_x = VersionMeta(name='current_2_1_x', family='2.1', 
> variant='current', version='2.1.20', min_proto_v=1, max_proto_v=3, 
> java_versions=(7, 8))
> indev_2_2_x = VersionMeta(name='indev_2_2_x', family='2.2', variant='indev', 
> version='github:apache/cassandra-2.2', min_proto_v=1, max_proto_v=4, 
> java_versions=(7, 8))
> current_2_2_x = VersionMeta(name='current_2_2_x', family='2.2', 
> variant='current', version='2.2.13', min_proto_v=1, max_proto_v=4, 
> java_versions=(7, 8))
> indev_3_0_x = VersionMeta(name='indev_3_0_x', family='3.0', variant='indev', 
> version='github:apache/cassandra-3.0', min_proto_v=3, max_proto_v=4, 
> java_versions=(8,))
> current_3_0_x = VersionMeta(name='current_3_0_x', family='3.0', 
> variant='current', version='3.0.23', min_proto_v=3, max_proto_v=4, 
> java_versions=(8,))
> indev_3_11_x = VersionMeta(name='indev_3_11_x', family='3.11', 
> variant='indev', version='github:apache/cassandra-3.11', min_proto_v=3, 
> max_proto_v=4, java_versions=(8,))
> current_3_11_x = VersionMeta(name='current_3_11_x', family='3.11', 
> variant='current', version='3.11.9', min_proto_v=3, max_proto_v=4, 
> java_versions=(8,))
> {code}
> later on in the code we have some manifest filtering:
> {code:python}
> if filter_for_current_family and not 
> origin_meta.matches_current_env_version_family and not 
> destination_meta.matches_current_env_version_family:
> logger.debug("skipping class creation, origin version {} and 
> destination version {} do not match target version {}, and 
> --upgrade-target-version-only was set".format(origin_meta.name, 
> destination_meta.name, VERSION_FAMILY))
> continue
> {code}
> This does not cause any problems for {{trunk}}, but when I tried to run some 
> upgrade tests on 3.11 build, I could not do 

[jira] [Comment Edited] (CASSANDRA-16433) Version family is probably broken for Cassandra 2 and 3.11 in dtests

[Tomasz Lasica (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281592#comment-17281592
 ] 

Tomasz Lasica edited comment on CASSANDRA-16433 at 2/9/21, 7:43 AM:


I am not a big fun of making things complex, and rather to abstract reality as 
simple as it is.

So to me family should reflect  family... means:
 * 2.0.15 and 2.0.2 will be 2.0 family.
 * 3.11.8 and 3.11.9 will be 3.11 family.
 * 3.10.1 and 3.10.20 will be 3.10 family.
 * And 4.0.0  or 4.0.1. will be 4.0 family.

 

Explicit is better than implicit. Versions are are added not often, so no need 
to be smart in the way upgrade tests will automatically switch to latest, 
because we will have to make sure we still have enough tests for both 
pre-latest and latest in such case.


was (Author: tomasz.lasica):
I am not a big fun of making things complex, and rather to abstract reality as 
simple as it is.

So to me family should reflect  family... means: 2.0.15 and 2.0.2 will be 
2.0 family. 3.11.8 and 3.11.9 will be 3.11 family. 3.10.1 and 3.10.20 will be 
3.10 family. And 4.0.0  or 4.0.1. will be 4.0 family.

 

Explicit is better than implicit. Versions are are added not often, so no need 
to be smart in the way upgrade tests will automatically switch to latest, 
because we will have to make sure we still have enough tests for both 
pre-latest and latest in such case.

> Version family is probably broken for Cassandra 2 and 3.11 in dtests
> 
>
> Key: CASSANDRA-16433
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16433
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/python
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> It looks like the version families are a bit confused - when we determine 
> version family for the current build, we do:
> {code:python}
> if current_version.vstring.startswith('2.0'):
> version_family = '2.0.x'
> elif current_version.vstring.startswith('2.1'):
> version_family = '2.1.x'
> elif current_version.vstring.startswith('2.2'):
> version_family = '2.2.x'
> elif current_version.vstring.startswith('3.0'):
> version_family = '3.0.x'
> elif '3.1' <= current_version < '4.0':
> version_family = '3.x'
> elif '4.0' <= current_version < '4.1':
> version_family = 'trunk'
> else:
> # when this occurs, it's time to update this manifest a bit!
> raise RuntimeError("4.1+ not yet supported on upgrade tests!")
> {code}
> but later, in the upgrade manifest we have:
> {code:python}
> indev_2_1_x = VersionMeta(name='indev_2_1_x', family='2.1', variant='indev', 
> version='github:apache/cassandra-2.1', min_proto_v=1, max_proto_v=3, 
> java_versions=(7, 8))
> current_2_1_x = VersionMeta(name='current_2_1_x', family='2.1', 
> variant='current', version='2.1.20', min_proto_v=1, max_proto_v=3, 
> java_versions=(7, 8))
> indev_2_2_x = VersionMeta(name='indev_2_2_x', family='2.2', variant='indev', 
> version='github:apache/cassandra-2.2', min_proto_v=1, max_proto_v=4, 
> java_versions=(7, 8))
> current_2_2_x = VersionMeta(name='current_2_2_x', family='2.2', 
> variant='current', version='2.2.13', min_proto_v=1, max_proto_v=4, 
> java_versions=(7, 8))
> indev_3_0_x = VersionMeta(name='indev_3_0_x', family='3.0', variant='indev', 
> version='github:apache/cassandra-3.0', min_proto_v=3, max_proto_v=4, 
> java_versions=(8,))
> current_3_0_x = VersionMeta(name='current_3_0_x', family='3.0', 
> variant='current', version='3.0.23', min_proto_v=3, max_proto_v=4, 
> java_versions=(8,))
> indev_3_11_x = VersionMeta(name='indev_3_11_x', family='3.11', 
> variant='indev', version='github:apache/cassandra-3.11', min_proto_v=3, 
> max_proto_v=4, java_versions=(8,))
> current_3_11_x = VersionMeta(name='current_3_11_x', family='3.11', 
> variant='current', version='3.11.9', min_proto_v=3, max_proto_v=4, 
> java_versions=(8,))
> {code}
> later on in the code we have some manifest filtering:
> {code:python}
> if filter_for_current_family and not 
> origin_meta.matches_current_env_version_family and not 
> destination_meta.matches_current_env_version_family:
> logger.debug("skipping class creation, origin version {} and 
> destination version {} do not match target version {}, and 
> --upgrade-target-version-only was set".format(origin_meta.name, 
> destination_meta.name, VERSION_FAMILY))
> continue
> {code}
> This does not cause any problems for {{trunk}}, but when I tried to run some 
> upgrade tests on 3.11 build, I could not do anything. 
> Therefore we need to either change families in manifest as follows:
> - 2.1 

[jira] [Commented] (CASSANDRA-16433) Version family is probably broken for Cassandra 2 and 3.11 in dtests

[Tomasz Lasica (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281592#comment-17281592
 ] 

Tomasz Lasica commented on CASSANDRA-16433:
---

I am not a big fun of making things complex, and rather to abstract reality as 
simple as it is.

So to me family should reflect  family... means: 2.0.15 and 2.0.2 will be 
2.0 family. 3.11.8 and 3.11.9 will be 3.11 family. 3.10.1 and 3.10.20 will be 
3.10 family. And 4.0.0  or 4.0.1. will be 4.0 family.

 

Explicit is better than implicit. Versions are are added not often, so no need 
to be smart in the way upgrade tests will automatically switch to latest, 
because we will have to make sure we still have enough tests for both 
pre-latest and latest in such case.

> Version family is probably broken for Cassandra 2 and 3.11 in dtests
> 
>
> Key: CASSANDRA-16433
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16433
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/python
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> It looks like the version families are a bit confused - when we determine 
> version family for the current build, we do:
> {code:python}
> if current_version.vstring.startswith('2.0'):
> version_family = '2.0.x'
> elif current_version.vstring.startswith('2.1'):
> version_family = '2.1.x'
> elif current_version.vstring.startswith('2.2'):
> version_family = '2.2.x'
> elif current_version.vstring.startswith('3.0'):
> version_family = '3.0.x'
> elif '3.1' <= current_version < '4.0':
> version_family = '3.x'
> elif '4.0' <= current_version < '4.1':
> version_family = 'trunk'
> else:
> # when this occurs, it's time to update this manifest a bit!
> raise RuntimeError("4.1+ not yet supported on upgrade tests!")
> {code}
> but later, in the upgrade manifest we have:
> {code:python}
> indev_2_1_x = VersionMeta(name='indev_2_1_x', family='2.1', variant='indev', 
> version='github:apache/cassandra-2.1', min_proto_v=1, max_proto_v=3, 
> java_versions=(7, 8))
> current_2_1_x = VersionMeta(name='current_2_1_x', family='2.1', 
> variant='current', version='2.1.20', min_proto_v=1, max_proto_v=3, 
> java_versions=(7, 8))
> indev_2_2_x = VersionMeta(name='indev_2_2_x', family='2.2', variant='indev', 
> version='github:apache/cassandra-2.2', min_proto_v=1, max_proto_v=4, 
> java_versions=(7, 8))
> current_2_2_x = VersionMeta(name='current_2_2_x', family='2.2', 
> variant='current', version='2.2.13', min_proto_v=1, max_proto_v=4, 
> java_versions=(7, 8))
> indev_3_0_x = VersionMeta(name='indev_3_0_x', family='3.0', variant='indev', 
> version='github:apache/cassandra-3.0', min_proto_v=3, max_proto_v=4, 
> java_versions=(8,))
> current_3_0_x = VersionMeta(name='current_3_0_x', family='3.0', 
> variant='current', version='3.0.23', min_proto_v=3, max_proto_v=4, 
> java_versions=(8,))
> indev_3_11_x = VersionMeta(name='indev_3_11_x', family='3.11', 
> variant='indev', version='github:apache/cassandra-3.11', min_proto_v=3, 
> max_proto_v=4, java_versions=(8,))
> current_3_11_x = VersionMeta(name='current_3_11_x', family='3.11', 
> variant='current', version='3.11.9', min_proto_v=3, max_proto_v=4, 
> java_versions=(8,))
> {code}
> later on in the code we have some manifest filtering:
> {code:python}
> if filter_for_current_family and not 
> origin_meta.matches_current_env_version_family and not 
> destination_meta.matches_current_env_version_family:
> logger.debug("skipping class creation, origin version {} and 
> destination version {} do not match target version {}, and 
> --upgrade-target-version-only was set".format(origin_meta.name, 
> destination_meta.name, VERSION_FAMILY))
> continue
> {code}
> This does not cause any problems for {{trunk}}, but when I tried to run some 
> upgrade tests on 3.11 build, I could not do anything. 
> Therefore we need to either change families in manifest as follows:
> - 2.1 -> 2.1.x
> - 2.2 -> 2.2.x
> - 3.0 -> 3.0.x
> - 3.11 -> 3.x
> or change how we assign version family for the current build



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16415) Broken test mixedModeReadColumnSubsetDigestCheck - org.apache.cassandra.distributed.upgrade.MixedModeReadTest

[Jacek Lewandowski (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281571#comment-17281571
 ] 

Jacek Lewandowski commented on CASSANDRA-16415:
---

:facepalm

You are absolutely right. I've blindly copied the test from trunk. That test 
does not use gossip. Let me fix that...

> Broken test mixedModeReadColumnSubsetDigestCheck - 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest
> -
>
> Key: CASSANDRA-16415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16415
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination, Test/dtest/java
>Reporter: Yifan Cai
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The test has been failing and can always be reproduced in the recent CI. 
> Stack trace: 
> {code:java}
> junit.framework.AssertionFailedError: Found Digest Mismatch
>  at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.checkTraceForDigestMismatch(MixedModeReadTest.java:89)
>  at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.lambda$mixedModeReadColumnSubsetDigestCheck$0(MixedModeReadTest.java:63)
>  at 
> org.apache.cassandra.distributed.upgrade.UpgradeTestBase$TestCase.run(UpgradeTestBase.java:171)
>  at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.mixedModeReadColumnSubsetDigestCheck(MixedModeReadTest.java:76)
>  {code}
> The initial investigation shows that 
>  * The test only fails in the setup phase of 
> mixedModeReadColumnSubsetDigestCheck. The cluster version is 
> *Versions.Major.v3X*
>  * The test failure is likely a consequence of CASSANDRA-15962. After 
> dropping the commit in branch cassandra-3.11 and rebuild the dtest jar, the 
> upgrade test can pass. Meanwhile, dropping the other commits does not help. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16433) Version family is probably broken for Cassandra 2 and 3.11 in dtests

[Jacek Lewandowski (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacek Lewandowski updated CASSANDRA-16433:
--
Test and Documentation Plan: run upgrade tests on 3.11
 Status: Patch Available  (was: In Progress)

> Version family is probably broken for Cassandra 2 and 3.11 in dtests
> 
>
> Key: CASSANDRA-16433
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16433
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/python
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> It looks like the version families are a bit confused - when we determine 
> version family for the current build, we do:
> {code:python}
> if current_version.vstring.startswith('2.0'):
> version_family = '2.0.x'
> elif current_version.vstring.startswith('2.1'):
> version_family = '2.1.x'
> elif current_version.vstring.startswith('2.2'):
> version_family = '2.2.x'
> elif current_version.vstring.startswith('3.0'):
> version_family = '3.0.x'
> elif '3.1' <= current_version < '4.0':
> version_family = '3.x'
> elif '4.0' <= current_version < '4.1':
> version_family = 'trunk'
> else:
> # when this occurs, it's time to update this manifest a bit!
> raise RuntimeError("4.1+ not yet supported on upgrade tests!")
> {code}
> but later, in the upgrade manifest we have:
> {code:python}
> indev_2_1_x = VersionMeta(name='indev_2_1_x', family='2.1', variant='indev', 
> version='github:apache/cassandra-2.1', min_proto_v=1, max_proto_v=3, 
> java_versions=(7, 8))
> current_2_1_x = VersionMeta(name='current_2_1_x', family='2.1', 
> variant='current', version='2.1.20', min_proto_v=1, max_proto_v=3, 
> java_versions=(7, 8))
> indev_2_2_x = VersionMeta(name='indev_2_2_x', family='2.2', variant='indev', 
> version='github:apache/cassandra-2.2', min_proto_v=1, max_proto_v=4, 
> java_versions=(7, 8))
> current_2_2_x = VersionMeta(name='current_2_2_x', family='2.2', 
> variant='current', version='2.2.13', min_proto_v=1, max_proto_v=4, 
> java_versions=(7, 8))
> indev_3_0_x = VersionMeta(name='indev_3_0_x', family='3.0', variant='indev', 
> version='github:apache/cassandra-3.0', min_proto_v=3, max_proto_v=4, 
> java_versions=(8,))
> current_3_0_x = VersionMeta(name='current_3_0_x', family='3.0', 
> variant='current', version='3.0.23', min_proto_v=3, max_proto_v=4, 
> java_versions=(8,))
> indev_3_11_x = VersionMeta(name='indev_3_11_x', family='3.11', 
> variant='indev', version='github:apache/cassandra-3.11', min_proto_v=3, 
> max_proto_v=4, java_versions=(8,))
> current_3_11_x = VersionMeta(name='current_3_11_x', family='3.11', 
> variant='current', version='3.11.9', min_proto_v=3, max_proto_v=4, 
> java_versions=(8,))
> {code}
> later on in the code we have some manifest filtering:
> {code:python}
> if filter_for_current_family and not 
> origin_meta.matches_current_env_version_family and not 
> destination_meta.matches_current_env_version_family:
> logger.debug("skipping class creation, origin version {} and 
> destination version {} do not match target version {}, and 
> --upgrade-target-version-only was set".format(origin_meta.name, 
> destination_meta.name, VERSION_FAMILY))
> continue
> {code}
> This does not cause any problems for {{trunk}}, but when I tried to run some 
> upgrade tests on 3.11 build, I could not do anything. 
> Therefore we need to either change families in manifest as follows:
> - 2.1 -> 2.1.x
> - 2.2 -> 2.2.x
> - 3.0 -> 3.0.x
> - 3.11 -> 3.x
> or change how we assign version family for the current build



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16433) Version family is probably broken for Cassandra 2 and 3.11 in dtests

[Jacek Lewandowski (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacek Lewandowski updated CASSANDRA-16433:
--
Source Control Link: https://github.com/apache/cassandra-dtest/pull/122

> Version family is probably broken for Cassandra 2 and 3.11 in dtests
> 
>
> Key: CASSANDRA-16433
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16433
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/python
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> It looks like the version families are a bit confused - when we determine 
> version family for the current build, we do:
> {code:python}
> if current_version.vstring.startswith('2.0'):
> version_family = '2.0.x'
> elif current_version.vstring.startswith('2.1'):
> version_family = '2.1.x'
> elif current_version.vstring.startswith('2.2'):
> version_family = '2.2.x'
> elif current_version.vstring.startswith('3.0'):
> version_family = '3.0.x'
> elif '3.1' <= current_version < '4.0':
> version_family = '3.x'
> elif '4.0' <= current_version < '4.1':
> version_family = 'trunk'
> else:
> # when this occurs, it's time to update this manifest a bit!
> raise RuntimeError("4.1+ not yet supported on upgrade tests!")
> {code}
> but later, in the upgrade manifest we have:
> {code:python}
> indev_2_1_x = VersionMeta(name='indev_2_1_x', family='2.1', variant='indev', 
> version='github:apache/cassandra-2.1', min_proto_v=1, max_proto_v=3, 
> java_versions=(7, 8))
> current_2_1_x = VersionMeta(name='current_2_1_x', family='2.1', 
> variant='current', version='2.1.20', min_proto_v=1, max_proto_v=3, 
> java_versions=(7, 8))
> indev_2_2_x = VersionMeta(name='indev_2_2_x', family='2.2', variant='indev', 
> version='github:apache/cassandra-2.2', min_proto_v=1, max_proto_v=4, 
> java_versions=(7, 8))
> current_2_2_x = VersionMeta(name='current_2_2_x', family='2.2', 
> variant='current', version='2.2.13', min_proto_v=1, max_proto_v=4, 
> java_versions=(7, 8))
> indev_3_0_x = VersionMeta(name='indev_3_0_x', family='3.0', variant='indev', 
> version='github:apache/cassandra-3.0', min_proto_v=3, max_proto_v=4, 
> java_versions=(8,))
> current_3_0_x = VersionMeta(name='current_3_0_x', family='3.0', 
> variant='current', version='3.0.23', min_proto_v=3, max_proto_v=4, 
> java_versions=(8,))
> indev_3_11_x = VersionMeta(name='indev_3_11_x', family='3.11', 
> variant='indev', version='github:apache/cassandra-3.11', min_proto_v=3, 
> max_proto_v=4, java_versions=(8,))
> current_3_11_x = VersionMeta(name='current_3_11_x', family='3.11', 
> variant='current', version='3.11.9', min_proto_v=3, max_proto_v=4, 
> java_versions=(8,))
> {code}
> later on in the code we have some manifest filtering:
> {code:python}
> if filter_for_current_family and not 
> origin_meta.matches_current_env_version_family and not 
> destination_meta.matches_current_env_version_family:
> logger.debug("skipping class creation, origin version {} and 
> destination version {} do not match target version {}, and 
> --upgrade-target-version-only was set".format(origin_meta.name, 
> destination_meta.name, VERSION_FAMILY))
> continue
> {code}
> This does not cause any problems for {{trunk}}, but when I tried to run some 
> upgrade tests on 3.11 build, I could not do anything. 
> Therefore we need to either change families in manifest as follows:
> - 2.1 -> 2.1.x
> - 2.2 -> 2.2.x
> - 3.0 -> 3.0.x
> - 3.11 -> 3.x
> or change how we assign version family for the current build



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16433) Version family is probably broken for Cassandra 2 and 3.11 in dtests

[Jacek Lewandowski (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacek Lewandowski updated CASSANDRA-16433:
--
 Bug Category: Parent values: Code(13163)Level 1 values: Bug - Unclear 
Impact(13164)
   Complexity: Low Hanging Fruit
Discovered By: User Report
 Severity: Low
   Status: Open  (was: Triage Needed)

> Version family is probably broken for Cassandra 2 and 3.11 in dtests
> 
>
> Key: CASSANDRA-16433
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16433
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/python
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> It looks like the version families are a bit confused - when we determine 
> version family for the current build, we do:
> {code:python}
> if current_version.vstring.startswith('2.0'):
> version_family = '2.0.x'
> elif current_version.vstring.startswith('2.1'):
> version_family = '2.1.x'
> elif current_version.vstring.startswith('2.2'):
> version_family = '2.2.x'
> elif current_version.vstring.startswith('3.0'):
> version_family = '3.0.x'
> elif '3.1' <= current_version < '4.0':
> version_family = '3.x'
> elif '4.0' <= current_version < '4.1':
> version_family = 'trunk'
> else:
> # when this occurs, it's time to update this manifest a bit!
> raise RuntimeError("4.1+ not yet supported on upgrade tests!")
> {code}
> but later, in the upgrade manifest we have:
> {code:python}
> indev_2_1_x = VersionMeta(name='indev_2_1_x', family='2.1', variant='indev', 
> version='github:apache/cassandra-2.1', min_proto_v=1, max_proto_v=3, 
> java_versions=(7, 8))
> current_2_1_x = VersionMeta(name='current_2_1_x', family='2.1', 
> variant='current', version='2.1.20', min_proto_v=1, max_proto_v=3, 
> java_versions=(7, 8))
> indev_2_2_x = VersionMeta(name='indev_2_2_x', family='2.2', variant='indev', 
> version='github:apache/cassandra-2.2', min_proto_v=1, max_proto_v=4, 
> java_versions=(7, 8))
> current_2_2_x = VersionMeta(name='current_2_2_x', family='2.2', 
> variant='current', version='2.2.13', min_proto_v=1, max_proto_v=4, 
> java_versions=(7, 8))
> indev_3_0_x = VersionMeta(name='indev_3_0_x', family='3.0', variant='indev', 
> version='github:apache/cassandra-3.0', min_proto_v=3, max_proto_v=4, 
> java_versions=(8,))
> current_3_0_x = VersionMeta(name='current_3_0_x', family='3.0', 
> variant='current', version='3.0.23', min_proto_v=3, max_proto_v=4, 
> java_versions=(8,))
> indev_3_11_x = VersionMeta(name='indev_3_11_x', family='3.11', 
> variant='indev', version='github:apache/cassandra-3.11', min_proto_v=3, 
> max_proto_v=4, java_versions=(8,))
> current_3_11_x = VersionMeta(name='current_3_11_x', family='3.11', 
> variant='current', version='3.11.9', min_proto_v=3, max_proto_v=4, 
> java_versions=(8,))
> {code}
> later on in the code we have some manifest filtering:
> {code:python}
> if filter_for_current_family and not 
> origin_meta.matches_current_env_version_family and not 
> destination_meta.matches_current_env_version_family:
> logger.debug("skipping class creation, origin version {} and 
> destination version {} do not match target version {}, and 
> --upgrade-target-version-only was set".format(origin_meta.name, 
> destination_meta.name, VERSION_FAMILY))
> continue
> {code}
> This does not cause any problems for {{trunk}}, but when I tried to run some 
> upgrade tests on 3.11 build, I could not do anything. 
> Therefore we need to either change families in manifest as follows:
> - 2.1 -> 2.1.x
> - 2.2 -> 2.2.x
> - 3.0 -> 3.0.x
> - 3.11 -> 3.x
> or change how we assign version family for the current build



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-16433) Version family is probably broken for Cassandra 2 and 3.11 in dtests

[Jacek Lewandowski (Jira)]

Jacek Lewandowski created CASSANDRA-16433:
-

 Summary: Version family is probably broken for Cassandra 2 and 
3.11 in dtests
 Key: CASSANDRA-16433
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16433
 Project: Cassandra
  Issue Type: Bug
  Components: Test/dtest/python
Reporter: Jacek Lewandowski
Assignee: Jacek Lewandowski


It looks like the version families are a bit confused - when we determine 
version family for the current build, we do:

{code:python}
if current_version.vstring.startswith('2.0'):
version_family = '2.0.x'
elif current_version.vstring.startswith('2.1'):
version_family = '2.1.x'
elif current_version.vstring.startswith('2.2'):
version_family = '2.2.x'
elif current_version.vstring.startswith('3.0'):
version_family = '3.0.x'
elif '3.1' <= current_version < '4.0':
version_family = '3.x'
elif '4.0' <= current_version < '4.1':
version_family = 'trunk'
else:
# when this occurs, it's time to update this manifest a bit!
raise RuntimeError("4.1+ not yet supported on upgrade tests!")
{code}

but later, in the upgrade manifest we have:

{code:python}
indev_2_1_x = VersionMeta(name='indev_2_1_x', family='2.1', variant='indev', 
version='github:apache/cassandra-2.1', min_proto_v=1, max_proto_v=3, 
java_versions=(7, 8))
current_2_1_x = VersionMeta(name='current_2_1_x', family='2.1', 
variant='current', version='2.1.20', min_proto_v=1, max_proto_v=3, 
java_versions=(7, 8))

indev_2_2_x = VersionMeta(name='indev_2_2_x', family='2.2', variant='indev', 
version='github:apache/cassandra-2.2', min_proto_v=1, max_proto_v=4, 
java_versions=(7, 8))
current_2_2_x = VersionMeta(name='current_2_2_x', family='2.2', 
variant='current', version='2.2.13', min_proto_v=1, max_proto_v=4, 
java_versions=(7, 8))

indev_3_0_x = VersionMeta(name='indev_3_0_x', family='3.0', variant='indev', 
version='github:apache/cassandra-3.0', min_proto_v=3, max_proto_v=4, 
java_versions=(8,))
current_3_0_x = VersionMeta(name='current_3_0_x', family='3.0', 
variant='current', version='3.0.23', min_proto_v=3, max_proto_v=4, 
java_versions=(8,))

indev_3_11_x = VersionMeta(name='indev_3_11_x', family='3.11', variant='indev', 
version='github:apache/cassandra-3.11', min_proto_v=3, max_proto_v=4, 
java_versions=(8,))
current_3_11_x = VersionMeta(name='current_3_11_x', family='3.11', 
variant='current', version='3.11.9', min_proto_v=3, max_proto_v=4, 
java_versions=(8,))
{code}

later on in the code we have some manifest filtering:

{code:python}
if filter_for_current_family and not 
origin_meta.matches_current_env_version_family and not 
destination_meta.matches_current_env_version_family:
logger.debug("skipping class creation, origin version {} and 
destination version {} do not match target version {}, and 
--upgrade-target-version-only was set".format(origin_meta.name, 
destination_meta.name, VERSION_FAMILY))
continue
{code}

This does not cause any problems for {{trunk}}, but when I tried to run some 
upgrade tests on 3.11 build, I could not do anything. 

Therefore we need to either change families in manifest as follows:
- 2.1 -> 2.1.x
- 2.2 -> 2.2.x
- 3.0 -> 3.0.x
- 3.11 -> 3.x

or change how we assign version family for the current build





--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16415) Broken test mixedModeReadColumnSubsetDigestCheck - org.apache.cassandra.distributed.upgrade.MixedModeReadTest

[Jacek Lewandowski (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281556#comment-17281556
 ] 

Jacek Lewandowski commented on CASSANDRA-16415:
---

[~yifanc] thank you for checking that. Maybe I'm doing something wrong with how 
I run that test. All in all, I rewrote that test as Python dtest and it passed 
without making any changes in production code. However the new test was failing 
because there was a bug in {{ColumnFilter}}. I fixed the bug and run both 
tests, both passed.


> Broken test mixedModeReadColumnSubsetDigestCheck - 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest
> -
>
> Key: CASSANDRA-16415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16415
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination, Test/dtest/java
>Reporter: Yifan Cai
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The test has been failing and can always be reproduced in the recent CI. 
> Stack trace: 
> {code:java}
> junit.framework.AssertionFailedError: Found Digest Mismatch
>  at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.checkTraceForDigestMismatch(MixedModeReadTest.java:89)
>  at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.lambda$mixedModeReadColumnSubsetDigestCheck$0(MixedModeReadTest.java:63)
>  at 
> org.apache.cassandra.distributed.upgrade.UpgradeTestBase$TestCase.run(UpgradeTestBase.java:171)
>  at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.mixedModeReadColumnSubsetDigestCheck(MixedModeReadTest.java:76)
>  {code}
> The initial investigation shows that 
>  * The test only fails in the setup phase of 
> mixedModeReadColumnSubsetDigestCheck. The cluster version is 
> *Versions.Major.v3X*
>  * The test failure is likely a consequence of CASSANDRA-15962. After 
> dropping the commit in branch cassandra-3.11 and rebuild the dtest jar, the 
> upgrade test can pass. Meanwhile, dropping the other commits does not help. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16411) Fix topology corruption on joining nodes without DC in dtests

[Berenguer Blasi (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Berenguer Blasi updated CASSANDRA-16411:

Status: Ready to Commit  (was: Review In Progress)

> Fix topology corruption on joining nodes without DC in dtests
> -
>
> Key: CASSANDRA-16411
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16411
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/python
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Dtests with multi-dc setups don't update the topology properties file of a 
> newly joining node without dc post cluster creation.
> The new node will join dc1 but it's snitch will report the default snitch 
> config as it doesn't get updated == dc1 == all nodes are in dc1 == there are 
> no other dcs. That leads to bootstrap, token, startup,... problems. Let's see 
> an example:
> - populate [1, 1, 1]
> - Node1 sees: dc1/ip1, dc2/ip2 and dc3/ip3
> - Node2 sees: dc1/ip1, dc2/ip2 and dc3/ip3
> - Node3 sees: dc1/ip1, dc2/ip2 and dc3/ip3
> - Add a new Node4
> - Node4 sees: dc1/ip1+ip2+ip3+ip4
> All multi-dc dtests that add a node after cluster creation suffer this 
> problem.
> Solution: When a new node is added without specifying a dc in dtests check 
> for existing dcs. If there are any force to specify one to update the 
> topology file.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16411) Fix topology corruption on joining nodes without DC in dtests

[Berenguer Blasi (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281531#comment-17281531
 ] 

Berenguer Blasi commented on CASSANDRA-16411:
-

I _think_ this is +1 from [~e.dimitrova] and [~adelapena] so we can move to 
commit? Please correct me if I am wrong.

> Fix topology corruption on joining nodes without DC in dtests
> -
>
> Key: CASSANDRA-16411
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16411
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/python
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Dtests with multi-dc setups don't update the topology properties file of a 
> newly joining node without dc post cluster creation.
> The new node will join dc1 but it's snitch will report the default snitch 
> config as it doesn't get updated == dc1 == all nodes are in dc1 == there are 
> no other dcs. That leads to bootstrap, token, startup,... problems. Let's see 
> an example:
> - populate [1, 1, 1]
> - Node1 sees: dc1/ip1, dc2/ip2 and dc3/ip3
> - Node2 sees: dc1/ip1, dc2/ip2 and dc3/ip3
> - Node3 sees: dc1/ip1, dc2/ip2 and dc3/ip3
> - Add a new Node4
> - Node4 sees: dc1/ip1+ip2+ip3+ip4
> All multi-dc dtests that add a node after cluster creation suffer this 
> problem.
> Solution: When a new node is added without specifying a dc in dtests check 
> for existing dcs. If there are any force to specify one to update the 
> topology file.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16310) Track top partitions (by size and tombstone count) and display in nodetool tablestats

[Yifan Cai (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yifan Cai updated CASSANDRA-16310:
--
Reviewers: Yifan Cai
   Status: Review In Progress  (was: Patch Available)

# Having setters for {{max_top_size_partition_count}} and 
{{max_top_tombstone_partition_count}} suggests that we want to tweak it at 
runtime. However, the property consumers, {{TopPartitionTracker}} and 
{{ColumnFamilyStore}} do not update the value once initialized. So it has no 
effects when operator update the values.
 # {{TopPartitionsTest}} only has the test cases for full repair. Can we add 
tests that run with preview repair? (I simply replaced the option in the 
nodetool commands in the tests.)
 ## The tests cases pass with the {{--preview}} option.
 ## They fail with the {{--validate}} option. No sstables are selected since 
the corresponding {{PreviewKind.REPAIRED}} filters out unrepaired ones. 
# The todo comment at TableStatsPrinter line#131 can be removed, since it is 
implemented. 
# Nit: consider adding a {{isValidation()}} method to {{PreviewKind}}. So in 
{{ValidationManager#doValidation}}, we can have 
{{validator.getPreviewKind().isValidation()}}, instead of 
{{validator.getPreviewKind() == PreviewKind.REPAIRED}}, which is less readable. 
# Document the feature. 

> Track top partitions (by size and tombstone count) and display in nodetool 
> tablestats
> -
>
> Key: CASSANDRA-16310
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16310
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Other
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 4.x
>
>
> We should track the top partitions by size and tombstone count and display in 
> {{nodetool tablestats}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16432) Fix flaky test mixedModeReadColumnSubsetDigestCheck - org.apache.cassandra.distributed.upgrade.MixedModeReadTest

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16432:
-
Resolution: Fixed
Status: Resolved  (was: Open)

> Fix flaky test mixedModeReadColumnSubsetDigestCheck - 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest
> 
>
> Key: CASSANDRA-16432
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16432
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/860/workflows/bde81400-ed6d-4d9c-8447-3dd33b14cddd/jobs/5117
> {code}
> junit.framework.AssertionFailedError: Found Digest Mismatch
>   at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.checkTraceForDigestMismatch(MixedModeReadTest.java:89)
>   at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.lambda$mixedModeReadColumnSubsetDigestCheck$0(MixedModeReadTest.java:63)
>   at 
> org.apache.cassandra.distributed.upgrade.UpgradeTestBase$TestCase.run(UpgradeTestBase.java:171)
>   at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.mixedModeReadColumnSubsetDigestCheck(MixedModeReadTest.java:76)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16432) Fix flaky test mixedModeReadColumnSubsetDigestCheck - org.apache.cassandra.distributed.upgrade.MixedModeReadTest

[David Capwell (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-16432:
--
 Bug Category: Parent values: Correctness(12982)Level 1 values: Test 
Failure(12990)
   Complexity: Normal
Discovered By: Unit Test
Fix Version/s: 4.0-beta
 Severity: Normal
   Status: Open  (was: Triage Needed)

> Fix flaky test mixedModeReadColumnSubsetDigestCheck - 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest
> 
>
> Key: CASSANDRA-16432
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16432
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/860/workflows/bde81400-ed6d-4d9c-8447-3dd33b14cddd/jobs/5117
> {code}
> junit.framework.AssertionFailedError: Found Digest Mismatch
>   at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.checkTraceForDigestMismatch(MixedModeReadTest.java:89)
>   at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.lambda$mixedModeReadColumnSubsetDigestCheck$0(MixedModeReadTest.java:63)
>   at 
> org.apache.cassandra.distributed.upgrade.UpgradeTestBase$TestCase.run(UpgradeTestBase.java:171)
>   at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.mixedModeReadColumnSubsetDigestCheck(MixedModeReadTest.java:76)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-16432) Fix flaky test mixedModeReadColumnSubsetDigestCheck - org.apache.cassandra.distributed.upgrade.MixedModeReadTest

[David Capwell (Jira)]

David Capwell created CASSANDRA-16432:
-

 Summary: Fix flaky test mixedModeReadColumnSubsetDigestCheck - 
org.apache.cassandra.distributed.upgrade.MixedModeReadTest
 Key: CASSANDRA-16432
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16432
 Project: Cassandra
  Issue Type: Bug
  Components: Test/dtest/java
Reporter: David Capwell


https://app.circleci.com/pipelines/github/dcapwell/cassandra/860/workflows/bde81400-ed6d-4d9c-8447-3dd33b14cddd/jobs/5117

{code}
junit.framework.AssertionFailedError: Found Digest Mismatch
at 
org.apache.cassandra.distributed.upgrade.MixedModeReadTest.checkTraceForDigestMismatch(MixedModeReadTest.java:89)
at 
org.apache.cassandra.distributed.upgrade.MixedModeReadTest.lambda$mixedModeReadColumnSubsetDigestCheck$0(MixedModeReadTest.java:63)
at 
org.apache.cassandra.distributed.upgrade.UpgradeTestBase$TestCase.run(UpgradeTestBase.java:171)
at 
org.apache.cassandra.distributed.upgrade.MixedModeReadTest.mixedModeReadColumnSubsetDigestCheck(MixedModeReadTest.java:76)
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable

[Brandon Williams (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281371#comment-17281371
 ] 

Brandon Williams commented on CASSANDRA-15897:
--

+1 on 3.0.

bq. Since 4.0 cannot even read the lb SSTables, do we need a check at all when 
we drop compact storage or we would expect that there won't be already any that 
old SSTables on a cluster with 4.0 nodes? 

After CASSANDRA-16063, shouldn't it refuse to start if there are old sstables?

As far as 4.0 blocker goes, I do feel that we need all upgrade procedures 
finalized, and compact storage  would be one of them.

> Dropping compact storage with 2.1-sstables on disk make them unreadable
> ---
>
> Key: CASSANDRA-15897
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15897
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Local Write-Read Paths
>Reporter: Marcus Eriksson
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> Test reproducing: 
> https://github.com/krummas/cassandra/commits/marcuse/dropcompactstorage



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15355) Schema push/pull race on continuous schema changes

[James Baker (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Baker updated CASSANDRA-15355:

 Bug Category: Parent values: Degradation(12984)Level 1 values: Slow Use 
Case(12996)
   Complexity: Low Hanging Fruit
Discovered By: User Report
 Severity: Normal
   Status: Open  (was: Triage Needed)

> Schema push/pull race on continuous schema changes
> --
>
> Key: CASSANDRA-15355
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15355
> Project: Cassandra
>  Issue Type: Bug
>  Components: Cluster/Gossip
>Reporter: James Baker
>Priority: Normal
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In https://issues.apache.org/jira/browse/CASSANDRA-5025, pull based schema 
> updates were scheduled 1 minute after the schema change was first visible, so 
> as to prefer the push codepath as much as possible.
> Unfortunately, this does not handle the case where there are many schema 
> changes happening - imagine a scenario where we create a table every 5 
> seconds for 2 minutes - the first update tasks execute 60 seconds in and the 
> schemas may well be out of sync between nodes at that point.
> In this case, there is some chance that when the task runs, the schemas are 
> out of sync because a subsequent schema update has occurred, and so the same 
> push/pull race has happened.
> A fix is to modify the codepath such that the scheduled task is only run if 
> the other node's schema version is the same as when the task was scheduled. A 
> different (later scheduled) task should run otherwise.
> For us, what we see is that when we have a reasonably large number of 
> changes, a few schema changes can have the unfortunate outcome of causing our 
> nodes to run out of memory and crash - if we have a 30 node cluster, create a 
> table every second for 2 minutes, and for some reason we pause for 10 seconds 
> after 60 seconds with no progress, we can easily end up currently running 300 
> schema pulls for a single node. These can cause further piling up which 
> causes cascading failures. This change stops that.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15355) Schema push/pull race on continuous schema changes

[James Baker (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Baker updated CASSANDRA-15355:

  Component/s: Cluster/Gossip
  Impacts:   (was: None)
Since Version: 2.2.18

> Schema push/pull race on continuous schema changes
> --
>
> Key: CASSANDRA-15355
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15355
> Project: Cassandra
>  Issue Type: Bug
>  Components: Cluster/Gossip
>Reporter: James Baker
>Priority: Normal
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In https://issues.apache.org/jira/browse/CASSANDRA-5025, pull based schema 
> updates were scheduled 1 minute after the schema change was first visible, so 
> as to prefer the push codepath as much as possible.
> Unfortunately, this does not handle the case where there are many schema 
> changes happening - imagine a scenario where we create a table every 5 
> seconds for 2 minutes - the first update tasks execute 60 seconds in and the 
> schemas may well be out of sync between nodes at that point.
> In this case, there is some chance that when the task runs, the schemas are 
> out of sync because a subsequent schema update has occurred, and so the same 
> push/pull race has happened.
> A fix is to modify the codepath such that the scheduled task is only run if 
> the other node's schema version is the same as when the task was scheduled. A 
> different (later scheduled) task should run otherwise.
> For us, what we see is that when we have a reasonably large number of 
> changes, a few schema changes can have the unfortunate outcome of causing our 
> nodes to run out of memory and crash - if we have a 30 node cluster, create a 
> table every second for 2 minutes, and for some reason we pause for 10 seconds 
> after 60 seconds with no progress, we can easily end up currently running 300 
> schema pulls for a single node. These can cause further piling up which 
> causes cascading failures. This change stops that.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15355) Schema push/pull race on continuous schema changes

[James Baker (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281320#comment-17281320
 ] 

James Baker commented on CASSANDRA-15355:
-

The code has now been refactored and I believe that the bug is located here: 
[https://github.com/apache/cassandra/blob/159e021aa36a44cdc7674dd234d4a6e189478280/src/java/org/apache/cassandra/schema/MigrationCoordinator.java#L205]
 though I might be mistaken. I think it's still a problem though. Would be 
happy to rebase my fix if it's desired.

> Schema push/pull race on continuous schema changes
> --
>
> Key: CASSANDRA-15355
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15355
> Project: Cassandra
>  Issue Type: Bug
>Reporter: James Baker
>Priority: Normal
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In https://issues.apache.org/jira/browse/CASSANDRA-5025, pull based schema 
> updates were scheduled 1 minute after the schema change was first visible, so 
> as to prefer the push codepath as much as possible.
> Unfortunately, this does not handle the case where there are many schema 
> changes happening - imagine a scenario where we create a table every 5 
> seconds for 2 minutes - the first update tasks execute 60 seconds in and the 
> schemas may well be out of sync between nodes at that point.
> In this case, there is some chance that when the task runs, the schemas are 
> out of sync because a subsequent schema update has occurred, and so the same 
> push/pull race has happened.
> A fix is to modify the codepath such that the scheduled task is only run if 
> the other node's schema version is the same as when the task was scheduled. A 
> different (later scheduled) task should run otherwise.
> For us, what we see is that when we have a reasonably large number of 
> changes, a few schema changes can have the unfortunate outcome of causing our 
> nodes to run out of memory and crash - if we have a 30 node cluster, create a 
> table every second for 2 minutes, and for some reason we pause for 10 seconds 
> after 60 seconds with no progress, we can easily end up currently running 300 
> schema pulls for a single node. These can cause further piling up which 
> causes cascading failures. This change stops that.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16415) Broken test mixedModeReadColumnSubsetDigestCheck - org.apache.cassandra.distributed.upgrade.MixedModeReadTest

[Yifan Cai (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281287#comment-17281287
 ] 

Yifan Cai commented on CASSANDRA-16415:
---

[~jlewandowski], it seems the only difference in the new 'MixedModeReadTest' is 
the static column. The old test already had the gossip and networking features 
enabled. The test runs after reverting the table definition. 

Not sure whether the behavior is unexpected. It is a separate issue and should 
be tracked in a different ticket, if so. 

> Broken test mixedModeReadColumnSubsetDigestCheck - 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest
> -
>
> Key: CASSANDRA-16415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16415
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination, Test/dtest/java
>Reporter: Yifan Cai
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The test has been failing and can always be reproduced in the recent CI. 
> Stack trace: 
> {code:java}
> junit.framework.AssertionFailedError: Found Digest Mismatch
>  at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.checkTraceForDigestMismatch(MixedModeReadTest.java:89)
>  at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.lambda$mixedModeReadColumnSubsetDigestCheck$0(MixedModeReadTest.java:63)
>  at 
> org.apache.cassandra.distributed.upgrade.UpgradeTestBase$TestCase.run(UpgradeTestBase.java:171)
>  at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.mixedModeReadColumnSubsetDigestCheck(MixedModeReadTest.java:76)
>  {code}
> The initial investigation shows that 
>  * The test only fails in the setup phase of 
> mixedModeReadColumnSubsetDigestCheck. The cluster version is 
> *Versions.Major.v3X*
>  * The test failure is likely a consequence of CASSANDRA-15962. After 
> dropping the commit in branch cassandra-3.11 and rebuild the dtest jar, the 
> upgrade test can pass. Meanwhile, dropping the other commits does not help. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15977) 4.0 Quality: Read Repair Test Audit

[Sergey Kandyla (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281267#comment-17281267
 ] 

Sergey Kandyla commented on CASSANDRA-15977:


[~adelapena]
we don't use UDTs and frozen collections.
Data model:
{code:java}
CREATE TABLE book_ticket (
 application_id text,
 user_id text,
 ticket_id text,
 book_type text,
 created_at_timestamp int,
 deleted_at int,
 ids set,
 PRIMARY KEY ((application_id, user_id), ticket_id)
) WITH CLUSTERING ORDER BY (ticket_id ASC)
 AND bloom_filter_fp_chance = 0.01
 AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
 AND comment = ''
 AND compaction = {'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
'max_threshold': '32', 'min_threshold': '4'}
 AND compression = {'chunk_length_in_kb': '64', 'class': 
'org.apache.cassandra.io.compress.LZ4Compressor'}
 AND crc_check_chance = 1.0
 AND dclocal_read_repair_chance = 0.0
 AND default_time_to_live = 0
 AND gc_grace_seconds = 864000
 AND max_index_interval = 2048
 AND memtable_flush_period_in_ms = 0
 AND min_index_interval = 128
 AND read_repair_chance = 0.0
 AND speculative_retry = '99PERCENTILE';{code}

Typical request from golang application something like this:
{code:java}
SELECT deleted_at,ids FROM book_ticket WHERE application_id = ? AND user_id IN 
('5fff6d0e-2658-4e93-8f8a-05120a0f021d');{code}

user_id can be multiple values.
cassandra 3.11.10, 3 nodes, RF=3. gocql with consistency local_quorum.

> 4.0 Quality: Read Repair Test Audit
> ---
>
> Key: CASSANDRA-15977
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15977
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/java, Test/unit
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 3.11.9, 4.0, 4.0-beta3
>
> Attachments: Screenshot 2021-02-05 at 18.01.10.png
>
>  Time Spent: 13h 50m
>  Remaining Estimate: 0h
>
> This is a subtask of CASSANDRA-15579 focusing on read repair.
> [This 
> document|https://docs.google.com/document/d/1-gldHcdLSMRbDhhI8ahs_tPeAZsjurjXr38xABVjWHE/edit?usp=sharing]
>  lists and describes the existing functional tests for read repair, so we can 
> have a broad view of what is currently covered. We can comment on this 
> document and add ideas for new cases/tests, so it can gradually evolve to a 
> more or less detailed test plan.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16431) Group By breaks range tombstone closer

[Sam Tunnicliffe (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-16431:

Reviewers: Sam Tunnicliffe

> Group By breaks range tombstone closer
> --
>
> Key: CASSANDRA-16431
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16431
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Normal
>
> This is caused by the same piece of GROUP BY code that has caused 
> CASSANDRA-16307: counting. Since we have to look one row ahead in order to 
> “finish” the group, we’ll check out the next row but will return null to 
> transformations (including RT closer one). RT Closer will not see the row 
> and, since the iterator is considered done, it’ll complain that that’s not 
> right.
> Similar result can be achieved without paging, but with LIMIT 1.
> Minimal repro:
> {code:java}
> try (Cluster cluster = init(builder().withNodes(1).start()))
> {
> cluster.schemaChange("CREATE TABLE 
> distributed_test_keyspace.table_5 " +
>  "(pk0 bigint,pk1 bigint,ck0 bigint,ck1 
> bigint, PRIMARY KEY ((pk0,pk1), ck0, ck1)) " +
>  "WITH  CLUSTERING ORDER BY (ck0 ASC,ck1 
> ASC);");
> long pk1 = 1;
> long pk2 = 1;
> cluster.coordinator(1).execute("DELETE FROM 
> distributed_test_keyspace.table_5 USING TIMESTAMP 10 WHERE pk0=? AND 
> pk1=? AND ck0>?;", ConsistencyLevel.QUORUM, pk1, pk2, 2L);
> cluster.coordinator(1).execute("INSERT INTO 
> distributed_test_keyspace.table_5 (pk0,pk1,ck0,ck1) VALUES (?,?,?,?) USING 
> TIMESTAMP 11;", ConsistencyLevel.QUORUM, pk1, pk2, 1L, 1L);
> cluster.coordinator(1).execute("INSERT INTO 
> distributed_test_keyspace.table_5 (pk0,pk1,ck0,ck1) VALUES (?,?,?,?) USING 
> TIMESTAMP 11;", ConsistencyLevel.QUORUM, pk1, pk2, 3L, 1L);
> cluster.coordinator(1).executeWithPaging("SELECT pk0,pk1,ck0,ck1 
> FROM distributed_test_keyspace.table_5 WHERE pk0=? AND pk1=? GROUP BY 
> pk0,pk1,ck0;",
>  ConsistencyLevel.QUORUM,
>  1,
>  pk1, pk2);
> }
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16431) Group By breaks range tombstone closer

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-16431:

 Bug Category: Parent values: Availability(12983)Level 1 values: Response 
Crash(12991)
   Complexity: Challenging
  Component/s: Consistency/Coordination
Discovered By: Fuzz Test
 Severity: Critical
   Status: Open  (was: Triage Needed)

> Group By breaks range tombstone closer
> --
>
> Key: CASSANDRA-16431
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16431
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Normal
>
> This is caused by the same piece of GROUP BY code that has caused 
> CASSANDRA-16307: counting. Since we have to look one row ahead in order to 
> “finish” the group, we’ll check out the next row but will return null to 
> transformations (including RT closer one). RT Closer will not see the row 
> and, since the iterator is considered done, it’ll complain that that’s not 
> right.
> Similar result can be achieved without paging, but with LIMIT 1.
> Minimal repro:
> {code:java}
> try (Cluster cluster = init(builder().withNodes(1).start()))
> {
> cluster.schemaChange("CREATE TABLE 
> distributed_test_keyspace.table_5 " +
>  "(pk0 bigint,pk1 bigint,ck0 bigint,ck1 
> bigint, PRIMARY KEY ((pk0,pk1), ck0, ck1)) " +
>  "WITH  CLUSTERING ORDER BY (ck0 ASC,ck1 
> ASC);");
> long pk1 = 1;
> long pk2 = 1;
> cluster.coordinator(1).execute("DELETE FROM 
> distributed_test_keyspace.table_5 USING TIMESTAMP 10 WHERE pk0=? AND 
> pk1=? AND ck0>?;", ConsistencyLevel.QUORUM, pk1, pk2, 2L);
> cluster.coordinator(1).execute("INSERT INTO 
> distributed_test_keyspace.table_5 (pk0,pk1,ck0,ck1) VALUES (?,?,?,?) USING 
> TIMESTAMP 11;", ConsistencyLevel.QUORUM, pk1, pk2, 1L, 1L);
> cluster.coordinator(1).execute("INSERT INTO 
> distributed_test_keyspace.table_5 (pk0,pk1,ck0,ck1) VALUES (?,?,?,?) USING 
> TIMESTAMP 11;", ConsistencyLevel.QUORUM, pk1, pk2, 3L, 1L);
> cluster.coordinator(1).executeWithPaging("SELECT pk0,pk1,ck0,ck1 
> FROM distributed_test_keyspace.table_5 WHERE pk0=? AND pk1=? GROUP BY 
> pk0,pk1,ck0;",
>  ConsistencyLevel.QUORUM,
>  1,
>  pk1, pk2);
> }
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-16431) Group By breaks range tombstone closer

[Alex Petrov (Jira)]

Alex Petrov created CASSANDRA-16431:
---

 Summary: Group By breaks range tombstone closer
 Key: CASSANDRA-16431
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16431
 Project: Cassandra
  Issue Type: Bug
Reporter: Alex Petrov
Assignee: Alex Petrov


This is caused by the same piece of GROUP BY code that has caused 
CASSANDRA-16307: counting. Since we have to look one row ahead in order to 
“finish” the group, we’ll check out the next row but will return null to 
transformations (including RT closer one). RT Closer will not see the row and, 
since the iterator is considered done, it’ll complain that that’s not right.

Similar result can be achieved without paging, but with LIMIT 1.

Minimal repro:
{code:java}
try (Cluster cluster = init(builder().withNodes(1).start()))
{
cluster.schemaChange("CREATE TABLE 
distributed_test_keyspace.table_5 " +
 "(pk0 bigint,pk1 bigint,ck0 bigint,ck1 bigint, 
PRIMARY KEY ((pk0,pk1), ck0, ck1)) " +
 "WITH  CLUSTERING ORDER BY (ck0 ASC,ck1 
ASC);");
long pk1 = 1;
long pk2 = 1;
cluster.coordinator(1).execute("DELETE FROM 
distributed_test_keyspace.table_5 USING TIMESTAMP 10 WHERE pk0=? AND pk1=? 
AND ck0>?;", ConsistencyLevel.QUORUM, pk1, pk2, 2L);
cluster.coordinator(1).execute("INSERT INTO 
distributed_test_keyspace.table_5 (pk0,pk1,ck0,ck1) VALUES (?,?,?,?) USING 
TIMESTAMP 11;", ConsistencyLevel.QUORUM, pk1, pk2, 1L, 1L);
cluster.coordinator(1).execute("INSERT INTO 
distributed_test_keyspace.table_5 (pk0,pk1,ck0,ck1) VALUES (?,?,?,?) USING 
TIMESTAMP 11;", ConsistencyLevel.QUORUM, pk1, pk2, 3L, 1L);
cluster.coordinator(1).executeWithPaging("SELECT pk0,pk1,ck0,ck1 
FROM distributed_test_keyspace.table_5 WHERE pk0=? AND pk1=? GROUP BY 
pk0,pk1,ck0;",
 ConsistencyLevel.QUORUM,
 1,
 pk1, pk2);
}
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16401) Remove the Jenkins plaintext reports

[Michael Semb Wever (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-16401:
---
Source Control Link: 
https://github.com/apache/cassandra/commit/9fa2c28dc2d701b7b2cc90d4282f05ee1a221943
 
https://github.com/apache/cassandra-builds/commit/b6a7a2db24bcdc139d08928ec08cafd30c9a4c10
  (was: 
https://github.com/apache/cassandra/commit/9fa2c28dc2d701b7b2cc90d4282f05ee1a221943)
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

Cassandra-builds committed with 
[b6a7a2db24bcdc139d08928ec08cafd30c9a4c10|https://github.com/apache/cassandra-builds/commit/b6a7a2db24bcdc139d08928ec08cafd30c9a4c10].

> Remove the Jenkins plaintext reports
> 
>
> Key: CASSANDRA-16401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16401
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0-beta5, 4.0
>
> Attachments: jenkins-local-consoleText.txt
>
>
> Remove the generation of the {{cassandra-test-report.txt}} plain text report 
> (that was added in CASSANDRA-15729). 
> The xml report is now at 3GB and too big to fit through any xsl 
> transformation. It is only going to get bigger as more tests and test stages 
> are added.
> This size failure is seen in the pipeline logs as
> {noformat}
> docker run --rm -v 
> /home/jenkins/jenkins-slave/workspace/Cassandra-trunk@2:/tmp 
> apache_cassandra_ci/generate_plaintext_test_report
> Warning: at xsl:stylesheet on line 4 column 49 of cassandra-test-report.xsl:  
> Running an XSLT 1.0 stylesheet with an XSLT 2.0 processor
>  java.lang.IndexOutOfBoundsException: [660540429,660540717]
>   at net.sf.saxon.tinytree.LargeStringBuffer.subSequence(Unknown Source)
>   at net.sf.saxon.tinytree.TinyTextImpl.getStringValue(Unknown Source)
>   at net.sf.saxon.tinytree.TinyParentNodeImpl.getStringValue(Unknown 
> Source)
> …
> {noformat}
> In addition we can and do now archive everything to 
> https://nightlies.apache.org/cassandra/ , solving the original requirements 
> of CASSANDRA-15729 of providing a permanent archive of test results.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra-builds] branch trunk updated: Jenkins builds to provide link to nightlies archive, and remove the Jenkins plaintext reports

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git


The following commit(s) were added to refs/heads/trunk by this push:
 new b6a7a2d  Jenkins builds to provide link to nightlies archive, and 
remove the Jenkins plaintext reports
b6a7a2d is described below

commit b6a7a2db24bcdc139d08928ec08cafd30c9a4c10
Author: Mick Semb Wever 
AuthorDate: Sat Jan 23 11:32:06 2021 +0100

Jenkins builds to provide link to nightlies archive, and remove the Jenkins 
plaintext reports

 Also add aggregated xml reports to the test jobs, and archive them to 
nightlies.a.o

 patch by Mick Semb Wever; reviewed by Tomek Łasica for CASSANDRA-16401
---
 build-scripts/cassandra-test-report.sh |   6 +-
 build-scripts/cassandra-test-report.xsl| 325 -
 .../jenkins/generate_plaintext_test_report.docker  |  23 --
 jenkins-dsl/cassandra_job_dsl_seed.groovy  |  32 +-
 jenkins-dsl/cassandra_pipeline.groovy  |   6 +-
 5 files changed, 22 insertions(+), 370 deletions(-)

diff --git a/build-scripts/cassandra-test-report.sh 
b/build-scripts/cassandra-test-report.sh
index d3b6d52..0fdee2c 100755
--- a/build-scripts/cassandra-test-report.sh
+++ b/build-scripts/cassandra-test-report.sh
@@ -1,8 +1,4 @@
 #!/bin/bash -x
 
-# merge all the unit test xml files into one TESTS-TestSuites.xml file
+# merge all the unit test xml files into one TESTS-TestSuites.xml file (which 
will get archived at nightlies.a.o)
 ant -f ./cassandra-builds/build-scripts/cassandra-test-report.xml
-
-# perform the xslt and plaintext transformation inside a docker container
-docker build --tag apache_cassandra_ci/generate_plaintext_test_report -f 
cassandra-builds/docker/jenkins/generate_plaintext_test_report.docker .
-docker run --rm -v `pwd`:/tmp 
apache_cassandra_ci/generate_plaintext_test_report > cassandra-test-report.txt
diff --git a/build-scripts/cassandra-test-report.xsl 
b/build-scripts/cassandra-test-report.xsl
deleted file mode 100644
index 9b37706..000
--- a/build-scripts/cassandra-test-report.xsl
+++ /dev/null
@@ -1,325 +0,0 @@
-
-http://www.w3.org/1999/XSL/Transform; version="1.0"
-xmlns:lxslt="http://xml.apache.org/xslt;
-xmlns:string="xalan://java.lang.String">
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-END
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-Failure
-Error
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-Packages
-
-
-
-
-
-
-
-
-
-
-
-
-
-Failures
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-Skipped
-
-
-
-
-
-
-
-
-
-
-Cassandra CI Results
-
-
-
-
-
-
-
-
-
-Tests
-Failures
-Errors
-Skipped
-Success rate
-Time
-
-
-
-
-Failure
-Error
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-Name
-Tests
-Errors
-Failures
-Skipped
-Time(s)
-
-
-
-
-
-
-Name
-Tests
-Errors
-Failures
-Skipped
-Time(s)
-
-
-
-
-
-
-
-
-
-Failure
-Error
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-   

[jira] [Updated] (CASSANDRA-16430) Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory

[sharanya (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

sharanya updated CASSANDRA-16430:
-
Description: 
I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
production environment. We are continuously facing below warnings from one of 
our monitoring tool, but there is no suspicious error from system.log file. 
Also there is sufficient memory available for Heap/server.

 
 warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
 Datasource: Cassandra JVM Memory Pools -Par Survivor Space
 Datapoint: UsedPoolMemory
 Level: warn
 Start: 2021-02-08 10:11:17 GMT
 Duration: 1h 0m
 Value: 100.0
 ClearValue: 1.3631
 Reason: UsedPoolMemory is not >= 90: the current value is 1.3631

The java memory pool Par Survivor Space is now using 100.0 % of the maximum.

 We see the below message in debug.log file.

2021-02-08 08:43:51,310 INFO  [ReadStage-1] NoSpamLogger.java:91 - Maximum 
memory usage reached (512.000MiB), cannot allocate chunk of 1.000MiB

2021-02-08 08:47:28,344 DEBUG [COMMIT-LOG-ALLOCATOR] 
AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a 
fresh one

 Please help us to fix this issue ASAP.

 Thanks!

  was:
I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
production environment. We are continuously facing below warnings from one of 
our monitoring tool, but there is no suspicious error from system.log file. 
Also there is sufficient memory available for Heap/server.

 
 warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
 Datasource: Cassandra JVM Memory Pools -Par Survivor Space
 Datapoint: UsedPoolMemory
 Level: warn
 Start: 2021-02-08 10:11:17 GMT
 Duration: 1h 0m
 Value: 100.0
 ClearValue: 1.3631
 Reason: UsedPoolMemory is not >= 90: the current value is 1.3631

The java memory pool Par Survivor Space is now using 100.0 % of the maximum.

 

We see the below message in debug.log file.

2021-02-08 08:43:51,310 INFO  [ReadStage-1] NoSpamLogger.java:91 - Maximum 
memory usage reached (512.000MiB), cannot allocate chunk of 1.000MiB

2021-02-08 08:47:28,344 DEBUG [COMMIT-LOG-ALLOCATOR] 
AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a 
fresh one

 Please help us to fix this issue ASAP.

 

 

 

Thanks!


> Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
> -
>
> Key: CASSANDRA-16430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16430
> Project: Cassandra
>  Issue Type: Task
>  Components: Local/Config, Observability/Metrics
>Reporter: sharanya
>Priority: Urgent
>
> I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
> production environment. We are continuously facing below warnings from one of 
> our monitoring tool, but there is no suspicious error from system.log file. 
> Also there is sufficient memory available for Heap/server.
>  
>  warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
>  Datasource: Cassandra JVM Memory Pools -Par Survivor Space
>  Datapoint: UsedPoolMemory
>  Level: warn
>  Start: 2021-02-08 10:11:17 GMT
>  Duration: 1h 0m
>  Value: 100.0
>  ClearValue: 1.3631
>  Reason: UsedPoolMemory is not >= 90: the current value is 1.3631
> The java memory pool Par Survivor Space is now using 100.0 % of the maximum.
>  We see the below message in debug.log file.
> 2021-02-08 08:43:51,310 INFO  [ReadStage-1] NoSpamLogger.java:91 - Maximum 
> memory usage reached (512.000MiB), cannot allocate chunk of 1.000MiB
> 2021-02-08 08:47:28,344 DEBUG [COMMIT-LOG-ALLOCATOR] 
> AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a 
> fresh one
>  Please help us to fix this issue ASAP.
>  Thanks!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16430) Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory

[sharanya (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

sharanya updated CASSANDRA-16430:
-
Description: 
I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
production environment. We are continuously facing below warnings from one of 
our monitoring tool, but there is no suspicious error from system.log file. 
Also there is sufficient memory available for Heap/server.

 
 warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
 Datasource: Cassandra JVM Memory Pools -Par Survivor Space
 Datapoint: UsedPoolMemory
 Level: warn
 Start: 2021-02-08 10:11:17 GMT
 Duration: 1h 0m
 Value: 100.0
 ClearValue: 1.3631
 Reason: UsedPoolMemory is not >= 90: the current value is 1.3631

The java memory pool Par Survivor Space is now using 100.0 % of the maximum.

 

We see the below message in debug.log file.

2021-02-08 08:43:51,310 INFO  [ReadStage-1] NoSpamLogger.java:91 - Maximum 
memory usage reached (512.000MiB), cannot allocate chunk of 1.000MiB

2021-02-08 08:47:28,344 DEBUG [COMMIT-LOG-ALLOCATOR] 
AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a 
fresh one

 Please help us to fix this issue ASAP.

 

 

 

Thanks!

  was:
I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
production environment. We are continuously facing below warnings from one of 
our monitoring tool, but there is no suspicious error from system.log file. 
Also there is sufficient memory available for Heap/server.

 
 warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
 Datasource: Cassandra JVM Memory Pools -Par Survivor Space
 Datapoint: UsedPoolMemory
 Level: warn
 Start: 2021-02-08 10:11:17 GMT
 Duration: 1h 0m
 Value: 100.0
 ClearValue: 1.3631
 Reason: UsedPoolMemory is not >= 90: the current value is 1.3631

The java memory pool Par Survivor Space is now using 100.0 % of the maximum.

 

We see the below message in debug.log file.

2021-02-08 08:43:51,310 INFO  [ReadStage-1] NoSpamLogger.java:91 - Maximum 
memory usage reached (512.000MiB), cannot allocate chunk of 1.000MiB

2021-02-08 08:47:28,344 DEBUG [COMMIT-LOG-ALLOCATOR] 
AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a 
fresh one

 

 

Please help us to fix this issue ASAP.

 

 

 

Thanks!


> Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
> -
>
> Key: CASSANDRA-16430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16430
> Project: Cassandra
>  Issue Type: Task
>  Components: Local/Config, Observability/Metrics
>Reporter: sharanya
>Priority: Urgent
>
> I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
> production environment. We are continuously facing below warnings from one of 
> our monitoring tool, but there is no suspicious error from system.log file. 
> Also there is sufficient memory available for Heap/server.
>  
>  warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
>  Datasource: Cassandra JVM Memory Pools -Par Survivor Space
>  Datapoint: UsedPoolMemory
>  Level: warn
>  Start: 2021-02-08 10:11:17 GMT
>  Duration: 1h 0m
>  Value: 100.0
>  ClearValue: 1.3631
>  Reason: UsedPoolMemory is not >= 90: the current value is 1.3631
> The java memory pool Par Survivor Space is now using 100.0 % of the maximum.
>  
> We see the below message in debug.log file.
> 2021-02-08 08:43:51,310 INFO  [ReadStage-1] NoSpamLogger.java:91 - Maximum 
> memory usage reached (512.000MiB), cannot allocate chunk of 1.000MiB
> 2021-02-08 08:47:28,344 DEBUG [COMMIT-LOG-ALLOCATOR] 
> AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a 
> fresh one
>  Please help us to fix this issue ASAP.
>  
>  
>  
> Thanks!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16430) Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory

[sharanya (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

sharanya updated CASSANDRA-16430:
-
Description: 
I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
production environment. We are continuously facing below warnings from one of 
our monitoring tool, but there is no suspicious error from system.log file. 
Also there is sufficient memory available for Heap/server.

 
 warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
 Datasource: Cassandra JVM Memory Pools -Par Survivor Space
 Datapoint: UsedPoolMemory
 Level: warn
 Start: 2021-02-08 10:11:17 GMT
 Duration: 1h 0m
 Value: 100.0
 ClearValue: 1.3631
 Reason: UsedPoolMemory is not >= 90: the current value is 1.3631

The java memory pool Par Survivor Space is now using 100.0 % of the maximum.

 

We see the below message in debug.log file.

2021-02-08 08:43:51,310 INFO  [ReadStage-1] NoSpamLogger.java:91 - Maximum 
memory usage reached (512.000MiB), cannot allocate chunk of 1.000MiB

2021-02-08 08:47:28,344 DEBUG [COMMIT-LOG-ALLOCATOR] 
AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a 
fresh one

 

 

Please help us to fix this issue ASAP.

 

 

 

Thanks!

  was:
I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
production environment. We are continuously facing below warnings from one of 
our monitoring tool, but there is no suspicious error from system.log file. 
Also there is sufficient memory available for Heap/server.

 
 warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
Datasource: Cassandra JVM Memory Pools -Par Survivor Space
Datapoint: UsedPoolMemory
Level: warn
Start: 2021-02-08 10:11:17 GMT
Duration: 1h 0m
Value: 100.0
ClearValue: 1.3631
Reason: UsedPoolMemory is not >= 90: the current value is 1.3631


The java memory pool Par Survivor Space on esg3-c-cassv3-03p is now using 100.0 
% of the maximum.



 

We see the below message in debug.log file.

2021-02-08 08:43:51,310 INFO  [ReadStage-1] NoSpamLogger.java:91 - Maximum 
memory usage reached (512.000MiB), cannot allocate chunk of 1.000MiB

2021-02-08 08:47:28,344 DEBUG [COMMIT-LOG-ALLOCATOR] 
AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a 
fresh one

 

 

Please help us to fix this issue ASAP.

 

 

 

Thanks!


> Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
> -
>
> Key: CASSANDRA-16430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16430
> Project: Cassandra
>  Issue Type: Task
>  Components: Local/Config, Observability/Metrics
>Reporter: sharanya
>Priority: Urgent
>
> I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
> production environment. We are continuously facing below warnings from one of 
> our monitoring tool, but there is no suspicious error from system.log file. 
> Also there is sufficient memory available for Heap/server.
>  
>  warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
>  Datasource: Cassandra JVM Memory Pools -Par Survivor Space
>  Datapoint: UsedPoolMemory
>  Level: warn
>  Start: 2021-02-08 10:11:17 GMT
>  Duration: 1h 0m
>  Value: 100.0
>  ClearValue: 1.3631
>  Reason: UsedPoolMemory is not >= 90: the current value is 1.3631
> The java memory pool Par Survivor Space is now using 100.0 % of the maximum.
>  
> We see the below message in debug.log file.
> 2021-02-08 08:43:51,310 INFO  [ReadStage-1] NoSpamLogger.java:91 - Maximum 
> memory usage reached (512.000MiB), cannot allocate chunk of 1.000MiB
> 2021-02-08 08:47:28,344 DEBUG [COMMIT-LOG-ALLOCATOR] 
> AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a 
> fresh one
>  
>  
> Please help us to fix this issue ASAP.
>  
>  
>  
> Thanks!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16425) Improve handling of repair sessions during startup/shutdown

[Chris Lohfink (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281190#comment-17281190
 ] 

Chris Lohfink commented on CASSANDRA-16425:
---

failed test looks like flakey unrelated thing if can double check that.

+1

> Improve handling of repair sessions during startup/shutdown
> ---
>
> Key: CASSANDRA-16425
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16425
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Repair
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 4.0-beta5
>
>
> We should send out a fail message on clean shutdown and if we find an ongoing 
> repair session during startup we should fail it and tell participants.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16425) Improve handling of repair sessions during startup/shutdown

[Chris Lohfink (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Lohfink updated CASSANDRA-16425:
--
Reviewers: Blake Eggleston, Chris Lohfink  (was: Blake Eggleston)

> Improve handling of repair sessions during startup/shutdown
> ---
>
> Key: CASSANDRA-16425
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16425
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Repair
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 4.0-beta5
>
>
> We should send out a fail message on clean shutdown and if we find an ongoing 
> repair session during startup we should fail it and tell participants.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16430) Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory

[sharanya (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281174#comment-17281174
 ] 

sharanya commented on CASSANDRA-16430:
--

Hi [~brandon.williams],

Thanks for the valued information.

 

 

> Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
> -
>
> Key: CASSANDRA-16430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16430
> Project: Cassandra
>  Issue Type: Task
>  Components: Local/Config, Observability/Metrics
>Reporter: sharanya
>Priority: Urgent
>
> I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
> production environment. We are continuously facing below warnings from one of 
> our monitoring tool, but there is no suspicious error from system.log file. 
> Also there is sufficient memory available for Heap/server.
>  
>  warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
> Datasource: Cassandra JVM Memory Pools -Par Survivor Space
> Datapoint: UsedPoolMemory
> Level: warn
> Start: 2021-02-08 10:11:17 GMT
> Duration: 1h 0m
> Value: 100.0
> ClearValue: 1.3631
> Reason: UsedPoolMemory is not >= 90: the current value is 1.3631
> The java memory pool Par Survivor Space on esg3-c-cassv3-03p is now using 
> 100.0 % of the maximum.
>  
> We see the below message in debug.log file.
> 2021-02-08 08:43:51,310 INFO  [ReadStage-1] NoSpamLogger.java:91 - Maximum 
> memory usage reached (512.000MiB), cannot allocate chunk of 1.000MiB
> 2021-02-08 08:47:28,344 DEBUG [COMMIT-LOG-ALLOCATOR] 
> AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a 
> fresh one
>  
>  
> Please help us to fix this issue ASAP.
>  
>  
>  
> Thanks!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16428) Fix selections of JDKs in debian docker images on arm64

[Brandon Williams (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281158#comment-17281158
 ] 

Brandon Williams commented on CASSANDRA-16428:
--

Perhaps using `{{dpkg --print-architecture}}` would be more correct?

> Fix selections of JDKs in debian docker images on arm64
> ---
>
> Key: CASSANDRA-16428
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16428
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
>
> The debian docker 
> [image|https://github.com/apache/cassandra-builds/blob/trunk/docker/buster-image.docker]
>  used by the 
> [cassandra-deb-packaging.sh|https://github.com/apache/cassandra-builds/blob/trunk/build-scripts/cassandra-deb-packaging.sh]
>  tries to set the jdk like
> {code}
> update-java-alternatives --set java-1.8.0-openjdk-amd64
> {code}
> This won't work on arm64.
> Suggestion is to replace it with the following:
> {code}
> update-java-alternatives --set $(update-java-alternatives -l | cut -d" " -f1 
> | grep java-1.8)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16296) Join new node to cluster failing on C* version 4

[Ekaterina Dimitrova (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-16296:

Reviewers: Andres de la Peña, Ekaterina Dimitrova, Tomasz Lasica  (was: 
Andres de la Peña, Tomasz Lasica)

> Join new node to cluster failing on C* version 4
> 
>
> Key: CASSANDRA-16296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16296
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Startup and Shutdown
>Reporter: Yakir Gibraltar
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.0-beta
>
>  Time Spent: 13h
>  Remaining Estimate: 0h
>
> Hi, i'm using cassandra 4.0-beta3,
> Join new node failing,
> Current status:
> {code}
> [root@rc801 ~]# nodetool status
> Datacenter: V4CH
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  Address  LoadTokens  Owns (effective)  Host ID
>Rack
> UN  1.1.1.1  356.55 KiB  128 ? 
> 92ae4c39-edb3-4e67-8623-b49fd8301b66  RAC1
> UN  2.2.2.2  424.36 KiB  128 ? 
> d80647a8-32b2-4a8f-8022-f5ae3ce8fbb2  RAC1
> Datacenter: V4NJ
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  Address  LoadTokens  Owns (effective)  Host ID
>Rack
> UN  3.3.3.3  322.97 KiB  128 ? 
> 0106ce0b-18be-4321-8247-843076ff42e6  RAC1
> UN  4.4.4.4  297.6 KiB   128 ? 
> d81e2b7d-9221-4b1b-86c8-e8956e3eec07  RAC1
> UN  5.5.5.5  324.75 KiB  128 ? 
> 5dfbaf14-8310-4d64-b61a-9da0a7a8a620  RAC1
> {code}
> Error on new host (7.7.7.7):
> {code}
> INFO  [main] 2020-11-22 09:27:36,592 RangeStreamer.java:323 - Bootstrap: 
> range Full(/7.7.7.7:7000,(-7734271291904743823,-7725025391901227341]) exists 
> on Full(/1.1.1.1:7000,(-7734271291904743823,-7725025391901227341]) for 
> keyspace system_traces
> ERROR [main] 2020-11-22 09:27:36,609 CassandraDaemon.java:817 - Exception 
> encountered during startup
> java.lang.IllegalStateException: Multiple strict sources found for 
> Full(/7.7.7.7:7000,(9208454697546654053,-9212763148842799409]), sources: 
> [Full(/2.2.2.2:7000,(9189373804905478622,-9212763148842799409]), 
> Full(/1.1.1.1:7000,(9189373804905478622,-9212763148842799409])]
> at 
> org.apache.cassandra.dht.RangeStreamer.calculateRangesToFetchWithPreferredEndpoints(RangeStreamer.java:517)
> at 
> org.apache.cassandra.dht.RangeStreamer.calculateRangesToFetchWithPreferredEndpoints(RangeStreamer.java:383)
> at 
> org.apache.cassandra.dht.RangeStreamer.addRanges(RangeStreamer.java:320)
> at 
> org.apache.cassandra.dht.BootStrapper.bootstrap(BootStrapper.java:81)
> at 
> org.apache.cassandra.service.StorageService.startBootstrap(StorageService.java:1635)
> at 
> org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1612)
> at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:931)
> at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:892)
> at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:699)
> at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:635)
> at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:407)
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:671)
> at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:795)
> INFO  [StorageServiceShutdownHook] 2020-11-22 09:27:36,616 
> HintsService.java:220 - Paused hints dispatch
> WARN  [StorageServiceShutdownHook] 2020-11-22 09:27:36,617 Gossiper.java:1849 
> - No local state, state is in silent shutdown, or node hasn't joined, not 
> announcing shutdown
> INFO  [StorageServiceShutdownHook] 2020-11-22 09:27:36,617 
> MessagingService.java:441 - Waiting for messaging service to quiesce
> {code}
> Errors on seed node:
> {code}
> INFO  [Messaging-EventLoop-3-37] 2020-11-22 09:26:54,121 
> InboundConnectionInitiator.java:459 - 
> /7.7.7.7:7000(/7.7.7.7:45490)->/1.1.1.1:7000-URGENT_MESSAGES-f412b9ba 
> messaging connection established, version = 12, framing = LZ4, encryption = 
> disabled
> INFO  [Messaging-EventLoop-3-40] 2020-11-22 09:26:54,157 
> OutboundConnection.java:1151 - 
> /1.1.1.1:7000(/1.1.1.1:38678)->/7.7.7.7:7000-URGENT_MESSAGES-3b23f639 
> successfully connected, version = 12, framing = LZ4, encryption = disabled
> INFO  [GossipStage:1] 2020-11-22 09:26:56,109 Gossiper.java:1248 - Node 
> /7.7.7.7:7000 is now part of the cluster
> INFO  [GossipStage:1] 

[jira] [Updated] (CASSANDRA-16411) Fix topology corruption on joining nodes without DC in dtests

[Ekaterina Dimitrova (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-16411:

Reviewers: Andres de la Peña, Ekaterina Dimitrova, Tomasz Lasica  (was: 
Andres de la Peña, Ekaterina Dimitrova)

> Fix topology corruption on joining nodes without DC in dtests
> -
>
> Key: CASSANDRA-16411
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16411
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/python
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Dtests with multi-dc setups don't update the topology properties file of a 
> newly joining node without dc post cluster creation.
> The new node will join dc1 but it's snitch will report the default snitch 
> config as it doesn't get updated == dc1 == all nodes are in dc1 == there are 
> no other dcs. That leads to bootstrap, token, startup,... problems. Let's see 
> an example:
> - populate [1, 1, 1]
> - Node1 sees: dc1/ip1, dc2/ip2 and dc3/ip3
> - Node2 sees: dc1/ip1, dc2/ip2 and dc3/ip3
> - Node3 sees: dc1/ip1, dc2/ip2 and dc3/ip3
> - Add a new Node4
> - Node4 sees: dc1/ip1+ip2+ip3+ip4
> All multi-dc dtests that add a node after cluster creation suffer this 
> problem.
> Solution: When a new node is added without specifying a dc in dtests check 
> for existing dcs. If there are any force to specify one to update the 
> topology file.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16411) Fix topology corruption on joining nodes without DC in dtests

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281154#comment-17281154
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16411:
-

Thanks [~Bereng], I found the links.

Posting here for visibility:
 * 
[2.1|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/370/]
 * 
[2.2|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/371/]
 * 
[3.0|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/372/]
 * 
[3.11|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/373/]
 * 
[trunk|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/374/]

I saw only one new commitlog related test failure on trunk but it is also 
failing with other unrelated patches in dev, also it seems it has nothing in 
common with this one.

+1 from me, thank you

> Fix topology corruption on joining nodes without DC in dtests
> -
>
> Key: CASSANDRA-16411
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16411
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/python
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Dtests with multi-dc setups don't update the topology properties file of a 
> newly joining node without dc post cluster creation.
> The new node will join dc1 but it's snitch will report the default snitch 
> config as it doesn't get updated == dc1 == all nodes are in dc1 == there are 
> no other dcs. That leads to bootstrap, token, startup,... problems. Let's see 
> an example:
> - populate [1, 1, 1]
> - Node1 sees: dc1/ip1, dc2/ip2 and dc3/ip3
> - Node2 sees: dc1/ip1, dc2/ip2 and dc3/ip3
> - Node3 sees: dc1/ip1, dc2/ip2 and dc3/ip3
> - Add a new Node4
> - Node4 sees: dc1/ip1+ip2+ip3+ip4
> All multi-dc dtests that add a node after cluster creation suffer this 
> problem.
> Solution: When a new node is added without specifying a dc in dtests check 
> for existing dcs. If there are any force to specify one to update the 
> topology file.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16430) Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16430:
-
Resolution: Invalid
Status: Resolved  (was: Open)

I'm not sure where this warning is coming from, since C* does not even monitor 
this, but as it states, this is a java memory pool.  JVM tuning is generally 
outside the scope of the project - but I will note this memory pool 
specifically is not something that has historically needed tuning.

 

The debug log messages are just that - debug information.

> Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
> -
>
> Key: CASSANDRA-16430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16430
> Project: Cassandra
>  Issue Type: Task
>  Components: Local/Config, Observability/Metrics
>Reporter: sharanya
>Priority: Urgent
>
> I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
> production environment. We are continuously facing below warnings from one of 
> our monitoring tool, but there is no suspicious error from system.log file. 
> Also there is sufficient memory available for Heap/server.
>  
>  warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
> Datasource: Cassandra JVM Memory Pools -Par Survivor Space
> Datapoint: UsedPoolMemory
> Level: warn
> Start: 2021-02-08 10:11:17 GMT
> Duration: 1h 0m
> Value: 100.0
> ClearValue: 1.3631
> Reason: UsedPoolMemory is not >= 90: the current value is 1.3631
> The java memory pool Par Survivor Space on esg3-c-cassv3-03p is now using 
> 100.0 % of the maximum.
>  
> We see the below message in debug.log file.
> 2021-02-08 08:43:51,310 INFO  [ReadStage-1] NoSpamLogger.java:91 - Maximum 
> memory usage reached (512.000MiB), cannot allocate chunk of 1.000MiB
> 2021-02-08 08:47:28,344 DEBUG [COMMIT-LOG-ALLOCATOR] 
> AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a 
> fresh one
>  
>  
> Please help us to fix this issue ASAP.
>  
>  
>  
> Thanks!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16415) Broken test mixedModeReadColumnSubsetDigestCheck - org.apache.cassandra.distributed.upgrade.MixedModeReadTest

[Jacek Lewandowski (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281118#comment-17281118
 ] 

Jacek Lewandowski commented on CASSANDRA-16415:
---

I encountered a weird problem - see my PR for 3.11: 
https://github.com/apache/cassandra/pull/891

Basically there are no changes in the production code. I modified a little bit 
the mixed read test mostly by adding gossip and networking features to the 
cluster configuration (as gossiper is required for this test run). Apparently I 
did something wrong because the cluster does not run properly and it complains 
with the following exception (repeated many, many times):

{noformat}
[junit-timeout] ERROR 14:35:29 Unable to merge schema from /127.0.0.2
[junit-timeout] java.util.concurrent.RejectedExecutionException: 
ThreadPoolExecutor has shut down
[junit-timeout] at 
org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:58)
 ~[dtest-3.11.11.jar:na]
[junit-timeout] at 
java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) 
[na:1.8.0_212]
[junit-timeout] at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) 
[na:1.8.0_212]
[junit-timeout] at 
org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.execute(DebuggableThreadPoolExecutor.java:162)
 ~[dtest-3.11.11.jar:na]
[junit-timeout] at 
org.apache.cassandra.db.ColumnFamilyStore.switchMemtable(ColumnFamilyStore.java:904)
 ~[dtest-3.11.11.jar:na]
[junit-timeout] at 
org.apache.cassandra.db.ColumnFamilyStore.switchMemtableIfCurrent(ColumnFamilyStore.java:885)
 ~[dtest-3.11.11.jar:na]
[junit-timeout] at 
org.apache.cassandra.db.ColumnFamilyStore.forceFlush(ColumnFamilyStore.java:954)
 ~[dtest-3.11.11.jar:na]
[junit-timeout] at 
org.apache.cassandra.schema.SchemaKeyspace.lambda$flush$1(SchemaKeyspace.java:310)
 ~[dtest-3.11.11.jar:na]
[junit-timeout] at java.lang.Iterable.forEach(Iterable.java:75) 
~[na:1.8.0_212]
[junit-timeout] at 
org.apache.cassandra.schema.SchemaKeyspace.flush(SchemaKeyspace.java:310) 
~[dtest-3.11.11.jar:na]
[junit-timeout] at 
org.apache.cassandra.schema.SchemaKeyspace.mergeSchema(SchemaKeyspace.java:1398)
 ~[dtest-3.11.11.jar:na]
[junit-timeout] at 
org.apache.cassandra.schema.SchemaKeyspace.mergeSchemaAndAnnounceVersion(SchemaKeyspace.java:1380)
 ~[dtest-3.11.11.jar:na]
[junit-timeout] at 
org.apache.cassandra.service.MigrationCoordinator.mergeSchemaFrom(MigrationCoordinator.java:367)
 ~[dtest-3.11.11.jar:na]
[junit-timeout] at 
org.apache.cassandra.service.MigrationCoordinator$Callback.response(MigrationCoordinator.java:404)
 [dtest-3.11.11.jar:na]
[junit-timeout] at 
org.apache.cassandra.service.MigrationCoordinator$Callback.response(MigrationCoordinator.java:393)
 [dtest-3.11.11.jar:na]
[junit-timeout] at 
org.apache.cassandra.net.ResponseVerbHandler.doVerb(ResponseVerbHandler.java:53)
 [dtest-3.11.11.jar:na]
[junit-timeout] at 
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66) 
[dtest-3.11.11.jar:na]
[junit-timeout] at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
[na:1.8.0_212]
[junit-timeout] at 
java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_212]
[junit-timeout] at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
[na:1.8.0_212]
[junit-timeout] at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
[na:1.8.0_212]
[junit-timeout] at 
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:84)
 [dtest-3.11.11.jar:na]
[junit-timeout] at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_212]
[junit-timeout] ERROR [node1_InternalResponseStage:1] node1 2021-02-08 
15:35:29,093 MigrationCoordinator.java:408 - Unable to merge schema from 
/127.0.0.2
[junit-timeout] java.util.concurrent.RejectedExecutionException: 
ThreadPoolExecutor has shut down
[junit-timeout] at 
org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:58)
 ~[dtest-3.11.11.jar:na]
[junit-timeout] at 
java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) 
[na:1.8.0_212]
[junit-timeout] at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) 
[na:1.8.0_212]
[junit-timeout] at 
org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.execute(DebuggableThreadPoolExecutor.java:162)
 ~[dtest-3.11.11.jar:na]
[junit-timeout] at 
org.apache.cassandra.db.ColumnFamilyStore.switchMemtable(ColumnFamilyStore.java:904)
 ~[dtest-3.11.11.jar:na]
[junit-timeout] at 

[jira] [Commented] (CASSANDRA-16066) Update and rework cassandra-website material to work with Antora

[Anthony Grasso (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281041#comment-17281041
 ] 

Anthony Grasso commented on CASSANDRA-16066:


The majority of proposed changes are now complete and are on [my 
fork|https://github.com/ossarga/cassandra-website] of the cassandra-website on 
branch 
[anthony/CASSANDRA-16066|https://github.com/ossarga/cassandra-website/tree/anthony/CASSANDRA-16066].

Items left to do:

* Complete wrapper script that simplifies the usage of the new tooling 
(currently in progress and about 75% complete).
* Update CI commands and triggers.
* Update repository README.

The bulk of the work is done and reviews of the proposed changes can probably 
start.

> Update and rework cassandra-website material to work with Antora
> 
>
> Key: CASSANDRA-16066
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16066
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation/Website
>Reporter: Anthony Grasso
>Assignee: Anthony Grasso
>Priority: Normal
> Attachments: image-2020-09-05-13-24-13-606.png, 
> image-2020-09-06-07-17-14-705.png
>
>
> *We want to modernise the way the website is built*
>  Rework the cassandra-website repository to generate a UI bundle containing 
> resources that Antora will use when generating the Cassandra documents and 
> website.
> *The existing method is starting to become dated*
>  The documentation and website templates are currently in markdown format. 
> Sphinx is used to generate the Cassandra documentation and Jekyll generates 
> the website content. One of the major issues with the existing website 
> tooling is that the live preview server (render site as it is being updated) 
> is really slow. There is a preview server that is really fast, however it is 
> unable to detect changes to the content and render automatically.
> *We are migrating the docs to be rendered with Antora*
>  The work in CASSANDRA-16029 is converting the document templates to AsciiDoc 
> format. Sphinx is being replaced by Antora to generate the documentation 
> content. This change has two advantages:
>  * More flexibility if the Apache Cassandra documentation look and feel needs 
> to be updated or redesigned.
>  * More modern look and feel to the documentation.
> *We can use Antora to generate the website as well*
>  Antora could also be used to generate the Cassandra website content. As 
> suggested on the [mailing 
> list|https://www.mail-archive.com/dev@cassandra.apache.org/msg15577.html] 
> this would require the existing markdown templates to be converted to 
> AsciiDoc as well.
> *Antora needs a UI bundle to style content*
>  For Antora to generate the document content and potentially the website 
> content it requires a UI bundle (ui-bundle.zip). The UI bundle contains the 
> HTML templates (layouts, partials, and helpers), CSS, JavaScript, fonts, and 
> (site-wide) images. As such, it provides both the visual theme and user 
> interactions for the documentation. Effectively the UI bundle is the 
> templates and styling that are applied to the documentation and website 
> content.
> *The [cassandra-website|https://github.com/apache/cassandra-website] 
> repository can be used to generate the UI bundle*
>  All the resources associated with templating and styling the documentation 
> and website can be placed in the 
> [cassandra-website|https://github.com/apache/cassandra-website] repository. 
> In this case the repository would serve two purposes;
>  * Generation of the UI bundle resources.
>  * Serve the production website content.
> *The [cassandra|https://github.com/apache/cassandra] repository would contain 
> the documentation material, while the rest of the non-versioned pages would 
> contain that material*
>  * All other material that is used to generate documentation would live in 
> the [cassandra|https://github.com/apache/cassandra] repository. In this case 
> Antora would run on the 
> [cassandra/doc|https://github.com/apache/cassandra/doc] directory and pull in 
> a UI bundle released on the GitHub 
> [cassandra-website|https://github.com/apache/cassandra-website] repository. 
> The content generated by Antora using the site.yml file located in this repo 
> can be used to preview the docs for review.
>  * All other non-versioned material, such as the blogs, development 
> instructions, and third-party page would live in the GitHub 
> [cassandra-website|https://github.com/apache/cassandra-website] repository. 
> Antora will generate the full website using the site.yml located in this 
> repo, using both as content sources the material located in both the 
> [cassandra|https://github.com/apache/cassandra] and 
> [cassandra-website|https://github.com/apache/cassandra-website] 

[jira] [Updated] (CASSANDRA-16430) Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory

[sharanya (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

sharanya updated CASSANDRA-16430:
-
Description: 
I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
production environment. We are continuously facing below warnings from one of 
our monitoring tool, but there is no suspicious error from system.log file. 
Also there is sufficient memory available for Heap/server.

 
 warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
Datasource: Cassandra JVM Memory Pools -Par Survivor Space
Datapoint: UsedPoolMemory
Level: warn
Start: 2021-02-08 10:11:17 GMT
Duration: 1h 0m
Value: 100.0
ClearValue: 1.3631
Reason: UsedPoolMemory is not >= 90: the current value is 1.3631


The java memory pool Par Survivor Space on esg3-c-cassv3-03p is now using 100.0 
% of the maximum.



 

We see the below message in debug.log file.

2021-02-08 08:43:51,310 INFO  [ReadStage-1] NoSpamLogger.java:91 - Maximum 
memory usage reached (512.000MiB), cannot allocate chunk of 1.000MiB

2021-02-08 08:47:28,344 DEBUG [COMMIT-LOG-ALLOCATOR] 
AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a 
fresh one

 

 

Please help us to fix this issue ASAP.

 

 

 

Thanks!

  was:
I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
production environment. We are continuously facing below warnings from one of 
our monitoring tool, but there is no suspicious error from system.log file. 
Also there is sufficient memory available for Heap/server.

 
 warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory

The java memory pool Par Survivor Space on esg3-c-cassv3-03p is now using 100.0 
% of the maximum.
 This started at 2021-02-08 11:44:17 GMT - or 0h 56m ago.

Datasource: Cassandra JVM Memory Pools 
 Instance: Cassandra JVM Memory Pools -Par Survivor SpaceDatapoint: 
UsedPoolMemoryEffective Thresholds: >= 90
 Please help us to fix this issue ASAP.

 

Thanks!


> Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
> -
>
> Key: CASSANDRA-16430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16430
> Project: Cassandra
>  Issue Type: Task
>  Components: Local/Config, Observability/Metrics
>Reporter: sharanya
>Priority: Urgent
>
> I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
> production environment. We are continuously facing below warnings from one of 
> our monitoring tool, but there is no suspicious error from system.log file. 
> Also there is sufficient memory available for Heap/server.
>  
>  warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
> Datasource: Cassandra JVM Memory Pools -Par Survivor Space
> Datapoint: UsedPoolMemory
> Level: warn
> Start: 2021-02-08 10:11:17 GMT
> Duration: 1h 0m
> Value: 100.0
> ClearValue: 1.3631
> Reason: UsedPoolMemory is not >= 90: the current value is 1.3631
> The java memory pool Par Survivor Space on esg3-c-cassv3-03p is now using 
> 100.0 % of the maximum.
>  
> We see the below message in debug.log file.
> 2021-02-08 08:43:51,310 INFO  [ReadStage-1] NoSpamLogger.java:91 - Maximum 
> memory usage reached (512.000MiB), cannot allocate chunk of 1.000MiB
> 2021-02-08 08:47:28,344 DEBUG [COMMIT-LOG-ALLOCATOR] 
> AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a 
> fresh one
>  
>  
> Please help us to fix this issue ASAP.
>  
>  
>  
> Thanks!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16216) Add tests to cover ClientMetrics metrics

[Benjamin Lerer (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-16216:
---
Status: Review In Progress  (was: Patch Available)

> Add tests to cover ClientMetrics metrics
> 
>
> Key: CASSANDRA-16216
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16216
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java, Test/unit
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 4.0-rc
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16430) Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory

[sharanya (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

sharanya updated CASSANDRA-16430:
-
Description: 
I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
production environment. We are continuously facing below warnings from one of 
our monitoring tool, but there is no suspicious error from system.log file. 
Also there is sufficient memory available for Heap/server.

 
 warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory

The java memory pool Par Survivor Space on esg3-c-cassv3-03p is now using 100.0 
% of the maximum.
 This started at 2021-02-08 11:44:17 GMT - or 0h 56m ago.

Datasource: Cassandra JVM Memory Pools 
 Instance: Cassandra JVM Memory Pools -Par Survivor SpaceDatapoint: 
UsedPoolMemoryEffective Thresholds: >= 90
 Please help us to fix this issue ASAP.

 

Thanks!

  was:
I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
production environment. We are continuously facing below warnings from one of 
our monitoring tool, but no error found from system.log file. Also there is 
sufficient memory available for Heap/server.

 
 warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory

The java memory pool Par Survivor Space on esg3-c-cassv3-03p is now using 100.0 
% of the maximum.
 This started at 2021-02-08 11:44:17 GMT - or 0h 56m ago.

Datasource: Cassandra JVM Memory Pools 
 Instance: Cassandra JVM Memory Pools -Par Survivor SpaceDatapoint: 
UsedPoolMemoryEffective Thresholds: >= 90
 Please help us to fix this issue ASAP.

 

Thanks!


> Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
> -
>
> Key: CASSANDRA-16430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16430
> Project: Cassandra
>  Issue Type: Task
>  Components: Local/Config, Observability/Metrics
>Reporter: sharanya
>Priority: Urgent
>
> I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
> production environment. We are continuously facing below warnings from one of 
> our monitoring tool, but there is no suspicious error from system.log file. 
> Also there is sufficient memory available for Heap/server.
>  
>  warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
> The java memory pool Par Survivor Space on esg3-c-cassv3-03p is now using 
> 100.0 % of the maximum.
>  This started at 2021-02-08 11:44:17 GMT - or 0h 56m ago.
> Datasource: Cassandra JVM Memory Pools 
>  Instance: Cassandra JVM Memory Pools -Par Survivor SpaceDatapoint: 
> UsedPoolMemoryEffective Thresholds: >= 90
>  Please help us to fix this issue ASAP.
>  
> Thanks!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16430) Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory

[sharanya (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

sharanya updated CASSANDRA-16430:
-
Description: 
I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
production environment. We are continuously facing below warnings from one of 
our monitoring tool, but no error found from system.log file. Also there is 
sufficient memory available for Heap/server.

 
 warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory

The java memory pool Par Survivor Space on esg3-c-cassv3-03p is now using 100.0 
% of the maximum.
 This started at 2021-02-08 11:44:17 GMT - or 0h 56m ago.

Datasource: Cassandra JVM Memory Pools 
 Instance: Cassandra JVM Memory Pools -Par Survivor SpaceDatapoint: 
UsedPoolMemoryEffective Thresholds: >= 90
 Please help us to fix this issue ASAP.

 

Thanks!

  was:
I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
production environment. We found below warning from one of our monitoring tool, 
but no error found from system.log file. Also there is sufficient memory 
available for Heap/server.

 
 warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory

The java memory pool Par Survivor Space on esg3-c-cassv3-03p is now using 100.0 
% of the maximum.
 This started at 2021-02-08 11:44:17 GMT - or 0h 56m ago.

Datasource: Cassandra JVM Memory Pools 
 Instance: Cassandra JVM Memory Pools -Par Survivor SpaceDatapoint: 
UsedPoolMemoryEffective Thresholds: >= 90
 Please help us to fix this issue ASAP.

 

Thanks!


> Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
> -
>
> Key: CASSANDRA-16430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16430
> Project: Cassandra
>  Issue Type: Task
>  Components: Local/Config, Observability/Metrics
>Reporter: sharanya
>Priority: Urgent
>
> I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
> production environment. We are continuously facing below warnings from one of 
> our monitoring tool, but no error found from system.log file. Also there is 
> sufficient memory available for Heap/server.
>  
>  warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
> The java memory pool Par Survivor Space on esg3-c-cassv3-03p is now using 
> 100.0 % of the maximum.
>  This started at 2021-02-08 11:44:17 GMT - or 0h 56m ago.
> Datasource: Cassandra JVM Memory Pools 
>  Instance: Cassandra JVM Memory Pools -Par Survivor SpaceDatapoint: 
> UsedPoolMemoryEffective Thresholds: >= 90
>  Please help us to fix this issue ASAP.
>  
> Thanks!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16430) Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory

[sharanya (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

sharanya updated CASSANDRA-16430:
-
Change Category: Performance
 Complexity: Challenging
   Priority: Urgent  (was: Normal)
 Status: Open  (was: Triage Needed)

> Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
> -
>
> Key: CASSANDRA-16430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16430
> Project: Cassandra
>  Issue Type: Task
>  Components: Local/Config, Observability/Metrics
>Reporter: sharanya
>Priority: Urgent
>
> I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
> production environment. We found below warning from one of our monitoring 
> tool, but no error found from system.log file. Also there is sufficient 
> memory available for Heap/server.
>  
>  warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
> The java memory pool Par Survivor Space on esg3-c-cassv3-03p is now using 
> 100.0 % of the maximum.
>  This started at 2021-02-08 11:44:17 GMT - or 0h 56m ago.
> Datasource: Cassandra JVM Memory Pools 
>  Instance: Cassandra JVM Memory Pools -Par Survivor SpaceDatapoint: 
> UsedPoolMemoryEffective Thresholds: >= 90
>  Please help us to fix this issue ASAP.
>  
> Thanks!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-16430) Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory

[sharanya (Jira)]

sharanya created CASSANDRA-16430:


 Summary: Cassandra JVM Memory Pools -Par Survivor Space 
UsedPoolMemory
 Key: CASSANDRA-16430
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16430
 Project: Cassandra
  Issue Type: Task
  Components: Local/Config, Observability/Metrics
Reporter: sharanya


I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
production environment. We found below warning from one of our monitoring tool, 
but no error found from system.log file. Also there is sufficient memory 
available for Heap/server.

 
warn - esg3-c-cassv3-03p Cassandra JVM Memory Pools -Par Survivor Space 
UsedPoolMemory

The java memory pool Par Survivor Space  on esg3-c-cassv3-03p is now using 
100.0 % of the maximum.
This started at 2021-02-08 11:44:17 GMT - or 0h 56m ago.

Datasource: Cassandra JVM Memory Pools 
Instance: Cassandra JVM Memory Pools -Par Survivor SpaceDatapoint: 
UsedPoolMemoryEffective Thresholds: >= 90
Please help us to fix this issue ASAP.

 

Thanks!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16430) Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory

[sharanya (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

sharanya updated CASSANDRA-16430:
-
Description: 
I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
production environment. We found below warning from one of our monitoring tool, 
but no error found from system.log file. Also there is sufficient memory 
available for Heap/server.

 
 warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory

The java memory pool Par Survivor Space on esg3-c-cassv3-03p is now using 100.0 
% of the maximum.
 This started at 2021-02-08 11:44:17 GMT - or 0h 56m ago.

Datasource: Cassandra JVM Memory Pools 
 Instance: Cassandra JVM Memory Pools -Par Survivor SpaceDatapoint: 
UsedPoolMemoryEffective Thresholds: >= 90
 Please help us to fix this issue ASAP.

 

Thanks!

  was:
I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
production environment. We found below warning from one of our monitoring tool, 
but no error found from system.log file. Also there is sufficient memory 
available for Heap/server.

 
warn - esg3-c-cassv3-03p Cassandra JVM Memory Pools -Par Survivor Space 
UsedPoolMemory

The java memory pool Par Survivor Space  on esg3-c-cassv3-03p is now using 
100.0 % of the maximum.
This started at 2021-02-08 11:44:17 GMT - or 0h 56m ago.

Datasource: Cassandra JVM Memory Pools 
Instance: Cassandra JVM Memory Pools -Par Survivor SpaceDatapoint: 
UsedPoolMemoryEffective Thresholds: >= 90
Please help us to fix this issue ASAP.

 

Thanks!


> Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
> -
>
> Key: CASSANDRA-16430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16430
> Project: Cassandra
>  Issue Type: Task
>  Components: Local/Config, Observability/Metrics
>Reporter: sharanya
>Priority: Normal
>
> I have a 9 node cluster of cassandra, with 2gb of data on each node in our 
> production environment. We found below warning from one of our monitoring 
> tool, but no error found from system.log file. Also there is sufficient 
> memory available for Heap/server.
>  
>  warn - Cassandra JVM Memory Pools -Par Survivor Space UsedPoolMemory
> The java memory pool Par Survivor Space on esg3-c-cassv3-03p is now using 
> 100.0 % of the maximum.
>  This started at 2021-02-08 11:44:17 GMT - or 0h 56m ago.
> Datasource: Cassandra JVM Memory Pools 
>  Instance: Cassandra JVM Memory Pools -Par Survivor SpaceDatapoint: 
> UsedPoolMemoryEffective Thresholds: >= 90
>  Please help us to fix this issue ASAP.
>  
> Thanks!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16307) GROUP BY queries with paging can return deleted data

[Sam Tunnicliffe (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-16307:

Reviewers: Benjamin Lerer, Sam Tunnicliffe  (was: Benjamin Lerer, beobal, 
Sam Tunnicliffe)

> GROUP BY queries with paging can return deleted data
> 
>
> Key: CASSANDRA-16307
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16307
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination
>Reporter: Andres de la Peña
>Assignee: Alex Petrov
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> {{GROUP BY}} queries using paging and CL>ONE/LOCAL_ONE. This dtest reproduces 
> the problem:
> {code:java}
> try (Cluster cluster = init(Cluster.create(2)))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (pk int, ck int, 
> PRIMARY KEY (pk, ck))"));
> ICoordinator coordinator = cluster.coordinator(1);
> coordinator.execute(withKeyspace("INSERT INTO %s.t (pk, ck) VALUES (0, 
> 0)"), ConsistencyLevel.ALL);
> coordinator.execute(withKeyspace("INSERT INTO %s.t (pk, ck) VALUES (1, 
> 1)"), ConsistencyLevel.ALL);
> 
> cluster.get(1).executeInternal(withKeyspace("DELETE FROM %s.t WHERE pk=0 
> AND ck=0"));
> cluster.get(2).executeInternal(withKeyspace("DELETE FROM %s.t WHERE pk=1 
> AND ck=1"));
> String query = withKeyspace("SELECT * FROM %s.t GROUP BY pk");
> Iterator rows = coordinator.executeWithPaging(query, 
> ConsistencyLevel.ALL, 1);
> assertRows(Iterators.toArray(rows, Object[].class));
> }
> {code}
> Using a 2-node cluster and RF=2, the test inserts two partitions in both 
> nodes. Then it locally deletes each row in a separate node, so each node sees 
> a different partition alive, but reconciliation should produce no alive 
> partitions. However, a {{GROUP BY}} query using a page size of 1 wrongly 
> returns one of the rows.
> This has been detected during CASSANDRA-16180, and it is probably related to 
> CASSANDRA-15459, which solved a similar problem for group-by queries with 
> limit, instead of paging.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16307) GROUP BY queries with paging can return deleted data

[Sam Tunnicliffe (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281001#comment-17281001
 ] 

Sam Tunnicliffe commented on CASSANDRA-16307:
-

Looks good to me, assuming that you're planning to include the logic inversion 
patch on the 3.11 branch too. One minor nit is the naming of the new methods. 
They feel slightly off because counters/limits don't have content, so maybe a 
better options could be:

{{maybeUnderCounted / maybeUnderCountedForPartition}} or {{maybeShort / 
maybeShortForPartition}} ?

> GROUP BY queries with paging can return deleted data
> 
>
> Key: CASSANDRA-16307
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16307
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination
>Reporter: Andres de la Peña
>Assignee: Alex Petrov
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> {{GROUP BY}} queries using paging and CL>ONE/LOCAL_ONE. This dtest reproduces 
> the problem:
> {code:java}
> try (Cluster cluster = init(Cluster.create(2)))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (pk int, ck int, 
> PRIMARY KEY (pk, ck))"));
> ICoordinator coordinator = cluster.coordinator(1);
> coordinator.execute(withKeyspace("INSERT INTO %s.t (pk, ck) VALUES (0, 
> 0)"), ConsistencyLevel.ALL);
> coordinator.execute(withKeyspace("INSERT INTO %s.t (pk, ck) VALUES (1, 
> 1)"), ConsistencyLevel.ALL);
> 
> cluster.get(1).executeInternal(withKeyspace("DELETE FROM %s.t WHERE pk=0 
> AND ck=0"));
> cluster.get(2).executeInternal(withKeyspace("DELETE FROM %s.t WHERE pk=1 
> AND ck=1"));
> String query = withKeyspace("SELECT * FROM %s.t GROUP BY pk");
> Iterator rows = coordinator.executeWithPaging(query, 
> ConsistencyLevel.ALL, 1);
> assertRows(Iterators.toArray(rows, Object[].class));
> }
> {code}
> Using a 2-node cluster and RF=2, the test inserts two partitions in both 
> nodes. Then it locally deletes each row in a separate node, so each node sees 
> a different partition alive, but reconciliation should produce no alive 
> partitions. However, a {{GROUP BY}} query using a page size of 1 wrongly 
> returns one of the rows.
> This has been detected during CASSANDRA-16180, and it is probably related to 
> CASSANDRA-15459, which solved a similar problem for group-by queries with 
> limit, instead of paging.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16307) GROUP BY queries with paging can return deleted data

[Sam Tunnicliffe (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-16307:

Reviewers: Benjamin Lerer, beobal, Sam Tunnicliffe  (was: Benjamin Lerer, 
beobal)
   Status: Review In Progress  (was: Patch Available)

> GROUP BY queries with paging can return deleted data
> 
>
> Key: CASSANDRA-16307
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16307
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination
>Reporter: Andres de la Peña
>Assignee: Alex Petrov
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> {{GROUP BY}} queries using paging and CL>ONE/LOCAL_ONE. This dtest reproduces 
> the problem:
> {code:java}
> try (Cluster cluster = init(Cluster.create(2)))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (pk int, ck int, 
> PRIMARY KEY (pk, ck))"));
> ICoordinator coordinator = cluster.coordinator(1);
> coordinator.execute(withKeyspace("INSERT INTO %s.t (pk, ck) VALUES (0, 
> 0)"), ConsistencyLevel.ALL);
> coordinator.execute(withKeyspace("INSERT INTO %s.t (pk, ck) VALUES (1, 
> 1)"), ConsistencyLevel.ALL);
> 
> cluster.get(1).executeInternal(withKeyspace("DELETE FROM %s.t WHERE pk=0 
> AND ck=0"));
> cluster.get(2).executeInternal(withKeyspace("DELETE FROM %s.t WHERE pk=1 
> AND ck=1"));
> String query = withKeyspace("SELECT * FROM %s.t GROUP BY pk");
> Iterator rows = coordinator.executeWithPaging(query, 
> ConsistencyLevel.ALL, 1);
> assertRows(Iterators.toArray(rows, Object[].class));
> }
> {code}
> Using a 2-node cluster and RF=2, the test inserts two partitions in both 
> nodes. Then it locally deletes each row in a separate node, so each node sees 
> a different partition alive, but reconciliation should produce no alive 
> partitions. However, a {{GROUP BY}} query using a page size of 1 wrongly 
> returns one of the rows.
> This has been detected during CASSANDRA-16180, and it is probably related to 
> CASSANDRA-15459, which solved a similar problem for group-by queries with 
> limit, instead of paging.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16377) Add dtest-upgrade and dtest-large-novnode to Jenkins pipeline

[Michael Semb Wever (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-16377:
---
Reviewers: Jacek Lewandowski, Tomasz Lasica  (was: Tomasz Lasica)

> Add dtest-upgrade and dtest-large-novnode to Jenkins pipeline
> -
>
> Key: CASSANDRA-16377
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16377
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Low
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> Now that dtest-upgrade test runs are passing they can be considered for 
> addition to the jenkins pipeline. And dtest-large-novnode only as one failure 
> so should be added too.  
> https://the-asf.slack.com/archives/CK23JSY2K/p1610132643162700 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16377) Add dtest-upgrade and dtest-large-novnode to Jenkins pipeline

[Michael Semb Wever (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17280955#comment-17280955
 ] 

Michael Semb Wever commented on CASSANDRA-16377:


Thanks [~jlewandowski].

Just waiting on CASSANDRA-16401 to be finished committed and all its 
post-commit builds to go through, before committing this one.

> Add dtest-upgrade and dtest-large-novnode to Jenkins pipeline
> -
>
> Key: CASSANDRA-16377
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16377
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Low
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> Now that dtest-upgrade test runs are passing they can be considered for 
> addition to the jenkins pipeline. And dtest-large-novnode only as one failure 
> so should be added too.  
> https://the-asf.slack.com/archives/CK23JSY2K/p1610132643162700 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16401) Remove the Jenkins plaintext reports

[Michael Semb Wever (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-16401:
---
Fix Version/s: (was: 4.0.x)
   (was: 3.11.x)
   (was: 3.0.x)
   (was: 2.2.x)
   4.0
   4.0-beta5
   3.11.11
   3.0.25
   2.2.20

> Remove the Jenkins plaintext reports
> 
>
> Key: CASSANDRA-16401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16401
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0-beta5, 4.0
>
> Attachments: jenkins-local-consoleText.txt
>
>
> Remove the generation of the {{cassandra-test-report.txt}} plain text report 
> (that was added in CASSANDRA-15729). 
> The xml report is now at 3GB and too big to fit through any xsl 
> transformation. It is only going to get bigger as more tests and test stages 
> are added.
> This size failure is seen in the pipeline logs as
> {noformat}
> docker run --rm -v 
> /home/jenkins/jenkins-slave/workspace/Cassandra-trunk@2:/tmp 
> apache_cassandra_ci/generate_plaintext_test_report
> Warning: at xsl:stylesheet on line 4 column 49 of cassandra-test-report.xsl:  
> Running an XSLT 1.0 stylesheet with an XSLT 2.0 processor
>  java.lang.IndexOutOfBoundsException: [660540429,660540717]
>   at net.sf.saxon.tinytree.LargeStringBuffer.subSequence(Unknown Source)
>   at net.sf.saxon.tinytree.TinyTextImpl.getStringValue(Unknown Source)
>   at net.sf.saxon.tinytree.TinyParentNodeImpl.getStringValue(Unknown 
> Source)
> …
> {noformat}
> In addition we can and do now archive everything to 
> https://nightlies.apache.org/cassandra/ , solving the original requirements 
> of CASSANDRA-15729 of providing a permanent archive of test results.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16401) Remove the Jenkins plaintext reports

[Michael Semb Wever (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-16401:
---
Source Control Link: 
https://github.com/apache/cassandra/commit/9fa2c28dc2d701b7b2cc90d4282f05ee1a221943

> Remove the Jenkins plaintext reports
> 
>
> Key: CASSANDRA-16401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16401
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0.x
>
> Attachments: jenkins-local-consoleText.txt
>
>
> Remove the generation of the {{cassandra-test-report.txt}} plain text report 
> (that was added in CASSANDRA-15729). 
> The xml report is now at 3GB and too big to fit through any xsl 
> transformation. It is only going to get bigger as more tests and test stages 
> are added.
> This size failure is seen in the pipeline logs as
> {noformat}
> docker run --rm -v 
> /home/jenkins/jenkins-slave/workspace/Cassandra-trunk@2:/tmp 
> apache_cassandra_ci/generate_plaintext_test_report
> Warning: at xsl:stylesheet on line 4 column 49 of cassandra-test-report.xsl:  
> Running an XSLT 1.0 stylesheet with an XSLT 2.0 processor
>  java.lang.IndexOutOfBoundsException: [660540429,660540717]
>   at net.sf.saxon.tinytree.LargeStringBuffer.subSequence(Unknown Source)
>   at net.sf.saxon.tinytree.TinyTextImpl.getStringValue(Unknown Source)
>   at net.sf.saxon.tinytree.TinyParentNodeImpl.getStringValue(Unknown 
> Source)
> …
> {noformat}
> In addition we can and do now archive everything to 
> https://nightlies.apache.org/cassandra/ , solving the original requirements 
> of CASSANDRA-15729 of providing a permanent archive of test results.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16401) Remove the Jenkins plaintext reports

[Michael Semb Wever (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17280953#comment-17280953
 ] 

Michael Semb Wever commented on CASSANDRA-16401:


Committed in-tree patches with 
[9fa2c28dc2d701b7b2cc90d4282f05ee1a221943|https://github.com/apache/cassandra/commit/9fa2c28dc2d701b7b2cc90d4282f05ee1a221943].

> Remove the Jenkins plaintext reports
> 
>
> Key: CASSANDRA-16401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16401
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0.x
>
> Attachments: jenkins-local-consoleText.txt
>
>
> Remove the generation of the {{cassandra-test-report.txt}} plain text report 
> (that was added in CASSANDRA-15729). 
> The xml report is now at 3GB and too big to fit through any xsl 
> transformation. It is only going to get bigger as more tests and test stages 
> are added.
> This size failure is seen in the pipeline logs as
> {noformat}
> docker run --rm -v 
> /home/jenkins/jenkins-slave/workspace/Cassandra-trunk@2:/tmp 
> apache_cassandra_ci/generate_plaintext_test_report
> Warning: at xsl:stylesheet on line 4 column 49 of cassandra-test-report.xsl:  
> Running an XSLT 1.0 stylesheet with an XSLT 2.0 processor
>  java.lang.IndexOutOfBoundsException: [660540429,660540717]
>   at net.sf.saxon.tinytree.LargeStringBuffer.subSequence(Unknown Source)
>   at net.sf.saxon.tinytree.TinyTextImpl.getStringValue(Unknown Source)
>   at net.sf.saxon.tinytree.TinyParentNodeImpl.getStringValue(Unknown 
> Source)
> …
> {noformat}
> In addition we can and do now archive everything to 
> https://nightlies.apache.org/cassandra/ , solving the original requirements 
> of CASSANDRA-15729 of providing a permanent archive of test results.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch cassandra-3.11 updated (b0a5a7c -> aa92e88)

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a change to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


from b0a5a7c  Merge branch 'cassandra-3.0' into cassandra-3.11
 new 9fa2c28  Jenkins builds to provide link to nightlies archive, and 
remove the Jenkins plaintext reports
 new 19b3945  Merge branch 'cassandra-2.2' into cassandra-3.0
 new aa92e88  Merge branch 'cassandra-3.0' into cassandra-3.11

The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .jenkins/Jenkinsfile | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git

commit aa92e8868800460908717f1a1a9dbb7ac67d79cc
Merge: b0a5a7c 19b3945
Author: Mick Semb Wever 
AuthorDate: Mon Feb 8 12:07:11 2021 +0100

Merge branch 'cassandra-3.0' into cassandra-3.11

 .jenkins/Jenkinsfile | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)



-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch trunk updated (637b242 -> 7cb3adc)

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a change to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


from 637b242  Merge branch 'cassandra-3.11' into trunk
 new 9fa2c28  Jenkins builds to provide link to nightlies archive, and 
remove the Jenkins plaintext reports
 new 19b3945  Merge branch 'cassandra-2.2' into cassandra-3.0
 new aa92e88  Merge branch 'cassandra-3.0' into cassandra-3.11
 new 7cb3adc  Merge branch 'cassandra-3.11' into trunk

The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .jenkins/Jenkinsfile | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch cassandra-3.0 updated (4333b5c -> 19b3945)

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a change to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


from 4333b5c  Merge branch 'cassandra-2.2' into cassandra-3.0
 new 9fa2c28  Jenkins builds to provide link to nightlies archive, and 
remove the Jenkins plaintext reports
 new 19b3945  Merge branch 'cassandra-2.2' into cassandra-3.0

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .jenkins/Jenkinsfile | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git

commit 7cb3adc3bf38221a0edf597cc7a0f08178347ff4
Merge: 637b242 aa92e88
Author: Mick Semb Wever 
AuthorDate: Mon Feb 8 12:10:01 2021 +0100

Merge branch 'cassandra-3.11' into trunk

 .jenkins/Jenkinsfile | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)



-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git

commit 19b3945297303bbaa4d18ae088e53bb65391557f
Merge: 4333b5c 9fa2c28
Author: Mick Semb Wever 
AuthorDate: Mon Feb 8 12:06:06 2021 +0100

Merge branch 'cassandra-2.2' into cassandra-3.0

 .jenkins/Jenkinsfile | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)



-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch cassandra-2.2 updated: Jenkins builds to provide link to nightlies archive, and remove the Jenkins plaintext reports

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to branch cassandra-2.2
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/cassandra-2.2 by this push:
 new 9fa2c28  Jenkins builds to provide link to nightlies archive, and 
remove the Jenkins plaintext reports
9fa2c28 is described below

commit 9fa2c28dc2d701b7b2cc90d4282f05ee1a221943
Author: Mick Semb Wever 
AuthorDate: Sat Jan 23 11:18:46 2021 +0100

Jenkins builds to provide link to nightlies archive, and remove the Jenkins 
plaintext reports

 patch by Mick Semb Wever; reviewed by Tomek Łasica for CASSANDRA-16401
---
 .jenkins/Jenkinsfile | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/.jenkins/Jenkinsfile b/.jenkins/Jenkinsfile
index aef9661..e03366b 100644
--- a/.jenkins/Jenkinsfile
+++ b/.jenkins/Jenkinsfile
@@ -232,17 +232,17 @@ ${CHANGES}
 Failed Tests:
 ${FAILED_TESTS,maxTests=500,showMessage=false,showStack=false}
 ---
-${FILE,path="cassandra-test-report.txt"}
+For complete test report and logs see 
https://nightlies.apache.org/cassandra/${JOB_NAME}/${BUILD_NUMBER}/
 '''
   }
   sh "echo \"cassandra-builds at: `git -C cassandra-builds log -1 
--pretty=format:'%h %an %ad %s'`\" > builds.head"
   sh "find . -type f -name \\*.head -exec cat {} \\;"
-  sh "xz cassandra-test-report.txt TESTS-TestSuites.xml"
+  sh "xz TESTS-TestSuites.xml"
+  sh "echo \"For test report and logs see 
https://nightlies.apache.org/cassandra/${JOB_NAME}/${BUILD_NUMBER}/\";
   }
   post {
   always {
-  archiveArtifacts artifacts: 'cassandra-test-report.txt.xz', 
fingerprint: true
-  sshPublisher(publishers: [sshPublisherDesc(configName: 
'Nightlies', transfers: [sshTransfer(remoteDirectory: 
'cassandra/${JOB_NAME}/${BUILD_NUMBER}/', sourceFiles: 
'cassandra-test-report.txt.xz, TESTS-TestSuites.xml.xz')])])
+  sshPublisher(publishers: [sshPublisherDesc(configName: 
'Nightlies', transfers: [sshTransfer(remoteDirectory: 
'cassandra/${JOB_NAME}/${BUILD_NUMBER}/', sourceFiles: 
'TESTS-TestSuites.xml.xz')])])
   }
   }
 }


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra-builds] branch trunk updated: Pin 'cryptography<3.4' (dep to urllib3) to maintain support for python2 in the docker image used for building the rpm packages

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git


The following commit(s) were added to refs/heads/trunk by this push:
 new 117f46c  Pin 'cryptography<3.4' (dep to urllib3) to maintain support 
for python2 in the docker image used for building the rpm packages
117f46c is described below

commit 117f46c09f3c304bccb67ea193401eb0afa73c86
Author: Mick Semb Wever 
AuthorDate: Mon Feb 8 11:58:57 2021 +0100

Pin 'cryptography<3.4' (dep to urllib3) to maintain support for python2 in 
the docker image used for building the rpm packages

 patch by Mick Semb Wever; reviewed by Sam Tunnicliffe
---
 docker/centos7-image.docker | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docker/centos7-image.docker b/docker/centos7-image.docker
index 8235633..a6c0da8 100644
--- a/docker/centos7-image.docker
+++ b/docker/centos7-image.docker
@@ -33,7 +33,7 @@ RUN yum -y install python2-pip
 
 # install ssl enabled urllib for retrieving python packages
 # this will produce some ssl related warnings, which will be resolved once the 
package has been installed
-RUN pip install 'idna<3' urllib3[secure]
+RUN pip install 'idna<3' 'cryptography<3.4' urllib3[secure]
 
 # upgrade to modern pip version
 RUN pip install --upgrade 'pip<21'


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16401) Remove the Jenkins plaintext reports

[Michael Semb Wever (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-16401:
---
Status: Ready to Commit  (was: Review In Progress)

> Remove the Jenkins plaintext reports
> 
>
> Key: CASSANDRA-16401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16401
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0.x
>
> Attachments: jenkins-local-consoleText.txt
>
>
> Remove the generation of the {{cassandra-test-report.txt}} plain text report 
> (that was added in CASSANDRA-15729). 
> The xml report is now at 3GB and too big to fit through any xsl 
> transformation. It is only going to get bigger as more tests and test stages 
> are added.
> This size failure is seen in the pipeline logs as
> {noformat}
> docker run --rm -v 
> /home/jenkins/jenkins-slave/workspace/Cassandra-trunk@2:/tmp 
> apache_cassandra_ci/generate_plaintext_test_report
> Warning: at xsl:stylesheet on line 4 column 49 of cassandra-test-report.xsl:  
> Running an XSLT 1.0 stylesheet with an XSLT 2.0 processor
>  java.lang.IndexOutOfBoundsException: [660540429,660540717]
>   at net.sf.saxon.tinytree.LargeStringBuffer.subSequence(Unknown Source)
>   at net.sf.saxon.tinytree.TinyTextImpl.getStringValue(Unknown Source)
>   at net.sf.saxon.tinytree.TinyParentNodeImpl.getStringValue(Unknown 
> Source)
> …
> {noformat}
> In addition we can and do now archive everything to 
> https://nightlies.apache.org/cassandra/ , solving the original requirements 
> of CASSANDRA-15729 of providing a permanent archive of test results.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16371) JMH Reports in ci-cassandra.apache.org

[Michael Semb Wever (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-16371:
---
  Fix Version/s: (was: 4.0.x)
 (was: 3.11.x)
 (was: 3.0.x)
 (was: 2.2.x)
 4.0
 4.0-beta5
 3.11.11
 3.0.25
 2.2.20
Source Control Link: 
https://github.com/apache/cassandra/commit/d19f35243733d211a81de149ef3b995fc7310854
 
https://github.com/apache/cassandra-builds/commit/8eff3855117b37d304711ce182edc516ea0fbd35
  (was: 
https://github.com/apache/cassandra/commit/d19f35243733d211a81de149ef3b995fc7310854)
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

> JMH Reports in ci-cassandra.apache.org
> --
>
> Key: CASSANDRA-16371
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16371
> Project: Cassandra
>  Issue Type: Task
>  Components: CI, Test/benchmark
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Low
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0-beta5, 4.0
>
> Attachments: JMH #1 [Jenkins .png, JMH #7 [Jenkins .png
>
>
> Enable the  jenkins [jmh reporting 
> plugin|https://plugins.jenkins.io/jmh-report/] and add a jenkins job to run 
> the `ant microbench` target.
> Patches
>  - cassandra-builds: 
> https://github.com/apache/cassandra-builds/compare/trunk...thelastpickle:mck/jenkins-microbench
>  - cassandra: 
> https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_jmh_report
> Example results are:
>  - CI JMH report: 
> https://ci-cassandra.apache.org/job/Cassandra-devbranch-microbench/jdk=jdk_11_latest,label=cassandra/1/jmh-run-report/
>   (screenshot attached)
> The value of these reports is limited, with different hardware jenkins agents 
> and two executors on each agent. But visibility from the reports shows what 
> is possible, how agents are different (and behave under saturation), and most 
> importantly when jmh classes are broken (there are no unit tests on these 
> benchmark tests).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16371) JMH Reports in ci-cassandra.apache.org

[Michael Semb Wever (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17280934#comment-17280934
 ] 

Michael Semb Wever commented on CASSANDRA-16371:


Committed 
[d19f35243733d211a81de149ef3b995fc7310854|https://github.com/apache/cassandra/commit/d19f35243733d211a81de149ef3b995fc7310854]
 and 
[8eff3855117b37d304711ce182edc516ea0fbd35|https://github.com/apache/cassandra-builds/commit/8eff3855117b37d304711ce182edc516ea0fbd35].

> JMH Reports in ci-cassandra.apache.org
> --
>
> Key: CASSANDRA-16371
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16371
> Project: Cassandra
>  Issue Type: Task
>  Components: CI, Test/benchmark
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Low
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0.x
>
> Attachments: JMH #1 [Jenkins .png, JMH #7 [Jenkins .png
>
>
> Enable the  jenkins [jmh reporting 
> plugin|https://plugins.jenkins.io/jmh-report/] and add a jenkins job to run 
> the `ant microbench` target.
> Patches
>  - cassandra-builds: 
> https://github.com/apache/cassandra-builds/compare/trunk...thelastpickle:mck/jenkins-microbench
>  - cassandra: 
> https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_jmh_report
> Example results are:
>  - CI JMH report: 
> https://ci-cassandra.apache.org/job/Cassandra-devbranch-microbench/jdk=jdk_11_latest,label=cassandra/1/jmh-run-report/
>   (screenshot attached)
> The value of these reports is limited, with different hardware jenkins agents 
> and two executors on each agent. But visibility from the reports shows what 
> is possible, how agents are different (and behave under saturation), and most 
> importantly when jmh classes are broken (there are no unit tests on these 
> benchmark tests).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra-builds] branch trunk updated: Add microbench as a target in cassandra-test.sh, and as a jenkins job

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git


The following commit(s) were added to refs/heads/trunk by this push:
 new 8eff385  Add microbench as a target in cassandra-test.sh, and as a 
jenkins job
8eff385 is described below

commit 8eff3855117b37d304711ce182edc516ea0fbd35
Author: Mick Semb Wever 
AuthorDate: Sat Dec 26 19:20:24 2020 +0100

Add microbench as a target in cassandra-test.sh, and as a jenkins job

The jenkins jmh report plugin needs the jmh-result.json remaining.

 patch by Mick Semb Wever; reviewed by Tomek Łasica for CASSANDRA-16371
---
 build-scripts/cassandra-test.sh   |  3 +++
 jenkins-dsl/cassandra_job_dsl_seed.groovy | 24 +---
 2 files changed, 20 insertions(+), 7 deletions(-)

diff --git a/build-scripts/cassandra-test.sh b/build-scripts/cassandra-test.sh
index 499039f..a9b0a8d 100755
--- a/build-scripts/cassandra-test.sh
+++ b/build-scripts/cassandra-test.sh
@@ -74,6 +74,9 @@ _main() {
   ant fqltool-build-test
   ant $target -Dtmp.dir="$(pwd)/tmp" || echo "failed $target"
   ;;
+"microbench")
+  ant $target -Dtmp.dir="$(pwd)/tmp" -Dmaven.test.failure.ignore=true
+  ;;
 "test")
   ant testclasslist -Dtest.classlistfile=<( _list_tests "unit" ) 
-Dtmp.dir="${TMP_DIR}" || echo "failed $target"
   ;;
diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy 
b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index f2dfcf9..16aa48e 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -49,7 +49,7 @@ if(binding.hasVariable("CASSANDRA_BRANCHES")) {
 cassandraBranches = "${CASSANDRA_BRANCHES}".split(",")
 }
 // Ant test targets
-def testTargets = ['test', 'test-burn', 'test-cdc', 'test-compression', 
'stress-test', 'fqltool-test', 'long-test', 'jvm-dtest', 'jvm-dtest-upgrade']
+def testTargets = ['test', 'test-burn', 'test-cdc', 'test-compression', 
'stress-test', 'fqltool-test', 'long-test', 'jvm-dtest', 'jvm-dtest-upgrade', 
'microbench']
 if(binding.hasVariable("CASSANDRA_ANT_TEST_TARGETS")) {
 testTargets = "${CASSANDRA_ANT_TEST_TARGETS}".split(",")
 }
@@ -233,7 +233,7 @@ matrixJob('Cassandra-template-test') {
 steps {
 buildDescription('', buildDescStr)
 shell("""
-git clean -xdff ;
+git clean -xdff -e build/test/jmh-result.json ;
 git clone --depth 1 --single-branch -b ${buildsBranch} 
${buildsRepo} ;
 echo "cassandra-builds at: `git -C cassandra-builds log -1 
--pretty=format:'%h %an %ad %s'`" ;
 echo "\${JOB_BASE_NAME}: `git log -1 --pretty=format:'%h %an 
%ad %s'`" > \${JOB_BASE_NAME}.head ;
@@ -253,7 +253,7 @@ matrixJob('Cassandra-template-test') {
 publishOverSsh {
 server('Nightlies') {
 transferSet {
-sourceFiles("build/test/logs/**")
+
sourceFiles("build/test/logs/**,build/test/jmh-result.json")
 remoteDirectory("cassandra/\${JOB_NAME}/\${BUILD_NUMBER}/")
 }
 }
@@ -262,7 +262,7 @@ matrixJob('Cassandra-template-test') {
 postBuildTask {
 task('.', """
 echo "Finding job process orphans…"; if pgrep -af 
"\${JOB_BASE_NAME}"; then pkill -9 -f "\${JOB_BASE_NAME}"; fi;
-echo "Cleaning project…"; git clean -xdff ;
+echo "Cleaning project…"; git clean -xdff -e 
build/test/jmh-result.json ;
 echo "Pruning docker…" ; docker system prune --all --force 
--filter "until=${maxJobHours}h"  ;
 echo "Reporting disk usage…"; du -xm / 2>/dev/null | sort -rn 
| head -n 30 ; df -h ;
 echo "Cleaning tmp…";
@@ -524,6 +524,16 @@ cassandraBranches.each {
  find build/test/logs -type f -name "*.log" | 
xargs xz -qq
   """)
 }
+if (targetName == 'microbench') {
+publishers {
+jmhReport {
+resultPath('build/test/jmh-result.json')
+}
+archiveJunit('build/test/**/TEST-*.xml') {
+allowEmptyResults(true)
+}
+}
+}
 }
 }
 }
@@ -776,7 +786,7 @@ testTargets.each {
 steps {
 buildDescription('', buildDescStr)
 shell("""
-git clean -xdff ;
+git clean -xdff ${targetName == 'microbench' ? '-e 
build/test/jmh-result.json' : ''};
 git clone --depth 1 --single-branch -b ${buildsBranch} 
${buildsRepo} ;
 echo "cassandra-builds at: `git -C cassandra-builds log -1 
--pretty=format:'%h %an %ad %s'`" ;

[jira] [Updated] (CASSANDRA-16371) JMH Reports in ci-cassandra.apache.org

[Michael Semb Wever (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-16371:
---
Source Control Link: 
https://github.com/apache/cassandra/commit/d19f35243733d211a81de149ef3b995fc7310854

> JMH Reports in ci-cassandra.apache.org
> --
>
> Key: CASSANDRA-16371
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16371
> Project: Cassandra
>  Issue Type: Task
>  Components: CI, Test/benchmark
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Low
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0.x
>
> Attachments: JMH #1 [Jenkins .png, JMH #7 [Jenkins .png
>
>
> Enable the  jenkins [jmh reporting 
> plugin|https://plugins.jenkins.io/jmh-report/] and add a jenkins job to run 
> the `ant microbench` target.
> Patches
>  - cassandra-builds: 
> https://github.com/apache/cassandra-builds/compare/trunk...thelastpickle:mck/jenkins-microbench
>  - cassandra: 
> https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_jmh_report
> Example results are:
>  - CI JMH report: 
> https://ci-cassandra.apache.org/job/Cassandra-devbranch-microbench/jdk=jdk_11_latest,label=cassandra/1/jmh-run-report/
>   (screenshot attached)
> The value of these reports is limited, with different hardware jenkins agents 
> and two executors on each agent. But visibility from the reports shows what 
> is possible, how agents are different (and behave under saturation), and most 
> importantly when jmh classes are broken (there are no unit tests on these 
> benchmark tests).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch cassandra-3.11 updated (a249e6c -> b0a5a7c)

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a change to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


from a249e6c  Merge branch 'cassandra-3.0' into cassandra-3.11
 new d19f352  Generate JMH json report with `ant microbench`
 new 4333b5c  Merge branch 'cassandra-2.2' into cassandra-3.0
 new b0a5a7c  Merge branch 'cassandra-3.0' into cassandra-3.11

The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 build.xml | 12 
 1 file changed, 12 insertions(+)


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch trunk updated (b04429b -> 637b242)

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a change to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


from b04429b  ninja-fix debian cassandra.yaml patch for CASSANDRA-14793
 new d19f352  Generate JMH json report with `ant microbench`
 new 4333b5c  Merge branch 'cassandra-2.2' into cassandra-3.0
 new b0a5a7c  Merge branch 'cassandra-3.0' into cassandra-3.11
 new 637b242  Merge branch 'cassandra-3.11' into trunk

The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 build.xml | 20 
 doc/source/development/ci.rst | 15 +--
 2 files changed, 25 insertions(+), 10 deletions(-)


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git

commit 4333b5c7d5127e4bf59c3fc76bd44880a2fe25ea
Merge: 08be5e0 d19f352
Author: Mick Semb Wever 
AuthorDate: Mon Feb 8 11:03:51 2021 +0100

Merge branch 'cassandra-2.2' into cassandra-3.0

 build.xml | 8 
 1 file changed, 8 insertions(+)



-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch cassandra-2.2 updated: Generate JMH json report with `ant microbench`

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to branch cassandra-2.2
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/cassandra-2.2 by this push:
 new d19f352  Generate JMH json report with `ant microbench`
d19f352 is described below

commit d19f35243733d211a81de149ef3b995fc7310854
Author: Mick Semb Wever 
AuthorDate: Mon Feb 8 10:55:58 2021 +0100

Generate JMH json report with `ant microbench`

 patch by Mick Semb Wever; reviewed by Tomek Łasica for CASSANDRA-16371
---
 build.xml | 8 
 1 file changed, 8 insertions(+)

diff --git a/build.xml b/build.xml
index 83d9aa4..1ac8d5d 100644
--- a/build.xml
+++ b/build.xml
@@ -1935,6 +1935,14 @@
   
   
   
+  
+  
+  
+  
+  
+  
+  
+  
   
   
   


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch cassandra-3.0 updated (08be5e0 -> 4333b5c)

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a change to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


from 08be5e0  Merge branch 'cassandra-2.2' into cassandra-3.0
 new d19f352  Generate JMH json report with `ant microbench`
 new 4333b5c  Merge branch 'cassandra-2.2' into cassandra-3.0

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 build.xml | 8 
 1 file changed, 8 insertions(+)


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git

commit 637b24284009c0e6c14c2817a6a673f5046dd763
Merge: b04429b b0a5a7c
Author: Mick Semb Wever 
AuthorDate: Mon Feb 8 11:28:19 2021 +0100

Merge branch 'cassandra-3.11' into trunk

 build.xml | 20 
 doc/source/development/ci.rst | 15 +--
 2 files changed, 25 insertions(+), 10 deletions(-)

diff --cc build.xml
index b2d0919,9e0bb73..c397019
--- a/build.xml
+++ b/build.xml
@@@ -157,105 -150,6 +157,105 @@@
  }
  
  
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +-Djdk.attach.allowAttachSelf=true
 +
 +-XX:+UseConcMarkSweepGC
 +-XX:+CMSParallelRemarkEnabled
 +-XX:SurvivorRatio=8
 +-XX:MaxTenuringThreshold=1
 +-XX:CMSInitiatingOccupancyFraction=75
 +-XX:+UseCMSInitiatingOccupancyOnly
 +-XX:CMSWaitDuration=1
 +-XX:+CMSParallelInitialMarkEnabled
 +-XX:+CMSEdenChunksRecordAlways
 +
 +--add-exports java.base/jdk.internal.misc=ALL-UNNAMED
 +--add-exports java.base/jdk.internal.ref=ALL-UNNAMED
 +--add-exports java.base/sun.nio.ch=ALL-UNNAMED
 +--add-exports 
java.management.rmi/com.sun.jmx.remote.internal.rmi=ALL-UNNAMED
 +--add-exports java.rmi/sun.rmi.registry=ALL-UNNAMED
 +--add-exports java.rmi/sun.rmi.server=ALL-UNNAMED
 +--add-exports java.sql/java.sql=ALL-UNNAMED
 +
 +--add-opens java.base/java.lang.module=ALL-UNNAMED
 +--add-opens java.base/java.net=ALL-UNNAMED
 +--add-opens java.base/jdk.internal.loader=ALL-UNNAMED
 +--add-opens java.base/jdk.internal.ref=ALL-UNNAMED
 +--add-opens 
java.base/jdk.internal.reflect=ALL-UNNAMED
 +--add-opens java.base/jdk.internal.math=ALL-UNNAMED
 +--add-opens java.base/jdk.internal.module=ALL-UNNAMED
 +--add-opens 
java.base/jdk.internal.util.jar=ALL-UNNAMED
 +--add-opens 
jdk.management/com.sun.management.internal=ALL-UNNAMED
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +  
 +
 +
 +
 +-XX:-CMSClassUnloadingEnabled
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
  
@@@ -2207,6 -1969,18 +2207,18 @@@



+   
+   
+   
+   
+   
+   
+   
+   
+ 
 -  
 -  
++  
++  
+ 



diff --cc doc/source/development/ci.rst
index d7a1bb6,000..4349259
mode 100644,00..100644
--- a/doc/source/development/ci.rst
+++ b/doc/source/development/ci.rst
@@@ -1,81 -1,0 +1,84 @@@
 +.. Licensed to the Apache Software Foundation (ASF) under one
 +.. or more contributor license agreements.  See the NOTICE file
 +.. distributed with this work for additional information
 +.. regarding copyright ownership.  The ASF licenses this file
 +.. to you under the Apache License, Version 2.0 (the
 +.. "License"); you may not use this file except in compliance
 +.. with the License.  You may obtain a copy of the License at
 +..
 +.. http://www.apache.org/licenses/LICENSE-2.0
 +..
 +.. Unless required by applicable law or agreed to in writing, software
 +.. distributed under the License is distributed on an "AS IS" BASIS,
 +.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 +.. See the License for the specific language governing permissions and
 +.. limitations under the License.
 +
 +Jenkins CI Environment
 +**
 +
 +About CI testing and Apache Cassandra
 +=
 +
 +Cassandra can be automatically tested using various test suites, that are 
either implemented based on JUnit or the `dtest 
`_ scripts written in Python. As 
outlined in :doc:`testing`, each kind of test suite addresses a different way 
how to test Cassandra. But in the end, all of them will be executed together on 
our CI platform at `builds.apache.org `_, running 
`Jenkins `_.
 +
 +
 +
 +Setting up your own Jenkins server
 +==
 +
 +Jenkins is an open source solution that can be installed on a large number of 
platforms. Setting up a custom Jenkins instance for Cassandra may be desirable 
for users who have hardware to spare, or organizations that want to run 
Cassandra tests for custom patches before contribution.
 +
 +Please refer to the Jenkins download and documentation pages for details on 
how to get Jenkins 

[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git

commit b0a5a7cfc73da86368848b9274fd44572fae12e9
Merge: a249e6c 4333b5c
Author: Mick Semb Wever 
AuthorDate: Mon Feb 8 11:17:04 2021 +0100

Merge branch 'cassandra-3.0' into cassandra-3.11

 build.xml | 12 
 1 file changed, 12 insertions(+)

diff --cc build.xml
index 4aff477,939ccd8..9e0bb73
--- a/build.xml
+++ b/build.xml
@@@ -1969,6 -1912,14 +1969,18 @@@



+   
+   
+   
+   
+   
+   
+   
+   
++
++  
++  
++





-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-16428) Fix selections of JDKs in debian docker images on arm64

[Michael Semb Wever (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17280446#comment-17280446
 ] 

Michael Semb Wever edited comment on CASSANDRA-16428 at 2/8/21, 10:23 AM:
--

[~eduard.tudenhoefner], i need an extra pair of eyes on this patch.

I'm removing the docker {{ENV}} for JAVA_HOME, JAVA8_HOME, JAVA11_HOME. The 
.bashrc is already setting them, and as far as I can see the only place we use 
them is 
[here|https://github.com/apache/cassandra-builds/blob/trunk/build-scripts/cassandra-dtest-pytest.sh#L27-L32]
 (and there the bashrc defined variables work).

But these were only recently added as part of CASSANDRA-15835 
[here|https://github.com/apache/cassandra-builds/commit/45dc010227d75bf05e34617729de5389e62f8f99#diff-9e34dd20086a6a0c4b9f9f238fabfffae47615da0416ef2929e61e3c1290005dR84-R86].


was (Author: michaelsembwever):
[~eduard.tudenhoefner], i need an extra pair of eyes on this patch.

I'm removing the docker {{ENV}} for JAVA_HOME, JAVA8_HOME, JAVA11_HOME. The 
.bashrc is already setting them, and as far as I can see the only place we use 
them is 
[here|https://github.com/apache/cassandra-builds/blob/trunk/build-scripts/cassandra-dtest-pytest.sh#L27-L32].

But these were only recently added as part of CASSANDRA-15835 
[here|https://github.com/apache/cassandra-builds/commit/45dc010227d75bf05e34617729de5389e62f8f99#diff-9e34dd20086a6a0c4b9f9f238fabfffae47615da0416ef2929e61e3c1290005dR84-R86].

> Fix selections of JDKs in debian docker images on arm64
> ---
>
> Key: CASSANDRA-16428
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16428
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
>
> The debian docker 
> [image|https://github.com/apache/cassandra-builds/blob/trunk/docker/buster-image.docker]
>  used by the 
> [cassandra-deb-packaging.sh|https://github.com/apache/cassandra-builds/blob/trunk/build-scripts/cassandra-deb-packaging.sh]
>  tries to set the jdk like
> {code}
> update-java-alternatives --set java-1.8.0-openjdk-amd64
> {code}
> This won't work on arm64.
> Suggestion is to replace it with the following:
> {code}
> update-java-alternatives --set $(update-java-alternatives -l | cut -d" " -f1 
> | grep java-1.8)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16415) Broken test mixedModeReadColumnSubsetDigestCheck - org.apache.cassandra.distributed.upgrade.MixedModeReadTest

[Benjamin Lerer (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-16415:
---
Fix Version/s: (was: 3.11.11)
   4.0-beta
   3.11.x

> Broken test mixedModeReadColumnSubsetDigestCheck - 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest
> -
>
> Key: CASSANDRA-16415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16415
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination, Test/dtest/java
>Reporter: Yifan Cai
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The test has been failing and can always be reproduced in the recent CI. 
> Stack trace: 
> {code:java}
> junit.framework.AssertionFailedError: Found Digest Mismatch
>  at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.checkTraceForDigestMismatch(MixedModeReadTest.java:89)
>  at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.lambda$mixedModeReadColumnSubsetDigestCheck$0(MixedModeReadTest.java:63)
>  at 
> org.apache.cassandra.distributed.upgrade.UpgradeTestBase$TestCase.run(UpgradeTestBase.java:171)
>  at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.mixedModeReadColumnSubsetDigestCheck(MixedModeReadTest.java:76)
>  {code}
> The initial investigation shows that 
>  * The test only fails in the setup phase of 
> mixedModeReadColumnSubsetDigestCheck. The cluster version is 
> *Versions.Major.v3X*
>  * The test failure is likely a consequence of CASSANDRA-15962. After 
> dropping the commit in branch cassandra-3.11 and rebuild the dtest jar, the 
> upgrade test can pass. Meanwhile, dropping the other commits does not help. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16426) Cannot read from C* during migration

[Marius Morlock (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17280917#comment-17280917
 ] 

Marius Morlock commented on CASSANDRA-16426:


Hello Alex,

here is the how we created the table that causes the issue:
{code:java}
CREATE TABLE file.log (
tenant text,
year smallint,
month tinyint,
day tinyint,
hour tinyint,
time timestamp,
category text,
format_version text,
message text,
user text,
PRIMARY KEY ((tenant, year, month, day, hour), time)
) WITH CLUSTERING ORDER BY (time DESC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
AND comment = ''
AND compaction = {'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}
AND compression = {'sstable_compression': 
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
{code}
We cannot read anything from the C* cluster during the migration process. The 
cluster has 3 nodes which are all C* version 2.2.9 before we start the upgrade 
and we want to upgrade all nodes to version 3.11.

Do you need any further information? In the datastax ticket that I linked above 
are more details to the tables before and after the migration.
Thank you and best regards!

 

> Cannot read from C* during migration
> 
>
> Key: CASSANDRA-16426
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16426
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marius Morlock
>Priority: Normal
>
> Hello,
> We tried to migrate our C* clusters from 2.2.9 to 3.11.6. During the 
> migration our clients (using C* driver version 3.4.0 ) failed with the 
> following exception and were continuously reporting the exception. 
> Interestingly after the migration, the same clients are running fine and no 
> issues are being reported. We have observed this on multiple migrations. As 
> this issue occurs we cannot read from C* which causes unwanted downtimes.
> {code:java}
> java.lang.RuntimeException: 
> com.datastax.driver.core.exceptions.CodecNotFoundException: Codec not found 
> for requested operation: ['org.apache.cassandra.db.marshal.ShortType' <-> 
> java.lang.Short] at com.test.LogReader.readFromDB(LogReader.java:150) at 
> com.test.readLogData(LogReader.java:113) at 
> com.test.retrieveLogs(LogApiServletV2.java:124) at 
> com.test.doGet(LogApiServletV2.java:82) 
> atjavax.servlet.http.HttpServlet.service(HttpServlet.java:626) at 
> javax.servlet.http.HttpServlet.service(HttpServlet.java:733) at 
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
>  at 
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
>  at 
> org.apache.tomee.webservices.CXFJAXRSFilter.doFilter(CXFJAXRSFilter.java:83) 
> at 
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
>  at 
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
>  at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at 
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
>  at 
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
>  at 
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
>  at 
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
>  at 
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
>  at 
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
>  at 
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
>  at 
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
>  at 
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
>  at 
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
>  at 
> org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:154)
>  at 
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
>  at 
> 

[jira] [Updated] (CASSANDRA-16307) GROUP BY queries with paging can return deleted data

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-16307:

Status: Patch Available  (was: Open)

> GROUP BY queries with paging can return deleted data
> 
>
> Key: CASSANDRA-16307
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16307
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination
>Reporter: Andres de la Peña
>Assignee: Alex Petrov
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> {{GROUP BY}} queries using paging and CL>ONE/LOCAL_ONE. This dtest reproduces 
> the problem:
> {code:java}
> try (Cluster cluster = init(Cluster.create(2)))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (pk int, ck int, 
> PRIMARY KEY (pk, ck))"));
> ICoordinator coordinator = cluster.coordinator(1);
> coordinator.execute(withKeyspace("INSERT INTO %s.t (pk, ck) VALUES (0, 
> 0)"), ConsistencyLevel.ALL);
> coordinator.execute(withKeyspace("INSERT INTO %s.t (pk, ck) VALUES (1, 
> 1)"), ConsistencyLevel.ALL);
> 
> cluster.get(1).executeInternal(withKeyspace("DELETE FROM %s.t WHERE pk=0 
> AND ck=0"));
> cluster.get(2).executeInternal(withKeyspace("DELETE FROM %s.t WHERE pk=1 
> AND ck=1"));
> String query = withKeyspace("SELECT * FROM %s.t GROUP BY pk");
> Iterator rows = coordinator.executeWithPaging(query, 
> ConsistencyLevel.ALL, 1);
> assertRows(Iterators.toArray(rows, Object[].class));
> }
> {code}
> Using a 2-node cluster and RF=2, the test inserts two partitions in both 
> nodes. Then it locally deletes each row in a separate node, so each node sees 
> a different partition alive, but reconciliation should produce no alive 
> partitions. However, a {{GROUP BY}} query using a page size of 1 wrongly 
> returns one of the rows.
> This has been detected during CASSANDRA-16180, and it is probably related to 
> CASSANDRA-15459, which solved a similar problem for group-by queries with 
> limit, instead of paging.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16371) JMH Reports in ci-cassandra.apache.org

[Michael Semb Wever (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-16371:
---
Status: Ready to Commit  (was: Review In Progress)

> JMH Reports in ci-cassandra.apache.org
> --
>
> Key: CASSANDRA-16371
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16371
> Project: Cassandra
>  Issue Type: Task
>  Components: CI, Test/benchmark
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Low
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0.x
>
> Attachments: JMH #1 [Jenkins .png, JMH #7 [Jenkins .png
>
>
> Enable the  jenkins [jmh reporting 
> plugin|https://plugins.jenkins.io/jmh-report/] and add a jenkins job to run 
> the `ant microbench` target.
> Patches
>  - cassandra-builds: 
> https://github.com/apache/cassandra-builds/compare/trunk...thelastpickle:mck/jenkins-microbench
>  - cassandra: 
> https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_jmh_report
> Example results are:
>  - CI JMH report: 
> https://ci-cassandra.apache.org/job/Cassandra-devbranch-microbench/jdk=jdk_11_latest,label=cassandra/1/jmh-run-report/
>   (screenshot attached)
> The value of these reports is limited, with different hardware jenkins agents 
> and two executors on each agent. But visibility from the reports shows what 
> is possible, how agents are different (and behave under saturation), and most 
> importantly when jmh classes are broken (there are no unit tests on these 
> benchmark tests).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16429) cqlsh garbles column names with Japanese characters

[Benjamin Lerer (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17280895#comment-17280895
 ] 

Benjamin Lerer commented on CASSANDRA-16429:


[~jeromatron]  CASSANDRA-14825 changed only the way the output of the 
`DESCRIBE` statement worked. According to your comment this part is 
functionning so I believe that the problem has been caused by some other change.

> cqlsh garbles column names with Japanese characters
> ---
>
> Key: CASSANDRA-16429
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16429
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL/Interpreter
>Reporter: Yoshi Kimoto
>Priority: Normal
> Fix For: 4.0-rc
>
> Attachments: jptest.cql
>
>
> Tables created with Japanese character name columns are working well in C* 
> 3.11.10 when doing a SELECT * in cqlsh but will show as garbled (shown as 
> "?") in 4.0-beta4. DESCRIBE shows the column names correctly in both cases.
> Run the attached jptest.cql script in both envs with cqlsh -f. They will 
> yield different results.
> My test env (MacOS 10.15.7):
> C* 3.11.10 with
>  - OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_252-b09)
>  - Python 2.7.16
> C* 4.0-beta4
>  - OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.9.1+1)
>  - Python 3.8.2



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16429) cqlsh garbles column names with Japanese characters

[Benjamin Lerer (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-16429:
---
Fix Version/s: (was: 4.0-rc)
   4.0-beta

> cqlsh garbles column names with Japanese characters
> ---
>
> Key: CASSANDRA-16429
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16429
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL/Interpreter
>Reporter: Yoshi Kimoto
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: jptest.cql
>
>
> Tables created with Japanese character name columns are working well in C* 
> 3.11.10 when doing a SELECT * in cqlsh but will show as garbled (shown as 
> "?") in 4.0-beta4. DESCRIBE shows the column names correctly in both cases.
> Run the attached jptest.cql script in both envs with cqlsh -f. They will 
> yield different results.
> My test env (MacOS 10.15.7):
> C* 3.11.10 with
>  - OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_252-b09)
>  - Python 2.7.16
> C* 4.0-beta4
>  - OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.9.1+1)
>  - Python 3.8.2



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16415) Broken test mixedModeReadColumnSubsetDigestCheck - org.apache.cassandra.distributed.upgrade.MixedModeReadTest

[Jacek Lewandowski (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17280890#comment-17280890
 ] 

Jacek Lewandowski commented on CASSANDRA-16415:
---

This problem does not affect 4.0 at all. The only reason we can see it in 4.0 
tests is that the JVM upgrade distributed tests is different in for {{trunk}} 
and for {{cassandra-3.11}} branches. I don't remember why is it different, but 
we should make it look same. 

Also this is interesting problem - tests tied to 4.0 fail during upgrade 
because C* we upgrade from is broken...

> Broken test mixedModeReadColumnSubsetDigestCheck - 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest
> -
>
> Key: CASSANDRA-16415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16415
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination, Test/dtest/java
>Reporter: Yifan Cai
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 3.11.11
>
>
> The test has been failing and can always be reproduced in the recent CI. 
> Stack trace: 
> {code:java}
> junit.framework.AssertionFailedError: Found Digest Mismatch
>  at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.checkTraceForDigestMismatch(MixedModeReadTest.java:89)
>  at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.lambda$mixedModeReadColumnSubsetDigestCheck$0(MixedModeReadTest.java:63)
>  at 
> org.apache.cassandra.distributed.upgrade.UpgradeTestBase$TestCase.run(UpgradeTestBase.java:171)
>  at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.mixedModeReadColumnSubsetDigestCheck(MixedModeReadTest.java:76)
>  {code}
> The initial investigation shows that 
>  * The test only fails in the setup phase of 
> mixedModeReadColumnSubsetDigestCheck. The cluster version is 
> *Versions.Major.v3X*
>  * The test failure is likely a consequence of CASSANDRA-15962. After 
> dropping the commit in branch cassandra-3.11 and rebuild the dtest jar, the 
> upgrade test can pass. Meanwhile, dropping the other commits does not help. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16415) Broken test mixedModeReadColumnSubsetDigestCheck - org.apache.cassandra.distributed.upgrade.MixedModeReadTest

[Jacek Lewandowski (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacek Lewandowski updated CASSANDRA-16415:
--
Fix Version/s: (was: 4.0-beta)

> Broken test mixedModeReadColumnSubsetDigestCheck - 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest
> -
>
> Key: CASSANDRA-16415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16415
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination, Test/dtest/java
>Reporter: Yifan Cai
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 3.11.11
>
>
> The test has been failing and can always be reproduced in the recent CI. 
> Stack trace: 
> {code:java}
> junit.framework.AssertionFailedError: Found Digest Mismatch
>  at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.checkTraceForDigestMismatch(MixedModeReadTest.java:89)
>  at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.lambda$mixedModeReadColumnSubsetDigestCheck$0(MixedModeReadTest.java:63)
>  at 
> org.apache.cassandra.distributed.upgrade.UpgradeTestBase$TestCase.run(UpgradeTestBase.java:171)
>  at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.mixedModeReadColumnSubsetDigestCheck(MixedModeReadTest.java:76)
>  {code}
> The initial investigation shows that 
>  * The test only fails in the setup phase of 
> mixedModeReadColumnSubsetDigestCheck. The cluster version is 
> *Versions.Major.v3X*
>  * The test failure is likely a consequence of CASSANDRA-15962. After 
> dropping the commit in branch cassandra-3.11 and rebuild the dtest jar, the 
> upgrade test can pass. Meanwhile, dropping the other commits does not help. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16415) Broken test mixedModeReadColumnSubsetDigestCheck - org.apache.cassandra.distributed.upgrade.MixedModeReadTest

[Jacek Lewandowski (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacek Lewandowski updated CASSANDRA-16415:
--
Fix Version/s: (was: 4.0-beta)
   3.11.11

> Broken test mixedModeReadColumnSubsetDigestCheck - 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest
> -
>
> Key: CASSANDRA-16415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16415
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination, Test/dtest/java
>Reporter: Yifan Cai
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 3.11.11
>
>
> The test has been failing and can always be reproduced in the recent CI. 
> Stack trace: 
> {code:java}
> junit.framework.AssertionFailedError: Found Digest Mismatch
>  at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.checkTraceForDigestMismatch(MixedModeReadTest.java:89)
>  at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.lambda$mixedModeReadColumnSubsetDigestCheck$0(MixedModeReadTest.java:63)
>  at 
> org.apache.cassandra.distributed.upgrade.UpgradeTestBase$TestCase.run(UpgradeTestBase.java:171)
>  at 
> org.apache.cassandra.distributed.upgrade.MixedModeReadTest.mixedModeReadColumnSubsetDigestCheck(MixedModeReadTest.java:76)
>  {code}
> The initial investigation shows that 
>  * The test only fails in the setup phase of 
> mixedModeReadColumnSubsetDigestCheck. The cluster version is 
> *Versions.Major.v3X*
>  * The test failure is likely a consequence of CASSANDRA-15962. After 
> dropping the commit in branch cassandra-3.11 and rebuild the dtest jar, the 
> upgrade test can pass. Meanwhile, dropping the other commits does not help. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16307) GROUP BY queries with paging can return deleted data

[Alex Petrov (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17280846#comment-17280846
 ] 

Alex Petrov commented on CASSANDRA-16307:
-

The problem here turned out to be that the Group By pager incorrectly interacts 
with SRP. 

In short, we have multiple implementations of paging counters: one is regular 
CQL one and, the other is Group By. CQL one is counting rows as it sees them. 
Group By is counting rows its going to return (in other words, groups) only 
when it encounters the next group (or partition/data end).

The problem is that when issuing SRP, we rely on correctness of these counters. 
And if counter says that iterator couldn’t give enough data to exhaust limits, 
we’re not going to ask for more data. In case of regular counters it all works 
as expected, but with group by counters, we’ll say that we have seen 1 row less 
every time, and won’t issue SRP. In other words, if one of the nodes holds a 
row followed by the tombstone, current behaviour will prevent SRP from getting 
triggered and tombstone from being visible, since group will be unfinished, and 
it doesn’t make sense to ask the node for more data since it couldn’t even 
satisfy the current data limit.

A straightforward solution to this would have been to just start counting 
groups as we’re encountering them. However, here the problem is that if we say 
that we have a group as soon as we see the first row that belongs to it, 
internal pager will think that there’s no need to continue, since data limits 
are satisfied.

The patch is introducing a distinction between needsMoreContents (better name 
pending) and isDone. This allows us to know we need to fetch more rows in case 
we haven’t seen either end of data, or next row. I think we can even start 
counting groups right away instead of delaying counting till we see the next 
group to (potentially) simplify the logic, however I wanted to leave a larger 
refactoring to a separate patch.

|[trunk|https://github.com/apache/cassandra/compare/trunk...ifesdjeen:16307-trunk]|[CI|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=16307-trunk]|
|[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...ifesdjeen:16307-3.11]|[CI|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=16307-3.11]|

> GROUP BY queries with paging can return deleted data
> 
>
> Key: CASSANDRA-16307
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16307
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination
>Reporter: Andres de la Peña
>Assignee: Alex Petrov
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> {{GROUP BY}} queries using paging and CL>ONE/LOCAL_ONE. This dtest reproduces 
> the problem:
> {code:java}
> try (Cluster cluster = init(Cluster.create(2)))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (pk int, ck int, 
> PRIMARY KEY (pk, ck))"));
> ICoordinator coordinator = cluster.coordinator(1);
> coordinator.execute(withKeyspace("INSERT INTO %s.t (pk, ck) VALUES (0, 
> 0)"), ConsistencyLevel.ALL);
> coordinator.execute(withKeyspace("INSERT INTO %s.t (pk, ck) VALUES (1, 
> 1)"), ConsistencyLevel.ALL);
> 
> cluster.get(1).executeInternal(withKeyspace("DELETE FROM %s.t WHERE pk=0 
> AND ck=0"));
> cluster.get(2).executeInternal(withKeyspace("DELETE FROM %s.t WHERE pk=1 
> AND ck=1"));
> String query = withKeyspace("SELECT * FROM %s.t GROUP BY pk");
> Iterator rows = coordinator.executeWithPaging(query, 
> ConsistencyLevel.ALL, 1);
> assertRows(Iterators.toArray(rows, Object[].class));
> }
> {code}
> Using a 2-node cluster and RF=2, the test inserts two partitions in both 
> nodes. Then it locally deletes each row in a separate node, so each node sees 
> a different partition alive, but reconciliation should produce no alive 
> partitions. However, a {{GROUP BY}} query using a page size of 1 wrongly 
> returns one of the rows.
> This has been detected during CASSANDRA-16180, and it is probably related to 
> CASSANDRA-15459, which solved a similar problem for group-by queries with 
> limit, instead of paging.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-16307) GROUP BY queries with paging can return deleted data

[Alex Petrov (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17280846#comment-17280846
 ] 

Alex Petrov edited comment on CASSANDRA-16307 at 2/8/21, 8:34 AM:
--

The problem here turned out to be that the Group By pager incorrectly interacts 
with SRP. 

In short, we have multiple implementations of paging counters: one is regular 
CQL one and, the other is Group By. CQL one is counting rows as it sees them. 
Group By is counting rows its going to return (in other words, groups) only 
when it encounters the next group (or partition/data end).

The problem is that when issuing SRP, we rely on correctness of these counters. 
And if counter says that iterator couldn’t give enough data to exhaust limits, 
we’re not going to ask for more data. In case of regular counters it all works 
as expected, but with group by counters, we’ll say that we have seen 1 row less 
every time, and won’t issue SRP. In other words, if one of the nodes holds a 
row followed by the tombstone, current behaviour will prevent SRP from getting 
triggered and tombstone from being visible, since group will be unfinished, and 
it doesn’t make sense to ask the node for more data since it couldn’t even 
satisfy the current data limit.

A straightforward solution to this would have been to just start counting 
groups as we’re encountering them. However, here the problem is that if we say 
that we have a group as soon as we see the first row that belongs to it, 
internal pager will think that there’s no need to continue, since data limits 
are satisfied.

The patch is introducing a distinction between {{needsMoreContents}} (better 
name pending) and {{isDone}}. This allows us to know we need to fetch more rows 
in case we haven’t seen either end of data, or next row. I think we can even 
start counting groups right away instead of delaying counting till we see the 
next group to (potentially) simplify the logic, however I wanted to leave a 
larger refactoring to a separate patch.

|[trunk|https://github.com/apache/cassandra/compare/trunk...ifesdjeen:16307-trunk]|[CI|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=16307-trunk]|
|[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...ifesdjeen:16307-3.11]|[CI|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=16307-3.11]|


was (Author: ifesdjeen):
The problem here turned out to be that the Group By pager incorrectly interacts 
with SRP. 

In short, we have multiple implementations of paging counters: one is regular 
CQL one and, the other is Group By. CQL one is counting rows as it sees them. 
Group By is counting rows its going to return (in other words, groups) only 
when it encounters the next group (or partition/data end).

The problem is that when issuing SRP, we rely on correctness of these counters. 
And if counter says that iterator couldn’t give enough data to exhaust limits, 
we’re not going to ask for more data. In case of regular counters it all works 
as expected, but with group by counters, we’ll say that we have seen 1 row less 
every time, and won’t issue SRP. In other words, if one of the nodes holds a 
row followed by the tombstone, current behaviour will prevent SRP from getting 
triggered and tombstone from being visible, since group will be unfinished, and 
it doesn’t make sense to ask the node for more data since it couldn’t even 
satisfy the current data limit.

A straightforward solution to this would have been to just start counting 
groups as we’re encountering them. However, here the problem is that if we say 
that we have a group as soon as we see the first row that belongs to it, 
internal pager will think that there’s no need to continue, since data limits 
are satisfied.

The patch is introducing a distinction between needsMoreContents (better name 
pending) and isDone. This allows us to know we need to fetch more rows in case 
we haven’t seen either end of data, or next row. I think we can even start 
counting groups right away instead of delaying counting till we see the next 
group to (potentially) simplify the logic, however I wanted to leave a larger 
refactoring to a separate patch.

|[trunk|https://github.com/apache/cassandra/compare/trunk...ifesdjeen:16307-trunk]|[CI|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=16307-trunk]|
|[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...ifesdjeen:16307-3.11]|[CI|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=16307-3.11]|

> GROUP BY queries with paging can return deleted data
> 
>
> Key: CASSANDRA-16307
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16307
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination
>

[jira] [Comment Edited] (CASSANDRA-16427) In-JVM dtest paging does not handle Group By correctly

[Alex Petrov (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279807#comment-17279807
 ] 

Alex Petrov edited comment on CASSANDRA-16427 at 2/8/21, 8:30 AM:
--

|[trunk|https://github.com/apache/cassandra/compare/trunk...ifesdjeen:16427-trunk]|[CI|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=16427-trunk]|
|[3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...ifesdjeen:16427-3.0]|[CI|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=16427-3.0]|
|[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...ifesdjeen:16427-3.11]|[CI|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=16427-3.11]|
|[2.2|https://github.com/apache/cassandra/compare/cassandra-2.2...ifesdjeen:16427-2.2]|[CI|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=16427-2.2]|

Disclaimer: some of the tests are still queued after several hours. However, CI 
seems to be ok with achange.


was (Author: ifesdjeen):
|[trunk|https://github.com/apache/cassandra/compare/trunk...ifesdjeen:16427-trunk]|[CI|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=
 16427-trunk]|
|[3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...ifesdjeen:16427-3.0]|[CI|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=16427-3.0]|
|[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...ifesdjeen:16427-3.11]|[CI|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=16427-3.11]|
|[2.2|https://github.com/apache/cassandra/compare/cassandra-2.2...ifesdjeen:16427-2.2]|[CI|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=16427-2.2]|

Disclaimer: some of the tests are still queued after several hours. However, CI 
seems to be ok with achange.

> In-JVM dtest paging does not handle Group By correctly
> --
>
> Key: CASSANDRA-16427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16427
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Normal
>
> In-JVM dtest paging is using a pager that disregards the type of the executed 
> query, resulting into `GROUP BY` queries being executed like normal SELECT 
> queries without GROUP BY clause.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16377) Add dtest-upgrade and dtest-large-novnode to Jenkins pipeline

[Jacek Lewandowski (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17280827#comment-17280827
 ] 

Jacek Lewandowski commented on CASSANDRA-16377:
---

+1

> Add dtest-upgrade and dtest-large-novnode to Jenkins pipeline
> -
>
> Key: CASSANDRA-16377
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16377
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Low
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> Now that dtest-upgrade test runs are passing they can be considered for 
> addition to the jenkins pipeline. And dtest-large-novnode only as one failure 
> so should be added too.  
> https://the-asf.slack.com/archives/CK23JSY2K/p1610132643162700 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org