[jira] [Comment Edited] (CASSANDRA-16228) Fix race condition on ColumnFamilyStore and TableMetrics initialization that makes MetricListener to see an instance in an unknown state
[
https://issues.apache.org/jira/browse/CASSANDRA-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221914#comment-17221914
]
Ekaterina Dimitrova edited comment on CASSANDRA-16228 at 10/28/20, 2:35 AM:
The CI results are out. The only failure that grabbed my attention was the one
of [test_table_metric_mbeans - jmx_test.TestJMX |
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/8df700c5-1775-433f-8dfc-3b3391f3d0ae/jobs/2606]
.
But it looks like it was random, unrelated, CircleCI resource related(I also
looped the test locally several times and it completed successfully; it also
didn't fail in Jenkins):
{code:java}
stdout = b"Couldn't start agent for PID 6894\nPossible reason could be that
port '8778' is already occupied.\nPlease check the standard output of the
target process for a detailed error message.\n"
{code}
was (Author: e.dimitrova):
The CI results are out. The only failure that grabbed my attention was the one
of [test_table_metric_mbeans - jmx_test.TestJMX |
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/8df700c5-1775-433f-8dfc-3b3391f3d0ae/jobs/2606]
.
But it looks like it was random, unrelated, CircleCI resource related(I also
looped the test locally several times and it completed successfully):
{code:java}
stdout = b"Couldn't start agent for PID 6894\nPossible reason could be that
port '8778' is already occupied.\nPlease check the standard output of the
target process for a detailed error message.\n"
{code}
> Fix race condition on ColumnFamilyStore and TableMetrics initialization that
> makes MetricListener to see an instance in an unknown state
>
>
> Key: CASSANDRA-16228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16228
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> There is a dependency between TableMetrics, SecondaryIndexManager and Tracker
> which requires certain order of the operations in ColumnFamilyStore
> constructor.
> The following test will be added to ColumnFamilyMetricTest to show the issue:
> {code:java}
> @Test
> public void testStartupRaceConditionOnMetricListeners()
> {
> // Since the ColumnFamilyStore instance reference escapes during the
> construction
> // we have a race condition and listeners can see an instance that is in an
> unknown state.
> // This test just check that all callbacks can access the data without
> throwing any exception.
> registerMetricListener();
> SchemaLoader.createKeyspace("Keyspace2",
> KeyspaceParams.simple(1),
> SchemaLoader.standardCFMD("Keyspace2", "Standard2"));
> }
> private void registerMetricListener()
> {
> CassandraMetricsRegistry.Metrics.addListener(new
> MetricRegistryListener.Base()
> {
> @Override
> public void onGaugeAdded(String name, Gauge gauge)
> {
> gauge.getValue();
> }
> @Override
> public void onGaugeRemoved(String name)
> {
> }
> @Override
> public void onCounterAdded(String name, Counter counter)
> {
> counter.getCount();
> }
> @Override
> public void onCounterRemoved(String name)
> {
> }
> @Override
> public void onHistogramAdded(String name, Histogram histogram)
> {
> histogram.getCount();
> }
> @Override
> public void onHistogramRemoved(String name)
> {
> }
> @Override
> public void onMeterAdded(String name, Meter meter)
> {
> meter.getCount();
> }
> @Override
> public void onMeterRemoved(String name)
> {
> }
> @Override
> public void onTimerAdded(String name, Timer timer)
> {
> timer.getCount();
> }
> @Override
> public void onTimerRemoved(String name)
> {
> }
> });{code}
> *NOTE:* In order to be able to run the test and use registerMetricListener()
> method a small modification to requests in CacheMetrics is needed. This will
> be submitted and further explained with a coming patch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16228) Fix race condition on ColumnFamilyStore and TableMetrics initialization that makes MetricListener to see an instance in an unknown state
[
https://issues.apache.org/jira/browse/CASSANDRA-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221916#comment-17221916
]
Ekaterina Dimitrova commented on CASSANDRA-16228:
-
[~blerer], do you think you will have the time to make a review?
> Fix race condition on ColumnFamilyStore and TableMetrics initialization that
> makes MetricListener to see an instance in an unknown state
>
>
> Key: CASSANDRA-16228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16228
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> There is a dependency between TableMetrics, SecondaryIndexManager and Tracker
> which requires certain order of the operations in ColumnFamilyStore
> constructor.
> The following test will be added to ColumnFamilyMetricTest to show the issue:
> {code:java}
> @Test
> public void testStartupRaceConditionOnMetricListeners()
> {
> // Since the ColumnFamilyStore instance reference escapes during the
> construction
> // we have a race condition and listeners can see an instance that is in an
> unknown state.
> // This test just check that all callbacks can access the data without
> throwing any exception.
> registerMetricListener();
> SchemaLoader.createKeyspace("Keyspace2",
> KeyspaceParams.simple(1),
> SchemaLoader.standardCFMD("Keyspace2", "Standard2"));
> }
> private void registerMetricListener()
> {
> CassandraMetricsRegistry.Metrics.addListener(new
> MetricRegistryListener.Base()
> {
> @Override
> public void onGaugeAdded(String name, Gauge gauge)
> {
> gauge.getValue();
> }
> @Override
> public void onGaugeRemoved(String name)
> {
> }
> @Override
> public void onCounterAdded(String name, Counter counter)
> {
> counter.getCount();
> }
> @Override
> public void onCounterRemoved(String name)
> {
> }
> @Override
> public void onHistogramAdded(String name, Histogram histogram)
> {
> histogram.getCount();
> }
> @Override
> public void onHistogramRemoved(String name)
> {
> }
> @Override
> public void onMeterAdded(String name, Meter meter)
> {
> meter.getCount();
> }
> @Override
> public void onMeterRemoved(String name)
> {
> }
> @Override
> public void onTimerAdded(String name, Timer timer)
> {
> timer.getCount();
> }
> @Override
> public void onTimerRemoved(String name)
> {
> }
> });{code}
> *NOTE:* In order to be able to run the test and use registerMetricListener()
> method a small modification to requests in CacheMetrics is needed. This will
> be submitted and further explained with a coming patch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16228) Fix race condition on ColumnFamilyStore and TableMetrics initialization that makes MetricListener to see an instance in an unknown state
[
https://issues.apache.org/jira/browse/CASSANDRA-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221914#comment-17221914
]
Ekaterina Dimitrova commented on CASSANDRA-16228:
-
The CI results are out. The only failure that grabbed my attention was the one
of [test_table_metric_mbeans - jmx_test.TestJMX |
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/8df700c5-1775-433f-8dfc-3b3391f3d0ae/jobs/2606]
.
But it looks like it was random, unrelated, CircleCI resource related(I also
looped the test locally several times and it completed successfully):
{code:java}
stdout = b"Couldn't start agent for PID 6894\nPossible reason could be that
port '8778' is already occupied.\nPlease check the standard output of the
target process for a detailed error message.\n"
{code}
> Fix race condition on ColumnFamilyStore and TableMetrics initialization that
> makes MetricListener to see an instance in an unknown state
>
>
> Key: CASSANDRA-16228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16228
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> There is a dependency between TableMetrics, SecondaryIndexManager and Tracker
> which requires certain order of the operations in ColumnFamilyStore
> constructor.
> The following test will be added to ColumnFamilyMetricTest to show the issue:
> {code:java}
> @Test
> public void testStartupRaceConditionOnMetricListeners()
> {
> // Since the ColumnFamilyStore instance reference escapes during the
> construction
> // we have a race condition and listeners can see an instance that is in an
> unknown state.
> // This test just check that all callbacks can access the data without
> throwing any exception.
> registerMetricListener();
> SchemaLoader.createKeyspace("Keyspace2",
> KeyspaceParams.simple(1),
> SchemaLoader.standardCFMD("Keyspace2", "Standard2"));
> }
> private void registerMetricListener()
> {
> CassandraMetricsRegistry.Metrics.addListener(new
> MetricRegistryListener.Base()
> {
> @Override
> public void onGaugeAdded(String name, Gauge gauge)
> {
> gauge.getValue();
> }
> @Override
> public void onGaugeRemoved(String name)
> {
> }
> @Override
> public void onCounterAdded(String name, Counter counter)
> {
> counter.getCount();
> }
> @Override
> public void onCounterRemoved(String name)
> {
> }
> @Override
> public void onHistogramAdded(String name, Histogram histogram)
> {
> histogram.getCount();
> }
> @Override
> public void onHistogramRemoved(String name)
> {
> }
> @Override
> public void onMeterAdded(String name, Meter meter)
> {
> meter.getCount();
> }
> @Override
> public void onMeterRemoved(String name)
> {
> }
> @Override
> public void onTimerAdded(String name, Timer timer)
> {
> timer.getCount();
> }
> @Override
> public void onTimerRemoved(String name)
> {
> }
> });{code}
> *NOTE:* In order to be able to run the test and use registerMetricListener()
> method a small modification to requests in CacheMetrics is needed. This will
> be submitted and further explained with a coming patch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16228) Fix race condition on ColumnFamilyStore and TableMetrics initialization that makes MetricListener to see an instance in an unknown state
[ https://issues.apache.org/jira/browse/CASSANDRA-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221755#comment-17221755 ] Ekaterina Dimitrova edited comment on CASSANDRA-16228 at 10/28/20, 2:26 AM: The patch adds a [test|https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R122] to cover the mentioned race condition. In 4.0, as part of CASSANDRA-14628, requests implementation in CacheMetrics was changed but this change (the 2nd suggestion [here|https://issues.apache.org/jira/browse/CASSANDRA-14626?focusedCommentId=16587227=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16587227] ) prevents us from using the codahale [addListener() |https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R136]. That is why I reverted the requests implementation to the original [version |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] from CASSANDRA-14628, changing its type from Metered to Meter and using the mark method. The race condition was fixed itself in 3.0, 3.11, 4.0. [4.0 pull request |https://github.com/ekaterinadimitrova2/cassandra/pull/67] | [JAVA 8 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/8df700c5-1775-433f-8dfc-3b3391f3d0ae] | [JAVA 11 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/4f8946b5-6448-4dd6-b231-8f288c861f34] [Jenkins CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/144/] [3.0 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/145/] [3.11 CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/146/] [3.0 patch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] to be merged also to 3.11 and 4.0 *after* applying first this [patch |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] to 4.0 was (Author: e.dimitrova): The patch adds a [test|https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R122] to cover the mentioned race condition. In 4.0, as part of CASSANDRA-14628, requests implementation in CacheMetrics was changed but this change (the 2nd suggestion [here|https://issues.apache.org/jira/browse/CASSANDRA-14626?focusedCommentId=16587227=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16587227] ) prevents us from using the codahale [addListener() |https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R136]. That is why I reverted the requests implementation to the original [version |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] from CASSANDRA-14628, changing its type from Metered to Meter and using the mark method. The race condition was fixed itself in 3.0, 3.11, 4.0. [4.0 pull request |https://github.com/ekaterinadimitrova2/cassandra/pull/67] | [JAVA 8 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/8df700c5-1775-433f-8dfc-3b3391f3d0ae] | [JAVA 11 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/4f8946b5-6448-4dd6-b231-8f288c861f34] [Jenkins CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/144/] [3.0 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/145/] [3.11 branch |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/146/] [3.0 patch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] to be merged also to 3.11 and 4.0 *after* applying first this [patch |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] to 4.0 > Fix race condition on ColumnFamilyStore and TableMetrics initialization that > makes MetricListener to see an instance in an unknown state > > > Key: CASSANDRA-16228 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16228 > Project: Cassandra > Issue Type: Bug > Components: Observability/Metrics >
[jira] [Comment Edited] (CASSANDRA-16228) Fix race condition on ColumnFamilyStore and TableMetrics initialization that makes MetricListener to see an instance in an unknown state
[ https://issues.apache.org/jira/browse/CASSANDRA-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221755#comment-17221755 ] Ekaterina Dimitrova edited comment on CASSANDRA-16228 at 10/28/20, 2:25 AM: The patch adds a [test|https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R122] to cover the mentioned race condition. In 4.0, as part of CASSANDRA-14628, requests implementation in CacheMetrics was changed but this change (the 2nd suggestion [here|https://issues.apache.org/jira/browse/CASSANDRA-14626?focusedCommentId=16587227=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16587227] ) prevents us from using the codahale [addListener() |https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R136]. That is why I reverted the requests implementation to the original [version |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] from CASSANDRA-14628, changing its type from Metered to Meter and using the mark method. The race condition was fixed itself in 3.0, 3.11, 4.0. [4.0 pull request |https://github.com/ekaterinadimitrova2/cassandra/pull/67] | [JAVA 8 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/8df700c5-1775-433f-8dfc-3b3391f3d0ae] | [JAVA 11 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/4f8946b5-6448-4dd6-b231-8f288c861f34] [Jenkins CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/144/] [3.0 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/145/] [3.11 branch |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/146/] [3.0 patch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] to be merged also to 3.11 and 4.0 *after* applying first this [patch |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] to 4.0 was (Author: e.dimitrova): The patch adds a [test|https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R122] to cover the mentioned race condition. In 4.0, as part of CASSANDRA-14628, requests implementation in CacheMetrics was changed but this change (the 2nd suggestion [here|https://issues.apache.org/jira/browse/CASSANDRA-14626?focusedCommentId=16587227=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16587227] ) prevents us from using the codahale [addListener() |https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R136]. That is why I reverted the requests implementation to the original [version |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] from CASSANDRA-14628, changing its type from Metered to Meter and using the mark method. The race condition was fixed itself in 3.0, 3.11, 4.0. [4.0 pull request |https://github.com/ekaterinadimitrova2/cassandra/pull/67] | [JAVA 8 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/8df700c5-1775-433f-8dfc-3b3391f3d0ae] | [JAVA 11 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/4f8946b5-6448-4dd6-b231-8f288c861f34] [Jenkins CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/144/] [3.0 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/145/] [3.11 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/dfcb3e82e952f8b1dbeb1fe91a3ed3fcb07abc5e] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/146/changes] [3.0 patch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] to be merged also to 3.11 and 4.0 *after* applying first this [patch |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] to 4.0 > Fix race condition on ColumnFamilyStore and TableMetrics initialization that > makes MetricListener to see an instance in an unknown state > > > Key: CASSANDRA-16228 > URL:
[jira] [Commented] (CASSANDRA-16103) Invalid serialized size for responses
[
https://issues.apache.org/jira/browse/CASSANDRA-16103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221877#comment-17221877
]
Yifan Cai commented on CASSANDRA-16103:
---
Updated CI result after rebase.
[https://app.circleci.com/pipelines/github/yifan-c/cassandra/140/workflows/573178eb-002f-4c57-93b8-0d7029689098]
(1 failure from dtest with vnodes. It does not look related)
> Invalid serialized size for responses
> -
>
> Key: CASSANDRA-16103
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16103
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: David Capwell
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Was running a benchmark at LOCAL_ONE and eventually saw the below exception;
> this is related to CASSANDRA-16097 as it was found during the same test.
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized
> size; expected 14, actual size at least 13, for verb HINT_RSP
> at
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
> at
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
> at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized
> size; expected 14, actual size at least 13, for verb MUTATION_RSP
> at
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
> at
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
> at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16220) python dtest pending_range_test.py::TestPendingRangeMovements::test_pending_range (@pytest.mark.resource_intensive) fails on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-16220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221836#comment-17221836
]
Ekaterina Dimitrova commented on CASSANDRA-16220:
-
[~dcapwell], [~brandon.williams], is anyone of you available also to
review/commit it?
> python dtest
> pending_range_test.py::TestPendingRangeMovements::test_pending_range
> (@pytest.mark.resource_intensive) fails on trunk
> --
>
> Key: CASSANDRA-16220
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16220
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: David Capwell
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta
>
>
> The test has the following section
> {code}
> if cluster.version() >= '2.2':
> node2.watch_log_for('127.0.0.1 state moving', timeout=10,
> filename='debug.log’)
> {code}
> The issue is that in trunk we have the port attached to the log, so the log
> is now
> {code}
> DEBUG [GossipStage:1] 2020-10-21 00:48:20,104 StorageService.java:2452 - Node
> /127.0.0.1:7000 state MOVING, tokens [-9223372036854775808]
> DEBUG [GossipStage:1] 2020-10-21 00:48:20,105 StorageService.java:2670 - Node
> /127.0.0.1:7000 state moving, new token -634023222112864484
> {code}
> Since the log now contains the port, this test always times out on trunk when
> it hits this
> {code}
> self =
> @pytest.mark.resource_intensive
> def test_pending_range(self):
> """
> @jira_ticket CASSANDRA-10887
> """
> cluster = self.cluster
> # If we are on 2.1, we need to set the log level to debug or higher,
> as debug.log does not exist.
> if cluster.version() < '2.2':
> cluster.set_log_level('DEBUG')
>
> # Create 5 node cluster
> cluster.populate(5).start()
> node1, node2 = cluster.nodelist()[0:2]
>
> # Set up RF=3 keyspace
> session = self.patient_cql_connection(node1)
> create_ks(session, 'ks', 3)
>
> session.execute("CREATE TABLE users (login text PRIMARY KEY, email
> text, name text, login_count int)")
>
> # We use the partition key 'jdoe3' because it belongs to node1.
> # The key MUST belong to node1 to repro the bug.
> session.execute("INSERT INTO users (login, email, name, login_count)
> VALUES ('jdoe3', 'j...@abc.com', 'Jane Doe', 1) IF NOT EXISTS;")
>
> lwt_query = SimpleStatement("UPDATE users SET email =
> 'jane...@abc.com' WHERE login = 'jdoe3' IF email = 'j...@abc.com'")
>
> # Show we can execute LWT no problem
> for i in range(1000):
> session.execute(lwt_query)
>
> token = '-634023222112864484'
>
> mark = node1.mark_log()
>
> # Move a node
> node1.nodetool('move {}'.format(token))
>
> # Watch the log so we know when the node is moving
> node1.watch_log_for('Moving .* to {}'.format(token), timeout=10,
> from_mark=mark)
> node1.watch_log_for('Sleeping 3 ms before start
> streaming/fetching ranges', timeout=10, from_mark=mark)
>
> if cluster.version() >= '2.2':
> > node2.watch_log_for('127.0.0.1 state moving', timeout=10,
> > filename='debug.log')
> pending_range_test.py:57:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> self =
> exprs = '127.0.0.1 state moving', from_mark = None, timeout = 10, process =
> None
> verbose = False, filename = 'debug.log'
> def watch_log_for(self, exprs, from_mark=None, timeout=600,
> process=None, verbose=False, filename='system.log'):
> """
> Watch the log until one or more (regular) expressions are found
> or timeouts (a
> TimeoutError is then raised). On successful completion, a list of
> pair (line matched,
> match object) is returned.
> """
> start = time.time()
> tofind = [exprs] if isinstance(exprs, string_types) else exprs
> tofind = [re.compile(e) for e in tofind]
> matchings = []
> reads = ""
> if len(tofind) == 0:
> return None
>
> log_file = os.path.join(self.log_directory(), filename)
> output_read = False
> while not os.path.exists(log_file):
> time.sleep(.5)
> if start + timeout < time.time():
> raise TimeoutError(time.strftime("%d %b %Y %H:%M:%S",
> time.gmtime()) + " [" + self.name + "] Timed out waiting for {} to be
> created.".format(log_file))
> if process and not output_read:
> process.poll()
>
[jira] [Updated] (CASSANDRA-16220) python dtest pending_range_test.py::TestPendingRangeMovements::test_pending_range (@pytest.mark.resource_intensive) fails on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-16220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-16220:
Status: Review In Progress (was: Changes Suggested)
> python dtest
> pending_range_test.py::TestPendingRangeMovements::test_pending_range
> (@pytest.mark.resource_intensive) fails on trunk
> --
>
> Key: CASSANDRA-16220
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16220
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: David Capwell
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta
>
>
> The test has the following section
> {code}
> if cluster.version() >= '2.2':
> node2.watch_log_for('127.0.0.1 state moving', timeout=10,
> filename='debug.log’)
> {code}
> The issue is that in trunk we have the port attached to the log, so the log
> is now
> {code}
> DEBUG [GossipStage:1] 2020-10-21 00:48:20,104 StorageService.java:2452 - Node
> /127.0.0.1:7000 state MOVING, tokens [-9223372036854775808]
> DEBUG [GossipStage:1] 2020-10-21 00:48:20,105 StorageService.java:2670 - Node
> /127.0.0.1:7000 state moving, new token -634023222112864484
> {code}
> Since the log now contains the port, this test always times out on trunk when
> it hits this
> {code}
> self =
> @pytest.mark.resource_intensive
> def test_pending_range(self):
> """
> @jira_ticket CASSANDRA-10887
> """
> cluster = self.cluster
> # If we are on 2.1, we need to set the log level to debug or higher,
> as debug.log does not exist.
> if cluster.version() < '2.2':
> cluster.set_log_level('DEBUG')
>
> # Create 5 node cluster
> cluster.populate(5).start()
> node1, node2 = cluster.nodelist()[0:2]
>
> # Set up RF=3 keyspace
> session = self.patient_cql_connection(node1)
> create_ks(session, 'ks', 3)
>
> session.execute("CREATE TABLE users (login text PRIMARY KEY, email
> text, name text, login_count int)")
>
> # We use the partition key 'jdoe3' because it belongs to node1.
> # The key MUST belong to node1 to repro the bug.
> session.execute("INSERT INTO users (login, email, name, login_count)
> VALUES ('jdoe3', 'j...@abc.com', 'Jane Doe', 1) IF NOT EXISTS;")
>
> lwt_query = SimpleStatement("UPDATE users SET email =
> 'jane...@abc.com' WHERE login = 'jdoe3' IF email = 'j...@abc.com'")
>
> # Show we can execute LWT no problem
> for i in range(1000):
> session.execute(lwt_query)
>
> token = '-634023222112864484'
>
> mark = node1.mark_log()
>
> # Move a node
> node1.nodetool('move {}'.format(token))
>
> # Watch the log so we know when the node is moving
> node1.watch_log_for('Moving .* to {}'.format(token), timeout=10,
> from_mark=mark)
> node1.watch_log_for('Sleeping 3 ms before start
> streaming/fetching ranges', timeout=10, from_mark=mark)
>
> if cluster.version() >= '2.2':
> > node2.watch_log_for('127.0.0.1 state moving', timeout=10,
> > filename='debug.log')
> pending_range_test.py:57:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> self =
> exprs = '127.0.0.1 state moving', from_mark = None, timeout = 10, process =
> None
> verbose = False, filename = 'debug.log'
> def watch_log_for(self, exprs, from_mark=None, timeout=600,
> process=None, verbose=False, filename='system.log'):
> """
> Watch the log until one or more (regular) expressions are found
> or timeouts (a
> TimeoutError is then raised). On successful completion, a list of
> pair (line matched,
> match object) is returned.
> """
> start = time.time()
> tofind = [exprs] if isinstance(exprs, string_types) else exprs
> tofind = [re.compile(e) for e in tofind]
> matchings = []
> reads = ""
> if len(tofind) == 0:
> return None
>
> log_file = os.path.join(self.log_directory(), filename)
> output_read = False
> while not os.path.exists(log_file):
> time.sleep(.5)
> if start + timeout < time.time():
> raise TimeoutError(time.strftime("%d %b %Y %H:%M:%S",
> time.gmtime()) + " [" + self.name + "] Timed out waiting for {} to be
> created.".format(log_file))
> if process and not output_read:
> process.poll()
> if process.returncode is not None:
>
[jira] [Updated] (CASSANDRA-16225) Followup CASSANDRA-14554
[ https://issues.apache.org/jira/browse/CASSANDRA-16225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-16225: Description: As per [~stefania]'s advice, additional synchronization should be added to LogTransaction.java. Without synchronization, we could have corrupted txn log files with JBOD. (was: As per [~stefania]'s advice, additional synchronization should be added to LogTransaction.java. Without synchronization, we could have corrupt txn log files with JBOD.) > Followup CASSANDRA-14554 > > > Key: CASSANDRA-16225 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16225 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Internode >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > > As per [~stefania]'s advice, additional synchronization should be added to > LogTransaction.java. Without synchronization, we could have corrupted txn log > files with JBOD. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16103) Invalid serialized size for responses
[
https://issues.apache.org/jira/browse/CASSANDRA-16103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221804#comment-17221804
]
Yifan Cai commented on CASSANDRA-16103:
---
Sure. Rebased and rerun CI
> Invalid serialized size for responses
> -
>
> Key: CASSANDRA-16103
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16103
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: David Capwell
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Was running a benchmark at LOCAL_ONE and eventually saw the below exception;
> this is related to CASSANDRA-16097 as it was found during the same test.
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized
> size; expected 14, actual size at least 13, for verb HINT_RSP
> at
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
> at
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
> at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized
> size; expected 14, actual size at least 13, for verb MUTATION_RSP
> at
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
> at
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
> at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16103) Invalid serialized size for responses
[
https://issues.apache.org/jira/browse/CASSANDRA-16103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221792#comment-17221792
]
David Capwell commented on CASSANDRA-16103:
---
[~yifanc] for the upgrade tests you would need to rebase to pick up changes
> Invalid serialized size for responses
> -
>
> Key: CASSANDRA-16103
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16103
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: David Capwell
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Was running a benchmark at LOCAL_ONE and eventually saw the below exception;
> this is related to CASSANDRA-16097 as it was found during the same test.
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized
> size; expected 14, actual size at least 13, for verb HINT_RSP
> at
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
> at
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
> at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized
> size; expected 14, actual size at least 13, for verb MUTATION_RSP
> at
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
> at
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
> at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16103) Invalid serialized size for responses
[
https://issues.apache.org/jira/browse/CASSANDRA-16103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221788#comment-17221788
]
David Capwell commented on CASSANDRA-16103:
---
my comments are only nits, so +1 from me.
> Invalid serialized size for responses
> -
>
> Key: CASSANDRA-16103
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16103
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: David Capwell
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Was running a benchmark at LOCAL_ONE and eventually saw the below exception;
> this is related to CASSANDRA-16097 as it was found during the same test.
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized
> size; expected 14, actual size at least 13, for verb HINT_RSP
> at
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
> at
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
> at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized
> size; expected 14, actual size at least 13, for verb MUTATION_RSP
> at
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
> at
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
> at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16144) TLS connections to the storage port on a node without server encryption configured causes java.io.IOException accessing missing keystore
[
https://issues.apache.org/jira/browse/CASSANDRA-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221776#comment-17221776
]
David Capwell commented on CASSANDRA-16144:
---
[~jmeredithco] can you take a look at the trunk tests, ssl tests are failing
https://app.circleci.com/pipelines/github/dcapwell/cassandra/742/workflows/438a85b4-a6b3-4d07-9ff0-b642fdef011c/jobs/4208
> TLS connections to the storage port on a node without server encryption
> configured causes java.io.IOException accessing missing keystore
>
>
> Key: CASSANDRA-16144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16144
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> If a TLS connection is requested against a node with all encryption disabled
> by configuration,
> configured with
> {code}
> server_encryption_options: {optional:false, internode_encryption: none}
> {code}
> it logs the following error if no keystore exists for the node.
> {code}
> INFO [Messaging-EventLoop-3-3] 2020-09-15T14:30:02,952 : -
> 127.0.0.1:7000->127.0.1.1:7000-URGENT_MESSAGES-[no-channel] failed to connect
> io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection
> refused: local1-i1/127.0.1.1:7000
> Caused by: java.net.ConnectException: Connection refused
>at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?]
>at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779) ~[?:?]
>at
> io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:702)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at java.lang.Thread.run(Thread.java:834) [?:?]
> WARN [Messaging-EventLoop-3-9] 2020-09-15T14:30:06,375 : - Failed to
> initialize a channel. Closing: [id: 0x0746c157, L:/127.0.0.1:7000 -
> R:/127.0.0.1:59623]
> java.io.IOException: failed to build trust manager store for secure
> connections
>at
> org.apache.cassandra.security.SSLFactory.buildKeyManagerFactory(SSLFactory.java:232)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.security.SSLFactory.createNettySslContext(SSLFactory.java:300)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.security.SSLFactory.getOrCreateSslContext(SSLFactory.java:276)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.security.SSLFactory.getOrCreateSslContext(SSLFactory.java:257)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.net.InboundConnectionInitiator$Initializer.initChannel(InboundConnectionInitiator.java:107)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.net.InboundConnectionInitiator$Initializer.initChannel(InboundConnectionInitiator.java:71)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> io.netty.channel.ChannelInitializer.initChannel(ChannelInitializer.java:129)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.ChannelInitializer.handlerAdded(ChannelInitializer.java:112)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.AbstractChannelHandlerContext.callHandlerAdded(AbstractChannelHandlerContext.java:938)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>
[jira] [Updated] (CASSANDRA-16144) TLS connections to the storage port on a node without server encryption configured causes java.io.IOException accessing missing keystore
[
https://issues.apache.org/jira/browse/CASSANDRA-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-16144:
--
Status: Changes Suggested (was: Ready to Commit)
> TLS connections to the storage port on a node without server encryption
> configured causes java.io.IOException accessing missing keystore
>
>
> Key: CASSANDRA-16144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16144
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> If a TLS connection is requested against a node with all encryption disabled
> by configuration,
> configured with
> {code}
> server_encryption_options: {optional:false, internode_encryption: none}
> {code}
> it logs the following error if no keystore exists for the node.
> {code}
> INFO [Messaging-EventLoop-3-3] 2020-09-15T14:30:02,952 : -
> 127.0.0.1:7000->127.0.1.1:7000-URGENT_MESSAGES-[no-channel] failed to connect
> io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection
> refused: local1-i1/127.0.1.1:7000
> Caused by: java.net.ConnectException: Connection refused
>at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?]
>at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779) ~[?:?]
>at
> io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:702)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at java.lang.Thread.run(Thread.java:834) [?:?]
> WARN [Messaging-EventLoop-3-9] 2020-09-15T14:30:06,375 : - Failed to
> initialize a channel. Closing: [id: 0x0746c157, L:/127.0.0.1:7000 -
> R:/127.0.0.1:59623]
> java.io.IOException: failed to build trust manager store for secure
> connections
>at
> org.apache.cassandra.security.SSLFactory.buildKeyManagerFactory(SSLFactory.java:232)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.security.SSLFactory.createNettySslContext(SSLFactory.java:300)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.security.SSLFactory.getOrCreateSslContext(SSLFactory.java:276)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.security.SSLFactory.getOrCreateSslContext(SSLFactory.java:257)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.net.InboundConnectionInitiator$Initializer.initChannel(InboundConnectionInitiator.java:107)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.net.InboundConnectionInitiator$Initializer.initChannel(InboundConnectionInitiator.java:71)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> io.netty.channel.ChannelInitializer.initChannel(ChannelInitializer.java:129)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.ChannelInitializer.handlerAdded(ChannelInitializer.java:112)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.AbstractChannelHandlerContext.callHandlerAdded(AbstractChannelHandlerContext.java:938)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:609)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
>
[jira] [Updated] (CASSANDRA-16228) Fix race condition on ColumnFamilyStore and TableMetrics initialization that makes MetricListener to see an instance in an unknown state
[
https://issues.apache.org/jira/browse/CASSANDRA-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-16228:
Test and Documentation Plan:
https://issues.apache.org/jira/browse/CASSANDRA-16228?focusedCommentId=17221755=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17221755
Status: Patch Available (was: In Progress)
> Fix race condition on ColumnFamilyStore and TableMetrics initialization that
> makes MetricListener to see an instance in an unknown state
>
>
> Key: CASSANDRA-16228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16228
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> There is a dependency between TableMetrics, SecondaryIndexManager and Tracker
> which requires certain order of the operations in ColumnFamilyStore
> constructor.
> The following test will be added to ColumnFamilyMetricTest to show the issue:
> {code:java}
> @Test
> public void testStartupRaceConditionOnMetricListeners()
> {
> // Since the ColumnFamilyStore instance reference escapes during the
> construction
> // we have a race condition and listeners can see an instance that is in an
> unknown state.
> // This test just check that all callbacks can access the data without
> throwing any exception.
> registerMetricListener();
> SchemaLoader.createKeyspace("Keyspace2",
> KeyspaceParams.simple(1),
> SchemaLoader.standardCFMD("Keyspace2", "Standard2"));
> }
> private void registerMetricListener()
> {
> CassandraMetricsRegistry.Metrics.addListener(new
> MetricRegistryListener.Base()
> {
> @Override
> public void onGaugeAdded(String name, Gauge gauge)
> {
> gauge.getValue();
> }
> @Override
> public void onGaugeRemoved(String name)
> {
> }
> @Override
> public void onCounterAdded(String name, Counter counter)
> {
> counter.getCount();
> }
> @Override
> public void onCounterRemoved(String name)
> {
> }
> @Override
> public void onHistogramAdded(String name, Histogram histogram)
> {
> histogram.getCount();
> }
> @Override
> public void onHistogramRemoved(String name)
> {
> }
> @Override
> public void onMeterAdded(String name, Meter meter)
> {
> meter.getCount();
> }
> @Override
> public void onMeterRemoved(String name)
> {
> }
> @Override
> public void onTimerAdded(String name, Timer timer)
> {
> timer.getCount();
> }
> @Override
> public void onTimerRemoved(String name)
> {
> }
> });{code}
> *NOTE:* In order to be able to run the test and use registerMetricListener()
> method a small modification to requests in CacheMetrics is needed. This will
> be submitted and further explained with a coming patch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16228) Fix race condition on ColumnFamilyStore and TableMetrics initialization that makes MetricListener to see an instance in an unknown state
[ https://issues.apache.org/jira/browse/CASSANDRA-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221755#comment-17221755 ] Ekaterina Dimitrova edited comment on CASSANDRA-16228 at 10/27/20, 9:11 PM: The patch adds a [test|https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R122] to cover the mentioned race condition. In 4.0, as part of CASSANDRA-14628, requests implementation in CacheMetrics was changed but this change (the 2nd suggestion [here|https://issues.apache.org/jira/browse/CASSANDRA-14626?focusedCommentId=16587227=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16587227] ) prevents us from using the codadhale [addListener() |https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R136]. That is why I reverted the requests implementation to the original [version |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] from CASSANDRA-14628, changing its type from Metered to Meter and using the mark method. The race condition was fixed itself in 3.0, 3.11, 4.0. [4.0 pull request |https://github.com/ekaterinadimitrova2/cassandra/pull/67] | [JAVA 8 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/8df700c5-1775-433f-8dfc-3b3391f3d0ae] | [JAVA 11 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/4f8946b5-6448-4dd6-b231-8f288c861f34] [Jenkins CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/144/] [3.0 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/145/] [3.11 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/dfcb3e82e952f8b1dbeb1fe91a3ed3fcb07abc5e] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/146/changes] [3.0 patch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] to be merged also to 3.11 and 4.0 *after* applying first this [patch |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] to 4.0 was (Author: e.dimitrova): The patch adds a [test|https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R122] to cover the mentioned race condition. In 4.0, as part of CASSANDRA-14628, requests implementation in CacheMetrics was changed but this change (the 2nd suggestion here ) prevents us from using the codadhale [addListener() |https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R136]. That is why I reverted the requests implementation to the original [version |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] from CASSANDRA-14628, changing its type from Metered to Meter and using the mark method. The race condition was fixed itself in 3.0, 3.11, 4.0. [4.0 pull request |https://github.com/ekaterinadimitrova2/cassandra/pull/67] | [JAVA 8 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/8df700c5-1775-433f-8dfc-3b3391f3d0ae] | [JAVA 11 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/4f8946b5-6448-4dd6-b231-8f288c861f34] [Jenkins CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/144/] [3.0 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/145/] [3.11 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/dfcb3e82e952f8b1dbeb1fe91a3ed3fcb07abc5e] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/146/changes] [3.0 patch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] to be merged also to 3.11 and 4.0 *after* applying first this [patch |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] to 4.0 > Fix race condition on ColumnFamilyStore and TableMetrics initialization that > makes MetricListener to see an instance in an unknown state > > > Key: CASSANDRA-16228 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16228 > Project: Cassandra >
[jira] [Comment Edited] (CASSANDRA-16228) Fix race condition on ColumnFamilyStore and TableMetrics initialization that makes MetricListener to see an instance in an unknown state
[ https://issues.apache.org/jira/browse/CASSANDRA-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221755#comment-17221755 ] Ekaterina Dimitrova edited comment on CASSANDRA-16228 at 10/27/20, 9:11 PM: The patch adds a [test|https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R122] to cover the mentioned race condition. In 4.0, as part of CASSANDRA-14628, requests implementation in CacheMetrics was changed but this change (the 2nd suggestion [here|https://issues.apache.org/jira/browse/CASSANDRA-14626?focusedCommentId=16587227=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16587227] ) prevents us from using the codahale [addListener() |https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R136]. That is why I reverted the requests implementation to the original [version |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] from CASSANDRA-14628, changing its type from Metered to Meter and using the mark method. The race condition was fixed itself in 3.0, 3.11, 4.0. [4.0 pull request |https://github.com/ekaterinadimitrova2/cassandra/pull/67] | [JAVA 8 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/8df700c5-1775-433f-8dfc-3b3391f3d0ae] | [JAVA 11 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/4f8946b5-6448-4dd6-b231-8f288c861f34] [Jenkins CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/144/] [3.0 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/145/] [3.11 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/dfcb3e82e952f8b1dbeb1fe91a3ed3fcb07abc5e] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/146/changes] [3.0 patch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] to be merged also to 3.11 and 4.0 *after* applying first this [patch |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] to 4.0 was (Author: e.dimitrova): The patch adds a [test|https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R122] to cover the mentioned race condition. In 4.0, as part of CASSANDRA-14628, requests implementation in CacheMetrics was changed but this change (the 2nd suggestion [here|https://issues.apache.org/jira/browse/CASSANDRA-14626?focusedCommentId=16587227=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16587227] ) prevents us from using the codadhale [addListener() |https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R136]. That is why I reverted the requests implementation to the original [version |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] from CASSANDRA-14628, changing its type from Metered to Meter and using the mark method. The race condition was fixed itself in 3.0, 3.11, 4.0. [4.0 pull request |https://github.com/ekaterinadimitrova2/cassandra/pull/67] | [JAVA 8 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/8df700c5-1775-433f-8dfc-3b3391f3d0ae] | [JAVA 11 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/4f8946b5-6448-4dd6-b231-8f288c861f34] [Jenkins CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/144/] [3.0 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/145/] [3.11 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/dfcb3e82e952f8b1dbeb1fe91a3ed3fcb07abc5e] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/146/changes] [3.0 patch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] to be merged also to 3.11 and 4.0 *after* applying first this [patch |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] to 4.0 > Fix race condition on ColumnFamilyStore and TableMetrics initialization that > makes MetricListener to see an instance in an unknown state >
[jira] [Comment Edited] (CASSANDRA-16228) Fix race condition on ColumnFamilyStore and TableMetrics initialization that makes MetricListener to see an instance in an unknown state
[ https://issues.apache.org/jira/browse/CASSANDRA-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221755#comment-17221755 ] Ekaterina Dimitrova edited comment on CASSANDRA-16228 at 10/27/20, 9:10 PM: The patch adds a [test|https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R122] to cover the mentioned race condition. In 4.0, as part of CASSANDRA-14628, requests implementation in CacheMetrics was changed but this change (the 2nd suggestion here ) prevents us from using the codadhale [addListener() |https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R136]. That is why I reverted the requests implementation to the original [version |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] from CASSANDRA-14628, changing its type from Metered to Meter and using the mark method. The race condition was fixed itself in 3.0, 3.11, 4.0. [4.0 pull request |https://github.com/ekaterinadimitrova2/cassandra/pull/67] | [JAVA 8 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/8df700c5-1775-433f-8dfc-3b3391f3d0ae] | [JAVA 11 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/4f8946b5-6448-4dd6-b231-8f288c861f34] [Jenkins CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/144/] [3.0 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/145/] [3.11 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/dfcb3e82e952f8b1dbeb1fe91a3ed3fcb07abc5e] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/146/changes] [3.0 patch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] to be merged also to 3.11 and 4.0 *after* applying first this [patch |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] to 4.0 was (Author: e.dimitrova): The patch adds a [test|https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R122] to cover the mentioned race condition. In 4.0, as part of CASSANDRA-14628, requests implementation in CacheMetrics was changed but this change (2n suggestion here ) prevents us from using the codadhale [addListener() |https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R136]. That is why I reverted the requests implementation to the original [version |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] from CASSANDRA-14628, changing its type from Metered to Meter and using the mark method. The race condition was fixed itself in 3.0, 3.11, 4.0. [4.0 pull request |https://github.com/ekaterinadimitrova2/cassandra/pull/67] | [JAVA 8 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/8df700c5-1775-433f-8dfc-3b3391f3d0ae] | [JAVA 11 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/4f8946b5-6448-4dd6-b231-8f288c861f34] [Jenkins CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/144/] [3.0 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/145/] [3.11 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/dfcb3e82e952f8b1dbeb1fe91a3ed3fcb07abc5e] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/146/changes] [3.0 patch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] to be merged also to 3.11 and 4.0 *after* applying first this [patch |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] to 4.0 > Fix race condition on ColumnFamilyStore and TableMetrics initialization that > makes MetricListener to see an instance in an unknown state > > > Key: CASSANDRA-16228 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16228 > Project: Cassandra > Issue Type: Bug > Components: Observability/Metrics >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority:
[jira] [Comment Edited] (CASSANDRA-16228) Fix race condition on ColumnFamilyStore and TableMetrics initialization that makes MetricListener to see an instance in an unknown state
[ https://issues.apache.org/jira/browse/CASSANDRA-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221755#comment-17221755 ] Ekaterina Dimitrova edited comment on CASSANDRA-16228 at 10/27/20, 9:10 PM: The patch adds a [test|https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R122] to cover the mentioned race condition. In 4.0, as part of CASSANDRA-14628, requests implementation in CacheMetrics was changed but this change (the 2nd suggestion here ) prevents us from using the codadhale [addListener() |https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R136]. That is why I reverted the requests implementation to the original [version |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] from CASSANDRA-14628, changing its type from Metered to Meter and using the mark method. The race condition was fixed itself in 3.0, 3.11, 4.0. [4.0 pull request |https://github.com/ekaterinadimitrova2/cassandra/pull/67] | [JAVA 8 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/8df700c5-1775-433f-8dfc-3b3391f3d0ae] | [JAVA 11 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/4f8946b5-6448-4dd6-b231-8f288c861f34] [Jenkins CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/144/] [3.0 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/145/] [3.11 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/dfcb3e82e952f8b1dbeb1fe91a3ed3fcb07abc5e] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/146/changes] [3.0 patch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] to be merged also to 3.11 and 4.0 *after* applying first this [patch |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] to 4.0 was (Author: e.dimitrova): The patch adds a [test|https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R122] to cover the mentioned race condition. In 4.0, as part of CASSANDRA-14628, requests implementation in CacheMetrics was changed but this change (the 2nd suggestion here ) prevents us from using the codadhale [addListener() |https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R136]. That is why I reverted the requests implementation to the original [version |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] from CASSANDRA-14628, changing its type from Metered to Meter and using the mark method. The race condition was fixed itself in 3.0, 3.11, 4.0. [4.0 pull request |https://github.com/ekaterinadimitrova2/cassandra/pull/67] | [JAVA 8 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/8df700c5-1775-433f-8dfc-3b3391f3d0ae] | [JAVA 11 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/4f8946b5-6448-4dd6-b231-8f288c861f34] [Jenkins CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/144/] [3.0 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/145/] [3.11 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/dfcb3e82e952f8b1dbeb1fe91a3ed3fcb07abc5e] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/146/changes] [3.0 patch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] to be merged also to 3.11 and 4.0 *after* applying first this [patch |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] to 4.0 > Fix race condition on ColumnFamilyStore and TableMetrics initialization that > makes MetricListener to see an instance in an unknown state > > > Key: CASSANDRA-16228 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16228 > Project: Cassandra > Issue Type: Bug > Components: Observability/Metrics >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >
[jira] [Comment Edited] (CASSANDRA-16228) Fix race condition on ColumnFamilyStore and TableMetrics initialization that makes MetricListener to see an instance in an unknown state
[ https://issues.apache.org/jira/browse/CASSANDRA-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221755#comment-17221755 ] Ekaterina Dimitrova edited comment on CASSANDRA-16228 at 10/27/20, 9:09 PM: The patch adds a [test|https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R122] to cover the mentioned race condition. In 4.0, as part of CASSANDRA-14628, requests implementation in CacheMetrics was changed but this change (2n suggestion here ) prevents us from using the codadhale [addListener() |https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R136]. That is why I reverted the requests implementation to the original [version |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] from CASSANDRA-14628, changing its type from Metered to Meter and using the mark method. The race condition was fixed itself in 3.0, 3.11, 4.0. [4.0 pull request |https://github.com/ekaterinadimitrova2/cassandra/pull/67] | [JAVA 8 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/8df700c5-1775-433f-8dfc-3b3391f3d0ae] | [JAVA 11 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/4f8946b5-6448-4dd6-b231-8f288c861f34] [Jenkins CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/144/] [3.0 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/145/] [3.11 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/dfcb3e82e952f8b1dbeb1fe91a3ed3fcb07abc5e] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/146/changes] [3.0 patch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] to be merged also to 3.11 and 4.0 *after* applying first this [patch |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] to 4.0 was (Author: e.dimitrova): The patch adds a [test|https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R122] to cover the mentioned race condition. In 4.0, as part of CASSANDRA-, requests implementation in CacheMetrics was changed but this change (2n suggestion here ) prevents us from using the codadhale [addListener() |https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R136]. That is why I reverted the requests implementation to the original [version |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] from CASSANDRA-, changing its type from Metered to Meter and using the mark method. The race condition was fixed itself in 3.0, 3.11, 4.0. [4.0 pull request |https://github.com/ekaterinadimitrova2/cassandra/pull/67] | [JAVA 8 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/8df700c5-1775-433f-8dfc-3b3391f3d0ae] | [JAVA 11 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/4f8946b5-6448-4dd6-b231-8f288c861f34] [Jenkins CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/144/] [3.0 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/145/] [3.11 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/dfcb3e82e952f8b1dbeb1fe91a3ed3fcb07abc5e] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/146/changes] [3.0 patch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] to be merged also to 3.11 and 4.0 *after* applying first this [patch |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] to 4.0 > Fix race condition on ColumnFamilyStore and TableMetrics initialization that > makes MetricListener to see an instance in an unknown state > > > Key: CASSANDRA-16228 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16228 > Project: Cassandra > Issue Type: Bug > Components: Observability/Metrics >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal >
[jira] [Comment Edited] (CASSANDRA-16228) Fix race condition on ColumnFamilyStore and TableMetrics initialization that makes MetricListener to see an instance in an unknown state
[ https://issues.apache.org/jira/browse/CASSANDRA-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221755#comment-17221755 ] Ekaterina Dimitrova edited comment on CASSANDRA-16228 at 10/27/20, 9:07 PM: The patch adds a [test|https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R122] to cover the mentioned race condition. In 4.0, as part of CASSANDRA-, requests implementation in CacheMetrics was changed but this change (2n suggestion here ) prevents us from using the codadhale [addListener() |https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R136]. That is why I reverted the requests implementation to the original [version |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] from CASSANDRA-, changing its type from Metered to Meter and using the mark method. The race condition was fixed itself in 3.0, 3.11, 4.0. [4.0 pull request |https://github.com/ekaterinadimitrova2/cassandra/pull/67] | [JAVA 8 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/8df700c5-1775-433f-8dfc-3b3391f3d0ae] | [JAVA 11 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/4f8946b5-6448-4dd6-b231-8f288c861f34] [Jenkins CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/144/] [3.0 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/145/] [3.11 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/dfcb3e82e952f8b1dbeb1fe91a3ed3fcb07abc5e] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/146/changes] [3.0 patch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] to be merged also to 3.11 and 4.0 *after* applying first this [patch |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] to 4.0 was (Author: e.dimitrova): The patch adds a [test|https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R122] to cover the mentioned race condition. In 4.0, as part of CASSANDRA-, requests implementation in CacheMetrics was changed but this change (2n suggestion [here | https://issues.apache.org/jira/browse/CASSANDRA-14626?focusedCommentId=16587227=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16587227]) prevents us from using the codadhale [addListener() | https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R136]. That is why I reverted the requests implementation to the original [version | https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] from CASSANDRA-, changing its type from Metered to Meter and using the mark method. The race condition was fixed itself in 3.0, 3.11, 4.0. [4.0 pull request | https://github.com/ekaterinadimitrova2/cassandra/pull/67] | [JAVA 8 CI | https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/8df700c5-1775-433f-8dfc-3b3391f3d0ae] | [JAVA 11 CI | https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/4f8946b5-6448-4dd6-b231-8f288c861f34] [Jenkins CI | https://jenkins-cm4.apache.org/job/Cassandra-devbranch/144/] [3.0 branch | https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] | [CI | https://jenkins-cm4.apache.org/job/Cassandra-devbranch/145/] [3.11 branch | https://github.com/ekaterinadimitrova2/cassandra/commit/dfcb3e82e952f8b1dbeb1fe91a3ed3fcb07abc5e] | [CI | https://jenkins-cm4.apache.org/job/Cassandra-devbranch/146/changes] [3.0 patch | https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] to be merged also to 3.11 and 4.0 *after * applying first this [patch | https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] to 4.0 > Fix race condition on ColumnFamilyStore and TableMetrics initialization that > makes MetricListener to see an instance in an unknown state > > > Key: CASSANDRA-16228 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16228 > Project: Cassandra > Issue Type: Bug >
[jira] [Commented] (CASSANDRA-16228) Fix race condition on ColumnFamilyStore and TableMetrics initialization that makes MetricListener to see an instance in an unknown state
[
https://issues.apache.org/jira/browse/CASSANDRA-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221755#comment-17221755
]
Ekaterina Dimitrova commented on CASSANDRA-16228:
-
The patch adds a
[test|https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R122]
to cover the mentioned race condition.
In 4.0, as part of CASSANDRA-, requests implementation in CacheMetrics was
changed but this change (2n suggestion [here |
https://issues.apache.org/jira/browse/CASSANDRA-14626?focusedCommentId=16587227=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16587227])
prevents us from using the codadhale [addListener() |
https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R136].
That is why I reverted the requests implementation to the original [version |
https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f]
from CASSANDRA-, changing its type from Metered to Meter and using the mark
method.
The race condition was fixed itself in 3.0, 3.11, 4.0.
[4.0 pull request | https://github.com/ekaterinadimitrova2/cassandra/pull/67] |
[JAVA 8 CI |
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/8df700c5-1775-433f-8dfc-3b3391f3d0ae]
| [JAVA 11 CI |
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/4f8946b5-6448-4dd6-b231-8f288c861f34]
[Jenkins CI | https://jenkins-cm4.apache.org/job/Cassandra-devbranch/144/]
[3.0 branch |
https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43]
| [CI | https://jenkins-cm4.apache.org/job/Cassandra-devbranch/145/]
[3.11 branch |
https://github.com/ekaterinadimitrova2/cassandra/commit/dfcb3e82e952f8b1dbeb1fe91a3ed3fcb07abc5e]
| [CI | https://jenkins-cm4.apache.org/job/Cassandra-devbranch/146/changes]
[3.0 patch |
https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43]
to be merged also to 3.11 and 4.0 *after * applying first this [patch |
https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f]
to 4.0
> Fix race condition on ColumnFamilyStore and TableMetrics initialization that
> makes MetricListener to see an instance in an unknown state
>
>
> Key: CASSANDRA-16228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16228
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
>
> There is a dependency between TableMetrics, SecondaryIndexManager and Tracker
> which requires certain order of the operations in ColumnFamilyStore
> constructor.
> The following test will be added to ColumnFamilyMetricTest to show the issue:
> {code:java}
> @Test
> public void testStartupRaceConditionOnMetricListeners()
> {
> // Since the ColumnFamilyStore instance reference escapes during the
> construction
> // we have a race condition and listeners can see an instance that is in an
> unknown state.
> // This test just check that all callbacks can access the data without
> throwing any exception.
> registerMetricListener();
> SchemaLoader.createKeyspace("Keyspace2",
> KeyspaceParams.simple(1),
> SchemaLoader.standardCFMD("Keyspace2", "Standard2"));
> }
> private void registerMetricListener()
> {
> CassandraMetricsRegistry.Metrics.addListener(new
> MetricRegistryListener.Base()
> {
> @Override
> public void onGaugeAdded(String name, Gauge gauge)
> {
> gauge.getValue();
> }
> @Override
> public void onGaugeRemoved(String name)
> {
> }
> @Override
> public void onCounterAdded(String name, Counter counter)
> {
> counter.getCount();
> }
> @Override
> public void onCounterRemoved(String name)
> {
> }
> @Override
> public void onHistogramAdded(String name, Histogram histogram)
> {
> histogram.getCount();
> }
> @Override
> public void onHistogramRemoved(String name)
> {
> }
> @Override
> public void onMeterAdded(String name, Meter meter)
> {
> meter.getCount();
> }
> @Override
> public void onMeterRemoved(String name)
> {
> }
> @Override
> public void onTimerAdded(String name, Timer timer)
> {
> timer.getCount();
> }
> @Override
> public void onTimerRemoved(String name)
> {
> }
> });{code}
> *NOTE:* In order to be able to run the test and use registerMetricListener()
> method a small modification to
[jira] [Updated] (CASSANDRA-16228) Fix race condition on ColumnFamilyStore and TableMetrics initialization that makes MetricListener to see an instance in an unknown state
[
https://issues.apache.org/jira/browse/CASSANDRA-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-16228:
Fix Version/s: 3.11.x
3.0.x
> Fix race condition on ColumnFamilyStore and TableMetrics initialization that
> makes MetricListener to see an instance in an unknown state
>
>
> Key: CASSANDRA-16228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16228
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> There is a dependency between TableMetrics, SecondaryIndexManager and Tracker
> which requires certain order of the operations in ColumnFamilyStore
> constructor.
> The following test will be added to ColumnFamilyMetricTest to show the issue:
> {code:java}
> @Test
> public void testStartupRaceConditionOnMetricListeners()
> {
> // Since the ColumnFamilyStore instance reference escapes during the
> construction
> // we have a race condition and listeners can see an instance that is in an
> unknown state.
> // This test just check that all callbacks can access the data without
> throwing any exception.
> registerMetricListener();
> SchemaLoader.createKeyspace("Keyspace2",
> KeyspaceParams.simple(1),
> SchemaLoader.standardCFMD("Keyspace2", "Standard2"));
> }
> private void registerMetricListener()
> {
> CassandraMetricsRegistry.Metrics.addListener(new
> MetricRegistryListener.Base()
> {
> @Override
> public void onGaugeAdded(String name, Gauge gauge)
> {
> gauge.getValue();
> }
> @Override
> public void onGaugeRemoved(String name)
> {
> }
> @Override
> public void onCounterAdded(String name, Counter counter)
> {
> counter.getCount();
> }
> @Override
> public void onCounterRemoved(String name)
> {
> }
> @Override
> public void onHistogramAdded(String name, Histogram histogram)
> {
> histogram.getCount();
> }
> @Override
> public void onHistogramRemoved(String name)
> {
> }
> @Override
> public void onMeterAdded(String name, Meter meter)
> {
> meter.getCount();
> }
> @Override
> public void onMeterRemoved(String name)
> {
> }
> @Override
> public void onTimerAdded(String name, Timer timer)
> {
> timer.getCount();
> }
> @Override
> public void onTimerRemoved(String name)
> {
> }
> });{code}
> *NOTE:* In order to be able to run the test and use registerMetricListener()
> method a small modification to requests in CacheMetrics is needed. This will
> be submitted and further explained with a coming patch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16144) TLS connections to the storage port on a node without server encryption configured causes java.io.IOException accessing missing keystore
[
https://issues.apache.org/jira/browse/CASSANDRA-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221716#comment-17221716
]
David Capwell edited comment on CASSANDRA-16144 at 10/27/20, 8:14 PM:
--
Starting commit
CI Results (pending):
||Branch||Source||Circle CI||Jenkins||
|cassandra-2.2
|[branch|https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-cassandra-2.2-6298EA10-5C5B-41A3-9575-78AA3A5A5249]
|[build|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-cassandra-2.2-6298EA10-5C5B-41A3-9575-78AA3A5A5249]
|[build|https://ci-cassandra.apache.org/job/Cassandra-devbranch/140/]|
|cassandra-3.0
|[branch|https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-cassandra-3.0-6298EA10-5C5B-41A3-9575-78AA3A5A5249]
|[build|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-cassandra-3.0-6298EA10-5C5B-41A3-9575-78AA3A5A5249]
|[build|https://ci-cassandra.apache.org/job/Cassandra-devbranch/141/]|
|cassandra-3.11
|[branch|https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-cassandra-3.11-6298EA10-5C5B-41A3-9575-78AA3A5A5249]
|[build|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-cassandra-3.11-6298EA10-5C5B-41A3-9575-78AA3A5A5249]
|[build|https://ci-cassandra.apache.org/job/Cassandra-devbranch/142/]|
|trunk
|[branch|https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-trunk-6298EA10-5C5B-41A3-9575-78AA3A5A5249]
|[build|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-trunk-6298EA10-5C5B-41A3-9575-78AA3A5A5249]
|[build|https://ci-cassandra.apache.org/job/Cassandra-devbranch/143/]|
was (Author: dcapwell):
Starting commit
CI Results (pending):
||Branch||Source||Circle CI||Jenkins||
|cassandra-2.2
|https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-cassandra-2.2-6298EA10-5C5B-41A3-9575-78AA3A5A5249
|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-cassandra-2.2-6298EA10-5C5B-41A3-9575-78AA3A5A5249
|https://ci-cassandra.apache.org/job/Cassandra-devbranch/140/|
|cassandra-3.0
|https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-cassandra-3.0-6298EA10-5C5B-41A3-9575-78AA3A5A5249
|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-cassandra-3.0-6298EA10-5C5B-41A3-9575-78AA3A5A5249
|https://ci-cassandra.apache.org/job/Cassandra-devbranch/141/|
|cassandra-3.11
|https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-cassandra-3.11-6298EA10-5C5B-41A3-9575-78AA3A5A5249
|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-cassandra-3.11-6298EA10-5C5B-41A3-9575-78AA3A5A5249
|https://ci-cassandra.apache.org/job/Cassandra-devbranch/142/|
|trunk
|https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-trunk-6298EA10-5C5B-41A3-9575-78AA3A5A5249
|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-trunk-6298EA10-5C5B-41A3-9575-78AA3A5A5249
|https://ci-cassandra.apache.org/job/Cassandra-devbranch/143/|
> TLS connections to the storage port on a node without server encryption
> configured causes java.io.IOException accessing missing keystore
>
>
> Key: CASSANDRA-16144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16144
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> If a TLS connection is requested against a node with all encryption disabled
> by configuration,
> configured with
> {code}
> server_encryption_options: {optional:false, internode_encryption: none}
> {code}
> it logs the following error if no keystore exists for the node.
> {code}
> INFO [Messaging-EventLoop-3-3] 2020-09-15T14:30:02,952 : -
> 127.0.0.1:7000->127.0.1.1:7000-URGENT_MESSAGES-[no-channel] failed to connect
> io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection
> refused: local1-i1/127.0.1.1:7000
> Caused by: java.net.ConnectException: Connection refused
>at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?]
>at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779) ~[?:?]
>
[jira] [Comment Edited] (CASSANDRA-16144) TLS connections to the storage port on a node without server encryption configured causes java.io.IOException accessing missing keystore
[
https://issues.apache.org/jira/browse/CASSANDRA-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221716#comment-17221716
]
David Capwell edited comment on CASSANDRA-16144 at 10/27/20, 8:08 PM:
--
Starting commit
CI Results (pending):
||Branch||Source||Circle CI||Jenkins||
|cassandra-2.2
|https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-cassandra-2.2-6298EA10-5C5B-41A3-9575-78AA3A5A5249
|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-cassandra-2.2-6298EA10-5C5B-41A3-9575-78AA3A5A5249
|https://ci-cassandra.apache.org/job/Cassandra-devbranch/140/|
|cassandra-3.0
|https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-cassandra-3.0-6298EA10-5C5B-41A3-9575-78AA3A5A5249
|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-cassandra-3.0-6298EA10-5C5B-41A3-9575-78AA3A5A5249
|https://ci-cassandra.apache.org/job/Cassandra-devbranch/141/|
|cassandra-3.11
|https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-cassandra-3.11-6298EA10-5C5B-41A3-9575-78AA3A5A5249
|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-cassandra-3.11-6298EA10-5C5B-41A3-9575-78AA3A5A5249
|https://ci-cassandra.apache.org/job/Cassandra-devbranch/142/|
|trunk
|https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-trunk-6298EA10-5C5B-41A3-9575-78AA3A5A5249
|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-trunk-6298EA10-5C5B-41A3-9575-78AA3A5A5249
|https://ci-cassandra.apache.org/job/Cassandra-devbranch/143/|
was (Author: dcapwell):
Starting commit
CI Results (pending):
Branch: cassandra-2.2
Source:
https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-cassandra-2.2-6298EA10-5C5B-41A3-9575-78AA3A5A5249
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-cassandra-2.2-6298EA10-5C5B-41A3-9575-78AA3A5A5249
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/140/
Branch: cassandra-3.0
Source:
https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-cassandra-3.0-6298EA10-5C5B-41A3-9575-78AA3A5A5249
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-cassandra-3.0-6298EA10-5C5B-41A3-9575-78AA3A5A5249
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/141/
Branch: cassandra-3.11
Source:
https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-cassandra-3.11-6298EA10-5C5B-41A3-9575-78AA3A5A5249
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-cassandra-3.11-6298EA10-5C5B-41A3-9575-78AA3A5A5249
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/142/
Branch: trunk
Source:
https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-trunk-6298EA10-5C5B-41A3-9575-78AA3A5A5249
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-trunk-6298EA10-5C5B-41A3-9575-78AA3A5A5249
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/143/
> TLS connections to the storage port on a node without server encryption
> configured causes java.io.IOException accessing missing keystore
>
>
> Key: CASSANDRA-16144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16144
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> If a TLS connection is requested against a node with all encryption disabled
> by configuration,
> configured with
> {code}
> server_encryption_options: {optional:false, internode_encryption: none}
> {code}
> it logs the following error if no keystore exists for the node.
> {code}
> INFO [Messaging-EventLoop-3-3] 2020-09-15T14:30:02,952 : -
> 127.0.0.1:7000->127.0.1.1:7000-URGENT_MESSAGES-[no-channel] failed to connect
> io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection
> refused: local1-i1/127.0.1.1:7000
> Caused by: java.net.ConnectException: Connection refused
>at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?]
>at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779) ~[?:?]
>at
>
[jira] [Commented] (CASSANDRA-16144) TLS connections to the storage port on a node without server encryption configured causes java.io.IOException accessing missing keystore
[
https://issues.apache.org/jira/browse/CASSANDRA-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221716#comment-17221716
]
David Capwell commented on CASSANDRA-16144:
---
Starting commit
CI Results (pending):
Branch: cassandra-2.2
Source:
https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-cassandra-2.2-6298EA10-5C5B-41A3-9575-78AA3A5A5249
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-cassandra-2.2-6298EA10-5C5B-41A3-9575-78AA3A5A5249
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/140/
Branch: cassandra-3.0
Source:
https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-cassandra-3.0-6298EA10-5C5B-41A3-9575-78AA3A5A5249
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-cassandra-3.0-6298EA10-5C5B-41A3-9575-78AA3A5A5249
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/141/
Branch: cassandra-3.11
Source:
https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-cassandra-3.11-6298EA10-5C5B-41A3-9575-78AA3A5A5249
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-cassandra-3.11-6298EA10-5C5B-41A3-9575-78AA3A5A5249
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/142/
Branch: trunk
Source:
https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-trunk-6298EA10-5C5B-41A3-9575-78AA3A5A5249
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-trunk-6298EA10-5C5B-41A3-9575-78AA3A5A5249
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/143/
> TLS connections to the storage port on a node without server encryption
> configured causes java.io.IOException accessing missing keystore
>
>
> Key: CASSANDRA-16144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16144
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> If a TLS connection is requested against a node with all encryption disabled
> by configuration,
> configured with
> {code}
> server_encryption_options: {optional:false, internode_encryption: none}
> {code}
> it logs the following error if no keystore exists for the node.
> {code}
> INFO [Messaging-EventLoop-3-3] 2020-09-15T14:30:02,952 : -
> 127.0.0.1:7000->127.0.1.1:7000-URGENT_MESSAGES-[no-channel] failed to connect
> io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection
> refused: local1-i1/127.0.1.1:7000
> Caused by: java.net.ConnectException: Connection refused
>at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?]
>at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779) ~[?:?]
>at
> io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:702)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at java.lang.Thread.run(Thread.java:834) [?:?]
> WARN [Messaging-EventLoop-3-9] 2020-09-15T14:30:06,375 : - Failed to
> initialize a channel. Closing: [id: 0x0746c157, L:/127.0.0.1:7000 -
> R:/127.0.0.1:59623]
> java.io.IOException: failed to build trust manager store for secure
> connections
>at
>
[jira] [Commented] (CASSANDRA-16144) TLS connections to the storage port on a node without server encryption configured causes java.io.IOException accessing missing keystore
[
https://issues.apache.org/jira/browse/CASSANDRA-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221715#comment-17221715
]
David Capwell commented on CASSANDRA-16144:
---
switch to trunk for dtest broke all python dtests... fixing...
> TLS connections to the storage port on a node without server encryption
> configured causes java.io.IOException accessing missing keystore
>
>
> Key: CASSANDRA-16144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16144
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> If a TLS connection is requested against a node with all encryption disabled
> by configuration,
> configured with
> {code}
> server_encryption_options: {optional:false, internode_encryption: none}
> {code}
> it logs the following error if no keystore exists for the node.
> {code}
> INFO [Messaging-EventLoop-3-3] 2020-09-15T14:30:02,952 : -
> 127.0.0.1:7000->127.0.1.1:7000-URGENT_MESSAGES-[no-channel] failed to connect
> io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection
> refused: local1-i1/127.0.1.1:7000
> Caused by: java.net.ConnectException: Connection refused
>at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?]
>at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779) ~[?:?]
>at
> io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:702)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at java.lang.Thread.run(Thread.java:834) [?:?]
> WARN [Messaging-EventLoop-3-9] 2020-09-15T14:30:06,375 : - Failed to
> initialize a channel. Closing: [id: 0x0746c157, L:/127.0.0.1:7000 -
> R:/127.0.0.1:59623]
> java.io.IOException: failed to build trust manager store for secure
> connections
>at
> org.apache.cassandra.security.SSLFactory.buildKeyManagerFactory(SSLFactory.java:232)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.security.SSLFactory.createNettySslContext(SSLFactory.java:300)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.security.SSLFactory.getOrCreateSslContext(SSLFactory.java:276)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.security.SSLFactory.getOrCreateSslContext(SSLFactory.java:257)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.net.InboundConnectionInitiator$Initializer.initChannel(InboundConnectionInitiator.java:107)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.net.InboundConnectionInitiator$Initializer.initChannel(InboundConnectionInitiator.java:71)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> io.netty.channel.ChannelInitializer.initChannel(ChannelInitializer.java:129)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.ChannelInitializer.handlerAdded(ChannelInitializer.java:112)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.AbstractChannelHandlerContext.callHandlerAdded(AbstractChannelHandlerContext.java:938)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:609)
>
[jira] [Updated] (CASSANDRA-15997) TestBootstrap::test_cleanup failing on unexpected number of SSTables
[
https://issues.apache.org/jira/browse/CASSANDRA-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-15997:
-
Resolution: Later
Status: Resolved (was: Open)
Closing for now with no further failures to drive this.
> TestBootstrap::test_cleanup failing on unexpected number of SSTables
>
>
> Key: CASSANDRA-15997
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15997
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Caleb Rackliffe
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-beta
>
>
> This failure has now occurred in a number of places on trunk (4.0), including
> both Java 8 and 11 dtest runs. Nominally, there appear to be more SSTables
> after cleanup than the test is expecting.
> {noformat}
> if len(sstables) > basecount + jobs:
> logger.debug("Current count is {}, basecount was
> {}".format(len(sstables), basecount))
> failed.set()
> {noformat}
> Examples:
> https://app.circleci.com/pipelines/github/maedhroz/cassandra/92/workflows/c59be4f8-329e-4d76-9c59-d49c38e58dd2/jobs/448
> https://app.circleci.com/pipelines/github/jolynch/cassandra/20/workflows/9d6c3b86-6207-4ead-aa4b-79022fc84182/jobs/893
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16226) COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by timestamp due to missing primary key liveness info
[
https://issues.apache.org/jira/browse/CASSANDRA-16226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-16226:
Status: In Progress (was: Patch Available)
> COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by
> timestamp due to missing primary key liveness info
> ---
>
> Key: CASSANDRA-16226
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16226
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: perfomance, upgrade
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This was discovered while tracking down a spike in the number of SSTables
> per read for a COMPACT STORAGE table after a 2.1 -> 3.0 upgrade. Before 3.0,
> there is no direct analog of 3.0's primary key liveness info. When we upgrade
> 2.1 COMPACT STORAGE SSTables to the mf format, we simply don't write row
> timestamps, even if the original mutations were INSERTs. On read, when we
> look at SSTables in order from newest to oldest max timestamp, we expect to
> have this primary key liveness information to determine whether we can skip
> older SSTables after finding completely populated rows.
> ex. I have three SSTables in a COMPACT STORAGE table with max timestamps
> 1000, 2000, and 3000. There are many rows in a particular partition, making
> filtering on the min and max clustering effectively a no-op. All data is
> inserted, and there are no partial updates. A fully specified row with
> timestamp 2500 exists in the SSTable with a max timestamp of 3000. With a
> proper row timestamp in hand, we can easily ignore the SSTables w/ max
> timestamps of 1000 and 2000. Without it, we read 3 SSTables instead of 1,
> which likely means a significant performance regression.
> The following test illustrates this difference in behavior between 2.1 and
> 3.0:
> https://github.com/maedhroz/cassandra/commit/84ce9242bedd735ca79d4f06007d127de6a82800
> A solution here might be as simple as having
> {{SinglePartitionReadCommand#canRemoveRow()}} only inspect primary key
> liveness information for non-compact/CQL tables. Tombstones seem to be
> handled at a level above that anyway. (One potential problem with that is
> whether or not the distinction will continue to exist in 4.0, and dropping
> compact storage from a table doesn't magically make pk liveness information
> appear.)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16226) COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by timestamp due to missing primary key liveness info
[
https://issues.apache.org/jira/browse/CASSANDRA-16226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221620#comment-17221620
]
Caleb Rackliffe commented on CASSANDRA-16226:
-
[~marcuse] [~jwest] [~ifesdjeen] What might make sense (and I need to
coordinate with you) is whether we could solve the above problem with a simple
flag at the SSTable level that indicates whether empty primary key liveness
info is valid. If we write this in {{upgradesstables}} and then require its
presence before running {{DROP COMPACT STORAGE}}, I think we would have the
guardrails necessary to make it basically impossible for people to accidentally
re-introduce this regression. (This would be coupled, of course, with changing
the current patch to look at that flag, not the table level {{isCQLTable()}}.)
> COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by
> timestamp due to missing primary key liveness info
> ---
>
> Key: CASSANDRA-16226
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16226
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: perfomance, upgrade
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This was discovered while tracking down a spike in the number of SSTables
> per read for a COMPACT STORAGE table after a 2.1 -> 3.0 upgrade. Before 3.0,
> there is no direct analog of 3.0's primary key liveness info. When we upgrade
> 2.1 COMPACT STORAGE SSTables to the mf format, we simply don't write row
> timestamps, even if the original mutations were INSERTs. On read, when we
> look at SSTables in order from newest to oldest max timestamp, we expect to
> have this primary key liveness information to determine whether we can skip
> older SSTables after finding completely populated rows.
> ex. I have three SSTables in a COMPACT STORAGE table with max timestamps
> 1000, 2000, and 3000. There are many rows in a particular partition, making
> filtering on the min and max clustering effectively a no-op. All data is
> inserted, and there are no partial updates. A fully specified row with
> timestamp 2500 exists in the SSTable with a max timestamp of 3000. With a
> proper row timestamp in hand, we can easily ignore the SSTables w/ max
> timestamps of 1000 and 2000. Without it, we read 3 SSTables instead of 1,
> which likely means a significant performance regression.
> The following test illustrates this difference in behavior between 2.1 and
> 3.0:
> https://github.com/maedhroz/cassandra/commit/84ce9242bedd735ca79d4f06007d127de6a82800
> A solution here might be as simple as having
> {{SinglePartitionReadCommand#canRemoveRow()}} only inspect primary key
> liveness information for non-compact/CQL tables. Tombstones seem to be
> handled at a level above that anyway. (One potential problem with that is
> whether or not the distinction will continue to exist in 4.0, and dropping
> compact storage from a table doesn't magically make pk liveness information
> appear.)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16144) TLS connections to the storage port on a node without server encryption configured causes java.io.IOException accessing missing keystore
[
https://issues.apache.org/jira/browse/CASSANDRA-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221618#comment-17221618
]
David Capwell commented on CASSANDRA-16144:
---
Starting commit
CI Results (pending):
Branch: cassandra-2.2
Source:
https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-cassandra-2.2-6F583124-6A59-40FF-832F-27F6AC26F5FD
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-cassandra-2.2-6F583124-6A59-40FF-832F-27F6AC26F5FD
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/136/
Branch: cassandra-3.0
Source:
https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-cassandra-3.0-6F583124-6A59-40FF-832F-27F6AC26F5FD
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-cassandra-3.0-6F583124-6A59-40FF-832F-27F6AC26F5FD
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/137/
Branch: cassandra-3.11
Source:
https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-cassandra-3.11-6F583124-6A59-40FF-832F-27F6AC26F5FD
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-cassandra-3.11-6F583124-6A59-40FF-832F-27F6AC26F5FD
Jenkins: unknown
Branch: trunk
Source:
https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16144-trunk-6F583124-6A59-40FF-832F-27F6AC26F5FD
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16144-trunk-6F583124-6A59-40FF-832F-27F6AC26F5FD
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/139/
> TLS connections to the storage port on a node without server encryption
> configured causes java.io.IOException accessing missing keystore
>
>
> Key: CASSANDRA-16144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16144
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> If a TLS connection is requested against a node with all encryption disabled
> by configuration,
> configured with
> {code}
> server_encryption_options: {optional:false, internode_encryption: none}
> {code}
> it logs the following error if no keystore exists for the node.
> {code}
> INFO [Messaging-EventLoop-3-3] 2020-09-15T14:30:02,952 : -
> 127.0.0.1:7000->127.0.1.1:7000-URGENT_MESSAGES-[no-channel] failed to connect
> io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection
> refused: local1-i1/127.0.1.1:7000
> Caused by: java.net.ConnectException: Connection refused
>at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?]
>at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779) ~[?:?]
>at
> io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:702)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at java.lang.Thread.run(Thread.java:834) [?:?]
> WARN [Messaging-EventLoop-3-9] 2020-09-15T14:30:06,375 : - Failed to
> initialize a channel. Closing: [id: 0x0746c157, L:/127.0.0.1:7000 -
> R:/127.0.0.1:59623]
> java.io.IOException: failed to build trust manager store for secure
> connections
>at
> org.apache.cassandra.security.SSLFactory.buildKeyManagerFactory(SSLFactory.java:232)
>
[jira] [Created] (CASSANDRA-16229) Flaky jvm-dtest: org.apache.cassandra.distributed.test.ring.NodeNotInRingTest.nodeNotInRingTest
Yifan Cai created CASSANDRA-16229:
-
Summary: Flaky jvm-dtest:
org.apache.cassandra.distributed.test.ring.NodeNotInRingTest.nodeNotInRingTest
Key: CASSANDRA-16229
URL: https://issues.apache.org/jira/browse/CASSANDRA-16229
Project: Cassandra
Issue Type: Bug
Components: Test/dtest/java
Reporter: Yifan Cai
Test failed during CI build. Stack trace attached below,
[https://app.circleci.com/pipelines/github/yifan-c/cassandra/135/workflows/371dee8e-36d2-41a1-b27f-f7578496a1fe/jobs/681]
{code:java}
Testcase:
nodeNotInRingTest(org.apache.cassandra.distributed.test.ring.NodeNotInRingTest):
FAILED
expected:<86> but was:<100>
junit.framework.AssertionFailedError: expected:<86> but was:<100>
at
org.apache.cassandra.distributed.test.ring.NodeNotInRingTest.nodeNotInRingTest(NodeNotInRingTest.java:60)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43){code}
Besides the flakiness, the expected and the actual values are swapped in the
assertion statement.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15663) DESCRIBE KEYSPACE does not properly quote table names
[
https://issues.apache.org/jira/browse/CASSANDRA-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221615#comment-17221615
]
Alan Boudreault commented on CASSANDRA-15663:
-
[~adelapena] This will be fixed for the next release:
https://datastax-oss.atlassian.net/browse/PYTHON-1269.
> DESCRIBE KEYSPACE does not properly quote table names
> -
>
> Key: CASSANDRA-15663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15663
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Syntax
>Reporter: Oskar Liljeblad
>Assignee: Aleksandr Sorokoumov
>Priority: Normal
> Labels: pull-request-available
> Fix For: 3.11.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> How to reproduce (3.11.6) - cqlsh:
> {code}
> CREATE KEYSPACE test1 WITH replication = \{'class': 'SimpleStrategy',
> 'replication_factor': '1'} AND durable_writes = true;
> CREATE TABLE test1."default" (id text PRIMARY KEY, data text, etag text);
> DESCRIBE KEYSPACE test1;
> {code}
> Output will be:
> {code}
> CREATE TABLE test1.default (
> id text PRIMARY KEY,
> data text,
> etag text
> ) WITH [..]
> {code}
> Output should be:
> {code}
> CREATE TABLE test1."default" (
> id text PRIMARY KEY,
> data text,
> etag text
> ) WITH [..]
> {code}
> If you try to run {{CREATE TABLE test1.default [..]}} you will get an error
> SyntaxException: line 1:19 no viable alternative at input 'default' (CREATE
> TABLE test1.[default]...)
> Oskar Liljeblad
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16226) COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by timestamp due to missing primary key liveness info
[
https://issues.apache.org/jira/browse/CASSANDRA-16226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221596#comment-17221596
]
Kornel Pal commented on CASSANDRA-16226:
Thank you [~maedhroz] for the details, now I understand that this is not an
upgrade issue. I don't have any more concerns about your patch, and thank you
for fixing this performance issue.
> COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by
> timestamp due to missing primary key liveness info
> ---
>
> Key: CASSANDRA-16226
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16226
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: perfomance, upgrade
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This was discovered while tracking down a spike in the number of SSTables
> per read for a COMPACT STORAGE table after a 2.1 -> 3.0 upgrade. Before 3.0,
> there is no direct analog of 3.0's primary key liveness info. When we upgrade
> 2.1 COMPACT STORAGE SSTables to the mf format, we simply don't write row
> timestamps, even if the original mutations were INSERTs. On read, when we
> look at SSTables in order from newest to oldest max timestamp, we expect to
> have this primary key liveness information to determine whether we can skip
> older SSTables after finding completely populated rows.
> ex. I have three SSTables in a COMPACT STORAGE table with max timestamps
> 1000, 2000, and 3000. There are many rows in a particular partition, making
> filtering on the min and max clustering effectively a no-op. All data is
> inserted, and there are no partial updates. A fully specified row with
> timestamp 2500 exists in the SSTable with a max timestamp of 3000. With a
> proper row timestamp in hand, we can easily ignore the SSTables w/ max
> timestamps of 1000 and 2000. Without it, we read 3 SSTables instead of 1,
> which likely means a significant performance regression.
> The following test illustrates this difference in behavior between 2.1 and
> 3.0:
> https://github.com/maedhroz/cassandra/commit/84ce9242bedd735ca79d4f06007d127de6a82800
> A solution here might be as simple as having
> {{SinglePartitionReadCommand#canRemoveRow()}} only inspect primary key
> liveness information for non-compact/CQL tables. Tombstones seem to be
> handled at a level above that anyway. (One potential problem with that is
> whether or not the distinction will continue to exist in 4.0, and dropping
> compact storage from a table doesn't magically make pk liveness information
> appear.)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16226) COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by timestamp due to missing primary key liveness info
[
https://issues.apache.org/jira/browse/CASSANDRA-16226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221593#comment-17221593
]
Caleb Rackliffe commented on CASSANDRA-16226:
-
There is a problem the current patch does not address, and that's dropping
compact storage when you've already written SSTables. {{DROP COMPACT STORAGE}}
doesn't leave any indication on individual SSTables as to whether the table
overall was COMPACT when they were written. Therefore, {{isCQLTable()}} can be
true after the drop, and we're back in the same situation. Also,
{{upgradesstables}} has to be run before {{DROP COMPACT STORAGE}} can be run,
so there is no source of liveness information either way. The only thing that
seems to reintroduce liveness information post-drop is compaction.
> COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by
> timestamp due to missing primary key liveness info
> ---
>
> Key: CASSANDRA-16226
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16226
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: perfomance, upgrade
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This was discovered while tracking down a spike in the number of SSTables
> per read for a COMPACT STORAGE table after a 2.1 -> 3.0 upgrade. Before 3.0,
> there is no direct analog of 3.0's primary key liveness info. When we upgrade
> 2.1 COMPACT STORAGE SSTables to the mf format, we simply don't write row
> timestamps, even if the original mutations were INSERTs. On read, when we
> look at SSTables in order from newest to oldest max timestamp, we expect to
> have this primary key liveness information to determine whether we can skip
> older SSTables after finding completely populated rows.
> ex. I have three SSTables in a COMPACT STORAGE table with max timestamps
> 1000, 2000, and 3000. There are many rows in a particular partition, making
> filtering on the min and max clustering effectively a no-op. All data is
> inserted, and there are no partial updates. A fully specified row with
> timestamp 2500 exists in the SSTable with a max timestamp of 3000. With a
> proper row timestamp in hand, we can easily ignore the SSTables w/ max
> timestamps of 1000 and 2000. Without it, we read 3 SSTables instead of 1,
> which likely means a significant performance regression.
> The following test illustrates this difference in behavior between 2.1 and
> 3.0:
> https://github.com/maedhroz/cassandra/commit/84ce9242bedd735ca79d4f06007d127de6a82800
> A solution here might be as simple as having
> {{SinglePartitionReadCommand#canRemoveRow()}} only inspect primary key
> liveness information for non-compact/CQL tables. Tombstones seem to be
> handled at a level above that anyway. (One potential problem with that is
> whether or not the distinction will continue to exist in 4.0, and dropping
> compact storage from a table doesn't magically make pk liveness information
> appear.)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16217) Minimal 4.0 COMPACT STORAGE backport
[ https://issues.apache.org/jira/browse/CASSANDRA-16217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-16217: Description: There are several behavioural changes related to compact storage, and these differences are larger than most of us have anticipated: we first thought there’ll be that “appearing column”, but there’s implicit nulls in clusterings thing, and row vs column deletion. Some of the recent issues on the subject are: CASSANDRA-16048, which allows to ignore these differences. The other one was trying to improve user experience of anyone still using compact storage: CASSANDRA-15811. Easily reproducible differernces are: (1) hidden columns show up, which breaks SELECT * queries (2) DELETE v and UPDATE v WITH TTL would result into row removals in non-dense compact tables (CASSANDRA-16069) (3) INSERT allows skipping clusterings, which are filled with nulls by default. Some of these are tricky to support, as 15811 has shown. Anyone on OSS side who might want to upgrade to 4.0 while still using compact storage might be affected by being forced into one of these behaviours. Possible solutions are to document these behaviours, or to bring back a minimal set of COMPACT STORAGE to keep supporting these. It looks like it is possible to leave some of the functionality related to DENSE flag and allow it to be present in 4.0, but only for these three (and potential related, however not direrclty visible) cases. [~e.dimitrova] since you were working on removal on compact storage, wanted to reassure that this is not a revert of your patch. On contrary: your patch was instrumental in identifying the right places. cc [~slebresne] [~aleksey] [~benedict] [~marcuse] |[patch|https://github.com/apache/cassandra/pull/785]|[ci|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=13994-followup]| was: There are several behavioural changes related to compact storage, and these differences are larger than most of us have anticipated: we first thought there’ll be that “appearing column”, but there’s implicit nulls in clusterings thing, and row vs column deletion. Some of the recent issues on the subject are: CASSANDRA-16048, which allows to ignore these differences. The other one was trying to improve user experience of anyone still using compact storage: CASSANDRA-15811. Easily reproducible differernces are: (1) hidden columns show up, which breaks SELECT * queries (2) DELETE v and UPDATE v WITH TTL would result into row removals in non-dense compact tables (CASSANDRA-16069) (3) INSERT allows skipping clusterings, which are filled with nulls by default. Some of these are tricky to support, as 15811 has shown. Anyone on OSS side who might want to upgrade to 4.0 while still using compact storage might be affected by being forced into one of these behaviours. Possible solutions are to document these behaviours, or to bring back a minimal set of COMPACT STORAGE to keep supporting these. It looks like it is possible to leave some of the functionality related to DENSE flag and allow it to be present in 4.0, but only for these three (and potential related, however not direrclty visible) cases. [~e.dimitrova] since you were working on removal on compact storage, wanted to reassure that this is not a revert of your patch. On contrary: your patch was instrumental in identifying the right places. cc [~slebresne] [~aleksey] [~benedict] [~marcuse] |[patch|https://github.com/apache/cassandra/pull/785]|[cihttps://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=13994-followup]| > Minimal 4.0 COMPACT STORAGE backport > > > Key: CASSANDRA-16217 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16217 > Project: Cassandra > Issue Type: Bug > Components: Legacy/CQL >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > > There are several behavioural changes related to compact storage, and these > differences are larger than most of us have anticipated: we first thought > there’ll be that “appearing column”, but there’s implicit nulls in > clusterings thing, and row vs column deletion. > Some of the recent issues on the subject are: CASSANDRA-16048, which allows > to ignore these differences. The other one was trying to improve user > experience of anyone still using compact storage: CASSANDRA-15811. > Easily reproducible differernces are: > (1) hidden columns show up, which breaks SELECT * queries > (2) DELETE v and UPDATE v WITH TTL would result into row removals in > non-dense compact tables (CASSANDRA-16069) > (3) INSERT allows skipping clusterings, which are filled with nulls by > default. > Some of these are tricky to support, as 15811 has shown. Anyone on OSS side > who might want to upgrade to
[jira] [Updated] (CASSANDRA-16228) Fix race condition on ColumnFamilyStore and TableMetrics initialization that makes MetricListener to see an instance in an unknown state
[
https://issues.apache.org/jira/browse/CASSANDRA-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-16228:
Summary: Fix race condition on ColumnFamilyStore and TableMetrics
initialization that makes MetricListener to see an instance in an unknown state
(was: Fixing race condition on ColumnFamilyStore and TableMetrics
initialization that makes MetricListener to see an instance in an unknown state)
> Fix race condition on ColumnFamilyStore and TableMetrics initialization that
> makes MetricListener to see an instance in an unknown state
>
>
> Key: CASSANDRA-16228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16228
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> There is a dependency between TableMetrics, SecondaryIndexManager and Tracker
> which requires certain order of the operations in ColumnFamilyStore
> constructor.
> The following test will be added to ColumnFamilyMetricTest to show the issue:
> {code:java}
> @Test
> public void testStartupRaceConditionOnMetricListeners()
> {
> // Since the ColumnFamilyStore instance reference escapes during the
> construction
> // we have a race condition and listeners can see an instance that is in an
> unknown state.
> // This test just check that all callbacks can access the data without
> throwing any exception.
> registerMetricListener();
> SchemaLoader.createKeyspace("Keyspace2",
> KeyspaceParams.simple(1),
> SchemaLoader.standardCFMD("Keyspace2", "Standard2"));
> }
> private void registerMetricListener()
> {
> CassandraMetricsRegistry.Metrics.addListener(new
> MetricRegistryListener.Base()
> {
> @Override
> public void onGaugeAdded(String name, Gauge gauge)
> {
> gauge.getValue();
> }
> @Override
> public void onGaugeRemoved(String name)
> {
> }
> @Override
> public void onCounterAdded(String name, Counter counter)
> {
> counter.getCount();
> }
> @Override
> public void onCounterRemoved(String name)
> {
> }
> @Override
> public void onHistogramAdded(String name, Histogram histogram)
> {
> histogram.getCount();
> }
> @Override
> public void onHistogramRemoved(String name)
> {
> }
> @Override
> public void onMeterAdded(String name, Meter meter)
> {
> meter.getCount();
> }
> @Override
> public void onMeterRemoved(String name)
> {
> }
> @Override
> public void onTimerAdded(String name, Timer timer)
> {
> timer.getCount();
> }
> @Override
> public void onTimerRemoved(String name)
> {
> }
> });{code}
> *NOTE:* In order to be able to run the test and use registerMetricListener()
> method a small modification to requests in CacheMetrics is needed. This will
> be submitted and further explained with a coming patch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16228) Fix race condition on ColumnFamilyStore and TableMetrics initialization that makes MetricListener to see an instance in an unknown state
[
https://issues.apache.org/jira/browse/CASSANDRA-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-16228:
Fix Version/s: (was: 3.11.x)
> Fix race condition on ColumnFamilyStore and TableMetrics initialization that
> makes MetricListener to see an instance in an unknown state
>
>
> Key: CASSANDRA-16228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16228
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
>
> There is a dependency between TableMetrics, SecondaryIndexManager and Tracker
> which requires certain order of the operations in ColumnFamilyStore
> constructor.
> The following test will be added to ColumnFamilyMetricTest to show the issue:
> {code:java}
> @Test
> public void testStartupRaceConditionOnMetricListeners()
> {
> // Since the ColumnFamilyStore instance reference escapes during the
> construction
> // we have a race condition and listeners can see an instance that is in an
> unknown state.
> // This test just check that all callbacks can access the data without
> throwing any exception.
> registerMetricListener();
> SchemaLoader.createKeyspace("Keyspace2",
> KeyspaceParams.simple(1),
> SchemaLoader.standardCFMD("Keyspace2", "Standard2"));
> }
> private void registerMetricListener()
> {
> CassandraMetricsRegistry.Metrics.addListener(new
> MetricRegistryListener.Base()
> {
> @Override
> public void onGaugeAdded(String name, Gauge gauge)
> {
> gauge.getValue();
> }
> @Override
> public void onGaugeRemoved(String name)
> {
> }
> @Override
> public void onCounterAdded(String name, Counter counter)
> {
> counter.getCount();
> }
> @Override
> public void onCounterRemoved(String name)
> {
> }
> @Override
> public void onHistogramAdded(String name, Histogram histogram)
> {
> histogram.getCount();
> }
> @Override
> public void onHistogramRemoved(String name)
> {
> }
> @Override
> public void onMeterAdded(String name, Meter meter)
> {
> meter.getCount();
> }
> @Override
> public void onMeterRemoved(String name)
> {
> }
> @Override
> public void onTimerAdded(String name, Timer timer)
> {
> timer.getCount();
> }
> @Override
> public void onTimerRemoved(String name)
> {
> }
> });{code}
> *NOTE:* In order to be able to run the test and use registerMetricListener()
> method a small modification to requests in CacheMetrics is needed. This will
> be submitted and further explained with a coming patch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16226) COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by timestamp due to missing primary key liveness info
[
https://issues.apache.org/jira/browse/CASSANDRA-16226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221547#comment-17221547
]
Caleb Rackliffe commented on CASSANDRA-16226:
-
I do need to test this w/ {{DROP COMPACT STORAGE}} though...
> COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by
> timestamp due to missing primary key liveness info
> ---
>
> Key: CASSANDRA-16226
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16226
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: perfomance, upgrade
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This was discovered while tracking down a spike in the number of SSTables
> per read for a COMPACT STORAGE table after a 2.1 -> 3.0 upgrade. Before 3.0,
> there is no direct analog of 3.0's primary key liveness info. When we upgrade
> 2.1 COMPACT STORAGE SSTables to the mf format, we simply don't write row
> timestamps, even if the original mutations were INSERTs. On read, when we
> look at SSTables in order from newest to oldest max timestamp, we expect to
> have this primary key liveness information to determine whether we can skip
> older SSTables after finding completely populated rows.
> ex. I have three SSTables in a COMPACT STORAGE table with max timestamps
> 1000, 2000, and 3000. There are many rows in a particular partition, making
> filtering on the min and max clustering effectively a no-op. All data is
> inserted, and there are no partial updates. A fully specified row with
> timestamp 2500 exists in the SSTable with a max timestamp of 3000. With a
> proper row timestamp in hand, we can easily ignore the SSTables w/ max
> timestamps of 1000 and 2000. Without it, we read 3 SSTables instead of 1,
> which likely means a significant performance regression.
> The following test illustrates this difference in behavior between 2.1 and
> 3.0:
> https://github.com/maedhroz/cassandra/commit/84ce9242bedd735ca79d4f06007d127de6a82800
> A solution here might be as simple as having
> {{SinglePartitionReadCommand#canRemoveRow()}} only inspect primary key
> liveness information for non-compact/CQL tables. Tombstones seem to be
> handled at a level above that anyway. (One potential problem with that is
> whether or not the distinction will continue to exist in 4.0, and dropping
> compact storage from a table doesn't magically make pk liveness information
> appear.)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16226) COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by timestamp due to missing primary key liveness info
[
https://issues.apache.org/jira/browse/CASSANDRA-16226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221542#comment-17221542
]
Caleb Rackliffe edited comment on CASSANDRA-16226 at 10/27/20, 4:16 PM:
Hi [~kornelpal]. Take a look at {{UpdateStatement#addUpdateForKey()}}...
{noformat}
// We update the row timestamp (ex-row marker) only on INSERT (#6782)
// Further, COMPACT tables semantic differs from "CQL3" ones in that a row
exists only if it has
// a non-null column, so we don't want to set the row timestamp for them.
if (type.isInsert() && cfm.isCQLTable())
params.addPrimaryKeyLivenessInfo();
{noformat}
...and {{LegacyLayout}}...
{noformat}
else if (column.isPrimaryKeyColumn() && metadata.isCQLTable())
{noformat}
COMPACT tables will never have primary key liveness info, even if those tables
are created in 3.0+, so running {{upgradesstables}} doesn't help (at least as
far as I can tell). The patch I've posted simply restores the way this
optimization worked for COMPACT tables before the 3.0 storage engine rewrite.
was (Author: maedhroz):
Hi [~kornelpal]. Take a look at {{UpdateStatement#addUpdateForKey()}}...
{noformat}
// We update the row timestamp (ex-row marker) only on INSERT (#6782)
// Further, COMPACT tables semantic differs from "CQL3" ones in that a row
exists only if it has
// a non-null column, so we don't want to set the row timestamp for them.
if (type.isInsert() && cfm.isCQLTable())
params.addPrimaryKeyLivenessInfo();
{noformat}
COMPACT tables will never have primary key liveness info, even if those tables
are created in 3.0+, so running {{upgradesstables}} doesn't help (at least as
far as I can tell). The patch I've posted simply restores the way this
optimization worked for COMPACT tables before the 3.0 storage engine rewrite.
> COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by
> timestamp due to missing primary key liveness info
> ---
>
> Key: CASSANDRA-16226
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16226
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: perfomance, upgrade
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This was discovered while tracking down a spike in the number of SSTables
> per read for a COMPACT STORAGE table after a 2.1 -> 3.0 upgrade. Before 3.0,
> there is no direct analog of 3.0's primary key liveness info. When we upgrade
> 2.1 COMPACT STORAGE SSTables to the mf format, we simply don't write row
> timestamps, even if the original mutations were INSERTs. On read, when we
> look at SSTables in order from newest to oldest max timestamp, we expect to
> have this primary key liveness information to determine whether we can skip
> older SSTables after finding completely populated rows.
> ex. I have three SSTables in a COMPACT STORAGE table with max timestamps
> 1000, 2000, and 3000. There are many rows in a particular partition, making
> filtering on the min and max clustering effectively a no-op. All data is
> inserted, and there are no partial updates. A fully specified row with
> timestamp 2500 exists in the SSTable with a max timestamp of 3000. With a
> proper row timestamp in hand, we can easily ignore the SSTables w/ max
> timestamps of 1000 and 2000. Without it, we read 3 SSTables instead of 1,
> which likely means a significant performance regression.
> The following test illustrates this difference in behavior between 2.1 and
> 3.0:
> https://github.com/maedhroz/cassandra/commit/84ce9242bedd735ca79d4f06007d127de6a82800
> A solution here might be as simple as having
> {{SinglePartitionReadCommand#canRemoveRow()}} only inspect primary key
> liveness information for non-compact/CQL tables. Tombstones seem to be
> handled at a level above that anyway. (One potential problem with that is
> whether or not the distinction will continue to exist in 4.0, and dropping
> compact storage from a table doesn't magically make pk liveness information
> appear.)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16226) COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by timestamp due to missing primary key liveness info
[
https://issues.apache.org/jira/browse/CASSANDRA-16226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221542#comment-17221542
]
Caleb Rackliffe commented on CASSANDRA-16226:
-
Hi [~kornelpal]. Take a look at {{UpdateStatement#addUpdateForKey()}}...
{noformat}
// We update the row timestamp (ex-row marker) only on INSERT (#6782)
// Further, COMPACT tables semantic differs from "CQL3" ones in that a row
exists only if it has
// a non-null column, so we don't want to set the row timestamp for them.
if (type.isInsert() && cfm.isCQLTable())
params.addPrimaryKeyLivenessInfo();
{noformat}
COMPACT tables will never have primary key liveness info, even if those tables
are created in 3.0+, so running {{upgradesstables}} doesn't help (at least as
far as I can tell). The patch I've posted simply restores the way this
optimization worked for COMPACT tables before the 3.0 storage engine rewrite.
> COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by
> timestamp due to missing primary key liveness info
> ---
>
> Key: CASSANDRA-16226
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16226
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: perfomance, upgrade
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This was discovered while tracking down a spike in the number of SSTables
> per read for a COMPACT STORAGE table after a 2.1 -> 3.0 upgrade. Before 3.0,
> there is no direct analog of 3.0's primary key liveness info. When we upgrade
> 2.1 COMPACT STORAGE SSTables to the mf format, we simply don't write row
> timestamps, even if the original mutations were INSERTs. On read, when we
> look at SSTables in order from newest to oldest max timestamp, we expect to
> have this primary key liveness information to determine whether we can skip
> older SSTables after finding completely populated rows.
> ex. I have three SSTables in a COMPACT STORAGE table with max timestamps
> 1000, 2000, and 3000. There are many rows in a particular partition, making
> filtering on the min and max clustering effectively a no-op. All data is
> inserted, and there are no partial updates. A fully specified row with
> timestamp 2500 exists in the SSTable with a max timestamp of 3000. With a
> proper row timestamp in hand, we can easily ignore the SSTables w/ max
> timestamps of 1000 and 2000. Without it, we read 3 SSTables instead of 1,
> which likely means a significant performance regression.
> The following test illustrates this difference in behavior between 2.1 and
> 3.0:
> https://github.com/maedhroz/cassandra/commit/84ce9242bedd735ca79d4f06007d127de6a82800
> A solution here might be as simple as having
> {{SinglePartitionReadCommand#canRemoveRow()}} only inspect primary key
> liveness information for non-compact/CQL tables. Tombstones seem to be
> handled at a level above that anyway. (One potential problem with that is
> whether or not the distinction will continue to exist in 4.0, and dropping
> compact storage from a table doesn't magically make pk liveness information
> appear.)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16021) auditlogviewer unit test hardening and docs update
[
https://issues.apache.org/jira/browse/CASSANDRA-16021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fábio Takeo Ueno updated CASSANDRA-16021:
-
Fix Version/s: 4.0-beta
> auditlogviewer unit test hardening and docs update
> --
>
> Key: CASSANDRA-16021
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16021
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/auditlogging
>Reporter: Berenguer Blasi
>Assignee: Fábio Takeo Ueno
>Priority: Normal
> Fix For: 4.0-beta
>
>
> During CASSANDRA-15883 / CASSANDRA-15991 it was detected unit test coverage
> for this tool is minimal. There is a unit test to enhance upon under
> {{test/unit/org/apache/cassandra/tools}}. Also docs need updating to reflect
> all available options
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14013) Data loss in snapshots keyspace after service restart
[
https://issues.apache.org/jira/browse/CASSANDRA-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic reassigned CASSANDRA-14013:
-
Assignee: Stefan Miklosovic (was: Vincent White)
> Data loss in snapshots keyspace after service restart
> -
>
> Key: CASSANDRA-14013
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14013
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core
>Reporter: Gregor Uhlenheuer
>Assignee: Stefan Miklosovic
>Priority: Normal
>
> I am posting this bug in hope to discover the stupid mistake I am doing
> because I can't imagine a reasonable answer for the behavior I see right now
> :-)
> In short words, I do observe data loss in a keyspace called *snapshots* after
> restarting the Cassandra service. Say I do have 1000 records in a table
> called *snapshots.test_idx* then after restart the table has less entries or
> is even empty.
> My kind of "mysterious" observation is that it happens only in a keyspace
> called *snapshots*...
> h3. Steps to reproduce
> These steps to reproduce show the described behavior in "most" attempts (not
> every single time though).
> {code}
> # create keyspace
> CREATE KEYSPACE snapshots WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> # create table
> CREATE TABLE snapshots.test_idx (key text, seqno bigint, primary key(key));
> # insert some test data
> INSERT INTO snapshots.test_idx (key,seqno) values ('key1', 1);
> ...
> INSERT INTO snapshots.test_idx (key,seqno) values ('key1000', 1000);
> # count entries
> SELECT count(*) FROM snapshots.test_idx;
> 1000
> # restart service
> kill
> cassandra -f
> # count entries
> SELECT count(*) FROM snapshots.test_idx;
> 0
> {code}
> I hope someone can point me to the obvious mistake I am doing :-)
> This happened to me using both Cassandra 3.9 and 3.11.0
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16021) auditlogviewer unit test hardening and docs update
[
https://issues.apache.org/jira/browse/CASSANDRA-16021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fábio Takeo Ueno reassigned CASSANDRA-16021:
Assignee: Fábio Takeo Ueno
> auditlogviewer unit test hardening and docs update
> --
>
> Key: CASSANDRA-16021
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16021
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/auditlogging
>Reporter: Berenguer Blasi
>Assignee: Fábio Takeo Ueno
>Priority: Normal
>
> During CASSANDRA-15883 / CASSANDRA-15991 it was detected unit test coverage
> for this tool is minimal. There is a unit test to enhance upon under
> {{test/unit/org/apache/cassandra/tools}}. Also docs need updating to reflect
> all available options
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16021) auditlogviewer unit test hardening and docs update
[
https://issues.apache.org/jira/browse/CASSANDRA-16021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-16021:
---
Bug Category: Parent values: Correctness(12982)
Complexity: Low Hanging Fruit
Component/s: Tool/auditlogging
Discovered By: Code Inspection
Severity: Low
Status: Open (was: Triage Needed)
> auditlogviewer unit test hardening and docs update
> --
>
> Key: CASSANDRA-16021
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16021
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/auditlogging
>Reporter: Berenguer Blasi
>Priority: Normal
>
> During CASSANDRA-15883 / CASSANDRA-15991 it was detected unit test coverage
> for this tool is minimal. There is a unit test to enhance upon under
> {{test/unit/org/apache/cassandra/tools}}. Also docs need updating to reflect
> all available options
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16010) sstableutil unit test hardening
[
https://issues.apache.org/jira/browse/CASSANDRA-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-16010:
---
Bug Category: Parent values: Correctness(12982)
Complexity: Low Hanging Fruit
Discovered By: Code Inspection
Fix Version/s: 4.0-beta
Severity: Low
Status: Open (was: Triage Needed)
> sstableutil unit test hardening
> ---
>
> Key: CASSANDRA-16010
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16010
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/sstable
>Reporter: Berenguer Blasi
>Assignee: Fábio Takeo Ueno
>Priority: Normal
> Labels: low-hanging-fruit
> Fix For: 4.0-beta
>
>
> During CASSANDRA-15883 / CASSANDRA-15991 it was detected unit test coverage
> for this tool is minimal. There is a unit test to enhance upon under
> {{test/unit/org/apache/cassandra/tools}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16191) Add tests for Repair metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-16191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fábio Takeo Ueno reassigned CASSANDRA-16191:
Assignee: (was: Fábio Takeo Ueno)
> Add tests for Repair metrics
>
>
> Key: CASSANDRA-16191
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16191
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest/java
>Reporter: Benjamin Lerer
>Priority: Normal
> Fix For: 4.0-beta
>
>
> We do not seems to have some tests for the {{RepairMetrics.previewFailures}}
> counter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16010) sstableutil unit test hardening
[
https://issues.apache.org/jira/browse/CASSANDRA-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fábio Takeo Ueno reassigned CASSANDRA-16010:
Assignee: Fábio Takeo Ueno
> sstableutil unit test hardening
> ---
>
> Key: CASSANDRA-16010
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16010
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/sstable
>Reporter: Berenguer Blasi
>Assignee: Fábio Takeo Ueno
>Priority: Normal
> Labels: low-hanging-fruit
>
> During CASSANDRA-15883 / CASSANDRA-15991 it was detected unit test coverage
> for this tool is minimal. There is a unit test to enhance upon under
> {{test/unit/org/apache/cassandra/tools}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16227) Nodetool Netstats and tablestats tests
[ https://issues.apache.org/jira/browse/CASSANDRA-16227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-16227: -- Reviewers: Andres de la Peña, Andres de la Peña (was: Andres de la Peña) Andres de la Peña, Andres de la Peña (was: Andres de la Peña) Status: Review In Progress (was: Patch Available) > Nodetool Netstats and tablestats tests > -- > > Key: CASSANDRA-16227 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16227 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0 > > Time Spent: 20m > Remaining Estimate: 0h > > As part of the CASSANDRA-15583 effort we're adding unit tests for netstats > and tablestats -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16228) Fixing race condition on ColumnFamilyStore and TableMetrics initialization that makes MetricListener to see an instance in an unknown state
[
https://issues.apache.org/jira/browse/CASSANDRA-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-16228:
Description:
There is a dependency between TableMetrics, SecondaryIndexManager and Tracker
which requires certain order of the operations in ColumnFamilyStore
constructor.
The following test will be added to ColumnFamilyMetricTest to show the issue:
{code:java}
@Test
public void testStartupRaceConditionOnMetricListeners()
{
// Since the ColumnFamilyStore instance reference escapes during the
construction
// we have a race condition and listeners can see an instance that is in an
unknown state.
// This test just check that all callbacks can access the data without
throwing any exception.
registerMetricListener();
SchemaLoader.createKeyspace("Keyspace2",
KeyspaceParams.simple(1),
SchemaLoader.standardCFMD("Keyspace2", "Standard2"));
}
private void registerMetricListener()
{
CassandraMetricsRegistry.Metrics.addListener(new MetricRegistryListener.Base()
{
@Override
public void onGaugeAdded(String name, Gauge gauge)
{
gauge.getValue();
}
@Override
public void onGaugeRemoved(String name)
{
}
@Override
public void onCounterAdded(String name, Counter counter)
{
counter.getCount();
}
@Override
public void onCounterRemoved(String name)
{
}
@Override
public void onHistogramAdded(String name, Histogram histogram)
{
histogram.getCount();
}
@Override
public void onHistogramRemoved(String name)
{
}
@Override
public void onMeterAdded(String name, Meter meter)
{
meter.getCount();
}
@Override
public void onMeterRemoved(String name)
{
}
@Override
public void onTimerAdded(String name, Timer timer)
{
timer.getCount();
}
@Override
public void onTimerRemoved(String name)
{
}
});{code}
*NOTE:* In order to be able to run the test and use registerMetricListener()
method a small modification to requests in CacheMetrics is needed. This will be
submitted and further explained with a coming patch.
was:
There is a dependency between TableMetrics, SecondaryIndexManager and Tracker
which requires certain order of the operations in ColumnFamilyStore
constructor.
The following test will be added to ColumnFamilyMetricTest to show the issue:
{code:java}
@Test
public void testStartupRaceConditionOnMetricListeners()
{
// Since the ColumnFamilyStore instance reference escapes during the
construction
// we have a race condition and listeners can see an instance that is in an
unknown state
// this test just check that all callbacks can access the data without
throwing any exception.
registerMetricListener();
SchemaLoader.createKeyspace("Keyspace2",
KeyspaceParams.simple(1),
SchemaLoader.standardCFMD("Keyspace2", "Standard2"));
}
private void registerMetricListener()
{
CassandraMetricsRegistry.Metrics.addListener(new MetricRegistryListener.Base()
{
@Override
public void onGaugeAdded(String name, Gauge gauge)
{
gauge.getValue();
}
@Override
public void onGaugeRemoved(String name)
{
}
@Override
public void onCounterAdded(String name, Counter counter)
{
counter.getCount();
}
@Override
public void onCounterRemoved(String name)
{
}
@Override
public void onHistogramAdded(String name, Histogram histogram)
{
histogram.getCount();
}
@Override
public void onHistogramRemoved(String name)
{
}
@Override
public void onMeterAdded(String name, Meter meter)
{
meter.getCount();
}
@Override
public void onMeterRemoved(String name)
{
}
@Override
public void onTimerAdded(String name, Timer timer)
{
timer.getCount();
}
@Override
public void onTimerRemoved(String name)
{
}
});{code}
*NOTE:* In order to be able to run the test and use registerMetricListener()
method a small modification to requests in CacheMetrics is needed. This will be
submitted and further explained with a coming patch.
> Fixing race condition on ColumnFamilyStore and TableMetrics initialization
> that makes MetricListener to see an instance in an unknown state
> ---
>
> Key: CASSANDRA-16228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16228
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> There is a dependency between TableMetrics, SecondaryIndexManager and Tracker
> which requires certain order of the operations in ColumnFamilyStore
> constructor.
> The following test will be added to ColumnFamilyMetricTest to show the issue:
> {code:java}
> @Test
> public void testStartupRaceConditionOnMetricListeners()
> {
> // Since the
[jira] [Updated] (CASSANDRA-16228) Fixing race condition on ColumnFamilyStore and TableMetrics initialization that makes MetricListener to see an instance in an unknown state
[
https://issues.apache.org/jira/browse/CASSANDRA-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-16228:
Description:
There is a dependency between TableMetrics, SecondaryIndexManager and Tracker
which requires certain order of the operations in ColumnFamilyStore
constructor.
The following test will be added to ColumnFamilyMetricTest to show the issue:
{code:java}
@Test
public void testStartupRaceConditionOnMetricListeners()
{
// Since the ColumnFamilyStore instance reference escapes during the
construction
// we have a race condition and listeners can see an instance that is in an
unknown state
// this test just check that all callbacks can access the data without
throwing any exception.
registerMetricListener();
SchemaLoader.createKeyspace("Keyspace2",
KeyspaceParams.simple(1),
SchemaLoader.standardCFMD("Keyspace2", "Standard2"));
}
private void registerMetricListener()
{
CassandraMetricsRegistry.Metrics.addListener(new MetricRegistryListener.Base()
{
@Override
public void onGaugeAdded(String name, Gauge gauge)
{
gauge.getValue();
}
@Override
public void onGaugeRemoved(String name)
{
}
@Override
public void onCounterAdded(String name, Counter counter)
{
counter.getCount();
}
@Override
public void onCounterRemoved(String name)
{
}
@Override
public void onHistogramAdded(String name, Histogram histogram)
{
histogram.getCount();
}
@Override
public void onHistogramRemoved(String name)
{
}
@Override
public void onMeterAdded(String name, Meter meter)
{
meter.getCount();
}
@Override
public void onMeterRemoved(String name)
{
}
@Override
public void onTimerAdded(String name, Timer timer)
{
timer.getCount();
}
@Override
public void onTimerRemoved(String name)
{
}
});{code}
*NOTE:* In order to be able to run the test and use registerMetricListener()
method a small modification to requests in CacheMetrics is needed. This will be
submitted and further explained with a coming patch.
was:
There is a dependency between TableMetrics, SecondaryIndexManager and Tracker
which requires certain order of the operations in ColumnFamilyStore
constructor.
The following test will be added to ColumnFamilyMetricTest to show the issue:
{code:java}
@Test
public void testStartupRaceConditionOnMetricListeners()
{
// Since the ColumnFamilyStore instance reference escapes during the
construction
// we have a race condition and listeners can see an instance that is in an
unknown state
// this test just check that all callbacks can access the data without
throwing any exception.
registerMetricListener();
SchemaLoader.createKeyspace("Keyspace2",
KeyspaceParams.simple(1),
SchemaLoader.standardCFMD("Keyspace2", "Standard2"));
}
private void registerMetricListener()
{
CassandraMetricsRegistry.Metrics.addListener(new MetricRegistryListener.Base()
{
@Override
public void onGaugeAdded(String name, Gauge gauge)
{
gauge.getValue();
}
@Override
public void onGaugeRemoved(String name)
{
}
@Override
public void onCounterAdded(String name, Counter counter)
{
counter.getCount();
}
@Override
public void onCounterRemoved(String name)
{
}
@Override
public void onHistogramAdded(String name, Histogram histogram)
{
histogram.getCount();
}
@Override
public void onHistogramRemoved(String name)
{
}
@Override
public void onMeterAdded(String name, Meter meter)
{
meter.getCount();
}
@Override
public void onMeterRemoved(String name)
{
}
@Override
public void onTimerAdded(String name, Timer timer)
{
timer.getCount();
}
@Override
public void onTimerRemoved(String name)
{
}
});{code}
*NOTE:* In order to be able to run the test and use registerMetricListener a
small modification to requests in CacheMetrics is needed. This will be
submitted and further explained with a coming patch.
> Fixing race condition on ColumnFamilyStore and TableMetrics initialization
> that makes MetricListener to see an instance in an unknown state
> ---
>
> Key: CASSANDRA-16228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16228
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> There is a dependency between TableMetrics, SecondaryIndexManager and Tracker
> which requires certain order of the operations in ColumnFamilyStore
> constructor.
> The following test will be added to ColumnFamilyMetricTest to show the issue:
> {code:java}
> @Test
> public void testStartupRaceConditionOnMetricListeners()
> {
> // Since the
[jira] [Updated] (CASSANDRA-16228) Fixing race condition on ColumnFamilyStore and TableMetrics initialization that makes MetricListener to see an instance in an unknown state
[
https://issues.apache.org/jira/browse/CASSANDRA-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-16228:
Bug Category: Parent values: Correctness(12982)
Complexity: Normal
Component/s: Observability/Metrics
Discovered By: User Report
Severity: Normal
Assignee: Ekaterina Dimitrova
Status: Open (was: Triage Needed)
> Fixing race condition on ColumnFamilyStore and TableMetrics initialization
> that makes MetricListener to see an instance in an unknown state
> ---
>
> Key: CASSANDRA-16228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16228
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> There is a dependency between TableMetrics, SecondaryIndexManager and Tracker
> which requires certain order of the operations in ColumnFamilyStore
> constructor.
> The following test will be added to ColumnFamilyMetricTest to show the issue:
> {code:java}
> @Test
> public void testStartupRaceConditionOnMetricListeners()
> {
> // Since the ColumnFamilyStore instance reference escapes during the
> construction
> // we have a race condition and listeners can see an instance that is in an
> unknown state
> // this test just check that all callbacks can access the data without
> throwing any exception.
> registerMetricListener();
> SchemaLoader.createKeyspace("Keyspace2",
> KeyspaceParams.simple(1),
> SchemaLoader.standardCFMD("Keyspace2", "Standard2"));
> }
> private void registerMetricListener()
> {
> CassandraMetricsRegistry.Metrics.addListener(new
> MetricRegistryListener.Base()
> {
> @Override
> public void onGaugeAdded(String name, Gauge gauge)
> {
> gauge.getValue();
> }
> @Override
> public void onGaugeRemoved(String name)
> {
> }
> @Override
> public void onCounterAdded(String name, Counter counter)
> {
> counter.getCount();
> }
> @Override
> public void onCounterRemoved(String name)
> {
> }
> @Override
> public void onHistogramAdded(String name, Histogram histogram)
> {
> histogram.getCount();
> }
> @Override
> public void onHistogramRemoved(String name)
> {
> }
> @Override
> public void onMeterAdded(String name, Meter meter)
> {
> meter.getCount();
> }
> @Override
> public void onMeterRemoved(String name)
> {
> }
> @Override
> public void onTimerAdded(String name, Timer timer)
> {
> timer.getCount();
> }
> @Override
> public void onTimerRemoved(String name)
> {
> }
> });{code}
> *NOTE:* In order to be able to run the test and use registerMetricListener a
> small modification to requests in CacheMetrics is needed. This will be
> submitted and further explained with a coming patch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16228) Fixing race condition on ColumnFamilyStore and TableMetrics initialization that makes MetricListener to see an instance in an unknown state
[
https://issues.apache.org/jira/browse/CASSANDRA-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-16228:
Fix Version/s: 4.0-beta
3.11.x
> Fixing race condition on ColumnFamilyStore and TableMetrics initialization
> that makes MetricListener to see an instance in an unknown state
> ---
>
> Key: CASSANDRA-16228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16228
> Project: Cassandra
> Issue Type: Bug
>Reporter: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> There is a dependency between TableMetrics, SecondaryIndexManager and Tracker
> which requires certain order of the operations in ColumnFamilyStore
> constructor.
> The following test will be added to ColumnFamilyMetricTest to show the issue:
> {code:java}
> @Test
> public void testStartupRaceConditionOnMetricListeners()
> {
> // Since the ColumnFamilyStore instance reference escapes during the
> construction
> // we have a race condition and listeners can see an instance that is in an
> unknown state
> // this test just check that all callbacks can access the data without
> throwing any exception.
> registerMetricListener();
> SchemaLoader.createKeyspace("Keyspace2",
> KeyspaceParams.simple(1),
> SchemaLoader.standardCFMD("Keyspace2", "Standard2"));
> }
> private void registerMetricListener()
> {
> CassandraMetricsRegistry.Metrics.addListener(new
> MetricRegistryListener.Base()
> {
> @Override
> public void onGaugeAdded(String name, Gauge gauge)
> {
> gauge.getValue();
> }
> @Override
> public void onGaugeRemoved(String name)
> {
> }
> @Override
> public void onCounterAdded(String name, Counter counter)
> {
> counter.getCount();
> }
> @Override
> public void onCounterRemoved(String name)
> {
> }
> @Override
> public void onHistogramAdded(String name, Histogram histogram)
> {
> histogram.getCount();
> }
> @Override
> public void onHistogramRemoved(String name)
> {
> }
> @Override
> public void onMeterAdded(String name, Meter meter)
> {
> meter.getCount();
> }
> @Override
> public void onMeterRemoved(String name)
> {
> }
> @Override
> public void onTimerAdded(String name, Timer timer)
> {
> timer.getCount();
> }
> @Override
> public void onTimerRemoved(String name)
> {
> }
> });{code}
> *NOTE:* In order to be able to run the test and use registerMetricListener a
> small modification to requests in CacheMetrics is needed. This will be
> submitted and further explained with a coming patch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16228) Fixing race condition on ColumnFamilyStore and TableMetrics initialization that makes MetricListener to see an instance in an unknown state
Ekaterina Dimitrova created CASSANDRA-16228:
---
Summary: Fixing race condition on ColumnFamilyStore and
TableMetrics initialization that makes MetricListener to see an instance in an
unknown state
Key: CASSANDRA-16228
URL: https://issues.apache.org/jira/browse/CASSANDRA-16228
Project: Cassandra
Issue Type: Bug
Reporter: Ekaterina Dimitrova
There is a dependency between TableMetrics, SecondaryIndexManager and Tracker
which requires certain order of the operations in ColumnFamilyStore
constructor.
The following test will be added to ColumnFamilyMetricTest to show the issue:
{code:java}
@Test
public void testStartupRaceConditionOnMetricListeners()
{
// Since the ColumnFamilyStore instance reference escapes during the
construction
// we have a race condition and listeners can see an instance that is in an
unknown state
// this test just check that all callbacks can access the data without
throwing any exception.
registerMetricListener();
SchemaLoader.createKeyspace("Keyspace2",
KeyspaceParams.simple(1),
SchemaLoader.standardCFMD("Keyspace2", "Standard2"));
}
private void registerMetricListener()
{
CassandraMetricsRegistry.Metrics.addListener(new MetricRegistryListener.Base()
{
@Override
public void onGaugeAdded(String name, Gauge gauge)
{
gauge.getValue();
}
@Override
public void onGaugeRemoved(String name)
{
}
@Override
public void onCounterAdded(String name, Counter counter)
{
counter.getCount();
}
@Override
public void onCounterRemoved(String name)
{
}
@Override
public void onHistogramAdded(String name, Histogram histogram)
{
histogram.getCount();
}
@Override
public void onHistogramRemoved(String name)
{
}
@Override
public void onMeterAdded(String name, Meter meter)
{
meter.getCount();
}
@Override
public void onMeterRemoved(String name)
{
}
@Override
public void onTimerAdded(String name, Timer timer)
{
timer.getCount();
}
@Override
public void onTimerRemoved(String name)
{
}
});{code}
*NOTE:* In order to be able to run the test and use registerMetricListener a
small modification to requests in CacheMetrics is needed. This will be
submitted and further explained with a coming patch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16220) python dtest pending_range_test.py::TestPendingRangeMovements::test_pending_range (@pytest.mark.resource_intensive) fails on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-16220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221439#comment-17221439
]
Berenguer Blasi commented on CASSANDRA-16220:
-
Ah right! my bad. +1.
> python dtest
> pending_range_test.py::TestPendingRangeMovements::test_pending_range
> (@pytest.mark.resource_intensive) fails on trunk
> --
>
> Key: CASSANDRA-16220
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16220
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: David Capwell
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta
>
>
> The test has the following section
> {code}
> if cluster.version() >= '2.2':
> node2.watch_log_for('127.0.0.1 state moving', timeout=10,
> filename='debug.log’)
> {code}
> The issue is that in trunk we have the port attached to the log, so the log
> is now
> {code}
> DEBUG [GossipStage:1] 2020-10-21 00:48:20,104 StorageService.java:2452 - Node
> /127.0.0.1:7000 state MOVING, tokens [-9223372036854775808]
> DEBUG [GossipStage:1] 2020-10-21 00:48:20,105 StorageService.java:2670 - Node
> /127.0.0.1:7000 state moving, new token -634023222112864484
> {code}
> Since the log now contains the port, this test always times out on trunk when
> it hits this
> {code}
> self =
> @pytest.mark.resource_intensive
> def test_pending_range(self):
> """
> @jira_ticket CASSANDRA-10887
> """
> cluster = self.cluster
> # If we are on 2.1, we need to set the log level to debug or higher,
> as debug.log does not exist.
> if cluster.version() < '2.2':
> cluster.set_log_level('DEBUG')
>
> # Create 5 node cluster
> cluster.populate(5).start()
> node1, node2 = cluster.nodelist()[0:2]
>
> # Set up RF=3 keyspace
> session = self.patient_cql_connection(node1)
> create_ks(session, 'ks', 3)
>
> session.execute("CREATE TABLE users (login text PRIMARY KEY, email
> text, name text, login_count int)")
>
> # We use the partition key 'jdoe3' because it belongs to node1.
> # The key MUST belong to node1 to repro the bug.
> session.execute("INSERT INTO users (login, email, name, login_count)
> VALUES ('jdoe3', 'j...@abc.com', 'Jane Doe', 1) IF NOT EXISTS;")
>
> lwt_query = SimpleStatement("UPDATE users SET email =
> 'jane...@abc.com' WHERE login = 'jdoe3' IF email = 'j...@abc.com'")
>
> # Show we can execute LWT no problem
> for i in range(1000):
> session.execute(lwt_query)
>
> token = '-634023222112864484'
>
> mark = node1.mark_log()
>
> # Move a node
> node1.nodetool('move {}'.format(token))
>
> # Watch the log so we know when the node is moving
> node1.watch_log_for('Moving .* to {}'.format(token), timeout=10,
> from_mark=mark)
> node1.watch_log_for('Sleeping 3 ms before start
> streaming/fetching ranges', timeout=10, from_mark=mark)
>
> if cluster.version() >= '2.2':
> > node2.watch_log_for('127.0.0.1 state moving', timeout=10,
> > filename='debug.log')
> pending_range_test.py:57:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> self =
> exprs = '127.0.0.1 state moving', from_mark = None, timeout = 10, process =
> None
> verbose = False, filename = 'debug.log'
> def watch_log_for(self, exprs, from_mark=None, timeout=600,
> process=None, verbose=False, filename='system.log'):
> """
> Watch the log until one or more (regular) expressions are found
> or timeouts (a
> TimeoutError is then raised). On successful completion, a list of
> pair (line matched,
> match object) is returned.
> """
> start = time.time()
> tofind = [exprs] if isinstance(exprs, string_types) else exprs
> tofind = [re.compile(e) for e in tofind]
> matchings = []
> reads = ""
> if len(tofind) == 0:
> return None
>
> log_file = os.path.join(self.log_directory(), filename)
> output_read = False
> while not os.path.exists(log_file):
> time.sleep(.5)
> if start + timeout < time.time():
> raise TimeoutError(time.strftime("%d %b %Y %H:%M:%S",
> time.gmtime()) + " [" + self.name + "] Timed out waiting for {} to be
> created.".format(log_file))
> if process and not output_read:
> process.poll()
> if process.returncode is not None:
>
[jira] [Assigned] (CASSANDRA-16160) Regression in cqlsh with regard to row id display
[
https://issues.apache.org/jira/browse/CASSANDRA-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams reassigned CASSANDRA-16160:
Assignee: Brandon Williams
> Regression in cqlsh with regard to row id display
> -
>
> Key: CASSANDRA-16160
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16160
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: David Capwell
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-beta
>
>
> When you run a query such as
> {code}
> expand on;
> select * from table_with_clustering_keys where token(partition_key) =
> 1192326969048244361;
> {code}
> We print out a header for each row that looks like the following
> @ Row 1
> In 3.0 all values printed were uniq, but in 4.0 they are no longer unique
> {code}
> $ grep Row 3.0-rows.results | sort | uniq -c | sort -k1 -h -r | head -n 10
> 1 @ Row 999
> 1 @ Row 998
> 1 @ Row 997
> 1 @ Row 996
> 1 @ Row 995
> 1 @ Row 994
> 1 @ Row 993
> 1 @ Row 992
> 1 @ Row 991
> 1 @ Row 990
> {code}
> {code}
> $ grep Row 4.0-rows.results | sort | uniq -c | sort -k1 -h -r | head -n 10
> 10 @ Row 9
> 10 @ Row 8
> 10 @ Row 7
> 10 @ Row 6
> 10 @ Row 5
> 10 @ Row 48
> 10 @ Row 47
> 10 @ Row 46
> 10 @ Row 45
> 10 @ Row 44
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16220) python dtest pending_range_test.py::TestPendingRangeMovements::test_pending_range (@pytest.mark.resource_intensive) fails on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-16220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221415#comment-17221415
]
Ekaterina Dimitrova commented on CASSANDRA-16220:
-
This test is still not running in Jenkins
> python dtest
> pending_range_test.py::TestPendingRangeMovements::test_pending_range
> (@pytest.mark.resource_intensive) fails on trunk
> --
>
> Key: CASSANDRA-16220
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16220
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: David Capwell
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta
>
>
> The test has the following section
> {code}
> if cluster.version() >= '2.2':
> node2.watch_log_for('127.0.0.1 state moving', timeout=10,
> filename='debug.log’)
> {code}
> The issue is that in trunk we have the port attached to the log, so the log
> is now
> {code}
> DEBUG [GossipStage:1] 2020-10-21 00:48:20,104 StorageService.java:2452 - Node
> /127.0.0.1:7000 state MOVING, tokens [-9223372036854775808]
> DEBUG [GossipStage:1] 2020-10-21 00:48:20,105 StorageService.java:2670 - Node
> /127.0.0.1:7000 state moving, new token -634023222112864484
> {code}
> Since the log now contains the port, this test always times out on trunk when
> it hits this
> {code}
> self =
> @pytest.mark.resource_intensive
> def test_pending_range(self):
> """
> @jira_ticket CASSANDRA-10887
> """
> cluster = self.cluster
> # If we are on 2.1, we need to set the log level to debug or higher,
> as debug.log does not exist.
> if cluster.version() < '2.2':
> cluster.set_log_level('DEBUG')
>
> # Create 5 node cluster
> cluster.populate(5).start()
> node1, node2 = cluster.nodelist()[0:2]
>
> # Set up RF=3 keyspace
> session = self.patient_cql_connection(node1)
> create_ks(session, 'ks', 3)
>
> session.execute("CREATE TABLE users (login text PRIMARY KEY, email
> text, name text, login_count int)")
>
> # We use the partition key 'jdoe3' because it belongs to node1.
> # The key MUST belong to node1 to repro the bug.
> session.execute("INSERT INTO users (login, email, name, login_count)
> VALUES ('jdoe3', 'j...@abc.com', 'Jane Doe', 1) IF NOT EXISTS;")
>
> lwt_query = SimpleStatement("UPDATE users SET email =
> 'jane...@abc.com' WHERE login = 'jdoe3' IF email = 'j...@abc.com'")
>
> # Show we can execute LWT no problem
> for i in range(1000):
> session.execute(lwt_query)
>
> token = '-634023222112864484'
>
> mark = node1.mark_log()
>
> # Move a node
> node1.nodetool('move {}'.format(token))
>
> # Watch the log so we know when the node is moving
> node1.watch_log_for('Moving .* to {}'.format(token), timeout=10,
> from_mark=mark)
> node1.watch_log_for('Sleeping 3 ms before start
> streaming/fetching ranges', timeout=10, from_mark=mark)
>
> if cluster.version() >= '2.2':
> > node2.watch_log_for('127.0.0.1 state moving', timeout=10,
> > filename='debug.log')
> pending_range_test.py:57:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> self =
> exprs = '127.0.0.1 state moving', from_mark = None, timeout = 10, process =
> None
> verbose = False, filename = 'debug.log'
> def watch_log_for(self, exprs, from_mark=None, timeout=600,
> process=None, verbose=False, filename='system.log'):
> """
> Watch the log until one or more (regular) expressions are found
> or timeouts (a
> TimeoutError is then raised). On successful completion, a list of
> pair (line matched,
> match object) is returned.
> """
> start = time.time()
> tofind = [exprs] if isinstance(exprs, string_types) else exprs
> tofind = [re.compile(e) for e in tofind]
> matchings = []
> reads = ""
> if len(tofind) == 0:
> return None
>
> log_file = os.path.join(self.log_directory(), filename)
> output_read = False
> while not os.path.exists(log_file):
> time.sleep(.5)
> if start + timeout < time.time():
> raise TimeoutError(time.strftime("%d %b %Y %H:%M:%S",
> time.gmtime()) + " [" + self.name + "] Timed out waiting for {} to be
> created.".format(log_file))
> if process and not output_read:
> process.poll()
> if process.returncode is not None:
>
[jira] [Updated] (CASSANDRA-16217) Minimal 4.0 COMPACT STORAGE backport
[ https://issues.apache.org/jira/browse/CASSANDRA-16217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-16217: Reviewers: Marcus Eriksson > Minimal 4.0 COMPACT STORAGE backport > > > Key: CASSANDRA-16217 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16217 > Project: Cassandra > Issue Type: Bug > Components: Legacy/CQL >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > > There are several behavioural changes related to compact storage, and these > differences are larger than most of us have anticipated: we first thought > there’ll be that “appearing column”, but there’s implicit nulls in > clusterings thing, and row vs column deletion. > Some of the recent issues on the subject are: CASSANDRA-16048, which allows > to ignore these differences. The other one was trying to improve user > experience of anyone still using compact storage: CASSANDRA-15811. > Easily reproducible differernces are: > (1) hidden columns show up, which breaks SELECT * queries > (2) DELETE v and UPDATE v WITH TTL would result into row removals in > non-dense compact tables (CASSANDRA-16069) > (3) INSERT allows skipping clusterings, which are filled with nulls by > default. > Some of these are tricky to support, as 15811 has shown. Anyone on OSS side > who might want to upgrade to 4.0 while still using compact storage might be > affected by being forced into one of these behaviours. > Possible solutions are to document these behaviours, or to bring back a > minimal set of COMPACT STORAGE to keep supporting these. > It looks like it is possible to leave some of the functionality related to > DENSE flag and allow it to be present in 4.0, but only for these three (and > potential related, however not direrclty visible) cases. > [~e.dimitrova] since you were working on removal on compact storage, wanted > to reassure that this is not a revert of your patch. On contrary: your patch > was instrumental in identifying the right places. > cc [~slebresne] [~aleksey] [~benedict] [~marcuse] > |[patch|https://github.com/apache/cassandra/pull/785]|[cihttps://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=13994-followup]| -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16217) Minimal 4.0 COMPACT STORAGE backport
[ https://issues.apache.org/jira/browse/CASSANDRA-16217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-16217: Test and Documentation Plan: Tests included; will be additionally tested wit Harry Status: Patch Available (was: Open) > Minimal 4.0 COMPACT STORAGE backport > > > Key: CASSANDRA-16217 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16217 > Project: Cassandra > Issue Type: Bug > Components: Legacy/CQL >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > > There are several behavioural changes related to compact storage, and these > differences are larger than most of us have anticipated: we first thought > there’ll be that “appearing column”, but there’s implicit nulls in > clusterings thing, and row vs column deletion. > Some of the recent issues on the subject are: CASSANDRA-16048, which allows > to ignore these differences. The other one was trying to improve user > experience of anyone still using compact storage: CASSANDRA-15811. > Easily reproducible differernces are: > (1) hidden columns show up, which breaks SELECT * queries > (2) DELETE v and UPDATE v WITH TTL would result into row removals in > non-dense compact tables (CASSANDRA-16069) > (3) INSERT allows skipping clusterings, which are filled with nulls by > default. > Some of these are tricky to support, as 15811 has shown. Anyone on OSS side > who might want to upgrade to 4.0 while still using compact storage might be > affected by being forced into one of these behaviours. > Possible solutions are to document these behaviours, or to bring back a > minimal set of COMPACT STORAGE to keep supporting these. > It looks like it is possible to leave some of the functionality related to > DENSE flag and allow it to be present in 4.0, but only for these three (and > potential related, however not direrclty visible) cases. > [~e.dimitrova] since you were working on removal on compact storage, wanted > to reassure that this is not a revert of your patch. On contrary: your patch > was instrumental in identifying the right places. > cc [~slebresne] [~aleksey] [~benedict] [~marcuse] > |[patch|https://github.com/apache/cassandra/pull/785]|[cihttps://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=13994-followup]| -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16217) Minimal 4.0 COMPACT STORAGE backport
[ https://issues.apache.org/jira/browse/CASSANDRA-16217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-16217: Bug Category: Parent values: Correctness(12982)Level 1 values: API / Semantic Implementation(12988) Complexity: Normal Component/s: Legacy/CQL Discovered By: User Report Severity: Critical Status: Open (was: Triage Needed) > Minimal 4.0 COMPACT STORAGE backport > > > Key: CASSANDRA-16217 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16217 > Project: Cassandra > Issue Type: Bug > Components: Legacy/CQL >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > > There are several behavioural changes related to compact storage, and these > differences are larger than most of us have anticipated: we first thought > there’ll be that “appearing column”, but there’s implicit nulls in > clusterings thing, and row vs column deletion. > Some of the recent issues on the subject are: CASSANDRA-16048, which allows > to ignore these differences. The other one was trying to improve user > experience of anyone still using compact storage: CASSANDRA-15811. > Easily reproducible differernces are: > (1) hidden columns show up, which breaks SELECT * queries > (2) DELETE v and UPDATE v WITH TTL would result into row removals in > non-dense compact tables (CASSANDRA-16069) > (3) INSERT allows skipping clusterings, which are filled with nulls by > default. > Some of these are tricky to support, as 15811 has shown. Anyone on OSS side > who might want to upgrade to 4.0 while still using compact storage might be > affected by being forced into one of these behaviours. > Possible solutions are to document these behaviours, or to bring back a > minimal set of COMPACT STORAGE to keep supporting these. > It looks like it is possible to leave some of the functionality related to > DENSE flag and allow it to be present in 4.0, but only for these three (and > potential related, however not direrclty visible) cases. > [~e.dimitrova] since you were working on removal on compact storage, wanted > to reassure that this is not a revert of your patch. On contrary: your patch > was instrumental in identifying the right places. > cc [~slebresne] [~aleksey] [~benedict] [~marcuse] > |[patch|https://github.com/apache/cassandra/pull/785]|[cihttps://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=13994-followup]| -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16217) Minimal 4.0 COMPACT STORAGE backport
[ https://issues.apache.org/jira/browse/CASSANDRA-16217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-16217: Description: There are several behavioural changes related to compact storage, and these differences are larger than most of us have anticipated: we first thought there’ll be that “appearing column”, but there’s implicit nulls in clusterings thing, and row vs column deletion. Some of the recent issues on the subject are: CASSANDRA-16048, which allows to ignore these differences. The other one was trying to improve user experience of anyone still using compact storage: CASSANDRA-15811. Easily reproducible differernces are: (1) hidden columns show up, which breaks SELECT * queries (2) DELETE v and UPDATE v WITH TTL would result into row removals in non-dense compact tables (CASSANDRA-16069) (3) INSERT allows skipping clusterings, which are filled with nulls by default. Some of these are tricky to support, as 15811 has shown. Anyone on OSS side who might want to upgrade to 4.0 while still using compact storage might be affected by being forced into one of these behaviours. Possible solutions are to document these behaviours, or to bring back a minimal set of COMPACT STORAGE to keep supporting these. It looks like it is possible to leave some of the functionality related to DENSE flag and allow it to be present in 4.0, but only for these three (and potential related, however not direrclty visible) cases. [~e.dimitrova] since you were working on removal on compact storage, wanted to reassure that this is not a revert of your patch. On contrary: your patch was instrumental in identifying the right places. cc [~slebresne] [~aleksey] [~benedict] [~marcuse] |[patch|https://github.com/apache/cassandra/pull/785]|[cihttps://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=13994-followup]| was: There are several behavioural changes related to compact storage, and these differences are larger than most of us have anticipated: we first thought there’ll be that “appearing column”, but there’s implicit nulls in clusterings thing, and row vs column deletion. Some of the recent issues on the subject are: CASSANDRA-16048, which allows to ignore these differences. The other one was trying to improve user experience of anyone still using compact storage: CASSANDRA-15811. Easily reproducible differernces are: (1) hidden columns show up, which breaks SELECT * queries (2) DELETE v and UPDATE v WITH TTL would result into row removals in non-dense compact tables (CASSANDRA-16069) (3) INSERT allows skipping clusterings, which are filled with nulls by default. Some of these are tricky to support, as 15811 has shown. Anyone on OSS side who might want to upgrade to 4.0 while still using compact storage might be affected by being forced into one of these behaviours. Possible solutions are to document these behaviours, or to bring back a minimal set of COMPACT STORAGE to keep supporting these. It looks like it is possible to leave some of the functionality related to DENSE flag and allow it to be present in 4.0, but only for these three (and potential related, however not direrclty visible) cases. [~e.dimitrova] since you were working on removal on compact storage, wanted to reassure that this is not a revert of your patch. On contrary: your patch was instrumental in identifying the right places. cc [~slebresne] [~aleksey] [~benedict] [~marcuse] Preliminary patch: [https://github.com/apache/cassandra/pull/785] > Minimal 4.0 COMPACT STORAGE backport > > > Key: CASSANDRA-16217 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16217 > Project: Cassandra > Issue Type: Bug >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > > There are several behavioural changes related to compact storage, and these > differences are larger than most of us have anticipated: we first thought > there’ll be that “appearing column”, but there’s implicit nulls in > clusterings thing, and row vs column deletion. > Some of the recent issues on the subject are: CASSANDRA-16048, which allows > to ignore these differences. The other one was trying to improve user > experience of anyone still using compact storage: CASSANDRA-15811. > Easily reproducible differernces are: > (1) hidden columns show up, which breaks SELECT * queries > (2) DELETE v and UPDATE v WITH TTL would result into row removals in > non-dense compact tables (CASSANDRA-16069) > (3) INSERT allows skipping clusterings, which are filled with nulls by > default. > Some of these are tricky to support, as 15811 has shown. Anyone on OSS side > who might want to upgrade to 4.0 while still using compact storage might be > affected by being forced into one of these behaviours. >
[jira] [Commented] (CASSANDRA-15823) Support for networking via identity instead of IP
[ https://issues.apache.org/jira/browse/CASSANDRA-15823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221344#comment-17221344 ] vincent royer commented on CASSANDRA-15823: --- Hi [~jjirsa], would you have the scenario that leads to inconsistency when replacing a node with the same IP as the dead node (A scenario also possible under kubernetes) ? > Support for networking via identity instead of IP > - > > Key: CASSANDRA-15823 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15823 > Project: Cassandra > Issue Type: Improvement >Reporter: Christopher Bradford >Priority: Normal > Labels: kubernetes > Attachments: consul-mesh-gateways.png, > istio-multicluster-with-gateways.svg, linkerd-service-mirroring.svg > > > TL;DR: Instead of mapping host ids to IPs, use hostnames. This allows > resolution to different IP addresses per DC that may then be forwarded to > nodes on remote networks without requiring node to node IP connectivity for > cross-dc links. > > This approach should not affect existing deployments as those could continue > to use IPs as the hostname and skip resolution. > > With orchestration platforms like Kubernetes and the usage of ephemeral > containers in environments today we should consider some changes to how we > handle the tracking of nodes and their network location. Currently we > maintain a mapping between host ids and IP addresses. > > With traditional infrastructure, if a node goes down it, usually, comes back > up with the same IP. In some environments this contract may be explicit with > virtual IPs that may move between hosts. In newer deployments, like on > Kubernetes, this contract is not possible. Pods (analogous to nodes) are > assigned an IP address at start time. Should the pod be restarted or > scheduled on a different host there is no guarantee we would have the same > IP. Cassandra is protected here as we already have logic in place to update > peers when we come up with the same host id, but a different IP address. > > There are ways to get Kubernetes to assign a specific IP per Pod. Most > recommendations involve the use of a service per pod. Communication with the > fixed service IP would automatically forward to the associated pod, > regardless of address. We _could_ use this approach, but it seems like this > would needlessly create a number of extra resources in our k8s cluster to get > around the problem. Which, to be fair, doesn't seem like much of a problem > with the aforementioned mitigations built into C*. > > So what is the _actual_ problem? *Cross-region, cross-cloud, > hybrid-deployment connectivity between pods is a pain.* This can be solved > with significant investment by those who want to deploy these types of > topologies. You can definitely configure connectivity between clouds over > dedicated connections, or VPN tunnels. With a big chunk of time insuring that > pod to pod connectivity just works even if those pods are managed by separate > control planes, but that again requires time and talent. There are a number > of edge cases to support between the ever so slight, but very important, > differences in cloud vendor networks. > > Recently there have been a number of innovations that aid in the deployment > and operation of these types of applications on Kubernetes. Service meshes > support distributed microservices running across multiple k8s cluster control > planes in disparate networks. Instead of directly connecting to IP addresses > of remote services instead they use a hostname. With this approach, hostname > traffic may then be routed to a proxy that sends traffic over the WAN > (sometimes with mTLS) to another proxy pod in the remote cluster which then > forwards the data along to the correct pod in that network. (See attached > diagrams) > > Which brings us to the point of this ticket. Instead of mapping host ids to > IPs, use hostnames (and update the underlying address periodically instead of > caching indefinitely). This allows resolution to different IP addresses per > DC (k8s cluster) that may then be forwarded to nodes (pods) on remote > networks (k8s clusters) without requiring node to node (pod to pod) IP > connectivity between them. Traditional deployments can still function like > they do today (even if operators opt to keep using IPs as identifiers instead > of hostnames). This proxy approach is then enabled like those we see in > service meshes. > > _Notes_ > C* already has the concept of broadcast addresses vs those which are bound on > the node. This approach _could_ be leveraged to provide the behavior we're > looking for, but then the broadcast values would need to be pre-computed > _*and match*_ across all k8s control planes. By
[jira] [Updated] (CASSANDRA-15369) Fake row deletions and range tombstones, causing digest mismatch and sstable growth
[
https://issues.apache.org/jira/browse/CASSANDRA-15369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao Yang updated CASSANDRA-15369:
--
Test and Documentation Plan:
Added unit tests to read RT with named queries and slice queries.
[CIrcle
CI|https://app.circleci.com/pipelines/github/jasonstack/cassandra/328/workflows/5bdb12cd-6c92-4d06-aed2-445f75e2b8f5]
[asf-ci|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/141/]running
was:
Added unit tests to read RT with named queries and slice queries.
[CIrcle
CI|https://app.circleci.com/pipelines/github/jasonstack/cassandra/328/workflows/2a6ac1f4-9241-498f-889a-cde07bff3c1e]
[asf-ci|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/135/]running
> Fake row deletions and range tombstones, causing digest mismatch and sstable
> growth
> ---
>
> Key: CASSANDRA-15369
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15369
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination, Local/Memtable, Local/SSTable
>Reporter: Benedict Elliott Smith
>Assignee: Zhao Yang
>Priority: Normal
> Fix For: 4.0-beta
>
>
> As assessed in CASSANDRA-15363, we generate fake row deletions and fake
> tombstone markers under various circumstances:
> * If we perform a clustering key query (or select a compact column):
> * Serving from a {{Memtable}}, we will generate fake row deletions
> * Serving from an sstable, we will generate fake row tombstone markers
> * If we perform a slice query, we will generate only fake row tombstone
> markers for any range tombstone that begins or ends outside of the limit of
> the requested slice
> * If we perform a multi-slice or IN query, this will occur for each
> slice/clustering
> Unfortunately, these different behaviours can lead to very different data
> stored in sstables until a full repair is run. When we read-repair, we only
> send these fake deletions or range tombstones. A fake row deletion,
> clustering RT and slice RT, each produces a different digest. So for each
> single point lookup we can produce a digest mismatch twice, and until a full
> repair is run we can encounter an unlimited number of digest mismatches
> across different overlapping queries.
> Relatedly, this seems a more problematic variant of our atomicity failures
> caused by our monotonic reads, since RTs can have an atomic effect across (up
> to) the entire partition, whereas the propagation may happen on an
> arbitrarily small portion. If the RT exists on only one node, this could
> plausibly lead to fairly problematic scenario if that node fails before the
> range can be repaired.
> At the very least, this behaviour can lead to an almost unlimited amount of
> extraneous data being stored until the range is repaired and compaction
> happens to overwrite the sub-range RTs and row deletions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16163) Rename master branches to trunk in all repositories
[ https://issues.apache.org/jira/browse/CASSANDRA-16163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16163: --- Resolution: Fixed Status: Resolved (was: Triage Needed) > Rename master branches to trunk in all repositories > --- > > Key: CASSANDRA-16163 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16163 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Michael Semb Wever >Priority: Normal > > Applies to the following repositories > * cassandra-builds > * cassandra-website > * cassandra-dtest > * cassandra-sidecar > * cassandra-diff > * cassandra-in-jvm-dtest-api > * cassandra-harry > This was discussed in > https://lists.apache.org/thread.html/r54db4cd870d2d665060d5fb50d925843be4b4d54dc64f3d21f04c367%40%3Cdev.cassandra.apache.org%3E > The general preference there was trunk over main, so to match the cassandra > repository. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16163) Rename master branches to trunk in all repositories
[ https://issues.apache.org/jira/browse/CASSANDRA-16163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16163: --- Fix Version/s: 4.0-beta3 3.11.9 3.0.23 2.2.19 > Rename master branches to trunk in all repositories > --- > > Key: CASSANDRA-16163 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16163 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Michael Semb Wever >Priority: Normal > Fix For: 2.2.19, 3.0.23, 3.11.9, 4.0-beta3 > > > Applies to the following repositories > * cassandra-builds > * cassandra-website > * cassandra-dtest > * cassandra-sidecar > * cassandra-diff > * cassandra-in-jvm-dtest-api > * cassandra-harry > This was discussed in > https://lists.apache.org/thread.html/r54db4cd870d2d665060d5fb50d925843be4b4d54dc64f3d21f04c367%40%3Cdev.cassandra.apache.org%3E > The general preference there was trunk over main, so to match the cassandra > repository. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16166) Rename master branch to trunk in cassandra-dtest
[ https://issues.apache.org/jira/browse/CASSANDRA-16166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever reassigned CASSANDRA-16166: -- Assignee: (was: Michael Semb Wever) > Rename master branch to trunk in cassandra-dtest > > > Key: CASSANDRA-16166 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16166 > Project: Cassandra > Issue Type: Sub-task > Components: Build >Reporter: Michael Semb Wever >Priority: Normal > Fix For: 2.2.19, 3.0.23, 3.11.9, 4.0-beta3 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16166) Rename master branch to trunk in cassandra-dtest
[ https://issues.apache.org/jira/browse/CASSANDRA-16166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16166: --- Fix Version/s: 4.0-beta3 3.11.9 3.0.23 2.2.19 Source Control Link: https://github.com/apache/cassandra/commit/5d269807778534fc4eaabdfbc90cd34f8927d968 https://github.com/apache/cassandra-dtest/commit/8d20f667ea9761517c51f4cf7a17e119fa8897c9 https://github.com/apache/cassandra-builds/commit/9a4a4eb0a1ec8b39cb1d27d7476d2 Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed as [5d269807778534fc4eaabdfbc90cd34f8927d968|https://github.com/apache/cassandra/commit/5d269807778534fc4eaabdfbc90cd34f8927d968] and [8d20f667ea9761517c51f4cf7a17e119fa8897c9|https://github.com/apache/cassandra-dtest/commit/8d20f667ea9761517c51f4cf7a17e119fa8897c9] and [9a4a4eb0a1ec8b39cb1d27d7476d2|https://github.com/apache/cassandra-builds/commit/9a4a4eb0a1ec8b39cb1d27d7476d2] > Rename master branch to trunk in cassandra-dtest > > > Key: CASSANDRA-16166 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16166 > Project: Cassandra > Issue Type: Sub-task > Components: Build >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 2.2.19, 3.0.23, 3.11.9, 4.0-beta3 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (d7f0361 -> f139c6a)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from d7f0361 ninja: remove CI configs added in CASSANDRA-16152 new 5d26980 Rename master branch to trunk in cassandra-dtest new f139c6a Merge branch 'cassandra-2.2' into cassandra-3.0 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .circleci/config-2_1.yml | 2 +- .circleci/config.yml | 20 ++-- .circleci/config.yml.HIGHRES | 20 ++-- .circleci/config.yml.LOWRES | 20 ++-- .jenkins/Jenkinsfile | 2 +- 5 files changed, 32 insertions(+), 32 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.2 updated: Rename master branch to trunk in cassandra-dtest
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch cassandra-2.2 in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/cassandra-2.2 by this push: new 5d26980 Rename master branch to trunk in cassandra-dtest 5d26980 is described below commit 5d269807778534fc4eaabdfbc90cd34f8927d968 Author: Mick Semb Wever AuthorDate: Sun Oct 25 15:26:14 2020 +0100 Rename master branch to trunk in cassandra-dtest patch by Mick Semb Wever; reviewed by Brandon Williams for CASSANDRA-16166 --- .circleci/config-2_1.yml | 2 +- .circleci/config.yml | 16 .circleci/config.yml.HIGHRES | 16 .circleci/config.yml.LOWRES | 16 .jenkins/Jenkinsfile | 2 +- 5 files changed, 26 insertions(+), 26 deletions(-) diff --git a/.circleci/config-2_1.yml b/.circleci/config-2_1.yml index e90ef4a..79332a8 100644 --- a/.circleci/config-2_1.yml +++ b/.circleci/config-2_1.yml @@ -13,7 +13,7 @@ default_env_vars: _env_vars #Skip all syncing to disk to avoid performance issues in flaky CI environments CASSANDRA_SKIP_SYNC: true DTEST_REPO: git://github.com/apache/cassandra-dtest.git -DTEST_BRANCH: master +DTEST_BRANCH: trunk CCM_MAX_HEAP_SIZE: 1024M CCM_HEAP_NEWSIZE: 256M diff --git a/.circleci/config.yml b/.circleci/config.yml index d5efe4f..0997d8a 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -70,7 +70,7 @@ jobs: - CASS_DRIVER_NO_CYTHON: true - CASSANDRA_SKIP_SYNC: true - DTEST_REPO: git://github.com/apache/cassandra-dtest.git -- DTEST_BRANCH: master +- DTEST_BRANCH: trunk - CCM_MAX_HEAP_SIZE: 1024M - CCM_HEAP_NEWSIZE: 256M - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64 @@ -159,7 +159,7 @@ jobs: - CASS_DRIVER_NO_CYTHON: true - CASSANDRA_SKIP_SYNC: true - DTEST_REPO: git://github.com/apache/cassandra-dtest.git -- DTEST_BRANCH: master +- DTEST_BRANCH: trunk - CCM_MAX_HEAP_SIZE: 1024M - CCM_HEAP_NEWSIZE: 256M - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64 @@ -248,7 +248,7 @@ jobs: - CASS_DRIVER_NO_CYTHON: true - CASSANDRA_SKIP_SYNC: true - DTEST_REPO: git://github.com/apache/cassandra-dtest.git -- DTEST_BRANCH: master +- DTEST_BRANCH: trunk - CCM_MAX_HEAP_SIZE: 1024M - CCM_HEAP_NEWSIZE: 256M - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64 @@ -291,7 +291,7 @@ jobs: - CASS_DRIVER_NO_CYTHON: true - CASSANDRA_SKIP_SYNC: true - DTEST_REPO: git://github.com/apache/cassandra-dtest.git -- DTEST_BRANCH: master +- DTEST_BRANCH: trunk - CCM_MAX_HEAP_SIZE: 1024M - CCM_HEAP_NEWSIZE: 256M - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64 @@ -380,7 +380,7 @@ jobs: - CASS_DRIVER_NO_CYTHON: true - CASSANDRA_SKIP_SYNC: true - DTEST_REPO: git://github.com/apache/cassandra-dtest.git -- DTEST_BRANCH: master +- DTEST_BRANCH: trunk - CCM_MAX_HEAP_SIZE: 1024M - CCM_HEAP_NEWSIZE: 256M - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64 @@ -438,7 +438,7 @@ jobs: - CASS_DRIVER_NO_CYTHON: true - CASSANDRA_SKIP_SYNC: true - DTEST_REPO: git://github.com/apache/cassandra-dtest.git -- DTEST_BRANCH: master +- DTEST_BRANCH: trunk - CCM_MAX_HEAP_SIZE: 1024M - CCM_HEAP_NEWSIZE: 256M - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64 @@ -496,7 +496,7 @@ jobs: - CASS_DRIVER_NO_CYTHON: true - CASSANDRA_SKIP_SYNC: true - DTEST_REPO: git://github.com/apache/cassandra-dtest.git -- DTEST_BRANCH: master +- DTEST_BRANCH: trunk - CCM_MAX_HEAP_SIZE: 1024M - CCM_HEAP_NEWSIZE: 256M - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64 @@ -595,7 +595,7 @@ jobs: - CASS_DRIVER_NO_CYTHON: true - CASSANDRA_SKIP_SYNC: true - DTEST_REPO: git://github.com/apache/cassandra-dtest.git -- DTEST_BRANCH: master +- DTEST_BRANCH: trunk - CCM_MAX_HEAP_SIZE: 1024M - CCM_HEAP_NEWSIZE: 256M - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64 diff --git a/.circleci/config.yml.HIGHRES b/.circleci/config.yml.HIGHRES index 77a396d..64e2e74 100644 --- a/.circleci/config.yml.HIGHRES +++ b/.circleci/config.yml.HIGHRES @@ -70,7 +70,7 @@ jobs: - CASS_DRIVER_NO_CYTHON: true - CASSANDRA_SKIP_SYNC: true - DTEST_REPO: git://github.com/apache/cassandra-dtest.git -- DTEST_BRANCH: master +- DTEST_BRANCH: trunk - CCM_MAX_HEAP_SIZE: 2048M - CCM_HEAP_NEWSIZE: 512M - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64 @@ -159,7 +159,7 @@ jobs: - CASS_DRIVER_NO_CYTHON: true - CASSANDRA_SKIP_SYNC: true - DTEST_REPO: git://github.com/apache/cassandra-dtest.git -- DTEST_BRANCH: master +- DTEST_BRANCH: trunk - CCM_MAX_HEAP_SIZE: 2048M - CCM_HEAP_NEWSIZE: 512M - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64 @@ -248,7 +248,7 @@ jobs: -
[cassandra] branch cassandra-3.11 updated (65a7997 -> 06e4e81)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 65a7997 Merge branch 'cassandra-3.0' into cassandra-3.11 new 5d26980 Rename master branch to trunk in cassandra-dtest new f139c6a Merge branch 'cassandra-2.2' into cassandra-3.0 new 06e4e81 Merge branch 'cassandra-3.0' into cassandra-3.11 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .circleci/config-2_1.yml | 2 +- .circleci/config.yml | 22 +++--- .circleci/config.yml.HIGHRES | 22 +++--- .circleci/config.yml.LOWRES| 22 +++--- .jenkins/Jenkinsfile | 2 +- doc/source/development/testing.rst | 2 +- 6 files changed, 36 insertions(+), 36 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit f139c6a6c171ec4a991f21da1dc97376bbf1d264
Merge: d7f0361 5d26980
Author: Mick Semb Wever
AuthorDate: Tue Oct 27 10:47:05 2020 +0100
Merge branch 'cassandra-2.2' into cassandra-3.0
.circleci/config-2_1.yml | 2 +-
.circleci/config.yml | 20 ++--
.circleci/config.yml.HIGHRES | 20 ++--
.circleci/config.yml.LOWRES | 20 ++--
.jenkins/Jenkinsfile | 2 +-
5 files changed, 32 insertions(+), 32 deletions(-)
diff --cc .circleci/config.yml
index f8526be,0997d8a..b8aa6ea
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@@ -1,97 -1,5 +1,97 @@@
version: 2
jobs:
+ j8_jvm_upgrade_dtests:
+docker:
+- image: spod/cassandra-testing-ubuntu1810-java11-w-dependencies:20190306
+resource_class: medium
+working_directory: ~/
+shell: /bin/bash -eo pipefail -l
+parallelism: 1
+steps:
+- attach_workspace:
+at: /home/cassandra
+- run:
+name: Determine distributed Tests to Run
+command: |
+ # reminder: this code (along with all the steps) is independently
executed on every circle container
+ # so the goal here is to get the circleci script to return the
tests *this* container will run
+ # which we do via the `circleci` cli tool.
+
+ rm -fr ~/cassandra-dtest/upgrade_tests
+ echo "***java tests***"
+
+ # get all of our unit test filenames
+ set -eo pipefail && circleci tests glob
"$HOME/cassandra/test/distributed/**/*.java" > /tmp/all_java_unit_tests.txt
+
+ # split up the unit tests into groups based on the number of
containers we have
+ set -eo pipefail && circleci tests split --split-by=timings
--timings-type=filename --index=${CIRCLE_NODE_INDEX}
--total=${CIRCLE_NODE_TOTAL} /tmp/all_java_unit_tests.txt >
/tmp/java_tests_${CIRCLE_NODE_INDEX}.txt
+ set -eo pipefail && cat /tmp/java_tests_${CIRCLE_NODE_INDEX}.txt |
sed "s;^/home/cassandra/cassandra/test/distributed/;;g" | grep "Test\.java$" |
grep upgrade > /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
+ echo "** /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt"
+ cat /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
+no_output_timeout: 15m
+- run:
+name: Log Environment Information
+command: |
+ echo '*** id ***'
+ id
+ echo '*** cat /proc/cpuinfo ***'
+ cat /proc/cpuinfo
+ echo '*** free -m ***'
+ free -m
+ echo '*** df -m ***'
+ df -m
+ echo '*** ifconfig -a ***'
+ ifconfig -a
+ echo '*** uname -a ***'
+ uname -a
+ echo '*** mount ***'
+ mount
+ echo '*** env ***'
+ env
+ echo '*** java ***'
+ which java
+ java -version
+- run:
+name: Run Unit Tests (testclasslist)
+command: |
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
+ test_timeout=$(grep 'name="test.distributed.timeout"' build.xml |
awk -F'"' '{print $4}' || true)
+ if [ -z "$test_timeout" ]; then
+test_timeout=$(grep 'name="test.timeout"' build.xml | awk -F'"'
'{print $4}')
+ fi
+ ant testclasslist -Dtest.timeout="$test_timeout"
-Dtest.classlistfile=/tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
-Dtest.classlistprefix=distributed
+no_output_timeout: 15m
+- store_test_results:
+path: /tmp/cassandra/build/test/output/
+- store_artifacts:
+path: /tmp/cassandra/build/test/output
+destination: junitxml
+- store_artifacts:
+path: /tmp/cassandra/build/test/logs
+destination: logs
+environment:
+- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+- ANT_HOME: /usr/share/ant
+- LANG: en_US.UTF-8
+- KEEP_TEST_DIR: true
+- DEFAULT_DIR: /home/cassandra/cassandra-dtest
+- PYTHONIOENCODING: utf-8
+- PYTHONUNBUFFERED: true
+- CASS_DRIVER_NO_EXTENSIONS: true
+- CASS_DRIVER_NO_CYTHON: true
+- CASSANDRA_SKIP_SYNC: true
+- DTEST_REPO: git://github.com/apache/cassandra-dtest.git
- - DTEST_BRANCH: master
++- DTEST_BRANCH: trunk
+- CCM_MAX_HEAP_SIZE: 1024M
+- CCM_HEAP_NEWSIZE: 256M
+- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
build:
docker:
- image: spod/cassandra-testing-ubuntu1810-java11-w-dependencies:20190306
@@@ -173,163 -75,6 +173,163 @@@
- CCM_HEAP_NEWSIZE:
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 06e4e81d87794b469d55d607e373387ab1a47072
Merge: 65a7997 f139c6a
Author: Mick Semb Wever
AuthorDate: Tue Oct 27 10:48:17 2020 +0100
Merge branch 'cassandra-3.0' into cassandra-3.11
.circleci/config-2_1.yml | 2 +-
.circleci/config.yml | 22 +++---
.circleci/config.yml.HIGHRES | 22 +++---
.circleci/config.yml.LOWRES| 22 +++---
.jenkins/Jenkinsfile | 2 +-
doc/source/development/testing.rst | 2 +-
6 files changed, 36 insertions(+), 36 deletions(-)
diff --cc .circleci/config.yml
index dd4b8db,b8aa6ea..b0db851
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@@ -330,52 -330,6 +330,52 @@@ jobs
- CCM_HEAP_NEWSIZE: 256M
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ utests_stress:
+docker:
+- image: spod/cassandra-testing-ubuntu1810-java11-w-dependencies:20190306
+resource_class: medium
+working_directory: ~/
+shell: /bin/bash -eo pipefail -l
+parallelism: 1
+steps:
+- attach_workspace:
+at: /home/cassandra
+- run:
+name: Run Unit Tests (stress-test)
+command: |
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
+ ant stress-test
-Dtest.classlistfile=/tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
-Dtest.classlistprefix=unit
+no_output_timeout: 15m
+- store_test_results:
+path: /tmp/cassandra/build/test/output/
+- store_artifacts:
+path: /tmp/cassandra/build/test/output
+destination: junitxml
+- store_artifacts:
+path: /tmp/cassandra/build/test/logs
+destination: logs
+environment:
+- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+- ANT_HOME: /usr/share/ant
+- LANG: en_US.UTF-8
+- KEEP_TEST_DIR: true
+- DEFAULT_DIR: /home/cassandra/cassandra-dtest
+- PYTHONIOENCODING: utf-8
+- PYTHONUNBUFFERED: true
+- CASS_DRIVER_NO_EXTENSIONS: true
+- CASS_DRIVER_NO_CYTHON: true
+- CASSANDRA_SKIP_SYNC: true
+- DTEST_REPO: git://github.com/apache/cassandra-dtest.git
- - DTEST_BRANCH: master
++- DTEST_BRANCH: trunk
+- CCM_MAX_HEAP_SIZE: 1024M
+- CCM_HEAP_NEWSIZE: 256M
+- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
j8_unit_tests:
docker:
- image: spod/cassandra-testing-ubuntu1810-java11-w-dependencies:20190306
diff --cc .circleci/config.yml.HIGHRES
index 37dcf7a,c21ad30..834f046
--- a/.circleci/config.yml.HIGHRES
+++ b/.circleci/config.yml.HIGHRES
@@@ -330,52 -330,6 +330,52 @@@ jobs
- CCM_HEAP_NEWSIZE: 512M
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ utests_stress:
+docker:
+- image: spod/cassandra-testing-ubuntu1810-java11-w-dependencies:20190306
+resource_class: xlarge
+working_directory: ~/
+shell: /bin/bash -eo pipefail -l
+parallelism: 1
+steps:
+- attach_workspace:
+at: /home/cassandra
+- run:
+name: Run Unit Tests (stress-test)
+command: |
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
+ ant stress-test
-Dtest.classlistfile=/tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
-Dtest.classlistprefix=unit
+no_output_timeout: 15m
+- store_test_results:
+path: /tmp/cassandra/build/test/output/
+- store_artifacts:
+path: /tmp/cassandra/build/test/output
+destination: junitxml
+- store_artifacts:
+path: /tmp/cassandra/build/test/logs
+destination: logs
+environment:
+- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+- ANT_HOME: /usr/share/ant
+- LANG: en_US.UTF-8
+- KEEP_TEST_DIR: true
+- DEFAULT_DIR: /home/cassandra/cassandra-dtest
+- PYTHONIOENCODING: utf-8
+- PYTHONUNBUFFERED: true
+- CASS_DRIVER_NO_EXTENSIONS: true
+- CASS_DRIVER_NO_CYTHON: true
+- CASSANDRA_SKIP_SYNC: true
+- DTEST_REPO: git://github.com/apache/cassandra-dtest.git
- - DTEST_BRANCH: master
++- DTEST_BRANCH: trunk
+- CCM_MAX_HEAP_SIZE: 2048M
+- CCM_HEAP_NEWSIZE: 512M
+- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
j8_unit_tests:
docker:
- image:
[cassandra] branch trunk updated (dfd9c74 -> 6ca34f8)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from dfd9c74 Merge branch 'cassandra-3.11' into trunk new 5d26980 Rename master branch to trunk in cassandra-dtest new f139c6a Merge branch 'cassandra-2.2' into cassandra-3.0 new 06e4e81 Merge branch 'cassandra-3.0' into cassandra-3.11 new 6ca34f8 Merge branch 'cassandra-3.11' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .circleci/config-2_1.yml | 2 +- .circleci/config.yml | 58 .circleci/config.yml.HIGHRES | 58 .circleci/config.yml.LOWRES | 58 .circleci/config.yml.MIDRES | 58 .jenkins/Jenkinsfile | 2 +- doc/source/development/documentation.rst | 2 +- doc/source/development/testing.rst | 2 +- 8 files changed, 120 insertions(+), 120 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-dtest] branch trunk updated: Rename master branch to trunk in cassandra-dtest
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/trunk by this push:
new 8d20f66 Rename master branch to trunk in cassandra-dtest
8d20f66 is described below
commit 8d20f667ea9761517c51f4cf7a17e119fa8897c9
Author: Mick Semb Wever
AuthorDate: Sun Oct 25 13:00:08 2020 +0100
Rename master branch to trunk in cassandra-dtest
patch by Mick Semb Wever; reviewed by Brandon Williams for CASSANDRA-16166
---
CONTRIBUTING.md | 2 +-
README.md | 2 +-
linter_check.sh | 4 ++--
3 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 5c9e72a..c881d1a 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -60,7 +60,7 @@ In some cases, we organize our test files by putting them in
directories. If you
## Assertions
-- When possible, you should use the assert functions from
[`tools/assertions.py`](https://github.com/apache/cassandra-dtest/blob/master/tools/assertions.py).
+- When possible, you should use the assert functions from
[`tools/assertions.py`](https://github.com/apache/cassandra-dtest/blob/trunk/tools/assertions.py).
- When none of these are applicable, use python's built in [unittest
assertions](https://docs.python.org/2/library/unittest.html#assert-methods).
## Byteman Files
diff --git a/README.md b/README.md
index 219..9afa7b4 100644
--- a/README.md
+++ b/README.md
@@ -35,7 +35,7 @@ bootstrap your development environment and then use
[Homebrew](https://brew.sh)
### Python Dependencies
There are multiple external Python dependencies required to run DTests.
The current Python dependency list is maintained in a file named
-[requirements.txt](https://github.com/apache/cassandra-dtest/blob/master/requirements.txt)
+[requirements.txt](https://github.com/apache/cassandra-dtest/blob/trunk/requirements.txt)
in the root of the `cassandra-dtest` repository.
The easiest way to install these dependencies is with pip and virtualenv.
diff --git a/linter_check.sh b/linter_check.sh
index fccb053..6e1acd9 100755
--- a/linter_check.sh
+++ b/linter_check.sh
@@ -7,11 +7,11 @@ flake8 --ignore=E501,F811,F812,F822,F823,F831,F841,N8,C9
--exclude=thrift_bindin
flake8_result=$?
# lint all files for everything but line length errors
-git diff apache/master...HEAD -U0 | pycodestyle --ignore=E501 --diff
+git diff apache/trunk...HEAD -U0 | pycodestyle --ignore=E501 --diff
pep8_style_check=$?
# lint all files except json_test.py for line length errors
-git diff apache/master...HEAD -U0 | pycodestyle --diff
--exclude='json_test.py' --exclude='meta_tests/assertion_test.py'
--max-line-length=200
+git diff apache/trunk...HEAD -U0 | pycodestyle --diff --exclude='json_test.py'
--exclude='meta_tests/assertion_test.py' --max-line-length=200
pep8_line_length=$?
echo -e "\nflake8 exited with ${flake8_result}."
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch trunk updated: Rename master branch to trunk in cassandra-dtest
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
The following commit(s) were added to refs/heads/trunk by this push:
new 9a4a4eb Rename master branch to trunk in cassandra-dtest
9a4a4eb is described below
commit 9a4a4eb0a1ec8b39cb1d27d7476d2fd617d1402a
Author: Mick Semb Wever
AuthorDate: Sun Oct 25 14:12:36 2020 +0100
Rename master branch to trunk in cassandra-dtest
patch by Mick Semb Wever; reviewed by Brandon Williams for CASSANDRA-16166
---
docker/testing/ubuntu1810_j11.docker | 2 +-
docker/testing/ubuntu18_j11.docker| 2 +-
docker/testing/ubuntu1910_j11.docker | 2 +-
jenkins-dsl/cassandra_job_dsl_seed.groovy | 8
4 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/docker/testing/ubuntu1810_j11.docker
b/docker/testing/ubuntu1810_j11.docker
index 58fea47..07d68ba 100644
--- a/docker/testing/ubuntu1810_j11.docker
+++ b/docker/testing/ubuntu1810_j11.docker
@@ -43,7 +43,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
# as we only need the requirements.txt file from the dtest repo, let's just
get it from GitHub as a raw asset
# so we can avoid needing to clone the entire repo just to get this file
-ADD
https://raw.githubusercontent.com/apache/cassandra-dtest/master/requirements.txt
/opt
+ADD
https://raw.githubusercontent.com/apache/cassandra-dtest/trunk/requirements.txt
/opt
RUN chmod 0644 /opt/requirements.txt
# now setup python via virtualenv with all of the python dependencies we need
according to requirements.txt
diff --git a/docker/testing/ubuntu18_j11.docker
b/docker/testing/ubuntu18_j11.docker
index 3e6fe6b..88a80d6 100644
--- a/docker/testing/ubuntu18_j11.docker
+++ b/docker/testing/ubuntu18_j11.docker
@@ -43,7 +43,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
# as we only need the requirements.txt file from the dtest repo, let's just
get it from GitHub as a raw asset
# so we can avoid needing to clone the entire repo just to get this file
-ADD
https://raw.githubusercontent.com/apache/cassandra-dtest/master/requirements.txt
/opt
+ADD
https://raw.githubusercontent.com/apache/cassandra-dtest/trunk/requirements.txt
/opt
RUN chmod 0644 /opt/requirements.txt
# now setup python via virtualenv with all of the python dependencies we need
according to requirements.txt
diff --git a/docker/testing/ubuntu1910_j11.docker
b/docker/testing/ubuntu1910_j11.docker
index 70dd44b..e56ac5c 100644
--- a/docker/testing/ubuntu1910_j11.docker
+++ b/docker/testing/ubuntu1910_j11.docker
@@ -55,7 +55,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
# as we only need the requirements.txt file from the dtest repo, let's just
get it from GitHub as a raw asset
# so we can avoid needing to clone the entire repo just to get this file
-ADD
https://raw.githubusercontent.com/apache/cassandra-dtest/master/requirements.txt
/opt
+ADD
https://raw.githubusercontent.com/apache/cassandra-dtest/trunk/requirements.txt
/opt
RUN chmod 0644 /opt/requirements.txt
# now setup python via virtualenv with all of the python dependencies we need
according to requirements.txt
diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy
b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index e55d339..7e20c5d 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -520,7 +520,7 @@ cassandraBranches.each {
}
steps {
shell("""
-sh ./cassandra-builds/docker/jenkins/jenkinscommand.sh
apache ${branchName} https://github.com/apache/cassandra-dtest.git master
${buildsRepo} ${buildsBranch} ${dtestDockerImage} ${targetName}
\${split}/${splits} ;
+sh ./cassandra-builds/docker/jenkins/jenkinscommand.sh
apache ${branchName} https://github.com/apache/cassandra-dtest.git trunk
${buildsRepo} ${buildsBranch} ${dtestDockerImage} ${targetName}
\${split}/${splits} ;
xz test_stdout.txt
""")
}
@@ -790,7 +790,7 @@ dtestTargets.each {
stringParam('REPO', 'apache', 'The github user/org to clone
cassandra repo from')
stringParam('BRANCH', 'trunk', 'The branch of cassandra to
checkout')
stringParam('DTEST_REPO', "${dtestRepo}", 'The cassandra-dtest
repo URL')
-stringParam('DTEST_BRANCH', 'master', 'The branch of
cassandra-dtest to checkout')
+stringParam('DTEST_BRANCH', 'trunk', 'The branch of
cassandra-dtest to checkout')
stringParam('DOCKER_IMAGE', "${dtestDockerImage}", 'Docker image
for running dtests')
}
axes {
@@ -895,7 +895,7 @@ matrixJob('Cassandra-devbranch-cqlsh-tests') {
stringParam('REPO', 'apache', 'The github user/org to clone cassandra
repo from')
stringParam('BRANCH', 'trunk', 'The branch of
[jira] [Comment Edited] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220921#comment-17220921
]
Sam Tunnicliffe edited comment on CASSANDRA-15299 at 10/27/20, 9:31 AM:
Here are some numbers from running tlp-stress against 4.0-beta2 vs the 15299
branch {{70923bae4}} with protocol v4 & v5.
The dataset was sized to be mostly in-memory and reading/writing at
{{CL.ONE}}. Each workload ran for 10 minutes, with a 1m warmup, on a 3 node
cluster.
Both latency and throughput are pretty much on a par, with v5 showing a bit of
an improvement where the workload is amenable to compression.
Memory usage and GC were pretty much the same too. If anything, during the v5
runs the servers spent less time in GC, but this was so close as not be
significant.
There are definitely some places we can improve the performance of v5, and I
don't think anything here indicates a serious regression in either v4 or v5
performance. Given the additional integrity checks in v5, I think not
regressing meets the perf bar here, at least in the first instance.
h4. 100% read workload, reads a single row at a time.
{code:java}
Count Latency (p99) 1min (req/s)
4.0-beta2 V4135449878 72.43 224222.86
15299 V4138424112 68.3 229765.45
15299 V5137618437 70.35 231602.36
4.0-beta2 V4 LZ4103348953 66.68 173437.38
15299 V4 LZ4105114560 68.83 176192.36
15299 V5 LZ4131833462 70.19 222092.99
{code}
h4. Mixed r/w workload (50/50). K/V with blobs upto 65k
{code:java}
Count Latency (p99) 1min (req/s) | Count
Latency (p99) 1min (req/s)
4.0-beta2 V434455009 76.59 56557.78 | 34464217
74.85 56546.40
15299 V433368171 74.90 54859.94 | 33361940
67.77 54848.57
15299 V532991815 76.00 54780.74 | 33001153
76.72 54856.99
4.0-beta2 V4 LZ432152220 83.41 53306.03 | 32147206
83.22 53334.00
15299 V4 LZ431158895 71.01 51106.60 | 31153453
72.75 51087.01
15299 V5 LZ432634296 75.73 54370.71 | 32644765
76.70 54396.15
{code}
h4. Mixed r/w/d workload (60/30/10). Wide rows with values up to 200b and
slicing on both the selects and deletions.
{code:java}
Count Latency (p99) 1min (req/s) | Count
Latency (p99) 1min (req/s) | Count Latency (p99) 1min (req/s)
4.0-beta2 V418725688 197.47 25377.16 | 9357971
193.86 12687.94 | 3117394 178.44 4221.90
15299 V417975636 185.88 24160.78 | 8986125
184.97 12087.13 | 2995562 179.99 4023.88
15299 V518429252 192.91 25349.35 | 9223277
188.15 12678.46 | 3073312 184.24 4232.23
4.0-beta2 V4 LZ418407719 179.94 25160.16 | 9197664
178.25 12575.03 | 3068134 180.90 4195.38
15299 V4 LZ417678994 171.39 24073.09 | 8842952
196.06 12064.18 | 2947344 170.53 4026.37
15299 V5 LZ418274085 208.57 25127.02 | 9138491
163.23 12558.28 | 3045264 203.54 4188.88
{code}
was (Author: beobal):
Here are some numbers from running tlp-stress against 4.0-beta2 vs the 15299
branch {{70923bae4}} with protocol v4 & v5.
The dataset was sized to be mostly in-memory and reading/writing at {{CL.ONE}}.
Each workload ran for 10 minutes, with a 1m warmup, on a 3 node cluster.
Both latency and throughput are pretty much on a par, with v5 showing a bit of
an improvement where the workload is amenable to compression.
Memory usage and GC were pretty much the same too. If anything, during the v5
runs the servers spent less time in GC, but this was so close as not be
significant.
There are definitely some places we can improve the performance of v5, and I
don't think anything here indicates a serious regression in either v4 or v5
performance. Given the additional integrity checks in v5, I think not
regressing meets the perf bar here, at least in the first instance.
h4. 100% read workload, reads a single row at a time.
{code}
Count Latency (p99) 1min (req/s)
4.0-beta2 V4135449878 72.43 224222.86
15299 V4138424112 68.3 229765.45
15299 V5137618437 70.35 231602.36
4.0-beta2 V4 LZ4103348953 66.68 173437.38
15299 V4 LZ4105114560 68.83
[jira] [Updated] (CASSANDRA-16166) Rename master branch to trunk in cassandra-dtest
[ https://issues.apache.org/jira/browse/CASSANDRA-16166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16166: --- Status: Ready to Commit (was: Review In Progress) > Rename master branch to trunk in cassandra-dtest > > > Key: CASSANDRA-16166 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16166 > Project: Cassandra > Issue Type: Sub-task > Components: Build >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16166) Rename master branch to trunk in cassandra-dtest
[ https://issues.apache.org/jira/browse/CASSANDRA-16166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221242#comment-17221242 ] Michael Semb Wever commented on CASSANDRA-16166: Created INFRA-21033 > Rename master branch to trunk in cassandra-dtest > > > Key: CASSANDRA-16166 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16166 > Project: Cassandra > Issue Type: Sub-task > Components: Build >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
