[jira] [Updated] (CASSANDRA-15616) Expose Cassandra related system properties in a virtual table
[ https://issues.apache.org/jira/browse/CASSANDRA-15616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15616: --- Fix Version/s: (was: 4.x) 4.0-alpha Source Control Link: https://github.com/apache/cassandra/commit/78deb3e6df31e81f1da40a09d7c40f3d4557ff75 Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed as 78deb3e6df31e81f1da40a09d7c40f3d4557ff75 > Expose Cassandra related system properties in a virtual table > - > > Key: CASSANDRA-15616 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15616 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Virtual Tables >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: High > Fix For: 4.0-alpha > > > Allow viewing Cassandra related system properties. > Viewing yaml config settings, as implemented by {{SettingsTable}} in > CASSANDRA-14573, is not enough for certain third-party tool use-cases. There > are a number of system properties that can be defined that contribute to the > configuration set that Cassandra runs on. > An example of such a use-case is described in CASSANDRA-15339. JMX ports need > to be discovered, especially in 4.0 where multiple nodes can run on one > ipaddress. Unlike the native port that is through gossip discoverable, the > jmx port is not. The jmx port is also not available via the SettingsTable. > With this patch and {{SystemPropertiesTable}} it becomes possible to first > discover all the native ports in the cluster and against each then discover > each node's jmx port. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: ninja-fix: CHANGES.txt for `Implement Virtual Table exposing Cassandra relevant system properties (CASSANDRA-15616)`
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new 250ccb5 ninja-fix: CHANGES.txt for `Implement Virtual Table exposing Cassandra relevant system properties (CASSANDRA-15616)` 250ccb5 is described below commit 250ccb5589cd8b8148d737ff8e30657cf3651ca6 Author: Mick Semb Wever AuthorDate: Thu Mar 5 08:40:34 2020 +0100 ninja-fix: CHANGES.txt for `Implement Virtual Table exposing Cassandra relevant system properties (CASSANDRA-15616)` --- CHANGES.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGES.txt b/CHANGES.txt index a07ff91..1fbf1df 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 4.0-alpha4 + * Added Virtual Table exposing Cassandra relevant system properties (CASSANDRA-15616) * Add data modeling introduction (CASSANDRA-15481) * Improve the algorithmic token allocation in case racks = RF (CASSANDRA-15600) * Fix ConnectionTest.testAcquireReleaseOutbound (CASSANDRA-15308) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Implement Virtual Table exposing Cassandra relevant system properties.
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new 78deb3e Implement Virtual Table exposing Cassandra relevant system properties. 78deb3e is described below commit 78deb3e6df31e81f1da40a09d7c40f3d4557ff75 Author: Mick Semb Wever AuthorDate: Mon Mar 2 14:15:47 2020 +0100 Implement Virtual Table exposing Cassandra relevant system properties. patch by Mick Semb Wever; reviewed by Chris Lohfink for CASSANDRA-15616 --- .../db/virtual/SystemPropertiesTable.java | 111 + .../cassandra/db/virtual/SystemViewsKeyspace.java | 1 + .../db/virtual/SystemPropertiesTableTest.java | 85 3 files changed, 197 insertions(+) diff --git a/src/java/org/apache/cassandra/db/virtual/SystemPropertiesTable.java b/src/java/org/apache/cassandra/db/virtual/SystemPropertiesTable.java new file mode 100644 index 000..864b11b --- /dev/null +++ b/src/java/org/apache/cassandra/db/virtual/SystemPropertiesTable.java @@ -0,0 +1,111 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.cassandra.db.virtual; + +import java.util.Set; + +import com.google.common.collect.Sets; + +import org.apache.cassandra.config.Config; +import org.apache.cassandra.db.DecoratedKey; +import org.apache.cassandra.db.marshal.*; +import org.apache.cassandra.dht.LocalPartitioner; +import org.apache.cassandra.schema.TableMetadata; + +final class SystemPropertiesTable extends AbstractVirtualTable +{ +private static final String NAME = "name"; +private static final String VALUE = "value"; + +private static final Set CASSANDRA_RELEVANT_PROPERTIES = Sets.newHashSet( +// base jvm properties +"java.home", +"java.io.tmpdir", +"java.library.path", +"java.security.egd", +"java.version", +"java.vm.name", +"line.separator", +"os.arch", +"os.name", +"user.home", +"sun.arch.data.model", +// jmx properties +"java.rmi.server.hostname", +"com.sun.management.jmxremote.authenticate", +"com.sun.management.jmxremote.rmi.port", +"com.sun.management.jmxremote.ssl", +"com.sun.management.jmxremote.ssl.need.client.auth", +"com.sun.management.jmxremote.access.file", +"com.sun.management.jmxremote.password.file", +"com.sun.management.jmxremote.port", +"com.sun.management.jmxremote.ssl.enabled.protocols", +"com.sun.management.jmxremote.ssl.enabled.cipher.suites", +"mx4jaddress", +"mx4jport", +// cassandra properties (without the "cassandra." prefix) +"cassandra-foreground", +"cassandra-pidfile", +"default.provide.overlapping.tombstones", +"org.apache.cassandra.disable_mbean_registration" +); + +SystemPropertiesTable(String keyspace) +{ +super(TableMetadata.builder(keyspace, "system_properties") + .comment("Cassandra relevant system properties") + .kind(TableMetadata.Kind.VIRTUAL) + .partitioner(new LocalPartitioner(UTF8Type.instance)) + .addPartitionKeyColumn(NAME, UTF8Type.instance) + .addRegularColumn(VALUE, UTF8Type.instance) + .build()); +} + +public DataSet data() +{ +SimpleDataSet result = new SimpleDataSet(metadata()); + +System.getProperties().stringPropertyNames() +.stream() +.filter(SystemPropertiesTable::isCassandraRelevant) +.forEach(name -> addRow(result, name, System.getProperty(name))); + +return result; +} + +@Override +public DataSet data(DecoratedKey partitionKey) +{ +SimpleDataSet result = new SimpleDataSet(metadata()); +String name = UTF8Type.instance.compose(partit
[jira] [Updated] (CASSANDRA-15616) Expose Cassandra related system properties in a virtual table
[ https://issues.apache.org/jira/browse/CASSANDRA-15616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15616: --- Status: Ready to Commit (was: Review In Progress) > Expose Cassandra related system properties in a virtual table > - > > Key: CASSANDRA-15616 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15616 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Virtual Tables >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: High > Fix For: 4.x > > > Allow viewing Cassandra related system properties. > Viewing yaml config settings, as implemented by {{SettingsTable}} in > CASSANDRA-14573, is not enough for certain third-party tool use-cases. There > are a number of system properties that can be defined that contribute to the > configuration set that Cassandra runs on. > An example of such a use-case is described in CASSANDRA-15339. JMX ports need > to be discovered, especially in 4.0 where multiple nodes can run on one > ipaddress. Unlike the native port that is through gossip discoverable, the > jmx port is not. The jmx port is also not available via the SettingsTable. > With this patch and {{SystemPropertiesTable}} it becomes possible to first > discover all the native ports in the cluster and against each then discover > each node's jmx port. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-8557) Default install from tarball is a bit puzzling
[ https://issues.apache.org/jira/browse/CASSANDRA-8557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CASSANDRA-8557: -- Labels: doc-impacting documentation easyfix pull-request-available (was: doc-impacting documentation easyfix) > Default install from tarball is a bit puzzling > -- > > Key: CASSANDRA-8557 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8557 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools, Local/Config > Environment: Tested with Crunchbang Waldorf built on Debian 7 Wheezy. > Java version: > {noformat} > $ java -version > java version "1.8.0_25" > Java(TM) SE Runtime Environment (build 1.8.0_25-b17) > Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode) > {noformat} > Python version: > {noformat} > $ python --version > Python 2.7.3 > {noformat} > I suspect this applies to all *nix tarball installs. >Reporter: Chris E >Assignee: Tanaka >Priority: Low > Labels: doc-impacting, documentation, easyfix, > pull-request-available > > The default tarball installation of Apache Cassandra is kind of confusing for > a newcomer. > There are several different points of confusion: > 1. The default installation doesn't allow remote connections. > 2. The config file (cassandra.yaml) specifies the use of the > AllowAllAuthenticator. This is almost immmediately replaced during setup by > the PasswordAuthenticator. Why not start there? > 3. The config file (cassandra.yaml) specifies the use of the > AllowAllAuthorizer. This is almost immediately replaced during setup by the > CassandraAuthorizer. Why not start there? > 4. Why does cassandra-cli exist? It even tells the user "This is being > deprecated." It's confusing figuring out whether to use cqlsh or > cassandra-cli. > 5. Running the cassandra script creates a background job that keeps > running--if you control-c the script, the process continues running. > 6. The config file (cassandra.yaml) has rpc_interface and rpc_address, and > the docs there don't spell out that that address is *also* required for using > remote logins from cqlsh. > 7. On a freshly-created VM, the localhost flag to rpc_address doesn't appear > to work (though this may be due to a misconfiguration of the VM). It seems to > need the assigned IP address of the VM to get it accepting external > connections. > 8. The config file (cassandra.yaml) has the request_scheduler using > NoScheduler--which is fine, but the docs are unclear as to whether or not > this means that client requests won't be scheduled (at all). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15616) Expose Cassandra related system properties in a virtual table
[ https://issues.apache.org/jira/browse/CASSANDRA-15616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051755#comment-17051755 ] Chris Lohfink commented on CASSANDRA-15616: --- lgtm +1 > Expose Cassandra related system properties in a virtual table > - > > Key: CASSANDRA-15616 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15616 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Virtual Tables >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: High > Fix For: 4.x > > > Allow viewing Cassandra related system properties. > Viewing yaml config settings, as implemented by {{SettingsTable}} in > CASSANDRA-14573, is not enough for certain third-party tool use-cases. There > are a number of system properties that can be defined that contribute to the > configuration set that Cassandra runs on. > An example of such a use-case is described in CASSANDRA-15339. JMX ports need > to be discovered, especially in 4.0 where multiple nodes can run on one > ipaddress. Unlike the native port that is through gossip discoverable, the > jmx port is not. The jmx port is also not available via the SettingsTable. > With this patch and {{SystemPropertiesTable}} it becomes possible to first > discover all the native ports in the cluster and against each then discover > each node's jmx port. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15613) Flakey test - TestBootstrap.test_local_quorum_bootstrap
[ https://issues.apache.org/jira/browse/CASSANDRA-15613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051693#comment-17051693 ] Ekaterina Dimitrova commented on CASSANDRA-15613: - LGTM +1 > Flakey test - TestBootstrap.test_local_quorum_bootstrap > --- > > Key: CASSANDRA-15613 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15613 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Ekaterina Dimitrova >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.0-alpha > > > Flakey test on trunk, java 8. > I did a couple of loops and it sometimes fail at the 5th run, sometimes at > the 50th. > > {noformat} > *_ > TestBootstrap.test_local_quorum_bootstrap > __* > > self = > > *def test_local_quorum_bootstrap(self):* > *"""* > *Test that CL local_quorum works while a node is bootstrapping.* > *@jira_ticket CASSANDRA-8058* > *"""* > *cluster = self.cluster* > *cluster.populate([1, 1])* > *cluster.start()* > ** > *node1 = cluster.nodes['node1']* > *yaml_config = """* > *# Create the keyspace and table* > *keyspace: keyspace1* > *keyspace_definition: |* > *CREATE KEYSPACE keyspace1 WITH replication = \{'class': > 'NetworkTopologyStrategy', 'dc1': 1, 'dc2': 1};* > *table: users* > *table_definition:* > *CREATE TABLE users (* > *username text,* > *first_name text,* > *last_name text,* > *email text,* > *PRIMARY KEY(username)* > *) WITH compaction = \{'class':'SizeTieredCompactionStrategy'};* > *insert:* > *partitions: fixed(1)* > *batchtype: UNLOGGED* > *queries:* > *read:* > *cql: select * from users where username = ?* > *fields: samerow* > *"""* > *with tempfile.NamedTemporaryFile(mode='w+') as stress_config:* > *stress_config.write(yaml_config)* > *stress_config.flush()* > *node1.stress(['user', 'profile=' + stress_config.name, 'n=200K', > 'no-warmup',* > *'ops(insert=1)', '-rate', 'threads=10'])* > ** > *node3 = new_node(cluster, data_center='dc2')* > *node3.start(no_wait=True)* > *time.sleep(3)* > ** > *ntout = node1.nodetool('status').stdout* > *> assert re.search(r'UJ\s+' + node3.ip_addr, ntout), ntout* > *E AssertionError: Datacenter: dc1* > *E ===* > *E Status=Up/Down* > *E |/ State=Normal/Leaving/Joining/Moving* > *E -- Address Load Owns (effective) Host ID > Token Rack* > *E UN 127.0.0.1 76.82 KiB ? > 3056e1d1-eac6-493a-b5bc-f8fe703d8d4b -9223372036854775808 r1 * > *E* > *E Datacenter: dc2* > *E ===* > *E Status=Up/Down* > *E |/ State=Normal/Leaving/Joining/Moving* > *E -- Address Load Owns (effective) Host ID > Token Rack* > *E UN 127.0.0.2 76.83 KiB ? > ae607eb3-5601-4b6f-9d01-cf9401da5a4c -9223372036854775708 r1 * > *E* > *E* > *E assert None* > *E + where None = (('UJ\\s+' + > '127.0.0.3'), 'Datacenter: dc1\n===\nStatus=Up/Down\n|/ > State=Normal/Leaving/Joining/Moving\n-- Address Load O...Rack\nUN > 127.0.0.2 76.83 KiB ? ae607eb3-5601-4b6f-9d01-cf9401da5a4c > -9223372036854775708 r1 \n\n')* > *E + where = re.search* > *E + and '127.0.0.3' = 0x106530b10>.ip_addr* > > *bootstrap_test.py*:488: AssertionError > --- > Captured stdout setup > > 11:42:12,744 ccm DEBUG Log-watching thread starting. > - > Captured log setup > -
[jira] [Updated] (CASSANDRA-15613) Flakey test - TestBootstrap.test_local_quorum_bootstrap
[ https://issues.apache.org/jira/browse/CASSANDRA-15613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-15613: Reviewers: Ekaterina Dimitrova, Ekaterina Dimitrova (was: Ekaterina Dimitrova) Ekaterina Dimitrova, Ekaterina Dimitrova (was: Ekaterina Dimitrova) Status: Review In Progress (was: Patch Available) > Flakey test - TestBootstrap.test_local_quorum_bootstrap > --- > > Key: CASSANDRA-15613 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15613 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Ekaterina Dimitrova >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.0-alpha > > > Flakey test on trunk, java 8. > I did a couple of loops and it sometimes fail at the 5th run, sometimes at > the 50th. > > {noformat} > *_ > TestBootstrap.test_local_quorum_bootstrap > __* > > self = > > *def test_local_quorum_bootstrap(self):* > *"""* > *Test that CL local_quorum works while a node is bootstrapping.* > *@jira_ticket CASSANDRA-8058* > *"""* > *cluster = self.cluster* > *cluster.populate([1, 1])* > *cluster.start()* > ** > *node1 = cluster.nodes['node1']* > *yaml_config = """* > *# Create the keyspace and table* > *keyspace: keyspace1* > *keyspace_definition: |* > *CREATE KEYSPACE keyspace1 WITH replication = \{'class': > 'NetworkTopologyStrategy', 'dc1': 1, 'dc2': 1};* > *table: users* > *table_definition:* > *CREATE TABLE users (* > *username text,* > *first_name text,* > *last_name text,* > *email text,* > *PRIMARY KEY(username)* > *) WITH compaction = \{'class':'SizeTieredCompactionStrategy'};* > *insert:* > *partitions: fixed(1)* > *batchtype: UNLOGGED* > *queries:* > *read:* > *cql: select * from users where username = ?* > *fields: samerow* > *"""* > *with tempfile.NamedTemporaryFile(mode='w+') as stress_config:* > *stress_config.write(yaml_config)* > *stress_config.flush()* > *node1.stress(['user', 'profile=' + stress_config.name, 'n=200K', > 'no-warmup',* > *'ops(insert=1)', '-rate', 'threads=10'])* > ** > *node3 = new_node(cluster, data_center='dc2')* > *node3.start(no_wait=True)* > *time.sleep(3)* > ** > *ntout = node1.nodetool('status').stdout* > *> assert re.search(r'UJ\s+' + node3.ip_addr, ntout), ntout* > *E AssertionError: Datacenter: dc1* > *E ===* > *E Status=Up/Down* > *E |/ State=Normal/Leaving/Joining/Moving* > *E -- Address Load Owns (effective) Host ID > Token Rack* > *E UN 127.0.0.1 76.82 KiB ? > 3056e1d1-eac6-493a-b5bc-f8fe703d8d4b -9223372036854775808 r1 * > *E* > *E Datacenter: dc2* > *E ===* > *E Status=Up/Down* > *E |/ State=Normal/Leaving/Joining/Moving* > *E -- Address Load Owns (effective) Host ID > Token Rack* > *E UN 127.0.0.2 76.83 KiB ? > ae607eb3-5601-4b6f-9d01-cf9401da5a4c -9223372036854775708 r1 * > *E* > *E* > *E assert None* > *E + where None = (('UJ\\s+' + > '127.0.0.3'), 'Datacenter: dc1\n===\nStatus=Up/Down\n|/ > State=Normal/Leaving/Joining/Moving\n-- Address Load O...Rack\nUN > 127.0.0.2 76.83 KiB ? ae607eb3-5601-4b6f-9d01-cf9401da5a4c > -9223372036854775708 r1 \n\n')* > *E + where = re.search* > *E + and '127.0.0.3' = 0x106530b10>.ip_addr* > > *bootstrap_test.py*:488: AssertionError > --- > Captured stdout setup > > 11:42:12,744 ccm DEBUG Log-watching thread starting. > --
[jira] [Updated] (CASSANDRA-15613) Flakey test - TestBootstrap.test_local_quorum_bootstrap
[ https://issues.apache.org/jira/browse/CASSANDRA-15613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-15613: Status: Ready to Commit (was: Review In Progress) > Flakey test - TestBootstrap.test_local_quorum_bootstrap > --- > > Key: CASSANDRA-15613 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15613 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Ekaterina Dimitrova >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.0-alpha > > > Flakey test on trunk, java 8. > I did a couple of loops and it sometimes fail at the 5th run, sometimes at > the 50th. > > {noformat} > *_ > TestBootstrap.test_local_quorum_bootstrap > __* > > self = > > *def test_local_quorum_bootstrap(self):* > *"""* > *Test that CL local_quorum works while a node is bootstrapping.* > *@jira_ticket CASSANDRA-8058* > *"""* > *cluster = self.cluster* > *cluster.populate([1, 1])* > *cluster.start()* > ** > *node1 = cluster.nodes['node1']* > *yaml_config = """* > *# Create the keyspace and table* > *keyspace: keyspace1* > *keyspace_definition: |* > *CREATE KEYSPACE keyspace1 WITH replication = \{'class': > 'NetworkTopologyStrategy', 'dc1': 1, 'dc2': 1};* > *table: users* > *table_definition:* > *CREATE TABLE users (* > *username text,* > *first_name text,* > *last_name text,* > *email text,* > *PRIMARY KEY(username)* > *) WITH compaction = \{'class':'SizeTieredCompactionStrategy'};* > *insert:* > *partitions: fixed(1)* > *batchtype: UNLOGGED* > *queries:* > *read:* > *cql: select * from users where username = ?* > *fields: samerow* > *"""* > *with tempfile.NamedTemporaryFile(mode='w+') as stress_config:* > *stress_config.write(yaml_config)* > *stress_config.flush()* > *node1.stress(['user', 'profile=' + stress_config.name, 'n=200K', > 'no-warmup',* > *'ops(insert=1)', '-rate', 'threads=10'])* > ** > *node3 = new_node(cluster, data_center='dc2')* > *node3.start(no_wait=True)* > *time.sleep(3)* > ** > *ntout = node1.nodetool('status').stdout* > *> assert re.search(r'UJ\s+' + node3.ip_addr, ntout), ntout* > *E AssertionError: Datacenter: dc1* > *E ===* > *E Status=Up/Down* > *E |/ State=Normal/Leaving/Joining/Moving* > *E -- Address Load Owns (effective) Host ID > Token Rack* > *E UN 127.0.0.1 76.82 KiB ? > 3056e1d1-eac6-493a-b5bc-f8fe703d8d4b -9223372036854775808 r1 * > *E* > *E Datacenter: dc2* > *E ===* > *E Status=Up/Down* > *E |/ State=Normal/Leaving/Joining/Moving* > *E -- Address Load Owns (effective) Host ID > Token Rack* > *E UN 127.0.0.2 76.83 KiB ? > ae607eb3-5601-4b6f-9d01-cf9401da5a4c -9223372036854775708 r1 * > *E* > *E* > *E assert None* > *E + where None = (('UJ\\s+' + > '127.0.0.3'), 'Datacenter: dc1\n===\nStatus=Up/Down\n|/ > State=Normal/Leaving/Joining/Moving\n-- Address Load O...Rack\nUN > 127.0.0.2 76.83 KiB ? ae607eb3-5601-4b6f-9d01-cf9401da5a4c > -9223372036854775708 r1 \n\n')* > *E + where = re.search* > *E + and '127.0.0.3' = 0x106530b10>.ip_addr* > > *bootstrap_test.py*:488: AssertionError > --- > Captured stdout setup > > 11:42:12,744 ccm DEBUG Log-watching thread starting. > - > Captured log setup >
[jira] [Updated] (CASSANDRA-15622) Unit tests throw UnknownHostException trying to use `InetAddress.getLocalHost()` instead of `FBUtilities.getLocalAddress()`
[ https://issues.apache.org/jira/browse/CASSANDRA-15622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15622: --- Description: Many of the unit tests in Jenkins fail because of the use of {{`InetAddress.getLocalHost()`}} in the test classes. The Jenkins agents need a public ipaddress (and a hostname associated to it) so the Jenkins master can connect to them (agents can be hosted externally, by donating third-parties). The call to {{`InetAddress.getLocalHost()`}} then resolves this public hostname. This is not listed in {{`/etc/hosts`}}, and we don't want it to be either (in case of accidental external port opening). (This is also ASF policy on this infrastructure.) The unit test code needs to replace these code occurrences with the call to {{`FBUtilities.getLocalAddress()`}} was: Many of the unit tests in Jenkins fail because of the use of {{`InetAddress.getLocalHost()`}} in the test classes. The Jenkins agents needs a public ipaddress (and a hostname associated to it) so the Jenkins master can connect to them (agents can be hosted externally, by donating third-parties). The call to {{`InetAddress.getLocalHost()`}} then resolves this public hostname. This is not listed in {{`/etc/hosts`}}, and we don't want it to be either (in case of accidental external port opening). (This is also ASF policy on this infrastructure.) The unit test code needs to replace these code occurrences with the call to {{`FBUtilities.getLocalAddress()`}} > Unit tests throw UnknownHostException trying to use > `InetAddress.getLocalHost()` instead of `FBUtilities.getLocalAddress()` > --- > > Key: CASSANDRA-15622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15622 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x > > > Many of the unit tests in Jenkins fail because of the use of > {{`InetAddress.getLocalHost()`}} in the test classes. > The Jenkins agents need a public ipaddress (and a hostname associated to it) > so the Jenkins master can connect to them (agents can be hosted externally, > by donating third-parties). > The call to {{`InetAddress.getLocalHost()`}} then resolves this public > hostname. > This is not listed in {{`/etc/hosts`}}, and we don't want it to be either (in > case of accidental external port opening). (This is also ASF policy on this > infrastructure.) > The unit test code needs to replace these code occurrences with the call to > {{`FBUtilities.getLocalAddress()`}} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15616) Expose Cassandra related system properties in a virtual table
[ https://issues.apache.org/jira/browse/CASSANDRA-15616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051667#comment-17051667 ] Michael Semb Wever commented on CASSANDRA-15616: Thanks [~cnlwsu]. All comments addressed. (Forced pushed the same branch.) > Expose Cassandra related system properties in a virtual table > - > > Key: CASSANDRA-15616 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15616 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Virtual Tables >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: High > Fix For: 4.x > > > Allow viewing Cassandra related system properties. > Viewing yaml config settings, as implemented by {{SettingsTable}} in > CASSANDRA-14573, is not enough for certain third-party tool use-cases. There > are a number of system properties that can be defined that contribute to the > configuration set that Cassandra runs on. > An example of such a use-case is described in CASSANDRA-15339. JMX ports need > to be discovered, especially in 4.0 where multiple nodes can run on one > ipaddress. Unlike the native port that is through gossip discoverable, the > jmx port is not. The jmx port is also not available via the SettingsTable. > With this patch and {{SystemPropertiesTable}} it becomes possible to first > discover all the native ports in the cluster and against each then discover > each node's jmx port. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15466) DOC - Improve installation pages, include YUM procedure
[ https://issues.apache.org/jira/browse/CASSANDRA-15466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Ramirez updated CASSANDRA-15466: -- Reviewers: Jon Haddad > DOC - Improve installation pages, include YUM procedure > --- > > Key: CASSANDRA-15466 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15466 > Project: Cassandra > Issue Type: Improvement > Components: Documentation/Website >Reporter: Erick Ramirez >Assignee: Erick Ramirez >Priority: Normal > > h2. Background > The installation instructions are a little lacking. I'm proposing we add more > explicit steps which are targeted towards users who are very new to Cassandra. > h2. Proposal > We can improve the installation pages with explicit instructions for the > following: > * steps for installing older versions to match nodes in an existing cluster > * steps for installing RHEL/CentOS package with YUM > I am preparing a draft with details to follow. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15622) Unit tests throw UnknownHostException trying to use `InetAddress.getLocalHost()` instead of `FBUtilities.getLocalAddress()`
[ https://issues.apache.org/jira/browse/CASSANDRA-15622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051658#comment-17051658 ] Michael Semb Wever edited comment on CASSANDRA-15622 at 3/4/20, 10:10 PM: -- initial patch against [cassandra-2.2|https://github.com/apache/cassandra/compare/cassandra-2.2...thelastpickle:mck/cassandra-2.2_15622] was (Author: michaelsembwever): initial patch at [here|https://github.com/apache/cassandra/compare/cassandra-2.2...thelastpickle:mck/cassandra-2.2_15622] > Unit tests throw UnknownHostException trying to use > `InetAddress.getLocalHost()` instead of `FBUtilities.getLocalAddress()` > --- > > Key: CASSANDRA-15622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15622 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x > > > Many of the unit tests in Jenkins fail because of the use of > {{`InetAddress.getLocalHost()`}} in the test classes. > The Jenkins agents needs a public ipaddress (and a hostname associated to it) > so the Jenkins master can connect to them (agents can be hosted externally, > by donating third-parties). > The call to {{`InetAddress.getLocalHost()`}} then resolves this public > hostname. > This is not listed in {{`/etc/hosts`}}, and we don't want it to be either (in > case of accidental external port opening). (This is also ASF policy on this > infrastructure.) > The unit test code needs to replace these code occurrences with the call to > {{`FBUtilities.getLocalAddress()`}} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15622) Unit tests throw UnknownHostException trying to use `InetAddress.getLocalHost()` instead of `FBUtilities.getLocalAddress()`
[ https://issues.apache.org/jira/browse/CASSANDRA-15622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051658#comment-17051658 ] Michael Semb Wever commented on CASSANDRA-15622: initial patch at [here|https://github.com/apache/cassandra/compare/cassandra-2.2...thelastpickle:mck/cassandra-2.2_15622] > Unit tests throw UnknownHostException trying to use > `InetAddress.getLocalHost()` instead of `FBUtilities.getLocalAddress()` > --- > > Key: CASSANDRA-15622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15622 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x > > > Many of the unit tests in Jenkins fail because of the use of > {{`InetAddress.getLocalHost()`}} in the test classes. > The Jenkins agents needs a public ipaddress (and a hostname associated to it) > so the Jenkins master can connect to them (agents can be hosted externally, > by donating third-parties). > The call to {{`InetAddress.getLocalHost()`}} then resolves this public > hostname. > This is not listed in {{`/etc/hosts`}}, and we don't want it to be either (in > case of accidental external port opening). (This is also ASF policy on this > infrastructure.) > The unit test code needs to replace these code occurrences with the call to > {{`FBUtilities.getLocalAddress()`}} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15622) Unit tests throw UnknownHostException trying to use `InetAddress.getLocalHost()` instead of `FBUtilities.getLocalAddress()`
[ https://issues.apache.org/jira/browse/CASSANDRA-15622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15622: --- Bug Category: Parent values: Correctness(12982)Level 1 values: Test Failure(12990) Complexity: Low Hanging Fruit Discovered By: Unit Test Fix Version/s: 4.x 3.11.x 3.0.x 2.2.x Severity: Normal Status: Open (was: Triage Needed) > Unit tests throw UnknownHostException trying to use > `InetAddress.getLocalHost()` instead of `FBUtilities.getLocalAddress()` > --- > > Key: CASSANDRA-15622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15622 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x > > > Many of the unit tests in Jenkins fail because of the use of > {{`InetAddress.getLocalHost()`}} in the test classes. > The Jenkins agents needs a public ipaddress (and a hostname associated to it) > so the Jenkins master can connect to them (agents can be hosted externally, > by donating third-parties). > The call to {{`InetAddress.getLocalHost()`}} then resolves this public > hostname. > This is not listed in {{`/etc/hosts`}}, and we don't want it to be either (in > case of accidental external port opening). (This is also ASF policy on this > infrastructure.) > The unit test code needs to replace these code occurrences with the call to > {{`FBUtilities.getLocalAddress()`}} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15564) Refactor repair coordinator so errors are consistent
[ https://issues.apache.org/jira/browse/CASSANDRA-15564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051648#comment-17051648 ] David Capwell commented on CASSANDRA-15564: --- Pushed a new [commit|https://github.com/dcapwell/cassandra/commit/641a31ec76e575a66ecda714f9059dbbdef1ff9b] which will support inbound/outbound filters, this allows preview repair test to block sending (behavior on trunk). [~ifesdjeen] can you review this commit? Since the changes are localized to jvm dtests I am relying on Circle CI to validate both branches. > Refactor repair coordinator so errors are consistent > > > Key: CASSANDRA-15564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15564 > Project: Cassandra > Issue Type: Sub-task > Components: Consistency/Repair >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Time Spent: 11h 10m > Remaining Estimate: 0h > > This is to split the change in CASSANDRA-15399 so the refactor is isolated > out. > Currently the repair coordinator special cases the exit cases at each call > site; this makes it so that errors can be inconsistent and there are cases > where proper complete isn't done (proper notifications, and forgetting to > update ActiveRepairService). > [Circle > CI|https://circleci.com/gh/dcapwell/cassandra/tree/bug%2FrepairCoordinatorJmxConsistency] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15372) Add Jon Haddad's GPG key to KEYS file
[ https://issues.apache.org/jira/browse/CASSANDRA-15372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Haddad updated CASSANDRA-15372: --- Change Category: Quality Assurance Complexity: Low Hanging Fruit Component/s: Packaging Status: Open (was: Triage Needed) > Add Jon Haddad's GPG key to KEYS file > - > > Key: CASSANDRA-15372 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15372 > Project: Cassandra > Issue Type: Task > Components: Packaging >Reporter: Jon Haddad >Assignee: Jon Haddad >Priority: Normal > > Following up CASSANDRA-15360 and the mailing list discussion, I'll add my GPG > key to the keys file. > References: > - [https://www.apache.org/dev/release-signing#keys-policy] > - [http://www.apache.org/legal/release-policy.html] > - [dev ML thread "Improving our frequency of (patch) releases, and letting > committers make > releases"|https://lists.apache.org/thread.html/660ed8c73e7b79afa610f3e45f37914ef43a4358a85a99c8b4b0288a@%3Cdev.cassandra.apache.org%3E] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15597) Correct Visibility and Improve Safety of Methods in LatencyMetrics
[ https://issues.apache.org/jira/browse/CASSANDRA-15597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051611#comment-17051611 ] Jordan West commented on CASSANDRA-15597: - [~Jeff] patch LGTM. Do you want to also include the visibility changes to add/removeChildren? Thanks. > Correct Visibility and Improve Safety of Methods in LatencyMetrics > -- > > Key: CASSANDRA-15597 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15597 > Project: Cassandra > Issue Type: Improvement > Components: Observability/Metrics >Reporter: Jordan West >Assignee: Jeff >Priority: Normal > Fix For: 4.0 > > > * add/removeChildren does not need to be public (and exposing addChildren is > unsafe since no lock is used). > * casting in the constructor is safer than casting each time in removeChildren -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15616) Expose Cassandra related system properties in a virtual table
[ https://issues.apache.org/jira/browse/CASSANDRA-15616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051589#comment-17051589 ] Chris Lohfink commented on CASSANDRA-15616: --- I walked through repo and your whitelist captured everything I could find. I would actually like to see removing `"java.class.path"` simply because it's so long makes the output unreadable without `EXPAND ON` and its pretty limited value. Added some small comments in https://github.com/apache/cassandra/commit/b226d3014b7cb13eb61b47bda2185fc1ccf0b1ef > Expose Cassandra related system properties in a virtual table > - > > Key: CASSANDRA-15616 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15616 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Virtual Tables >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: High > Fix For: 4.x > > > Allow viewing Cassandra related system properties. > Viewing yaml config settings, as implemented by {{SettingsTable}} in > CASSANDRA-14573, is not enough for certain third-party tool use-cases. There > are a number of system properties that can be defined that contribute to the > configuration set that Cassandra runs on. > An example of such a use-case is described in CASSANDRA-15339. JMX ports need > to be discovered, especially in 4.0 where multiple nodes can run on one > ipaddress. Unlike the native port that is through gossip discoverable, the > jmx port is not. The jmx port is also not available via the SettingsTable. > With this patch and {{SystemPropertiesTable}} it becomes possible to first > discover all the native ports in the cluster and against each then discover > each node's jmx port. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15353) GSoD Documentation - Cassandra 4.0
[ https://issues.apache.org/jira/browse/CASSANDRA-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Haddad updated CASSANDRA-15353: --- Status: Ready to Commit (was: Review In Progress) > GSoD Documentation - Cassandra 4.0 > -- > > Key: CASSANDRA-15353 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15353 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: Deepak Vohra >Priority: Normal > Fix For: 4.0-rc > > > Please review, and add comments to, some of the documentation preview for > Apache Cassandra 4.0. > New Features > --- > *Audit Logging* > https://docs.google.com/document/d/1WsQYjDQ9UZ7f8P3eAATht6Vz5oNibTmhsH8cu9viLTk/edit?usp=sharing > *Full Query Logging* > https://docs.google.com/document/d/1ZNfs68ZNJXqDc_2XGM6W-tpvnejUfSh8HmkbG8u35Es/edit?usp=sharing > *Support for Java 11* > https://docs.google.com/document/d/1v7ffccqk_5Son4iwfuwZae8YUam41quewKi_6Z_PQGw/edit?usp=sharing > *Transient Replication and Cheap Quorums* > https://docs.google.com/document/d/1GyJVCmwoMb7OO9NkbGdk26YxHi9fWtfzuTBTiIEpli8/edit?usp=sharing > *Virtual Tables* > https://docs.google.com/document/d/1U8qAzQnA2HcoIzTg5IXFUySUZXD1tEIl2bam-514we8/edit?usp=sharing > Improvements > > *Improved Streaming* > https://docs.google.com/document/d/1zd6AuHBgC82v598cuDtVIkxJVDV9c0A2rrMEVFHyB4Y/edit?usp=sharing > *Improved Internode Messaging* > https://docs.google.com/document/d/1ub2DBHE7hNEKe4tuCOpdbCZ7qGqCdMAYvSTC-YamuMA/edit?usp=sharing -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15353) GSoD Documentation - Cassandra 4.0
[ https://issues.apache.org/jira/browse/CASSANDRA-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Haddad updated CASSANDRA-15353: --- Source Control Link: https://github.com/apache/cassandra/commit/4585bbd131aaab367ceee6833ca1e8287724a5b1 Resolution: Fixed Status: Resolved (was: Ready to Commit) Last commit added as source control link. > GSoD Documentation - Cassandra 4.0 > -- > > Key: CASSANDRA-15353 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15353 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: Deepak Vohra >Priority: Normal > Fix For: 4.0-rc > > > Please review, and add comments to, some of the documentation preview for > Apache Cassandra 4.0. > New Features > --- > *Audit Logging* > https://docs.google.com/document/d/1WsQYjDQ9UZ7f8P3eAATht6Vz5oNibTmhsH8cu9viLTk/edit?usp=sharing > *Full Query Logging* > https://docs.google.com/document/d/1ZNfs68ZNJXqDc_2XGM6W-tpvnejUfSh8HmkbG8u35Es/edit?usp=sharing > *Support for Java 11* > https://docs.google.com/document/d/1v7ffccqk_5Son4iwfuwZae8YUam41quewKi_6Z_PQGw/edit?usp=sharing > *Transient Replication and Cheap Quorums* > https://docs.google.com/document/d/1GyJVCmwoMb7OO9NkbGdk26YxHi9fWtfzuTBTiIEpli8/edit?usp=sharing > *Virtual Tables* > https://docs.google.com/document/d/1U8qAzQnA2HcoIzTg5IXFUySUZXD1tEIl2bam-514we8/edit?usp=sharing > Improvements > > *Improved Streaming* > https://docs.google.com/document/d/1zd6AuHBgC82v598cuDtVIkxJVDV9c0A2rrMEVFHyB4Y/edit?usp=sharing > *Improved Internode Messaging* > https://docs.google.com/document/d/1ub2DBHE7hNEKe4tuCOpdbCZ7qGqCdMAYvSTC-YamuMA/edit?usp=sharing -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15353) GSoD Documentation - Cassandra 4.0
[ https://issues.apache.org/jira/browse/CASSANDRA-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Haddad updated CASSANDRA-15353: --- Reviewers: Dinesh Joshi, Jon Haddad, Jon Haddad (was: Dinesh Joshi, Jon Haddad) Dinesh Joshi, Jon Haddad, Jon Haddad (was: Dinesh Joshi) Status: Review In Progress (was: Patch Available) > GSoD Documentation - Cassandra 4.0 > -- > > Key: CASSANDRA-15353 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15353 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: Deepak Vohra >Priority: Normal > Fix For: 4.0-rc > > > Please review, and add comments to, some of the documentation preview for > Apache Cassandra 4.0. > New Features > --- > *Audit Logging* > https://docs.google.com/document/d/1WsQYjDQ9UZ7f8P3eAATht6Vz5oNibTmhsH8cu9viLTk/edit?usp=sharing > *Full Query Logging* > https://docs.google.com/document/d/1ZNfs68ZNJXqDc_2XGM6W-tpvnejUfSh8HmkbG8u35Es/edit?usp=sharing > *Support for Java 11* > https://docs.google.com/document/d/1v7ffccqk_5Son4iwfuwZae8YUam41quewKi_6Z_PQGw/edit?usp=sharing > *Transient Replication and Cheap Quorums* > https://docs.google.com/document/d/1GyJVCmwoMb7OO9NkbGdk26YxHi9fWtfzuTBTiIEpli8/edit?usp=sharing > *Virtual Tables* > https://docs.google.com/document/d/1U8qAzQnA2HcoIzTg5IXFUySUZXD1tEIl2bam-514we8/edit?usp=sharing > Improvements > > *Improved Streaming* > https://docs.google.com/document/d/1zd6AuHBgC82v598cuDtVIkxJVDV9c0A2rrMEVFHyB4Y/edit?usp=sharing > *Improved Internode Messaging* > https://docs.google.com/document/d/1ub2DBHE7hNEKe4tuCOpdbCZ7qGqCdMAYvSTC-YamuMA/edit?usp=sharing -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15353) GSoD Documentation - Cassandra 4.0
[ https://issues.apache.org/jira/browse/CASSANDRA-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Haddad updated CASSANDRA-15353: --- Test and Documentation Plan: Reviewed by reading Status: Patch Available (was: In Progress) > GSoD Documentation - Cassandra 4.0 > -- > > Key: CASSANDRA-15353 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15353 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: Deepak Vohra >Priority: Normal > Fix For: 4.0-rc > > > Please review, and add comments to, some of the documentation preview for > Apache Cassandra 4.0. > New Features > --- > *Audit Logging* > https://docs.google.com/document/d/1WsQYjDQ9UZ7f8P3eAATht6Vz5oNibTmhsH8cu9viLTk/edit?usp=sharing > *Full Query Logging* > https://docs.google.com/document/d/1ZNfs68ZNJXqDc_2XGM6W-tpvnejUfSh8HmkbG8u35Es/edit?usp=sharing > *Support for Java 11* > https://docs.google.com/document/d/1v7ffccqk_5Son4iwfuwZae8YUam41quewKi_6Z_PQGw/edit?usp=sharing > *Transient Replication and Cheap Quorums* > https://docs.google.com/document/d/1GyJVCmwoMb7OO9NkbGdk26YxHi9fWtfzuTBTiIEpli8/edit?usp=sharing > *Virtual Tables* > https://docs.google.com/document/d/1U8qAzQnA2HcoIzTg5IXFUySUZXD1tEIl2bam-514we8/edit?usp=sharing > Improvements > > *Improved Streaming* > https://docs.google.com/document/d/1zd6AuHBgC82v598cuDtVIkxJVDV9c0A2rrMEVFHyB4Y/edit?usp=sharing > *Improved Internode Messaging* > https://docs.google.com/document/d/1ub2DBHE7hNEKe4tuCOpdbCZ7qGqCdMAYvSTC-YamuMA/edit?usp=sharing -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15481) Data Modeling
[ https://issues.apache.org/jira/browse/CASSANDRA-15481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Haddad updated CASSANDRA-15481: --- Status: Ready to Commit (was: Review In Progress) > Data Modeling > - > > Key: CASSANDRA-15481 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15481 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > Labels: pull-request-available > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Added a page on data modeling. > https://github.com/apache/cassandra/pull/410 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15481) Data Modeling
[ https://issues.apache.org/jira/browse/CASSANDRA-15481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Haddad updated CASSANDRA-15481: --- Reviewers: Erick Ramirez, Jon Haddad, Jon Haddad (was: Erick Ramirez, Jon Haddad) Erick Ramirez, Jon Haddad, Jon Haddad (was: Erick Ramirez, Jon Haddad) Status: Review In Progress (was: Patch Available) > Data Modeling > - > > Key: CASSANDRA-15481 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15481 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > Labels: pull-request-available > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Added a page on data modeling. > https://github.com/apache/cassandra/pull/410 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15481) Data Modeling
[ https://issues.apache.org/jira/browse/CASSANDRA-15481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Haddad updated CASSANDRA-15481: --- Fix Version/s: (was: 4.0) 4.0-alpha Source Control Link: https://github.com/apache/cassandra/commit/4585bbd131aaab367ceee6833ca1e8287724a5b1 Resolution: Fixed Status: Resolved (was: Ready to Commit) > Data Modeling > - > > Key: CASSANDRA-15481 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15481 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-alpha > > Time Spent: 10m > Remaining Estimate: 0h > > Added a page on data modeling. > https://github.com/apache/cassandra/pull/410 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Added intro to data modeling section.
This is an automated email from the ASF dual-hosted git repository. rustyrazorblade pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new 4585bbd Added intro to data modeling section. 4585bbd is described below commit 4585bbd131aaab367ceee6833ca1e8287724a5b1 Author: dvohra AuthorDate: Wed Mar 4 11:37:56 2020 + Added intro to data modeling section. Patch by dvohra; Reviewed by Erick Ramirez and Jon Haddad for CASSANDRA-15481 --- CHANGES.txt| 1 + .../data_modeling/images/Figure_1_data_model.jpg | Bin 0 -> 17469 bytes .../data_modeling/images/Figure_2_data_model.jpg | Bin 0 -> 20925 bytes doc/source/data_modeling/index.rst | 1 + doc/source/data_modeling/intro.rst | 146 + 5 files changed, 148 insertions(+) diff --git a/CHANGES.txt b/CHANGES.txt index 072b7c3..a07ff91 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 4.0-alpha4 + * Add data modeling introduction (CASSANDRA-15481) * Improve the algorithmic token allocation in case racks = RF (CASSANDRA-15600) * Fix ConnectionTest.testAcquireReleaseOutbound (CASSANDRA-15308) * Include finalized pending sstables in preview repair (CASSANDRA-15553) diff --git a/doc/source/data_modeling/images/Figure_1_data_model.jpg b/doc/source/data_modeling/images/Figure_1_data_model.jpg new file mode 100644 index 000..a3b330e Binary files /dev/null and b/doc/source/data_modeling/images/Figure_1_data_model.jpg differ diff --git a/doc/source/data_modeling/images/Figure_2_data_model.jpg b/doc/source/data_modeling/images/Figure_2_data_model.jpg new file mode 100644 index 000..7acdeac Binary files /dev/null and b/doc/source/data_modeling/images/Figure_2_data_model.jpg differ diff --git a/doc/source/data_modeling/index.rst b/doc/source/data_modeling/index.rst index f01c92c..2f799dc 100644 --- a/doc/source/data_modeling/index.rst +++ b/doc/source/data_modeling/index.rst @@ -20,6 +20,7 @@ Data Modeling .. toctree:: :maxdepth: 2 + intro data_modeling_conceptual data_modeling_rdbms data_modeling_queries diff --git a/doc/source/data_modeling/intro.rst b/doc/source/data_modeling/intro.rst new file mode 100644 index 000..630a7d1 --- /dev/null +++ b/doc/source/data_modeling/intro.rst @@ -0,0 +1,146 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed under the License is distributed on an "AS IS" BASIS, +.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +.. See the License for the specific language governing permissions and +.. limitations under the License. + +Introduction + + +Apache Cassandra stores data in tables, with each table consisting of rows and columns. CQL (Cassandra Query Language) is used to query the data stored in tables. Apache Cassandra data model is based around and optimized for querying. Cassandra does not support relational data modeling intended for relational databases. + +What is Data Modeling? +^^ + +Data modeling is the process of identifying entities and their relationships. In relational databases, data is placed in normalized tables with foreign keys used to reference related data in other tables. Queries that the application will make are driven by the structure of the tables and related data are queried as table joins. + +In Cassandra, data modeling is query-driven. The data access patterns and application queries determine the structure and organization of data which then used to design the database tables. + +Data is modeled around specific queries. Queries are best designed to access a single table, which implies that all entities involved in a query must be in the same table to make data access (reads) very fast. Data is modeled to best suit a query or a set of queries. A table could have one or more entities as best suits a query. As entities do typically have relationships among them and queries could involve entities with relationships among them, a single entity may be included in multi [...] + +Query-driven modeling +^ + +Unlike a relational database model in which queries make use of table joins to get data from multiple tables, joins are not supported in Cassandra so all required fields (columns) must be grouped together in a single tabl
[jira] [Commented] (CASSANDRA-15564) Refactor repair coordinator so errors are consistent
[ https://issues.apache.org/jira/browse/CASSANDRA-15564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051538#comment-17051538 ] Alex Petrov commented on CASSANDRA-15564: - Wondering if we should have both in- and outbound- filters. For most of the purposes inbound are probably preferred, but it'll be a tiny addition to have both. We can do it in a separate ticket though. +1 on the dtest part. > Refactor repair coordinator so errors are consistent > > > Key: CASSANDRA-15564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15564 > Project: Cassandra > Issue Type: Sub-task > Components: Consistency/Repair >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Time Spent: 11h 10m > Remaining Estimate: 0h > > This is to split the change in CASSANDRA-15399 so the refactor is isolated > out. > Currently the repair coordinator special cases the exit cases at each call > site; this makes it so that errors can be inconsistent and there are cases > where proper complete isn't done (proper notifications, and forgetting to > update ActiveRepairService). > [Circle > CI|https://circleci.com/gh/dcapwell/cassandra/tree/bug%2FrepairCoordinatorJmxConsistency] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15622) Unit tests throw UnknownHostException trying to use `InetAddress.getLocalHost()` instead of `FBUtilities.getLocalAddress()`
[ https://issues.apache.org/jira/browse/CASSANDRA-15622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever reassigned CASSANDRA-15622: -- Assignee: Michael Semb Wever > Unit tests throw UnknownHostException trying to use > `InetAddress.getLocalHost()` instead of `FBUtilities.getLocalAddress()` > --- > > Key: CASSANDRA-15622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15622 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > > Many of the unit tests in Jenkins fail because of the use of > {{`InetAddress.getLocalHost()`}} in the test classes. > The Jenkins agents needs a public ipaddress (and a hostname associated to it) > so the Jenkins master can connect to them (agents can be hosted externally, > by donating third-parties). > The call to {{`InetAddress.getLocalHost()`}} then resolves this public > hostname. > This is not listed in {{`/etc/hosts`}}, and we don't want it to be either (in > case of accidental external port opening). (This is also ASF policy on this > infrastructure.) > The unit test code needs to replace these code occurrences with the call to > {{`FBUtilities.getLocalAddress()`}} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15622) Unit tests throw UnknownHostException trying to use `InetAddress.getLocalHost()` instead of `FBUtilities.getLocalAddress()`
Michael Semb Wever created CASSANDRA-15622: -- Summary: Unit tests throw UnknownHostException trying to use `InetAddress.getLocalHost()` instead of `FBUtilities.getLocalAddress()` Key: CASSANDRA-15622 URL: https://issues.apache.org/jira/browse/CASSANDRA-15622 Project: Cassandra Issue Type: Bug Components: Test/unit Reporter: Michael Semb Wever Many of the unit tests in Jenkins fail because of the use of {{`InetAddress.getLocalHost()`}} in the test classes. The Jenkins agents needs a public ipaddress (and a hostname associated to it) so the Jenkins master can connect to them (agents can be hosted externally, by donating third-parties). The call to {{`InetAddress.getLocalHost()`}} then resolves this public hostname. This is not listed in {{`/etc/hosts`}}, and we don't want it to be either (in case of accidental external port opening). (This is also ASF policy on this infrastructure.) The unit test code needs to replace these code occurrences with the call to {{`FBUtilities.getLocalAddress()`}} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15564) Refactor repair coordinator so errors are consistent
[ https://issues.apache.org/jira/browse/CASSANDRA-15564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051527#comment-17051527 ] Chris Lohfink commented on CASSANDRA-15564: --- I like response time between the two, since service time being bad would still show up in it, it would be able to flag a wider range of problems > Refactor repair coordinator so errors are consistent > > > Key: CASSANDRA-15564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15564 > Project: Cassandra > Issue Type: Sub-task > Components: Consistency/Repair >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Time Spent: 11h 10m > Remaining Estimate: 0h > > This is to split the change in CASSANDRA-15399 so the refactor is isolated > out. > Currently the repair coordinator special cases the exit cases at each call > site; this makes it so that errors can be inconsistent and there are cases > where proper complete isn't done (proper notifications, and forgetting to > update ActiveRepairService). > [Circle > CI|https://circleci.com/gh/dcapwell/cassandra/tree/bug%2FrepairCoordinatorJmxConsistency] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15564) Refactor repair coordinator so errors are consistent
[ https://issues.apache.org/jira/browse/CASSANDRA-15564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051515#comment-17051515 ] David Capwell edited comment on CASSANDRA-15564 at 3/4/20, 6:46 PM: org.apache.cassandra.distributed.test.PreviewRepairTest uses a blocking filter to delay the validation message; in trunk we block the MessagingService.send method and in this patch we do this as part of inbound filtering (this is performed in the Stage pool, for repair this is size 1). In order for dtest filters to be able to replicate specific failure modes inbound filtering is more desirable and has been brought up CASSANDRA-15568 since outbound made testing CAS harder. I will try to figure out how to deal with that test, ill see if there is any way to replicate with inbound filtering and if I can't I will try to revert the inbound filtering logic and try to figure out how to rewrite my tests to not rely on it. was (Author: dcapwell): org.apache.cassandra.distributed.test.PreviewRepairTest uses a blocking filter to delay the validation message; in trunk we block the MessagingService.send method and in this patch we do this as part of inbound filtering (this is performed in the Stage pool). In order for dtest filters to be able to replicate specific failure modes inbound filtering is more desirable and has been brought up CASSANDRA-15568 since outbound made testing CAS harder. I will try to figure out how to deal with that test, ill see if there is any way to replicate with inbound filtering and if I can't I will try to revert the inbound filtering logic and try to figure out how to rewrite my tests to not rely on it. > Refactor repair coordinator so errors are consistent > > > Key: CASSANDRA-15564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15564 > Project: Cassandra > Issue Type: Sub-task > Components: Consistency/Repair >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Time Spent: 11h 10m > Remaining Estimate: 0h > > This is to split the change in CASSANDRA-15399 so the refactor is isolated > out. > Currently the repair coordinator special cases the exit cases at each call > site; this makes it so that errors can be inconsistent and there are cases > where proper complete isn't done (proper notifications, and forgetting to > update ActiveRepairService). > [Circle > CI|https://circleci.com/gh/dcapwell/cassandra/tree/bug%2FrepairCoordinatorJmxConsistency] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15564) Refactor repair coordinator so errors are consistent
[ https://issues.apache.org/jira/browse/CASSANDRA-15564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051515#comment-17051515 ] David Capwell commented on CASSANDRA-15564: --- org.apache.cassandra.distributed.test.PreviewRepairTest uses a blocking filter to delay the validation message; in trunk we block the MessagingService.send method and in this patch we do this as part of inbound filtering (this is performed in the Stage pool). In order for dtest filters to be able to replicate specific failure modes inbound filtering is more desirable and has been brought up CASSANDRA-15568 since outbound made testing CAS harder. I will try to figure out how to deal with that test, ill see if there is any way to replicate with inbound filtering and if I can't I will try to revert the inbound filtering logic and try to figure out how to rewrite my tests to not rely on it. > Refactor repair coordinator so errors are consistent > > > Key: CASSANDRA-15564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15564 > Project: Cassandra > Issue Type: Sub-task > Components: Consistency/Repair >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Time Spent: 11h 10m > Remaining Estimate: 0h > > This is to split the change in CASSANDRA-15399 so the refactor is isolated > out. > Currently the repair coordinator special cases the exit cases at each call > site; this makes it so that errors can be inconsistent and there are cases > where proper complete isn't done (proper notifications, and forgetting to > update ActiveRepairService). > [Circle > CI|https://circleci.com/gh/dcapwell/cassandra/tree/bug%2FrepairCoordinatorJmxConsistency] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15604) Remove obsolete table/MV options from schema tables
[ https://issues.apache.org/jira/browse/CASSANDRA-15604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-15604: Resolution: Won't Fix Status: Resolved (was: Awaiting Feedback) Ok thanks. Closing since there is a more general proposed solution that should theoretically address the backwards compatibility concerns as well. > Remove obsolete table/MV options from schema tables > --- > > Key: CASSANDRA-15604 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15604 > Project: Cassandra > Issue Type: Improvement >Reporter: Olivier Michallat >Priority: Normal > > CASSANDRA-13910 removed the {{read_repair_chance}} and > {{dclocal_read_repair_chance}} options for tables and materialized views. > CASSANDRA-12868 made {{default_time_to_live}} illegal for materialized views. > However the corresponding columns still exist in {{system_schema.tables}} and > {{system_schema.views}}. Maybe it would make sense to remove them in > Cassandra 4. Historically, obsolete options have been removed in similar > cases, for example {{compaction_strategy_class}}, > {{compaction_strategy_options}} and {{compression_parameters}} were present > in 2.2 but removed in 3.0 (arguably there was also a major refactoring of > schema tables during that transition). > This is not a huge deal, but it would make things a bit easier for the Java > driver (our metadata exposes a "client-side describe" feature, and currently > we assume that any column in the schema table must be added to the CREATE > TABLE statement). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15604) Remove obsolete table/MV options from schema tables
[ https://issues.apache.org/jira/browse/CASSANDRA-15604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051472#comment-17051472 ] Olivier Michallat commented on CASSANDRA-15604: --- Yes, backward compatibility for direct schema table queries is a legitimate concern as well. CASSANDRA-14825 will indeed address this, as the driver will be able to get the correct statement directly from Cassandra instead of piecing it together itself. As I said it's not a big deal and we have a workaround (even until 14825 gets in), so feel free to close this if backward compatibility is considered more important. > Remove obsolete table/MV options from schema tables > --- > > Key: CASSANDRA-15604 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15604 > Project: Cassandra > Issue Type: Improvement >Reporter: Olivier Michallat >Priority: Normal > > CASSANDRA-13910 removed the {{read_repair_chance}} and > {{dclocal_read_repair_chance}} options for tables and materialized views. > CASSANDRA-12868 made {{default_time_to_live}} illegal for materialized views. > However the corresponding columns still exist in {{system_schema.tables}} and > {{system_schema.views}}. Maybe it would make sense to remove them in > Cassandra 4. Historically, obsolete options have been removed in similar > cases, for example {{compaction_strategy_class}}, > {{compaction_strategy_options}} and {{compression_parameters}} were present > in 2.2 but removed in 3.0 (arguably there was also a major refactoring of > schema tables during that transition). > This is not a huge deal, but it would make things a bit easier for the Java > driver (our metadata exposes a "client-side describe" feature, and currently > we assume that any column in the schema table must be added to the CREATE > TABLE statement). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15604) Remove obsolete table/MV options from schema tables
[ https://issues.apache.org/jira/browse/CASSANDRA-15604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West reassigned CASSANDRA-15604: --- Assignee: Jordan West > Remove obsolete table/MV options from schema tables > --- > > Key: CASSANDRA-15604 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15604 > Project: Cassandra > Issue Type: Improvement >Reporter: Olivier Michallat >Assignee: Jordan West >Priority: Normal > > CASSANDRA-13910 removed the {{read_repair_chance}} and > {{dclocal_read_repair_chance}} options for tables and materialized views. > CASSANDRA-12868 made {{default_time_to_live}} illegal for materialized views. > However the corresponding columns still exist in {{system_schema.tables}} and > {{system_schema.views}}. Maybe it would make sense to remove them in > Cassandra 4. Historically, obsolete options have been removed in similar > cases, for example {{compaction_strategy_class}}, > {{compaction_strategy_options}} and {{compression_parameters}} were present > in 2.2 but removed in 3.0 (arguably there was also a major refactoring of > schema tables during that transition). > This is not a huge deal, but it would make things a bit easier for the Java > driver (our metadata exposes a "client-side describe" feature, and currently > we assume that any column in the schema table must be added to the CREATE > TABLE statement). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15604) Remove obsolete table/MV options from schema tables
[ https://issues.apache.org/jira/browse/CASSANDRA-15604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West reassigned CASSANDRA-15604: --- Assignee: (was: Jordan West) > Remove obsolete table/MV options from schema tables > --- > > Key: CASSANDRA-15604 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15604 > Project: Cassandra > Issue Type: Improvement >Reporter: Olivier Michallat >Priority: Normal > > CASSANDRA-13910 removed the {{read_repair_chance}} and > {{dclocal_read_repair_chance}} options for tables and materialized views. > CASSANDRA-12868 made {{default_time_to_live}} illegal for materialized views. > However the corresponding columns still exist in {{system_schema.tables}} and > {{system_schema.views}}. Maybe it would make sense to remove them in > Cassandra 4. Historically, obsolete options have been removed in similar > cases, for example {{compaction_strategy_class}}, > {{compaction_strategy_options}} and {{compression_parameters}} were present > in 2.2 but removed in 3.0 (arguably there was also a major refactoring of > schema tables during that transition). > This is not a huge deal, but it would make things a bit easier for the Java > driver (our metadata exposes a "client-side describe" feature, and currently > we assume that any column in the schema table must be added to the CREATE > TABLE statement). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15604) Remove obsolete table/MV options from schema tables
[ https://issues.apache.org/jira/browse/CASSANDRA-15604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-15604: Status: Awaiting Feedback (was: Triage Needed) [~omichallat] Could removing cause issues for clients that explicitly state which columns to select (instead of select *)? My concern is that there could be some issues on upgrade for some clients (which we may not have taken as much care to check for during the 2.x -> 3.x changes). Also would https://issues.apache.org/jira/browse/CASSANDRA-14825 address this? > Remove obsolete table/MV options from schema tables > --- > > Key: CASSANDRA-15604 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15604 > Project: Cassandra > Issue Type: Improvement >Reporter: Olivier Michallat >Priority: Normal > > CASSANDRA-13910 removed the {{read_repair_chance}} and > {{dclocal_read_repair_chance}} options for tables and materialized views. > CASSANDRA-12868 made {{default_time_to_live}} illegal for materialized views. > However the corresponding columns still exist in {{system_schema.tables}} and > {{system_schema.views}}. Maybe it would make sense to remove them in > Cassandra 4. Historically, obsolete options have been removed in similar > cases, for example {{compaction_strategy_class}}, > {{compaction_strategy_options}} and {{compression_parameters}} were present > in 2.2 but removed in 3.0 (arguably there was also a major refactoring of > schema tables during that transition). > This is not a huge deal, but it would make things a bit easier for the Java > driver (our metadata exposes a "client-side describe" feature, and currently > we assume that any column in the schema table must be added to the CREATE > TABLE statement). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15388) Add compaction allocation measurement test to support compaction gc optimization.
[ https://issues.apache.org/jira/browse/CASSANDRA-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051445#comment-17051445 ] Blake Eggleston commented on CASSANDRA-15388: - I've added a few comments addressing questions. However, regarding: {quote}Personally would love if test-memory target could go away in favor of testclasslist so it matches the rest of CI. {quote} and {quote}Make this overridable; a system properly would be fine {quote} and {quote}Once you complete you can add to the logger for now (though more value if you generate a test report, but fine if this is a different JIRA) {quote} This is basically some ad-hoc code intended as a tool to identify optimization opportunities, and quantify changes. It's being contributed so the methodology can be reviewed, to quantify / justify the changes in the other tickets, and so that it can be re-used / built upon for later work. This is not meant to be in a state where it can be plugged into our ci process. I think that we _should_ have allocation measurement as part of a performance test suite, but that's not the goal of this patch. I also don't think it's a good use of time to start making changes like these, because they'll make it more awkward to use this test as currently intended, and will probably just be re-written once we have a better idea of what we want a performance test suite to do. {quote}Why all the sleeps? {quote} The ones when profiling is to give you time to start and stop recording. The one after compactions was to work around some issues with ending recording immediately after compactions. The one at the end is for logging. When running the test in IntelliJ, the report doesn't always make it to the output without them. > Add compaction allocation measurement test to support compaction gc > optimization. > -- > > Key: CASSANDRA-15388 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15388 > Project: Cassandra > Issue Type: Sub-task > Components: Local/Compaction >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Normal > Fix For: 4.0 > > > This adds a test that is able to quickly and accurately measure the effect of > potential gc optimizations against a wide range of (synthetic) compaction > workloads. This test accurately measures allocation rates from 16 workloads > in less that 2 minutes. > This test uses google’s {{java-allocation-instrumenter}} agent to measure the > workloads. Measurements using this agent are very accurate and pretty > repeatable from run to run, with most variance being negligible (1-2 bytes > per partition), although workloads with larger but fewer partitions vary a > bit more (still less that 0.03%). > The thinking behind this patch is that with compaction, we’re generally > interested in the memory allocated per partition, since garbage scales more > or less linearly with the number of partitions compacted. So measuring > allocation from a small number of partitions that otherwise represent real > world use cases is a good enough approximation. > In addition to helping with compaction optimizations, this test could be used > as a template for future optimization work. This pattern could also be used > to set allocation limits on workloads/operations and fail CI if the > allocation behavior changes past some threshold. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15337) Document usage of Ec2Snitch for multiple regions cluster
[ https://issues.apache.org/jira/browse/CASSANDRA-15337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Haddad updated CASSANDRA-15337: --- Change Category: Semantic Complexity: Low Hanging Fruit Reviewers: Jon Haddad Status: Open (was: Triage Needed) > Document usage of Ec2Snitch for multiple regions cluster > > > Key: CASSANDRA-15337 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15337 > Project: Cassandra > Issue Type: Improvement > Components: Documentation/Website >Reporter: Serban Teodorescu >Assignee: Serban Teodorescu >Priority: Normal > > Ec2MultiRegionSnitch was (probably) done because it was not possible in AWS > to have intra-region VPC peering. This changed pin 2017, see [AWS > announcement|https://aws.amazon.com/about-aws/whats-new/2017/11/announcing-support-for-inter-region-vpc-peering/] > (and extended with 9 more regions in 2018, see > [link|https://aws.amazon.com/about-aws/whats-new/2018/02/inter-region-vpc-peering-is-now-available-in-nine-additional-aws-regions/]): > {quote}Inter-Region VPC Peering allows VPC resources like EC2 instances[...] > running in different AWS regions to communicate with each other using private > IP addresses, without requiring gateways, VPN connections or separate network > appliances. > {quote} > Since the datacenter/rack names are loaded in Ec2Snitch using AWS API there > is no reason why this snitch would not work in a multi-datacenter setup. I > tested and used this in a production environment. So the documentation should > be changed to include this usage. > Proposed change: > [https://github.com/apache/cassandra/compare/trunk...serban21:15337-trunk] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15358) Cassandra alpha 4 testing - Nodes crashing due to bufferpool allocator issue
[ https://issues.apache.org/jira/browse/CASSANDRA-15358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051418#comment-17051418 ] Benedict Elliott Smith commented on CASSANDRA-15358: bq. We should support JDK8, that isn't going away any time soon and we would still be able to keep it even if it didn't use the pool. We're only talking about dropping {{Deflate}} support for JDK8, and I believe the project is unlikely to support JDK8 in our next major release anyway (since JDK8 is no longer freely supported). Since {{Deflate}} is very much not recommended, the restriction is minimal and solves this problem neatly. bq. though would still prefer isolated memory ? bq. "if you are requested unpooled buffers you should not be using this API", To also be clear, none of the users of this API when requesting {{ON_HEAP}} really require it to be pooled. There is no guarantee that {{get}} must supply a pooled buffer, and in fact will not pool if the pool threshold has been exceeded (which can be modelled to be zero for heap buffers, which is a reasonable pool size given the very different impact on GC). So at best we want to reconfigure your expectations of a combination of method and class name to better self-document, or otherwise actual documention making it clear that {{get}} only returns a pooled buffer for {{OFF_HEAP}} would also be fine. Literally none of this is a real issue in my book, and I am actually fine leaving this unchanged in all honesty, so I'm not going to spend ages agonising over it. The only thing I wish is that we do not even permit the option of {{ON_HEAP}}, because it only exists for something nobody uses in practice, since {{Deflate}} is a really bad idea (since it is cripplingly slow) and completely superseded by {{Zstd}}. bq. Works for me, can you create a JIRA to address it and link with this one? It already exists, you and I have even discussed it recently (CASSANDRA-15229). > Cassandra alpha 4 testing - Nodes crashing due to bufferpool allocator issue > > > Key: CASSANDRA-15358 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15358 > Project: Cassandra > Issue Type: Bug > Components: Test/benchmark >Reporter: Santhosh Kumar Ramalingam >Assignee: Benedict Elliott Smith >Priority: Normal > Labels: 4.0, alpha > Fix For: 4.0, 4.0-beta > > Attachments: all_errors.txt, debug_logs_during_repair.txt, > repair_1_trace.txt, verbose_logs.diff, verbose_logs.txt > > > Hitting a bug with cassandra 4 alpha version. The same bug is repeated with > difefrent version of Java(8,11 &12) [~benedict] > > Stack trace: > {code:java} > INFO [main] 2019-10-11 16:07:12,024 Server.java:164 - Starting listening for > CQL clients on /1.3.0.6:9042 (unencrypted)... > WARN [OptionalTasks:1] 2019-10-11 16:07:13,961 CassandraRoleManager.java:343 > - CassandraRoleManager skipped default role setup: some nodes were not ready > INFO [OptionalTasks:1] 2019-10-11 16:07:13,961 CassandraRoleManager.java:369 > - Setup task failed with error, rescheduling > WARN [Messaging-EventLoop-3-2] 2019-10-11 16:07:22,038 NoSpamLogger.java:94 - > 10.3x.4x.5x:7000->1.3.0.5:7000-LARGE_MESSAGES-[no-channel] dropping message > of type PING_REQ whose timeout expired before reaching the network > WARN [OptionalTasks:1] 2019-10-11 16:07:23,963 CassandraRoleManager.java:343 > - CassandraRoleManager skipped default role setup: some nodes were not ready > INFO [OptionalTasks:1] 2019-10-11 16:07:23,963 CassandraRoleManager.java:369 > - Setup task failed with error, rescheduling > INFO [Messaging-EventLoop-3-6] 2019-10-11 16:07:32,759 NoSpamLogger.java:91 - > 10.3x.4x.5x:7000->1.3.0.2:7000-URGENT_MESSAGES-[no-channel] failed to connect > io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) > failed: Connection refused: /1.3.0.2:7000 > Caused by: java.net.ConnectException: finishConnect(..) failed: Connection > refused > at io.netty.channel.unix.Errors.throwConnectException(Errors.java:124) > at io.netty.channel.unix.Socket.finishConnect(Socket.java:243) > at > io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:667) > at > io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:644) > at > io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:524) > at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:414) > at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:326) > at > io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918) > at io.netty.util.intern
[jira] [Commented] (CASSANDRA-15564) Refactor repair coordinator so errors are consistent
[ https://issues.apache.org/jira/browse/CASSANDRA-15564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051414#comment-17051414 ] David Capwell commented on CASSANDRA-15564: --- Looking into org.apache.cassandra.distributed.test.PreviewRepairTest (rebase branch) > Refactor repair coordinator so errors are consistent > > > Key: CASSANDRA-15564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15564 > Project: Cassandra > Issue Type: Sub-task > Components: Consistency/Repair >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Time Spent: 11h 10m > Remaining Estimate: 0h > > This is to split the change in CASSANDRA-15399 so the refactor is isolated > out. > Currently the repair coordinator special cases the exit cases at each call > site; this makes it so that errors can be inconsistent and there are cases > where proper complete isn't done (proper notifications, and forgetting to > update ActiveRepairService). > [Circle > CI|https://circleci.com/gh/dcapwell/cassandra/tree/bug%2FrepairCoordinatorJmxConsistency] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15358) Cassandra alpha 4 testing - Nodes crashing due to bufferpool allocator issue
[ https://issues.apache.org/jira/browse/CASSANDRA-15358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051402#comment-17051402 ] David Capwell commented on CASSANDRA-15358: --- bq. but this means dropping support for Deflate, at least for JDK8. I would be fine requiring JDK11 for Deflate support in 4.0 though, WDYT? We should support JDK8, that isn't going away any time soon and we would still be able to keep it even if it didn't use the pool. To be clear, my statement was that "if you are requested unpooled buffers you should not be using this API", if you request a native buffer I am fine with using it (though would still prefer isolated memory, thats a different issue though ^_^). bq. We anyway need to address the ChunkCache use of BufferPool in a separate ticket. Works for me, can you create a JIRA to address it and link with this one? bq. rename the get method to e.g. getPooledIfOffHeap, or renaming BufferPool to OffHeapBufferPool so that it's clear we never pool on-heap The class rename makes the most sense to me (self documenting) but we would still need to guard access at the call sites {code} ByteBuffer bb = compressor.preferredBufferType() == ON_HEAP ? ByteBuffer.allocate(size) : OffHeapBufferPool.get(size) {code} bq. Or alternatively we could start pooling on-heap, but that should probably be a separate ticket (and is probably unjustified compared to the above options). This really should be a different JIRA. There is still a lot of code which assumes the array isn't shared so would need to do a lot of work to find all those patterns and change them (found org.apache.cassandra.gms.TokenSerializer#serialize with a quick search, this isn't the only one). In doing that scan, most of the sites which require array buffers could be changed to support native (or do and special case) as well which makes me ask a different question, "why not drop array buffer support"; again, different JIRA should tackle this question. > Cassandra alpha 4 testing - Nodes crashing due to bufferpool allocator issue > > > Key: CASSANDRA-15358 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15358 > Project: Cassandra > Issue Type: Bug > Components: Test/benchmark >Reporter: Santhosh Kumar Ramalingam >Assignee: Benedict Elliott Smith >Priority: Normal > Labels: 4.0, alpha > Fix For: 4.0, 4.0-beta > > Attachments: all_errors.txt, debug_logs_during_repair.txt, > repair_1_trace.txt, verbose_logs.diff, verbose_logs.txt > > > Hitting a bug with cassandra 4 alpha version. The same bug is repeated with > difefrent version of Java(8,11 &12) [~benedict] > > Stack trace: > {code:java} > INFO [main] 2019-10-11 16:07:12,024 Server.java:164 - Starting listening for > CQL clients on /1.3.0.6:9042 (unencrypted)... > WARN [OptionalTasks:1] 2019-10-11 16:07:13,961 CassandraRoleManager.java:343 > - CassandraRoleManager skipped default role setup: some nodes were not ready > INFO [OptionalTasks:1] 2019-10-11 16:07:13,961 CassandraRoleManager.java:369 > - Setup task failed with error, rescheduling > WARN [Messaging-EventLoop-3-2] 2019-10-11 16:07:22,038 NoSpamLogger.java:94 - > 10.3x.4x.5x:7000->1.3.0.5:7000-LARGE_MESSAGES-[no-channel] dropping message > of type PING_REQ whose timeout expired before reaching the network > WARN [OptionalTasks:1] 2019-10-11 16:07:23,963 CassandraRoleManager.java:343 > - CassandraRoleManager skipped default role setup: some nodes were not ready > INFO [OptionalTasks:1] 2019-10-11 16:07:23,963 CassandraRoleManager.java:369 > - Setup task failed with error, rescheduling > INFO [Messaging-EventLoop-3-6] 2019-10-11 16:07:32,759 NoSpamLogger.java:91 - > 10.3x.4x.5x:7000->1.3.0.2:7000-URGENT_MESSAGES-[no-channel] failed to connect > io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) > failed: Connection refused: /1.3.0.2:7000 > Caused by: java.net.ConnectException: finishConnect(..) failed: Connection > refused > at io.netty.channel.unix.Errors.throwConnectException(Errors.java:124) > at io.netty.channel.unix.Socket.finishConnect(Socket.java:243) > at > io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:667) > at > io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:644) > at > io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:524) > at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:414) > at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:326) > at > io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918) > a
[jira] [Updated] (CASSANDRA-15613) Flakey test - TestBootstrap.test_local_quorum_bootstrap
[ https://issues.apache.org/jira/browse/CASSANDRA-15613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-15613: Reviewers: Ekaterina Dimitrova > Flakey test - TestBootstrap.test_local_quorum_bootstrap > --- > > Key: CASSANDRA-15613 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15613 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Ekaterina Dimitrova >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.0-alpha > > > Flakey test on trunk, java 8. > I did a couple of loops and it sometimes fail at the 5th run, sometimes at > the 50th. > > {noformat} > *_ > TestBootstrap.test_local_quorum_bootstrap > __* > > self = > > *def test_local_quorum_bootstrap(self):* > *"""* > *Test that CL local_quorum works while a node is bootstrapping.* > *@jira_ticket CASSANDRA-8058* > *"""* > *cluster = self.cluster* > *cluster.populate([1, 1])* > *cluster.start()* > ** > *node1 = cluster.nodes['node1']* > *yaml_config = """* > *# Create the keyspace and table* > *keyspace: keyspace1* > *keyspace_definition: |* > *CREATE KEYSPACE keyspace1 WITH replication = \{'class': > 'NetworkTopologyStrategy', 'dc1': 1, 'dc2': 1};* > *table: users* > *table_definition:* > *CREATE TABLE users (* > *username text,* > *first_name text,* > *last_name text,* > *email text,* > *PRIMARY KEY(username)* > *) WITH compaction = \{'class':'SizeTieredCompactionStrategy'};* > *insert:* > *partitions: fixed(1)* > *batchtype: UNLOGGED* > *queries:* > *read:* > *cql: select * from users where username = ?* > *fields: samerow* > *"""* > *with tempfile.NamedTemporaryFile(mode='w+') as stress_config:* > *stress_config.write(yaml_config)* > *stress_config.flush()* > *node1.stress(['user', 'profile=' + stress_config.name, 'n=200K', > 'no-warmup',* > *'ops(insert=1)', '-rate', 'threads=10'])* > ** > *node3 = new_node(cluster, data_center='dc2')* > *node3.start(no_wait=True)* > *time.sleep(3)* > ** > *ntout = node1.nodetool('status').stdout* > *> assert re.search(r'UJ\s+' + node3.ip_addr, ntout), ntout* > *E AssertionError: Datacenter: dc1* > *E ===* > *E Status=Up/Down* > *E |/ State=Normal/Leaving/Joining/Moving* > *E -- Address Load Owns (effective) Host ID > Token Rack* > *E UN 127.0.0.1 76.82 KiB ? > 3056e1d1-eac6-493a-b5bc-f8fe703d8d4b -9223372036854775808 r1 * > *E* > *E Datacenter: dc2* > *E ===* > *E Status=Up/Down* > *E |/ State=Normal/Leaving/Joining/Moving* > *E -- Address Load Owns (effective) Host ID > Token Rack* > *E UN 127.0.0.2 76.83 KiB ? > ae607eb3-5601-4b6f-9d01-cf9401da5a4c -9223372036854775708 r1 * > *E* > *E* > *E assert None* > *E + where None = (('UJ\\s+' + > '127.0.0.3'), 'Datacenter: dc1\n===\nStatus=Up/Down\n|/ > State=Normal/Leaving/Joining/Moving\n-- Address Load O...Rack\nUN > 127.0.0.2 76.83 KiB ? ae607eb3-5601-4b6f-9d01-cf9401da5a4c > -9223372036854775708 r1 \n\n')* > *E + where = re.search* > *E + and '127.0.0.3' = 0x106530b10>.ip_addr* > > *bootstrap_test.py*:488: AssertionError > --- > Captured stdout setup > > 11:42:12,744 ccm DEBUG Log-watching thread starting. > - > Captured log setup > - > 11:42:12,608
[jira] [Commented] (CASSANDRA-14842) SSL connection problems when upgrading to 4.0 when upgrading from 3.0.x
[ https://issues.apache.org/jira/browse/CASSANDRA-14842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051349#comment-17051349 ] Brandon Williams commented on CASSANDRA-14842: -- Tommy, is this still an issue after CASSANDRA-15066? If so, can you rebase? > SSL connection problems when upgrading to 4.0 when upgrading from 3.0.x > --- > > Key: CASSANDRA-14842 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14842 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Internode >Reporter: Tommy Stendahl >Assignee: Tommy Stendahl >Priority: Normal > Fix For: 4.0 > > > While testing to upgrade from 3.0.15 to 4.0 the old nodes fails to connect to > the 4.0 node, I get this exception on the 4.0 node: > > {noformat} > 2018-10-22T11:57:44.366+0200 ERROR [MessagingService-NettyInbound-Thread-3-8] > InboundHandshakeHandler.java:300 Failed to properly handshake with peer > /10.216.193.246:58296. Closing the channel. > io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: > SSLv2Hello is disabled > at > io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459) > at > io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) > at > io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) > at > io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) > at > io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:808) > at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:417) > at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:317) > at > io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884) > at > io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > at java.lang.Thread.run(Thread.java:748) > Caused by: javax.net.ssl.SSLHandshakeException: SSLv2Hello is disabled > at sun.security.ssl.InputRecord.handleUnknownRecord(InputRecord.java:637) > at sun.security.ssl.InputRecord.read(InputRecord.java:527) > at sun.security.ssl.EngineInputRecord.read(EngineInputRecord.java:382) > at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:962) > at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:907) > at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781) > at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) > at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:294) > at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1275) > at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1177) > at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1221) > at > io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) > at > io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) > ... 14 common frames omitted{noformat} > In the server encryption options on the 4.0 node I have both "enabled and > "enable_legacy_ssl_storage_port" set to true so it should accept incoming > connections on the "ssl_storage_port". > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14842) SSL connection problems when upgrading to 4.0 when upgrading from 3.0.x
[ https://issues.apache.org/jira/browse/CASSANDRA-14842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-14842: - Status: Open (was: Patch Available) > SSL connection problems when upgrading to 4.0 when upgrading from 3.0.x > --- > > Key: CASSANDRA-14842 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14842 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Internode >Reporter: Tommy Stendahl >Assignee: Tommy Stendahl >Priority: Normal > Fix For: 4.0 > > > While testing to upgrade from 3.0.15 to 4.0 the old nodes fails to connect to > the 4.0 node, I get this exception on the 4.0 node: > > {noformat} > 2018-10-22T11:57:44.366+0200 ERROR [MessagingService-NettyInbound-Thread-3-8] > InboundHandshakeHandler.java:300 Failed to properly handshake with peer > /10.216.193.246:58296. Closing the channel. > io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: > SSLv2Hello is disabled > at > io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459) > at > io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) > at > io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) > at > io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) > at > io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:808) > at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:417) > at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:317) > at > io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884) > at > io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > at java.lang.Thread.run(Thread.java:748) > Caused by: javax.net.ssl.SSLHandshakeException: SSLv2Hello is disabled > at sun.security.ssl.InputRecord.handleUnknownRecord(InputRecord.java:637) > at sun.security.ssl.InputRecord.read(InputRecord.java:527) > at sun.security.ssl.EngineInputRecord.read(EngineInputRecord.java:382) > at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:962) > at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:907) > at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781) > at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) > at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:294) > at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1275) > at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1177) > at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1221) > at > io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) > at > io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) > ... 14 common frames omitted{noformat} > In the server encryption options on the 4.0 node I have both "enabled and > "enable_legacy_ssl_storage_port" set to true so it should accept incoming > connections on the "ssl_storage_port". > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15605) Broken dtest replication_test.py::TestSnitchConfigurationUpdate
[ https://issues.apache.org/jira/browse/CASSANDRA-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051343#comment-17051343 ] Brandon Williams commented on CASSANDRA-15605: -- I'm unable to view the CI failure, and I looped this test for 48 hours without failure, so with nowhere to go, I give up. > Broken dtest replication_test.py::TestSnitchConfigurationUpdate > --- > > Key: CASSANDRA-15605 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15605 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Sam Tunnicliffe >Priority: Normal > Fix For: 4.0-alpha > > > Noticed this failing on a couple of CI runs and repros when running trunk > locally and on CircleCI > 2 or 3 tests are consistently failing: > * {{test_rf_expand_gossiping_property_file_snitch}} > * {{test_rf_expand_property_file_snitch}} > * {{test_move_forwards_between_and_cleanup}} > [https://circleci.com/workflow-run/f23f13a9-bbdc-4764-8336-109517e137f1] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15605) Broken dtest replication_test.py::TestSnitchConfigurationUpdate
[ https://issues.apache.org/jira/browse/CASSANDRA-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams reassigned CASSANDRA-15605: Assignee: (was: Brandon Williams) > Broken dtest replication_test.py::TestSnitchConfigurationUpdate > --- > > Key: CASSANDRA-15605 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15605 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Sam Tunnicliffe >Priority: Normal > Fix For: 4.0-alpha > > > Noticed this failing on a couple of CI runs and repros when running trunk > locally and on CircleCI > 2 or 3 tests are consistently failing: > * {{test_rf_expand_gossiping_property_file_snitch}} > * {{test_rf_expand_property_file_snitch}} > * {{test_move_forwards_between_and_cleanup}} > [https://circleci.com/workflow-run/f23f13a9-bbdc-4764-8336-109517e137f1] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15613) Flakey test - TestBootstrap.test_local_quorum_bootstrap
[ https://issues.apache.org/jira/browse/CASSANDRA-15613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051329#comment-17051329 ] Brandon Williams edited comment on CASSANDRA-15613 at 3/4/20, 3:35 PM: --- There are two problem with this test. First, 3 seconds may not be long enough to see the bootstrapping node. Second, it assumes that after inserting 10k rows, the node will still be bootstrapping, and there's simply no logical guarantee that will be the case. The simplest thing to do, since we don't actually care if the node ever finishes joining, is to put it into write survey mode so it will perpetually be joining. Patch to increase the initial sleep slightly and use survey mode here: https://github.com/driftx/cassandra-dtest/tree/CASSANDRA-15613 was (Author: brandon.williams): There are two problem with this test. First, 3 second may not be long enough to see the bootstrapping node. Second, it assumes that after inserting 10k rows, the node will still be bootstrapping, and there's simply no logical guarantee that will be the case. The simplest thing to do, since we don't actually care if the node ever finishes joining, is to put it into write survey mode so it will perpetually be joining. Patch to increase the initial sleep slightly and use survey mode here: https://github.com/driftx/cassandra-dtest/tree/CASSANDRA-15613 > Flakey test - TestBootstrap.test_local_quorum_bootstrap > --- > > Key: CASSANDRA-15613 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15613 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Ekaterina Dimitrova >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.0-alpha > > > Flakey test on trunk, java 8. > I did a couple of loops and it sometimes fail at the 5th run, sometimes at > the 50th. > > {noformat} > *_ > TestBootstrap.test_local_quorum_bootstrap > __* > > self = > > *def test_local_quorum_bootstrap(self):* > *"""* > *Test that CL local_quorum works while a node is bootstrapping.* > *@jira_ticket CASSANDRA-8058* > *"""* > *cluster = self.cluster* > *cluster.populate([1, 1])* > *cluster.start()* > ** > *node1 = cluster.nodes['node1']* > *yaml_config = """* > *# Create the keyspace and table* > *keyspace: keyspace1* > *keyspace_definition: |* > *CREATE KEYSPACE keyspace1 WITH replication = \{'class': > 'NetworkTopologyStrategy', 'dc1': 1, 'dc2': 1};* > *table: users* > *table_definition:* > *CREATE TABLE users (* > *username text,* > *first_name text,* > *last_name text,* > *email text,* > *PRIMARY KEY(username)* > *) WITH compaction = \{'class':'SizeTieredCompactionStrategy'};* > *insert:* > *partitions: fixed(1)* > *batchtype: UNLOGGED* > *queries:* > *read:* > *cql: select * from users where username = ?* > *fields: samerow* > *"""* > *with tempfile.NamedTemporaryFile(mode='w+') as stress_config:* > *stress_config.write(yaml_config)* > *stress_config.flush()* > *node1.stress(['user', 'profile=' + stress_config.name, 'n=200K', > 'no-warmup',* > *'ops(insert=1)', '-rate', 'threads=10'])* > ** > *node3 = new_node(cluster, data_center='dc2')* > *node3.start(no_wait=True)* > *time.sleep(3)* > ** > *ntout = node1.nodetool('status').stdout* > *> assert re.search(r'UJ\s+' + node3.ip_addr, ntout), ntout* > *E AssertionError: Datacenter: dc1* > *E ===* > *E Status=Up/Down* > *E |/ State=Normal/Leaving/Joining/Moving* > *E -- Address Load Owns (effective) Host ID > Token Rack* > *E UN 127.0.0.1 76.82 KiB ? > 3056e1d1-eac6-493a-b5bc-f8fe703d8d4b -9223372036854775808 r1 * > *E* > *E Datacenter: dc2* > *E ===* > *E Status=Up/Down* > *E |/ State=Normal/Leaving/Joining/Moving* > *E -- Address Load Owns (effective) Host ID > Token R
[jira] [Commented] (CASSANDRA-15613) Flakey test - TestBootstrap.test_local_quorum_bootstrap
[ https://issues.apache.org/jira/browse/CASSANDRA-15613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051329#comment-17051329 ] Brandon Williams commented on CASSANDRA-15613: -- There are two problem with this test. First, 3 second may not be long enough to see the bootstrapping node. Second, it assumes that after inserting 10k rows, the node will still be bootstrapping, and there's simply no logical guarantee that will be the case. The simplest thing to do, since we don't actually care if the node ever finishes joining, is to put it into write survey mode so it will perpetually be joining. Patch to increase the initial sleep slightly and use survey mode here: https://github.com/driftx/cassandra-dtest/tree/CASSANDRA-15613 > Flakey test - TestBootstrap.test_local_quorum_bootstrap > --- > > Key: CASSANDRA-15613 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15613 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Ekaterina Dimitrova >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.0-alpha > > > Flakey test on trunk, java 8. > I did a couple of loops and it sometimes fail at the 5th run, sometimes at > the 50th. > > {noformat} > *_ > TestBootstrap.test_local_quorum_bootstrap > __* > > self = > > *def test_local_quorum_bootstrap(self):* > *"""* > *Test that CL local_quorum works while a node is bootstrapping.* > *@jira_ticket CASSANDRA-8058* > *"""* > *cluster = self.cluster* > *cluster.populate([1, 1])* > *cluster.start()* > ** > *node1 = cluster.nodes['node1']* > *yaml_config = """* > *# Create the keyspace and table* > *keyspace: keyspace1* > *keyspace_definition: |* > *CREATE KEYSPACE keyspace1 WITH replication = \{'class': > 'NetworkTopologyStrategy', 'dc1': 1, 'dc2': 1};* > *table: users* > *table_definition:* > *CREATE TABLE users (* > *username text,* > *first_name text,* > *last_name text,* > *email text,* > *PRIMARY KEY(username)* > *) WITH compaction = \{'class':'SizeTieredCompactionStrategy'};* > *insert:* > *partitions: fixed(1)* > *batchtype: UNLOGGED* > *queries:* > *read:* > *cql: select * from users where username = ?* > *fields: samerow* > *"""* > *with tempfile.NamedTemporaryFile(mode='w+') as stress_config:* > *stress_config.write(yaml_config)* > *stress_config.flush()* > *node1.stress(['user', 'profile=' + stress_config.name, 'n=200K', > 'no-warmup',* > *'ops(insert=1)', '-rate', 'threads=10'])* > ** > *node3 = new_node(cluster, data_center='dc2')* > *node3.start(no_wait=True)* > *time.sleep(3)* > ** > *ntout = node1.nodetool('status').stdout* > *> assert re.search(r'UJ\s+' + node3.ip_addr, ntout), ntout* > *E AssertionError: Datacenter: dc1* > *E ===* > *E Status=Up/Down* > *E |/ State=Normal/Leaving/Joining/Moving* > *E -- Address Load Owns (effective) Host ID > Token Rack* > *E UN 127.0.0.1 76.82 KiB ? > 3056e1d1-eac6-493a-b5bc-f8fe703d8d4b -9223372036854775808 r1 * > *E* > *E Datacenter: dc2* > *E ===* > *E Status=Up/Down* > *E |/ State=Normal/Leaving/Joining/Moving* > *E -- Address Load Owns (effective) Host ID > Token Rack* > *E UN 127.0.0.2 76.83 KiB ? > ae607eb3-5601-4b6f-9d01-cf9401da5a4c -9223372036854775708 r1 * > *E* > *E* > *E assert None* > *E + where None = (('UJ\\s+' + > '127.0.0.3'), 'Datacenter: dc1\n===\nStatus=Up/Down\n|/ > State=Normal/Leaving/Joining/Moving\n-- Address Load O...Rack\nUN > 127.0.0.2 76.83 KiB ? ae607eb3-5601-4b6f-9d01-cf9401da5a4c > -9223372036854775708 r1 \n\n')* > *E + where = re.search* > *E + and '127.0.0.3' = 0x106530b10>.ip_addr* > > *bootstrap_test.py*:48
[jira] [Updated] (CASSANDRA-15613) Flakey test - TestBootstrap.test_local_quorum_bootstrap
[ https://issues.apache.org/jira/browse/CASSANDRA-15613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-15613: - Test and Documentation Plan: none Status: Patch Available (was: Open) > Flakey test - TestBootstrap.test_local_quorum_bootstrap > --- > > Key: CASSANDRA-15613 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15613 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Ekaterina Dimitrova >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.0-alpha > > > Flakey test on trunk, java 8. > I did a couple of loops and it sometimes fail at the 5th run, sometimes at > the 50th. > > {noformat} > *_ > TestBootstrap.test_local_quorum_bootstrap > __* > > self = > > *def test_local_quorum_bootstrap(self):* > *"""* > *Test that CL local_quorum works while a node is bootstrapping.* > *@jira_ticket CASSANDRA-8058* > *"""* > *cluster = self.cluster* > *cluster.populate([1, 1])* > *cluster.start()* > ** > *node1 = cluster.nodes['node1']* > *yaml_config = """* > *# Create the keyspace and table* > *keyspace: keyspace1* > *keyspace_definition: |* > *CREATE KEYSPACE keyspace1 WITH replication = \{'class': > 'NetworkTopologyStrategy', 'dc1': 1, 'dc2': 1};* > *table: users* > *table_definition:* > *CREATE TABLE users (* > *username text,* > *first_name text,* > *last_name text,* > *email text,* > *PRIMARY KEY(username)* > *) WITH compaction = \{'class':'SizeTieredCompactionStrategy'};* > *insert:* > *partitions: fixed(1)* > *batchtype: UNLOGGED* > *queries:* > *read:* > *cql: select * from users where username = ?* > *fields: samerow* > *"""* > *with tempfile.NamedTemporaryFile(mode='w+') as stress_config:* > *stress_config.write(yaml_config)* > *stress_config.flush()* > *node1.stress(['user', 'profile=' + stress_config.name, 'n=200K', > 'no-warmup',* > *'ops(insert=1)', '-rate', 'threads=10'])* > ** > *node3 = new_node(cluster, data_center='dc2')* > *node3.start(no_wait=True)* > *time.sleep(3)* > ** > *ntout = node1.nodetool('status').stdout* > *> assert re.search(r'UJ\s+' + node3.ip_addr, ntout), ntout* > *E AssertionError: Datacenter: dc1* > *E ===* > *E Status=Up/Down* > *E |/ State=Normal/Leaving/Joining/Moving* > *E -- Address Load Owns (effective) Host ID > Token Rack* > *E UN 127.0.0.1 76.82 KiB ? > 3056e1d1-eac6-493a-b5bc-f8fe703d8d4b -9223372036854775808 r1 * > *E* > *E Datacenter: dc2* > *E ===* > *E Status=Up/Down* > *E |/ State=Normal/Leaving/Joining/Moving* > *E -- Address Load Owns (effective) Host ID > Token Rack* > *E UN 127.0.0.2 76.83 KiB ? > ae607eb3-5601-4b6f-9d01-cf9401da5a4c -9223372036854775708 r1 * > *E* > *E* > *E assert None* > *E + where None = (('UJ\\s+' + > '127.0.0.3'), 'Datacenter: dc1\n===\nStatus=Up/Down\n|/ > State=Normal/Leaving/Joining/Moving\n-- Address Load O...Rack\nUN > 127.0.0.2 76.83 KiB ? ae607eb3-5601-4b6f-9d01-cf9401da5a4c > -9223372036854775708 r1 \n\n')* > *E + where = re.search* > *E + and '127.0.0.3' = 0x106530b10>.ip_addr* > > *bootstrap_test.py*:488: AssertionError > --- > Captured stdout setup > > 11:42:12,744 ccm DEBUG Log-watching thread starting. > - > Captured log setup > -
[jira] [Updated] (CASSANDRA-15621) LOCAL_QUORUM is waiting on remote datacenter
[ https://issues.apache.org/jira/browse/CASSANDRA-15621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-15621: - Resolution: Not A Problem Status: Resolved (was: Triage Needed) > LOCAL_QUORUM is waiting on remote datacenter > > > Key: CASSANDRA-15621 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15621 > Project: Cassandra > Issue Type: Bug >Reporter: Stefan Genov >Priority: Normal > > We have a 2 datacenters ( One in EU and one in US ) cluster with 4 nodes each > deployed in AWS. > The nodes are separated in 3 racks ( Availability zones ) each. > In the cluster we have a keyspace test with replication: > NetworkTopologyStrategy, eu-west:3, us-east:3 > In the keyspace we have a table called mytable that has only one row 'id' text > Now, we were doing some tests on the performance of the database. > In CQLSH with a consistency level of LOCAL_QUORUM we were doing some inserts > with TRACING ON and we noticed that the requests were not working as we > expected them. > From the tracing data we found out that the coordinator node was hitting as > expected 2 other local nodes and was also sending a request to one of the > remote datacenter nodes. Now the problem here was that the coordinator was > waiting not only for the local nodes ( who finished in no time ) but for the > remote nodes too. > Now since our 2 datacenters are geographically far away from each other, our > requests were taking a very long time to complete. > Notes: > - This does not happen with DSE but our understanding was we don't need to > pay crazy money for LOCAL_QUORUM to work as is expected -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15621) LOCAL_QUORUM is waiting on remote datacenter
[ https://issues.apache.org/jira/browse/CASSANDRA-15621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051298#comment-17051298 ] Brandon Williams commented on CASSANDRA-15621: -- That's your problem. http://cassandra.apache.org/doc/latest/operating/security.html#enabling-password-authentication bq. During login, the credentials for the default superuser are read with a consistency level of QUORUM, whereas those for all other users (including superusers) are read at LOCAL_ONE. In the interests of performance and availability, as well as security, operators should create another superuser and disable the default one. > LOCAL_QUORUM is waiting on remote datacenter > > > Key: CASSANDRA-15621 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15621 > Project: Cassandra > Issue Type: Bug >Reporter: Stefan Genov >Priority: Normal > > We have a 2 datacenters ( One in EU and one in US ) cluster with 4 nodes each > deployed in AWS. > The nodes are separated in 3 racks ( Availability zones ) each. > In the cluster we have a keyspace test with replication: > NetworkTopologyStrategy, eu-west:3, us-east:3 > In the keyspace we have a table called mytable that has only one row 'id' text > Now, we were doing some tests on the performance of the database. > In CQLSH with a consistency level of LOCAL_QUORUM we were doing some inserts > with TRACING ON and we noticed that the requests were not working as we > expected them. > From the tracing data we found out that the coordinator node was hitting as > expected 2 other local nodes and was also sending a request to one of the > remote datacenter nodes. Now the problem here was that the coordinator was > waiting not only for the local nodes ( who finished in no time ) but for the > remote nodes too. > Now since our 2 datacenters are geographically far away from each other, our > requests were taking a very long time to complete. > Notes: > - This does not happen with DSE but our understanding was we don't need to > pay crazy money for LOCAL_QUORUM to work as is expected -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15621) LOCAL_QUORUM is waiting on remote datacenter
[ https://issues.apache.org/jira/browse/CASSANDRA-15621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051285#comment-17051285 ] Stefan Genov commented on CASSANDRA-15621: -- yes i was using the default cassandra user > LOCAL_QUORUM is waiting on remote datacenter > > > Key: CASSANDRA-15621 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15621 > Project: Cassandra > Issue Type: Bug >Reporter: Stefan Genov >Priority: Normal > > We have a 2 datacenters ( One in EU and one in US ) cluster with 4 nodes each > deployed in AWS. > The nodes are separated in 3 racks ( Availability zones ) each. > In the cluster we have a keyspace test with replication: > NetworkTopologyStrategy, eu-west:3, us-east:3 > In the keyspace we have a table called mytable that has only one row 'id' text > Now, we were doing some tests on the performance of the database. > In CQLSH with a consistency level of LOCAL_QUORUM we were doing some inserts > with TRACING ON and we noticed that the requests were not working as we > expected them. > From the tracing data we found out that the coordinator node was hitting as > expected 2 other local nodes and was also sending a request to one of the > remote datacenter nodes. Now the problem here was that the coordinator was > waiting not only for the local nodes ( who finished in no time ) but for the > remote nodes too. > Now since our 2 datacenters are geographically far away from each other, our > requests were taking a very long time to complete. > Notes: > - This does not happen with DSE but our understanding was we don't need to > pay crazy money for LOCAL_QUORUM to work as is expected -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15621) LOCAL_QUORUM is waiting on remote datacenter
[ https://issues.apache.org/jira/browse/CASSANDRA-15621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051281#comment-17051281 ] Brandon Williams commented on CASSANDRA-15621: -- Are you using the cassandra user? > LOCAL_QUORUM is waiting on remote datacenter > > > Key: CASSANDRA-15621 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15621 > Project: Cassandra > Issue Type: Bug >Reporter: Stefan Genov >Priority: Normal > > We have a 2 datacenters ( One in EU and one in US ) cluster with 4 nodes each > deployed in AWS. > The nodes are separated in 3 racks ( Availability zones ) each. > In the cluster we have a keyspace test with replication: > NetworkTopologyStrategy, eu-west:3, us-east:3 > In the keyspace we have a table called mytable that has only one row 'id' text > Now, we were doing some tests on the performance of the database. > In CQLSH with a consistency level of LOCAL_QUORUM we were doing some inserts > with TRACING ON and we noticed that the requests were not working as we > expected them. > From the tracing data we found out that the coordinator node was hitting as > expected 2 other local nodes and was also sending a request to one of the > remote datacenter nodes. Now the problem here was that the coordinator was > waiting not only for the local nodes ( who finished in no time ) but for the > remote nodes too. > Now since our 2 datacenters are geographically far away from each other, our > requests were taking a very long time to complete. > Notes: > - This does not happen with DSE but our understanding was we don't need to > pay crazy money for LOCAL_QUORUM to work as is expected -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15621) LOCAL_QUORUM is waiting on remote datacenter
[ https://issues.apache.org/jira/browse/CASSANDRA-15621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051237#comment-17051237 ] Stefan Genov commented on CASSANDRA-15621: -- Our problem seemed to be related to the Authentication and the Authorization. For some reason with them we were seeing CRAZY slow requests and with them disabled the tracing shows everything as expected. Maybe this is a potential bug? > LOCAL_QUORUM is waiting on remote datacenter > > > Key: CASSANDRA-15621 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15621 > Project: Cassandra > Issue Type: Bug >Reporter: Stefan Genov >Priority: Normal > > We have a 2 datacenters ( One in EU and one in US ) cluster with 4 nodes each > deployed in AWS. > The nodes are separated in 3 racks ( Availability zones ) each. > In the cluster we have a keyspace test with replication: > NetworkTopologyStrategy, eu-west:3, us-east:3 > In the keyspace we have a table called mytable that has only one row 'id' text > Now, we were doing some tests on the performance of the database. > In CQLSH with a consistency level of LOCAL_QUORUM we were doing some inserts > with TRACING ON and we noticed that the requests were not working as we > expected them. > From the tracing data we found out that the coordinator node was hitting as > expected 2 other local nodes and was also sending a request to one of the > remote datacenter nodes. Now the problem here was that the coordinator was > waiting not only for the local nodes ( who finished in no time ) but for the > remote nodes too. > Now since our 2 datacenters are geographically far away from each other, our > requests were taking a very long time to complete. > Notes: > - This does not happen with DSE but our understanding was we don't need to > pay crazy money for LOCAL_QUORUM to work as is expected -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15621) LOCAL_QUORUM is waiting on remote datacenter
Stefan Genov created CASSANDRA-15621: Summary: LOCAL_QUORUM is waiting on remote datacenter Key: CASSANDRA-15621 URL: https://issues.apache.org/jira/browse/CASSANDRA-15621 Project: Cassandra Issue Type: Bug Reporter: Stefan Genov We have a 2 datacenters ( One in EU and one in US ) cluster with 4 nodes each deployed in AWS. The nodes are separated in 3 racks ( Availability zones ) each. In the cluster we have a keyspace test with replication: NetworkTopologyStrategy, eu-west:3, us-east:3 In the keyspace we have a table called mytable that has only one row 'id' text Now, we were doing some tests on the performance of the database. In CQLSH with a consistency level of LOCAL_QUORUM we were doing some inserts with TRACING ON and we noticed that the requests were not working as we expected them. >From the tracing data we found out that the coordinator node was hitting as >expected 2 other local nodes and was also sending a request to one of the >remote datacenter nodes. Now the problem here was that the coordinator was >waiting not only for the local nodes ( who finished in no time ) but for the >remote nodes too. Now since our 2 datacenters are geographically far away from each other, our requests were taking a very long time to complete. Notes: - This does not happen with DSE but our understanding was we don't need to pay crazy money for LOCAL_QUORUM to work as is expected -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15481) Data Modeling
[ https://issues.apache.org/jira/browse/CASSANDRA-15481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051136#comment-17051136 ] Erick Ramirez edited comment on CASSANDRA-15481 at 3/4/20, 11:54 AM: - [~rustyrazorblade], I've completed my review and made some changes -- PR [#462|https://github.com/apache/cassandra/pull/462/files] submitted. was (Author: flightc): [~rustyrazorblade] PR [#462|https://github.com/apache/cassandra/pull/462/files] submitted. > Data Modeling > - > > Key: CASSANDRA-15481 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15481 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > Labels: pull-request-available > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Added a page on data modeling. > https://github.com/apache/cassandra/pull/410 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15481) Data Modeling
[ https://issues.apache.org/jira/browse/CASSANDRA-15481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051136#comment-17051136 ] Erick Ramirez edited comment on CASSANDRA-15481 at 3/4/20, 11:53 AM: - [~rustyrazorblade] PR [#462|https://github.com/apache/cassandra/pull/462/files] submitted. was (Author: flightc): [~rustyrazorblade] PR [#462|[https://github.com/apache/cassandra/pull/462/files]] submitted. > Data Modeling > - > > Key: CASSANDRA-15481 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15481 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > Labels: pull-request-available > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Added a page on data modeling. > https://github.com/apache/cassandra/pull/410 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15481) Data Modeling
[ https://issues.apache.org/jira/browse/CASSANDRA-15481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Ramirez updated CASSANDRA-15481: -- Authors: DeepakVohra, Erick Ramirez (was: DeepakVohra) Status: Patch Available (was: In Progress) [~rustyrazorblade] PR [#462|[https://github.com/apache/cassandra/pull/462/files]] submitted. > Data Modeling > - > > Key: CASSANDRA-15481 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15481 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > Labels: pull-request-available > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Added a page on data modeling. > https://github.com/apache/cassandra/pull/410 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15557) Fix flaky test org.apache.cassandra.cql3.validation.operations.AlterTest testDropListAndAddListWithSameName
[ https://issues.apache.org/jira/browse/CASSANDRA-15557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051100#comment-17051100 ] Benjamin Lerer edited comment on CASSANDRA-15557 at 3/4/20, 11:51 AM: -- I believe that the test is fine and should not be changed. The {{DroppedColum}} should filter out every row with a timestamp less or equal to the {{DroppedColum.timestamp}}. So even if the row and the {{DroppedColum}} have the same timestamp, it should be ignored. Now the interesting question is why if you fix the timestamps for the row and the dropped column, the test is not failing anymore? was (Author: blerer): I believe that the test is fine and should not be changed. The {{DroppedColum}} should filter out every row with a timestamp less or equal to the {{DroppedColum.timestamp}}. So even if the row and the {{DroppedColum}} have the same timestamp something it should be ignored. Now the interesting question is why if you fix the timestamps for the row and the dropped column, the test is not failing anymore? > Fix flaky test org.apache.cassandra.cql3.validation.operations.AlterTest > testDropListAndAddListWithSameName > --- > > Key: CASSANDRA-15557 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15557 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: David Capwell >Assignee: Ryan Svihla >Priority: Normal > Fix For: 4.0-alpha > > > https://app.circleci.com/jobs/github/dcapwell/cassandra/482/tests > {code} > junit.framework.AssertionFailedError: Invalid value for row 0 column 2 > (mycollection of type list), expected but got <[first element]> > at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1070) > at > org.apache.cassandra.cql3.validation.operations.AlterTest.testDropListAndAddListWithSameName(AlterTest.java:91) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15367) Memtable memory allocations may deadlock
[ https://issues.apache.org/jira/browse/CASSANDRA-15367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051130#comment-17051130 ] Benedict Elliott Smith edited comment on CASSANDRA-15367 at 3/4/20, 11:46 AM: -- Sorry for the delay on this, I've been trying to figure out what's wrong with CircleCI for 3.0 and 3.11, but have given up. It appears the issues occur on HEAD for both as well. patch:[3.0|https://github.com/belliottsmith/cassandra/tree/15367-3.0] ci:[3.0|https://circleci.com/workflow-run/b6396d79-d2bb-42a1-a76f-805715e896de] head ci:[3.0|https://circleci.com/workflow-run/759d8331-7e2f-431f-8161-847fe46af4d3] patch:[3.11|https://github.com/belliottsmith/cassandra/tree/15367-311] ci:[3.11|https://circleci.com/workflow-run/7c74fea5-3730-44bf-bd62-5d112cf974d4] head ci:[3.11|https://circleci.com/workflow-run/72a173e0-713e-479d-b5db-9fd7e94eb3d6] patch:[trunk|https://github.com/belliottsmith/cassandra/tree/15367-trunk] ci:[trunk|https://circleci.com/workflow-run/b2b24c56-d191-4923-9f66-b05036d62ea1] I will commit tomorrow after double checking the test failures are consistent and assuming you don't say otherwise. was (Author: benedict): Sorry for the delay on this, I've been trying to figure out what's wrong with CircleCI for 3.0 and 3.11, but have given up. It appears the issues occur on HEAD for both as well. patch:[3.0|https://github.com/belliottsmith/cassandra/tree/15367-3.0] ci:[3.0|https://circleci.com/workflow-run/b6396d79-d2bb-42a1-a76f-805715e896de] head ci:[3.0|https://circleci.com/workflow-run/759d8331-7e2f-431f-8161-847fe46af4d3] patch:[3.11|https://github.com/belliottsmith/cassandra/tree/15367-311] ci:[3.11|https://circleci.com/workflow-run/7c74fea5-3730-44bf-bd62-5d112cf974d4] head ci:[3.11|https://circleci.com/workflow-run/72a173e0-713e-479d-b5db-9fd7e94eb3d6] patch: [trunk|https://github.com/belliottsmith/cassandra/tree/15367-trunk] ci:[trunk|https://circleci.com/workflow-run/b2b24c56-d191-4923-9f66-b05036d62ea1] I will commit tomorrow after double checking the test failures are consistent and assuming you don't say otherwise. > Memtable memory allocations may deadlock > > > Key: CASSANDRA-15367 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15367 > Project: Cassandra > Issue Type: Bug > Components: Local/Commit Log, Local/Memtable >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > Fix For: 4.0, 2.2.x, 3.0.x, 3.11.x > > > * Under heavy contention, we guard modifications to a partition with a mutex, > for the lifetime of the memtable. > * Memtables block for the completion of all {{OpOrder.Group}} started before > their flush began > * Memtables permit operations from this cohort to fall-through to the > following Memtable, in order to guarantee a precise commitLogUpperBound > * Memtable memory limits may be lifted for operations in the first cohort, > since they block flush (and hence block future memory allocation) > With very unfortunate scheduling > * A contended partition may rapidly escalate to a mutex > * The system may reach memory limits that prevent allocations for the new > Memtable’s cohort (C2) > * An operation from C2 may hold the mutex when this occurs > * Operations from a prior Memtable’s cohort (C1), for a contended partition, > may fall-through to the next Memtable > * The operations from C1 may execute after the above is encountered by those > from C2 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15367) Memtable memory allocations may deadlock
[ https://issues.apache.org/jira/browse/CASSANDRA-15367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051130#comment-17051130 ] Benedict Elliott Smith edited comment on CASSANDRA-15367 at 3/4/20, 11:46 AM: -- Sorry for the delay on this, I've been trying to figure out what's wrong with CircleCI for 3.0 and 3.11, but have given up. It appears the issues occur on HEAD for both as well. patch:[3.0|https://github.com/belliottsmith/cassandra/tree/15367-3.0] ci:[3.0|https://circleci.com/workflow-run/b6396d79-d2bb-42a1-a76f-805715e896de] head ci:[3.0|https://circleci.com/workflow-run/759d8331-7e2f-431f-8161-847fe46af4d3] patch:[3.11|https://github.com/belliottsmith/cassandra/tree/15367-311] ci:[3.11|https://circleci.com/workflow-run/7c74fea5-3730-44bf-bd62-5d112cf974d4] head ci:[3.11|https://circleci.com/workflow-run/72a173e0-713e-479d-b5db-9fd7e94eb3d6] patch: [trunk|https://github.com/belliottsmith/cassandra/tree/15367-trunk] ci:[trunk|https://circleci.com/workflow-run/b2b24c56-d191-4923-9f66-b05036d62ea1] I will commit tomorrow after double checking the test failures are consistent and assuming you don't say otherwise. was (Author: benedict): Sorry for the delay on this, I've been trying to figure out what's wrong with CircleCI for 3.0 and 3.11, but have given up. It appears the issues occur on HEAD for both as well. [3.0|https://github.com/belliottsmith/cassandra/tree/15367-3.0] [3.0ci|https://circleci.com/workflow-run/b6396d79-d2bb-42a1-a76f-805715e896de] [3.0-HEADci|https://circleci.com/workflow-run/759d8331-7e2f-431f-8161-847fe46af4d3] [3.11|https://github.com/belliottsmith/cassandra/tree/15367-311] [3.11ci|https://circleci.com/workflow-run/7c74fea5-3730-44bf-bd62-5d112cf974d4] [3.11-HEADci|https://circleci.com/workflow-run/72a173e0-713e-479d-b5db-9fd7e94eb3d6][trunk|https://github.com/belliottsmith/cassandra/tree/15367-trunk] [trunkci|https://circleci.com/workflow-run/b2b24c56-d191-4923-9f66-b05036d62ea1] I will commit tomorrow after double checking the test failures are consistent and assuming you don't say otherwise. > Memtable memory allocations may deadlock > > > Key: CASSANDRA-15367 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15367 > Project: Cassandra > Issue Type: Bug > Components: Local/Commit Log, Local/Memtable >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > Fix For: 4.0, 2.2.x, 3.0.x, 3.11.x > > > * Under heavy contention, we guard modifications to a partition with a mutex, > for the lifetime of the memtable. > * Memtables block for the completion of all {{OpOrder.Group}} started before > their flush began > * Memtables permit operations from this cohort to fall-through to the > following Memtable, in order to guarantee a precise commitLogUpperBound > * Memtable memory limits may be lifted for operations in the first cohort, > since they block flush (and hence block future memory allocation) > With very unfortunate scheduling > * A contended partition may rapidly escalate to a mutex > * The system may reach memory limits that prevent allocations for the new > Memtable’s cohort (C2) > * An operation from C2 may hold the mutex when this occurs > * Operations from a prior Memtable’s cohort (C1), for a contended partition, > may fall-through to the next Memtable > * The operations from C1 may execute after the above is encountered by those > from C2 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15367) Memtable memory allocations may deadlock
[ https://issues.apache.org/jira/browse/CASSANDRA-15367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051130#comment-17051130 ] Benedict Elliott Smith commented on CASSANDRA-15367: Sorry for the delay on this, I've been trying to figure out what's wrong with CircleCI for 3.0 and 3.11, but have given up. It appears the issues occur on HEAD for both as well. [3.0|https://github.com/belliottsmith/cassandra/tree/15367-3.0] [3.0ci|https://circleci.com/workflow-run/b6396d79-d2bb-42a1-a76f-805715e896de] [3.0-HEADci|https://circleci.com/workflow-run/759d8331-7e2f-431f-8161-847fe46af4d3] [3.11|https://github.com/belliottsmith/cassandra/tree/15367-311] [3.11ci|https://circleci.com/workflow-run/7c74fea5-3730-44bf-bd62-5d112cf974d4] [3.11-HEADci|https://circleci.com/workflow-run/72a173e0-713e-479d-b5db-9fd7e94eb3d6][trunk|https://github.com/belliottsmith/cassandra/tree/15367-trunk] [trunkci|https://circleci.com/workflow-run/b2b24c56-d191-4923-9f66-b05036d62ea1] I will commit tomorrow after double checking the test failures are consistent and assuming you don't say otherwise. > Memtable memory allocations may deadlock > > > Key: CASSANDRA-15367 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15367 > Project: Cassandra > Issue Type: Bug > Components: Local/Commit Log, Local/Memtable >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > Fix For: 4.0, 2.2.x, 3.0.x, 3.11.x > > > * Under heavy contention, we guard modifications to a partition with a mutex, > for the lifetime of the memtable. > * Memtables block for the completion of all {{OpOrder.Group}} started before > their flush began > * Memtables permit operations from this cohort to fall-through to the > following Memtable, in order to guarantee a precise commitLogUpperBound > * Memtable memory limits may be lifted for operations in the first cohort, > since they block flush (and hence block future memory allocation) > With very unfortunate scheduling > * A contended partition may rapidly escalate to a mutex > * The system may reach memory limits that prevent allocations for the new > Memtable’s cohort (C2) > * An operation from C2 may hold the mutex when this occurs > * Operations from a prior Memtable’s cohort (C1), for a contended partition, > may fall-through to the next Memtable > * The operations from C1 may execute after the above is encountered by those > from C2 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15481) Data Modeling
[ https://issues.apache.org/jira/browse/CASSANDRA-15481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CASSANDRA-15481: --- Labels: pull-request-available (was: ) > Data Modeling > - > > Key: CASSANDRA-15481 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15481 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > Labels: pull-request-available > Fix For: 4.0 > > > Added a page on data modeling. > https://github.com/apache/cassandra/pull/410 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15358) Cassandra alpha 4 testing - Nodes crashing due to bufferpool allocator issue
[ https://issues.apache.org/jira/browse/CASSANDRA-15358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051127#comment-17051127 ] Benedict Elliott Smith commented on CASSANDRA-15358: bq. I don't think it makes sense to have the get methods take a BufferType, if you write Ok, so I went to do this and it's not trivial to do as part of this ticket. I think the right thing to do is to entirely remove {{compressor.preferredBufferType()}}, as the need for the parameter vanishes, but this means dropping support for {{Deflate}}, at least for JDK8. I would be fine requiring JDK11 for Deflate support in 4.0 though, WDYT? We anyway need to address the {{ChunkCache}} use of {{BufferPool}} in a separate ticket. We do have a simple option, which is to rename the {{get}} method to e.g. {{getPooledIfOffHeap}}, or renaming {{BufferPool}} to {{OffHeapBufferPool}} so that it's clear we never pool on-heap. Or alternatively we could start pooling on-heap, but that should probably be a separate ticket (and is probably unjustified compared to the above options). > Cassandra alpha 4 testing - Nodes crashing due to bufferpool allocator issue > > > Key: CASSANDRA-15358 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15358 > Project: Cassandra > Issue Type: Bug > Components: Test/benchmark >Reporter: Santhosh Kumar Ramalingam >Assignee: Benedict Elliott Smith >Priority: Normal > Labels: 4.0, alpha > Fix For: 4.0, 4.0-beta > > Attachments: all_errors.txt, debug_logs_during_repair.txt, > repair_1_trace.txt, verbose_logs.diff, verbose_logs.txt > > > Hitting a bug with cassandra 4 alpha version. The same bug is repeated with > difefrent version of Java(8,11 &12) [~benedict] > > Stack trace: > {code:java} > INFO [main] 2019-10-11 16:07:12,024 Server.java:164 - Starting listening for > CQL clients on /1.3.0.6:9042 (unencrypted)... > WARN [OptionalTasks:1] 2019-10-11 16:07:13,961 CassandraRoleManager.java:343 > - CassandraRoleManager skipped default role setup: some nodes were not ready > INFO [OptionalTasks:1] 2019-10-11 16:07:13,961 CassandraRoleManager.java:369 > - Setup task failed with error, rescheduling > WARN [Messaging-EventLoop-3-2] 2019-10-11 16:07:22,038 NoSpamLogger.java:94 - > 10.3x.4x.5x:7000->1.3.0.5:7000-LARGE_MESSAGES-[no-channel] dropping message > of type PING_REQ whose timeout expired before reaching the network > WARN [OptionalTasks:1] 2019-10-11 16:07:23,963 CassandraRoleManager.java:343 > - CassandraRoleManager skipped default role setup: some nodes were not ready > INFO [OptionalTasks:1] 2019-10-11 16:07:23,963 CassandraRoleManager.java:369 > - Setup task failed with error, rescheduling > INFO [Messaging-EventLoop-3-6] 2019-10-11 16:07:32,759 NoSpamLogger.java:91 - > 10.3x.4x.5x:7000->1.3.0.2:7000-URGENT_MESSAGES-[no-channel] failed to connect > io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) > failed: Connection refused: /1.3.0.2:7000 > Caused by: java.net.ConnectException: finishConnect(..) failed: Connection > refused > at io.netty.channel.unix.Errors.throwConnectException(Errors.java:124) > at io.netty.channel.unix.Socket.finishConnect(Socket.java:243) > at > io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:667) > at > io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:644) > at > io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:524) > at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:414) > at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:326) > at > io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918) > at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) > at > io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > at java.base/java.lang.Thread.run(Thread.java:834) > WARN [Messaging-EventLoop-3-3] 2019-10-11 16:11:32,639 NoSpamLogger.java:94 - > 1.3.4.6:7000->1.3.4.5:7000-URGENT_MESSAGES-[no-channel] dropping message of > type GOSSIP_DIGEST_SYN whose timeout expired before reaching the network > INFO [Messaging-EventLoop-3-18] 2019-10-11 16:11:33,077 NoSpamLogger.java:91 > - 1.3.4.5:7000->1.3.4.4:7000-URGENT_MESSAGES-[no-channel] failed to connect > > ERROR [Messaging-EventLoop-3-11] 2019-10-10 01:34:34,407 > InboundMessageHandler.java:657 - > 1.3.4.5:7000->1.3.4.8:7000-LARGE_MESSAGES-0b7d09cd unexpected exception > caught while processing inbound messages; terminating connection > java.lang.IllegalArgumentException: initialB
[jira] [Comment Edited] (CASSANDRA-15565) Fix flaky test org.apache.cassandra.index.internal.CassandraIndexTest indexCorrectlyMarkedAsBuildAndRemoved
[ https://issues.apache.org/jira/browse/CASSANDRA-15565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17050184#comment-17050184 ] Jorge Bay edited comment on CASSANDRA-15565 at 3/4/20, 11:02 AM: - I've been trying to reproduce with no luck. I've run the test and the suite in loop in CircleCI: - [https://circleci.com/gh/jorgebay/cassandra/7] - [https://circleci.com/gh/jorgebay/cassandra/12] I've also run it in loop in my local machine. >From the error message in the ticket, it's related to the initial condition of >the test, it expects no other index present. The failure is mostly due to an >issue with the cleanup of a prior test. I'll submit a patch -using {{KEYSPACE_PER_TEST}}-, that way it will less likely to fail in cascade after another test. was (Author: jorgebg): I've been trying to reproduce with no luck. I've run the test and the suite in loop in CircleCI: - https://circleci.com/gh/jorgebay/cassandra/7 - https://circleci.com/gh/jorgebay/cassandra/12 I've also run it in loop in my local machine. >From the error message in the ticket, it's related to the initial condition of >the test, it expects no other index present. The failure is mostly due to an >issue with the cleanup of a prior test. I'll submit a patch using {{KEYSPACE_PER_TEST}}, that way it will less likely to fail in cascade after another test. > Fix flaky test org.apache.cassandra.index.internal.CassandraIndexTest > indexCorrectlyMarkedAsBuildAndRemoved > --- > > Key: CASSANDRA-15565 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15565 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: David Capwell >Assignee: Jorge Bay >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-alpha > > Time Spent: 10m > Remaining Estimate: 0h > > {code} > junit.framework.AssertionFailedError: Got more rows than expected. Expected 1 > but got 4. > at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1098) > at > org.apache.cassandra.index.internal.CassandraIndexTest.indexCorrectlyMarkedAsBuildAndRemoved(CassandraIndexTest.java:499) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > {code} > The failure was seen on java 11. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15557) Fix flaky test org.apache.cassandra.cql3.validation.operations.AlterTest testDropListAndAddListWithSameName
[ https://issues.apache.org/jira/browse/CASSANDRA-15557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1705#comment-1705 ] Ryan Svihla commented on CASSANDRA-15557: - Yes silly me, easy enough to prove when setting the timestamps to the same value the tests still pass, and just to totally eliminate weird side effects, setting the timestamp in reverse breaks the test as expected so I don't think there is anything specific about setting the timestamp to any value that makes it pass. So I'm not sure I have a good theory as to why breaks in the first place. It does then act as if somehow the drop is happening "before" the insert. I'll keep digging thanks for the review. > Fix flaky test org.apache.cassandra.cql3.validation.operations.AlterTest > testDropListAndAddListWithSameName > --- > > Key: CASSANDRA-15557 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15557 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: David Capwell >Assignee: Ryan Svihla >Priority: Normal > Fix For: 4.0-alpha > > > https://app.circleci.com/jobs/github/dcapwell/cassandra/482/tests > {code} > junit.framework.AssertionFailedError: Invalid value for row 0 column 2 > (mycollection of type list), expected but got <[first element]> > at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1070) > at > org.apache.cassandra.cql3.validation.operations.AlterTest.testDropListAndAddListWithSameName(AlterTest.java:91) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15565) Fix flaky test org.apache.cassandra.index.internal.CassandraIndexTest indexCorrectlyMarkedAsBuildAndRemoved
[ https://issues.apache.org/jira/browse/CASSANDRA-15565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Bay updated CASSANDRA-15565: -- Test and Documentation Plan: The fix involves using a before/after size comparison to avoid failing as a result of another test failure. Status: Patch Available (was: Open) [PR|https://github.com/apache/cassandra/pull/461] and [CI|https://app.circleci.com/pipelines/github/jorgebay/cassandra?branch=CASSANDRA-15565-fix]. > Fix flaky test org.apache.cassandra.index.internal.CassandraIndexTest > indexCorrectlyMarkedAsBuildAndRemoved > --- > > Key: CASSANDRA-15565 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15565 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: David Capwell >Assignee: Jorge Bay >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-alpha > > Time Spent: 10m > Remaining Estimate: 0h > > {code} > junit.framework.AssertionFailedError: Got more rows than expected. Expected 1 > but got 4. > at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1098) > at > org.apache.cassandra.index.internal.CassandraIndexTest.indexCorrectlyMarkedAsBuildAndRemoved(CassandraIndexTest.java:499) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > {code} > The failure was seen on java 11. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15375) Remove BackPressureStrategy
[ https://issues.apache.org/jira/browse/CASSANDRA-15375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051102#comment-17051102 ] Benedict Elliott Smith commented on CASSANDRA-15375: That would be great, thanks! > Remove BackPressureStrategy > --- > > Key: CASSANDRA-15375 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15375 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Client, Observability/Logging >Reporter: Jon Haddad >Assignee: Jon Haddad >Priority: Low > > This is odd: > {{INFO [main] 2019-10-25 10:33:07,985 DatabaseDescriptor.java:803 - > Back-pressure is disabled with strategy > org.apache.cassandra.net.RateBasedBackPressure\{high_ratio=0.9, factor=5, > flow=FAST}.}} > When I saw that, I wasn't sure if back pressure was actually disabled, or if > I was really using {{RateBasedBackPressure.}} > This should change to output either: > {{Back-pressure is disabled}} > {{or}} > {{Back-pressure is enabled with strategy > org.apache.cassandra.net.RateBasedBackPressure\{high_ratio=0.9, factor=5, > flow=FAST}.}}{{}} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15557) Fix flaky test org.apache.cassandra.cql3.validation.operations.AlterTest testDropListAndAddListWithSameName
[ https://issues.apache.org/jira/browse/CASSANDRA-15557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051100#comment-17051100 ] Benjamin Lerer commented on CASSANDRA-15557: I believe that the test is fine and should not be changed. The {{DroppedColum}} should filter out every row with a timestamp less or equal to the {{DroppedColum.timestamp}}. So even if the row and the {{DroppedColum}} have the same timestamp something it should be ignored. Now the interesting question is why if you fix the timestamps for the row and the dropped column, the test is not failing anymore? > Fix flaky test org.apache.cassandra.cql3.validation.operations.AlterTest > testDropListAndAddListWithSameName > --- > > Key: CASSANDRA-15557 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15557 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: David Capwell >Assignee: Ryan Svihla >Priority: Normal > Fix For: 4.0-alpha > > > https://app.circleci.com/jobs/github/dcapwell/cassandra/482/tests > {code} > junit.framework.AssertionFailedError: Invalid value for row 0 column 2 > (mycollection of type list), expected but got <[first element]> > at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1070) > at > org.apache.cassandra.cql3.validation.operations.AlterTest.testDropListAndAddListWithSameName(AlterTest.java:91) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15375) Remove BackPressureStrategy
[ https://issues.apache.org/jira/browse/CASSANDRA-15375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051099#comment-17051099 ] Sergio Bossa commented on CASSANDRA-15375: -- bq. I'd like to disclaim any desire to discredit the original work Of course, and as I said it was experimental and meant to evolve once widely used, which never happened. Do you want me to review your patch? > Remove BackPressureStrategy > --- > > Key: CASSANDRA-15375 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15375 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Client, Observability/Logging >Reporter: Jon Haddad >Assignee: Jon Haddad >Priority: Low > > This is odd: > {{INFO [main] 2019-10-25 10:33:07,985 DatabaseDescriptor.java:803 - > Back-pressure is disabled with strategy > org.apache.cassandra.net.RateBasedBackPressure\{high_ratio=0.9, factor=5, > flow=FAST}.}} > When I saw that, I wasn't sure if back pressure was actually disabled, or if > I was really using {{RateBasedBackPressure.}} > This should change to output either: > {{Back-pressure is disabled}} > {{or}} > {{Back-pressure is enabled with strategy > org.apache.cassandra.net.RateBasedBackPressure\{high_ratio=0.9, factor=5, > flow=FAST}.}}{{}} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15565) Fix flaky test org.apache.cassandra.index.internal.CassandraIndexTest indexCorrectlyMarkedAsBuildAndRemoved
[ https://issues.apache.org/jira/browse/CASSANDRA-15565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CASSANDRA-15565: --- Labels: pull-request-available (was: ) > Fix flaky test org.apache.cassandra.index.internal.CassandraIndexTest > indexCorrectlyMarkedAsBuildAndRemoved > --- > > Key: CASSANDRA-15565 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15565 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: David Capwell >Assignee: Jorge Bay >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-alpha > > > {code} > junit.framework.AssertionFailedError: Got more rows than expected. Expected 1 > but got 4. > at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1098) > at > org.apache.cassandra.index.internal.CassandraIndexTest.indexCorrectlyMarkedAsBuildAndRemoved(CassandraIndexTest.java:499) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > {code} > The failure was seen on java 11. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15358) Cassandra alpha 4 testing - Nodes crashing due to bufferpool allocator issue
[ https://issues.apache.org/jira/browse/CASSANDRA-15358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051081#comment-17051081 ] Benedict Elliott Smith commented on CASSANDRA-15358: {quote} +1 to removing ALLOCATE_ON_HEAP_WHEN_EXAHUSTED and DISABLED I don't think it makes sense to have the get methods take a BufferType, if you write {quote} Great, I'll rustle up a modified patch with those changes, as I agree. At some point we might want to start offering pooled heap buffers (which would be a simple change), but there's no call for it at the moment, and I prefer to remove disused options. > Cassandra alpha 4 testing - Nodes crashing due to bufferpool allocator issue > > > Key: CASSANDRA-15358 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15358 > Project: Cassandra > Issue Type: Bug > Components: Test/benchmark >Reporter: Santhosh Kumar Ramalingam >Assignee: Benedict Elliott Smith >Priority: Normal > Labels: 4.0, alpha > Fix For: 4.0, 4.0-beta > > Attachments: all_errors.txt, debug_logs_during_repair.txt, > repair_1_trace.txt, verbose_logs.diff, verbose_logs.txt > > > Hitting a bug with cassandra 4 alpha version. The same bug is repeated with > difefrent version of Java(8,11 &12) [~benedict] > > Stack trace: > {code:java} > INFO [main] 2019-10-11 16:07:12,024 Server.java:164 - Starting listening for > CQL clients on /1.3.0.6:9042 (unencrypted)... > WARN [OptionalTasks:1] 2019-10-11 16:07:13,961 CassandraRoleManager.java:343 > - CassandraRoleManager skipped default role setup: some nodes were not ready > INFO [OptionalTasks:1] 2019-10-11 16:07:13,961 CassandraRoleManager.java:369 > - Setup task failed with error, rescheduling > WARN [Messaging-EventLoop-3-2] 2019-10-11 16:07:22,038 NoSpamLogger.java:94 - > 10.3x.4x.5x:7000->1.3.0.5:7000-LARGE_MESSAGES-[no-channel] dropping message > of type PING_REQ whose timeout expired before reaching the network > WARN [OptionalTasks:1] 2019-10-11 16:07:23,963 CassandraRoleManager.java:343 > - CassandraRoleManager skipped default role setup: some nodes were not ready > INFO [OptionalTasks:1] 2019-10-11 16:07:23,963 CassandraRoleManager.java:369 > - Setup task failed with error, rescheduling > INFO [Messaging-EventLoop-3-6] 2019-10-11 16:07:32,759 NoSpamLogger.java:91 - > 10.3x.4x.5x:7000->1.3.0.2:7000-URGENT_MESSAGES-[no-channel] failed to connect > io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) > failed: Connection refused: /1.3.0.2:7000 > Caused by: java.net.ConnectException: finishConnect(..) failed: Connection > refused > at io.netty.channel.unix.Errors.throwConnectException(Errors.java:124) > at io.netty.channel.unix.Socket.finishConnect(Socket.java:243) > at > io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:667) > at > io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:644) > at > io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:524) > at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:414) > at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:326) > at > io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918) > at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) > at > io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > at java.base/java.lang.Thread.run(Thread.java:834) > WARN [Messaging-EventLoop-3-3] 2019-10-11 16:11:32,639 NoSpamLogger.java:94 - > 1.3.4.6:7000->1.3.4.5:7000-URGENT_MESSAGES-[no-channel] dropping message of > type GOSSIP_DIGEST_SYN whose timeout expired before reaching the network > INFO [Messaging-EventLoop-3-18] 2019-10-11 16:11:33,077 NoSpamLogger.java:91 > - 1.3.4.5:7000->1.3.4.4:7000-URGENT_MESSAGES-[no-channel] failed to connect > > ERROR [Messaging-EventLoop-3-11] 2019-10-10 01:34:34,407 > InboundMessageHandler.java:657 - > 1.3.4.5:7000->1.3.4.8:7000-LARGE_MESSAGES-0b7d09cd unexpected exception > caught while processing inbound messages; terminating connection > java.lang.IllegalArgumentException: initialBuffer is not a direct buffer. > at io.netty.buffer.UnpooledDirectByteBuf.(UnpooledDirectByteBuf.java:87) > at > io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:59) > at > org.apache.cassandra.net.BufferPoolAllocator$Wrapped.(BufferPoolAllocator.java:95) > at > org.apache.cassandra.net.BufferPoolAllocator.newDirectBuffer(BufferPoolAllocator.java:56) > at > io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocato
[jira] [Commented] (CASSANDRA-15339) Make available the known JMX endpoints across the cluster
[ https://issues.apache.org/jira/browse/CASSANDRA-15339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051068#comment-17051068 ] Michael Semb Wever commented on CASSANDRA-15339: [~brandon.williams], thanks for the heads up, have updated the patch. It's still wip though. > Make available the known JMX endpoints across the cluster > - > > Key: CASSANDRA-15339 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15339 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Gossip >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Labels: 4.0-feature-freeze-review-requested > > With the addition of multiple nodes running on the same server using > different ports: CASSANDRA-7544 ; it becomes more difficult for third-party > tools to easily connect to all nodes based on the jmx connection details to > just one node. > By adding jmx host and port to gossip, and saving it in {{system.peers_v2}}, > the list of all jmx endpoints in a cluster can be fetch after just the > initial successful jmx connection and the > {{StorageServiceMBean.getJmxEndpoints()}} method. > And example of the difficulty can be illustrated through the potential > workaround… > Such a third-party tool could make a native protocol connection, and via the > driver obtain the list of all possible `host:port` native protocol > connections, and make a connection to each of these then requesting the > configuration virtual table, from which the jmx port can be obtained. This is > a rather cumbersome approach, and can involve third-party tools having to add > native connection functionality and dependencies. It's also currently not > possible because CASSANDRA-14573 does not provide the jmx port (it only > offers the yaml settings). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15620) Add "unleveled sstables" table metric
[ https://issues.apache.org/jira/browse/CASSANDRA-15620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051057#comment-17051057 ] Stefan Podkowinski commented on CASSANDRA-15620: * [CASSANDRA-15620|https://github.com/spodkowinski/cassandra/tree/CASSANDRA-15620] * [!https://circleci.com/gh/spodkowinski/cassandra/tree/CASSANDRA-15620.svg?style=svg!|https://circleci.com/gh/spodkowinski/cassandra/tree/CASSANDRA-15620] > Add "unleveled sstables" table metric > - > > Key: CASSANDRA-15620 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15620 > Project: Cassandra > Issue Type: Improvement > Components: Observability/Metrics >Reporter: Stefan Podkowinski >Assignee: Stefan Podkowinski >Priority: Normal > > The number of unleveled sstables is an important indicator that deserves to > be a dedicated table metric on its own. This will also add a global gauge > that is convenient to query. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15557) Fix flaky test org.apache.cassandra.cql3.validation.operations.AlterTest testDropListAndAddListWithSameName
[ https://issues.apache.org/jira/browse/CASSANDRA-15557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051025#comment-17051025 ] Ryan Svihla edited comment on CASSANDRA-15557 at 3/4/20, 9:22 AM: -- Not sure what's going on there, maybe some Jira macro run amok. Take 2 Code as it was when the test was originally fixed [https://github.com/pcmanus/cassandra/commit/020960d727b882349aff00c17a1c46bf3c7f7a24#diff-43b2d530031e2f7ad4286bd05fed4ca0R253] EDIT dunno why the highlight isn't being added to the links..[src/java/org/apache/cassandra/cql3/statements/AlterTableStatement.java|https://github.com/pcmanus/cassandra/commit/020960d727b882349aff00c17a1c46bf3c7f7a24#diff-43b2d530031e2f7ad4286bd05fed4ca0] line 253 Change below that I think stripped out the client state timestamp [https://github.com/apache/cassandra/commit/207c80c1fd63dfbd8ca7e615ec8002ee8983c5d6#diff-50a7bf78bc4f943f42ce60c8768484a6R400] EDIT ( [src/java/org/apache/cassandra/cql3/statements/schema/AlterTableStatement.java|https://github.com/apache/cassandra/commit/207c80c1fd63dfbd8ca7e615ec8002ee8983c5d6#diff-50a7bf78bc4f943f42ce60c8768484a6] line 400) was (Author: rssvihla): Not sure what's going on there, maybe some Jira macro run amok. Take 2 Code as it was when the test was originally fixed [https://github.com/pcmanus/cassandra/commit/020960d727b882349aff00c17a1c46bf3c7f7a24#diff-43b2d530031e2f7ad4286bd05fed4ca0R253] Change below that I think stripped out the client state timestamp [https://github.com/apache/cassandra/commit/207c80c1fd63dfbd8ca7e615ec8002ee8983c5d6#diff-50a7bf78bc4f943f42ce60c8768484a6R400] > Fix flaky test org.apache.cassandra.cql3.validation.operations.AlterTest > testDropListAndAddListWithSameName > --- > > Key: CASSANDRA-15557 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15557 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: David Capwell >Assignee: Ryan Svihla >Priority: Normal > Fix For: 4.0-alpha > > > https://app.circleci.com/jobs/github/dcapwell/cassandra/482/tests > {code} > junit.framework.AssertionFailedError: Invalid value for row 0 column 2 > (mycollection of type list), expected but got <[first element]> > at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1070) > at > org.apache.cassandra.cql3.validation.operations.AlterTest.testDropListAndAddListWithSameName(AlterTest.java:91) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15557) Fix flaky test org.apache.cassandra.cql3.validation.operations.AlterTest testDropListAndAddListWithSameName
[ https://issues.apache.org/jira/browse/CASSANDRA-15557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051025#comment-17051025 ] Ryan Svihla commented on CASSANDRA-15557: - Not sure what's going on there, maybe some Jira macro run amok. Take 2 Code as it was when the test was originally fixed [https://github.com/pcmanus/cassandra/commit/020960d727b882349aff00c17a1c46bf3c7f7a24#diff-43b2d530031e2f7ad4286bd05fed4ca0R253] Change below that I think stripped out the client state timestamp [https://github.com/apache/cassandra/commit/207c80c1fd63dfbd8ca7e615ec8002ee8983c5d6#diff-50a7bf78bc4f943f42ce60c8768484a6R400] > Fix flaky test org.apache.cassandra.cql3.validation.operations.AlterTest > testDropListAndAddListWithSameName > --- > > Key: CASSANDRA-15557 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15557 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: David Capwell >Assignee: Ryan Svihla >Priority: Normal > Fix For: 4.0-alpha > > > https://app.circleci.com/jobs/github/dcapwell/cassandra/482/tests > {code} > junit.framework.AssertionFailedError: Invalid value for row 0 column 2 > (mycollection of type list), expected but got <[first element]> > at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1070) > at > org.apache.cassandra.cql3.validation.operations.AlterTest.testDropListAndAddListWithSameName(AlterTest.java:91) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15557) Fix flaky test org.apache.cassandra.cql3.validation.operations.AlterTest testDropListAndAddListWithSameName
[ https://issues.apache.org/jira/browse/CASSANDRA-15557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051020#comment-17051020 ] Benjamin Lerer commented on CASSANDRA-15557: It seems that something is wrong on your github {{here}} github links. They bring me back to this page. > Fix flaky test org.apache.cassandra.cql3.validation.operations.AlterTest > testDropListAndAddListWithSameName > --- > > Key: CASSANDRA-15557 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15557 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: David Capwell >Assignee: Ryan Svihla >Priority: Normal > Fix For: 4.0-alpha > > > https://app.circleci.com/jobs/github/dcapwell/cassandra/482/tests > {code} > junit.framework.AssertionFailedError: Invalid value for row 0 column 2 > (mycollection of type list), expected but got <[first element]> > at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1070) > at > org.apache.cassandra.cql3.validation.operations.AlterTest.testDropListAndAddListWithSameName(AlterTest.java:91) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15557) Fix flaky test org.apache.cassandra.cql3.validation.operations.AlterTest testDropListAndAddListWithSameName
[ https://issues.apache.org/jira/browse/CASSANDRA-15557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051007#comment-17051007 ] Ryan Svihla commented on CASSANDRA-15557: - [PR|[https://github.com/rssvihla/cassandra/pull/1]][Code|[https://github.com/rssvihla/cassandra/tree/failing-test-15557]] Ran the tests all night on two different servers and had no issues (whereas before on the same hardware I had 30 or 40 failures usually within 5 to 10 minute). The theory I have is this test got broken when the schema management changed in https://issues.apache.org/jira/browse/CASSANDRA-13426 and the test no longer was relying on client query state to set a timestamp as it was [here|[https://github.com/pcmanus/cassandra/commit/020960d727b882349aff00c17a1c46bf3c7f7a24#diff-43b2d530031e2f7ad4286bd05fed4ca0R253]|https://github.com/pcmanus/cassandra/commit/020960d727b882349aff00c17a1c46bf3c7f7a24#diff-43b2d530031e2f7ad4286bd05fed4ca0R253]and this logic appears to not have made it back in [here|[https://github.com/apache/cassandra/commit/207c80c1fd63dfbd8ca7e615ec8002ee8983c5d6#diff-50a7bf78bc4f943f42ce60c8768484a6R400]] which meant an occasional tie, but maybe I'm misreading it all, lots of code to go through in that PR. The fix I used was one of the two proposed in 12997 (ie just using manual time stamp). > Fix flaky test org.apache.cassandra.cql3.validation.operations.AlterTest > testDropListAndAddListWithSameName > --- > > Key: CASSANDRA-15557 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15557 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: David Capwell >Assignee: Ryan Svihla >Priority: Normal > Fix For: 4.0-alpha > > > https://app.circleci.com/jobs/github/dcapwell/cassandra/482/tests > {code} > junit.framework.AssertionFailedError: Invalid value for row 0 column 2 > (mycollection of type list), expected but got <[first element]> > at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1070) > at > org.apache.cassandra.cql3.validation.operations.AlterTest.testDropListAndAddListWithSameName(AlterTest.java:91) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15620) Add "unleveled sstables" table metric
[ https://issues.apache.org/jira/browse/CASSANDRA-15620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Podkowinski updated CASSANDRA-15620: --- Change Category: Operability Complexity: Low Hanging Fruit Reviewers: Chris Lohfink Status: Open (was: Triage Needed) > Add "unleveled sstables" table metric > - > > Key: CASSANDRA-15620 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15620 > Project: Cassandra > Issue Type: Improvement > Components: Observability/Metrics >Reporter: Stefan Podkowinski >Assignee: Stefan Podkowinski >Priority: Normal > > The number of unleveled sstables is an important indicator that deserves to > be a dedicated table metric on its own. This will also add a global gauge > that is convenient to query. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15620) Add "unleveled sstables" table metric
Stefan Podkowinski created CASSANDRA-15620: -- Summary: Add "unleveled sstables" table metric Key: CASSANDRA-15620 URL: https://issues.apache.org/jira/browse/CASSANDRA-15620 Project: Cassandra Issue Type: Improvement Components: Observability/Metrics Reporter: Stefan Podkowinski Assignee: Stefan Podkowinski The number of unleveled sstables is an important indicator that deserves to be a dedicated table metric on its own. This will also add a global gauge that is convenient to query. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15557) Fix flaky test org.apache.cassandra.cql3.validation.operations.AlterTest testDropListAndAddListWithSameName
[ https://issues.apache.org/jira/browse/CASSANDRA-15557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-15557: --- Reviewers: Benjamin Lerer > Fix flaky test org.apache.cassandra.cql3.validation.operations.AlterTest > testDropListAndAddListWithSameName > --- > > Key: CASSANDRA-15557 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15557 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: David Capwell >Assignee: Ryan Svihla >Priority: Normal > Fix For: 4.0-alpha > > > https://app.circleci.com/jobs/github/dcapwell/cassandra/482/tests > {code} > junit.framework.AssertionFailedError: Invalid value for row 0 column 2 > (mycollection of type list), expected but got <[first element]> > at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1070) > at > org.apache.cassandra.cql3.validation.operations.AlterTest.testDropListAndAddListWithSameName(AlterTest.java:91) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org