date:20230503



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Miklosovic updated CASSANDRA-12937:
--
Reviewers: Stefan Miklosovic

> Default setting (yaml) for SSTable compression
> --
>
> Key: CASSANDRA-12937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12937
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Config
>Reporter: Michael Semb Wever
>Assignee: Claude Warren
>Priority: Low
>  Labels: AdventCalendar2021, lhf
> Fix For: 5.x
>
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> In many situations the choice of compression for sstables is more relevant to 
> the disks attached than to the schema and data.
> This issue is to add to cassandra.yaml a default value for sstable 
> compression that new tables will inherit (instead of the defaults found in 
> {{CompressionParams.DEFAULT}}.
> Examples where this can be relevant are filesystems that do on-the-fly 
> compression (btrfs, zfs) or specific disk configurations or even specific C* 
> versions (see CASSANDRA-10995 ).
> +Additional information for newcomers+
> Some new fields need to be added to {{cassandra.yaml}} to allow specifying 
> the field required for defining the default compression parameters. In 
> {{DatabaseDescriptor}} a new {{CompressionParams}} field should be added for 
> the default compression. This field should be initialized in 
> {{DatabaseDescriptor.applySimpleConfig()}}. At the different places where 
> {{CompressionParams.DEFAULT}} was used the code should call 
> {{DatabaseDescriptor#getDefaultCompressionParams}} that should return some 
> copy of configured {{CompressionParams}}.
> Some unit test using {{OverrideConfigurationLoader}} should be used to test 
> that the table schema use the new default when a new table is created (see 
> CreateTest for some example).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-12937) Default setting (yaml) for SSTable compression



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Miklosovic updated CASSANDRA-12937:
--
Status: Review In Progress  (was: Changes Suggested)

> Default setting (yaml) for SSTable compression
> --
>
> Key: CASSANDRA-12937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12937
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Config
>Reporter: Michael Semb Wever
>Assignee: Claude Warren
>Priority: Low
>  Labels: AdventCalendar2021, lhf
> Fix For: 5.x
>
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> In many situations the choice of compression for sstables is more relevant to 
> the disks attached than to the schema and data.
> This issue is to add to cassandra.yaml a default value for sstable 
> compression that new tables will inherit (instead of the defaults found in 
> {{CompressionParams.DEFAULT}}.
> Examples where this can be relevant are filesystems that do on-the-fly 
> compression (btrfs, zfs) or specific disk configurations or even specific C* 
> versions (see CASSANDRA-10995 ).
> +Additional information for newcomers+
> Some new fields need to be added to {{cassandra.yaml}} to allow specifying 
> the field required for defining the default compression parameters. In 
> {{DatabaseDescriptor}} a new {{CompressionParams}} field should be added for 
> the default compression. This field should be initialized in 
> {{DatabaseDescriptor.applySimpleConfig()}}. At the different places where 
> {{CompressionParams.DEFAULT}} was used the code should call 
> {{DatabaseDescriptor#getDefaultCompressionParams}} that should return some 
> copy of configured {{CompressionParams}}.
> Some unit test using {{OverrideConfigurationLoader}} should be used to test 
> that the table schema use the new default when a new table is created (see 
> CreateTest for some example).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14319) nodetool rebuild from DC lets you pass invalid datacenters



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719065#comment-17719065
 ] 

Stefan Miklosovic commented on CASSANDRA-14319:
---

[~vinaykumarcse] do you plan to finish this? Am I ok to assign this to myself? 
It seems like this patch is abandoned.

> nodetool rebuild from DC lets you pass invalid datacenters 
> ---
>
> Key: CASSANDRA-14319
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14319
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tool/nodetool
>Reporter: Jon Haddad
>Assignee: Vinay Chella
>Priority: Normal
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.11.x, 5.x
>
> Attachments: CASSANDRA-14319-trunk.txt
>
>
> If you pass an invalid datacenter to nodetool rebuild, you'll get an error 
> like this:
> {code}
> Unable to find sufficient sources for streaming range 
> (3074457345618258602,-9223372036854775808] in keyspace system_distributed
> {code}
> Unfortunately, this is a rabbit hole of frustration if you are using caps for 
> your DC names and you pass in a lowercase DC name, or you just typo the DC.  
> Let's do the following:
> # Check the DC name that's passed in against the list of DCs we know about
> # If we don't find it, let's output a reasonable error, and list all the DCs 
> someone could put in.
> # Ideally we indicate which keyspaces are set to replicate to this DC and 
> which aren't



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-18047) fix flaky o.a.c.distributed.test.PaxosRepair2Test.paxosRepairHistoryIsntUpdatedInForcedRepair

2023-05-03 Thread Jon Meredith (Jira)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-18047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jon Meredith updated CASSANDRA-18047:
-
  Fix Version/s: 4.1.2
 5.0
 (was: 5.x)
 (was: 4.1.x)
  Since Version: 4.1.0  (was: 4.1.x)
Source Control Link: 
https://github.com/apache/cassandra/commit/602ffcbf3e4ead4732fdf46d506165f63d80a9a4
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

> fix flaky 
> o.a.c.distributed.test.PaxosRepair2Test.paxosRepairHistoryIsntUpdatedInForcedRepair
> -
>
> Key: CASSANDRA-18047
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18047
> Project: Cassandra
>  Issue Type: Bug
>  Components: CI
>Reporter: Stefan Miklosovic
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.1.2, 5.0
>
>
> This test was introduced by CASSANDRA-18029
>  
> {code:java}
> junit.framework.AssertionFailedError: Repair failed with errors: [Repair 
> session 864c53d0-61fe-11ed-935f-5103a8e332f7 for range 
> [(-3074457345618258603,3074457345618258601], 
> (9223372036854775805,-3074457345618258603], 
> (3074457345618258601,9223372036854775805]] failed with error UNKNOWN failure 
> response from /127.0.0.3:7012, Repair command #1 finished with error] at 
> org.apache.cassandra.distributed.test.PaxosRepair2Test.lambda$repair$54f7d7c2$1(PaxosRepair2Test.java:186)
>  at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96) at 
> org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61) at 
> org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71) at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>  at java.lang.Thread.run(Thread.java:748)
> {code}
> different error:
> {code:java}
> junit.framework.AssertionFailedError: Repair failed with errors: [Endpoint 
> not alive: /127.0.0.3:7012, Repair command #1 finished with error]
>   at 
> org.apache.cassandra.distributed.test.PaxosRepair2Test.lambda$repair$54f7d7c2$1(PaxosRepair2Test.java:186)
>   at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
>   at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
>   at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>   at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] 01/01: Merge branch 'cassandra-4.1' into trunk

2023-05-03 Thread jonmeredith

This is an automated email from the ASF dual-hosted git repository.

jonmeredith pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git

commit ccf37892199d76c1bebcaa98e0e1b15400bd8a91
Merge: 73da05f83b 602ffcbf3e
Author: Jon Meredith 
AuthorDate: Wed May 3 14:42:57 2023 -0600

Merge branch 'cassandra-4.1' into trunk

 .../distributed/test/PaxosRepair2Test.java | 246 ++---
 1 file changed, 122 insertions(+), 124 deletions(-)

diff --cc 
test/distributed/org/apache/cassandra/distributed/test/PaxosRepair2Test.java
index a66bf1bd32,574b84f2eb..804353028b
--- 
a/test/distributed/org/apache/cassandra/distributed/test/PaxosRepair2Test.java
+++ 
b/test/distributed/org/apache/cassandra/distributed/test/PaxosRepair2Test.java
@@@ -66,10 -63,8 +63,9 @@@ import org.apache.cassandra.distributed
  import org.apache.cassandra.distributed.api.ConsistencyLevel;
  import org.apache.cassandra.distributed.api.Feature;
  import org.apache.cassandra.distributed.api.IInvokableInstance;
 +import org.apache.cassandra.distributed.shared.ClusterUtils;
  import org.apache.cassandra.exceptions.CasWriteTimeoutException;
  import org.apache.cassandra.gms.FailureDetector;
- import org.apache.cassandra.gms.Gossiper;
  import org.apache.cassandra.locator.InetAddressAndPort;
  import org.apache.cassandra.net.Verb;
  import org.apache.cassandra.repair.RepairParallelism;
@@@ -256,14 -239,12 +240,12 @@@ public class PaxosRepair2Test extends T
  )
  {
  cluster.schemaChange("CREATE TABLE " + KEYSPACE + '.' + TABLE + " 
(k int primary key, v int)");
 -cluster.get(3).shutdown().get();
 +ClusterUtils.stopUnchecked(cluster.get(3));
  InetAddressAndPort node3 = 
InetAddressAndPort.getByAddress(cluster.get(3).broadcastAddress());
  
- for (int i = 0; i < 10; i++)
- {
- if (!cluster.get(1).callOnInstance(() -> 
FailureDetector.instance.isAlive(node3)))
- break;
- }
+ // make sure node1 knows node3 is down
+ Awaitility.waitAtMost(1,TimeUnit.MINUTES).until(
+ () -> !cluster.get(1).callOnInstance(() -> 
FailureDetector.instance.isAlive(node3)));
  
  repair(cluster, KEYSPACE, TABLE, true);
  for (int i = 0; i < cluster.size() - 1; i++)
@@@ -356,8 -338,13 +339,13 @@@
   
.set("truncate_request_timeout_in_ms", 1000L)))
  )
  {
+ cluster.forEach(i -> {
+ 
Assert.assertFalse(CassandraRelevantProperties.CLOCK_GLOBAL.isPresent());
+ Assert.assertEquals("1", 
System.getProperty("cassandra.auto_repair_frequency_seconds"));
+ Assert.assertEquals("true", 
System.getProperty("cassandra.disable_paxos_auto_repairs"));
+ });
  cluster.schemaChange("CREATE TABLE " + KEYSPACE + '.' + TABLE + " 
(pk int, ck int, v int, PRIMARY KEY (pk, ck))");
 -cluster.get(3).shutdown().get();
 +ClusterUtils.stopUnchecked(cluster.get(3));
  cluster.verbs(Verb.PAXOS_COMMIT_REQ).drop();
  try
  {


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch trunk updated (73da05f83b -> ccf3789219)

2023-05-03 Thread jonmeredith

This is an automated email from the ASF dual-hosted git repository.

jonmeredith pushed a change to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git


from 73da05f83b Replace usages of json-simple dependency by Jackson
 new 602ffcbf3e fix flaky 
o.a.c.distributed.test.PaxosRepair2Test.paxosRepairHistoryIsntUpdatedInForcedRepair
 new ccf3789219 Merge branch 'cassandra-4.1' into trunk

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../distributed/test/PaxosRepair2Test.java | 246 ++---
 1 file changed, 122 insertions(+), 124 deletions(-)


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch cassandra-4.1 updated: fix flaky o.a.c.distributed.test.PaxosRepair2Test.paxosRepairHistoryIsntUpdatedInForcedRepair

2023-05-03 Thread jonmeredith

This is an automated email from the ASF dual-hosted git repository.

jonmeredith pushed a commit to branch cassandra-4.1
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/cassandra-4.1 by this push:
 new 602ffcbf3e fix flaky 
o.a.c.distributed.test.PaxosRepair2Test.paxosRepairHistoryIsntUpdatedInForcedRepair
602ffcbf3e is described below

commit 602ffcbf3e4ead4732fdf46d506165f63d80a9a4
Author: Jon Meredith 
AuthorDate: Wed May 3 10:27:48 2023 -0600

fix flaky 
o.a.c.distributed.test.PaxosRepair2Test.paxosRepairHistoryIsntUpdatedInForcedRepair

patch by Jon Meredith; reviewed by Blake Eggleston for CASSANDRA-18047
---
 .../distributed/test/PaxosRepair2Test.java | 250 ++---
 1 file changed, 124 insertions(+), 126 deletions(-)

diff --git 
a/test/distributed/org/apache/cassandra/distributed/test/PaxosRepair2Test.java 
b/test/distributed/org/apache/cassandra/distributed/test/PaxosRepair2Test.java
index 8371bd4c4e..574b84f2eb 100644
--- 
a/test/distributed/org/apache/cassandra/distributed/test/PaxosRepair2Test.java
+++ 
b/test/distributed/org/apache/cassandra/distributed/test/PaxosRepair2Test.java
@@ -18,19 +18,16 @@
 
 package org.apache.cassandra.distributed.test;
 
-import java.net.InetAddress;
 import java.util.Collection;
 import java.util.Collections;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
-import java.util.Set;
 import java.util.concurrent.TimeUnit;
-import java.util.stream.Collectors;
 
-import com.google.common.collect.ImmutableSet;
 import com.google.common.collect.Iterators;
 import com.google.common.collect.Sets;
+import org.awaitility.Awaitility;
 import org.junit.Assert;
 import org.junit.Test;
 import org.slf4j.Logger;
@@ -68,7 +65,6 @@ import org.apache.cassandra.distributed.api.Feature;
 import org.apache.cassandra.distributed.api.IInvokableInstance;
 import org.apache.cassandra.exceptions.CasWriteTimeoutException;
 import org.apache.cassandra.gms.FailureDetector;
-import org.apache.cassandra.gms.Gossiper;
 import org.apache.cassandra.locator.InetAddressAndPort;
 import org.apache.cassandra.net.Verb;
 import org.apache.cassandra.repair.RepairParallelism;
@@ -106,6 +102,7 @@ public class PaxosRepair2Test extends TestBaseImpl
 {
 private static final Logger logger = 
LoggerFactory.getLogger(PaxosRepair2Test.class);
 private static final String TABLE = "tbl";
+public static final String OFFSETTABLE_CLOCK_NAME = 
OffsettableClock.class.getName();
 
 static
 {
@@ -125,19 +122,6 @@ public class PaxosRepair2Test extends TestBaseImpl
 return uncommitted;
 }
 
-private static void assertAllAlive(Cluster cluster)
-{
-Set allEndpoints = cluster.stream().map(i -> 
i.broadcastAddress().getAddress()).collect(Collectors.toSet());
-cluster.stream().forEach(instance -> {
-instance.runOnInstance(() -> {
-ImmutableSet endpoints = 
Gossiper.instance.getEndpoints();
-Assert.assertEquals(allEndpoints, endpoints);
-for (InetAddressAndPort endpoint : endpoints)
-
Assert.assertTrue(FailureDetector.instance.isAlive(endpoint));
-});
-});
-}
-
 private static void assertUncommitted(IInvokableInstance instance, String 
ks, String table, int expected)
 {
 Assert.assertEquals(expected, getUncommitted(instance, ks, table));
@@ -255,19 +239,18 @@ public class PaxosRepair2Test extends TestBaseImpl
 )
 {
 cluster.schemaChange("CREATE TABLE " + KEYSPACE + '.' + TABLE + " 
(k int primary key, v int)");
-cluster.get(3).shutdown();
+cluster.get(3).shutdown().get();
 InetAddressAndPort node3 = 
InetAddressAndPort.getByAddress(cluster.get(3).broadcastAddress());
 
-for (int i = 0; i < 10; i++)
-{
-if (!cluster.get(1).callOnInstance(() -> 
FailureDetector.instance.isAlive(node3)))
-break;
-}
+// make sure node1 knows node3 is down
+Awaitility.waitAtMost(1,TimeUnit.MINUTES).until(
+() -> !cluster.get(1).callOnInstance(() -> 
FailureDetector.instance.isAlive(node3)));
 
 repair(cluster, KEYSPACE, TABLE, true);
 for (int i = 0; i < cluster.size() - 1; i++)
 {
 cluster.get(i + 1).runOnInstance(() -> {
+
Assert.assertFalse(CassandraRelevantProperties.CLOCK_GLOBAL.isPresent());
 ColumnFamilyStore cfs = 
Keyspace.open(KEYSPACE).getColumnFamilyStore(TABLE);
 DecoratedKey key = 
cfs.decorateKey(ByteBufferUtil.bytes(1));
 
Assert.assertTrue(FBUtilities.getBroadcastAddressAndPort().toString(), 
Commit.isAfter(staleBallot, cfs.getPaxosRepairLowBound(key)));
@@ -355,8 +338,13 @@ public class PaxosRepair2Test extends TestBaseImpl

[jira] [Commented] (CASSANDRA-18047) fix flaky o.a.c.distributed.test.PaxosRepair2Test.paxosRepairHistoryIsntUpdatedInForcedRepair

2023-05-03 Thread Jon Meredith (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-18047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719055#comment-17719055
 ] 

Jon Meredith commented on CASSANDRA-18047:
--

Only test failure was OOM in the junit formatter. Proceeding with commit.


for future ref
https://app.circleci.com/pipelines/github/jonmeredith/cassandra/804/workflows/aeae322e-da34-4dca-9e01-a2dd257d2ce0/jobs/9578/parallel-runs/8?filterBy=FAILED
{code}
[junit-timeout] Testsuite: 
org.apache.cassandra.distributed.test.PaxosRepair2Test
[junit-timeout] Exception in thread "main" java.lang.OutOfMemoryError: Java 
heap space
[junit-timeout] at java.util.Arrays.copyOf(Arrays.java:3332)
[junit-timeout] at 
java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
[junit-timeout] at 
java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448)
[junit-timeout] at java.lang.StringBuffer.append(StringBuffer.java:276)
[junit-timeout] at 
org.apache.cassandra.CassandraBriefJUnitResultFormatter.endTestSuite(CassandraBriefJUnitResultFormatter.java:174)
[junit-timeout] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.fireEndTestSuite(JUnitTestRunner.java:853)
[junit-timeout] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:577)
[junit-timeout] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1196)
[junit-timeout] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1041)
[junit-timeout] Testsuite: 
org.apache.cassandra.distributed.test.PaxosRepair2Test
[junit-timeout] Testsuite: 
org.apache.cassandra.distributed.test.PaxosRepair2Test Tests run: 1, Failures: 
0, Errors: 1, Skipped: 0, Time elapsed: 0 sec
[junit-timeout]
[junit-timeout] Testcase: 
org.apache.cassandra.distributed.test.PaxosRepair2Test:paxosRepairHistoryIsntUpdatedInForcedRepair:
   Caused an ERROR
[junit-timeout] Forked Java VM exited abnormally. Please note the time in the 
report does not reflect the time until the VM exit.
[junit-timeout] junit.framework.AssertionFailedError: Forked Java VM exited 
abnormally. Please note the time in the report does not reflect the time until 
the VM
exit.
[junit-timeout] at java.util.Vector.forEach(Vector.java:1277)
[junit-timeout] at java.util.Vector.forEach(Vector.java:1277)
{code}

> fix flaky 
> o.a.c.distributed.test.PaxosRepair2Test.paxosRepairHistoryIsntUpdatedInForcedRepair
> -
>
> Key: CASSANDRA-18047
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18047
> Project: Cassandra
>  Issue Type: Bug
>  Components: CI
>Reporter: Stefan Miklosovic
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.1.x, 5.x
>
>
> This test was introduced by CASSANDRA-18029
>  
> {code:java}
> junit.framework.AssertionFailedError: Repair failed with errors: [Repair 
> session 864c53d0-61fe-11ed-935f-5103a8e332f7 for range 
> [(-3074457345618258603,3074457345618258601], 
> (9223372036854775805,-3074457345618258603], 
> (3074457345618258601,9223372036854775805]] failed with error UNKNOWN failure 
> response from /127.0.0.3:7012, Repair command #1 finished with error] at 
> org.apache.cassandra.distributed.test.PaxosRepair2Test.lambda$repair$54f7d7c2$1(PaxosRepair2Test.java:186)
>  at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96) at 
> org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61) at 
> org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71) at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>  at java.lang.Thread.run(Thread.java:748)
> {code}
> different error:
> {code:java}
> junit.framework.AssertionFailedError: Repair failed with errors: [Endpoint 
> not alive: /127.0.0.3:7012, Repair command #1 finished with error]
>   at 
> org.apache.cassandra.distributed.test.PaxosRepair2Test.lambda$repair$54f7d7c2$1(PaxosRepair2Test.java:186)
>   at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
>   at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
>   at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at 
>

[jira] [Updated] (CASSANDRA-18355) CEP-15: Transaction Result Serialization Efficiency

2023-05-03 Thread Caleb Rackliffe (Jira)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-18355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Caleb Rackliffe updated CASSANDRA-18355:

Reviewers: Blake Eggleston, David Capwell  (was: David Capwell)

> CEP-15: Transaction Result Serialization Efficiency
> ---
>
> Key: CASSANDRA-18355
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18355
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Accord
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: NA
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> There are two things we probably don’t need to serialize and write to the 
> Accord state tables:
>  
> 1.) Internal/external read responses
> 2.) The full result of the transaction



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8720) Provide tools for finding wide row/partition keys



[ 
https://issues.apache.org/jira/browse/CASSANDRA-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719025#comment-17719025
 ] 

Stefan Miklosovic commented on CASSANDRA-8720:
--

Sure, lets just ship this. If somebody saves the output and finds the max, it 
is not hard to additionally grep it to see what particular SSTable it belongs 
to. I am ok to have another ticket as you suggested.

> Provide tools for finding wide row/partition keys
> -
>
> Key: CASSANDRA-8720
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8720
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Legacy/Tools
>Reporter: J.B. Langston
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 5.x
>
> Attachments: 8720.txt
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Multiple users have requested some sort of tool to help identify wide row 
> keys. They get into a situation where they know a wide row/partition has been 
> inserted and it's causing problems for them but they have no idea what the 
> row key is in order to remove it.  
> Maintaining the widest row key currently encountered and displaying it in 
> cfstats would be one possible approach.
> Another would be an offline tool (possibly an enhancement to sstablekeys) to 
> show the number of columns/bytes per key in each sstable. If a tool to 
> aggregate the information at a CF-level could be provided that would be a 
> bonus, but it shouldn't be too hard to write a script wrapper to aggregate 
> them if not.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8720) Provide tools for finding wide row/partition keys



[ 
https://issues.apache.org/jira/browse/CASSANDRA-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719016#comment-17719016
 ] 

Andres de la Peña commented on CASSANDRA-8720:
--

[~brandon.williams] thanks for the review and creating the ticket for 
documentation.

[~smiklosovic] are you ok with the current approach for directories? I think we 
can have a followup ticket for adding a kind of {{--aggregate}} flag to 
aggregate the statistics of all the sstables on the same directory/table. That 
wouldn't reconcile the partitions but just sum their metrics, so if a partition 
has 100MiB and 50MiB on another, it would be listed once as a 150Mib partition.

> Provide tools for finding wide row/partition keys
> -
>
> Key: CASSANDRA-8720
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8720
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Legacy/Tools
>Reporter: J.B. Langston
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 5.x
>
> Attachments: 8720.txt
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Multiple users have requested some sort of tool to help identify wide row 
> keys. They get into a situation where they know a wide row/partition has been 
> inserted and it's causing problems for them but they have no idea what the 
> row key is in order to remove it.  
> Maintaining the widest row key currently encountered and displaying it in 
> cfstats would be one possible approach.
> Another would be an offline tool (possibly an enhancement to sstablekeys) to 
> show the number of columns/bytes per key in each sstable. If a tool to 
> aggregate the information at a CF-level could be provided that would be a 
> bonus, but it shouldn't be too hard to write a script wrapper to aggregate 
> them if not.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-18047) fix flaky o.a.c.distributed.test.PaxosRepair2Test.paxosRepairHistoryIsntUpdatedInForcedRepair

2023-05-03 Thread Jon Meredith (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-18047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718998#comment-17718998
 ] 

Jon Meredith commented on CASSANDRA-18047:
--

Starting commit

CI Results (pending):
||Branch||Source||Circle CI||Jenkins||
|cassandra-4.1|[branch|https://github.com/jonmeredith/cassandra/tree/commit_remote_branch/CASSANDRA-18047-cassandra-4.1-0D9D9D6B-4A3B-4AF4-A734-514C7025FF98]|[build|https://app.circleci.com/pipelines/github/jonmeredith/cassandra?branch=commit_remote_branch%2FCASSANDRA-18047-cassandra-4.1-0D9D9D6B-4A3B-4AF4-A734-514C7025FF98]|[build|https://ci-cassandra.apache.org/job/Cassandra-devbranch/2446/]|
|trunk|[branch|https://github.com/jonmeredith/cassandra/tree/commit_remote_branch/CASSANDRA-18047-trunk-0D9D9D6B-4A3B-4AF4-A734-514C7025FF98]|[build|https://app.circleci.com/pipelines/github/jonmeredith/cassandra?branch=commit_remote_branch%2FCASSANDRA-18047-trunk-0D9D9D6B-4A3B-4AF4-A734-514C7025FF98]|[build|https://ci-cassandra.apache.org/job/Cassandra-devbranch/2447/]|


> fix flaky 
> o.a.c.distributed.test.PaxosRepair2Test.paxosRepairHistoryIsntUpdatedInForcedRepair
> -
>
> Key: CASSANDRA-18047
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18047
> Project: Cassandra
>  Issue Type: Bug
>  Components: CI
>Reporter: Stefan Miklosovic
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.1.x, 5.x
>
>
> This test was introduced by CASSANDRA-18029
>  
> {code:java}
> junit.framework.AssertionFailedError: Repair failed with errors: [Repair 
> session 864c53d0-61fe-11ed-935f-5103a8e332f7 for range 
> [(-3074457345618258603,3074457345618258601], 
> (9223372036854775805,-3074457345618258603], 
> (3074457345618258601,9223372036854775805]] failed with error UNKNOWN failure 
> response from /127.0.0.3:7012, Repair command #1 finished with error] at 
> org.apache.cassandra.distributed.test.PaxosRepair2Test.lambda$repair$54f7d7c2$1(PaxosRepair2Test.java:186)
>  at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96) at 
> org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61) at 
> org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71) at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>  at java.lang.Thread.run(Thread.java:748)
> {code}
> different error:
> {code:java}
> junit.framework.AssertionFailedError: Repair failed with errors: [Endpoint 
> not alive: /127.0.0.3:7012, Repair command #1 finished with error]
>   at 
> org.apache.cassandra.distributed.test.PaxosRepair2Test.lambda$repair$54f7d7c2$1(PaxosRepair2Test.java:186)
>   at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
>   at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
>   at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>   at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16999) system.peers and system.peers_v2 do not contain the native_transport and/or native_transport_port_ssl



[ 
https://issues.apache.org/jira/browse/CASSANDRA-16999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718990#comment-17718990
 ] 

Brandon Williams commented on CASSANDRA-16999:
--

[~absurdfarce] what do you think?  It would be nice to get this in before 5.0 
releases.

> system.peers and system.peers_v2 do not contain the native_transport and/or 
> native_transport_port_ssl
> -
>
> Key: CASSANDRA-16999
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16999
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Other
>Reporter: Steve Lacerda
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.x
>
>
> system.peers_v2 includes a “native_port” but has no notion of 
> native_transport_port vs. native_transport_port_ssl.  Given this limited 
> information, there’s no clear way for the driver to know that different ports 
> are being used for SSL vs. non-SSL or which of those two ports is identified 
> by “native_port”.
>  
> The issue we ran into is that the java driver, since it has no notion of the 
> transport port SSL, the driver was only using the contact points and was not 
> load balancing.
>  
> The customer had both set:
> native_transport_port: 9042
> native_transport_port_ssl: 9142
>  
> They were attempting to connect to 9142, but that was failing. They could 
> only use 9042, and so their applications load balancing was failing. We found 
> that any node that was a contact point was connecting, but the other nodes 
> were never acting as coordinators.
>  
> There are still issues in the driver, for which I have created JAVA-2967, 
> which also refers to JAVA-2638, but the system.peers and system.peers_v2 
> tables should both contain native_transport_port and 
> native_transport_port_ssl.
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-18047) fix flaky o.a.c.distributed.test.PaxosRepair2Test.paxosRepairHistoryIsntUpdatedInForcedRepair

2023-05-03 Thread Blake Eggleston (Jira)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-18047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-18047:

Status: Ready to Commit  (was: Review In Progress)

LGTM, thanks

> fix flaky 
> o.a.c.distributed.test.PaxosRepair2Test.paxosRepairHistoryIsntUpdatedInForcedRepair
> -
>
> Key: CASSANDRA-18047
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18047
> Project: Cassandra
>  Issue Type: Bug
>  Components: CI
>Reporter: Stefan Miklosovic
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.1.x, 5.x
>
>
> This test was introduced by CASSANDRA-18029
>  
> {code:java}
> junit.framework.AssertionFailedError: Repair failed with errors: [Repair 
> session 864c53d0-61fe-11ed-935f-5103a8e332f7 for range 
> [(-3074457345618258603,3074457345618258601], 
> (9223372036854775805,-3074457345618258603], 
> (3074457345618258601,9223372036854775805]] failed with error UNKNOWN failure 
> response from /127.0.0.3:7012, Repair command #1 finished with error] at 
> org.apache.cassandra.distributed.test.PaxosRepair2Test.lambda$repair$54f7d7c2$1(PaxosRepair2Test.java:186)
>  at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96) at 
> org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61) at 
> org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71) at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>  at java.lang.Thread.run(Thread.java:748)
> {code}
> different error:
> {code:java}
> junit.framework.AssertionFailedError: Repair failed with errors: [Endpoint 
> not alive: /127.0.0.3:7012, Repair command #1 finished with error]
>   at 
> org.apache.cassandra.distributed.test.PaxosRepair2Test.lambda$repair$54f7d7c2$1(PaxosRepair2Test.java:186)
>   at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
>   at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
>   at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>   at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-18047) fix flaky o.a.c.distributed.test.PaxosRepair2Test.paxosRepairHistoryIsntUpdatedInForcedRepair

2023-05-03 Thread Blake Eggleston (Jira)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-18047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-18047:

Test and Documentation Plan: circle
 Status: Patch Available  (was: In Progress)

> fix flaky 
> o.a.c.distributed.test.PaxosRepair2Test.paxosRepairHistoryIsntUpdatedInForcedRepair
> -
>
> Key: CASSANDRA-18047
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18047
> Project: Cassandra
>  Issue Type: Bug
>  Components: CI
>Reporter: Stefan Miklosovic
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.1.x, 5.x
>
>
> This test was introduced by CASSANDRA-18029
>  
> {code:java}
> junit.framework.AssertionFailedError: Repair failed with errors: [Repair 
> session 864c53d0-61fe-11ed-935f-5103a8e332f7 for range 
> [(-3074457345618258603,3074457345618258601], 
> (9223372036854775805,-3074457345618258603], 
> (3074457345618258601,9223372036854775805]] failed with error UNKNOWN failure 
> response from /127.0.0.3:7012, Repair command #1 finished with error] at 
> org.apache.cassandra.distributed.test.PaxosRepair2Test.lambda$repair$54f7d7c2$1(PaxosRepair2Test.java:186)
>  at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96) at 
> org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61) at 
> org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71) at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>  at java.lang.Thread.run(Thread.java:748)
> {code}
> different error:
> {code:java}
> junit.framework.AssertionFailedError: Repair failed with errors: [Endpoint 
> not alive: /127.0.0.3:7012, Repair command #1 finished with error]
>   at 
> org.apache.cassandra.distributed.test.PaxosRepair2Test.lambda$repair$54f7d7c2$1(PaxosRepair2Test.java:186)
>   at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
>   at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
>   at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>   at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-18047) fix flaky o.a.c.distributed.test.PaxosRepair2Test.paxosRepairHistoryIsntUpdatedInForcedRepair

2023-05-03 Thread Blake Eggleston (Jira)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-18047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-18047:

Reviewers: Blake Eggleston, Blake Eggleston  (was: Caleb Rackliffe)
   Blake Eggleston, Blake Eggleston  (was: Blake Eggleston)
   Status: Review In Progress  (was: Patch Available)

> fix flaky 
> o.a.c.distributed.test.PaxosRepair2Test.paxosRepairHistoryIsntUpdatedInForcedRepair
> -
>
> Key: CASSANDRA-18047
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18047
> Project: Cassandra
>  Issue Type: Bug
>  Components: CI
>Reporter: Stefan Miklosovic
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.1.x, 5.x
>
>
> This test was introduced by CASSANDRA-18029
>  
> {code:java}
> junit.framework.AssertionFailedError: Repair failed with errors: [Repair 
> session 864c53d0-61fe-11ed-935f-5103a8e332f7 for range 
> [(-3074457345618258603,3074457345618258601], 
> (9223372036854775805,-3074457345618258603], 
> (3074457345618258601,9223372036854775805]] failed with error UNKNOWN failure 
> response from /127.0.0.3:7012, Repair command #1 finished with error] at 
> org.apache.cassandra.distributed.test.PaxosRepair2Test.lambda$repair$54f7d7c2$1(PaxosRepair2Test.java:186)
>  at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96) at 
> org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61) at 
> org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71) at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>  at java.lang.Thread.run(Thread.java:748)
> {code}
> different error:
> {code:java}
> junit.framework.AssertionFailedError: Repair failed with errors: [Endpoint 
> not alive: /127.0.0.3:7012, Repair command #1 finished with error]
>   at 
> org.apache.cassandra.distributed.test.PaxosRepair2Test.lambda$repair$54f7d7c2$1(PaxosRepair2Test.java:186)
>   at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
>   at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
>   at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>   at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-18047) fix flaky o.a.c.distributed.test.PaxosRepair2Test.paxosRepairHistoryIsntUpdatedInForcedRepair

2023-05-03 Thread Caleb Rackliffe (Jira)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-18047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Caleb Rackliffe updated CASSANDRA-18047:

Reviewers: Caleb Rackliffe

> fix flaky 
> o.a.c.distributed.test.PaxosRepair2Test.paxosRepairHistoryIsntUpdatedInForcedRepair
> -
>
> Key: CASSANDRA-18047
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18047
> Project: Cassandra
>  Issue Type: Bug
>  Components: CI
>Reporter: Stefan Miklosovic
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.1.x, 5.x
>
>
> This test was introduced by CASSANDRA-18029
>  
> {code:java}
> junit.framework.AssertionFailedError: Repair failed with errors: [Repair 
> session 864c53d0-61fe-11ed-935f-5103a8e332f7 for range 
> [(-3074457345618258603,3074457345618258601], 
> (9223372036854775805,-3074457345618258603], 
> (3074457345618258601,9223372036854775805]] failed with error UNKNOWN failure 
> response from /127.0.0.3:7012, Repair command #1 finished with error] at 
> org.apache.cassandra.distributed.test.PaxosRepair2Test.lambda$repair$54f7d7c2$1(PaxosRepair2Test.java:186)
>  at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96) at 
> org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61) at 
> org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71) at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>  at java.lang.Thread.run(Thread.java:748)
> {code}
> different error:
> {code:java}
> junit.framework.AssertionFailedError: Repair failed with errors: [Endpoint 
> not alive: /127.0.0.3:7012, Repair command #1 finished with error]
>   at 
> org.apache.cassandra.distributed.test.PaxosRepair2Test.lambda$repair$54f7d7c2$1(PaxosRepair2Test.java:186)
>   at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
>   at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
>   at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>   at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-8720) Provide tools for finding wide row/partition keys



[ 
https://issues.apache.org/jira/browse/CASSANDRA-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718962#comment-17718962
 ] 

Brandon Williams edited comment on CASSANDRA-8720 at 5/3/23 3:24 PM:
-

To avoid needing to wait on Jenkins, I manually built and tested the debs and 
rpms and they look good.  I also created CASSANDRA-18496 to review the docs 
about needing to stop the server, since it wasn't necessary here.  +1 from me.


was (Author: brandon.williams):
To avoid needing to wait on Jenkins, I manually build and tested the debs and 
rpms and they look good.  I also created CASSANDRA-18496 to review the docs 
about needing to stop the server, since it wasn't necessary here.  +1 from me.

> Provide tools for finding wide row/partition keys
> -
>
> Key: CASSANDRA-8720
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8720
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Legacy/Tools
>Reporter: J.B. Langston
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 5.x
>
> Attachments: 8720.txt
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Multiple users have requested some sort of tool to help identify wide row 
> keys. They get into a situation where they know a wide row/partition has been 
> inserted and it's causing problems for them but they have no idea what the 
> row key is in order to remove it.  
> Maintaining the widest row key currently encountered and displaying it in 
> cfstats would be one possible approach.
> Another would be an offline tool (possibly an enhancement to sstablekeys) to 
> show the number of columns/bytes per key in each sstable. If a tool to 
> aggregate the information at a CF-level could be provided that would be a 
> bonus, but it shouldn't be too hard to write a script wrapper to aggregate 
> them if not.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-8720) Provide tools for finding wide row/partition keys



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-8720:

Status: Ready to Commit  (was: Review In Progress)

> Provide tools for finding wide row/partition keys
> -
>
> Key: CASSANDRA-8720
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8720
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Legacy/Tools
>Reporter: J.B. Langston
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 5.x
>
> Attachments: 8720.txt
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Multiple users have requested some sort of tool to help identify wide row 
> keys. They get into a situation where they know a wide row/partition has been 
> inserted and it's causing problems for them but they have no idea what the 
> row key is in order to remove it.  
> Maintaining the widest row key currently encountered and displaying it in 
> cfstats would be one possible approach.
> Another would be an offline tool (possibly an enhancement to sstablekeys) to 
> show the number of columns/bytes per key in each sstable. If a tool to 
> aggregate the information at a CF-level could be provided that would be a 
> bonus, but it shouldn't be too hard to write a script wrapper to aggregate 
> them if not.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8720) Provide tools for finding wide row/partition keys



[ 
https://issues.apache.org/jira/browse/CASSANDRA-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718962#comment-17718962
 ] 

Brandon Williams commented on CASSANDRA-8720:
-

To avoid needing to wait on Jenkins, I manually build and tested the debs and 
rpms and they look good.  I also created CASSANDRA-18496 to review the docs 
about needing to stop the server, since it wasn't necessary here.  +1 from me.

> Provide tools for finding wide row/partition keys
> -
>
> Key: CASSANDRA-8720
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8720
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Legacy/Tools
>Reporter: J.B. Langston
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 5.x
>
> Attachments: 8720.txt
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Multiple users have requested some sort of tool to help identify wide row 
> keys. They get into a situation where they know a wide row/partition has been 
> inserted and it's causing problems for them but they have no idea what the 
> row key is in order to remove it.  
> Maintaining the widest row key currently encountered and displaying it in 
> cfstats would be one possible approach.
> Another would be an offline tool (possibly an enhancement to sstablekeys) to 
> show the number of columns/bytes per key in each sstable. If a tool to 
> aggregate the information at a CF-level could be provided that would be a 
> bonus, but it shouldn't be too hard to write a script wrapper to aggregate 
> them if not.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-18496) Review documentation about needing to stop the server for sstable tools

Brandon Williams created CASSANDRA-18496:


 Summary: Review documentation about needing to stop the server for 
sstable tools
 Key: CASSANDRA-18496
 URL: https://issues.apache.org/jira/browse/CASSANDRA-18496
 Project: Cassandra
  Issue Type: Bug
Reporter: Brandon Williams


For sstable tools the docs often mention:

bq. Cassandra must be stopped before this tool is executed, or unexpected 
results will occur. 

In CASSANDRA-8720 it was noticed this is not required, and we should probably 
review the instances of it that we have for accuracy.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-18496) Review documentation about needing to stop the server for sstable tools



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-18496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-18496:
-
 Bug Category: Parent values: Correctness(12982)
   Complexity: Normal
  Component/s: Documentation
Discovered By: User Report
Fix Version/s: 3.0.x
   3.11.x
   4.0.x
   5.x
 Severity: Normal
   Status: Open  (was: Triage Needed)

> Review documentation about needing to stop the server for sstable tools
> ---
>
> Key: CASSANDRA-18496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18496
> Project: Cassandra
>  Issue Type: Bug
>  Components: Documentation
>Reporter: Brandon Williams
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 5.x
>
>
> For sstable tools the docs often mention:
> bq. Cassandra must be stopped before this tool is executed, or unexpected 
> results will occur. 
> In CASSANDRA-8720 it was noticed this is not required, and we should probably 
> review the instances of it that we have for accuracy.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-18495) Warnings when using the perl driver to connect to Cassandra



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-18495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-18495:
-
Resolution: Duplicate
Status: Resolved  (was: Triage Needed)

I believe this is a duplicate of CASSANDRA-18322, only in this case preparing a 
"USE" statement doesn't make sense.

> Warnings when using the perl driver to connect to Cassandra
> ---
>
> Key: CASSANDRA-18495
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18495
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Aswin Karthik
>Priority: Normal
>
> When I use the [perl driver|https://github.com/TvdW/perl-DBD-Cassandra] to 
> connect to Cassandra 4.0.3 and onwards, I get the following error just for 
> initialization of a connection.
> The error I get is
> {noformat}
> `USE ` with prepared statements is considered to be an anti-pattern 
> due to ambiguity in non-qualified table names. Please consider removing 
> instances of `Session#setKeyspace()`, `Session#execute("USE 
> ")` and `cluster.newSession()` from your code, and always 
> use fully qualified table names (e.g. .). Keyspace used: 
> null, statement keyspace: null, statement id:  at 
> Cassandra/Client/Connection.pm line 957.
> {noformat}
>  
> This is just from initialization of the perl driver connection while choosing 
> the keyspace. 
> [https://github.com/TvdW/perl-DBD-Cassandra/blob/master/Cassandra-Client/lib/Cassandra/Client/Connection.pm#L562]
>  before running any queries. It is emitted even if I use qualified prepared 
> statement.
> The same warning does not pop up in Datastax java driver initialization.
> On debugging, I found that this warning does not emit for all unqualified 
> prepared statements. It only emits for unqualified prepared "USE ks" 
> statement. But the "USE ks" can never be qualified. So the warning is a bit 
> vague on what is the recommended approach.
> And from the perspective of the driver, it is setting the keyspace of the 
> connection for the first time. The same warning does not happen on the 
> datastax java driver and that is because it uses QUERY to set the keyspace on 
> connection. (I tried to follow the same approach on the perl driver - 
> https://github.com/TvdW/perl-DBD-Cassandra/pull/35 )
> The warnings are not very clear on what is deprecated and what is not. Does 
> it deprecate only the use of prepared statement of "USE ks"? or does it 
> deprecate "USE ks" completely? And it is not being emitted for other 
> unqualified prepared statements but only for a USE statement which cannot be 
> qualified at all.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-8720) Provide tools for finding wide row/partition keys



[ 
https://issues.apache.org/jira/browse/CASSANDRA-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718946#comment-17718946
 ] 

Andres de la Peña edited comment on CASSANDRA-8720 at 5/3/23 2:17 PM:
--

The tool accepts directories as arguments. It runs on every sstable on that 
directory, printing the stats for each sstable. What it doesn't do is any kind 
of reconciliation nor aggregation between the different fragments of a 
partition on each sstable.


was (Author: adelapena):
The tool accepts directories as arguments. It runs on every sstable on that 
directory, printing the stats for each sstable. What it doesn't do is any kind 
of reconciliation between the different fragments of a partition on each 
sstable.

> Provide tools for finding wide row/partition keys
> -
>
> Key: CASSANDRA-8720
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8720
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Legacy/Tools
>Reporter: J.B. Langston
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 5.x
>
> Attachments: 8720.txt
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Multiple users have requested some sort of tool to help identify wide row 
> keys. They get into a situation where they know a wide row/partition has been 
> inserted and it's causing problems for them but they have no idea what the 
> row key is in order to remove it.  
> Maintaining the widest row key currently encountered and displaying it in 
> cfstats would be one possible approach.
> Another would be an offline tool (possibly an enhancement to sstablekeys) to 
> show the number of columns/bytes per key in each sstable. If a tool to 
> aggregate the information at a CF-level could be provided that would be a 
> bonus, but it shouldn't be too hard to write a script wrapper to aggregate 
> them if not.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8720) Provide tools for finding wide row/partition keys



[ 
https://issues.apache.org/jira/browse/CASSANDRA-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718946#comment-17718946
 ] 

Andres de la Peña commented on CASSANDRA-8720:
--

The tool accepts directories as arguments. It runs on every sstable on that 
directory, printing the stats for each sstable. What it doesn't do is any kind 
of reconciliation between the different fragments of a partition on each 
sstable.

> Provide tools for finding wide row/partition keys
> -
>
> Key: CASSANDRA-8720
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8720
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Legacy/Tools
>Reporter: J.B. Langston
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 5.x
>
> Attachments: 8720.txt
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Multiple users have requested some sort of tool to help identify wide row 
> keys. They get into a situation where they know a wide row/partition has been 
> inserted and it's causing problems for them but they have no idea what the 
> row key is in order to remove it.  
> Maintaining the widest row key currently encountered and displaying it in 
> cfstats would be one possible approach.
> Another would be an offline tool (possibly an enhancement to sstablekeys) to 
> show the number of columns/bytes per key in each sstable. If a tool to 
> aggregate the information at a CF-level could be provided that would be a 
> bonus, but it shouldn't be too hard to write a script wrapper to aggregate 
> them if not.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8720) Provide tools for finding wide row/partition keys



[ 
https://issues.apache.org/jira/browse/CASSANDRA-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718942#comment-17718942
 ] 

Stefan Miklosovic commented on CASSANDRA-8720:
--

Having the possibility to specify a directory would be nice. A user could "grep 
it" on "max" line, "cut it" to take the second column where the size is and 
"sort it" to have the max. However, while doing so, we would lose the 
information what table that is. So a user would need to be a little bit smart 
about that. 

> Provide tools for finding wide row/partition keys
> -
>
> Key: CASSANDRA-8720
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8720
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Legacy/Tools
>Reporter: J.B. Langston
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 5.x
>
> Attachments: 8720.txt
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Multiple users have requested some sort of tool to help identify wide row 
> keys. They get into a situation where they know a wide row/partition has been 
> inserted and it's causing problems for them but they have no idea what the 
> row key is in order to remove it.  
> Maintaining the widest row key currently encountered and displaying it in 
> cfstats would be one possible approach.
> Another would be an offline tool (possibly an enhancement to sstablekeys) to 
> show the number of columns/bytes per key in each sstable. If a tool to 
> aggregate the information at a CF-level could be provided that would be a 
> bonus, but it shouldn't be too hard to write a script wrapper to aggregate 
> them if not.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8720) Provide tools for finding wide row/partition keys



[ 
https://issues.apache.org/jira/browse/CASSANDRA-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718941#comment-17718941
 ] 

Brandon Williams commented on CASSANDRA-8720:
-

bq. While this command works just fine for a single SSTable, if there is a 
table consisting of 300 SSTables, how it would look like? 

The doc says:

bq. You can supply any number of sstables file paths, or directories containing 
sstables.

If that is correct, can't we just pass the data dir?  You may have to collate 
the data a bit but you should still be able to find the partitions.

> Provide tools for finding wide row/partition keys
> -
>
> Key: CASSANDRA-8720
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8720
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Legacy/Tools
>Reporter: J.B. Langston
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 5.x
>
> Attachments: 8720.txt
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Multiple users have requested some sort of tool to help identify wide row 
> keys. They get into a situation where they know a wide row/partition has been 
> inserted and it's causing problems for them but they have no idea what the 
> row key is in order to remove it.  
> Maintaining the widest row key currently encountered and displaying it in 
> cfstats would be one possible approach.
> Another would be an offline tool (possibly an enhancement to sstablekeys) to 
> show the number of columns/bytes per key in each sstable. If a tool to 
> aggregate the information at a CF-level could be provided that would be a 
> bonus, but it shouldn't be too hard to write a script wrapper to aggregate 
> them if not.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-17292) Move cassandra.yaml toward a nested structure around major database concepts

2023-05-03 Thread Maxim Muzafarov (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718936#comment-17718936
 ] 

Maxim Muzafarov commented on CASSANDRA-17292:
-

Hello [~maedhroz],



I'd like to offer a few thoughts on that as well. I have been delving into the 
configuration usages while making the SettingsTable virtual table updatable and 
I think we can support nested structured configuration (with some limitations) 
as well as the flat one including storing the configuration in multiple files. 
Why can we do this? Well, we should move away from using the configuration as a 
POJO nested classes and store the configuration properties internally as a 
tree-based runtime data structure (the same concept is provided by Apache 
Commons Configuration, Lightbend Config etc.). This will give users a good deal 
of flexibility, so they can use/split their configuration as they wish.

I also mentioned some thoughts here in the {{= Alternatives =}} section, with 
all the drawbacks we might face:
[https://lists.apache.org/thread/gdtr3vp375d3nyj6h8xo7owth1s556lz]
h3. Why we can do so?

*The first thing* to note is that there is no need to map the yaml file 
structure directly to the POJO configuration classes, as these classes are not 
directly available to users and are only used in internal components. The only 
requirement is that we must clearly define the configuration properties on 
which the naming conversion is to be based: sub-components must be properly 
prefixed (we can align properties using @Replaces annotation or mapping).

So a user can use any kind of configuration below, we just need to load the 
configuration into our internal structure (or a POJO class) with an appropriate 
YamlLoader.

This is valid:
{code:java}
commitlog_directory: String
commitlog_max_compression_buffers_in_pool: int
commitlog_periodic_queue_size: int
{code}
This is also valid:
{code:java}
commitlog:
  directory: String
  max_compression_buffers_in_pool: int
  periodic_queue_size: int
{code}
This is a valid case if we split the configuration into multiple files and put 
them in the classpath to load:
{code:java}
// Let's assume Cassandra configurations yaml has 'cassandra.(.*).yaml' pattern.
cassandra.accord.yaml
cassandra.yaml
{code}
*The second thing* to note is how the whole configuration can be validated. I 
guess the answer here is relatively simple - we can reuse all the apply methods 
we have now (applySSTableFormats(), applySimpleConfig(), applyPartitioner()) 
keeping them almost 'as is'.

*The third thing* is that if we use a runtime tree-based structure to configure 
the Cassandra cluster, we are able to inject a configuration subtree right 
where it is needed. For example, @Configuration(prefix="commiglog"), so there 
will be no need to keep a layer with thousands of lines e.g. DatabaseDescriptor 
class in the source code to access the configuration. Of course, we will keep 
it to minimise the initial changes, but eventually, we can get rid of it.

{*}Last but not least{*}, we should think carefully about the performance of 
accessing configuration fields, as this could affect the performance of the 
cluster as a whole. Direct class field access is the fastest way we read a 
property value, but I think in the Cassandra project it might be OK to have 
O(1) guarantees. Some of the frameworks have configuration variables caching 
under the hood. For example, the Netflix/archaius has this 
[https://github.com/Netflix/archaius/blob/2.x/archaius2-core/src/main/java/com/netflix/archaius/DefaultPropertyFactory.java#L213],
 but the commons configuration doesn't seem to. If we go this way we will have 
to do benchmarks, but I think it will be faster enough within measurement error.
h3. Tree-based configuration frameworks

There are a lot of frameworks that store configuration in a runtime tree-based 
structure that might be considered for Cassandra: [Apache Commons 
Configuration|https://github.com/apache/commons-configuration], [Lightbend 
Config|https://github.com/lightbend/config], [Netflix 
Archaius|https://github.com/Netflix/archaius], and as I mentioned in the 
{{=Alternatives=}} section, we can consider adding the Apache Commons 
configuration. Adding something from 'apache commons' looks safer as we already 
have some libraries from 'commons', rather than adding a completely different 
configuration framework.

But whatever framework we consider, the following things need to be taken into 
account:
 - We have custom configuration datatypes such as DataStorageSpec, 
DataStorageSpec;
 - We have custom DurationSpec, so we either move them to Duration, preserving 
backwards compatibility for all supported APIs (yaml, JMX), or extend a 
considered framework with new types, we have to provide data type converters in 
the latter case;
 - An additional dependency, so the key component (configuration) of the 
project becomes

[jira] [Commented] (CASSANDRA-8720) Provide tools for finding wide row/partition keys



[ 
https://issues.apache.org/jira/browse/CASSANDRA-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718928#comment-17718928
 ] 

Andres de la Peña commented on CASSANDRA-8720:
--

I'm afraid that out of the scope of a sstable tool like the proposed 
{{sstablepartitions}}. Indeed it would be great to have a {{tablepartitions}} 
tool able to do that over reconciled partitions living on separate sstables, 
but I would do that on a separate ticket. That {{tablepartitions}} tool could 
probably reuse some of the {{sstablepartitions}} code. 

> Provide tools for finding wide row/partition keys
> -
>
> Key: CASSANDRA-8720
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8720
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Legacy/Tools
>Reporter: J.B. Langston
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 5.x
>
> Attachments: 8720.txt
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Multiple users have requested some sort of tool to help identify wide row 
> keys. They get into a situation where they know a wide row/partition has been 
> inserted and it's causing problems for them but they have no idea what the 
> row key is in order to remove it.  
> Maintaining the widest row key currently encountered and displaying it in 
> cfstats would be one possible approach.
> Another would be an offline tool (possibly an enhancement to sstablekeys) to 
> show the number of columns/bytes per key in each sstable. If a tool to 
> aggregate the information at a CF-level could be provided that would be a 
> bonus, but it shouldn't be too hard to write a script wrapper to aggregate 
> them if not.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-8720) Provide tools for finding wide row/partition keys



[ 
https://issues.apache.org/jira/browse/CASSANDRA-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718919#comment-17718919
 ] 

Stefan Miklosovic edited comment on CASSANDRA-8720 at 5/3/23 12:35 PM:
---

How would user find the biggest partition in a table consisting of multiple 
SSTables? A user would execute this command for each sstable? (and the he would 
need to compare results) Can not we have some kind of a "global" execution 
which would look into every table and the biggest partition would be reported? 
While this command works just fine for a single SSTable, if there is a table 
consisting of 300 SSTables, how it would look like? 


was (Author: smiklosovic):
How would user find the biggest partition in a table consisting of multiple 
SSTables? A user would execute this command for each sstable? Can not we have 
some kind of a "global" execution which would look into every table and the 
biggest partition would be reported? While this command works just fine for a 
single SSTable, if there is a table consisting of 300 SSTables, how it would 
look like? 

> Provide tools for finding wide row/partition keys
> -
>
> Key: CASSANDRA-8720
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8720
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Legacy/Tools
>Reporter: J.B. Langston
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 5.x
>
> Attachments: 8720.txt
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Multiple users have requested some sort of tool to help identify wide row 
> keys. They get into a situation where they know a wide row/partition has been 
> inserted and it's causing problems for them but they have no idea what the 
> row key is in order to remove it.  
> Maintaining the widest row key currently encountered and displaying it in 
> cfstats would be one possible approach.
> Another would be an offline tool (possibly an enhancement to sstablekeys) to 
> show the number of columns/bytes per key in each sstable. If a tool to 
> aggregate the information at a CF-level could be provided that would be a 
> bonus, but it shouldn't be too hard to write a script wrapper to aggregate 
> them if not.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8720) Provide tools for finding wide row/partition keys



[ 
https://issues.apache.org/jira/browse/CASSANDRA-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718919#comment-17718919
 ] 

Stefan Miklosovic commented on CASSANDRA-8720:
--

How would user find the biggest partition in a table consisting of multiple 
SSTables? A user would execute this command for each sstable? Can not we have 
some kind of a "global" execution which would look into every table and the 
biggest partition would be reported? While this command works just fine for a 
single SSTable, if there is a table consisting of 300 SSTables, how it would 
look like? 

> Provide tools for finding wide row/partition keys
> -
>
> Key: CASSANDRA-8720
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8720
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Legacy/Tools
>Reporter: J.B. Langston
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 5.x
>
> Attachments: 8720.txt
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Multiple users have requested some sort of tool to help identify wide row 
> keys. They get into a situation where they know a wide row/partition has been 
> inserted and it's causing problems for them but they have no idea what the 
> row key is in order to remove it.  
> Maintaining the widest row key currently encountered and displaying it in 
> cfstats would be one possible approach.
> Another would be an offline tool (possibly an enhancement to sstablekeys) to 
> show the number of columns/bytes per key in each sstable. If a tool to 
> aggregate the information at a CF-level could be provided that would be a 
> bonus, but it shouldn't be too hard to write a script wrapper to aggregate 
> them if not.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-8720) Provide tools for finding wide row/partition keys



[ 
https://issues.apache.org/jira/browse/CASSANDRA-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718911#comment-17718911
 ] 

Andres de la Peña edited comment on CASSANDRA-8720 at 5/3/23 12:06 PM:
---

Thanks :)

The data in the table comes from 
[{{EstimatedHistogram}}|https://github.com/apache/cassandra/blob/d2923275e360a1ee9db498e748c269f701bb3a8b/src/java/org/apache/cassandra/utils/EstimatedHistogram.java]s,
 like the ones we use on metrics such as the ones on 
[{{MetadataCollector}}|https://github.com/apache/cassandra/blob/b7e1e44a909c3a1d11e9c387db680c74d31b879f/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java#L60-L71].
 Histograms do sampling and they don't provide entirely accurate results.

However, it would be easy to track the min and max metrics so they are exact. 
Indeed max seems particularly important if when trying to detect large 
partitions.

I have updated the patch to do that exact calculation of min/max. I have also 
added a tilde (~) prefix to the percentiles coming from the histogram in an 
attempt to indicate that they are approximate. So the output of the tool now 
looks like:
{code:java}
> sstablepartitions 
> data/data/k/t-d7be5e90e90111ed8b54efe3c39cb0bb/nc-8-big-Data.db --min-size 
> 100MiB

Processing  #8 (big-nc) (1.368 GiB uncompressed, 534.979 MiB on disk)
  Partition: '13' (000d) live, size: 105.056 MiB, rows: 91490, cells: 
274470, tombstones: 50 (row:50, range:0, complex:0, cell:0, row-TTLd:0, 
cell-TTLd:0)
  Partition: '1' (0001) live, size: 127.241 MiB, rows: 111065, cells: 
333195, tombstones: 50 (row:50, range:0, complex:0, cell:0, row-TTLd:0, 
cell-TTLd:0)
  Partition: '8' (0008) live, size: 356.067 MiB, rows: 310706, cells: 
932118, tombstones: 0 (row:0, range:0, complex:0, cell:0, row-TTLd:0, 
cell-TTLd:0)
  Partition: '2' (0002) live, size: 213.341 MiB, rows: 186582, cells: 
559125, tombstones: 978 (row:978, range:0, complex:0, cell:0, row-TTLd:0, 
cell-TTLd:0)
Summary of  #8 (big-nc):
  File: /Users/adelapena/Desktop/sstablepartitions/nc-8-big-Data.db
  4 partitions match
  Keys: 13 1 8 2
   Partition sizeRow count   Cell count  
Tombstone count
  ~p50767.519 KiB  770 1916 
   0
  ~p75  2.238 MiB 2299 5722 
   0
  ~p90  3.867 MiB 3311 9887 
  50
  ~p95 16.629 MiB1423742510 
 446
  ~p99148.267 MiB   126934   379022 
1331
  ~p999   368.936 MiB   315852   943127 
2759
  min  56.854 KiB  100  150 
   0
  max 356.067 MiB   310706   932118 
2450
  count   21
{code}
Note also that the min/max rows on the table don't represent a single 
partition. In the previous example the max size of 356.067 MiB comes from 
partition '8', whereas the max number of tombstones (2450) comes from other 
partition. That partition is not listed because its size is below the 100MiB 
threshold. We can find the key of the partition with 2450 tombstones if we add 
a {{--min-tombstones}} threshold to the command:
{code:java}
> sstablepartitions 
> data/data/k/t-d7be5e90e90111ed8b54efe3c39cb0bb/nc-8-big-Data.db --min-size 
> 100MiB --min-tombstones 2450

Processing  #8 (big-nc) (1.368 GiB uncompressed, 534.979 MiB on disk)
  Partition: '13' (000d) live, size: 105.056 MiB, rows: 91490, cells: 
274470, tombstones: 50 (row:50, range:0, complex:0, cell:0, row-TTLd:0, 
cell-TTLd:0)
  Partition: '1' (0001) live, size: 127.241 MiB, rows: 111065, cells: 
333195, tombstones: 50 (row:50, range:0, complex:0, cell:0, row-TTLd:0, 
cell-TTLd:0)
  Partition: '8' (0008) live, size: 356.067 MiB, rows: 310706, cells: 
932118, tombstones: 0 (row:0, range:0, complex:0, cell:0, row-TTLd:0, 
cell-TTLd:0)
  Partition: '2' (0002) live, size: 213.341 MiB, rows: 186582, cells: 
559125, tombstones: 978 (row:978, range:0, complex:0, cell:0, row-TTLd:0, 
cell-TTLd:0)
  Partition: '21' (0015) live, size: 3.853 MiB, rows: 4900, cells: 9927, 
tombstones: 2450 (row:2450, range:0, complex:0, cell:0, row-TTLd:0, cell-TTLd:0)
Summary of  #8 (big-nc):
  File: /Users/adelapena/Desktop/sstablepartitions/nc-8-big-Data.db
  5 partitions match
  Keys: 13 1 8 2 21
   Partition sizeRow count   Cell count  
Tombstone count
  ~p50767.519 KiB  770 1916 
   0
  ~p75  2.238 MiB 2299 5722 
   0
  ~p90  3.867 MiB

[jira] [Commented] (CASSANDRA-8720) Provide tools for finding wide row/partition keys



[ 
https://issues.apache.org/jira/browse/CASSANDRA-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718911#comment-17718911
 ] 

Andres de la Peña commented on CASSANDRA-8720:
--

Thanks :)

The data in the table comes from 
[{{EstimatedHistogram}}|https://github.com/apache/cassandra/blob/d2923275e360a1ee9db498e748c269f701bb3a8b/src/java/org/apache/cassandra/utils/EstimatedHistogram.java]s,
 like the ones we use on metrics such as the ones on 
[{{MetadataCollector}}|https://github.com/apache/cassandra/blob/b7e1e44a909c3a1d11e9c387db680c74d31b879f/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java#L60-L71].
 Histograms do sampling and they don't provide entirely accurate results.

However, it would be easy to track the min and max metrics so they are exact. 
Indeed max seems particularly important if when trying to detect large 
partitions.

I have updated the patch to do that exact calculation of min/max. I have also 
added a tilde (~) prefix to the percentiles coming from the histogram in an 
attempt to indicate that they are approximate. So the output of the tool now 
looks like:
{code:java}
> sstablepartitions 
> data/data/k/t-d7be5e90e90111ed8b54efe3c39cb0bb/nc-8-big-Data.db --min-size 
> 100MiB

Processing  #8 (big-nc) (1.368 GiB uncompressed, 534.979 MiB on disk)
  Partition: '13' (000d) live, size: 105.056 MiB, rows: 91490, cells: 
274470, tombstones: 50 (row:50, range:0, complex:0, cell:0, row-TTLd:0, 
cell-TTLd:0)
  Partition: '1' (0001) live, size: 127.241 MiB, rows: 111065, cells: 
333195, tombstones: 50 (row:50, range:0, complex:0, cell:0, row-TTLd:0, 
cell-TTLd:0)
  Partition: '8' (0008) live, size: 356.067 MiB, rows: 310706, cells: 
932118, tombstones: 0 (row:0, range:0, complex:0, cell:0, row-TTLd:0, 
cell-TTLd:0)
  Partition: '2' (0002) live, size: 213.341 MiB, rows: 186582, cells: 
559125, tombstones: 978 (row:978, range:0, complex:0, cell:0, row-TTLd:0, 
cell-TTLd:0)
Summary of  #8 (big-nc):
  File: /Users/adelapena/Desktop/sstablepartitions/nc-8-big-Data.db
  4 partitions match
  Keys: 13 1 8 2
   Partition sizeRow count   Cell count  
Tombstone count
  ~p50767.519 KiB  770 1916 
   0
  ~p75  2.238 MiB 2299 5722 
   0
  ~p90  3.867 MiB 3311 9887 
  50
  ~p95 16.629 MiB1423742510 
 446
  ~p99148.267 MiB   126934   379022 
1331
  ~p999   368.936 MiB   315852   943127 
2759
  min  56.854 KiB  100  150 
   0
  max 356.067 MiB   310706   932118 
2450
  count   21
{code}
Note also that the min/max rows on the table don't represent a single 
partition. In the previous example the max size of 356.067 MiB comes from 
partition '8', whereas the max number of tombstones (2450) comes from other 
partition. That partition is not listed because its size is below the 100MiB 
threshold. We can find the key of that partition if we add a `--min-tombstones` 
threshold to the command:
{code:java}
> sstablepartitions 
> data/data/k/t-d7be5e90e90111ed8b54efe3c39cb0bb/nc-8-big-Data.db --min-size 
> 100MiB --min-tombstones 2000

Processing  #8 (big-nc) (1.368 GiB uncompressed, 534.979 MiB on disk)
  Partition: '13' (000d) live, size: 105.056 MiB, rows: 91490, cells: 
274470, tombstones: 50 (row:50, range:0, complex:0, cell:0, row-TTLd:0, 
cell-TTLd:0)
  Partition: '1' (0001) live, size: 127.241 MiB, rows: 111065, cells: 
333195, tombstones: 50 (row:50, range:0, complex:0, cell:0, row-TTLd:0, 
cell-TTLd:0)
  Partition: '8' (0008) live, size: 356.067 MiB, rows: 310706, cells: 
932118, tombstones: 0 (row:0, range:0, complex:0, cell:0, row-TTLd:0, 
cell-TTLd:0)
  Partition: '2' (0002) live, size: 213.341 MiB, rows: 186582, cells: 
559125, tombstones: 978 (row:978, range:0, complex:0, cell:0, row-TTLd:0, 
cell-TTLd:0)
  Partition: '21' (0015) live, size: 3.853 MiB, rows: 4900, cells: 9927, 
tombstones: 2450 (row:2450, range:0, complex:0, cell:0, row-TTLd:0, cell-TTLd:0)
Summary of  #8 (big-nc):
  File: /Users/adelapena/Desktop/sstablepartitions/nc-8-big-Data.db
  5 partitions match
  Keys: 13 1 8 2 21
   Partition sizeRow count   Cell count  
Tombstone count
  ~p50767.519 KiB  770 1916 
   0
  ~p75  2.238 MiB 2299 5722 
   0
  ~p90  3.867 MiB 3311 9887 
  50
  ~p95

[jira] [Commented] (CASSANDRA-12937) Default setting (yaml) for SSTable compression



[ 
https://issues.apache.org/jira/browse/CASSANDRA-12937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718882#comment-17718882
 ] 

Stefan Miklosovic commented on CASSANDRA-12937:
---

That is a good question. I think we are going too deep here if it is not 
removable easily. I would just leave it out and remove only stuff which does 
not spill over to the other parts of the code base. We may eventually create a 
ticket for looking into this specifically. 

> Default setting (yaml) for SSTable compression
> --
>
> Key: CASSANDRA-12937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12937
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Config
>Reporter: Michael Semb Wever
>Assignee: Claude Warren
>Priority: Low
>  Labels: AdventCalendar2021, lhf
> Fix For: 5.x
>
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> In many situations the choice of compression for sstables is more relevant to 
> the disks attached than to the schema and data.
> This issue is to add to cassandra.yaml a default value for sstable 
> compression that new tables will inherit (instead of the defaults found in 
> {{CompressionParams.DEFAULT}}.
> Examples where this can be relevant are filesystems that do on-the-fly 
> compression (btrfs, zfs) or specific disk configurations or even specific C* 
> versions (see CASSANDRA-10995 ).
> +Additional information for newcomers+
> Some new fields need to be added to {{cassandra.yaml}} to allow specifying 
> the field required for defining the default compression parameters. In 
> {{DatabaseDescriptor}} a new {{CompressionParams}} field should be added for 
> the default compression. This field should be initialized in 
> {{DatabaseDescriptor.applySimpleConfig()}}. At the different places where 
> {{CompressionParams.DEFAULT}} was used the code should call 
> {{DatabaseDescriptor#getDefaultCompressionParams}} that should return some 
> copy of configured {{CompressionParams}}.
> Some unit test using {{OverrideConfigurationLoader}} should be used to test 
> that the table schema use the new default when a new table is created (see 
> CreateTest for some example).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-18084) Introduce tags to snitch for better decision making for replica placement in topology strategies



[ 
https://issues.apache.org/jira/browse/CASSANDRA-18084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718881#comment-17718881
 ] 

Stefan Miklosovic commented on CASSANDRA-18084:
---

Thanks [~samt] for letting me know, [~jmckenzie] [~paulo] any chance you would 
take a quick look to evaluate this?

> Introduce tags to snitch for better decision making for replica placement in 
> topology strategies
> 
>
> Key: CASSANDRA-18084
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18084
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Cluster/Gossip, Cluster/Schema, Legacy/Distributed 
> Metadata
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> We would like to have extra meta-information in cassandra-rackdc.properties 
> which would further differentiate nodes in dc / racks.
> The main motivation behind this is that we have special requirements around 
> node's characteristics based on which we want to make further decisions when 
> it comes to replica placement in topology strategies. (New topology strategy 
> would be mostly derived from NTS / would be extended)
> The most reasonable way to do that is to introduce a new property into 
> cassandra-rackdc.properties called "tags"
> {code:java}
> # Arbitrary tag to assign to this node, they serve as additional 
> identificator of a node based on which operators might act. 
> # Value of this property is meant to be a comma-separated list of strings.
> #tags=tag1,tag2 
> {code}
> We also want to introduce new application state called TAGS. On startup of a 
> node, this node would advertise its tags to cluster and vice versa, all nodes 
> would tell to that respective node what tags they have so everybody would see 
> the same state of tags across the cluster based on which topology strategies 
> would do same decisions.
> These tags are not meant to be changed during whole runtime of a node, 
> similarly as datacenter and rack is not.
> For performance reasons, we might limit the maximum size of tags (sum of 
> their lenght), to be, for example, 64 characters and anything bigger would be 
> either shortened or the start would fail.
> Once we have tags for all nodes, we can have access to them, cluster-wide, 
> from TokenMetadata which is used quite heavily in topology strategies and it 
> exposes other relevant topology information (dc's, racks ...). We would just 
> add a new way to look at nodes.
> Tags would be a set.
> This would be persisted to the system.local to see what tags a local node has 
> and it would be persisted to system.peers_v2 to see what tags all other nodes 
> have. Column would be called "tags".
> {code:java}
> admin@cqlsh> select * from system.local ;
> @ Row 1
>  key | local
>  bootstrapped| COMPLETED
>  broadcast_address   | 172.19.0.5
>  broadcast_port  | 7000
>  cluster_name| Test Cluster
>  cql_version | 3.4.6
>  data_center | dc1
>  gossip_generation   | 1669739177
>  host_id | 54f8c6ea-a6ba-40c5-8fa5-484b2b4184c9
>  listen_address  | 172.19.0.5
>  listen_port | 7000
>  native_protocol_version | 5
>  partitioner | org.apache.cassandra.dht.Murmur3Partitioner
>  rack| rack1
>  release_version | 4.2-SNAPSHOT
>  rpc_address | 172.19.0.5
>  rpc_port| 9042
>  schema_version  | ef865449-2491-33b8-95b0-47c09cb14ea9
>  tags| {'tag1', 'tag2'}
>  tokens  | {'6504358681601109713'} 
> {code}
> for system.peers_v2:
> {code:java}
> admin@cqlsh> select peer,tags from system.peers_v2 ;
> @ Row 1
> --+-
>  peer | 172.19.0.15
>  tags | {'tag2', 'tag3'}
> @ Row 2
> --+-
>  peer | 172.19.0.11
>  tags | null
> {code}
> the POC implementation doing exactly that is here:
> We do not want to provide our custom topology strategies which are using this 
> feature as that will be the most probably a proprietary solution. This might 
> indeed change in the future. For now, we just want to implement hooks we can 
> base our in-house implementation on. All other people can benefit from this 
> as well if they choose so as this feature enables them to do that.
> Adding tags is not only about custom topology strategies. Operators could tag 
> their nodes if they wish to make further distinctions on topology level for 
> their operational needs.
> [https://github.com/instaclustr/cassandra/commit/eddd4681d8678515454dabb926d5f56b0c225eea]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (CASSANDRA-18228) Reorganize Apache C* docs in trunk branch to match the IA

2023-05-03 Thread Michael Semb Wever (Jira)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-18228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-18228:
---
Resolution: (was: Fixed)
Status: Open  (was: Resolved)

If you look at cassandra.staged.apache.org the changes are not working, despite 
actually being there. In particular the left-bar nav. I don't know antora well 
enough to say what the problem is…

> Reorganize Apache C* docs in trunk branch to match the IA
> -
>
> Key: CASSANDRA-18228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18228
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Documentation
>Reporter: Lorina Poland
>Assignee: Lorina Poland
>Priority: Normal
> Fix For: 5.0
>
>
> An agreed-upon Information Architecture will be used to reorganize the C* 5.0 
> docs.
> IA doc: 
> https://docs.google.com/document/d/1A96K73vj9MbJoD7wJNgIKWrOkLq-ZL2cNZAEXSWrciY/edit?usp=sharing



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-18084) Introduce tags to snitch for better decision making for replica placement in topology strategies



[ 
https://issues.apache.org/jira/browse/CASSANDRA-18084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718871#comment-17718871
 ] 

Sam Tunnicliffe commented on CASSANDRA-18084:
-

I can put it on my todo list [~smiklosovic], but I'm afraid it's unlikely that 
I'll get any spare time any time soon, so if you can find an alternative 
reviewer I would go ahead and do that. 

> Introduce tags to snitch for better decision making for replica placement in 
> topology strategies
> 
>
> Key: CASSANDRA-18084
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18084
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Cluster/Gossip, Cluster/Schema, Legacy/Distributed 
> Metadata
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> We would like to have extra meta-information in cassandra-rackdc.properties 
> which would further differentiate nodes in dc / racks.
> The main motivation behind this is that we have special requirements around 
> node's characteristics based on which we want to make further decisions when 
> it comes to replica placement in topology strategies. (New topology strategy 
> would be mostly derived from NTS / would be extended)
> The most reasonable way to do that is to introduce a new property into 
> cassandra-rackdc.properties called "tags"
> {code:java}
> # Arbitrary tag to assign to this node, they serve as additional 
> identificator of a node based on which operators might act. 
> # Value of this property is meant to be a comma-separated list of strings.
> #tags=tag1,tag2 
> {code}
> We also want to introduce new application state called TAGS. On startup of a 
> node, this node would advertise its tags to cluster and vice versa, all nodes 
> would tell to that respective node what tags they have so everybody would see 
> the same state of tags across the cluster based on which topology strategies 
> would do same decisions.
> These tags are not meant to be changed during whole runtime of a node, 
> similarly as datacenter and rack is not.
> For performance reasons, we might limit the maximum size of tags (sum of 
> their lenght), to be, for example, 64 characters and anything bigger would be 
> either shortened or the start would fail.
> Once we have tags for all nodes, we can have access to them, cluster-wide, 
> from TokenMetadata which is used quite heavily in topology strategies and it 
> exposes other relevant topology information (dc's, racks ...). We would just 
> add a new way to look at nodes.
> Tags would be a set.
> This would be persisted to the system.local to see what tags a local node has 
> and it would be persisted to system.peers_v2 to see what tags all other nodes 
> have. Column would be called "tags".
> {code:java}
> admin@cqlsh> select * from system.local ;
> @ Row 1
>  key | local
>  bootstrapped| COMPLETED
>  broadcast_address   | 172.19.0.5
>  broadcast_port  | 7000
>  cluster_name| Test Cluster
>  cql_version | 3.4.6
>  data_center | dc1
>  gossip_generation   | 1669739177
>  host_id | 54f8c6ea-a6ba-40c5-8fa5-484b2b4184c9
>  listen_address  | 172.19.0.5
>  listen_port | 7000
>  native_protocol_version | 5
>  partitioner | org.apache.cassandra.dht.Murmur3Partitioner
>  rack| rack1
>  release_version | 4.2-SNAPSHOT
>  rpc_address | 172.19.0.5
>  rpc_port| 9042
>  schema_version  | ef865449-2491-33b8-95b0-47c09cb14ea9
>  tags| {'tag1', 'tag2'}
>  tokens  | {'6504358681601109713'} 
> {code}
> for system.peers_v2:
> {code:java}
> admin@cqlsh> select peer,tags from system.peers_v2 ;
> @ Row 1
> --+-
>  peer | 172.19.0.15
>  tags | {'tag2', 'tag3'}
> @ Row 2
> --+-
>  peer | 172.19.0.11
>  tags | null
> {code}
> the POC implementation doing exactly that is here:
> We do not want to provide our custom topology strategies which are using this 
> feature as that will be the most probably a proprietary solution. This might 
> indeed change in the future. For now, we just want to implement hooks we can 
> base our in-house implementation on. All other people can benefit from this 
> as well if they choose so as this feature enables them to do that.
> Adding tags is not only about custom topology strategies. Operators could tag 
> their nodes if they wish to make further distinctions on topology level for 
> their operational needs.
> [https://github.com/instaclustr/cassandra/commit/eddd4681d8678515454dabb926d5f56b0c225eea]



--
This message was

[jira] [Commented] (CASSANDRA-12937) Default setting (yaml) for SSTable compression

2023-05-03 Thread Claude Warren (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-12937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718870#comment-17718870
 ] 

Claude Warren commented on CASSANDRA-12937:
---

[~smiklosovic], since the aliases are defined in the CompressionParams class 
they could be used anywhere.  Good observation.

The crcCheckChance parameter was removed but the crcCheckChance variable still 
exists because there are getter/setter functions that are used in other code.  
Is the intention to remove the crcCheckChance from all code, just the 
CompressionParams or ???

 

> Default setting (yaml) for SSTable compression
> --
>
> Key: CASSANDRA-12937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12937
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Config
>Reporter: Michael Semb Wever
>Assignee: Claude Warren
>Priority: Low
>  Labels: AdventCalendar2021, lhf
> Fix For: 5.x
>
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> In many situations the choice of compression for sstables is more relevant to 
> the disks attached than to the schema and data.
> This issue is to add to cassandra.yaml a default value for sstable 
> compression that new tables will inherit (instead of the defaults found in 
> {{CompressionParams.DEFAULT}}.
> Examples where this can be relevant are filesystems that do on-the-fly 
> compression (btrfs, zfs) or specific disk configurations or even specific C* 
> versions (see CASSANDRA-10995 ).
> +Additional information for newcomers+
> Some new fields need to be added to {{cassandra.yaml}} to allow specifying 
> the field required for defining the default compression parameters. In 
> {{DatabaseDescriptor}} a new {{CompressionParams}} field should be added for 
> the default compression. This field should be initialized in 
> {{DatabaseDescriptor.applySimpleConfig()}}. At the different places where 
> {{CompressionParams.DEFAULT}} was used the code should call 
> {{DatabaseDescriptor#getDefaultCompressionParams}} that should return some 
> copy of configured {{CompressionParams}}.
> Some unit test using {{OverrideConfigurationLoader}} should be used to test 
> that the table schema use the new default when a new table is created (see 
> CreateTest for some example).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-17047) Dropping a column can break queries until the schema is fully propagated (TAKE 2)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-17047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718869#comment-17718869
 ] 

Stefan Miklosovic edited comment on CASSANDRA-17047 at 5/3/23 9:57 AM:
---

Got it. It would be great if [~blerer] refreshed this work and rebased things 
etc ... 


was (Author: smiklosovic):
Got it. I would be great if [~blerer] refreshed this work and rebased things 
etc ... 

> Dropping a column can break queries until the schema is fully propagated 
> (TAKE 2)
> -
>
> Key: CASSANDRA-17047
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17047
> Project: Cassandra
>  Issue Type: Bug
>  Components: Cluster/Schema
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> With a table like:
> {code}
> CREATE TABLE ks.tbl (id int primary key, v1 int, v2 int, v3 int)
> {code}
> and we drop {{v2}}, we get this exception on the replicas which haven't seen 
> the schema change:
> {code}
> ERROR [SharedPool-Worker-1] node2 2020-06-24 09:49:08,107 
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread 
> Thread[SharedPool-Worker-1,5,node2]
> java.lang.IllegalStateException: [ColumnDefinition{name=v1, 
> type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=v2, type=org.apache.cassandra.db.marshal.Int32Type, 
> kind=REGULAR, position=-1}, ColumnDefinition{name=v3, 
> type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}] 
> is not a subset of [v1 v3]
>   at 
> org.apache.cassandra.db.Columns$Serializer.encodeBitmap(Columns.java:546) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.Columns$Serializer.serializeSubset(Columns.java:478) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:184)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:114)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:102)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:132)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
>  ~[main/:na]
> ...
> {code}
> Note that it doesn't matter if we {{SELECT *}} or {{SELECT id, v1}}
> CASSANDRA-15899 tried to fix the problem when columns are dropped as well as 
> when columns are added. Unfortunately the fix introduced an issue and had to 
> be reverted in CASSANDRA-16735. 
> If the scenario for ADDED columns is tricky, the original scenario for 
> DROPPED columns can  be solved in a safe way at the {{ColumnFilter}} level. 
> By consequence, I think that we should at least solve that scenario.
> [~bdeggleston], [~samt], [~ifesdjeen] does my proposal makes sense to you?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-17047) Dropping a column can break queries until the schema is fully propagated (TAKE 2)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-17047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718869#comment-17718869
 ] 

Stefan Miklosovic commented on CASSANDRA-17047:
---

Got it. I would be great if [~blerer] refreshed this work and rebased things 
etc ... 

> Dropping a column can break queries until the schema is fully propagated 
> (TAKE 2)
> -
>
> Key: CASSANDRA-17047
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17047
> Project: Cassandra
>  Issue Type: Bug
>  Components: Cluster/Schema
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> With a table like:
> {code}
> CREATE TABLE ks.tbl (id int primary key, v1 int, v2 int, v3 int)
> {code}
> and we drop {{v2}}, we get this exception on the replicas which haven't seen 
> the schema change:
> {code}
> ERROR [SharedPool-Worker-1] node2 2020-06-24 09:49:08,107 
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread 
> Thread[SharedPool-Worker-1,5,node2]
> java.lang.IllegalStateException: [ColumnDefinition{name=v1, 
> type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=v2, type=org.apache.cassandra.db.marshal.Int32Type, 
> kind=REGULAR, position=-1}, ColumnDefinition{name=v3, 
> type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}] 
> is not a subset of [v1 v3]
>   at 
> org.apache.cassandra.db.Columns$Serializer.encodeBitmap(Columns.java:546) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.Columns$Serializer.serializeSubset(Columns.java:478) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:184)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:114)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:102)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:132)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
>  ~[main/:na]
> ...
> {code}
> Note that it doesn't matter if we {{SELECT *}} or {{SELECT id, v1}}
> CASSANDRA-15899 tried to fix the problem when columns are dropped as well as 
> when columns are added. Unfortunately the fix introduced an issue and had to 
> be reverted in CASSANDRA-16735. 
> If the scenario for ADDED columns is tricky, the original scenario for 
> DROPPED columns can  be solved in a safe way at the {{ColumnFilter}} level. 
> By consequence, I think that we should at least solve that scenario.
> [~bdeggleston], [~samt], [~ifesdjeen] does my proposal makes sense to you?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-18084) Introduce tags to snitch for better decision making for replica placement in topology strategies



[ 
https://issues.apache.org/jira/browse/CASSANDRA-18084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718865#comment-17718865
 ] 

Stefan Miklosovic edited comment on CASSANDRA-18084 at 5/3/23 9:53 AM:
---

https://github.com/apache/cassandra/pull/2305

I also implemented pluggable mechanism where to fetch the metadata from - there 
are 3 sources out of the box - system properties, environment properties and a 
file. A user is switching between them in cassandra.yaml by choosing the 
correct implementation class, the same mechanism we use across whole yaml.

And chance you take a look at this? [~samt]


was (Author: smiklosovic):
https://github.com/apache/cassandra/pull/2305

I also implemented pluggable mechanism where to fetch the metadata from - there 
are 3 sources out of the box - system properties, environment properties and a 
file. A user is swichting between them in cassandra.yaml by choosing the 
correct implementation class, the same mechanism we use across whole yaml.

And chance you take a look at this? [~samt]

> Introduce tags to snitch for better decision making for replica placement in 
> topology strategies
> 
>
> Key: CASSANDRA-18084
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18084
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Cluster/Gossip, Cluster/Schema, Legacy/Distributed 
> Metadata
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> We would like to have extra meta-information in cassandra-rackdc.properties 
> which would further differentiate nodes in dc / racks.
> The main motivation behind this is that we have special requirements around 
> node's characteristics based on which we want to make further decisions when 
> it comes to replica placement in topology strategies. (New topology strategy 
> would be mostly derived from NTS / would be extended)
> The most reasonable way to do that is to introduce a new property into 
> cassandra-rackdc.properties called "tags"
> {code:java}
> # Arbitrary tag to assign to this node, they serve as additional 
> identificator of a node based on which operators might act. 
> # Value of this property is meant to be a comma-separated list of strings.
> #tags=tag1,tag2 
> {code}
> We also want to introduce new application state called TAGS. On startup of a 
> node, this node would advertise its tags to cluster and vice versa, all nodes 
> would tell to that respective node what tags they have so everybody would see 
> the same state of tags across the cluster based on which topology strategies 
> would do same decisions.
> These tags are not meant to be changed during whole runtime of a node, 
> similarly as datacenter and rack is not.
> For performance reasons, we might limit the maximum size of tags (sum of 
> their lenght), to be, for example, 64 characters and anything bigger would be 
> either shortened or the start would fail.
> Once we have tags for all nodes, we can have access to them, cluster-wide, 
> from TokenMetadata which is used quite heavily in topology strategies and it 
> exposes other relevant topology information (dc's, racks ...). We would just 
> add a new way to look at nodes.
> Tags would be a set.
> This would be persisted to the system.local to see what tags a local node has 
> and it would be persisted to system.peers_v2 to see what tags all other nodes 
> have. Column would be called "tags".
> {code:java}
> admin@cqlsh> select * from system.local ;
> @ Row 1
>  key | local
>  bootstrapped| COMPLETED
>  broadcast_address   | 172.19.0.5
>  broadcast_port  | 7000
>  cluster_name| Test Cluster
>  cql_version | 3.4.6
>  data_center | dc1
>  gossip_generation   | 1669739177
>  host_id | 54f8c6ea-a6ba-40c5-8fa5-484b2b4184c9
>  listen_address  | 172.19.0.5
>  listen_port | 7000
>  native_protocol_version | 5
>  partitioner | org.apache.cassandra.dht.Murmur3Partitioner
>  rack| rack1
>  release_version | 4.2-SNAPSHOT
>  rpc_address | 172.19.0.5
>  rpc_port| 9042
>  schema_version  | ef865449-2491-33b8-95b0-47c09cb14ea9
>  tags| {'tag1', 'tag2'}
>  tokens  | {'6504358681601109713'} 
> {code}
> for system.peers_v2:
> {code:java}
> admin@cqlsh> select peer,tags from system.peers_v2 ;
> @ Row 1
> --+-
>  peer | 172.19.0.15
>  tags | {'tag2', 'tag3'}
> @ Row 2
> --+-
>  peer | 172.19.0.11
>  tags | null
> {code}
> the POC implementation doing exactly that is here:
> We

[jira] [Comment Edited] (CASSANDRA-18084) Introduce tags to snitch for better decision making for replica placement in topology strategies



[ 
https://issues.apache.org/jira/browse/CASSANDRA-18084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718865#comment-17718865
 ] 

Stefan Miklosovic edited comment on CASSANDRA-18084 at 5/3/23 9:53 AM:
---

https://github.com/apache/cassandra/pull/2305

I also implemented pluggable mechanism where to fetch the metadata from - there 
are 3 sources out of the box - system properties, environment properties and a 
file. A user is swichting between them in cassandra.yaml by choosing the 
correct implementation class, the same mechanism we use across whole yaml.

And chance you take a look at this? [~samt]


was (Author: smiklosovic):
https://github.com/apache/cassandra/pull/2305

I also implemented pluggable mechanism where to fetch the metadata from - there 
are 3 sources of the out of the box - system properties, environment properties 
and a file. A user is swichting between them in cassandra.yaml by choosing the 
correct implementation class, the same mechanism we use across whole yaml.

And chance you take a look at this? [~samt]

> Introduce tags to snitch for better decision making for replica placement in 
> topology strategies
> 
>
> Key: CASSANDRA-18084
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18084
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Cluster/Gossip, Cluster/Schema, Legacy/Distributed 
> Metadata
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> We would like to have extra meta-information in cassandra-rackdc.properties 
> which would further differentiate nodes in dc / racks.
> The main motivation behind this is that we have special requirements around 
> node's characteristics based on which we want to make further decisions when 
> it comes to replica placement in topology strategies. (New topology strategy 
> would be mostly derived from NTS / would be extended)
> The most reasonable way to do that is to introduce a new property into 
> cassandra-rackdc.properties called "tags"
> {code:java}
> # Arbitrary tag to assign to this node, they serve as additional 
> identificator of a node based on which operators might act. 
> # Value of this property is meant to be a comma-separated list of strings.
> #tags=tag1,tag2 
> {code}
> We also want to introduce new application state called TAGS. On startup of a 
> node, this node would advertise its tags to cluster and vice versa, all nodes 
> would tell to that respective node what tags they have so everybody would see 
> the same state of tags across the cluster based on which topology strategies 
> would do same decisions.
> These tags are not meant to be changed during whole runtime of a node, 
> similarly as datacenter and rack is not.
> For performance reasons, we might limit the maximum size of tags (sum of 
> their lenght), to be, for example, 64 characters and anything bigger would be 
> either shortened or the start would fail.
> Once we have tags for all nodes, we can have access to them, cluster-wide, 
> from TokenMetadata which is used quite heavily in topology strategies and it 
> exposes other relevant topology information (dc's, racks ...). We would just 
> add a new way to look at nodes.
> Tags would be a set.
> This would be persisted to the system.local to see what tags a local node has 
> and it would be persisted to system.peers_v2 to see what tags all other nodes 
> have. Column would be called "tags".
> {code:java}
> admin@cqlsh> select * from system.local ;
> @ Row 1
>  key | local
>  bootstrapped| COMPLETED
>  broadcast_address   | 172.19.0.5
>  broadcast_port  | 7000
>  cluster_name| Test Cluster
>  cql_version | 3.4.6
>  data_center | dc1
>  gossip_generation   | 1669739177
>  host_id | 54f8c6ea-a6ba-40c5-8fa5-484b2b4184c9
>  listen_address  | 172.19.0.5
>  listen_port | 7000
>  native_protocol_version | 5
>  partitioner | org.apache.cassandra.dht.Murmur3Partitioner
>  rack| rack1
>  release_version | 4.2-SNAPSHOT
>  rpc_address | 172.19.0.5
>  rpc_port| 9042
>  schema_version  | ef865449-2491-33b8-95b0-47c09cb14ea9
>  tags| {'tag1', 'tag2'}
>  tokens  | {'6504358681601109713'} 
> {code}
> for system.peers_v2:
> {code:java}
> admin@cqlsh> select peer,tags from system.peers_v2 ;
> @ Row 1
> --+-
>  peer | 172.19.0.15
>  tags | {'tag2', 'tag3'}
> @ Row 2
> --+-
>  peer | 172.19.0.11
>  tags | null
> {code}
> the POC implementation doing exactly that is

[jira] [Commented] (CASSANDRA-18084) Introduce tags to snitch for better decision making for replica placement in topology strategies



[ 
https://issues.apache.org/jira/browse/CASSANDRA-18084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718865#comment-17718865
 ] 

Stefan Miklosovic commented on CASSANDRA-18084:
---

https://github.com/apache/cassandra/pull/2305

I also implemented pluggable mechanism where to fetch the metadata from - there 
are 3 sources of the out of the box - system properties, environment properties 
and a file. A user is swichting between them in cassandra.yaml by choosing the 
correct implementation class, the same mechanism we use across whole yaml.

And chance you take a look at this? [~samt]

> Introduce tags to snitch for better decision making for replica placement in 
> topology strategies
> 
>
> Key: CASSANDRA-18084
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18084
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Cluster/Gossip, Cluster/Schema, Legacy/Distributed 
> Metadata
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> We would like to have extra meta-information in cassandra-rackdc.properties 
> which would further differentiate nodes in dc / racks.
> The main motivation behind this is that we have special requirements around 
> node's characteristics based on which we want to make further decisions when 
> it comes to replica placement in topology strategies. (New topology strategy 
> would be mostly derived from NTS / would be extended)
> The most reasonable way to do that is to introduce a new property into 
> cassandra-rackdc.properties called "tags"
> {code:java}
> # Arbitrary tag to assign to this node, they serve as additional 
> identificator of a node based on which operators might act. 
> # Value of this property is meant to be a comma-separated list of strings.
> #tags=tag1,tag2 
> {code}
> We also want to introduce new application state called TAGS. On startup of a 
> node, this node would advertise its tags to cluster and vice versa, all nodes 
> would tell to that respective node what tags they have so everybody would see 
> the same state of tags across the cluster based on which topology strategies 
> would do same decisions.
> These tags are not meant to be changed during whole runtime of a node, 
> similarly as datacenter and rack is not.
> For performance reasons, we might limit the maximum size of tags (sum of 
> their lenght), to be, for example, 64 characters and anything bigger would be 
> either shortened or the start would fail.
> Once we have tags for all nodes, we can have access to them, cluster-wide, 
> from TokenMetadata which is used quite heavily in topology strategies and it 
> exposes other relevant topology information (dc's, racks ...). We would just 
> add a new way to look at nodes.
> Tags would be a set.
> This would be persisted to the system.local to see what tags a local node has 
> and it would be persisted to system.peers_v2 to see what tags all other nodes 
> have. Column would be called "tags".
> {code:java}
> admin@cqlsh> select * from system.local ;
> @ Row 1
>  key | local
>  bootstrapped| COMPLETED
>  broadcast_address   | 172.19.0.5
>  broadcast_port  | 7000
>  cluster_name| Test Cluster
>  cql_version | 3.4.6
>  data_center | dc1
>  gossip_generation   | 1669739177
>  host_id | 54f8c6ea-a6ba-40c5-8fa5-484b2b4184c9
>  listen_address  | 172.19.0.5
>  listen_port | 7000
>  native_protocol_version | 5
>  partitioner | org.apache.cassandra.dht.Murmur3Partitioner
>  rack| rack1
>  release_version | 4.2-SNAPSHOT
>  rpc_address | 172.19.0.5
>  rpc_port| 9042
>  schema_version  | ef865449-2491-33b8-95b0-47c09cb14ea9
>  tags| {'tag1', 'tag2'}
>  tokens  | {'6504358681601109713'} 
> {code}
> for system.peers_v2:
> {code:java}
> admin@cqlsh> select peer,tags from system.peers_v2 ;
> @ Row 1
> --+-
>  peer | 172.19.0.15
>  tags | {'tag2', 'tag3'}
> @ Row 2
> --+-
>  peer | 172.19.0.11
>  tags | null
> {code}
> the POC implementation doing exactly that is here:
> We do not want to provide our custom topology strategies which are using this 
> feature as that will be the most probably a proprietary solution. This might 
> indeed change in the future. For now, we just want to implement hooks we can 
> base our in-house implementation on. All other people can benefit from this 
> as well if they choose so as this feature enables them to do that.
> Adding tags is not only about custom topology strategies. Operators could tag 
>

[jira] [Commented] (CASSANDRA-17047) Dropping a column can break queries until the schema is fully propagated (TAKE 2)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-17047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718863#comment-17718863
 ] 

Sam Tunnicliffe commented on CASSANDRA-17047:
-

Not really. Until the CEP-21 stuff actually lands we shouldn't act as though it 
has IMO. So if this patch is ok to merge now, then we should probably do so (I 
don't have cycles to review at the moment though sorry).

> Dropping a column can break queries until the schema is fully propagated 
> (TAKE 2)
> -
>
> Key: CASSANDRA-17047
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17047
> Project: Cassandra
>  Issue Type: Bug
>  Components: Cluster/Schema
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> With a table like:
> {code}
> CREATE TABLE ks.tbl (id int primary key, v1 int, v2 int, v3 int)
> {code}
> and we drop {{v2}}, we get this exception on the replicas which haven't seen 
> the schema change:
> {code}
> ERROR [SharedPool-Worker-1] node2 2020-06-24 09:49:08,107 
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread 
> Thread[SharedPool-Worker-1,5,node2]
> java.lang.IllegalStateException: [ColumnDefinition{name=v1, 
> type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=v2, type=org.apache.cassandra.db.marshal.Int32Type, 
> kind=REGULAR, position=-1}, ColumnDefinition{name=v3, 
> type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}] 
> is not a subset of [v1 v3]
>   at 
> org.apache.cassandra.db.Columns$Serializer.encodeBitmap(Columns.java:546) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.Columns$Serializer.serializeSubset(Columns.java:478) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:184)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:114)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:102)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:132)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
>  ~[main/:na]
> ...
> {code}
> Note that it doesn't matter if we {{SELECT *}} or {{SELECT id, v1}}
> CASSANDRA-15899 tried to fix the problem when columns are dropped as well as 
> when columns are added. Unfortunately the fix introduced an issue and had to 
> be reverted in CASSANDRA-16735. 
> If the scenario for ADDED columns is tricky, the original scenario for 
> DROPPED columns can  be solved in a safe way at the {{ColumnFilter}} level. 
> By consequence, I think that we should at least solve that scenario.
> [~bdeggleston], [~samt], [~ifesdjeen] does my proposal makes sense to you?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-18420) Connection without username not logged in auditlog



[ 
https://issues.apache.org/jira/browse/CASSANDRA-18420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718862#comment-17718862
 ] 

Sam Tunnicliffe commented on CASSANDRA-18420:
-

This description is not quite accurate.
{quote}the client needs to send the STARTUP message and then wait for the 
AUTHENTICATE message from the Cassandra server
{quote}
This is correct, but not this:
{quote}If there is no username in the STARTUP message, there is no way that the 
Cassandra server can send the AUTHENTICATE message.
{quote}
STARTUP never contains a username. If authentication is configured on the 
server, it always responds to the STARTUP with an AUTHENTICATE message. The 
actual authentication is then performed via the 
[SASL|https://datatracker.ietf.org/doc/html/rfc4422] protocol, with 
[PLAIN|https://datatracker.ietf.org/doc/html/rfc4616] being the only mechanism 
supported by out of the box by the OSS distro.
{quote}The mechanism consists of a single message, a string of [UTF-8]
encoded [Unicode] characters, from the client to the server. The
client presents the authorization identity (identity to act as),
followed by a NUL (U+) character, followed by the authentication
identity (identity whose password will be used), followed by a NUL
(U+) character, followed by the clear-text password.
{quote}
So the situation I think you're describing is that the client receives the 
AUTHENTICATE then simply doesn't respond and this is what is happening in the 
cqlsh example posted here. If no credentials are supplied to cqlsh at the 
outset, it doesn't configure an {{AuthProvider}} then throws when it receives 
the AUTHENTICATE response from the server 
([here|https://github.com/datastax/python-driver/blob/master/cassandra/connection.py#L1387-L1395C3]).

> Connection without username not logged in auditlog 
> ---
>
> Key: CASSANDRA-18420
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18420
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/auditlogging
>Reporter: Yakir Gibraltar
>Assignee: Ningzi Zhan
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.x
>
>
> Hi,
> If making connection *without username* to cassandra cluster with 
> PasswordAuthenticator enabled, 
> Connection will fail but not logged on auditlog.
> How to reproduce:
>  # Enable "authenticator: PasswordAuthenticator" on cluster
>  # Enable audit : "nodetool enableauditlog"
>  # Open a new screen and run "auditlogviewer -f /audit/"
>  # Try to connect, and connection will fail:
> {code:java}
> [root@c1 ~]# cqlsh
> Connection error: ('Unable to connect to any servers', {'127.0.0.1:9042': 
> AuthenticationFailed('Remote end requires authentication',)}){code}
>  # *But nothing in auditlogviewer*.
> Connection with incorrect usernames or password logged correct on auditlog , 
> the problem only on connection without username. 
> How it's affecting:
>  # Security reason, hard to find unauthorized connections attempt  .
>  # When migrating cluster into PasswordAuthenticator, hard to find 
> applications that didn't add username/password. 
> Thank you. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-17047) Dropping a column can break queries until the schema is fully propagated (TAKE 2)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-17047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718859#comment-17718859
 ] 

Stefan Miklosovic edited comment on CASSANDRA-17047 at 5/3/23 9:24 AM:
---

So that means that this patch is eligible to be merged only up to 4.1 included, 
I guess.


was (Author: smiklosovic):
So that means that this patch is eligible to merge only up to 4.1 included, I 
guess.

> Dropping a column can break queries until the schema is fully propagated 
> (TAKE 2)
> -
>
> Key: CASSANDRA-17047
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17047
> Project: Cassandra
>  Issue Type: Bug
>  Components: Cluster/Schema
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> With a table like:
> {code}
> CREATE TABLE ks.tbl (id int primary key, v1 int, v2 int, v3 int)
> {code}
> and we drop {{v2}}, we get this exception on the replicas which haven't seen 
> the schema change:
> {code}
> ERROR [SharedPool-Worker-1] node2 2020-06-24 09:49:08,107 
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread 
> Thread[SharedPool-Worker-1,5,node2]
> java.lang.IllegalStateException: [ColumnDefinition{name=v1, 
> type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=v2, type=org.apache.cassandra.db.marshal.Int32Type, 
> kind=REGULAR, position=-1}, ColumnDefinition{name=v3, 
> type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}] 
> is not a subset of [v1 v3]
>   at 
> org.apache.cassandra.db.Columns$Serializer.encodeBitmap(Columns.java:546) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.Columns$Serializer.serializeSubset(Columns.java:478) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:184)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:114)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:102)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:132)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
>  ~[main/:na]
> ...
> {code}
> Note that it doesn't matter if we {{SELECT *}} or {{SELECT id, v1}}
> CASSANDRA-15899 tried to fix the problem when columns are dropped as well as 
> when columns are added. Unfortunately the fix introduced an issue and had to 
> be reverted in CASSANDRA-16735. 
> If the scenario for ADDED columns is tricky, the original scenario for 
> DROPPED columns can  be solved in a safe way at the {{ColumnFilter}} level. 
> By consequence, I think that we should at least solve that scenario.
> [~bdeggleston], [~samt], [~ifesdjeen] does my proposal makes sense to you?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-17047) Dropping a column can break queries until the schema is fully propagated (TAKE 2)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-17047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718859#comment-17718859
 ] 

Stefan Miklosovic commented on CASSANDRA-17047:
---

So that means that this patch is eligible to merge only up to 4.1 included, I 
guess.

> Dropping a column can break queries until the schema is fully propagated 
> (TAKE 2)
> -
>
> Key: CASSANDRA-17047
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17047
> Project: Cassandra
>  Issue Type: Bug
>  Components: Cluster/Schema
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> With a table like:
> {code}
> CREATE TABLE ks.tbl (id int primary key, v1 int, v2 int, v3 int)
> {code}
> and we drop {{v2}}, we get this exception on the replicas which haven't seen 
> the schema change:
> {code}
> ERROR [SharedPool-Worker-1] node2 2020-06-24 09:49:08,107 
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread 
> Thread[SharedPool-Worker-1,5,node2]
> java.lang.IllegalStateException: [ColumnDefinition{name=v1, 
> type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=v2, type=org.apache.cassandra.db.marshal.Int32Type, 
> kind=REGULAR, position=-1}, ColumnDefinition{name=v3, 
> type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}] 
> is not a subset of [v1 v3]
>   at 
> org.apache.cassandra.db.Columns$Serializer.encodeBitmap(Columns.java:546) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.Columns$Serializer.serializeSubset(Columns.java:478) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:184)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:114)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:102)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:132)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
>  ~[main/:na]
> ...
> {code}
> Note that it doesn't matter if we {{SELECT *}} or {{SELECT id, v1}}
> CASSANDRA-15899 tried to fix the problem when columns are dropped as well as 
> when columns are added. Unfortunately the fix introduced an issue and had to 
> be reverted in CASSANDRA-16735. 
> If the scenario for ADDED columns is tricky, the original scenario for 
> DROPPED columns can  be solved in a safe way at the {{ColumnFilter}} level. 
> By consequence, I think that we should at least solve that scenario.
> [~bdeggleston], [~samt], [~ifesdjeen] does my proposal makes sense to you?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-18495) Warnings when using the perl driver to connect to Cassandra

2023-05-03 Thread Aswin Karthik (Jira)

Aswin Karthik created CASSANDRA-18495:
-

Summary: Warnings when using the perl driver to connect to
Cassandra
Key: CASSANDRA-18495
URL: https://issues.apache.org/jira/browse/CASSANDRA-18495
Project: Cassandra
Issue Type: Bug
Reporter: Aswin Karthik

When I use the [perl driver|https://github.com/TvdW/perl-DBD-Cassandra] to
connect to Cassandra 4.0.3 and onwards, I get the following error just for
initialization of a connection.

The error I get is
{noformat}
`USE ` with prepared statements is considered to be an anti-pattern
due to ambiguity in non-qualified table names. Please consider removing
instances of `Session#setKeyspace()`, `Session#execute("USE
")` and `cluster.newSession()` from your code, and always
use fully qualified table names (e.g. .). Keyspace used: null,
statement keyspace: null, statement id: at Cassandra/Client/Connection.pm
line 957.
{noformat}

This is just from initialization of the perl driver connection while choosing
the keyspace.
[https://github.com/TvdW/perl-DBD-Cassandra/blob/master/Cassandra-Client/lib/Cassandra/Client/Connection.pm#L562]
before running any queries. It is emitted even if I use qualified prepared
statement.

The same warning does not pop up in Datastax java driver initialization.

On debugging, I found that this warning does not emit for all unqualified
prepared statements. It only emits for unqualified prepared "USE ks" statement.
But the "USE ks" can never be qualified. So the warning is a bit vague on what
is the recommended approach.

And from the perspective of the driver, it is setting the keyspace of the
connection for the first time. The same warning does not happen on the datastax
java driver and that is because it uses QUERY to set the keyspace on
connection. (I tried to follow the same approach on the perl driver -
https://github.com/TvdW/perl-DBD-Cassandra/pull/35 )

The warnings are not very clear on what is deprecated and what is not. Does it
deprecate only the use of prepared statement of "USE ks"? or does it deprecate
"USE ks" completely? And it is not being emitted for other unqualified prepared
statements but only for a USE statement which cannot be qualified at all.

--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-18493) Add LIKE prefix/suffix support to SAI

2023-05-03 Thread Benjamin Lerer (Jira)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-18493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-18493:
---
Summary: Add LIKE prefix/suffix support to SAI  (was: Add LIKE 
prefix/suffix support)

> Add LIKE prefix/suffix support to SAI
> -
>
> Key: CASSANDRA-18493
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18493
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Feature/2i Index
>Reporter: Mike Adamson
>Priority: Normal
>
> This should provide the following functionality:
> * LIKE abc%  - prefix support
> * LIKE %bcd  - suffix support
> * LIKE %bc%  - prefix/suffix support



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14667) Upgrade Dropwizard Metrics to 4.x



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718844#comment-17718844
 ] 

Stefan Miklosovic commented on CASSANDRA-14667:
---

What is there to cover if we do not use it. If you look closely they are really 
used almost exclusively in the tests. There is even the usage of 
'withoutMetrics' already in some "production class" in FQL tool or something 
similar. JavaDriverClient used for Stress tool uses "withoutMetrics()" already 
btw so this seems like pretty innocent change.

> Upgrade Dropwizard Metrics to 4.x
> -
>
> Key: CASSANDRA-14667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14667
> Project: Cassandra
>  Issue Type: Task
>  Components: Observability/Metrics
>Reporter: Stig Rohde Døssing
>Assignee: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.x
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Cassandra currently uses Metrics 3.1.5. Version 4.0.0 added some fixes for 
> Java 9 compatibility. It would be good to upgrade the Metrics library as part 
> of the version of Cassandra that adds Java 9 compatibility 
> (https://issues.apache.org/jira/browse/CASSANDRA-9608). 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14667) Upgrade Dropwizard Metrics to 4.x



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718844#comment-17718844
 ] 

Stefan Miklosovic edited comment on CASSANDRA-14667 at 5/3/23 8:37 AM:
---

What is there to cover if we do not use it. If you look closely they are really 
used almost exclusively in the tests. JavaDriverClient used for Stress tool 
uses "withoutMetrics()" already btw so this seems like pretty innocent change.


was (Author: smiklosovic):
What is there to cover if we do not use it. If you look closely they are really 
used almost exclusively in the tests. There is even the usage of 
'withoutMetrics' already in some "production class" in FQL tool or something 
similar. JavaDriverClient used for Stress tool uses "withoutMetrics()" already 
btw so this seems like pretty innocent change.

> Upgrade Dropwizard Metrics to 4.x
> -
>
> Key: CASSANDRA-14667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14667
> Project: Cassandra
>  Issue Type: Task
>  Components: Observability/Metrics
>Reporter: Stig Rohde Døssing
>Assignee: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.x
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Cassandra currently uses Metrics 3.1.5. Version 4.0.0 added some fixes for 
> Java 9 compatibility. It would be good to upgrade the Metrics library as part 
> of the version of Cassandra that adds Java 9 compatibility 
> (https://issues.apache.org/jira/browse/CASSANDRA-9608). 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-17047) Dropping a column can break queries until the schema is fully propagated (TAKE 2)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-17047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718843#comment-17718843
 ] 

Sam Tunnicliffe commented on CASSANDRA-17047:
-

[~smiklosovic] yes, replicas can detect and repair schema mismatches on the 
query path

> Dropping a column can break queries until the schema is fully propagated 
> (TAKE 2)
> -
>
> Key: CASSANDRA-17047
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17047
> Project: Cassandra
>  Issue Type: Bug
>  Components: Cluster/Schema
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> With a table like:
> {code}
> CREATE TABLE ks.tbl (id int primary key, v1 int, v2 int, v3 int)
> {code}
> and we drop {{v2}}, we get this exception on the replicas which haven't seen 
> the schema change:
> {code}
> ERROR [SharedPool-Worker-1] node2 2020-06-24 09:49:08,107 
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread 
> Thread[SharedPool-Worker-1,5,node2]
> java.lang.IllegalStateException: [ColumnDefinition{name=v1, 
> type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=v2, type=org.apache.cassandra.db.marshal.Int32Type, 
> kind=REGULAR, position=-1}, ColumnDefinition{name=v3, 
> type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}] 
> is not a subset of [v1 v3]
>   at 
> org.apache.cassandra.db.Columns$Serializer.encodeBitmap(Columns.java:546) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.Columns$Serializer.serializeSubset(Columns.java:478) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:184)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:114)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:102)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:132)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
>  ~[main/:na]
> ...
> {code}
> Note that it doesn't matter if we {{SELECT *}} or {{SELECT id, v1}}
> CASSANDRA-15899 tried to fix the problem when columns are dropped as well as 
> when columns are added. Unfortunately the fix introduced an issue and had to 
> be reverted in CASSANDRA-16735. 
> If the scenario for ADDED columns is tricky, the original scenario for 
> DROPPED columns can  be solved in a safe way at the {{ColumnFilter}} level. 
> By consequence, I think that we should at least solve that scenario.
> [~bdeggleston], [~samt], [~ifesdjeen] does my proposal makes sense to you?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14667) Upgrade Dropwizard Metrics to 4.x

2023-05-03 Thread Ekaterina Dimitrova (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718838#comment-17718838
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-14667 at 5/3/23 8:28 AM:
-

{quote}I am not completely sure if we use these "metrics" via driver, I do not 
think so but should be double checked. I think that for the sake of tests we do 
not need them at all.
{quote}
[https://docs.datastax.com/en/developer/java-driver/3.11/manual/metrics/]

We might not need them in our tests but it appears to me by adding everywhere 
*withoutMetrics()* we just cover a problem, no?

[~blerer] do I recall correctly you mentioned once you looked into this 
dropwizard upgrade? Any thoughts?


was (Author: e.dimitrova):
{quote}I am not completely sure if we use these "metrics" via driver, I do not 
think so but should be double checked. I think that for the sake of tests we do 
not need them at all.
{quote}
[https://docs.datastax.com/en/developer/java-driver/3.11/manual/metrics/]

We might not need them in our tests but it appears to me by adding everywhere 
*withoutMetrics()* we just cover a problem, no?

[~blerer] do I recall correctly you mentioned once you looked into that? Any 
thoughts?

> Upgrade Dropwizard Metrics to 4.x
> -
>
> Key: CASSANDRA-14667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14667
> Project: Cassandra
>  Issue Type: Task
>  Components: Observability/Metrics
>Reporter: Stig Rohde Døssing
>Assignee: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.x
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Cassandra currently uses Metrics 3.1.5. Version 4.0.0 added some fixes for 
> Java 9 compatibility. It would be good to upgrade the Metrics library as part 
> of the version of Cassandra that adds Java 9 compatibility 
> (https://issues.apache.org/jira/browse/CASSANDRA-9608). 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14667) Upgrade Dropwizard Metrics to 4.x

2023-05-03 Thread Ekaterina Dimitrova (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718838#comment-17718838
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-14667 at 5/3/23 8:28 AM:
-

{quote}I am not completely sure if we use these "metrics" via driver, I do not 
think so but should be double checked. I think that for the sake of tests we do 
not need them at all.
{quote}
[https://docs.datastax.com/en/developer/java-driver/3.11/manual/metrics/]

We might not need them in our tests but it appears to me by adding everywhere 
*withoutMetrics()* we just cover a problem, no?

[~blerer] do I recall correctly you mentioned once you looked into that? Any 
thoughts?


was (Author: e.dimitrova):
{quote}I am not completely sure if we use these "metrics" via driver, I do not 
think so but should be double checked. I think that for the sake of tests we do 
not need them at all.
{quote}
[https://docs.datastax.com/en/developer/java-driver/3.11/manual/metrics/]

We might not need them but it appears to me by adding everywhere 
*withoutMetrics()* we just cover a problem, no?

[~blerer] do I recall correctly you mentioned once you looked into that? Any 
thoughts?

> Upgrade Dropwizard Metrics to 4.x
> -
>
> Key: CASSANDRA-14667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14667
> Project: Cassandra
>  Issue Type: Task
>  Components: Observability/Metrics
>Reporter: Stig Rohde Døssing
>Assignee: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.x
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Cassandra currently uses Metrics 3.1.5. Version 4.0.0 added some fixes for 
> Java 9 compatibility. It would be good to upgrade the Metrics library as part 
> of the version of Cassandra that adds Java 9 compatibility 
> (https://issues.apache.org/jira/browse/CASSANDRA-9608). 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14667) Upgrade Dropwizard Metrics to 4.x

2023-05-03 Thread Ekaterina Dimitrova (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718838#comment-17718838
 ] 

Ekaterina Dimitrova commented on CASSANDRA-14667:
-

{quote}I am not completely sure if we use these "metrics" via driver, I do not 
think so but should be double checked. I think that for the sake of tests we do 
not need them at all.
{quote}
[https://docs.datastax.com/en/developer/java-driver/3.11/manual/metrics/]

We might not need them but it appears to me by adding everywhere 
*withoutMetrics()* we just cover a problem, no?

[~blerer] do I recall correctly you mentioned once you looked into that? Any 
thoughts?

> Upgrade Dropwizard Metrics to 4.x
> -
>
> Key: CASSANDRA-14667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14667
> Project: Cassandra
>  Issue Type: Task
>  Components: Observability/Metrics
>Reporter: Stig Rohde Døssing
>Assignee: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.x
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Cassandra currently uses Metrics 3.1.5. Version 4.0.0 added some fixes for 
> Java 9 compatibility. It would be good to upgrade the Metrics library as part 
> of the version of Cassandra that adds Java 9 compatibility 
> (https://issues.apache.org/jira/browse/CASSANDRA-9608). 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14667) Upgrade Dropwizard Metrics to 4.x



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718836#comment-17718836
 ] 

Stefan Miklosovic edited comment on CASSANDRA-14667 at 5/3/23 8:21 AM:
---

[~e.dimitrova] [~mmuzaf] [~brandonwilliams] any feedback on my last comment 
here? This relates to CASSANDRA-15472. I think that bumping Dropwizard to 4.x 
for Cassandra 5.0 would be ideal.

Bumping Dropwizard to 4.x is significantly easier than bumping Datastax Java 
Driver to 4. Bumping Dropwizard to 4 does not prevent us from bumping the 
driver later on as driver of 4 already includes Dropwizard 4.x.


was (Author: smiklosovic):
[~e.dimitrova] [~mmuzaf] [~brandonwilliams] any feedback on my last comment 
here? This relates to CASSANDRA-15472. I think that bumping Dropwizard to 4.x 
for Cassandra 5.0 would be ideal.

> Upgrade Dropwizard Metrics to 4.x
> -
>
> Key: CASSANDRA-14667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14667
> Project: Cassandra
>  Issue Type: Task
>  Components: Observability/Metrics
>Reporter: Stig Rohde Døssing
>Assignee: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.x
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Cassandra currently uses Metrics 3.1.5. Version 4.0.0 added some fixes for 
> Java 9 compatibility. It would be good to upgrade the Metrics library as part 
> of the version of Cassandra that adds Java 9 compatibility 
> (https://issues.apache.org/jira/browse/CASSANDRA-9608). 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14667) Upgrade Dropwizard Metrics to 4.x



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718836#comment-17718836
 ] 

Stefan Miklosovic edited comment on CASSANDRA-14667 at 5/3/23 8:20 AM:
---

[~e.dimitrova] [~mmuzaf] [~brandonwilliams] any feedback on my last comment 
here? This relates to CASSANDRA-15472. I think that bumping Dropwizard to 4.x 
for Cassandra 5.0 would be ideal.


was (Author: smiklosovic):
[~e.dimitrova] [~mmuzaf] [~brandonwilliams] any feedback on my last comment 
here? This relates to CASSANDRA-15472. I think that by bumping Dropwizard to 
4.x for Cassandra 5.0 would be ideal.

> Upgrade Dropwizard Metrics to 4.x
> -
>
> Key: CASSANDRA-14667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14667
> Project: Cassandra
>  Issue Type: Task
>  Components: Observability/Metrics
>Reporter: Stig Rohde Døssing
>Assignee: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.x
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Cassandra currently uses Metrics 3.1.5. Version 4.0.0 added some fixes for 
> Java 9 compatibility. It would be good to upgrade the Metrics library as part 
> of the version of Cassandra that adds Java 9 compatibility 
> (https://issues.apache.org/jira/browse/CASSANDRA-9608). 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14667) Upgrade Dropwizard Metrics to 4.x



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718836#comment-17718836
 ] 

Stefan Miklosovic commented on CASSANDRA-14667:
---

[~e.dimitrova] [~mmuzaf] [~brandonwilliams] any feedback on my last comment 
here? This relates to CASSANDRA-15472. I think that by bumping Dropwizard to 
4.x for Cassandra 5.0 would be ideal.

> Upgrade Dropwizard Metrics to 4.x
> -
>
> Key: CASSANDRA-14667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14667
> Project: Cassandra
>  Issue Type: Task
>  Components: Observability/Metrics
>Reporter: Stig Rohde Døssing
>Assignee: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.x
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Cassandra currently uses Metrics 3.1.5. Version 4.0.0 added some fixes for 
> Java 9 compatibility. It would be good to upgrade the Metrics library as part 
> of the version of Cassandra that adds Java 9 compatibility 
> (https://issues.apache.org/jira/browse/CASSANDRA-9608). 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15472) Read failure due to exception from metrics-core dependency



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718835#comment-17718835
 ] 

Stefan Miklosovic edited comment on CASSANDRA-15472 at 5/3/23 8:16 AM:
---

The investigation of [~andrew.tolbert] is confirmed in CASSANDRA-14667. I think 
that we should bump the version of dropwizard to latest 4 but there are some 
caveats as described in my comment here (1)

I am not completely sure if we want to bump the version of Dropwizard from 3.0 
included. I would focus just on current trunk (future 5.0).

(1) 
https://issues.apache.org/jira/browse/CASSANDRA-14667?focusedCommentId=17703962=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17703962


was (Author: smiklosovic):
The investigation of [~andrew.tolbert] is confirmed in CASSANDRA-14667. I think 
that we should bump the version of dropwizard to latest 4 but there are some 
caveats as described in my comment here (1)

( 1) 
https://issues.apache.org/jira/browse/CASSANDRA-14667?focusedCommentId=17703962=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17703962

> Read failure due to exception from metrics-core dependency
> --
>
> Key: CASSANDRA-15472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15472
> Project: Cassandra
>  Issue Type: Bug
>  Components: Dependencies
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> Stacktrace
> {code:java}
> Uncaught exception on thread Thread[SharedPool-Worker-27,5,main]: {}
> java.util.NoSuchElementException: null
>   at 
> java.util.concurrent.ConcurrentSkipListMap.firstKey(ConcurrentSkipListMap.java:2053)
>  ~[na:1.8.0_222]
>   at 
> com.yammer.metrics.stats.ExponentiallyDecayingSample.update(ExponentiallyDecayingSample.java:102)
>  ~[metrics-core-2.2.0.jar:na]
>   at 
> com.yammer.metrics.stats.ExponentiallyDecayingSample.update(ExponentiallyDecayingSample.java:81)
>  ~[metrics-core-2.2.0.jar:na]
>   at com.yammer.metrics.core.Histogram.update(Histogram.java:110) 
> ~[metrics-core-2.2.0.jar:na]
>   at com.yammer.metrics.core.Timer.update(Timer.java:198) 
> ~[metrics-core-2.2.0.jar:na]
>   at com.yammer.metrics.core.Timer.update(Timer.java:76) 
> ~[metrics-core-2.2.0.jar:na]
>   at 
> org.apache.cassandra.metrics.LatencyMetrics.addNano(LatencyMetrics.java:108) 
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
>   at 
> org.apache.cassandra.metrics.LatencyMetrics.addNano(LatencyMetrics.java:114) 
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1897)
>  ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
>   at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:353) 
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
>   at 
> org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:85)
>  ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
>   at 
> org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47) 
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
>   at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64) 
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_222]
>   at 
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
>  ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [nf-cassandra-2.1.19.10.jar:2.1.19.10]
>   at java.lang.Thread.run(Thread.java:748) [na:1.8.0_222]
> {code}
> This [issue|https://github.com/dropwizard/metrics/issues/1278] has been 
> [fixed|https://github.com/dropwizard/metrics/pull/1436] in 
> [v4.0.6|https://github.com/dropwizard/metrics/releases/tag/v4.0.6].
> This is observed on a 2.1.19 cluster, but this would impact pretty much any 
> version of C* since we depend on lower versions of metrics-core that do not 
> have the fix.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15472) Read failure due to exception from metrics-core dependency



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718835#comment-17718835
 ] 

Stefan Miklosovic commented on CASSANDRA-15472:
---

The investigation of [~andrew.tolbert] is confirmed in CASSANDRA-14667. I think 
that we should bump the version of dropwizard to latest 4 but there are some 
caveats as described in my comment here (1)

( 1) 
https://issues.apache.org/jira/browse/CASSANDRA-14667?focusedCommentId=17703962=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17703962

> Read failure due to exception from metrics-core dependency
> --
>
> Key: CASSANDRA-15472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15472
> Project: Cassandra
>  Issue Type: Bug
>  Components: Dependencies
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> Stacktrace
> {code:java}
> Uncaught exception on thread Thread[SharedPool-Worker-27,5,main]: {}
> java.util.NoSuchElementException: null
>   at 
> java.util.concurrent.ConcurrentSkipListMap.firstKey(ConcurrentSkipListMap.java:2053)
>  ~[na:1.8.0_222]
>   at 
> com.yammer.metrics.stats.ExponentiallyDecayingSample.update(ExponentiallyDecayingSample.java:102)
>  ~[metrics-core-2.2.0.jar:na]
>   at 
> com.yammer.metrics.stats.ExponentiallyDecayingSample.update(ExponentiallyDecayingSample.java:81)
>  ~[metrics-core-2.2.0.jar:na]
>   at com.yammer.metrics.core.Histogram.update(Histogram.java:110) 
> ~[metrics-core-2.2.0.jar:na]
>   at com.yammer.metrics.core.Timer.update(Timer.java:198) 
> ~[metrics-core-2.2.0.jar:na]
>   at com.yammer.metrics.core.Timer.update(Timer.java:76) 
> ~[metrics-core-2.2.0.jar:na]
>   at 
> org.apache.cassandra.metrics.LatencyMetrics.addNano(LatencyMetrics.java:108) 
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
>   at 
> org.apache.cassandra.metrics.LatencyMetrics.addNano(LatencyMetrics.java:114) 
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1897)
>  ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
>   at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:353) 
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
>   at 
> org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:85)
>  ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
>   at 
> org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47) 
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
>   at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64) 
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_222]
>   at 
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
>  ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [nf-cassandra-2.1.19.10.jar:2.1.19.10]
>   at java.lang.Thread.run(Thread.java:748) [na:1.8.0_222]
> {code}
> This [issue|https://github.com/dropwizard/metrics/issues/1278] has been 
> [fixed|https://github.com/dropwizard/metrics/pull/1436] in 
> [v4.0.6|https://github.com/dropwizard/metrics/releases/tag/v4.0.6].
> This is observed on a 2.1.19 cluster, but this would impact pretty much any 
> version of C* since we depend on lower versions of metrics-core that do not 
> have the fix.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-17249) DOC - Fix dead link for KDM in Data Modeling section



[ 
https://issues.apache.org/jira/browse/CASSANDRA-17249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718829#comment-17718829
 ] 

Stefan Miklosovic commented on CASSANDRA-17249:
---

[http://kdm.kashliev.com/] does not work anymore

> DOC - Fix dead link for KDM in Data Modeling section
> 
>
> Key: CASSANDRA-17249
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17249
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation/Website
>Reporter: Artem Chebotko
>Assignee: Erick Ramirez
>Priority: Normal
> Fix For: 4.0.x
>
>
> The link for Kashlev Data Modeler at 
> [https://cassandra.apache.org/doc/latest/cassandra/data_modeling/data_modeling_tools.html]
>  needs to be updated to [http://kdm.kashliev.com/].



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-17251) USING writetime + ttl is non-idempotent leading to non-deterministic merge iteration results



[ 
https://issues.apache.org/jira/browse/CASSANDRA-17251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718825#comment-17718825
 ] 

Stefan Miklosovic commented on CASSANDRA-17251:
---

Would you please prepare patches for all branches up to trunk, [~jwest] ? (if 
this issue happens there as well)

> USING writetime + ttl is non-idempotent leading to non-deterministic merge 
> iteration results
> 
>
> Key: CASSANDRA-17251
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17251
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Other
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> The combination of {{USING writetime = timestamp and ttl = ttl}} can result 
> in non-deterministic MergeIterator results causing DigestMismatchExceptions 
> and increased latencies. The increased latencies are caused by additional 
> round trips due to the digest mismatch as well as read repair rewriting the 
> data. The additional writes lead to an increase in the number of sstables the 
> key is stored in and must be scanned on read.
> The order of events is:
> 1. for a given partition a write is performed with {{USING timestamp = 
> sometime and ttl = ttl1}}.
> 2. Cassandra records this write with timestamp = sometime, ttl = ttl1, 
> expires_at = now + ttl1
> 3. after N seconds, for the same partition, another write is performed with 
> {{USING timestamp = sometime and ttl = ttl2 where ttl2 = ttl1 - N}}. This 
> write only makes it to a subset of replicas* for some reason (e.g. partial 
> write, node down, etc).
> 4. Cassandra records this write with timestamp = sometime, ttl = ttl2, 
> expires_at = now + ttl2. Its important to note that at this point, expires_at 
> in 2 above is equal to expires at here. This is because it is calculated 
> relative to the current write time not the provided timestamp and the ttl has 
> been adjusted by the time passed. This write also makes it to a subset of 
> replicas*.
> 5. A read of the data is performed.
> 5a. The MergeIterator resolves conflicts locally (accross sstables) using 
> {{Conflicts.resolveRegular}} or {{Cells.resolveRegular}}. The resolution 
> takes into account the write timestamp , the liveness of the cell, the values 
> themselves, and how much time is left to live via the expires_at field. In 
> this scenario, all of these fields are equal, leading to Cassandra picking 
> the sstable "on the right" – this is non-deterministic. The only item that 
> differs is the ttl itself. 
> 5b. One node returns the non-deterministically chosen value for the row, the 
> other two calculate and send a digest to the coordinator. The digest includes 
> the relative ttl field which may not match. This results in a 
> DigestMismatchException at the coordinator.
> 6. Read repair is triggered 
> *NOTE: its not strictly necessary for the write to make it to a subset of 
> replicas. sstables can also be ordered in random orders for reasons like 
> compaction or repair when returned from the live set which can lead to the 
> same behavior. This also affects repair from what we can tell. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-17047) Dropping a column can break queries until the schema is fully propagated (TAKE 2)