[jira] [Updated] (CASSANDRA-14573) Expose settings in virtual table
[
https://issues.apache.org/jira/browse/CASSANDRA-14573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CASSANDRA-14573:
---
Labels: pull-request-available virtual-tables (was: virtual-tables)
> Expose settings in virtual table
>
>
> Key: CASSANDRA-14573
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14573
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Labels: pull-request-available, virtual-tables
> Fix For: 4.x
>
>
> Allow both viewing what the settings are (currently impossible for some) and
> allow changing some settings.
> Example:
> {code:java}
> UPDATE system_info.settings SET value = 'false' WHERE setting =
> 'hinted_handoff_enabled';
> SELECT * FROM system_info.settings WHERE writable = True;
> setting | value | writable
> --++--
> batch_size_fail_threshold_in_kb | 50 | True
> batch_size_warn_threshold_in_kb | 5 | True
> cas_contention_timeout_in_ms | 1000 | True
> compaction_throughput_mb_per_sec | 16 | True
> concurrent_compactors | 2 | True
>concurrent_validations | 2147483647 | True
> counter_write_request_timeout_in_ms | 5000 | True
>hinted_handoff_enabled | false | True
> hinted_handoff_throttle_in_kb | 1024 | True
> incremental_backups | false | True
> inter_dc_stream_throughput_outbound_megabits_per_sec |200 | True
> phi_convict_threshold |8.0 | True
> range_request_timeout_in_ms | 1 | True
>read_request_timeout_in_ms | 5000 | True
> request_timeout_in_ms | 1 | True
> stream_throughput_outbound_megabits_per_sec |200 | True
> tombstone_failure_threshold | 10 | True
> tombstone_warn_threshold | 1000 | True
>truncate_request_timeout_in_ms | 6 | True
> write_request_timeout_in_ms | 2000 |
> True{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14621) Refactor CompactionStrategyManager
[ https://issues.apache.org/jira/browse/CASSANDRA-14621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580044#comment-16580044 ] Marcus Eriksson commented on CASSANDRA-14621: - [~aweisberg] yeah I'll review this week > Refactor CompactionStrategyManager > -- > > Key: CASSANDRA-14621 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14621 > Project: Cassandra > Issue Type: Improvement > Components: Compaction >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Minor > Fix For: 4.x > > > CompactionStrategyManager grew a decent amount of duplicated code as part of > CASSANDRA-9143, which added pendingRepairs alongside the repaired and > unrepaired buckets. At this point, the logic that routes sstables between the > different buckets, and the different partition range divisions has gotten a > little complex, and editing it is tedious and error prone. With transient > replication requiring yet another bucket for, this seems like a good time to > split some of the functionality of CSM into other classes, and make sstable > routing a bit more generalized. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14641) Nodetool toppartitions raises java.lang.IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/CASSANDRA-14641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580097#comment-16580097
]
Irina Truong commented on CASSANDRA-14641:
--
Added that.
> Nodetool toppartitions raises java.lang.IndexOutOfBoundsException
> -
>
> Key: CASSANDRA-14641
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14641
> Project: Cassandra
> Issue Type: Bug
> Environment: Linux 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2
> 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
>Reporter: Irina Truong
>Priority: Minor
>
> I get this exception in most cases when using nodetool.toppartitions command:
> {noformat}
> irina@host:/usr/local/cassandra/bin$ ./nodetool toppartitions casterisk
> events_2018_08_13 1000
> error: index (4) must be less than size (4)
> -- StackTrace --
> java.lang.IndexOutOfBoundsException: index (4) must be less than size (4)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:292)
> at
> com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:65)
> at
> org.apache.cassandra.db.marshal.CompositeType.getAndAppendComparator(CompositeType.java:148)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getString(AbstractCompositeType.java:174)
> at
> org.apache.cassandra.db.ColumnFamilyStore.finishLocalSampling(ColumnFamilyStore.java:1588)
> at sun.reflect.GeneratedMethodAccessor1899.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
> at
> javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
> at
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
> at
> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
> at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:324)
> at sun.rmi.transport.Transport$1.run(Transport.java:200)
> at sun.rmi.transport.Transport$1.run(Transport.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
> at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> Occasionally, nodetool will return a proper response. But in most cases, no.
> Cassandra version 3.0.8.
> table schema:
> {noformat}
> cqlsh:casterisk> describe table events_2018_08_14;
> CREATE TABLE
[jira] [Updated] (CASSANDRA-14641) Nodetool toppartitions raises java.lang.IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/CASSANDRA-14641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Irina Truong updated CASSANDRA-14641:
-
Description:
I get this exception in most cases when using nodetool.toppartitions command:
{noformat}
irina@host:/usr/local/cassandra/bin$ ./nodetool toppartitions casterisk
events_2018_08_13 1000
error: index (4) must be less than size (4)
-- StackTrace --
java.lang.IndexOutOfBoundsException: index (4) must be less than size (4)
at
com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310)
at
com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:292)
at
com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:65)
at
org.apache.cassandra.db.marshal.CompositeType.getAndAppendComparator(CompositeType.java:148)
at
org.apache.cassandra.db.marshal.AbstractCompositeType.getString(AbstractCompositeType.java:174)
at
org.apache.cassandra.db.ColumnFamilyStore.finishLocalSampling(ColumnFamilyStore.java:1588)
at sun.reflect.GeneratedMethodAccessor1899.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
at
javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
at
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:324)
at sun.rmi.transport.Transport$1.run(Transport.java:200)
at sun.rmi.transport.Transport$1.run(Transport.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683)
at java.security.AccessController.doPrivileged(Native Method)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}
Occasionally, nodetool will return a proper response. But in most cases, no.
Cassandra version 3.0.8.
table schema:
{noformat}
cqlsh:casterisk> describe table events_2018_08_14;
CREATE TABLE casterisk.events_2018_08_14 (
event_type text,
apikey text,
url text,
period timestamp,
ts timestamp,
event_id blob,
d blob,
PRIMARY KEY ((event_type, apikey, url, period), ts, event_id)
) WITH COMPACT STORAGE
AND CLUSTERING ORDER BY (ts ASC, event_id ASC)
AND bloom_filter_fp_chance = 0.1
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class':
'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
AND compression = {'chunk_length_in_kb': '64', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND
[jira] [Updated] (CASSANDRA-14573) Expose settings in virtual table
[
https://issues.apache.org/jira/browse/CASSANDRA-14573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CASSANDRA-14573:
---
Labels: pull-request-available virtual-tables (was: virtual-tables)
> Expose settings in virtual table
>
>
> Key: CASSANDRA-14573
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14573
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Labels: pull-request-available, virtual-tables
> Fix For: 4.x
>
>
> Allow both viewing what the settings are (currently impossible for some) and
> allow changing some settings.
> Example:
> {code:java}
> UPDATE system_info.settings SET value = 'false' WHERE setting =
> 'hinted_handoff_enabled';
> SELECT * FROM system_info.settings WHERE writable = True;
> setting | value | writable
> --++--
> batch_size_fail_threshold_in_kb | 50 | True
> batch_size_warn_threshold_in_kb | 5 | True
> cas_contention_timeout_in_ms | 1000 | True
> compaction_throughput_mb_per_sec | 16 | True
> concurrent_compactors | 2 | True
>concurrent_validations | 2147483647 | True
> counter_write_request_timeout_in_ms | 5000 | True
>hinted_handoff_enabled | false | True
> hinted_handoff_throttle_in_kb | 1024 | True
> incremental_backups | false | True
> inter_dc_stream_throughput_outbound_megabits_per_sec |200 | True
> phi_convict_threshold |8.0 | True
> range_request_timeout_in_ms | 1 | True
>read_request_timeout_in_ms | 5000 | True
> request_timeout_in_ms | 1 | True
> stream_throughput_outbound_megabits_per_sec |200 | True
> tombstone_failure_threshold | 10 | True
> tombstone_warn_threshold | 1000 | True
>truncate_request_timeout_in_ms | 6 | True
> write_request_timeout_in_ms | 2000 |
> True{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
cassandra git commit: Allocate ReentrantLock on-demand in java11 AtomicBTreePartitionerBase
Repository: cassandra
Updated Branches:
refs/heads/trunk 35750e805 -> 7925f91dc
Allocate ReentrantLock on-demand in java11 AtomicBTreePartitionerBase
patch by jasobrown; reviewed by Robert Stupp for CASSANDRA-14637
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7925f91d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7925f91d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7925f91d
Branch: refs/heads/trunk
Commit: 7925f91dc842834c39504b5f9a68db9b5c9fd879
Parents: 35750e8
Author: Jason Brown
Authored: Fri Aug 10 07:31:28 2018 -0700
Committer: Jason Brown
Committed: Tue Aug 14 09:48:40 2018 -0700
--
CHANGES.txt | 1 +
.../cassandra/db/partitions/AtomicBTreePartitionBase.java | 7 ++-
2 files changed, 7 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/7925f91d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 52933c8..67e85f6 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0
+ * Allocate ReentrantLock on-demand in java11 AtomicBTreePartitionerBase
(CASSANDRA-14637)
* Make all existing virtual tables use LocalPartitioner (CASSANDRA-14640)
* Revert 4.0 GC alg back to CMS (CASANDRA-14636)
* Remove hardcoded java11 jvm args in idea workspace files (CASSANDRA-14627)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/7925f91d/src/java11/org/apache/cassandra/db/partitions/AtomicBTreePartitionBase.java
--
diff --git
a/src/java11/org/apache/cassandra/db/partitions/AtomicBTreePartitionBase.java
b/src/java11/org/apache/cassandra/db/partitions/AtomicBTreePartitionBase.java
index ac1a11d..56359a2 100644
---
a/src/java11/org/apache/cassandra/db/partitions/AtomicBTreePartitionBase.java
+++
b/src/java11/org/apache/cassandra/db/partitions/AtomicBTreePartitionBase.java
@@ -18,6 +18,7 @@
package org.apache.cassandra.db.partitions;
+import java.util.concurrent.atomic.AtomicReferenceFieldUpdater;
import java.util.concurrent.locks.ReentrantLock;
import org.slf4j.Logger;
@@ -38,7 +39,8 @@ public abstract class AtomicBTreePartitionBase extends
AbstractBTreePartition
}
// Replacement for Unsafe.monitorEnter/monitorExit.
-private final ReentrantLock lock = new ReentrantLock();
+private volatile ReentrantLock lock;
+private static final AtomicReferenceFieldUpdater lockUpdater =
AtomicReferenceFieldUpdater.newUpdater(AtomicBTreePartitionBase.class,
ReentrantLock.class, "lock");
static
{
@@ -50,6 +52,9 @@ public abstract class AtomicBTreePartitionBase extends
AbstractBTreePartition
protected final void acquireLock()
{
+if (lock == null)
+lockUpdater.compareAndSet(this, null, new ReentrantLock());
+
lock.lock();
}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14637) Allocate ReentrantLock on-demand in java11 AtomicBTreePartitionerBase
[
https://issues.apache.org/jira/browse/CASSANDRA-14637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-14637:
-
Reviewer: Robert Stupp
+1
> Allocate ReentrantLock on-demand in java11 AtomicBTreePartitionerBase
> -
>
> Key: CASSANDRA-14637
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14637
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Jason Brown
>Assignee: Jason Brown
>Priority: Minor
> Labels: Java11
> Fix For: 4.x
>
>
> As an intermediate step before CASSANDRA-14607, let's allocate the
> {{ReentrantLock}} in the java11 version of {{AtomicBTreePartitionerBase}} on
> demand rather than with every instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14637) Allocate ReentrantLock on-demand in java11 AtomicBTreePartitionerBase
[
https://issues.apache.org/jira/browse/CASSANDRA-14637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-14637:
-
Status: Ready to Commit (was: Patch Available)
> Allocate ReentrantLock on-demand in java11 AtomicBTreePartitionerBase
> -
>
> Key: CASSANDRA-14637
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14637
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Jason Brown
>Assignee: Jason Brown
>Priority: Minor
> Labels: Java11
> Fix For: 4.x
>
>
> As an intermediate step before CASSANDRA-14607, let's allocate the
> {{ReentrantLock}} in the java11 version of {{AtomicBTreePartitionerBase}} on
> demand rather than with every instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14637) Allocate ReentrantLock on-demand in java11 AtomicBTreePartitionerBase
[
https://issues.apache.org/jira/browse/CASSANDRA-14637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Brown updated CASSANDRA-14637:
Resolution: Fixed
Fix Version/s: (was: 4.x)
4.0
Status: Resolved (was: Ready to Commit)
committed as sha {{7925f91dc842834c39504b5f9a68db9b5c9fd879}}
> Allocate ReentrantLock on-demand in java11 AtomicBTreePartitionerBase
> -
>
> Key: CASSANDRA-14637
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14637
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Jason Brown
>Assignee: Jason Brown
>Priority: Minor
> Labels: Java11
> Fix For: 4.0
>
>
> As an intermediate step before CASSANDRA-14607, let's allocate the
> {{ReentrantLock}} in the java11 version of {{AtomicBTreePartitionerBase}} on
> demand rather than with every instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14638) Column result order can change in 'SELECT *' results when upgrading from 2.1 to 3.0 causing response corruption for queries using prepared statements when static co
[
https://issues.apache.org/jira/browse/CASSANDRA-14638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
C. Scott Andreas reassigned CASSANDRA-14638:
Assignee: Aleksey Yeschenko (was: C. Scott Andreas)
> Column result order can change in 'SELECT *' results when upgrading from 2.1
> to 3.0 causing response corruption for queries using prepared statements when
> static columns are used
> --
>
> Key: CASSANDRA-14638
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14638
> Project: Cassandra
> Issue Type: Bug
> Environment: Single C* node ccm cluster upgraded from C* 2.1.20 to
> 3.0.17
>Reporter: Andy Tolbert
>Assignee: Aleksey Yeschenko
>Priority: Major
>
> When performing an upgrade from C* 2.1.20 to 3.0.17 I observed that the order
> of columns returned from a 'SELECT *' query changes, particularly when static
> columns are involved.
> This may not seem like that much of a problem, however if using Prepared
> Statements, any clients that remain connected during the upgrade may
> encounter issues consuming results from these queries, as data is reordered
> and the client not aware of it. The result definition is sent in the
> original prepared statement response, so if order changes the client has no
> way of knowing (until C* 4.0 via CASSANDRA-10786) without re-preparing, which
> is non-trivial as most client drivers cache prepared statements.
> This could lead to reading the wrong values for columns, which could result
> in some kind of deserialization exception or if the data types of the
> switched columns are compatible, the wrong values. This happens even if the
> client attempts to retrieve a column value by name (i.e. row.getInt("colx")).
> Unfortunately I don't think there is an easy fix for this. If the order was
> changed back to the previous format, you risk issues for users upgrading from
> older 3.0 version. I think it would be nice to add a note in the NEWS file
> in the 3.0 upgrade section that describes this issue, and how to work around
> it (specify all column names of interest explicitly in query).
> Example schema and code to reproduce:
>
> {noformat}
> create keyspace ks with replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> create table ks.tbl (p0 text,
> p1 text,
> m map static,
> t text,
> u text static,
> primary key (p0, p1)
> );
> insert into ks.tbl (p0, p1, m, t, u) values ('p0', 'p1', { 'm0' : 'm1' },
> 't', 'u');{noformat}
>
> When querying with 2.1 you'll observe the following order via cqlsh:
> {noformat}
> p0 | p1 | m| u | t
> ++--+---+---
> p0 | p1 | {'m0': 'm1'} | u | t{noformat}
>
> With 3.0, observe that u and m are transposed:
>
> {noformat}
> p0 | p1 | u | m | t
> ++---+--+---
> p0 | p1 | u | {'m0': 'm1'} | t{noformat}
>
>
> {code:java}
> import com.datastax.driver.core.BoundStatement;
> import com.datastax.driver.core.Cluster;
> import com.datastax.driver.core.ColumnDefinitions;
> import com.datastax.driver.core.PreparedStatement;
> import com.datastax.driver.core.ResultSet;
> import com.datastax.driver.core.Row;
> import com.datastax.driver.core.Session;
> import com.google.common.util.concurrent.Uninterruptibles;
> import java.util.concurrent.TimeUnit;
> public class LiveUpgradeTest {
> public static void main(String args[]) {
> Cluster cluster = Cluster.builder().addContactPoints("127.0.0.1").build();
> try {
> Session session = cluster.connect();
> PreparedStatement p = session.prepare("SELECT * from ks.tbl");
> BoundStatement bs = p.bind();
> // continually query every 30 seconds
> while (true) {
> try {
> ResultSet r = session.execute(bs);
> Row row = r.one();
> int i = 0;
> // iterate over the result metadata in order printing the
> // index, name, type, and length of the first row of data.
> for (ColumnDefinitions.Definition d : r.getColumnDefinitions()) {
> System.out.println(
> i++
> + ": "
> + d.getName()
> + " -> "
> + d.getType()
> + " -> val = "
> + row.getBytesUnsafe(d.getName()).array().length);
> }
> } catch (Throwable t) {
> t.printStackTrace();
> } finally {
> Uninterruptibles.sleepUninterruptibly(30, TimeUnit.SECONDS);
> }
> }
> } finally {
> cluster.close();
> }
> }
> }
> {code}
> To reproduce, set up a cluster, the schema, and run
[jira] [Assigned] (CASSANDRA-14638) Column result order can change in 'SELECT *' results when upgrading from 2.1 to 3.0 causing response corruption for queries using prepared statements when static co
[
https://issues.apache.org/jira/browse/CASSANDRA-14638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
C. Scott Andreas reassigned CASSANDRA-14638:
Assignee: C. Scott Andreas
> Column result order can change in 'SELECT *' results when upgrading from 2.1
> to 3.0 causing response corruption for queries using prepared statements when
> static columns are used
> --
>
> Key: CASSANDRA-14638
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14638
> Project: Cassandra
> Issue Type: Bug
> Environment: Single C* node ccm cluster upgraded from C* 2.1.20 to
> 3.0.17
>Reporter: Andy Tolbert
>Assignee: C. Scott Andreas
>Priority: Major
>
> When performing an upgrade from C* 2.1.20 to 3.0.17 I observed that the order
> of columns returned from a 'SELECT *' query changes, particularly when static
> columns are involved.
> This may not seem like that much of a problem, however if using Prepared
> Statements, any clients that remain connected during the upgrade may
> encounter issues consuming results from these queries, as data is reordered
> and the client not aware of it. The result definition is sent in the
> original prepared statement response, so if order changes the client has no
> way of knowing (until C* 4.0 via CASSANDRA-10786) without re-preparing, which
> is non-trivial as most client drivers cache prepared statements.
> This could lead to reading the wrong values for columns, which could result
> in some kind of deserialization exception or if the data types of the
> switched columns are compatible, the wrong values. This happens even if the
> client attempts to retrieve a column value by name (i.e. row.getInt("colx")).
> Unfortunately I don't think there is an easy fix for this. If the order was
> changed back to the previous format, you risk issues for users upgrading from
> older 3.0 version. I think it would be nice to add a note in the NEWS file
> in the 3.0 upgrade section that describes this issue, and how to work around
> it (specify all column names of interest explicitly in query).
> Example schema and code to reproduce:
>
> {noformat}
> create keyspace ks with replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> create table ks.tbl (p0 text,
> p1 text,
> m map static,
> t text,
> u text static,
> primary key (p0, p1)
> );
> insert into ks.tbl (p0, p1, m, t, u) values ('p0', 'p1', { 'm0' : 'm1' },
> 't', 'u');{noformat}
>
> When querying with 2.1 you'll observe the following order via cqlsh:
> {noformat}
> p0 | p1 | m| u | t
> ++--+---+---
> p0 | p1 | {'m0': 'm1'} | u | t{noformat}
>
> With 3.0, observe that u and m are transposed:
>
> {noformat}
> p0 | p1 | u | m | t
> ++---+--+---
> p0 | p1 | u | {'m0': 'm1'} | t{noformat}
>
>
> {code:java}
> import com.datastax.driver.core.BoundStatement;
> import com.datastax.driver.core.Cluster;
> import com.datastax.driver.core.ColumnDefinitions;
> import com.datastax.driver.core.PreparedStatement;
> import com.datastax.driver.core.ResultSet;
> import com.datastax.driver.core.Row;
> import com.datastax.driver.core.Session;
> import com.google.common.util.concurrent.Uninterruptibles;
> import java.util.concurrent.TimeUnit;
> public class LiveUpgradeTest {
> public static void main(String args[]) {
> Cluster cluster = Cluster.builder().addContactPoints("127.0.0.1").build();
> try {
> Session session = cluster.connect();
> PreparedStatement p = session.prepare("SELECT * from ks.tbl");
> BoundStatement bs = p.bind();
> // continually query every 30 seconds
> while (true) {
> try {
> ResultSet r = session.execute(bs);
> Row row = r.one();
> int i = 0;
> // iterate over the result metadata in order printing the
> // index, name, type, and length of the first row of data.
> for (ColumnDefinitions.Definition d : r.getColumnDefinitions()) {
> System.out.println(
> i++
> + ": "
> + d.getName()
> + " -> "
> + d.getType()
> + " -> val = "
> + row.getBytesUnsafe(d.getName()).array().length);
> }
> } catch (Throwable t) {
> t.printStackTrace();
> } finally {
> Uninterruptibles.sleepUninterruptibly(30, TimeUnit.SECONDS);
> }
> }
> } finally {
> cluster.close();
> }
> }
> }
> {code}
> To reproduce, set up a cluster, the schema, and run this script. Then
> upgrade
[jira] [Assigned] (CASSANDRA-14635) Support table level configuration of monotonic reads
[ https://issues.apache.org/jira/browse/CASSANDRA-14635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Blake Eggleston reassigned CASSANDRA-14635: --- Assignee: Blake Eggleston > Support table level configuration of monotonic reads > > > Key: CASSANDRA-14635 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14635 > Project: Cassandra > Issue Type: Improvement > Components: Coordination >Reporter: Ariel Weisberg >Assignee: Blake Eggleston >Priority: Major > > In CASSANDRA-10726 it was discussed that allowing expert users to forgo > monotonic reads might be desirable. It is practical to control monotonicity > of reads at a fine grained level because it involves changing the behavior of > read repair on a per read basis. > Per CASSANDRA-14593 we already don't preserve update atomicity down to the > column level. You could read the key out of a row and read repair the key, > pass the key to another process which attempts to read the value, but finds > the value is null because read repair only repairs the data (including > columns) that is part of the read. IMO it's a stretch to say that reads are > monotonic. It is technically correct, the best kind of correct, but far from > as useful as it should be. > An initial implementation could make read repair asynchronous or forgo read > repair entirely. This would improve the throughput and latency of reads. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14298) cqlshlib tests broken on b.a.o
[ https://issues.apache.org/jira/browse/CASSANDRA-14298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580629#comment-16580629 ] Patrick Bannister commented on CASSANDRA-14298: --- [~spo...@gmail.com], I built a CentOS system, checked out your copy of CASSANDRA-14298, and ran some of the tests that failed on build 597. I have LANG=en_US.utf8 and nothing set for LC_ALL or LC_CTYPE. Almost all of the Unicode related tests passed: * test_eat_glass * test_source_glass * test_unicode_syntax_error * test_with_empty_values * test_copy_to I have two suggestions: * Please check the output of locale -a on your test system and confirm that you have an en_US.utf8 locale generated. * If locale -a lists en_US.utf8 but not en_US.UTF-8, please try setting the environment variable LANG=en_US.utf8. Alternatively - is it possible for me to duplicate your Jenkins build and do that troubleshooting myself? I haven't set up on Jenkins yet, but I'd be willing to do it to work more efficiently on this ticket and eat up a little less of your time. Other failures...test_unicode_invalid_request_error fails for me too, but it looks like a superficial problem. test_materialized_view looks like a more substantive issue. I'll check into both. > cqlshlib tests broken on b.a.o > -- > > Key: CASSANDRA-14298 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14298 > Project: Cassandra > Issue Type: Bug > Components: Build, Testing >Reporter: Stefan Podkowinski >Assignee: Patrick Bannister >Priority: Major > Labels: cqlsh, dtest > Attachments: CASSANDRA-14298-old.txt, CASSANDRA-14298.txt, > cqlsh_tests_notes.md > > > It appears that cqlsh-tests on builds.apache.org on all branches stopped > working since we removed nosetests from the system environment. See e.g. > [here|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-cqlsh-tests/458/cython=no,jdk=JDK%201.8%20(latest),label=cassandra/console]. > Looks like we either have to make nosetests available again or migrate to > pytest as we did with dtests. Giving pytest a quick try resulted in many > errors locally, but I haven't inspected them in detail yet. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14346) Scheduled Repair in Cassandra
[
https://issues.apache.org/jira/browse/CASSANDRA-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580715#comment-16580715
]
Joseph Lynch commented on CASSANDRA-14346:
--
Ok, [~vinaykumarcse] and I have a Cassandra trunk patchset we believe is ready
for review to start. We are still tying up a few loose ends, especially around
integration tests (all of our e2e tests used the drivers CCMBridge system, not
python dtests so we're still working on porting those over), but I think in the
interest of giving folks like [~bdeggleston] or others a chance to start
reviewing we should get it started. Below is the patch-set broken into
hopefully easily reviewable 13 commits.
||trunk||
|[patch|https://github.com/apache/cassandra/compare/trunk...jolynch:trunk_CASSANDRA-14346_review]|
|[pull request for reviewing|https://github.com/jolynch/cassandra/pull/2]|
|[unit
tests|https://circleci.com/gh/jolynch/workflows/cassandra/tree/trunk_CASSANDRA-14346_review]
[!https://circleci.com/gh/jolynch/cassandra/tree/trunk_CASSANDRA-14346_review.png?circle-token=
1102a59698d04899ec971dd36e925928f7b521f5!|https://circleci.com/gh/jolynch/workflows/cassandra/tree/trunk_CASSANDRA-14346_review]|
I believe that we implement all proposed scope for V1 of the repair scheduler
as per the design document. A short summary of included functionality:
# A basic HTTP sidecar integrated with the ant build (CASSANDRA-14395)
# Circleci compiles the sidecar and runs the sidecar unit tests
# A fully decentralized orchestration engine for running repair and hooks in a
resilient, reliable manner
# Pluggable interface for abstracting different repair APIs with 4.x
implementation by default, we believe that 3.0, 3.11 and 4.x can all be
supported via {{CassandraInteraction}} implementations.
# Supports typical repair types: vnodes, no vnodes, incremental repair,
full+subrange repair. These are also abstracted (see #4) so that maintenance
should hopefully be low going
# Pluggable configuration with yaml implementation by default.
# Pluggable post repair hooks (actions run after repair is done on all
neighbors) with cleanup and compaction implementations by default (cleanup is
enabled by default)
# Additional repair scheduling metrics
# Schema/infra support for schedules although only one schedule is supported
at this time.
We are also missing some things, in particular we have to finish porting our
dtests over to python (from {{CCMBridge}} and the rest of our unit tests to
{{EmbeddedCasandra}} (from {{CassandraUnit}}) and there are still a few TODOs
to track down. [~bdeggleston], [~kohlisankalp] if we can get help getting the
review started that would be amazing as I imagine there is going to be a lot of
feedback and we'll need time to incorporate it all and battle test it for 4.0 .
> Scheduled Repair in Cassandra
> -
>
> Key: CASSANDRA-14346
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14346
> Project: Cassandra
> Issue Type: Improvement
> Components: Repair
>Reporter: Joseph Lynch
>Assignee: Joseph Lynch
>Priority: Major
> Labels: 4.0-feature-freeze-review-requested,
> CommunityFeedbackRequested
> Fix For: 4.0
>
> Attachments: ScheduledRepairV1_20180327.pdf
>
>
> There have been many attempts to automate repair in Cassandra, which makes
> sense given that it is necessary to give our users eventual consistency. Most
> recently CASSANDRA-10070, CASSANDRA-8911 and CASSANDRA-13924 have all looked
> for ways to solve this problem.
> At Netflix we've built a scheduled repair service within Priam (our sidecar),
> which we spoke about last year at NGCC. Given the positive feedback at NGCC
> we focussed on getting it production ready and have now been using it in
> production to repair hundreds of clusters, tens of thousands of nodes, and
> petabytes of data for the past six months. Also based on feedback at NGCC we
> have invested effort in figuring out how to integrate this natively into
> Cassandra rather than open sourcing it as an external service (e.g. in Priam).
> As such, [~vinaykumarcse] and I would like to re-work and merge our
> implementation into Cassandra, and have created a [design
> document|https://docs.google.com/document/d/1RV4rOrG1gwlD5IljmrIq_t45rz7H3xs9GbFSEyGzEtM/edit?usp=sharing]
> showing how we plan to make it happen, including the the user interface.
> As we work on the code migration from Priam to Cassandra, any feedback would
> be greatly appreciated about the interface or v1 implementation features. I
> have tried to call out in the document features which we explicitly consider
> future work (as well as a path forward to implement them in the future)
> because I would very much like to get this done before the 4.0 merge window
> closes,
[jira] [Updated] (CASSANDRA-14346) Scheduled Repair in Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Lynch updated CASSANDRA-14346: - Status: Patch Available (was: Open) > Scheduled Repair in Cassandra > - > > Key: CASSANDRA-14346 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14346 > Project: Cassandra > Issue Type: Improvement > Components: Repair >Reporter: Joseph Lynch >Assignee: Joseph Lynch >Priority: Major > Labels: 4.0-feature-freeze-review-requested, > CommunityFeedbackRequested > Fix For: 4.0 > > Attachments: ScheduledRepairV1_20180327.pdf > > > There have been many attempts to automate repair in Cassandra, which makes > sense given that it is necessary to give our users eventual consistency. Most > recently CASSANDRA-10070, CASSANDRA-8911 and CASSANDRA-13924 have all looked > for ways to solve this problem. > At Netflix we've built a scheduled repair service within Priam (our sidecar), > which we spoke about last year at NGCC. Given the positive feedback at NGCC > we focussed on getting it production ready and have now been using it in > production to repair hundreds of clusters, tens of thousands of nodes, and > petabytes of data for the past six months. Also based on feedback at NGCC we > have invested effort in figuring out how to integrate this natively into > Cassandra rather than open sourcing it as an external service (e.g. in Priam). > As such, [~vinaykumarcse] and I would like to re-work and merge our > implementation into Cassandra, and have created a [design > document|https://docs.google.com/document/d/1RV4rOrG1gwlD5IljmrIq_t45rz7H3xs9GbFSEyGzEtM/edit?usp=sharing] > showing how we plan to make it happen, including the the user interface. > As we work on the code migration from Priam to Cassandra, any feedback would > be greatly appreciated about the interface or v1 implementation features. I > have tried to call out in the document features which we explicitly consider > future work (as well as a path forward to implement them in the future) > because I would very much like to get this done before the 4.0 merge window > closes, and to do that I think aggressively pruning scope is going to be a > necessity. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14346) Scheduled Repair in Cassandra
[
https://issues.apache.org/jira/browse/CASSANDRA-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580715#comment-16580715
]
Joseph Lynch edited comment on CASSANDRA-14346 at 8/15/18 5:07 AM:
---
Ok, [~vinaykumarcse] and I have a Cassandra trunk patchset we believe is ready
for review to start. We are still tying up a few loose ends, especially around
integration tests (all of our e2e tests used the drivers CCMBridge system, not
python dtests so we're still working on porting those over), but I think in the
interest of giving folks like [~bdeggleston] or others a chance to start
reviewing we should get it started. Below is the patch-set broken into
hopefully easily reviewable 13 commits.
||trunk||
|[patch|https://github.com/apache/cassandra/compare/trunk...jolynch:trunk_CASSANDRA-14346_review]|
|[pull request for reviewing|https://github.com/jolynch/cassandra/pull/2]|
|[unit
tests|https://circleci.com/gh/jolynch/workflows/cassandra/tree/trunk_CASSANDRA-14346_review]
[!https://circleci.com/gh/jolynch/cassandra/tree/trunk_CASSANDRA-14346_review.png?circle-token=
1102a59698d04899ec971dd36e925928f7b521f5!|https://circleci.com/gh/jolynch/workflows/cassandra/tree/trunk_CASSANDRA-14346_review]|
I believe that we implement all proposed scope for V1 of the repair scheduler
as per the design document. A short summary of included functionality:
# A basic HTTP sidecar integrated with the ant build (CASSANDRA-14395)
# Circleci compiles the sidecar and runs the sidecar unit tests
# A fully decentralized orchestration engine for running repair and hooks in a
resilient, reliable manner
# Pluggable interface for abstracting different repair APIs with 4.x
implementation by default, we believe that 3.0, 3.11 and 4.x can all be
supported via {{CassandraInteraction}} implementations.
# Supports typical repair types: vnodes, no vnodes, incremental repair,
full+subrange repair. These are also abstracted (see #4) so that maintenance
should hopefully be low going. Range splits support splitting on partition
count, size, or adaptive (auto).
# Pluggable configuration with yaml implementation by default.
# Pluggable post repair hooks (actions run after repair is done on all
neighbors) with cleanup and compaction implementations by default (cleanup is
enabled by default)
# Additional repair scheduling metrics
# Schema/infra support for schedules although only one schedule is supported
at this time.
We are also missing some things, in particular we have to finish porting our
dtests over to python (from {{CCMBridge}} and the rest of our unit tests to
{{EmbeddedCasandra}} (from {{CassandraUnit}}) and there are still a few TODOs
to track down. [~bdeggleston], [~kohlisankalp] if we can get help getting the
review started that would be amazing as I imagine there is going to be a lot of
feedback and we'll need time to incorporate it all and battle test it for 4.0 .
was (Author: jolynch):
Ok, [~vinaykumarcse] and I have a Cassandra trunk patchset we believe is ready
for review to start. We are still tying up a few loose ends, especially around
integration tests (all of our e2e tests used the drivers CCMBridge system, not
python dtests so we're still working on porting those over), but I think in the
interest of giving folks like [~bdeggleston] or others a chance to start
reviewing we should get it started. Below is the patch-set broken into
hopefully easily reviewable 13 commits.
||trunk||
|[patch|https://github.com/apache/cassandra/compare/trunk...jolynch:trunk_CASSANDRA-14346_review]|
|[pull request for reviewing|https://github.com/jolynch/cassandra/pull/2]|
|[unit
tests|https://circleci.com/gh/jolynch/workflows/cassandra/tree/trunk_CASSANDRA-14346_review]
[!https://circleci.com/gh/jolynch/cassandra/tree/trunk_CASSANDRA-14346_review.png?circle-token=
1102a59698d04899ec971dd36e925928f7b521f5!|https://circleci.com/gh/jolynch/workflows/cassandra/tree/trunk_CASSANDRA-14346_review]|
I believe that we implement all proposed scope for V1 of the repair scheduler
as per the design document. A short summary of included functionality:
# A basic HTTP sidecar integrated with the ant build (CASSANDRA-14395)
# Circleci compiles the sidecar and runs the sidecar unit tests
# A fully decentralized orchestration engine for running repair and hooks in a
resilient, reliable manner
# Pluggable interface for abstracting different repair APIs with 4.x
implementation by default, we believe that 3.0, 3.11 and 4.x can all be
supported via {{CassandraInteraction}} implementations.
# Supports typical repair types: vnodes, no vnodes, incremental repair,
full+subrange repair. These are also abstracted (see #4) so that maintenance
should hopefully be low going
# Pluggable configuration with yaml implementation by default.
# Pluggable post repair hooks (actions run after repair is done on all
neighbors)
[jira] [Commented] (CASSANDRA-14346) Scheduled Repair in Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580734#comment-16580734 ] sankalp kohli commented on CASSANDRA-14346: --- 4.0 freeze is 2-3 weeks away, I am not sure this can make it to 4.0. We can always start the review but I dont think it can merge by 1st September. > Scheduled Repair in Cassandra > - > > Key: CASSANDRA-14346 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14346 > Project: Cassandra > Issue Type: Improvement > Components: Repair >Reporter: Joseph Lynch >Assignee: Joseph Lynch >Priority: Major > Labels: 4.0-feature-freeze-review-requested, > CommunityFeedbackRequested > Fix For: 4.0 > > Attachments: ScheduledRepairV1_20180327.pdf > > > There have been many attempts to automate repair in Cassandra, which makes > sense given that it is necessary to give our users eventual consistency. Most > recently CASSANDRA-10070, CASSANDRA-8911 and CASSANDRA-13924 have all looked > for ways to solve this problem. > At Netflix we've built a scheduled repair service within Priam (our sidecar), > which we spoke about last year at NGCC. Given the positive feedback at NGCC > we focussed on getting it production ready and have now been using it in > production to repair hundreds of clusters, tens of thousands of nodes, and > petabytes of data for the past six months. Also based on feedback at NGCC we > have invested effort in figuring out how to integrate this natively into > Cassandra rather than open sourcing it as an external service (e.g. in Priam). > As such, [~vinaykumarcse] and I would like to re-work and merge our > implementation into Cassandra, and have created a [design > document|https://docs.google.com/document/d/1RV4rOrG1gwlD5IljmrIq_t45rz7H3xs9GbFSEyGzEtM/edit?usp=sharing] > showing how we plan to make it happen, including the the user interface. > As we work on the code migration from Priam to Cassandra, any feedback would > be greatly appreciated about the interface or v1 implementation features. I > have tried to call out in the document features which we explicitly consider > future work (as well as a path forward to implement them in the future) > because I would very much like to get this done before the 4.0 merge window > closes, and to do that I think aggressively pruning scope is going to be a > necessity. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14395) C* Management process
[ https://issues.apache.org/jira/browse/CASSANDRA-14395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580720#comment-16580720 ] Joseph Lynch commented on CASSANDRA-14395: -- We have a basic patch including the base infrastructure over on [CASSANDRA-14346|https://issues.apache.org/jira/browse/CASSANDRA-14346?focusedCommentId=16580715=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16580715] if anyone wants to give feedback there on the basic integration patch. > C* Management process > - > > Key: CASSANDRA-14395 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14395 > Project: Cassandra > Issue Type: New Feature >Reporter: Dinesh Joshi >Assignee: Dinesh Joshi >Priority: Major > > I would like to propose amending Cassandra's architecture to include a > management process. The detailed description is here: > https://docs.google.com/document/d/1UV9pE81NaIUF3g4L1wxq09nT11AkSQcMijgLFwGsY3s/edit > I'd like to propose seeding this with a few simple use-cases such as Health > Checks, Bulk Commands with a simple REST API interface. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14621) Refactor CompactionStrategyManager
[ https://issues.apache.org/jira/browse/CASSANDRA-14621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16579943#comment-16579943 ] Ariel Weisberg commented on CASSANDRA-14621: Fix version listed here is 4.x, but this is a dependency for transient replication. [~krummas] are you realistically going to be able to review this for 4.0? [~cscotta] ^ > Refactor CompactionStrategyManager > -- > > Key: CASSANDRA-14621 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14621 > Project: Cassandra > Issue Type: Improvement > Components: Compaction >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Minor > Fix For: 4.x > > > CompactionStrategyManager grew a decent amount of duplicated code as part of > CASSANDRA-9143, which added pendingRepairs alongside the repaired and > unrepaired buckets. At this point, the logic that routes sstables between the > different buckets, and the different partition range divisions has gotten a > little complex, and editing it is tedious and error prone. With transient > replication requiring yet another bucket for, this seems like a good time to > split some of the functionality of CSM into other classes, and make sstable > routing a bit more generalized. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14640) Set local partitioner for existing virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-14640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16579791#comment-16579791 ] Aleksey Yeschenko edited comment on CASSANDRA-14640 at 8/14/18 2:11 PM: Committed to trunk as [35750e80589e61dbef0c85f20691764a6c7c3f81|https://github.com/apache/cassandra/commit/35750e80589e61dbef0c85f20691764a6c7c3f81], thanks. EDIT: for posterity, CI results [here|https://circleci.com/workflow-run/8c55d0f9-89fb-4d1d-a05a-80c34b5a92e6]. was (Author: iamaleksey): Committed to trunk as [35750e80589e61dbef0c85f20691764a6c7c3f81|https://github.com/apache/cassandra/commit/35750e80589e61dbef0c85f20691764a6c7c3f81], thanks. > Set local partitioner for existing virtual tables > - > > Key: CASSANDRA-14640 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14640 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Minor > Labels: pull-request-available, virtual-tables > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
cassandra git commit: Make all existing virtual tables use LocalPartitioner
Repository: cassandra
Updated Branches:
refs/heads/trunk ed806594e -> 35750e805
Make all existing virtual tables use LocalPartitioner
patch by Chris Lohfink; reviewed by Aleksey Yeschenko for CASSANDRA-14640
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/35750e80
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/35750e80
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/35750e80
Branch: refs/heads/trunk
Commit: 35750e80589e61dbef0c85f20691764a6c7c3f81
Parents: ed80659
Author: Chris Lohfink
Authored: Tue Aug 14 14:11:57 2018 +0100
Committer: Aleksey Yeshchenko
Committed: Tue Aug 14 14:11:57 2018 +0100
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/db/virtual/ClientsTable.java | 2 ++
src/java/org/apache/cassandra/db/virtual/SSTableTasksTable.java | 2 ++
.../org/apache/cassandra/db/virtual/VirtualSchemaKeyspace.java | 4
4 files changed, 9 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/35750e80/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6a45f95..52933c8 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0
+ * Make all existing virtual tables use LocalPartitioner (CASSANDRA-14640)
* Revert 4.0 GC alg back to CMS (CASANDRA-14636)
* Remove hardcoded java11 jvm args in idea workspace files (CASSANDRA-14627)
* Update netty to 4.1.128 (CASSANDRA-14633)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/35750e80/src/java/org/apache/cassandra/db/virtual/ClientsTable.java
--
diff --git a/src/java/org/apache/cassandra/db/virtual/ClientsTable.java
b/src/java/org/apache/cassandra/db/virtual/ClientsTable.java
index 98d1a28..40e175b 100644
--- a/src/java/org/apache/cassandra/db/virtual/ClientsTable.java
+++ b/src/java/org/apache/cassandra/db/virtual/ClientsTable.java
@@ -20,6 +20,7 @@ package org.apache.cassandra.db.virtual;
import java.net.InetSocketAddress;
import org.apache.cassandra.db.marshal.*;
+import org.apache.cassandra.dht.LocalPartitioner;
import org.apache.cassandra.metrics.ClientMetrics;
import org.apache.cassandra.schema.TableMetadata;
import org.apache.cassandra.transport.ConnectedClient;
@@ -44,6 +45,7 @@ final class ClientsTable extends AbstractVirtualTable
super(TableMetadata.builder(keyspace, "clients")
.comment("currently connected clients")
.kind(TableMetadata.Kind.VIRTUAL)
+ .partitioner(new
LocalPartitioner(InetAddressType.instance))
.addPartitionKeyColumn(ADDRESS,
InetAddressType.instance)
.addClusteringColumn(PORT, Int32Type.instance)
.addRegularColumn(HOSTNAME, UTF8Type.instance)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/35750e80/src/java/org/apache/cassandra/db/virtual/SSTableTasksTable.java
--
diff --git a/src/java/org/apache/cassandra/db/virtual/SSTableTasksTable.java
b/src/java/org/apache/cassandra/db/virtual/SSTableTasksTable.java
index 8fb12ba..b387b88 100644
--- a/src/java/org/apache/cassandra/db/virtual/SSTableTasksTable.java
+++ b/src/java/org/apache/cassandra/db/virtual/SSTableTasksTable.java
@@ -22,6 +22,7 @@ import org.apache.cassandra.db.compaction.CompactionManager;
import org.apache.cassandra.db.marshal.LongType;
import org.apache.cassandra.db.marshal.UTF8Type;
import org.apache.cassandra.db.marshal.UUIDType;
+import org.apache.cassandra.dht.LocalPartitioner;
import org.apache.cassandra.schema.TableMetadata;
final class SSTableTasksTable extends AbstractVirtualTable
@@ -39,6 +40,7 @@ final class SSTableTasksTable extends AbstractVirtualTable
super(TableMetadata.builder(keyspace, "sstable_tasks")
.comment("current sstable tasks")
.kind(TableMetadata.Kind.VIRTUAL)
+ .partitioner(new
LocalPartitioner(UTF8Type.instance))
.addPartitionKeyColumn(KEYSPACE_NAME,
UTF8Type.instance)
.addClusteringColumn(TABLE_NAME, UTF8Type.instance)
.addClusteringColumn(TASK_ID, UUIDType.instance)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/35750e80/src/java/org/apache/cassandra/db/virtual/VirtualSchemaKeyspace.java
--
diff --git
a/src/java/org/apache/cassandra/db/virtual/VirtualSchemaKeyspace.java
b/src/java/org/apache/cassandra/db/virtual/VirtualSchemaKeyspace.java
index
[jira] [Updated] (CASSANDRA-14640) Set local partitioner for existing virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-14640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14640: -- Reviewer: Aleksey Yeschenko Fix Version/s: 4.0 > Set local partitioner for existing virtual tables > - > > Key: CASSANDRA-14640 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14640 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Minor > Labels: pull-request-available, virtual-tables > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14640) Set local partitioner for existing virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-14640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16579791#comment-16579791 ] Aleksey Yeschenko commented on CASSANDRA-14640: --- Committed to trunk as [35750e80589e61dbef0c85f20691764a6c7c3f81|https://github.com/apache/cassandra/commit/35750e80589e61dbef0c85f20691764a6c7c3f81], thanks. > Set local partitioner for existing virtual tables > - > > Key: CASSANDRA-14640 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14640 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Minor > Labels: pull-request-available, virtual-tables > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14640) Set local partitioner for existing virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-14640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14640: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Set local partitioner for existing virtual tables > - > > Key: CASSANDRA-14640 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14640 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Minor > Labels: pull-request-available, virtual-tables > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14408) Transient Replication: Incremental & Validation repair handling of transient replicas
[ https://issues.apache.org/jira/browse/CASSANDRA-14408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16579948#comment-16579948 ] Ariel Weisberg commented on CASSANDRA-14408: [Projecting out CASSANDRA-14621|https://github.com/bdeggleston/cassandra/compare/4201772d647935fc65c64484cce6bcd98f8cba4d...bdeggleston:14408?expand=1]. Does that looks right? > Transient Replication: Incremental & Validation repair handling of transient > replicas > - > > Key: CASSANDRA-14408 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14408 > Project: Cassandra > Issue Type: Sub-task > Components: Repair >Reporter: Ariel Weisberg >Assignee: Blake Eggleston >Priority: Major > Fix For: 4.0 > > > At transient replicas anti-compaction shouldn't output any data for transient > ranges as the data will be dropped after repair. > Transient replicas should also never have data streamed to them. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14408) Transient Replication: Incremental & Validation repair handling of transient replicas
[ https://issues.apache.org/jira/browse/CASSANDRA-14408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16579948#comment-16579948 ] Ariel Weisberg edited comment on CASSANDRA-14408 at 8/14/18 3:19 PM: - [Projecting out CASSANDRA-14621|https://github.com/bdeggleston/cassandra/compare/4201772d647935fc65c64484cce6bcd98f8cba4d...bdeggleston:14408?expand=1]. Does that look right? was (Author: aweisberg): [Projecting out CASSANDRA-14621|https://github.com/bdeggleston/cassandra/compare/4201772d647935fc65c64484cce6bcd98f8cba4d...bdeggleston:14408?expand=1]. Does that looks right? > Transient Replication: Incremental & Validation repair handling of transient > replicas > - > > Key: CASSANDRA-14408 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14408 > Project: Cassandra > Issue Type: Sub-task > Components: Repair >Reporter: Ariel Weisberg >Assignee: Blake Eggleston >Priority: Major > Fix For: 4.0 > > > At transient replicas anti-compaction shouldn't output any data for transient > ranges as the data will be dropped after repair. > Transient replicas should also never have data streamed to them. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14408) Transient Replication: Incremental & Validation repair handling of transient replicas
[ https://issues.apache.org/jira/browse/CASSANDRA-14408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16579989#comment-16579989 ] Ariel Weisberg commented on CASSANDRA-14408: I would like to create a PR for us to comment on and get the comments in the work log, but I don't want to include CASSANDRA-14621 unless we are going to review it as part of this. We will need a target branch that has the csm refactor and other stuff in it already. > Transient Replication: Incremental & Validation repair handling of transient > replicas > - > > Key: CASSANDRA-14408 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14408 > Project: Cassandra > Issue Type: Sub-task > Components: Repair >Reporter: Ariel Weisberg >Assignee: Blake Eggleston >Priority: Major > Fix For: 4.0 > > > At transient replicas anti-compaction shouldn't output any data for transient > ranges as the data will be dropped after repair. > Transient replicas should also never have data streamed to them. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14408) Transient Replication: Incremental & Validation repair handling of transient replicas
[ https://issues.apache.org/jira/browse/CASSANDRA-14408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16579992#comment-16579992 ] Blake Eggleston commented on CASSANDRA-14408: - [~aweisberg] opened a PR [here|https://github.com/bdeggleston/cassandra/pull/2] > Transient Replication: Incremental & Validation repair handling of transient > replicas > - > > Key: CASSANDRA-14408 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14408 > Project: Cassandra > Issue Type: Sub-task > Components: Repair >Reporter: Ariel Weisberg >Assignee: Blake Eggleston >Priority: Major > Fix For: 4.0 > > > At transient replicas anti-compaction shouldn't output any data for transient > ranges as the data will be dropped after repair. > Transient replicas should also never have data streamed to them. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
