[jira] [Commented] (CASSANDRA-14396) Error about JNA on Startup

[Joe (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441913#comment-16441913
 ] 

Joe commented on CASSANDRA-14396:
-

[~jasobrown] 

I already gave all permissions to tmp folder but not working. even removed 
folder and ran.

thank you for ur reply :)

> Error about JNA on Startup 
> ---
>
> Key: CASSANDRA-14396
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14396
> Project: Cassandra
>  Issue Type: Test
>  Components: Build, Configuration, Core, Lifecycle
> Environment:  
>  * CentOS release 6.8
>  * Cassandra 3.11.2
>  * Java 1.8.0_152
>  * No provide network
>  
>Reporter: Joe
>Priority: Major
> Fix For: 3.11.2
>
> Attachments: cassandra.log
>
>
>  
> Hi, all.
> I got some error on startup.
> this is my own backup server which can't use network.
> I just extracted 'apache-cassandra-3.11.2-bin.tar.gz' file to 
> /usr/local/cassandra.
> and then ran like this 'cassandra -f'.
> but log displayed below's error.
>  
> I found some way to solve. but it's not working.
> after JNA library download, make JNA library symbolic link - not working
> can anyone advise to me about this issue?(I attached full logging file)
>  
> ERROR [main] 2018-04-18 10:44:59,536 NativeLibraryLinux.java:62 - Failed to 
> link the C library against JNA. Native methods will be unavailable.
>  java.lang.UnsatisfiedLinkError: /tmp/jna-3506402/jna5682737284440877593.tmp: 
> /tmp/jna-3506402/jna5682737284440877593.tmp: failed to map segment from 
> shared object: Operation not permitted
>  at java.lang.ClassLoader$NativeLibrary.load(Native Method) ~[na:1.8.0_152]
>  at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941) ~[na:1.8.0_152]
>  at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824) ~[na:1.8.0_152]
>  at java.lang.Runtime.load0(Runtime.java:809) ~[na:1.8.0_152]
>  at java.lang.System.load(System.java:1086) ~[na:1.8.0_152]
>  at 
> com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:851) 
> ~[jna-4.2.2.jar:4.2.2 (b0)]
>  at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:826) 
> ~[jna-4.2.2.jar:4.2.2 (b0)]
>  at com.sun.jna.Native.(Native.java:140) ~[jna-4.2.2.jar:4.2.2 (b0)]
>  at 
> org.apache.cassandra.utils.NativeLibraryLinux.(NativeLibraryLinux.java:53)
>  ~[apache-cassandra-3.11.2.jar:3.11.2]
>  at org.apache.cassandra.utils.NativeLibrary.(NativeLibrary.java:93) 
> [apache-cassandra-3.11.2.jar:3.11.2]
>  at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:196) 
> [apache-cassandra-3.11.2.jar:3.11.2]
>  at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:602)
>  [apache-cassandra-3.11.2.jar:3.11.2]
>  at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:691) 
> [apache-cassandra-3.11.2.jar:3.11.2]
>  WARN [main] 2018-04-18 10:44:59,537 StartupChecks.java:136 - jemalloc shared 
> library could not be preloaded to speed up memory allocations
>  WARN [main] 2018-04-18 10:44:59,537 StartupChecks.java:169 - JMX is not 
> enabled to receive remote connections. Please see cassandra-env.sh for more 
> info.
>  ERROR [main] 2018-04-18 10:44:59,539 CassandraDaemon.java:708 - The native 
> library could not be initialized properly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14396) Error about JNA on Startup

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441897#comment-16441897
 ] 

Jason Brown commented on CASSANDRA-14396:
-

Some naive internet searches for "failed to map segment from shared object: 
Operation not permitted" suggest that {{/tmp}} needs to be executable. (Note: 
{{/tmp}} is where java apps often decompress/move native libraries from jar 
files so they can be used at runtime)

> Error about JNA on Startup 
> ---
>
> Key: CASSANDRA-14396
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14396
> Project: Cassandra
>  Issue Type: Test
>  Components: Build, Configuration, Core, Lifecycle
> Environment:  
>  * CentOS release 6.8
>  * Cassandra 3.11.2
>  * Java 1.8.0_152
>  * No provide network
>  
>Reporter: Joe
>Priority: Major
> Fix For: 3.11.2
>
> Attachments: cassandra.log
>
>
>  
> Hi, all.
> I got some error on startup.
> this is my own backup server which can't use network.
> I just extracted 'apache-cassandra-3.11.2-bin.tar.gz' file to 
> /usr/local/cassandra.
> and then ran like this 'cassandra -f'.
> but log displayed below's error.
>  
> I found some way to solve. but it's not working.
> after JNA library download, make JNA library symbolic link - not working
> can anyone advise to me about this issue?(I attached full logging file)
>  
> ERROR [main] 2018-04-18 10:44:59,536 NativeLibraryLinux.java:62 - Failed to 
> link the C library against JNA. Native methods will be unavailable.
>  java.lang.UnsatisfiedLinkError: /tmp/jna-3506402/jna5682737284440877593.tmp: 
> /tmp/jna-3506402/jna5682737284440877593.tmp: failed to map segment from 
> shared object: Operation not permitted
>  at java.lang.ClassLoader$NativeLibrary.load(Native Method) ~[na:1.8.0_152]
>  at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941) ~[na:1.8.0_152]
>  at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824) ~[na:1.8.0_152]
>  at java.lang.Runtime.load0(Runtime.java:809) ~[na:1.8.0_152]
>  at java.lang.System.load(System.java:1086) ~[na:1.8.0_152]
>  at 
> com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:851) 
> ~[jna-4.2.2.jar:4.2.2 (b0)]
>  at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:826) 
> ~[jna-4.2.2.jar:4.2.2 (b0)]
>  at com.sun.jna.Native.(Native.java:140) ~[jna-4.2.2.jar:4.2.2 (b0)]
>  at 
> org.apache.cassandra.utils.NativeLibraryLinux.(NativeLibraryLinux.java:53)
>  ~[apache-cassandra-3.11.2.jar:3.11.2]
>  at org.apache.cassandra.utils.NativeLibrary.(NativeLibrary.java:93) 
> [apache-cassandra-3.11.2.jar:3.11.2]
>  at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:196) 
> [apache-cassandra-3.11.2.jar:3.11.2]
>  at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:602)
>  [apache-cassandra-3.11.2.jar:3.11.2]
>  at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:691) 
> [apache-cassandra-3.11.2.jar:3.11.2]
>  WARN [main] 2018-04-18 10:44:59,537 StartupChecks.java:136 - jemalloc shared 
> library could not be preloaded to speed up memory allocations
>  WARN [main] 2018-04-18 10:44:59,537 StartupChecks.java:169 - JMX is not 
> enabled to receive remote connections. Please see cassandra-env.sh for more 
> info.
>  ERROR [main] 2018-04-18 10:44:59,539 CassandraDaemon.java:708 - The native 
> library could not be initialized properly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-14365) Commit log replay failure for static columns with collections in clustering keys

[Kurt Greaves (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kurt Greaves reassigned CASSANDRA-14365:


Assignee: Vincent White

> Commit log replay failure for static columns with collections in clustering 
> keys
> 
>
> Key: CASSANDRA-14365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14365
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Vincent White
>Assignee: Vincent White
>Priority: Major
>
> In the old storage engine, static cells with a collection as part of the 
> clustering key fail to validate because a 0 byte collection (like in the cell 
> name of a static cell) isn't valid.
> To reproduce:
> 1.
> {code:java}
> CREATE TABLE test.x (
> id int,
> id2 frozen>,
> st int static,
> PRIMARY KEY (id, id2)
> );
> INSERT INTO test.x (id, st) VALUES (1, 2);
> {code}
> 2.
>  Kill the cassandra process
> 3.
>  Restart cassandra to replay the commitlog
> Outcome:
> {noformat}
> ERROR [main] 2018-04-05 04:58:23,741 JVMStabilityInspector.java:99 - Exiting 
> due to error while processing commit log during initialization.
> org.apache.cassandra.db.commitlog.CommitLogReplayer$CommitLogReplayException: 
> Unexpected error deserializing mutation; saved to 
> /tmp/mutation3825739904516830950dat.  This may be caused by replaying a 
> mutation against a table with the same name but incompatible schema.  
> Exception follows: org.apache.cassandra.serializers.MarshalException: Not 
> enough bytes to read a set
> at 
> org.apache.cassandra.db.commitlog.CommitLogReplayer.handleReplayError(CommitLogReplayer.java:638)
>  [main/:na]
> at 
> org.apache.cassandra.db.commitlog.CommitLogReplayer.replayMutation(CommitLogReplayer.java:565)
>  [main/:na]
> at 
> org.apache.cassandra.db.commitlog.CommitLogReplayer.replaySyncSection(CommitLogReplayer.java:517)
>  [main/:na]
> at 
> org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:397)
>  [main/:na]
> at 
> org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:143)
>  [main/:na]
> at 
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:181) 
> [main/:na]
> at 
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:161) 
> [main/:na]
> at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:284) 
> [main/:na]
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:533)
>  [main/:na]
> at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:642) 
> [main/:na]
> {noformat}
> I haven't investigated if there are other more subtle issues caused by these 
> cells failing to validate other places in the code, but I believe the fix for 
> this is to check for 0 byte length collections and accept them as valid as we 
> do with other types.
> I haven't had a chance for any extensive testing but this naive patch seems 
> to have the desired affect. [2.2 PoC 
> Patch|https://github.com/vincewhite/cassandra/commits/zero_length_collection]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14304) DELETE after INSERT IF NOT EXISTS does not work

[Kurt Greaves (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441876#comment-16441876
 ] 

Kurt Greaves commented on CASSANDRA-14304:
--

bq. Should we make our own retry policy to prevent retry of such batch? We use 
batches to write large binary BLOBs chunked in smaller statements.
You can, but you'd also have to ensure you don't use speculative execution 
either and possibly other things using client timestamps I don't know about, 
otherwise you risk mixing client and server timestamps (more so than LWT).

bq. However using LWT everywhere seems overkill.
It probably is, but can you break out the statements that absolutely need to be 
ordered into another table? That might be a better alternative...

> DELETE after INSERT IF NOT EXISTS does not work
> ---
>
> Key: CASSANDRA-14304
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14304
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Julien
>Assignee: Vinay Chella
>Priority: Major
> Attachments: debug.log, system.log
>
>
> DELETE a row immediately after INSERT IF NOT EXISTS does not work.
> Can be reproduced with this CQL script:
> {code:java}
> CREATE KEYSPACE ks WITH REPLICATION = { 'class' : 'SimpleStrategy', 
> 'replication_factor' : 1 };
> CREATE TABLE ks.ta ( id text PRIMARY KEY, col text );
> INSERT INTO ks.ta (id, col) VALUES ('myId', 'myCol') IF NOT EXISTS;
> DELETE FROM ks.ta WHERE id = 'myId';
> SELECT * FROM ks.ta WHERE id='myId';
> {code}
> {code:java}
> [cqlsh 5.0.1 | Cassandra 3.11.2 | CQL spec 3.4.4 | Native protocol v4]
> Use HELP for help.
> WARNING: pyreadline dependency missing.  Install to enable tab completion.
> cqlsh> CREATE KEYSPACE ks WITH REPLICATION = { 'class' : 'SimpleStrategy', 
> 'replication_factor' : 1 };
> cqlsh> CREATE TABLE ks.ta ( id text PRIMARY KEY, col text );
> cqlsh> INSERT INTO ks.ta (id, col) VALUES ('myId', 'myCol') IF NOT EXISTS;
>  [applied]
> ---
>   True
> cqlsh> DELETE FROM ks.ta WHERE id = 'myId';
> cqlsh> SELECT * FROM ks.ta WHERE id='myId';
>  id   | col
> --+---
>  myId | myCol
> {code}
>  * Only happens if the client is on a different host (works as expected on 
> the same host)
>  * Works as expected without IF NOT EXISTS
>  * A ~500 ms delay between INSERT and DELETE fixes the issue.
> Logs attached.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14281) Improve LatencyMetrics performance by reducing write path processing

[Chris Lohfink (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Lohfink updated CASSANDRA-14281:
--
Attachment: benchmark.html

> Improve LatencyMetrics performance by reducing write path processing
> 
>
> Key: CASSANDRA-14281
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14281
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Michael Burman
>Assignee: Michael Burman
>Priority: Major
> Attachments: bench.png, bench2.png, benchmark.html, benchmark2.png
>
>
> Currently for each write/read/rangequery/CAS touching the CFS we write a 
> latency metric which takes a lot of processing time (up to 66% of the total 
> processing time if the update was empty). 
> The way latencies are recorded is to use both a dropwizard "Timer" as well as 
> "Counter". Latter is used for totalLatency and the previous is decaying 
> metric for rates and certain percentile metrics. We then replicate all of 
> these CFS writes to the KeyspaceMetrics and globalWriteLatencies. 
> Instead of doing this on the write phase we should merge the metrics when 
> they're read. This is much less common occurrence and thus we save a lot of 
> CPU time in total. This also speeds up the write path.
> Currently, the DecayingEstimatedHistogramReservoir acquires a lock for each 
> update operation, which causes a contention if there are more than one thread 
> updating the histogram. This impacts scalability when using larger machines. 
> We should make it lock-free as much as possible and also avoid a single 
> CAS-update from blocking all the concurrent threads from making an update.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14281) Improve LatencyMetrics performance by reducing write path processing

[Chris Lohfink (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441870#comment-16441870
 ] 

Chris Lohfink commented on CASSANDRA-14281:
---

fwiw tried this again on a beefier linux box instead of my laptop and saw the 
improvement:

 !benchmark2.png|width=799,height=417! 

> Improve LatencyMetrics performance by reducing write path processing
> 
>
> Key: CASSANDRA-14281
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14281
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Michael Burman
>Assignee: Michael Burman
>Priority: Major
> Attachments: bench.png, bench2.png, benchmark2.png
>
>
> Currently for each write/read/rangequery/CAS touching the CFS we write a 
> latency metric which takes a lot of processing time (up to 66% of the total 
> processing time if the update was empty). 
> The way latencies are recorded is to use both a dropwizard "Timer" as well as 
> "Counter". Latter is used for totalLatency and the previous is decaying 
> metric for rates and certain percentile metrics. We then replicate all of 
> these CFS writes to the KeyspaceMetrics and globalWriteLatencies. 
> Instead of doing this on the write phase we should merge the metrics when 
> they're read. This is much less common occurrence and thus we save a lot of 
> CPU time in total. This also speeds up the write path.
> Currently, the DecayingEstimatedHistogramReservoir acquires a lock for each 
> update operation, which causes a contention if there are more than one thread 
> updating the histogram. This impacts scalability when using larger machines. 
> We should make it lock-free as much as possible and also avoid a single 
> CAS-update from blocking all the concurrent threads from making an update.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14281) Improve LatencyMetrics performance by reducing write path processing

[Chris Lohfink (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Lohfink updated CASSANDRA-14281:
--
Attachment: benchmark2.png

> Improve LatencyMetrics performance by reducing write path processing
> 
>
> Key: CASSANDRA-14281
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14281
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Michael Burman
>Assignee: Michael Burman
>Priority: Major
> Attachments: bench.png, bench2.png, benchmark2.png
>
>
> Currently for each write/read/rangequery/CAS touching the CFS we write a 
> latency metric which takes a lot of processing time (up to 66% of the total 
> processing time if the update was empty). 
> The way latencies are recorded is to use both a dropwizard "Timer" as well as 
> "Counter". Latter is used for totalLatency and the previous is decaying 
> metric for rates and certain percentile metrics. We then replicate all of 
> these CFS writes to the KeyspaceMetrics and globalWriteLatencies. 
> Instead of doing this on the write phase we should merge the metrics when 
> they're read. This is much less common occurrence and thus we save a lot of 
> CPU time in total. This also speeds up the write path.
> Currently, the DecayingEstimatedHistogramReservoir acquires a lock for each 
> update operation, which causes a contention if there are more than one thread 
> updating the histogram. This impacts scalability when using larger machines. 
> We should make it lock-free as much as possible and also avoid a single 
> CAS-update from blocking all the concurrent threads from making an update.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14396) Error about JNA on Startup

[Joe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joe updated CASSANDRA-14396:

Description: 
 

Hi, all.

I got some error on startup.

this is my own backup server which can't use network.

I just extracted 'apache-cassandra-3.11.2-bin.tar.gz' file to 
/usr/local/cassandra.

and then ran like this 'cassandra -f'.

but log displayed below's error.

 

I found some way to solve. but it's not working.

after JNA library download, make JNA library symbolic link - not working

can anyone advise to me about this issue?(I attached full logging file)

 

ERROR [main] 2018-04-18 10:44:59,536 NativeLibraryLinux.java:62 - Failed to 
link the C library against JNA. Native methods will be unavailable.
 java.lang.UnsatisfiedLinkError: /tmp/jna-3506402/jna5682737284440877593.tmp: 
/tmp/jna-3506402/jna5682737284440877593.tmp: failed to map segment from shared 
object: Operation not permitted
 at java.lang.ClassLoader$NativeLibrary.load(Native Method) ~[na:1.8.0_152]
 at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941) ~[na:1.8.0_152]
 at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824) ~[na:1.8.0_152]
 at java.lang.Runtime.load0(Runtime.java:809) ~[na:1.8.0_152]
 at java.lang.System.load(System.java:1086) ~[na:1.8.0_152]
 at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:851) 
~[jna-4.2.2.jar:4.2.2 (b0)]
 at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:826) 
~[jna-4.2.2.jar:4.2.2 (b0)]
 at com.sun.jna.Native.(Native.java:140) ~[jna-4.2.2.jar:4.2.2 (b0)]
 at 
org.apache.cassandra.utils.NativeLibraryLinux.(NativeLibraryLinux.java:53)
 ~[apache-cassandra-3.11.2.jar:3.11.2]
 at org.apache.cassandra.utils.NativeLibrary.(NativeLibrary.java:93) 
[apache-cassandra-3.11.2.jar:3.11.2]
 at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:196) 
[apache-cassandra-3.11.2.jar:3.11.2]
 at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:602) 
[apache-cassandra-3.11.2.jar:3.11.2]
 at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:691) 
[apache-cassandra-3.11.2.jar:3.11.2]
 WARN [main] 2018-04-18 10:44:59,537 StartupChecks.java:136 - jemalloc shared 
library could not be preloaded to speed up memory allocations
 WARN [main] 2018-04-18 10:44:59,537 StartupChecks.java:169 - JMX is not 
enabled to receive remote connections. Please see cassandra-env.sh for more 
info.
 ERROR [main] 2018-04-18 10:44:59,539 CassandraDaemon.java:708 - The native 
library could not be initialized properly.

  was:
 

Hi, all.

I got some error on startup.

this is my own backup server which can't use network.

I just extracted 'apache-cassandra-3.11.2-bin.tar.gz' file to 
/usr/local/cassandra.

and then ran like this 'cassandra -f'.

but log displayed below's error.

 

I found some way to solve. but it's not working.

after JNA library download, make JNA library symbolic link - not working

can anyone advise to me about this issue?

 

ERROR [main] 2018-04-18 10:44:59,536 NativeLibraryLinux.java:62 - Failed to 
link the C library against JNA. Native methods will be unavailable.
java.lang.UnsatisfiedLinkError: /tmp/jna-3506402/jna5682737284440877593.tmp: 
/tmp/jna-3506402/jna5682737284440877593.tmp: failed to map segment from shared 
object: Operation not permitted
 at java.lang.ClassLoader$NativeLibrary.load(Native Method) ~[na:1.8.0_152]
 at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941) ~[na:1.8.0_152]
 at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824) ~[na:1.8.0_152]
 at java.lang.Runtime.load0(Runtime.java:809) ~[na:1.8.0_152]
 at java.lang.System.load(System.java:1086) ~[na:1.8.0_152]
 at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:851) 
~[jna-4.2.2.jar:4.2.2 (b0)]
 at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:826) 
~[jna-4.2.2.jar:4.2.2 (b0)]
 at com.sun.jna.Native.(Native.java:140) ~[jna-4.2.2.jar:4.2.2 (b0)]
 at 
org.apache.cassandra.utils.NativeLibraryLinux.(NativeLibraryLinux.java:53)
 ~[apache-cassandra-3.11.2.jar:3.11.2]
 at org.apache.cassandra.utils.NativeLibrary.(NativeLibrary.java:93) 
[apache-cassandra-3.11.2.jar:3.11.2]
 at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:196) 
[apache-cassandra-3.11.2.jar:3.11.2]
 at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:602) 
[apache-cassandra-3.11.2.jar:3.11.2]
 at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:691) 
[apache-cassandra-3.11.2.jar:3.11.2]
WARN [main] 2018-04-18 10:44:59,537 StartupChecks.java:136 - jemalloc shared 
library could not be preloaded to speed up memory allocations
WARN [main] 2018-04-18 10:44:59,537 StartupChecks.java:169 - JMX is not enabled 
to receive remote connections. Please see cassandra-env.sh for more info.
ERROR [main] 2018-04-18 10:44:59,539 CassandraDaemon.java:708 - The native 

[jira] [Updated] (CASSANDRA-14396) Error about JNA on Startup

[Joe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joe updated CASSANDRA-14396:

Attachment: cassandra.log

> Error about JNA on Startup 
> ---
>
> Key: CASSANDRA-14396
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14396
> Project: Cassandra
>  Issue Type: Test
>  Components: Build, Configuration, Core, Lifecycle
> Environment:  
>  * CentOS release 6.8
>  * Cassandra 3.11.2
>  * Java 1.8.0_152
>  * No provide network
>  
>Reporter: Joe
>Priority: Major
> Fix For: 3.11.2
>
> Attachments: cassandra.log
>
>
>  
> Hi, all.
> I got some error on startup.
> this is my own backup server which can't use network.
> I just extracted 'apache-cassandra-3.11.2-bin.tar.gz' file to 
> /usr/local/cassandra.
> and then ran like this 'cassandra -f'.
> but log displayed below's error.
>  
> I found some way to solve. but it's not working.
> after JNA library download, make JNA library symbolic link - not working
> can anyone advise to me about this issue?
>  
> ERROR [main] 2018-04-18 10:44:59,536 NativeLibraryLinux.java:62 - Failed to 
> link the C library against JNA. Native methods will be unavailable.
> java.lang.UnsatisfiedLinkError: /tmp/jna-3506402/jna5682737284440877593.tmp: 
> /tmp/jna-3506402/jna5682737284440877593.tmp: failed to map segment from 
> shared object: Operation not permitted
>  at java.lang.ClassLoader$NativeLibrary.load(Native Method) ~[na:1.8.0_152]
>  at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941) ~[na:1.8.0_152]
>  at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824) ~[na:1.8.0_152]
>  at java.lang.Runtime.load0(Runtime.java:809) ~[na:1.8.0_152]
>  at java.lang.System.load(System.java:1086) ~[na:1.8.0_152]
>  at 
> com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:851) 
> ~[jna-4.2.2.jar:4.2.2 (b0)]
>  at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:826) 
> ~[jna-4.2.2.jar:4.2.2 (b0)]
>  at com.sun.jna.Native.(Native.java:140) ~[jna-4.2.2.jar:4.2.2 (b0)]
>  at 
> org.apache.cassandra.utils.NativeLibraryLinux.(NativeLibraryLinux.java:53)
>  ~[apache-cassandra-3.11.2.jar:3.11.2]
>  at org.apache.cassandra.utils.NativeLibrary.(NativeLibrary.java:93) 
> [apache-cassandra-3.11.2.jar:3.11.2]
>  at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:196) 
> [apache-cassandra-3.11.2.jar:3.11.2]
>  at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:602)
>  [apache-cassandra-3.11.2.jar:3.11.2]
>  at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:691) 
> [apache-cassandra-3.11.2.jar:3.11.2]
> WARN [main] 2018-04-18 10:44:59,537 StartupChecks.java:136 - jemalloc shared 
> library could not be preloaded to speed up memory allocations
> WARN [main] 2018-04-18 10:44:59,537 StartupChecks.java:169 - JMX is not 
> enabled to receive remote connections. Please see cassandra-env.sh for more 
> info.
> ERROR [main] 2018-04-18 10:44:59,539 CassandraDaemon.java:708 - The native 
> library could not be initialized properly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14396) Error about JNA on Startup

[Joe (JIRA)]

Joe created CASSANDRA-14396:
---

 Summary: Error about JNA on Startup 
 Key: CASSANDRA-14396
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14396
 Project: Cassandra
  Issue Type: Test
  Components: Build, Configuration, Core, Lifecycle
 Environment:  
 * CentOS release 6.8
 * Cassandra 3.11.2
 * Java 1.8.0_152
 * No provide network

 
Reporter: Joe
 Fix For: 3.11.2


 

Hi, all.

I got some error on startup.

this is my own backup server which can't use network.

I just extracted 'apache-cassandra-3.11.2-bin.tar.gz' file to 
/usr/local/cassandra.

and then ran like this 'cassandra -f'.

but log displayed below's error.

 

I found some way to solve. but it's not working.

after JNA library download, make JNA library symbolic link - not working

can anyone advise to me about this issue?

 

ERROR [main] 2018-04-18 10:44:59,536 NativeLibraryLinux.java:62 - Failed to 
link the C library against JNA. Native methods will be unavailable.
java.lang.UnsatisfiedLinkError: /tmp/jna-3506402/jna5682737284440877593.tmp: 
/tmp/jna-3506402/jna5682737284440877593.tmp: failed to map segment from shared 
object: Operation not permitted
 at java.lang.ClassLoader$NativeLibrary.load(Native Method) ~[na:1.8.0_152]
 at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941) ~[na:1.8.0_152]
 at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824) ~[na:1.8.0_152]
 at java.lang.Runtime.load0(Runtime.java:809) ~[na:1.8.0_152]
 at java.lang.System.load(System.java:1086) ~[na:1.8.0_152]
 at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:851) 
~[jna-4.2.2.jar:4.2.2 (b0)]
 at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:826) 
~[jna-4.2.2.jar:4.2.2 (b0)]
 at com.sun.jna.Native.(Native.java:140) ~[jna-4.2.2.jar:4.2.2 (b0)]
 at 
org.apache.cassandra.utils.NativeLibraryLinux.(NativeLibraryLinux.java:53)
 ~[apache-cassandra-3.11.2.jar:3.11.2]
 at org.apache.cassandra.utils.NativeLibrary.(NativeLibrary.java:93) 
[apache-cassandra-3.11.2.jar:3.11.2]
 at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:196) 
[apache-cassandra-3.11.2.jar:3.11.2]
 at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:602) 
[apache-cassandra-3.11.2.jar:3.11.2]
 at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:691) 
[apache-cassandra-3.11.2.jar:3.11.2]
WARN [main] 2018-04-18 10:44:59,537 StartupChecks.java:136 - jemalloc shared 
library could not be preloaded to speed up memory allocations
WARN [main] 2018-04-18 10:44:59,537 StartupChecks.java:169 - JMX is not enabled 
to receive remote connections. Please see cassandra-env.sh for more info.
ERROR [main] 2018-04-18 10:44:59,539 CassandraDaemon.java:708 - The native 
library could not be initialized properly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14298) cqlshlib tests broken on b.a.o

[Patrick Bannister (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440276#comment-16440276
 ] 

Patrick Bannister edited comment on CASSANDRA-14298 at 4/18/18 3:05 AM:


All tests in cqlsh_tests/cqlsh_tests.py pass on trunk, cassandra-3.11, and 
cassandra-3.0 on Ubuntu 16.04 LTS.


was (Author: ptbannister):
All tests in cqlsh_tests/cqlsh_tests.py pass on trunk on Ubuntu 16.04 LTS 
(except one test that is skipped by pytest - probably version related).

> cqlshlib tests broken on b.a.o
> --
>
> Key: CASSANDRA-14298
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14298
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Patrick Bannister
>Priority: Major
> Attachments: cqlsh_tests_notes.md
>
>
> It appears that cqlsh-tests on builds.apache.org on all branches stopped 
> working since we removed nosetests from the system environment. See e.g. 
> [here|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-cqlsh-tests/458/cython=no,jdk=JDK%201.8%20(latest),label=cassandra/console].
>  Looks like we either have to make nosetests available again or migrate to 
> pytest as we did with dtests. Giving pytest a quick try resulted in many 
> errors locally, but I haven't inspected them in detail yet. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Issue Comment Deleted] (CASSANDRA-14298) cqlshlib tests broken on b.a.o

[Patrick Bannister (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Bannister updated CASSANDRA-14298:
--
Comment: was deleted

(was: I just updated cqlsh_tests_nodes.md - my initial analysis of a ccm usage 
problem was wrong.)

> cqlshlib tests broken on b.a.o
> --
>
> Key: CASSANDRA-14298
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14298
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Patrick Bannister
>Priority: Major
> Attachments: cqlsh_tests_notes.md
>
>
> It appears that cqlsh-tests on builds.apache.org on all branches stopped 
> working since we removed nosetests from the system environment. See e.g. 
> [here|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-cqlsh-tests/458/cython=no,jdk=JDK%201.8%20(latest),label=cassandra/console].
>  Looks like we either have to make nosetests available again or migrate to 
> pytest as we did with dtests. Giving pytest a quick try resulted in many 
> errors locally, but I haven't inspected them in detail yet. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14394) Streaming fails with AssertionError

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441707#comment-16441707
 ] 

Paulo Motta edited comment on CASSANDRA-14394 at 4/18/18 2:58 AM:
--

{quote}Ah, looks like 13938. Now I feel less bad about breaking something in 
what was meant to be a minor refactor. Encourage either of you to commit the 
dtest with/without proving it's a dupe.
{quote}
-The funny thing is that during bissect I noticed that this starts failing 
consistently after CASSANDRA-14260, but after investigating I haven't found 
anything there that broken this, so I think this is some timing/synchronization 
bug that became more likely after 14260. o-

edit: I confirmed this reproduces consistently after CASSANDRA-12229 (and not 
CASSANDRA-14260), I probably screwed something up in the previous bisect.
{quote}I think it might be a dupe of CASSANDRA-13938, introduced by 
CASANDRA-12229
{quote}
Good call, I did a quick search on google of " stream can only read forward" 
and found only CASSANDRA-8314 so I thought it was a new issue, next time I will 
use the JIRA search bar. :) I will close this as dupe of CASSANDRA-13938 for 
now. Feel free to reopen if this is not the case and to include the dtest on 
that

Thanks for the quick feedback!


was (Author: pauloricardomg):
bq. Ah, looks like 13938. Now I feel less bad about breaking something in what 
was meant to be a minor refactor. Encourage either of you to commit the dtest 
with/without proving it's a dupe. 

The funny thing is that during bissect I noticed that this starts failing 
consistently after CASSANDRA-14260, but after investigating I haven't found 
anything there that broken this, so I think this is some timing/synchronization 
bug that became more likely after 14260.

bq. I think it might be a dupe of CASSANDRA-13938, introduced by CASANDRA-12229 

Good call, I did a quick search on google of " stream can only read forward" 
and found only CASSANDRA-8314 so I thought it was a new issue, next time I will 
use the JIRA search bar. :) I will close this as dupe of CASSANDRA-13938 for 
now. Feel free to reopen if this is not the case and to include the dtest on 
that

Thanks for the quick feedback!

> Streaming fails with AssertionError
> ---
>
> Key: CASSANDRA-14394
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14394
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Paulo Motta
>Priority: Major
> Fix For: 4.x
>
>
> The bootstrap from the attached dtest fails with the following error on trunk:
> {code:none}
> ERROR [Stream-Deserializer-127.0.0.1:38369-af884494] 2018-04-17 12:36:53,420 
> StreamingInboundHandler.java:210 - [Stream channel: af884494] stream 
> operation from 127.0.0.1:38369 failedjava.lang.AssertionError: stream can 
> only read forward.    at 
> org.apache.cassandra.db.streaming.CompressedInputStream.position(CompressedInputStream.java:108)
>     at 
> org.apache.cassandra.db.streaming.CompressedCassandraStreamReader.read(CompressedCassandraStreamReader.java:93)
>     at 
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:63)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
>     at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
>     at 
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:177)
>     at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13971) Automatic certificate management using Vault

[Nate McCall (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441812#comment-16441812
 ] 

Nate McCall commented on CASSANDRA-13971:
-

[~jasobrown] MPLv2.0 is on the Category-B list as you pointed out, so we are 
good to go there. [~spo...@gmail.com] Just make sure you put a reference to the 
version and license following the pattern in 
[https://github.com/apache/cassandra/tree/trunk/lib/licenses|https://github.com/apache/cassandra/tree/trunk/lib/licenses.]
 

> Automatic certificate management using Vault
> 
>
> Key: CASSANDRA-13971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13971
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>  Labels: security
> Fix For: 4.x
>
>
> We've been adding security features during the last years to enable users to 
> secure their clusters, if they are willing to use them and do so correctly. 
> Some features are powerful and easy to work with, such as role based 
> authorization. Other features that require to manage a local keystore are 
> rather painful to deal with. Think about setting up SSL..
> To be fair, keystore related issues and certificate handling hasn't been 
> invented by us. We're just following Java standards there. But that doesn't 
> mean that we absolutely have to, if there are better options. I'd like to 
> give it a shoot and find out if we can automate certificate/key handling 
> (PKI) by using external APIs. In this case, the implementation will be based 
> on [Vault|https://vaultproject.io]. But certificate management services 
> offered by cloud providers may also be able to handle the use-case and I 
> intend to create a generic, pluggable API for that.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-13971) Automatic certificate management using Vault

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441788#comment-16441788
 ] 

Jason Brown edited comment on CASSANDRA-13971 at 4/18/18 2:32 AM:
--

Getting to it this week. Can you just check with ASF legal about the MPL? It 
should be legit, but let's be sure.

Also, looks like Vault 0.10.0 has been released. Can you upgrade to that? Also, 
let me know if there's any significant changes because of that.


was (Author: jasobrown):
Getting to it this week. Can you just check with ASF legal about the MPL? It 
should be legit, but let's be sure.

> Automatic certificate management using Vault
> 
>
> Key: CASSANDRA-13971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13971
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>  Labels: security
> Fix For: 4.x
>
>
> We've been adding security features during the last years to enable users to 
> secure their clusters, if they are willing to use them and do so correctly. 
> Some features are powerful and easy to work with, such as role based 
> authorization. Other features that require to manage a local keystore are 
> rather painful to deal with. Think about setting up SSL..
> To be fair, keystore related issues and certificate handling hasn't been 
> invented by us. We're just following Java standards there. But that doesn't 
> mean that we absolutely have to, if there are better options. I'd like to 
> give it a shoot and find out if we can automate certificate/key handling 
> (PKI) by using external APIs. In this case, the implementation will be based 
> on [Vault|https://vaultproject.io]. But certificate management services 
> offered by cloud providers may also be able to handle the use-case and I 
> intend to create a generic, pluggable API for that.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13971) Automatic certificate management using Vault

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441788#comment-16441788
 ] 

Jason Brown commented on CASSANDRA-13971:
-

Getting to it this week. Can you just check with ASF legal about the MPL? It 
should be legit, but let's be sure.

> Automatic certificate management using Vault
> 
>
> Key: CASSANDRA-13971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13971
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>  Labels: security
> Fix For: 4.x
>
>
> We've been adding security features during the last years to enable users to 
> secure their clusters, if they are willing to use them and do so correctly. 
> Some features are powerful and easy to work with, such as role based 
> authorization. Other features that require to manage a local keystore are 
> rather painful to deal with. Think about setting up SSL..
> To be fair, keystore related issues and certificate handling hasn't been 
> invented by us. We're just following Java standards there. But that doesn't 
> mean that we absolutely have to, if there are better options. I'd like to 
> give it a shoot and find out if we can automate certificate/key handling 
> (PKI) by using external APIs. In this case, the implementation will be based 
> on [Vault|https://vaultproject.io]. But certificate management services 
> offered by cloud providers may also be able to handle the use-case and I 
> intend to create a generic, pluggable API for that.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14394) Streaming fails with AssertionError

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441765#comment-16441765
 ] 

Jason Brown commented on CASSANDRA-14394:
-

[~pauloricardomg] cool, thanks for the research. I may end up stealing your 
dtest if it's handy :)

> Streaming fails with AssertionError
> ---
>
> Key: CASSANDRA-14394
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14394
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Paulo Motta
>Priority: Major
> Fix For: 4.x
>
>
> The bootstrap from the attached dtest fails with the following error on trunk:
> {code:none}
> ERROR [Stream-Deserializer-127.0.0.1:38369-af884494] 2018-04-17 12:36:53,420 
> StreamingInboundHandler.java:210 - [Stream channel: af884494] stream 
> operation from 127.0.0.1:38369 failedjava.lang.AssertionError: stream can 
> only read forward.    at 
> org.apache.cassandra.db.streaming.CompressedInputStream.position(CompressedInputStream.java:108)
>     at 
> org.apache.cassandra.db.streaming.CompressedCassandraStreamReader.read(CompressedCassandraStreamReader.java:93)
>     at 
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:63)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
>     at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
>     at 
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:177)
>     at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13047) Point cqlsh help to the new doc

[Patrick Bannister (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441758#comment-16441758
 ] 

Patrick Bannister commented on CASSANDRA-13047:
---

Concur with recommendation by [~spo...@gmail.com] that cqlsh should link to 
versioned documentation for the corresponding CQL spec.

> Point cqlsh help to the new doc
> ---
>
> Key: CASSANDRA-13047
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13047
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Sylvain Lebresne
>Assignee: Patrick Bannister
>Priority: Major
> Fix For: 4.0
>
> Attachments: 13047-3.11.txt
>
>
> Cqlsh still points to the "old" textile CQL doc, but that's not really 
> maintain anymore now that we have the new doc (which include everything the 
> old doc had and more). We should update cqlsh to point to the new doc so we 
> can remove the old one completely.
> Any takers?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13047) Point cqlsh help to the new doc

[Patrick Bannister (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441752#comment-16441752
 ] 

Patrick Bannister commented on CASSANDRA-13047:
---

I would be happy to revise my work after 13907, I think it would provide much 
better documentation!

> Point cqlsh help to the new doc
> ---
>
> Key: CASSANDRA-13047
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13047
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Sylvain Lebresne
>Assignee: Patrick Bannister
>Priority: Major
> Fix For: 4.0
>
> Attachments: 13047-3.11.txt
>
>
> Cqlsh still points to the "old" textile CQL doc, but that's not really 
> maintain anymore now that we have the new doc (which include everything the 
> old doc had and more). We should update cqlsh to point to the new doc so we 
> can remove the old one completely.
> Any takers?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14394) Streaming fails with AssertionError

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441707#comment-16441707
 ] 

Paulo Motta edited comment on CASSANDRA-14394 at 4/18/18 12:40 AM:
---

bq. Ah, looks like 13938. Now I feel less bad about breaking something in what 
was meant to be a minor refactor. Encourage either of you to commit the dtest 
with/without proving it's a dupe. 

The funny thing is that during bissect I noticed that this starts failing 
consistently after CASSANDRA-14260, but after investigating I haven't found 
anything there that broken this, so I think this is some timing/synchronization 
bug that became more likely after 14260.

bq. I think it might be a dupe of CASSANDRA-13938, introduced by CASANDRA-12229 

Good call, I did a quick search on google of " stream can only read forward" 
and found only CASSANDRA-8314 so I thought it was a new issue, next time I will 
use the JIRA search bar. :) I will close this as dupe of CASSANDRA-13938 for 
now. Feel free to reopen if this is not the case and to include the dtest on 
that

Thanks for the quick feedback!


was (Author: pauloricardomg):
bq. Ah, looks like 13938. Now I feel less bad about breaking something in what 
was meant to be a minor refactor. Encourage either of you to commit the dtest 
with/without proving it's a dupe. 

The funny thing is that during bissect I noticed that this starts failing more 
often after CASSANDRA-14260, but after investigating I haven't found anything 
there that broken this, so I think this is some timing/synchronization bug 
which became more likely after 14260.

bq. I think it might be a dupe of CASSANDRA-13938, introduced by CASANDRA-12229 

Good call, I did a quick search on google of " stream can only read forward" 
and found only CASSANDRA-8314 so I thought it was a new issue, next time I will 
use the JIRA search bar. :) I will close this as dupe of CASSANDRA-13938 for 
now. Feel free to reopen if this is not the case and to include the dtest on 
that

Thanks for the quick feedback!

> Streaming fails with AssertionError
> ---
>
> Key: CASSANDRA-14394
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14394
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Paulo Motta
>Priority: Major
> Fix For: 4.x
>
>
> The bootstrap from the attached dtest fails with the following error on trunk:
> {code:none}
> ERROR [Stream-Deserializer-127.0.0.1:38369-af884494] 2018-04-17 12:36:53,420 
> StreamingInboundHandler.java:210 - [Stream channel: af884494] stream 
> operation from 127.0.0.1:38369 failedjava.lang.AssertionError: stream can 
> only read forward.    at 
> org.apache.cassandra.db.streaming.CompressedInputStream.position(CompressedInputStream.java:108)
>     at 
> org.apache.cassandra.db.streaming.CompressedCassandraStreamReader.read(CompressedCassandraStreamReader.java:93)
>     at 
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:63)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
>     at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
>     at 
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:177)
>     at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14394) Streaming fails with AssertionError

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441707#comment-16441707
 ] 

Paulo Motta edited comment on CASSANDRA-14394 at 4/18/18 12:37 AM:
---

bq. Ah, looks like 13938. Now I feel less bad about breaking something in what 
was meant to be a minor refactor. Encourage either of you to commit the dtest 
with/without proving it's a dupe. 

The funny thing is that during bissect I noticed that this starts failing more 
often after CASSANDRA-14260, but after investigating I haven't found anything 
there that broken this, so I think this is some timing/synchronization bug 
which became more likely after 14260.

bq. I think it might be a dupe of CASSANDRA-13938, introduced by CASANDRA-12229 

Good call, I did a quick search on google of " stream can only read forward" 
and found only CASSANDRA-8314 so I thought it was a new issue, next time I will 
use the JIRA search bar. :) I will close this as dupe of CASSANDRA-13938 for 
now. Feel free to reopen if this is not the case and to include the dtest on 
that

Thanks for the quick feedback!


was (Author: pauloricardomg):
bq. Ah, looks like 13938. Now I feel less bad about breaking something in what 
was meant to be a minor 
refactor. Encourage either of you to commit the dtest with/without proving it's 
a dupe. 

The funny thing is that during bissect I noticed that this starts failing more 
often after CASSANDRA-14260, but after investigating I haven't found anything 
there that broken this, so I think this is some timing/synchronization bug 
which became more likely after 14260.

bq. I think it might be a dupe of CASSANDRA-13938, introduced by CASANDRA-12229 

Good call, I did a quick search on google of " stream can only read forward" 
and found only CASSANDRA-8314 so I thought it was a new issue, next time I will 
use the JIRA search bar. :) I will close this as dupe of CASSANDRA-13938 for 
now. Feel free to reopen if this is not the case and to include the dtest on 
that

Thanks for the quick feedback!

> Streaming fails with AssertionError
> ---
>
> Key: CASSANDRA-14394
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14394
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Paulo Motta
>Priority: Major
> Fix For: 4.x
>
>
> The bootstrap from the attached dtest fails with the following error on trunk:
> {code:none}
> ERROR [Stream-Deserializer-127.0.0.1:38369-af884494] 2018-04-17 12:36:53,420 
> StreamingInboundHandler.java:210 - [Stream channel: af884494] stream 
> operation from 127.0.0.1:38369 failedjava.lang.AssertionError: stream can 
> only read forward.    at 
> org.apache.cassandra.db.streaming.CompressedInputStream.position(CompressedInputStream.java:108)
>     at 
> org.apache.cassandra.db.streaming.CompressedCassandraStreamReader.read(CompressedCassandraStreamReader.java:93)
>     at 
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:63)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
>     at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
>     at 
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:177)
>     at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Resolved] (CASSANDRA-14394) Streaming fails with AssertionError

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta resolved CASSANDRA-14394.
-
Resolution: Duplicate

bq. Ah, looks like 13938. Now I feel less bad about breaking something in what 
was meant to be a minor 
refactor. Encourage either of you to commit the dtest with/without proving it's 
a dupe. 

The funny thing is that during bissect I noticed that this starts failing more 
often after CASSANDRA-14260, but after investigating I haven't found anything 
there that broken this, so I think this is some timing/synchronization bug 
which became more likely after 14260.

bq. I think it might be a dupe of CASSANDRA-13938, introduced by CASANDRA-12229 

Good call, I did a quick search on google of " stream can only read forward" 
and found only CASSANDRA-8314 so I thought it was a new issue, next time I will 
use the JIRA search bar. :) I will close this as dupe of CASSANDRA-13938 for 
now. Feel free to reopen if this is not the case and to include the dtest on 
that

Thanks for the quick feedback!

> Streaming fails with AssertionError
> ---
>
> Key: CASSANDRA-14394
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14394
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Paulo Motta
>Priority: Major
> Fix For: 4.x
>
>
> The bootstrap from the attached dtest fails with the following error on trunk:
> {code:none}
> ERROR [Stream-Deserializer-127.0.0.1:38369-af884494] 2018-04-17 12:36:53,420 
> StreamingInboundHandler.java:210 - [Stream channel: af884494] stream 
> operation from 127.0.0.1:38369 failedjava.lang.AssertionError: stream can 
> only read forward.    at 
> org.apache.cassandra.db.streaming.CompressedInputStream.position(CompressedInputStream.java:108)
>     at 
> org.apache.cassandra.db.streaming.CompressedCassandraStreamReader.read(CompressedCassandraStreamReader.java:93)
>     at 
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:63)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
>     at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
>     at 
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:177)
>     at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13851) Allow existing nodes to use all peers in shadow round

[Kurt Greaves (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441687#comment-16441687
 ] 

Kurt Greaves commented on CASSANDRA-13851:
--

Thanks heaps [~beobal]!

> Allow existing nodes to use all peers in shadow round
> -
>
> Key: CASSANDRA-13851
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13851
> Project: Cassandra
>  Issue Type: Bug
>  Components: Lifecycle
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Major
> Fix For: 4.0, 3.11.3
>
>
> In CASSANDRA-10134 we made collision checks necessary on every startup. A 
> side-effect was introduced that then requires a nodes seeds to be contacted 
> on every startup. Prior to this change an existing node could start up 
> regardless whether it could contact a seed node or not (because 
> checkForEndpointCollision() was only called for bootstrapping nodes). 
> Now if a nodes seeds are removed/deleted/fail it will no longer be able to 
> start up until live seeds are configured (or itself is made a seed), even 
> though it already knows about the rest of the ring. This is inconvenient for 
> operators and has the potential to cause some nasty surprises and increase 
> downtime.
> One solution would be to use all a nodes existing peers as seeds in the 
> shadow round. Not a Gossip guru though so not sure of implications.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14395) C* Management process

[Dinesh Joshi (JIRA)]

Dinesh Joshi created CASSANDRA-14395:


 Summary: C* Management process
 Key: CASSANDRA-14395
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14395
 Project: Cassandra
  Issue Type: New Feature
Reporter: Dinesh Joshi
Assignee: Dinesh Joshi


I would like to propose amending Cassandra's architecture to include a 
management process. The detailed description is here: 
https://docs.google.com/document/d/1UV9pE81NaIUF3g4L1wxq09nT11AkSQcMijgLFwGsY3s/edit

I'd like to propose seeding this with a few simple use-cases such as Health 
Checks, Bulk Commands with a simple REST API interface. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14394) Streaming fails with AssertionError

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441561#comment-16441561
 ] 

Jason Brown edited comment on CASSANDRA-14394 at 4/17/18 9:59 PM:
--

I can't get to fixing this issue this week, but I can take a at the dtest and 
compare it with the instructions [~zznate] added on CASSANDRA-13938 - to 
confirm if the tickets are indeed the same problem


was (Author: jasobrown):
I can't get to fixing this issue this week, but I can take a at the dtest and 
compare it with the instructions [~zznate] added on CASSANDRA-13938.

> Streaming fails with AssertionError
> ---
>
> Key: CASSANDRA-14394
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14394
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Paulo Motta
>Priority: Major
> Fix For: 4.x
>
>
> The bootstrap from the attached dtest fails with the following error on trunk:
> {code:none}
> ERROR [Stream-Deserializer-127.0.0.1:38369-af884494] 2018-04-17 12:36:53,420 
> StreamingInboundHandler.java:210 - [Stream channel: af884494] stream 
> operation from 127.0.0.1:38369 failedjava.lang.AssertionError: stream can 
> only read forward.    at 
> org.apache.cassandra.db.streaming.CompressedInputStream.position(CompressedInputStream.java:108)
>     at 
> org.apache.cassandra.db.streaming.CompressedCassandraStreamReader.read(CompressedCassandraStreamReader.java:93)
>     at 
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:63)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
>     at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
>     at 
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:177)
>     at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14394) Streaming fails with AssertionError

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441561#comment-16441561
 ] 

Jason Brown commented on CASSANDRA-14394:
-

I can't get to fixing this issue this week, but I can take a at the dtest and 
compare it with the instructions [~zznate] added on CASSANDRA-13938.

> Streaming fails with AssertionError
> ---
>
> Key: CASSANDRA-14394
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14394
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Paulo Motta
>Priority: Major
> Fix For: 4.x
>
>
> The bootstrap from the attached dtest fails with the following error on trunk:
> {code:none}
> ERROR [Stream-Deserializer-127.0.0.1:38369-af884494] 2018-04-17 12:36:53,420 
> StreamingInboundHandler.java:210 - [Stream channel: af884494] stream 
> operation from 127.0.0.1:38369 failedjava.lang.AssertionError: stream can 
> only read forward.    at 
> org.apache.cassandra.db.streaming.CompressedInputStream.position(CompressedInputStream.java:108)
>     at 
> org.apache.cassandra.db.streaming.CompressedCassandraStreamReader.read(CompressedCassandraStreamReader.java:93)
>     at 
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:63)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
>     at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
>     at 
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:177)
>     at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14394) Streaming fails with AssertionError

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-14394:
---
Fix Version/s: 4.x

> Streaming fails with AssertionError
> ---
>
> Key: CASSANDRA-14394
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14394
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Paulo Motta
>Priority: Major
> Fix For: 4.x
>
>
> The bootstrap from the attached dtest fails with the following error on trunk:
> {code:none}
> ERROR [Stream-Deserializer-127.0.0.1:38369-af884494] 2018-04-17 12:36:53,420 
> StreamingInboundHandler.java:210 - [Stream channel: af884494] stream 
> operation from 127.0.0.1:38369 failedjava.lang.AssertionError: stream can 
> only read forward.    at 
> org.apache.cassandra.db.streaming.CompressedInputStream.position(CompressedInputStream.java:108)
>     at 
> org.apache.cassandra.db.streaming.CompressedCassandraStreamReader.read(CompressedCassandraStreamReader.java:93)
>     at 
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:63)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
>     at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
>     at 
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:177)
>     at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13896) Improving Cassandra write performance

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441532#comment-16441532
 ] 

Ariel Weisberg commented on CASSANDRA-13896:


The ticket was filed against 3.10 so I wonder if there is a performance 
difference here. Also are we comparing a cluster to a single node to determine 
scaling issues or Michael did you also test with a single node?

> Improving Cassandra write performance  
> ---
>
> Key: CASSANDRA-13896
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13896
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
> Environment: Skylake server with 2 sockets, 192GB RAM, 3xPCIe SSDs
> OS: Centos 7.3 
> Java: Oracle JDK1.8.0_121
>Reporter: Prince Nana Owusu Boateng
>Priority: Major
>  Labels: Performance
> Fix For: 4.x
>
> Attachments: Screen Shot 2017-09-22 at 11.22.43 AM.png, Screen Shot 
> 2017-09-22 at 3.31.09 PM.png
>
>
> During our Cassandra performance testing, we see high percentage of the CPU 
> spent in *org.apache.cassandra.utils.memory.SlabAllocator.allocate(int, 
> OpOrder Group) * method.  Appears to be high contention of the 
> ** atomic Integer in write workloads.   This structure is 
> used by the threads for keeping track of the region bytebuffer allocation.  
> When the contention appears, adding more clients, modifications of write 
> specific parameters does not change write throughput performance.  Attached 
> are the details of Java Flight Recorder (JFR), showing hot functions and also 
> performance results.   When we see this contention, we still have plenty of 
> CPU and throughput left ( *<20%*  Total average CPU utilization and  *<11%* 
> of the storage write total throughput).   This occurs on Cassandra 3.10.0-src 
> version using the Cassandra-Stress.
> Proposal:
> We will like to introduce a solution which eliminates the atomic operations 
> on the ** atomic Integer. This implementation will allow 
> concurrent allocation of bytebuffers without an atomic compareAndSet and 
> incrementAndGet operations. The solution is expected to increase overall 
> write performance while improving CPU utilization.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14394) Streaming fails with AssertionError

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-14394:
---
Component/s: Streaming and Messaging

> Streaming fails with AssertionError
> ---
>
> Key: CASSANDRA-14394
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14394
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Paulo Motta
>Priority: Major
> Fix For: 4.x
>
>
> The bootstrap from the attached dtest fails with the following error on trunk:
> {code:none}
> ERROR [Stream-Deserializer-127.0.0.1:38369-af884494] 2018-04-17 12:36:53,420 
> StreamingInboundHandler.java:210 - [Stream channel: af884494] stream 
> operation from 127.0.0.1:38369 failedjava.lang.AssertionError: stream can 
> only read forward.    at 
> org.apache.cassandra.db.streaming.CompressedInputStream.position(CompressedInputStream.java:108)
>     at 
> org.apache.cassandra.db.streaming.CompressedCassandraStreamReader.read(CompressedCassandraStreamReader.java:93)
>     at 
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:63)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
>     at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
>     at 
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:177)
>     at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-14394) Streaming fails with AssertionError

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa reassigned CASSANDRA-14394:
--

Assignee: (was: Jeff Jirsa)

> Streaming fails with AssertionError
> ---
>
> Key: CASSANDRA-14394
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14394
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Paulo Motta
>Priority: Major
>
> The bootstrap from the attached dtest fails with the following error on trunk:
> {code:none}
> ERROR [Stream-Deserializer-127.0.0.1:38369-af884494] 2018-04-17 12:36:53,420 
> StreamingInboundHandler.java:210 - [Stream channel: af884494] stream 
> operation from 127.0.0.1:38369 failedjava.lang.AssertionError: stream can 
> only read forward.    at 
> org.apache.cassandra.db.streaming.CompressedInputStream.position(CompressedInputStream.java:108)
>     at 
> org.apache.cassandra.db.streaming.CompressedCassandraStreamReader.read(CompressedCassandraStreamReader.java:93)
>     at 
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:63)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
>     at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
>     at 
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:177)
>     at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14394) Streaming fails with AssertionError

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441533#comment-16441533
 ] 

Jeff Jirsa commented on CASSANDRA-14394:


Ah, looks like 13938. Now I feel less bad about breaking something in what was 
meant to be a minor refactor. Encourage either of you to commit the dtest 
with/without proving it's a dupe.


> Streaming fails with AssertionError
> ---
>
> Key: CASSANDRA-14394
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14394
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Paulo Motta
>Assignee: Jeff Jirsa
>Priority: Major
>
> The bootstrap from the attached dtest fails with the following error on trunk:
> {code:none}
> ERROR [Stream-Deserializer-127.0.0.1:38369-af884494] 2018-04-17 12:36:53,420 
> StreamingInboundHandler.java:210 - [Stream channel: af884494] stream 
> operation from 127.0.0.1:38369 failedjava.lang.AssertionError: stream can 
> only read forward.    at 
> org.apache.cassandra.db.streaming.CompressedInputStream.position(CompressedInputStream.java:108)
>     at 
> org.apache.cassandra.db.streaming.CompressedCassandraStreamReader.read(CompressedCassandraStreamReader.java:93)
>     at 
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:63)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
>     at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
>     at 
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:177)
>     at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14394) Streaming fails with AssertionError

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441525#comment-16441525
 ] 

Jason Brown commented on CASSANDRA-14394:
-

I think it might be a dupe of CASSANDRA-13938, introduced by CASANDRA-12229

> Streaming fails with AssertionError
> ---
>
> Key: CASSANDRA-14394
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14394
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Paulo Motta
>Assignee: Jeff Jirsa
>Priority: Major
>
> The bootstrap from the attached dtest fails with the following error on trunk:
> {code:none}
> ERROR [Stream-Deserializer-127.0.0.1:38369-af884494] 2018-04-17 12:36:53,420 
> StreamingInboundHandler.java:210 - [Stream channel: af884494] stream 
> operation from 127.0.0.1:38369 failedjava.lang.AssertionError: stream can 
> only read forward.    at 
> org.apache.cassandra.db.streaming.CompressedInputStream.position(CompressedInputStream.java:108)
>     at 
> org.apache.cassandra.db.streaming.CompressedCassandraStreamReader.read(CompressedCassandraStreamReader.java:93)
>     at 
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:63)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
>     at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
>     at 
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:177)
>     at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14332) Fix unbounded validation compactions on repair

[Blake Eggleston (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-14332:

   Resolution: Fixed
Fix Version/s: 3.11.3
   3.0.17
   Status: Resolved  (was: Patch Available)

committed as {{00e5a3d508eb41944ce01c6cc96ae18cb16dad8c}} thanks!

> Fix unbounded validation compactions on repair
> --
>
> Key: CASSANDRA-14332
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14332
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Major
> Fix For: 3.0.17, 3.11.3
>
>
> After CASSANDRA-13797 it's possible to cause unbounded, simultaneous 
> validation compactions as we no longer wait for validations to finish. 
> Potential fix is to have a sane default for the # of concurrent validation 
> compactions by backporting CASSANDRA-13521 and setting a sane default.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[bdeggleston]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/703db6ff
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/703db6ff
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/703db6ff

Branch: refs/heads/trunk
Commit: 703db6fff5fdf30f98e61d175a2b268846f7b692
Parents: f165e72 e9462b9
Author: Blake Eggleston 
Authored: Tue Apr 17 14:18:59 2018 -0700
Committer: Blake Eggleston 
Committed: Tue Apr 17 14:19:29 2018 -0700

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/703db6ff/CHANGES.txt
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[1/6] cassandra git commit: Fix unbounded validation compactions on repair / revert CASSANDRA-13797

[bdeggleston]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 598008d43 -> 00e5a3d50
  refs/heads/cassandra-3.11 28ccf3fe3 -> e9462b9af
  refs/heads/trunk f165e72bf -> 703db6fff


Fix unbounded validation compactions on repair / revert CASSANDRA-13797

This reverts commit e7299c08f940057e8fd4dfa3f24dcc6e0cb5f78d.
Patch by Kurt Greaves; Reviewed by Blake Eggleston for CASSANDRA-14332


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/00e5a3d5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/00e5a3d5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/00e5a3d5

Branch: refs/heads/cassandra-3.0
Commit: 00e5a3d508eb41944ce01c6cc96ae18cb16dad8c
Parents: 598008d
Author: kurt 
Authored: Tue Feb 27 05:24:25 2018 +
Committer: Blake Eggleston 
Committed: Tue Apr 17 14:04:09 2018 -0700

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/repair/RepairJob.java | 3 +++
 2 files changed, 4 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/00e5a3d5/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d3d8036..1aa291f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.17
+ * Fix unbounded validation compactions on repair / revert CASSANDRA-13797 
(CASSANDRA-14332)
  * Avoid deadlock when running nodetool refresh before node is fully up 
(CASSANDRA-14310)
  * Handle all exceptions when opening sstables (CASSANDRA-14202)
  * Handle incompletely written hint descriptors during startup 
(CASSANDRA-14080)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/00e5a3d5/src/java/org/apache/cassandra/repair/RepairJob.java
--
diff --git a/src/java/org/apache/cassandra/repair/RepairJob.java 
b/src/java/org/apache/cassandra/repair/RepairJob.java
index 0711a64..cba176c 100644
--- a/src/java/org/apache/cassandra/repair/RepairJob.java
+++ b/src/java/org/apache/cassandra/repair/RepairJob.java
@@ -155,6 +155,9 @@ public class RepairJob extends AbstractFuture 
implements Runnable
 setException(t);
 }
 }, taskExecutor);
+
+// Wait for validation to complete
+Futures.getUnchecked(validations);
 }
 
 /**


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[3/6] cassandra git commit: Fix unbounded validation compactions on repair / revert CASSANDRA-13797

[bdeggleston]

Fix unbounded validation compactions on repair / revert CASSANDRA-13797

This reverts commit e7299c08f940057e8fd4dfa3f24dcc6e0cb5f78d.
Patch by Kurt Greaves; Reviewed by Blake Eggleston for CASSANDRA-14332


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/00e5a3d5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/00e5a3d5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/00e5a3d5

Branch: refs/heads/trunk
Commit: 00e5a3d508eb41944ce01c6cc96ae18cb16dad8c
Parents: 598008d
Author: kurt 
Authored: Tue Feb 27 05:24:25 2018 +
Committer: Blake Eggleston 
Committed: Tue Apr 17 14:04:09 2018 -0700

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/repair/RepairJob.java | 3 +++
 2 files changed, 4 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/00e5a3d5/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d3d8036..1aa291f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.17
+ * Fix unbounded validation compactions on repair / revert CASSANDRA-13797 
(CASSANDRA-14332)
  * Avoid deadlock when running nodetool refresh before node is fully up 
(CASSANDRA-14310)
  * Handle all exceptions when opening sstables (CASSANDRA-14202)
  * Handle incompletely written hint descriptors during startup 
(CASSANDRA-14080)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/00e5a3d5/src/java/org/apache/cassandra/repair/RepairJob.java
--
diff --git a/src/java/org/apache/cassandra/repair/RepairJob.java 
b/src/java/org/apache/cassandra/repair/RepairJob.java
index 0711a64..cba176c 100644
--- a/src/java/org/apache/cassandra/repair/RepairJob.java
+++ b/src/java/org/apache/cassandra/repair/RepairJob.java
@@ -155,6 +155,9 @@ public class RepairJob extends AbstractFuture 
implements Runnable
 setException(t);
 }
 }, taskExecutor);
+
+// Wait for validation to complete
+Futures.getUnchecked(validations);
 }
 
 /**


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[bdeggleston]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e9462b9a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e9462b9a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e9462b9a

Branch: refs/heads/trunk
Commit: e9462b9af7c85bc5e1de6bc1662768da71c1061b
Parents: 28ccf3f 00e5a3d
Author: Blake Eggleston 
Authored: Tue Apr 17 14:15:12 2018 -0700
Committer: Blake Eggleston 
Committed: Tue Apr 17 14:15:12 2018 -0700

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/repair/RepairJob.java | 3 +++
 2 files changed, 4 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e9462b9a/CHANGES.txt
--
diff --cc CHANGES.txt
index 39213a1,1aa291f..7253ec6
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,15 -1,5 +1,16 @@@
 -3.0.17
 +3.11.3
 + * Allow existing nodes to use all peers in shadow round (CASSANDRA-13851)
 + * Fix cqlsh to read connection.ssl cqlshrc option again (CASSANDRA-14299)
 + * Downgrade log level to trace for CommitLogSegmentManager (CASSANDRA-14370)
 + * CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)
 + * Serialize empty buffer as empty string for json output format 
(CASSANDRA-14245)
 + * Allow logging implementation to be interchanged for embedded testing 
(CASSANDRA-13396)
 + * SASI tokenizer for simple delimiter based entries (CASSANDRA-14247)
 + * Fix Loss of digits when doing CAST from varint/bigint to decimal 
(CASSANDRA-14170)
 + * RateBasedBackPressure unnecessarily invokes a lock on the Guava 
RateLimiter (CASSANDRA-14163)
 + * Fix wildcard GROUP BY queries (CASSANDRA-14209)
 +Merged from 3.0:
+  * Fix unbounded validation compactions on repair / revert CASSANDRA-13797 
(CASSANDRA-14332)
   * Avoid deadlock when running nodetool refresh before node is fully up 
(CASSANDRA-14310)
   * Handle all exceptions when opening sstables (CASSANDRA-14202)
   * Handle incompletely written hint descriptors during startup 
(CASSANDRA-14080)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e9462b9a/src/java/org/apache/cassandra/repair/RepairJob.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[bdeggleston]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e9462b9a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e9462b9a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e9462b9a

Branch: refs/heads/cassandra-3.11
Commit: e9462b9af7c85bc5e1de6bc1662768da71c1061b
Parents: 28ccf3f 00e5a3d
Author: Blake Eggleston 
Authored: Tue Apr 17 14:15:12 2018 -0700
Committer: Blake Eggleston 
Committed: Tue Apr 17 14:15:12 2018 -0700

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/repair/RepairJob.java | 3 +++
 2 files changed, 4 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e9462b9a/CHANGES.txt
--
diff --cc CHANGES.txt
index 39213a1,1aa291f..7253ec6
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,15 -1,5 +1,16 @@@
 -3.0.17
 +3.11.3
 + * Allow existing nodes to use all peers in shadow round (CASSANDRA-13851)
 + * Fix cqlsh to read connection.ssl cqlshrc option again (CASSANDRA-14299)
 + * Downgrade log level to trace for CommitLogSegmentManager (CASSANDRA-14370)
 + * CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)
 + * Serialize empty buffer as empty string for json output format 
(CASSANDRA-14245)
 + * Allow logging implementation to be interchanged for embedded testing 
(CASSANDRA-13396)
 + * SASI tokenizer for simple delimiter based entries (CASSANDRA-14247)
 + * Fix Loss of digits when doing CAST from varint/bigint to decimal 
(CASSANDRA-14170)
 + * RateBasedBackPressure unnecessarily invokes a lock on the Guava 
RateLimiter (CASSANDRA-14163)
 + * Fix wildcard GROUP BY queries (CASSANDRA-14209)
 +Merged from 3.0:
+  * Fix unbounded validation compactions on repair / revert CASSANDRA-13797 
(CASSANDRA-14332)
   * Avoid deadlock when running nodetool refresh before node is fully up 
(CASSANDRA-14310)
   * Handle all exceptions when opening sstables (CASSANDRA-14202)
   * Handle incompletely written hint descriptors during startup 
(CASSANDRA-14080)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e9462b9a/src/java/org/apache/cassandra/repair/RepairJob.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[2/6] cassandra git commit: Fix unbounded validation compactions on repair / revert CASSANDRA-13797

[bdeggleston]

Fix unbounded validation compactions on repair / revert CASSANDRA-13797

This reverts commit e7299c08f940057e8fd4dfa3f24dcc6e0cb5f78d.
Patch by Kurt Greaves; Reviewed by Blake Eggleston for CASSANDRA-14332


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/00e5a3d5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/00e5a3d5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/00e5a3d5

Branch: refs/heads/cassandra-3.11
Commit: 00e5a3d508eb41944ce01c6cc96ae18cb16dad8c
Parents: 598008d
Author: kurt 
Authored: Tue Feb 27 05:24:25 2018 +
Committer: Blake Eggleston 
Committed: Tue Apr 17 14:04:09 2018 -0700

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/repair/RepairJob.java | 3 +++
 2 files changed, 4 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/00e5a3d5/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d3d8036..1aa291f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.17
+ * Fix unbounded validation compactions on repair / revert CASSANDRA-13797 
(CASSANDRA-14332)
  * Avoid deadlock when running nodetool refresh before node is fully up 
(CASSANDRA-14310)
  * Handle all exceptions when opening sstables (CASSANDRA-14202)
  * Handle incompletely written hint descriptors during startup 
(CASSANDRA-14080)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/00e5a3d5/src/java/org/apache/cassandra/repair/RepairJob.java
--
diff --git a/src/java/org/apache/cassandra/repair/RepairJob.java 
b/src/java/org/apache/cassandra/repair/RepairJob.java
index 0711a64..cba176c 100644
--- a/src/java/org/apache/cassandra/repair/RepairJob.java
+++ b/src/java/org/apache/cassandra/repair/RepairJob.java
@@ -155,6 +155,9 @@ public class RepairJob extends AbstractFuture 
implements Runnable
 setException(t);
 }
 }, taskExecutor);
+
+// Wait for validation to complete
+Futures.getUnchecked(validations);
 }
 
 /**


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-14394) Streaming fails with AssertionError

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa reassigned CASSANDRA-14394:
--

Assignee: Jeff Jirsa

> Streaming fails with AssertionError
> ---
>
> Key: CASSANDRA-14394
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14394
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Paulo Motta
>Assignee: Jeff Jirsa
>Priority: Major
>
> The bootstrap from the attached dtest fails with the following error on trunk:
> {code:none}
> ERROR [Stream-Deserializer-127.0.0.1:38369-af884494] 2018-04-17 12:36:53,420 
> StreamingInboundHandler.java:210 - [Stream channel: af884494] stream 
> operation from 127.0.0.1:38369 failedjava.lang.AssertionError: stream can 
> only read forward.    at 
> org.apache.cassandra.db.streaming.CompressedInputStream.position(CompressedInputStream.java:108)
>     at 
> org.apache.cassandra.db.streaming.CompressedCassandraStreamReader.read(CompressedCassandraStreamReader.java:93)
>     at 
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:63)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
>     at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
>     at 
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:177)
>     at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14394) Streaming fails with AssertionError

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441506#comment-16441506
 ] 

Jeff Jirsa commented on CASSANDRA-14394:


Assuming without evidence that this is probably a bug from CASSANDRA-14260 - 
I'll check it (though I'm really surprised we didn't have another test that 
caught it)



> Streaming fails with AssertionError
> ---
>
> Key: CASSANDRA-14394
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14394
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Paulo Motta
>Priority: Major
>
> The bootstrap from the attached dtest fails with the following error on trunk:
> {code:none}
> ERROR [Stream-Deserializer-127.0.0.1:38369-af884494] 2018-04-17 12:36:53,420 
> StreamingInboundHandler.java:210 - [Stream channel: af884494] stream 
> operation from 127.0.0.1:38369 failedjava.lang.AssertionError: stream can 
> only read forward.    at 
> org.apache.cassandra.db.streaming.CompressedInputStream.position(CompressedInputStream.java:108)
>     at 
> org.apache.cassandra.db.streaming.CompressedCassandraStreamReader.read(CompressedCassandraStreamReader.java:93)
>     at 
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:63)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
>     at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
>     at 
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:177)
>     at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13910) Remove read_repair_chance/dclocal_read_repair_chance

[Blake Eggleston (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441403#comment-16441403
 ] 

Blake Eggleston commented on CASSANDRA-13910:
-

+1, although please remove the unused ClientWarn imports  in 
AlterTableStatement, AlterViewStatement, and CreateTableStatement on commit

> Remove read_repair_chance/dclocal_read_repair_chance
> 
>
> Key: CASSANDRA-13910
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13910
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 4.0
>
>
> First, let me clarify so this is not misunderstood that I'm not *at all* 
> suggesting to remove the read-repair mechanism of detecting and repairing 
> inconsistencies between read responses: that mechanism is imo fine and 
> useful.  But the {{read_repair_chance}} and {{dclocal_read_repair_chance}} 
> have never been about _enabling_ that mechanism, they are about querying all 
> replicas (even when this is not required by the consistency level) for the 
> sole purpose of maybe read-repairing some of the replica that wouldn't have 
> been queried otherwise. Which btw, bring me to reason 1 for considering their 
> removal: their naming/behavior is super confusing. Over the years, I've seen 
> countless users (and not only newbies) misunderstanding what those options 
> do, and as a consequence misunderstand when read-repair itself was happening.
> But my 2nd reason for suggesting this is that I suspect 
> {{read_repair_chance}}/{{dclocal_read_repair_chance}} are, especially 
> nowadays, more harmful than anything else when enabled. When those option 
> kick in, what you trade-off is additional resources consumption (all nodes 
> have to execute the read) for a _fairly remote chance_ of having some 
> inconsistencies repaired on _some_ replica _a bit faster_ than they would 
> otherwise be. To justify that last part, let's recall that:
> # most inconsistencies are actually fixed by hints in practice; and in the 
> case where a node stay dead for a long time so that hints ends up timing-out, 
> you really should repair the node when it comes back (if not simply 
> re-bootstrapping it).  Read-repair probably don't fix _that_ much stuff in 
> the first place.
> # again, read-repair do happen without those options kicking in. If you do 
> reads at {{QUORUM}}, inconsistencies will eventually get read-repaired all 
> the same.  Just a tiny bit less quickly.
> # I suspect almost everyone use a low "chance" for those options at best 
> (because the extra resources consumption is real), so at the end of the day, 
> it's up to chance how much faster this fixes inconsistencies.
> Overall, I'm having a hard time imagining real cases where that trade-off 
> really make sense. Don't get me wrong, those options had their places a long 
> time ago when hints weren't working all that well, but I think they bring 
> more confusion than benefits now.
> And I think it's sane to reconsider stuffs every once in a while, and to 
> clean up anything that may not make all that much sense anymore, which I 
> think is the case here.
> Tl;dr, I feel the benefits brought by those options are very slim at best and 
> well overshadowed by the confusion they bring, and not worth maintaining the 
> code that supports them (which, to be fair, isn't huge, but getting rid of 
> {{ReadCallback.AsyncRepairRunner}} wouldn't hurt for instance).
> Lastly, if the consensus here ends up being that they can have their use in 
> weird case and that we fill supporting those cases is worth confusing 
> everyone else and maintaining that code, I would still suggest disabling them 
> totally by default.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13910) Remove read_repair_chance/dclocal_read_repair_chance

[Blake Eggleston (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-13910:

Status: Ready to Commit  (was: Patch Available)

> Remove read_repair_chance/dclocal_read_repair_chance
> 
>
> Key: CASSANDRA-13910
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13910
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 4.0
>
>
> First, let me clarify so this is not misunderstood that I'm not *at all* 
> suggesting to remove the read-repair mechanism of detecting and repairing 
> inconsistencies between read responses: that mechanism is imo fine and 
> useful.  But the {{read_repair_chance}} and {{dclocal_read_repair_chance}} 
> have never been about _enabling_ that mechanism, they are about querying all 
> replicas (even when this is not required by the consistency level) for the 
> sole purpose of maybe read-repairing some of the replica that wouldn't have 
> been queried otherwise. Which btw, bring me to reason 1 for considering their 
> removal: their naming/behavior is super confusing. Over the years, I've seen 
> countless users (and not only newbies) misunderstanding what those options 
> do, and as a consequence misunderstand when read-repair itself was happening.
> But my 2nd reason for suggesting this is that I suspect 
> {{read_repair_chance}}/{{dclocal_read_repair_chance}} are, especially 
> nowadays, more harmful than anything else when enabled. When those option 
> kick in, what you trade-off is additional resources consumption (all nodes 
> have to execute the read) for a _fairly remote chance_ of having some 
> inconsistencies repaired on _some_ replica _a bit faster_ than they would 
> otherwise be. To justify that last part, let's recall that:
> # most inconsistencies are actually fixed by hints in practice; and in the 
> case where a node stay dead for a long time so that hints ends up timing-out, 
> you really should repair the node when it comes back (if not simply 
> re-bootstrapping it).  Read-repair probably don't fix _that_ much stuff in 
> the first place.
> # again, read-repair do happen without those options kicking in. If you do 
> reads at {{QUORUM}}, inconsistencies will eventually get read-repaired all 
> the same.  Just a tiny bit less quickly.
> # I suspect almost everyone use a low "chance" for those options at best 
> (because the extra resources consumption is real), so at the end of the day, 
> it's up to chance how much faster this fixes inconsistencies.
> Overall, I'm having a hard time imagining real cases where that trade-off 
> really make sense. Don't get me wrong, those options had their places a long 
> time ago when hints weren't working all that well, but I think they bring 
> more confusion than benefits now.
> And I think it's sane to reconsider stuffs every once in a while, and to 
> clean up anything that may not make all that much sense anymore, which I 
> think is the case here.
> Tl;dr, I feel the benefits brought by those options are very slim at best and 
> well overshadowed by the confusion they bring, and not worth maintaining the 
> code that supports them (which, to be fair, isn't huge, but getting rid of 
> {{ReadCallback.AsyncRepairRunner}} wouldn't hurt for instance).
> Lastly, if the consensus here ends up being that they can have their use in 
> weird case and that we fill supporting those cases is worth confusing 
> everyone else and maintaining that code, I would still suggest disabling them 
> totally by default.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14394) Streaming fails with AssertionError

[Paulo Motta (JIRA)]

Paulo Motta created CASSANDRA-14394:
---

 Summary: Streaming fails with AssertionError
 Key: CASSANDRA-14394
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14394
 Project: Cassandra
  Issue Type: Bug
Reporter: Paulo Motta


The bootstrap from the attached dtest fails with the following error on trunk:
{noformat}
ERROR [Stream-Deserializer-127.0.0.1:38369-af884494] 2018-04-17 12:36:53,420 
StreamingInboundHandler.java:210 - [Stream channel: af884494] stream operation 
from 127.0.0.1:38369 failedjava.lang.AssertionError: stream can only read 
forward.    at 
org.apache.cassandra.db.streaming.CompressedInputStream.position(CompressedInputStream.java:108)
    at 
org.apache.cassandra.db.streaming.CompressedCassandraStreamReader.read(CompressedCassandraStreamReader.java:93)
    at 
org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:63)
    at 
org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
    at 
org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
    at 
org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
    at 
org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:177)
    at java.lang.Thread.run(Thread.java:748)
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14394) Streaming fails with AssertionError

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-14394:

Description: 
The bootstrap from the attached dtest fails with the following error on trunk:
{code:none}
ERROR [Stream-Deserializer-127.0.0.1:38369-af884494] 2018-04-17 12:36:53,420 
StreamingInboundHandler.java:210 - [Stream channel: af884494] stream operation 
from 127.0.0.1:38369 failedjava.lang.AssertionError: stream can only read 
forward.    at 
org.apache.cassandra.db.streaming.CompressedInputStream.position(CompressedInputStream.java:108)
    at 
org.apache.cassandra.db.streaming.CompressedCassandraStreamReader.read(CompressedCassandraStreamReader.java:93)
    at 
org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:63)
    at 
org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
    at 
org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
    at 
org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
    at 
org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:177)
    at java.lang.Thread.run(Thread.java:748)
{code}

  was:
The bootstrap from the attached dtest fails with the following error on trunk:
{noformat}
ERROR [Stream-Deserializer-127.0.0.1:38369-af884494] 2018-04-17 12:36:53,420 
StreamingInboundHandler.java:210 - [Stream channel: af884494] stream operation 
from 127.0.0.1:38369 failedjava.lang.AssertionError: stream can only read 
forward.    at 
org.apache.cassandra.db.streaming.CompressedInputStream.position(CompressedInputStream.java:108)
    at 
org.apache.cassandra.db.streaming.CompressedCassandraStreamReader.read(CompressedCassandraStreamReader.java:93)
    at 
org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:63)
    at 
org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
    at 
org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
    at 
org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
    at 
org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:177)
    at java.lang.Thread.run(Thread.java:748)
{noformat}


> Streaming fails with AssertionError
> ---
>
> Key: CASSANDRA-14394
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14394
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Paulo Motta
>Priority: Major
>
> The bootstrap from the attached dtest fails with the following error on trunk:
> {code:none}
> ERROR [Stream-Deserializer-127.0.0.1:38369-af884494] 2018-04-17 12:36:53,420 
> StreamingInboundHandler.java:210 - [Stream channel: af884494] stream 
> operation from 127.0.0.1:38369 failedjava.lang.AssertionError: stream can 
> only read forward.    at 
> org.apache.cassandra.db.streaming.CompressedInputStream.position(CompressedInputStream.java:108)
>     at 
> org.apache.cassandra.db.streaming.CompressedCassandraStreamReader.read(CompressedCassandraStreamReader.java:93)
>     at 
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:63)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
>     at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
>     at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
>     at 
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:177)
>     at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14374) Speculative retry parsing breaks on non-english locale

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-14374:

Resolution: Fixed
Status: Resolved  (was: Ready to Commit)

Committed as {{f165e72bf19e8d12457b8f569517012628513d24}} to trunk. Thanks for 
the review!

> Speculative retry parsing breaks on non-english locale
> --
>
> Key: CASSANDRA-14374
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14374
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Paulo Motta
>Assignee: Paulo Motta
>Priority: Minor
> Fix For: 4.0
>
> Attachments: 
> 0001-Use-Locale.US-on-PercentileSpeculativeRetryPolicy.to.patch, 
> 0002-Use-Locale.US-on-PercentileSpeculativeRetryPolicy.to.patch, 
> 0003-Use-Locale.US-on-PercentileSpeculativeRetryPolicy.to.patch, 
> 0004-Use-Locale.US-on-PercentileSpeculativeRetryPolicy.to.patch
>
>
> I was getting the following error when running unit tests on my machine:
> {code:none}
> Error setting schema for test (query was: CREATE TABLE 
> cql_test_keyspace.table_32 (a int, b int, c text, primary key (a, b)))
> java.lang.RuntimeException: Error setting schema for test (query was: CREATE 
> TABLE cql_test_keyspace.table_32 (a int, b int, c text, primary key (a, b)))
>   at org.apache.cassandra.cql3.CQLTester.schemaChange(CQLTester.java:819)
>   at org.apache.cassandra.cql3.CQLTester.createTable(CQLTester.java:632)
>   at org.apache.cassandra.cql3.CQLTester.createTable(CQLTester.java:624)
>   at 
> org.apache.cassandra.cql3.validation.operations.DeleteTest.testDeleteWithNonoverlappingRange(DeleteTest.java:663)
> Caused by: org.apache.cassandra.exceptions.ConfigurationException: Specified 
> Speculative Retry Policy [99,00p] is not supported
>   at 
> org.apache.cassandra.service.reads.SpeculativeRetryPolicy.fromString(SpeculativeRetryPolicy.java:135)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.createTableParamsFromRow(SchemaKeyspace.java:1006)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:981)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:941)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:900)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspaces(SchemaKeyspace.java:1301)
>   at org.apache.cassandra.schema.Schema.merge(Schema.java:608)
>   at 
> org.apache.cassandra.schema.MigrationManager.announce(MigrationManager.java:425)
>   at 
> org.apache.cassandra.schema.MigrationManager.announceNewTable(MigrationManager.java:239)
>   at 
> org.apache.cassandra.schema.MigrationManager.announceNewTable(MigrationManager.java:224)
>   at 
> org.apache.cassandra.schema.MigrationManager.announceNewTable(MigrationManager.java:204)
>   at 
> org.apache.cassandra.cql3.statements.CreateTableStatement.announceMigration(CreateTableStatement.java:88)
>   at 
> org.apache.cassandra.cql3.statements.SchemaAlteringStatement.executeInternal(SchemaAlteringStatement.java:120)
>   at org.apache.cassandra.cql3.CQLTester.schemaChange(CQLTester.java:814)
> {code}
> It turns out that my machine is configured with {{pt_BR}} locale, which uses 
> comma instead of dot for decimal separator, so the speculative retry option 
> parsing introduced by CASSANDRA-14293, which assumed {{en_US}} locale was not 
> working.
> To reproduce on Linux:
> {code:none}
> export LC_CTYPE=pt_BR.UTF-8
> ant test -Dtest.name="DeleteTest"
> ant test -Dtest.name="SpeculativeRetryParseTest"
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13896) Improving Cassandra write performance

[Michael Burman (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441330#comment-16441330
 ] 

Michael Burman commented on CASSANDRA-13896:


I compared 4.0-trunk vs patched version.

> Improving Cassandra write performance  
> ---
>
> Key: CASSANDRA-13896
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13896
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
> Environment: Skylake server with 2 sockets, 192GB RAM, 3xPCIe SSDs
> OS: Centos 7.3 
> Java: Oracle JDK1.8.0_121
>Reporter: Prince Nana Owusu Boateng
>Priority: Major
>  Labels: Performance
> Fix For: 4.x
>
> Attachments: Screen Shot 2017-09-22 at 11.22.43 AM.png, Screen Shot 
> 2017-09-22 at 3.31.09 PM.png
>
>
> During our Cassandra performance testing, we see high percentage of the CPU 
> spent in *org.apache.cassandra.utils.memory.SlabAllocator.allocate(int, 
> OpOrder Group) * method.  Appears to be high contention of the 
> ** atomic Integer in write workloads.   This structure is 
> used by the threads for keeping track of the region bytebuffer allocation.  
> When the contention appears, adding more clients, modifications of write 
> specific parameters does not change write throughput performance.  Attached 
> are the details of Java Flight Recorder (JFR), showing hot functions and also 
> performance results.   When we see this contention, we still have plenty of 
> CPU and throughput left ( *<20%*  Total average CPU utilization and  *<11%* 
> of the storage write total throughput).   This occurs on Cassandra 3.10.0-src 
> version using the Cassandra-Stress.
> Proposal:
> We will like to introduce a solution which eliminates the atomic operations 
> on the ** atomic Integer. This implementation will allow 
> concurrent allocation of bytebuffers without an atomic compareAndSet and 
> incrementAndGet operations. The solution is expected to increase overall 
> write performance while improving CPU utilization.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra git commit: Fix speculative retry parsing on non-english default Locale

[paulo]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 88b01c866 -> f165e72bf


Fix speculative retry parsing on non-english default Locale

Patch by Paulo Motta; Reviewed by Aleksey Yeschenko for CASSANDRA-14374


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f165e72b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f165e72b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f165e72b

Branch: refs/heads/trunk
Commit: f165e72bf19e8d12457b8f569517012628513d24
Parents: 88b01c8
Author: Paulo Motta 
Authored: Tue Apr 17 15:47:29 2018 -0300
Committer: Paulo Motta 
Committed: Tue Apr 17 15:47:29 2018 -0300

--
 .../reads/PercentileSpeculativeRetryPolicy.java   |  7 ++-
 .../service/reads/SpeculativeRetryParseTest.java  | 14 --
 2 files changed, 18 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f165e72b/src/java/org/apache/cassandra/service/reads/PercentileSpeculativeRetryPolicy.java
--
diff --git 
a/src/java/org/apache/cassandra/service/reads/PercentileSpeculativeRetryPolicy.java
 
b/src/java/org/apache/cassandra/service/reads/PercentileSpeculativeRetryPolicy.java
index 9bf5d95..42f90fe 100644
--- 
a/src/java/org/apache/cassandra/service/reads/PercentileSpeculativeRetryPolicy.java
+++ 
b/src/java/org/apache/cassandra/service/reads/PercentileSpeculativeRetryPolicy.java
@@ -18,6 +18,8 @@
 package org.apache.cassandra.service.reads;
 
 import java.text.DecimalFormat;
+import java.text.DecimalFormatSymbols;
+import java.util.Locale;
 import java.util.regex.Matcher;
 import java.util.regex.Pattern;
 
@@ -32,7 +34,10 @@ public class PercentileSpeculativeRetryPolicy implements 
SpeculativeRetryPolicy
 public static final PercentileSpeculativeRetryPolicy NINETY_NINE_P = new 
PercentileSpeculativeRetryPolicy(99.0);
 
 private static final Pattern PATTERN = 
Pattern.compile("^(?[0-9.]+)p(ercentile)?$", Pattern.CASE_INSENSITIVE);
-private static final DecimalFormat FORMATTER = new DecimalFormat("#.");
+/**
+ * The pattern above uses dot as decimal separator, so we use {@link 
Locale#ENGLISH} to enforce that. (CASSANDRA-14374)
+ */
+private static final DecimalFormat FORMATTER = new DecimalFormat("#.", 
new DecimalFormatSymbols(Locale.ENGLISH));
 
 private final double percentile;
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f165e72b/test/unit/org/apache/cassandra/service/reads/SpeculativeRetryParseTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/service/reads/SpeculativeRetryParseTest.java 
b/test/unit/org/apache/cassandra/service/reads/SpeculativeRetryParseTest.java
index 8c742be..86b307e 100644
--- 
a/test/unit/org/apache/cassandra/service/reads/SpeculativeRetryParseTest.java
+++ 
b/test/unit/org/apache/cassandra/service/reads/SpeculativeRetryParseTest.java
@@ -20,6 +20,7 @@ package org.apache.cassandra.service.reads;
 
 import java.util.Arrays;
 import java.util.Collection;
+import java.util.Locale;
 
 import org.junit.Test;
 import org.junit.experimental.runners.Enclosed;
@@ -36,10 +37,11 @@ import static 
org.apache.cassandra.service.reads.HybridSpeculativeRetryPolicy.Fu
 @RunWith(Enclosed.class)
 public class SpeculativeRetryParseTest
 {
-
 @RunWith(Parameterized.class)
 public static class SuccessfulParseTest
 {
+static final Locale defaultLocale = Locale.getDefault();
+
 private final String string;
 private final SpeculativeRetryPolicy expectedValue;
 
@@ -104,9 +106,17 @@ public class SpeculativeRetryParseTest
 }
 
 @Test
-public void testToStringRoundTrip()
+public void testToStringRoundTripDefaultLocale()
+{
+assertEquals(expectedValue, 
SpeculativeRetryPolicy.fromString(expectedValue.toString()));
+}
+
+@Test
+public void testToStringRoundTripCommaDecimalSeparatorLocale()
 {
+Locale.setDefault(new Locale("pt","BR")); // CASSANDRA-14374: 
Brazil uses comma instead of dot as decimal separator
 assertEquals(expectedValue, 
SpeculativeRetryPolicy.fromString(expectedValue.toString()));
+Locale.setDefault(defaultLocale);
 }
 }
 


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13896) Improving Cassandra write performance

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441324#comment-16441324
 ] 

Ariel Weisberg commented on CASSANDRA-13896:


Did we compare different versions of Casasndra here? 3.10 vs ???

> Improving Cassandra write performance  
> ---
>
> Key: CASSANDRA-13896
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13896
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
> Environment: Skylake server with 2 sockets, 192GB RAM, 3xPCIe SSDs
> OS: Centos 7.3 
> Java: Oracle JDK1.8.0_121
>Reporter: Prince Nana Owusu Boateng
>Priority: Major
>  Labels: Performance
> Fix For: 4.x
>
> Attachments: Screen Shot 2017-09-22 at 11.22.43 AM.png, Screen Shot 
> 2017-09-22 at 3.31.09 PM.png
>
>
> During our Cassandra performance testing, we see high percentage of the CPU 
> spent in *org.apache.cassandra.utils.memory.SlabAllocator.allocate(int, 
> OpOrder Group) * method.  Appears to be high contention of the 
> ** atomic Integer in write workloads.   This structure is 
> used by the threads for keeping track of the region bytebuffer allocation.  
> When the contention appears, adding more clients, modifications of write 
> specific parameters does not change write throughput performance.  Attached 
> are the details of Java Flight Recorder (JFR), showing hot functions and also 
> performance results.   When we see this contention, we still have plenty of 
> CPU and throughput left ( *<20%*  Total average CPU utilization and  *<11%* 
> of the storage write total throughput).   This occurs on Cassandra 3.10.0-src 
> version using the Cassandra-Stress.
> Proposal:
> We will like to introduce a solution which eliminates the atomic operations 
> on the ** atomic Integer. This implementation will allow 
> concurrent allocation of bytebuffers without an atomic compareAndSet and 
> incrementAndGet operations. The solution is expected to increase overall 
> write performance while improving CPU utilization.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13896) Improving Cassandra write performance

[Michael Burman (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441314#comment-16441314
 ] 

Michael Burman commented on CASSANDRA-13896:


[~aweisberg] I think my patch was misunderstood. It's meant to rather prove 
that this isn't the contention point that Cassandra is hitting at this point. 
If it were, the patch would at least increase performance somewhat, since it 
reduces the contention on that single CAS. But there's no effect at all to 
performance. 

The most probable reason of scaling bottleneck is long before this place in the 
pipeline and it's most likely somewhere in our threading model executors, we 
can't process enough data from the network. There is a very large change in CPI 
(and reduce in front-end bound) when batching multiple data points. This 
reduces the hits to do_futex for example considerably and perf-map-agent shows 
reduced hits to our SharedExecutorPool's CSLM. We probably do either too much 
in the Netty pipeline before sending off the work or the pipeline for mutations 
is simply too long to be effective (instead of smaller parts where we could 
reuse the same thread and potentially same L1/L2/L3 cache hits for better 
performance). But I have no patch for that yet, profilers are not exactly 
brilliant in showing hotpoint when it's about "not doing anything". 

And you're right about contended annotation, I only tried several places to 
make sure no false sharing was responsible for no added performance. That place 
just happened to be in the pushed commit as I wasn't intending this to be the 
fix but instead to prove this isn't the correct scaling limitation - yet. 

> Improving Cassandra write performance  
> ---
>
> Key: CASSANDRA-13896
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13896
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
> Environment: Skylake server with 2 sockets, 192GB RAM, 3xPCIe SSDs
> OS: Centos 7.3 
> Java: Oracle JDK1.8.0_121
>Reporter: Prince Nana Owusu Boateng
>Priority: Major
>  Labels: Performance
> Fix For: 4.x
>
> Attachments: Screen Shot 2017-09-22 at 11.22.43 AM.png, Screen Shot 
> 2017-09-22 at 3.31.09 PM.png
>
>
> During our Cassandra performance testing, we see high percentage of the CPU 
> spent in *org.apache.cassandra.utils.memory.SlabAllocator.allocate(int, 
> OpOrder Group) * method.  Appears to be high contention of the 
> ** atomic Integer in write workloads.   This structure is 
> used by the threads for keeping track of the region bytebuffer allocation.  
> When the contention appears, adding more clients, modifications of write 
> specific parameters does not change write throughput performance.  Attached 
> are the details of Java Flight Recorder (JFR), showing hot functions and also 
> performance results.   When we see this contention, we still have plenty of 
> CPU and throughput left ( *<20%*  Total average CPU utilization and  *<11%* 
> of the storage write total throughput).   This occurs on Cassandra 3.10.0-src 
> version using the Cassandra-Stress.
> Proposal:
> We will like to introduce a solution which eliminates the atomic operations 
> on the ** atomic Integer. This implementation will allow 
> concurrent allocation of bytebuffers without an atomic compareAndSet and 
> incrementAndGet operations. The solution is expected to increase overall 
> write performance while improving CPU utilization.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14393) Incorrect view updates

[Duarte Nunes (JIRA)]

Duarte Nunes created CASSANDRA-14393:


 Summary: Incorrect view updates
 Key: CASSANDRA-14393
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14393
 Project: Cassandra
  Issue Type: Bug
  Components: Materialized Views
Reporter: Duarte Nunes


Consider the following:
{noformat}
create table t (p int, c int, v1 int, v2 int, primary key(p, c));
create materialized view mv as select p, c, v1 from t 
where p is not null and c is not null primary key (c, p);

insert into t (p, c, v1, v2) VALUES(1, 1, 1, 1) using ttl 5;
update t using ttl 1000 set v2 = 1 where p = 1 and c = 1;
delete v2 from t where p = 1 and c = 1;

// Wait 5 seconds
select * from mv;

c | p | v1
---+---+--
1 | 1 | null{noformat}
The view row should be dead after 5 seconds, but it is not.

This is because the liveness info calculated when deleting v2 is based on the 
base table update liveness info, which has the timestamp of the first insert 
statement. That liveness info is shadowed by the liveness info created in the 
update, which has a higher timestamp.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14381) nodetool listsnapshots is missing snapshots

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441224#comment-16441224
 ] 

Ariel Weisberg commented on CASSANDRA-14381:


The cicleci change is not part of the change it's just necessary to get 
CircleCI to run the dtests for me.

> nodetool listsnapshots is missing snapshots
> ---
>
> Key: CASSANDRA-14381
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14381
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: MacOs 10.12.5
> Java 1.8.0_144
> Cassandra 3.11.2 (brew install)
>Reporter: Cyril Scetbon
>Assignee: Ariel Weisberg
>Priority: Major
>
> The output of *nodetool listsnapshots* is inconsistent with the snapshots 
> created :
> {code:java}
> $ nodetool listsnapshots
> Snapshot Details:
> There are no snapshots
> $ nodetool snapshot -t tag1 --table local system
> Requested creating snapshot(s) for [system] with snapshot name [tag1] and 
> options {skipFlush=false}
> Snapshot directory: tag1
> $ nodetool snapshot -t tag2 --table local system
> Requested creating snapshot(s) for [system] with snapshot name [tag2] and 
> options {skipFlush=false}
> Snapshot directory: tag2
> $ nodetool listsnapshots
> Snapshot Details:
> There are no snapshots
> $ ls 
> /usr/local/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/snapshots/
> tag1 tag2{code}
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13896) Improving Cassandra write performance

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441223#comment-16441223
 ] 

Ariel Weisberg commented on CASSANDRA-13896:


I'm not sure what [~PrinceNana] was proposing to eliminate the contention 
entirely. Unless we have each thread allocate from its own memory region they 
will be shared and it will have to be an atomic operation.

If we could {{incrementAndGet}} instead of {{compareAndSet}} it might also be 
faster under contention.

I think splitting the contended resource is a classic solution to scaling 
contended access to shared state. The way Cassandra currently works with large 
numbers of threads it's the right kind of optimization. It's not totally crazy 
to optimize this based on microbenchmarks of the allocator rather then end to 
end benchmarks. Having headroom in the allocator is worth the risk of this 
particular change IMO, but we should know what we are getting. If we attempted 
to prove the correctness of the allocator we could be more confident about 
changing it. Like have multiple threads do a bunch of allocations and then 
compare the allocations across threads for correctness. They don't overlap, 
they are one after another, maybe other checks.

The contended annotation is not used correctly (I think) as we don't 
technically know where the contended atomic integer is stored and it would 
probably be stored away from the object just due to the annotation. The 
annotation only applies to the memory around the class or field being annotated 
(I think). We also have two contended fields, the allocation and the count. I 
think maybe count can just go away or alternatively be CASed into a single 
atomic long along with the offset. Assuming we can't manage to make that 
incrementAndGet.

Once we have consensus on this approach vs some other I can look closer at is 
this correct. And the answer for that is usually going to be well is there a 
test that convinces me it's correct?

> Improving Cassandra write performance  
> ---
>
> Key: CASSANDRA-13896
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13896
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
> Environment: Skylake server with 2 sockets, 192GB RAM, 3xPCIe SSDs
> OS: Centos 7.3 
> Java: Oracle JDK1.8.0_121
>Reporter: Prince Nana Owusu Boateng
>Priority: Major
>  Labels: Performance
> Fix For: 4.x
>
> Attachments: Screen Shot 2017-09-22 at 11.22.43 AM.png, Screen Shot 
> 2017-09-22 at 3.31.09 PM.png
>
>
> During our Cassandra performance testing, we see high percentage of the CPU 
> spent in *org.apache.cassandra.utils.memory.SlabAllocator.allocate(int, 
> OpOrder Group) * method.  Appears to be high contention of the 
> ** atomic Integer in write workloads.   This structure is 
> used by the threads for keeping track of the region bytebuffer allocation.  
> When the contention appears, adding more clients, modifications of write 
> specific parameters does not change write throughput performance.  Attached 
> are the details of Java Flight Recorder (JFR), showing hot functions and also 
> performance results.   When we see this contention, we still have plenty of 
> CPU and throughput left ( *<20%*  Total average CPU utilization and  *<11%* 
> of the storage write total throughput).   This occurs on Cassandra 3.10.0-src 
> version using the Cassandra-Stress.
> Proposal:
> We will like to introduce a solution which eliminates the atomic operations 
> on the ** atomic Integer. This implementation will allow 
> concurrent allocation of bytebuffers without an atomic compareAndSet and 
> incrementAndGet operations. The solution is expected to increase overall 
> write performance while improving CPU utilization.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra-dtest git commit: Add tests for running shadow round using seeds and/or peers

[samt]

Repository: cassandra-dtest
Updated Branches:
  refs/heads/master 6edfe7fb5 -> 2ee611a6d


Add tests for running shadow round using seeds and/or peers

Patch by Kurt Greaves; reveiewd by Sam Tunnicliffe for CASSANDRA-13851


Project: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/commit/2ee611a6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/tree/2ee611a6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/diff/2ee611a6

Branch: refs/heads/master
Commit: 2ee611a6d2bf5090d52856c7bc2368a2dc37f153
Parents: 6edfe7f
Author: kurt 
Authored: Wed Nov 1 09:57:10 2017 +
Committer: Sam Tunnicliffe 
Committed: Tue Apr 17 18:22:04 2018 +0100

--
 dtest.py |  13 +++
 seed_test.py | 109 ++
 2 files changed, 122 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra-dtest/blob/2ee611a6/dtest.py
--
diff --git a/dtest.py b/dtest.py
index 914f2f7..2aa30b8 100644
--- a/dtest.py
+++ b/dtest.py
@@ -276,6 +276,19 @@ class Tester:
 runner.start()
 return runner
 
+def assert_log_had_msg(self, node, msg, timeout=600, **kwargs):
+"""
+Wrapper for ccmlib.node.Node#watch_log_for to cause an assertion 
failure when a log message isn't found
+within the timeout.
+:param node: Node which logs we should watch
+:param msg: String message we expect to see in the logs.
+:param timeout: Seconds to wait for msg to appear
+"""
+try:
+node.watch_log_for(msg, timeout=timeout, **kwargs)
+except TimeoutError:
+pytest.fail("Log message was not seen within 
timeout:\n{0}".format(msg))
+
 def get_eager_protocol_version(cassandra_version):
 """
 Returns the highest protocol version accepted

http://git-wip-us.apache.org/repos/asf/cassandra-dtest/blob/2ee611a6/seed_test.py
--
diff --git a/seed_test.py b/seed_test.py
new file mode 100644
index 000..1f00d8b
--- /dev/null
+++ b/seed_test.py
@@ -0,0 +1,109 @@
+from ccmlib import node
+from dtest import Tester
+from time import sleep
+
+import pytest
+
+since = pytest.mark.since
+
+
+class TestGossiper(Tester):
+"""
+Test gossip states
+"""
+
+@since('3.11.2')
+def test_startup_no_live_seeds(self):
+"""
+Test that a node won't start with no live seeds.
+@jira_ticket CASSANDRA-13851
+"""
+
+self.fixture_dtest_setup.allow_log_errors = True
+n1 = self.cluster.create_node('node1', True, None, ('127.0.0.1', 
7000), '7100',
+  None, None, 
binary_interface=('127.0.0.1', 9042))
+self.cluster.add(n1, False)
+node1 = self.cluster.nodelist()[0]
+self.cluster.set_configuration_options({
+'seed_provider': [{'class_name': 
'org.apache.cassandra.locator.SimpleSeedProvider',
+   'parameters': [{'seeds': '127.0.0.2'}]  # dummy 
node doesn't exist
+   }]
+})
+
+try:
+STARTUP_TIMEOUT = 15  # seconds
+RING_DELAY = 1  # ms
+# set startup timeout > ring delay so that startup failure happens 
before the call to start returns
+node1.start(wait_for_binary_proto=STARTUP_TIMEOUT, 
jvm_args=['-Dcassandra.ring_delay_ms={}'.format(RING_DELAY)])
+except node.TimeoutError:
+self.assert_log_had_msg(node1, "Unable to gossip with any peers")
+except Exception as e:
+raise e
+else:
+pytest.fail("Expecting startup to raise a TimeoutError, but 
nothing was raised.")
+
+@since('3.11.2')
+def test_startup_non_seed_with_peers(self):
+"""
+Test that a node can start if peers are alive, or if a node has been 
bootstrapped
+but there are no live seeds or peers
+@jira_ticket CASSANDRA-13851
+"""
+
+self.fixture_dtest_setup.allow_log_errors = True
+
+n1 = self.cluster.create_node('node1', True, None, ('127.0.0.1', 
7000), '7100',
+  None, None, 
binary_interface=('127.0.0.1', 9042))
+n2 = self.cluster.create_node('node2', True, None, ('127.0.0.2', 
7000), '7101',
+  None, None, 
binary_interface=('127.0.0.2', 9042))
+n3 = self.cluster.create_node('node3', True, None, ('127.0.0.3', 
7000), '7102',
+  None, None, 
binary_interface=('127.0.0.3', 9042))
+self.cluster.add(n1, True)
+self.cluster.add(n2, True)
+

[jira] [Updated] (CASSANDRA-13851) Allow existing nodes to use all peers in shadow round

[Sam Tunnicliffe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-13851:

   Resolution: Fixed
Fix Version/s: (was: 3.11.x)
   (was: 4.x)
   3.11.3
   4.0
   Status: Resolved  (was: Patch Available)

Thanks [~KurtG], latest CI (after rebases) looks good committed so I've to 
cassandra-3.11 in {{28ccf3fe3989d9d80063fe4d4bb048efe471936b}} and merged to 
trunk. 
I'll get the dtest committed directly.

> Allow existing nodes to use all peers in shadow round
> -
>
> Key: CASSANDRA-13851
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13851
> Project: Cassandra
>  Issue Type: Bug
>  Components: Lifecycle
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Major
> Fix For: 4.0, 3.11.3
>
>
> In CASSANDRA-10134 we made collision checks necessary on every startup. A 
> side-effect was introduced that then requires a nodes seeds to be contacted 
> on every startup. Prior to this change an existing node could start up 
> regardless whether it could contact a seed node or not (because 
> checkForEndpointCollision() was only called for bootstrapping nodes). 
> Now if a nodes seeds are removed/deleted/fail it will no longer be able to 
> start up until live seeds are configured (or itself is made a seed), even 
> though it already knows about the rest of the ring. This is inconvenient for 
> operators and has the potential to cause some nasty surprises and increase 
> downtime.
> One solution would be to use all a nodes existing peers as seeds in the 
> shadow round. Not a Gossip guru though so not sure of implications.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[3/3] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[samt]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/88b01c86
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/88b01c86
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/88b01c86

Branch: refs/heads/trunk
Commit: 88b01c866d420c58e104640ddb78e3ebeef4d915
Parents: 70d9535 28ccf3f
Author: Sam Tunnicliffe 
Authored: Tue Apr 17 18:05:00 2018 +0100
Committer: Sam Tunnicliffe 
Committed: Tue Apr 17 18:13:58 2018 +0100

--
 CHANGES.txt |  3 ++
 src/java/org/apache/cassandra/gms/Gossiper.java | 38 +++-
 .../cassandra/service/StorageService.java   | 10 --
 3 files changed, 40 insertions(+), 11 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/88b01c86/CHANGES.txt
--
diff --cc CHANGES.txt
index 5e36674,39213a1..c9f6685
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,229 -1,8 +1,232 @@@
 +4.0
 + * Bind to correct local address in 4.0 streaming (CASSANDRA-14362)
 + * Use standard Amazon naming for datacenter and rack in Ec2Snitch 
(CASSANDRA-7839)
 + * Fix junit failure for SSTableReaderTest (CASSANDRA-14387)
 + * Abstract write path for pluggable storage (CASSANDRA-14118)
 + * nodetool describecluster should be more informative (CASSANDRA-13853)
 + * Compaction performance improvements (CASSANDRA-14261) 
 + * Refactor Pair usage to avoid boxing ints/longs (CASSANDRA-14260)
 + * Add options to nodetool tablestats to sort and limit output 
(CASSANDRA-13889)
 + * Rename internals to reflect CQL vocabulary (CASSANDRA-14354)
 + * Add support for hybrid MIN(), MAX() speculative retry policies
 +   (CASSANDRA-14293, CASSANDRA-14338, CASSANDRA-14352)
 + * Fix some regressions caused by 14058 (CASSANDRA-14353)
 + * Abstract repair for pluggable storage (CASSANDRA-14116)
 + * Add meaningful toString() impls (CASSANDRA-13653)
 + * Add sstableloader option to accept target keyspace name (CASSANDRA-13884)
 + * Move processing of EchoMessage response to gossip stage (CASSANDRA-13713)
 + * Add coordinator write metric per CF (CASSANDRA-14232)
 + * Correct and clarify SSLFactory.getSslContext method and call sites 
(CASSANDRA-14314)
 + * Handle static and partition deletion properly on 
ThrottledUnfilteredIterator (CASSANDRA-14315)
 + * NodeTool clientstats should show SSL Cipher (CASSANDRA-14322)
 + * Add ability to specify driver name and version (CASSANDRA-14275)
 + * Abstract streaming for pluggable storage (CASSANDRA-14115)
 + * Forced incremental repairs should promote sstables if they can 
(CASSANDRA-14294)
 + * Use Murmur3 for validation compactions (CASSANDRA-14002)
 + * Comma at the end of the seed list is interpretated as localhost 
(CASSANDRA-14285)
 + * Refactor read executor and response resolver, abstract read repair 
(CASSANDRA-14058)
 + * Add optional startup delay to wait until peers are ready (CASSANDRA-13993)
 + * Add a few options to nodetool verify (CASSANDRA-14201)
 + * CVE-2017-5929 Security vulnerability and redefine default log rotation 
policy (CASSANDRA-14183)
 + * Use JVM default SSL validation algorithm instead of custom default 
(CASSANDRA-13259)
 + * Better document in code InetAddressAndPort usage post 7544, incorporate 
port into UUIDGen node (CASSANDRA-14226)
 + * Fix sstablemetadata date string for minLocalDeletionTime (CASSANDRA-14132)
 + * Make it possible to change neverPurgeTombstones during runtime 
(CASSANDRA-14214)
 + * Remove GossipDigestSynVerbHandler#doSort() (CASSANDRA-14174)
 + * Add nodetool clientlist (CASSANDRA-13665)
 + * Revert ProtocolVersion changes from CASSANDRA-7544 (CASSANDRA-14211)
 + * Non-disruptive seed node list reload (CASSANDRA-14190)
 + * Nodetool tablehistograms to print statics for all the tables 
(CASSANDRA-14185)
 + * Migrate dtests to use pytest and python3 (CASSANDRA-14134)
 + * Allow storage port to be configurable per node (CASSANDRA-7544)
 + * Make sub-range selection for non-frozen collections return null instead of 
empty (CASSANDRA-14182)
 + * BloomFilter serialization format should not change byte ordering 
(CASSANDRA-9067)
 + * Remove unused on-heap BloomFilter implementation (CASSANDRA-14152)
 + * Delete temp test files on exit (CASSANDRA-14153)
 + * Make PartitionUpdate and Mutation immutable (CASSANDRA-13867)
 + * Fix CommitLogReplayer exception for CDC data (CASSANDRA-14066)
 + * Fix cassandra-stress startup failure (CASSANDRA-14106)
 + * Remove initialDirectories from CFS (CASSANDRA-13928)
 + * Fix trivial log format error (CASSANDRA-14015)
 + * Allow sstabledump to do a json object per partition (CASSANDRA-13848)
 + * Add option to optimise merkle tree comparison across replicas 
(CASSANDRA-3200)
 + * 

[1/3] cassandra git commit: Don't fail startup if peers aren't live

[samt]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.11 95cfee623 -> 28ccf3fe3
  refs/heads/trunk 70d95359d -> 88b01c866


Don't fail startup if peers aren't live

Patch by Kurt Greaves; reviewed by Sam Tunnicliffe for CASSANDRA-13851


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/28ccf3fe
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/28ccf3fe
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/28ccf3fe

Branch: refs/heads/cassandra-3.11
Commit: 28ccf3fe3989d9d80063fe4d4bb048efe471936b
Parents: 95cfee6
Author: kurt 
Authored: Wed Nov 1 08:58:54 2017 +
Committer: Sam Tunnicliffe 
Committed: Tue Apr 17 17:59:48 2018 +0100

--
 CHANGES.txt |  1 +
 src/java/org/apache/cassandra/gms/Gossiper.java | 38 +++-
 .../cassandra/service/StorageService.java   | 10 --
 3 files changed, 38 insertions(+), 11 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/28ccf3fe/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 42ea3b4..39213a1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.11.3
+ * Allow existing nodes to use all peers in shadow round (CASSANDRA-13851)
  * Fix cqlsh to read connection.ssl cqlshrc option again (CASSANDRA-14299)
  * Downgrade log level to trace for CommitLogSegmentManager (CASSANDRA-14370)
  * CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/28ccf3fe/src/java/org/apache/cassandra/gms/Gossiper.java
--
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java 
b/src/java/org/apache/cassandra/gms/Gossiper.java
index 2dac5c2..ea05525 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -1361,6 +1361,11 @@ public class Gossiper implements 
IFailureDetectionEventListener, GossiperMBean
   
TimeUnit.MILLISECONDS);
 }
 
+public synchronized Map doShadowRound()
+{
+return doShadowRound(Collections.EMPTY_SET);
+}
+
 /**
  * Do a single 'shadow' round of gossip by retrieving endpoint states that 
will be stored exclusively in the
  * map return value, instead of endpointStateMap.
@@ -1376,16 +1381,21 @@ public class Gossiper implements 
IFailureDetectionEventListener, GossiperMBean
  * caller of {@link Gossiper#doShadowRound()}. Therefor only a single 
shadow round execution is permitted at
  * the same time.
  *
+ * @param peers Additional peers to try gossiping with.
  * @return endpoint states gathered during shadow round or empty map
  */
-public synchronized Map doShadowRound()
+public synchronized Map 
doShadowRound(Set peers)
 {
 buildSeedsList();
-// it may be that the local address is the only entry in the seed
+// it may be that the local address is the only entry in the seed + 
peers
 // list in which case, attempting a shadow round is pointless
-if (seeds.isEmpty())
+if (seeds.isEmpty() && peers.isEmpty())
 return endpointShadowStateMap;
 
+boolean isSeed = 
DatabaseDescriptor.getSeeds().contains(FBUtilities.getBroadcastAddress());
+// We double RING_DELAY if we're not a seed to increase chance of 
successful startup during a full cluster bounce,
+// giving the seeds a chance to startup before we fail the shadow round
+int shadowRoundDelay =  isSeed ? StorageService.RING_DELAY : 
StorageService.RING_DELAY * 2;
 seedsInShadowRound.clear();
 endpointShadowStateMap.clear();
 // send a completely empty syn
@@ -1398,6 +1408,7 @@ public class Gossiper implements 
IFailureDetectionEventListener, GossiperMBean
 GossipDigestSyn.serializer);
 
 inShadowRound = true;
+boolean includePeers = false;
 int slept = 0;
 try
 {
@@ -1409,6 +1420,15 @@ public class Gossiper implements 
IFailureDetectionEventListener, GossiperMBean
 
 for (InetAddress seed : seeds)
 MessagingService.instance().sendOneWay(message, seed);
+
+// Send to any peers we already know about, but only if a 
seed didn't respond.
+if (includePeers)
+{
+logger.trace("Sending shadow round GOSSIP DIGEST SYN 
to known peers {}", peers);
+for (InetAddress peer : 

[2/3] cassandra git commit: Don't fail startup if peers aren't live

[samt]

Don't fail startup if peers aren't live

Patch by Kurt Greaves; reviewed by Sam Tunnicliffe for CASSANDRA-13851


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/28ccf3fe
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/28ccf3fe
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/28ccf3fe

Branch: refs/heads/trunk
Commit: 28ccf3fe3989d9d80063fe4d4bb048efe471936b
Parents: 95cfee6
Author: kurt 
Authored: Wed Nov 1 08:58:54 2017 +
Committer: Sam Tunnicliffe 
Committed: Tue Apr 17 17:59:48 2018 +0100

--
 CHANGES.txt |  1 +
 src/java/org/apache/cassandra/gms/Gossiper.java | 38 +++-
 .../cassandra/service/StorageService.java   | 10 --
 3 files changed, 38 insertions(+), 11 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/28ccf3fe/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 42ea3b4..39213a1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.11.3
+ * Allow existing nodes to use all peers in shadow round (CASSANDRA-13851)
  * Fix cqlsh to read connection.ssl cqlshrc option again (CASSANDRA-14299)
  * Downgrade log level to trace for CommitLogSegmentManager (CASSANDRA-14370)
  * CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/28ccf3fe/src/java/org/apache/cassandra/gms/Gossiper.java
--
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java 
b/src/java/org/apache/cassandra/gms/Gossiper.java
index 2dac5c2..ea05525 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -1361,6 +1361,11 @@ public class Gossiper implements 
IFailureDetectionEventListener, GossiperMBean
   
TimeUnit.MILLISECONDS);
 }
 
+public synchronized Map doShadowRound()
+{
+return doShadowRound(Collections.EMPTY_SET);
+}
+
 /**
  * Do a single 'shadow' round of gossip by retrieving endpoint states that 
will be stored exclusively in the
  * map return value, instead of endpointStateMap.
@@ -1376,16 +1381,21 @@ public class Gossiper implements 
IFailureDetectionEventListener, GossiperMBean
  * caller of {@link Gossiper#doShadowRound()}. Therefor only a single 
shadow round execution is permitted at
  * the same time.
  *
+ * @param peers Additional peers to try gossiping with.
  * @return endpoint states gathered during shadow round or empty map
  */
-public synchronized Map doShadowRound()
+public synchronized Map 
doShadowRound(Set peers)
 {
 buildSeedsList();
-// it may be that the local address is the only entry in the seed
+// it may be that the local address is the only entry in the seed + 
peers
 // list in which case, attempting a shadow round is pointless
-if (seeds.isEmpty())
+if (seeds.isEmpty() && peers.isEmpty())
 return endpointShadowStateMap;
 
+boolean isSeed = 
DatabaseDescriptor.getSeeds().contains(FBUtilities.getBroadcastAddress());
+// We double RING_DELAY if we're not a seed to increase chance of 
successful startup during a full cluster bounce,
+// giving the seeds a chance to startup before we fail the shadow round
+int shadowRoundDelay =  isSeed ? StorageService.RING_DELAY : 
StorageService.RING_DELAY * 2;
 seedsInShadowRound.clear();
 endpointShadowStateMap.clear();
 // send a completely empty syn
@@ -1398,6 +1408,7 @@ public class Gossiper implements 
IFailureDetectionEventListener, GossiperMBean
 GossipDigestSyn.serializer);
 
 inShadowRound = true;
+boolean includePeers = false;
 int slept = 0;
 try
 {
@@ -1409,6 +1420,15 @@ public class Gossiper implements 
IFailureDetectionEventListener, GossiperMBean
 
 for (InetAddress seed : seeds)
 MessagingService.instance().sendOneWay(message, seed);
+
+// Send to any peers we already know about, but only if a 
seed didn't respond.
+if (includePeers)
+{
+logger.trace("Sending shadow round GOSSIP DIGEST SYN 
to known peers {}", peers);
+for (InetAddress peer : peers)
+MessagingService.instance().sendOneWay(message, 
peer);
+}
+

[jira] [Commented] (CASSANDRA-14381) nodetool listsnapshots is missing snapshots

[Cyril Scetbon (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441199#comment-16441199
 ] 

Cyril Scetbon commented on CASSANDRA-14381:
---

Thanks [~aweisberg]

> nodetool listsnapshots is missing snapshots
> ---
>
> Key: CASSANDRA-14381
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14381
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: MacOs 10.12.5
> Java 1.8.0_144
> Cassandra 3.11.2 (brew install)
>Reporter: Cyril Scetbon
>Assignee: Ariel Weisberg
>Priority: Major
>
> The output of *nodetool listsnapshots* is inconsistent with the snapshots 
> created :
> {code:java}
> $ nodetool listsnapshots
> Snapshot Details:
> There are no snapshots
> $ nodetool snapshot -t tag1 --table local system
> Requested creating snapshot(s) for [system] with snapshot name [tag1] and 
> options {skipFlush=false}
> Snapshot directory: tag1
> $ nodetool snapshot -t tag2 --table local system
> Requested creating snapshot(s) for [system] with snapshot name [tag2] and 
> options {skipFlush=false}
> Snapshot directory: tag2
> $ nodetool listsnapshots
> Snapshot Details:
> There are no snapshots
> $ ls 
> /usr/local/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/snapshots/
> tag1 tag2{code}
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14381) nodetool listsnapshots is missing snapshots

[Jay Zhuang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441171#comment-16441171
 ] 

Jay Zhuang commented on CASSANDRA-14381:


+1

LGTM. Is the circleci configuration change needed?

> nodetool listsnapshots is missing snapshots
> ---
>
> Key: CASSANDRA-14381
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14381
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: MacOs 10.12.5
> Java 1.8.0_144
> Cassandra 3.11.2 (brew install)
>Reporter: Cyril Scetbon
>Assignee: Ariel Weisberg
>Priority: Major
>
> The output of *nodetool listsnapshots* is inconsistent with the snapshots 
> created :
> {code:java}
> $ nodetool listsnapshots
> Snapshot Details:
> There are no snapshots
> $ nodetool snapshot -t tag1 --table local system
> Requested creating snapshot(s) for [system] with snapshot name [tag1] and 
> options {skipFlush=false}
> Snapshot directory: tag1
> $ nodetool snapshot -t tag2 --table local system
> Requested creating snapshot(s) for [system] with snapshot name [tag2] and 
> options {skipFlush=false}
> Snapshot directory: tag2
> $ nodetool listsnapshots
> Snapshot Details:
> There are no snapshots
> $ ls 
> /usr/local/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/snapshots/
> tag1 tag2{code}
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13896) Improving Cassandra write performance

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441167#comment-16441167
 ] 

Jeff Jirsa commented on CASSANDRA-13896:


[~PrinceNana] - do you have an opinion on [~burmanm]'s patch?

cc [~aweisberg] for visibility.

> Improving Cassandra write performance  
> ---
>
> Key: CASSANDRA-13896
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13896
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
> Environment: Skylake server with 2 sockets, 192GB RAM, 3xPCIe SSDs
> OS: Centos 7.3 
> Java: Oracle JDK1.8.0_121
>Reporter: Prince Nana Owusu Boateng
>Priority: Major
>  Labels: Performance
> Fix For: 4.x
>
> Attachments: Screen Shot 2017-09-22 at 11.22.43 AM.png, Screen Shot 
> 2017-09-22 at 3.31.09 PM.png
>
>
> During our Cassandra performance testing, we see high percentage of the CPU 
> spent in *org.apache.cassandra.utils.memory.SlabAllocator.allocate(int, 
> OpOrder Group) * method.  Appears to be high contention of the 
> ** atomic Integer in write workloads.   This structure is 
> used by the threads for keeping track of the region bytebuffer allocation.  
> When the contention appears, adding more clients, modifications of write 
> specific parameters does not change write throughput performance.  Attached 
> are the details of Java Flight Recorder (JFR), showing hot functions and also 
> performance results.   When we see this contention, we still have plenty of 
> CPU and throughput left ( *<20%*  Total average CPU utilization and  *<11%* 
> of the storage write total throughput).   This occurs on Cassandra 3.10.0-src 
> version using the Cassandra-Stress.
> Proposal:
> We will like to introduce a solution which eliminates the atomic operations 
> on the ** atomic Integer. This implementation will allow 
> concurrent allocation of bytebuffers without an atomic compareAndSet and 
> incrementAndGet operations. The solution is expected to increase overall 
> write performance while improving CPU utilization.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-6719) redesign loadnewsstables

[Jordan West (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441160#comment-16441160
 ] 

Jordan West commented on CASSANDRA-6719:


Changes look good [~krummas]. +1. 

> redesign loadnewsstables
> 
>
> Key: CASSANDRA-6719
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6719
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Tools
>Reporter: Jonathan Ellis
>Assignee: Marcus Eriksson
>Priority: Minor
>  Labels: lhf
> Fix For: 4.x
>
> Attachments: 6719.patch
>
>
> CFSMBean.loadNewSSTables scans data directories for new sstables dropped 
> there by an external agent.  This is dangerous because of possible filename 
> conflicts with existing or newly generated sstables.
> Instead, we should support leaving the new sstables in a separate directory 
> (specified by a parameter, or configured as a new location in yaml) and take 
> care of renaming as necessary automagically.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14381) nodetool listsnapshots is missing snapshots

[Ariel Weisberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-14381:
---
Assignee: Ariel Weisberg
  Status: Patch Available  (was: Open)

I had a free minute so put up a patch for this.
[Trunk 
code|https://github.com/apache/cassandra/compare/trunk...aweisberg:cassandra-14381-trunk?expand=1]
[CircleCI|https://circleci.com/gh/aweisberg/cassandra/tree/cassandra-14381-trunk]

> nodetool listsnapshots is missing snapshots
> ---
>
> Key: CASSANDRA-14381
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14381
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: MacOs 10.12.5
> Java 1.8.0_144
> Cassandra 3.11.2 (brew install)
>Reporter: Cyril Scetbon
>Assignee: Ariel Weisberg
>Priority: Major
>
> The output of *nodetool listsnapshots* is inconsistent with the snapshots 
> created :
> {code:java}
> $ nodetool listsnapshots
> Snapshot Details:
> There are no snapshots
> $ nodetool snapshot -t tag1 --table local system
> Requested creating snapshot(s) for [system] with snapshot name [tag1] and 
> options {skipFlush=false}
> Snapshot directory: tag1
> $ nodetool snapshot -t tag2 --table local system
> Requested creating snapshot(s) for [system] with snapshot name [tag2] and 
> options {skipFlush=false}
> Snapshot directory: tag2
> $ nodetool listsnapshots
> Snapshot Details:
> There are no snapshots
> $ ls 
> /usr/local/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/snapshots/
> tag1 tag2{code}
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14392) Rename nodetool --with-port to --print-port to disambiguate from --port

[Ariel Weisberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-14392:
---
Status: Patch Available  (was: Open)

[Trunk 
code|https://github.com/apache/cassandra/compare/trunk...aweisberg:cassandra-14392-trunk?expand=1]

[CircleCI|https://circleci.com/gh/aweisberg/cassandra/tree/cassandra-14392-trunk]

> Rename nodetool --with-port to --print-port to disambiguate from --port
> ---
>
> Key: CASSANDRA-14392
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14392
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Major
> Fix For: 4.0
>
>
> Right now it looks kind of like a third way to set the port number.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Resolved] (CASSANDRA-14301) Error running cqlsh: unexpected keyword argument 'no_compact'

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa resolved CASSANDRA-14301.

Resolution: Not A Problem

> Error running cqlsh: unexpected keyword argument 'no_compact'
> -
>
> Key: CASSANDRA-14301
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14301
> Project: Cassandra
>  Issue Type: Bug
>Reporter: M. Justin
>Priority: Major
>  Labels: cqlsh, proposed-wontfix
>
> I recently installed Cassandra 3.11.2 on my Mac using Homebrew.  When I run 
> the "cqlsh" command, I get the following error:
> {code:none}
> $ cqlsh
> Traceback (most recent call last):
>   File "/usr/local/Cellar/cassandra/3.11.2/libexec/bin/cqlsh.py", line 2443, 
> in 
>     main(*read_options(sys.argv[1:], os.environ))
>   File "/usr/local/Cellar/cassandra/3.11.2/libexec/bin/cqlsh.py", line 2421, 
> in main
>     encoding=options.encoding)
>   File "/usr/local/Cellar/cassandra/3.11.2/libexec/bin/cqlsh.py", line 488, 
> in __init__
>     **kwargs)
>   File "cassandra/cluster.py", line 735, in 
> cassandra.cluster.Cluster.__init__ (cassandra/cluster.c:10935)
> TypeError: __init__() got an unexpected keyword argument 'no_compact'
> {code}
> Commenting out [line 483 of 
> cqlsh.py|https://github.com/apache/cassandra/blob/cassandra-3.11.2/bin/cqlsh.py#L483]
>  works around the issue:
> {code}
> # no_compact=no_compact
> {code}
>  I am not the only person impacted, as evidenced by [this existing Stack 
> Overflow 
> post|https://stackoverflow.com/questions/48885984/was-cqlsh-5-0-1-broken-in-cassandra-3-11-2-release]
>  from 2/20/2018.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14160) maxPurgeableTimestamp should traverse tables in order of minTimestamp

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441124#comment-16441124
 ] 

Jeff Jirsa commented on CASSANDRA-14160:


cc [~cnlwsu] as well (we've seen some meaningful perf improvements with 
NEVER_PURGE_TOMBSTONES=true, tagging Chris in case he remembers if the benefit 
was in the purge evaluator or getFullyExpiredSSTables() )



> maxPurgeableTimestamp should traverse tables in order of minTimestamp
> -
>
> Key: CASSANDRA-14160
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14160
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Josh Snyder
>Assignee: Josh Snyder
>Priority: Major
>  Labels: performance
> Fix For: 4.x
>
>
> In maxPurgeableTimestamp, we iterate over the bloom filters of each 
> overlapping SSTable. Of the bloom filter hits, we take the SSTable with the 
> lowest minTimestamp. If we kept the SSTables in sorted order of minTimestamp, 
> then we could short-circuit the operation at the first bloom filter hit, 
> reducing cache pressure (or worse, I/O) and CPU time.
> I've written (but not yet benchmarked) [some 
> code|https://github.com/hashbrowncipher/cassandra/commit/29859a4a2e617f6775be49448858bc59fdafab44]
>  to demonstrate this possibility.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14332) Fix unbounded validation compactions on repair

[Blake Eggleston (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-14332:

Reviewer: Blake Eggleston

> Fix unbounded validation compactions on repair
> --
>
> Key: CASSANDRA-14332
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14332
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Major
>
> After CASSANDRA-13797 it's possible to cause unbounded, simultaneous 
> validation compactions as we no longer wait for validations to finish. 
> Potential fix is to have a sane default for the # of concurrent validation 
> compactions by backporting CASSANDRA-13521 and setting a sane default.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14392) Rename nodetool --with-port to --print-port to disambiguate from --port

[Ariel Weisberg (JIRA)]

Ariel Weisberg created CASSANDRA-14392:
--

 Summary: Rename nodetool --with-port to --print-port to 
disambiguate from --port
 Key: CASSANDRA-14392
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14392
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 4.0


Right now it looks kind of like a third way to set the port number.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14301) Error running cqlsh: unexpected keyword argument 'no_compact'

[M. Justin (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441081#comment-16441081
 ] 

M. Justin commented on CASSANDRA-14301:
---

Homebrew ([has 
reported|https://github.com/Homebrew/homebrew-core/issues/24977#issuecomment-372047111])
 that they've fixed the bug on their end.  I believe this can be closed as a 
(now-fixed) issue with Homebrew and not an issue with Cassandra.

> Error running cqlsh: unexpected keyword argument 'no_compact'
> -
>
> Key: CASSANDRA-14301
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14301
> Project: Cassandra
>  Issue Type: Bug
>Reporter: M. Justin
>Priority: Major
>  Labels: cqlsh, proposed-wontfix
>
> I recently installed Cassandra 3.11.2 on my Mac using Homebrew.  When I run 
> the "cqlsh" command, I get the following error:
> {code:none}
> $ cqlsh
> Traceback (most recent call last):
>   File "/usr/local/Cellar/cassandra/3.11.2/libexec/bin/cqlsh.py", line 2443, 
> in 
>     main(*read_options(sys.argv[1:], os.environ))
>   File "/usr/local/Cellar/cassandra/3.11.2/libexec/bin/cqlsh.py", line 2421, 
> in main
>     encoding=options.encoding)
>   File "/usr/local/Cellar/cassandra/3.11.2/libexec/bin/cqlsh.py", line 488, 
> in __init__
>     **kwargs)
>   File "cassandra/cluster.py", line 735, in 
> cassandra.cluster.Cluster.__init__ (cassandra/cluster.c:10935)
> TypeError: __init__() got an unexpected keyword argument 'no_compact'
> {code}
> Commenting out [line 483 of 
> cqlsh.py|https://github.com/apache/cassandra/blob/cassandra-3.11.2/bin/cqlsh.py#L483]
>  works around the issue:
> {code}
> # no_compact=no_compact
> {code}
>  I am not the only person impacted, as evidenced by [this existing Stack 
> Overflow 
> post|https://stackoverflow.com/questions/48885984/was-cqlsh-5-0-1-broken-in-cassandra-3-11-2-release]
>  from 2/20/2018.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14301) Error running cqlsh: unexpected keyword argument 'no_compact'

[M. Justin (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441081#comment-16441081
 ] 

M. Justin edited comment on CASSANDRA-14301 at 4/17/18 4:06 PM:


Homebrew [has 
reported|https://github.com/Homebrew/homebrew-core/issues/24977#issuecomment-372047111]
 that they've fixed the bug on their end.  I believe this can be closed as a 
(now-fixed) issue with Homebrew and not an issue with Cassandra.


was (Author: mjjustin):
Homebrew ([has 
reported|https://github.com/Homebrew/homebrew-core/issues/24977#issuecomment-372047111])
 that they've fixed the bug on their end.  I believe this can be closed as a 
(now-fixed) issue with Homebrew and not an issue with Cassandra.

> Error running cqlsh: unexpected keyword argument 'no_compact'
> -
>
> Key: CASSANDRA-14301
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14301
> Project: Cassandra
>  Issue Type: Bug
>Reporter: M. Justin
>Priority: Major
>  Labels: cqlsh, proposed-wontfix
>
> I recently installed Cassandra 3.11.2 on my Mac using Homebrew.  When I run 
> the "cqlsh" command, I get the following error:
> {code:none}
> $ cqlsh
> Traceback (most recent call last):
>   File "/usr/local/Cellar/cassandra/3.11.2/libexec/bin/cqlsh.py", line 2443, 
> in 
>     main(*read_options(sys.argv[1:], os.environ))
>   File "/usr/local/Cellar/cassandra/3.11.2/libexec/bin/cqlsh.py", line 2421, 
> in main
>     encoding=options.encoding)
>   File "/usr/local/Cellar/cassandra/3.11.2/libexec/bin/cqlsh.py", line 488, 
> in __init__
>     **kwargs)
>   File "cassandra/cluster.py", line 735, in 
> cassandra.cluster.Cluster.__init__ (cassandra/cluster.c:10935)
> TypeError: __init__() got an unexpected keyword argument 'no_compact'
> {code}
> Commenting out [line 483 of 
> cqlsh.py|https://github.com/apache/cassandra/blob/cassandra-3.11.2/bin/cqlsh.py#L483]
>  works around the issue:
> {code}
> # no_compact=no_compact
> {code}
>  I am not the only person impacted, as evidenced by [this existing Stack 
> Overflow 
> post|https://stackoverflow.com/questions/48885984/was-cqlsh-5-0-1-broken-in-cassandra-3-11-2-release]
>  from 2/20/2018.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14391) COPY FROM ignores headers

[M. Justin (JIRA)]

M. Justin created CASSANDRA-14391:
-

 Summary: COPY FROM ignores headers
 Key: CASSANDRA-14391
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14391
 Project: Cassandra
  Issue Type: Bug
  Components: CQL
 Environment: cqlsh 5.0.1 and Cassandra 3.11.2 on macOS 10.13.2.
Reporter: M. Justin


COPY FROM appears to ignore the headers value, even when "headers = true" is 
specified. This means that if the columns are reordered, the import process 
will save values in the wrong columns.
h2. Example
{noformat:title=temp.csv}
col2,col1,col3
column value 1,key2,3
column value 2,key4,3
column value 3,key3,3
column value 4,key1,3
{noformat}
{code:sql}
create keyspace copy_to_from_test WITH replication = { 'class' : 
'SimpleStrategy', 'replication_factor' : 1 };
use copy_to_from_test;
create table test_table (col1 text primary key, col2 text, col3 bigint);
copy test_table from 'temp.csv' with header = true;
{code}
The above code will incorrectly swap the "col2" and "col1" values, since it 
expects the first column to be "col1".  If I had instead swapped the order of 
"col3", I would have received an error on input, as it would have attempted to 
store text in a numerical column.
h2.  Expected Behavior

I would expect specifying "with header = true" on a COPY FROM statement to use 
the headers as column names for insertion, rather than merely skipping the 
header row.

A question is whether missing columns should be an error, or just not imported.

h2. Other

I ran across this issue when copying between two of my environments.  One of 
the environments had changed the columns in the primary key, but the other had 
not yet.  This caused the order of the columns to vary between the environments.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14381) nodetool listsnapshots is missing snapshots

[Cyril Scetbon (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440995#comment-16440995
 ] 

Cyril Scetbon commented on CASSANDRA-14381:
---

Okay that's what I thought. Whenever I find some time, I should be able to 
remove that piece of code that skips the system keyspace

> nodetool listsnapshots is missing snapshots
> ---
>
> Key: CASSANDRA-14381
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14381
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: MacOs 10.12.5
> Java 1.8.0_144
> Cassandra 3.11.2 (brew install)
>Reporter: Cyril Scetbon
>Priority: Major
>
> The output of *nodetool listsnapshots* is inconsistent with the snapshots 
> created :
> {code:java}
> $ nodetool listsnapshots
> Snapshot Details:
> There are no snapshots
> $ nodetool snapshot -t tag1 --table local system
> Requested creating snapshot(s) for [system] with snapshot name [tag1] and 
> options {skipFlush=false}
> Snapshot directory: tag1
> $ nodetool snapshot -t tag2 --table local system
> Requested creating snapshot(s) for [system] with snapshot name [tag2] and 
> options {skipFlush=false}
> Snapshot directory: tag2
> $ nodetool listsnapshots
> Snapshot Details:
> There are no snapshots
> $ ls 
> /usr/local/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/snapshots/
> tag1 tag2{code}
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-7544) Allow storage port to be configurable per node

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440983#comment-16440983
 ] 

Ariel Weisberg commented on CASSANDRA-7544:
---

I think it's fine to tweak what that is called. You are right --with-port seems 
wrong as you would expect it to accept a port number as an argument with the 
Java builder idiom and maybe also CLI idioms.

I'll create a separate ticket for it.

> Allow storage port to be configurable per node
> --
>
> Key: CASSANDRA-7544
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7544
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sam Overton
>Assignee: Ariel Weisberg
>Priority: Major
> Fix For: 4.0
>
>
> Currently storage_port must be configured identically on all nodes in a 
> cluster and it is assumed that this is the case when connecting to a remote 
> node.
> This prevents running in any environment that requires multiple nodes to be 
> able to bind to the same network interface, such as with many automatic 
> provisioning/deployment frameworks.
> The current solutions seems to be
> * use a separate network interface for each node deployed to the same box. 
> This puts a big requirement on IP allocation at large scale.
> * allow multiple clusters to be provisioned from the same resource pool, but 
> restrict allocation to a maximum of one node per host from each cluster, 
> assuming each cluster is running on a different storage port.
> It would make operations much simpler in these kind of environments if the 
> environment provisioning the resources could assign the ports to be used when 
> bringing up a new node on shared hardware.
> The changes required would be at least the following:
> 1. configure seeds as IP:port instead of just IP
> 2. gossip the storage port as part of a node's ApplicationState
> 3. refer internally to nodes by hostID instead of IP, since there will be 
> multiple nodes with the same IP
> (1) & (2) are mostly trivial and I already have a patch for these. The bulk 
> of the work to enable this is (3), and I would structure this as a separate 
> pre-requisite patch. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Resolved] (CASSANDRA-14371) dtest failure: sstablesplit_test.TestSSTableSplit.test_single_file_split

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-14371.
---
Resolution: Fixed

Looks like we are good now after the ccm PR got merged. Thanks.

> dtest failure: sstablesplit_test.TestSSTableSplit.test_single_file_split
> 
>
> Key: CASSANDRA-14371
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14371
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Patrick Bannister
>Priority: Major
>
> https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-dtest/489/testReport/sstablesplit_test/TestSSTableSplit/test_single_file_split/
> {code}
> for (stdout, stderr, rc) in result:
> logger.debug(stderr)
> >   failure = stderr.find("java.lang.AssertionError: Data component 
> > is missing")
> E   TypeError: a bytes-like object is required, not 'str'
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14388) Fix setting min/max compaction threshold with LCS

[Chris Lohfink (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440896#comment-16440896
 ] 

Chris Lohfink commented on CASSANDRA-14388:
---

I like the alternative personally
{quote}An alternative would be to check if there is more than 32 
({{MAX_COMPACTING_L0}}) sstables in L0, if so, grab {{max_threshold}} sstables 
and run STCS on them
{quote}
Thinking mostly for cases when its critically far behind (ie 50k in L0) and a 
lot are tiny (from bad repairs), just want to quickly reduce the number of 
them. The {{MAX_COMPACTING_L0}} I think makes sense for normal use, but when 
theres a huge backlog just want STCS to chew up backlog faster than 32 at a 
time. But if increase L0 max compacting to 1000 it may not kick off STCS until 
already negatively impacting things.

Might be nice to have {{MAX_COMPACTING_L0}} as another table option but that 
could be done another ticket.

> Fix setting min/max compaction threshold with LCS
> -
>
> Key: CASSANDRA-14388
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14388
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
> Fix For: 4.x
>
>
> To be able to actually set max/min_threshold in compaction options we need to 
> remove it from the options map when validating.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14160) maxPurgeableTimestamp should traverse tables in order of minTimestamp

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440885#comment-16440885
 ] 

Marcus Eriksson commented on CASSANDRA-14160:
-

hey [~josnyder] the code looks good to me, but, as [~jjirsa] mentioned above, a 
unit test that makes sure the sstables are returned in the correct order should 
be added. You also mentioned a that you had not yet benchmarked the change, 
have you done that? If not, that would also be nice.

> maxPurgeableTimestamp should traverse tables in order of minTimestamp
> -
>
> Key: CASSANDRA-14160
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14160
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Josh Snyder
>Assignee: Josh Snyder
>Priority: Major
>  Labels: performance
> Fix For: 4.x
>
>
> In maxPurgeableTimestamp, we iterate over the bloom filters of each 
> overlapping SSTable. Of the bloom filter hits, we take the SSTable with the 
> lowest minTimestamp. If we kept the SSTables in sorted order of minTimestamp, 
> then we could short-circuit the operation at the first bloom filter hit, 
> reducing cache pressure (or worse, I/O) and CPU time.
> I've written (but not yet benchmarked) [some 
> code|https://github.com/hashbrowncipher/cassandra/commit/29859a4a2e617f6775be49448858bc59fdafab44]
>  to demonstrate this possibility.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14390) Cassandra's Debian package depends on java-X-jre which requires GUI components

[Richard Whitehouse (JIRA)]

Richard Whitehouse created CASSANDRA-14390:
--

 Summary: Cassandra's Debian package depends on java-X-jre which 
requires GUI components
 Key: CASSANDRA-14390
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14390
 Project: Cassandra
  Issue Type: Bug
  Components: Packaging
Reporter: Richard Whitehouse


Cassandra can't be installed against a later version of the JRE without also 
installing a bunch of GUI packages which aren't relevant on a server 
installation.

e.g. Cassandra 3 can't be installed against a Java 9 JRE without also 
installing GUI components.

This is because of the Debian package dependencies.

Cassandra 3+ depends on `openjdk-8-jre-headless | java-8-jre`.

Cassandra 2.x depends on `opendjk-7-jre-headless | java-8-jre`

`java-X-jre` is a metapackage which specifies a Java version compatible with 
the given Java version that includes GUI components. It's supplied by 
`openjdk-X-jre` - e.g. `java-8-jre` is supplied by `openjdk-8-jre` and 
`openjdk-9-jre`.

In comparison, `java-X-jre-headless` is a metapackage which specifies a Java 
version compatible with the given Java version but doesn't require GUI 
components.It's supplied by `openjdk-X-jre-headless` - e.g. 
`java-8-jre-headless` is supplied by `openjdk-8-jre-headless` and 
`openjdk-9-jre-headless`.

Can Cassandra be changed to depend on `java-8-jre-headless` instead of 
`java-8-jre`?

This affects all releases since Debian packaging was introduced according to 
the commit logs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14362) Bind to correct local address in 4.0 streaming

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440818#comment-16440818
 ] 

Jason Brown edited comment on CASSANDRA-14362 at 4/17/18 12:54 PM:
---

[~Lerh Low]'s patch is correct and fixes a bug I introduced with 
CASSANDRA-12229. Further, there is something wrong with my handling of 
CASSANDRA-12673, but I don't want to hold up Lerh or anyone else looking at 4.0 
in ec2. Thus, I've decided to commit Lerh's patch and will open a followup 
ticket to address the local address binding issue.

Committed as sha {{70d95359d2dca1c35f573776d11ed87bb9b4b441}}. Nice find and 
fix, Lehr.


was (Author: jasobrown):
[~Lerh Low]'s patch is correct and fixes a bug I introduced with 
CASSANDRA-12229. Further, there is something wrong with my handling of 
CASSANDRA-12673, but I don't want to hold up Lerh or anyone else looking at 4.0 
in ec2. Thus, I've decided to commit Lerh's patch and will open a followup 
ticket to address the local address binding issue.

Committed as sha {{stF4fFtT9pMYpabtyomEEFKARMeB8A7H}}. Nice find and fix, Lehr.

> Bind to correct local address in 4.0 streaming
> --
>
> Key: CASSANDRA-14362
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14362
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Fix For: 4.0
>
>
> I'm not sure if anybody has tried using {{trunk}} and starting an actual EC2 
> cluster, but with a 3 node setup that works on 3.11, it fails to work on 
> {{trunk}}. I mistakenly stumbled into it while testing my own code changes. 
> My setup is as follows:
> broadcast_rpc_address: publicIP
> broadcast_address: publicIP
> listen_address: omitted. Ends up as privateIP. 
> Works on 3.11 just fine. 
> On {{trunk}} though, it never works. My node is never able to join the 
> cluster:
> {code}
> Apr 03 05:57:06 ip-10-0-47-122 cassandra[13914]: INFO  [main] 2018-04-03 
> 05:57:05,895 RangeStreamer.java:195 - Bootstrap: range 
> (-128373781239966537,-122439194129870521] exists on 52.88.241.181:7000 for 
> keyspace system_traces
> Apr 03 05:57:06 ip-10-0-47-122 cassandra[13914]: INFO  [main] 2018-04-03 
> 05:57:05,895 RangeStreamer.java:195 - Bootstrap: range 
> (6968670424536541270,6973888347502882935] exists on 52.88.241.181:7000 for 
> keyspace system_traces
> Apr 03 05:57:42 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:57:42,298 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 26215173446ns > 50ns
> Apr 03 05:57:53 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:57:53,035 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 10736485907ns > 50ns
> Apr 03 05:58:30 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:58:30,790 Gossiper.java:814 - Gossip stage has 28 pending 
> tasks; skipping status check (no nodes will be marked down)
> Apr 03 05:58:33 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:58:33,060 Gossiper.java:814 - Gossip stage has 20 pending 
> tasks; skipping status check (no nodes will be marked down)
> Apr 03 06:04:33 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 06:04:33,826 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 400790432954ns > 50ns
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 06:04:49,133 Gossiper.java:814 - Gossip stage has 2 pending tasks; 
> skipping status check (no nodes will be marked down)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: ERROR 
> [StreamConnectionEstablisher:1] 2018-04-03 06:04:49,138 
> StreamSession.java:524 - [Stream #d4cd6420-3703-11e8-a6a5-e51ddc10cfe6] 
> Streaming error occurred on session with peer 52.88.241.181:7000
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: java.io.IOException: failed 
> to connect to 52.88.241.181:7000 (STREAM) for streaming data
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:98)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:57)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.createChannel(NettyStreamingMessageSender.java:183)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.setupControlMessageChannel(NettyStreamingMessageSender.java:165)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]:

[jira] [Commented] (CASSANDRA-14362) Bind to correct local address in 4.0 streaming

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440820#comment-16440820
 ] 

Jason Brown commented on CASSANDRA-14362:
-

Opened CASSANDRA-14389 for the local address binding regression. Thanks, 
[~djoshi3] for also looking at this ticket.

> Bind to correct local address in 4.0 streaming
> --
>
> Key: CASSANDRA-14362
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14362
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Fix For: 4.0
>
>
> I'm not sure if anybody has tried using {{trunk}} and starting an actual EC2 
> cluster, but with a 3 node setup that works on 3.11, it fails to work on 
> {{trunk}}. I mistakenly stumbled into it while testing my own code changes. 
> My setup is as follows:
> broadcast_rpc_address: publicIP
> broadcast_address: publicIP
> listen_address: omitted. Ends up as privateIP. 
> Works on 3.11 just fine. 
> On {{trunk}} though, it never works. My node is never able to join the 
> cluster:
> {code}
> Apr 03 05:57:06 ip-10-0-47-122 cassandra[13914]: INFO  [main] 2018-04-03 
> 05:57:05,895 RangeStreamer.java:195 - Bootstrap: range 
> (-128373781239966537,-122439194129870521] exists on 52.88.241.181:7000 for 
> keyspace system_traces
> Apr 03 05:57:06 ip-10-0-47-122 cassandra[13914]: INFO  [main] 2018-04-03 
> 05:57:05,895 RangeStreamer.java:195 - Bootstrap: range 
> (6968670424536541270,6973888347502882935] exists on 52.88.241.181:7000 for 
> keyspace system_traces
> Apr 03 05:57:42 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:57:42,298 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 26215173446ns > 50ns
> Apr 03 05:57:53 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:57:53,035 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 10736485907ns > 50ns
> Apr 03 05:58:30 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:58:30,790 Gossiper.java:814 - Gossip stage has 28 pending 
> tasks; skipping status check (no nodes will be marked down)
> Apr 03 05:58:33 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:58:33,060 Gossiper.java:814 - Gossip stage has 20 pending 
> tasks; skipping status check (no nodes will be marked down)
> Apr 03 06:04:33 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 06:04:33,826 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 400790432954ns > 50ns
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 06:04:49,133 Gossiper.java:814 - Gossip stage has 2 pending tasks; 
> skipping status check (no nodes will be marked down)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: ERROR 
> [StreamConnectionEstablisher:1] 2018-04-03 06:04:49,138 
> StreamSession.java:524 - [Stream #d4cd6420-3703-11e8-a6a5-e51ddc10cfe6] 
> Streaming error occurred on session with peer 52.88.241.181:7000
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: java.io.IOException: failed 
> to connect to 52.88.241.181:7000 (STREAM) for streaming data
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:98)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:57)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.createChannel(NettyStreamingMessageSender.java:183)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.setupControlMessageChannel(NettyStreamingMessageSender.java:165)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.sendMessage(NettyStreamingMessageSender.java:222)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.initialize(NettyStreamingMessageSender.java:146)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:271)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:273)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> 

[jira] [Created] (CASSANDRA-14389) Resolve local address binding in 4.0

[Jason Brown (JIRA)]

Jason Brown created CASSANDRA-14389:
---

 Summary: Resolve local address binding in 4.0
 Key: CASSANDRA-14389
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14389
 Project: Cassandra
  Issue Type: Bug
Reporter: Jason Brown
Assignee: Jason Brown
 Fix For: 4.x


CASSANDRA-8457/CASSANDRA-12229 introduced a regression against CASSANDRA-12673. 
This was discovered with CASSANDRA-14362 and moved here for resolution 
independent of that ticket.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14362) Bind to correct local address in 4.0 streaming

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-14362:

Reviewer: Jason Brown

> Bind to correct local address in 4.0 streaming
> --
>
> Key: CASSANDRA-14362
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14362
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Fix For: 4.0
>
>
> I'm not sure if anybody has tried using {{trunk}} and starting an actual EC2 
> cluster, but with a 3 node setup that works on 3.11, it fails to work on 
> {{trunk}}. I mistakenly stumbled into it while testing my own code changes. 
> My setup is as follows:
> broadcast_rpc_address: publicIP
> broadcast_address: publicIP
> listen_address: omitted. Ends up as privateIP. 
> Works on 3.11 just fine. 
> On {{trunk}} though, it never works. My node is never able to join the 
> cluster:
> {code}
> Apr 03 05:57:06 ip-10-0-47-122 cassandra[13914]: INFO  [main] 2018-04-03 
> 05:57:05,895 RangeStreamer.java:195 - Bootstrap: range 
> (-128373781239966537,-122439194129870521] exists on 52.88.241.181:7000 for 
> keyspace system_traces
> Apr 03 05:57:06 ip-10-0-47-122 cassandra[13914]: INFO  [main] 2018-04-03 
> 05:57:05,895 RangeStreamer.java:195 - Bootstrap: range 
> (6968670424536541270,6973888347502882935] exists on 52.88.241.181:7000 for 
> keyspace system_traces
> Apr 03 05:57:42 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:57:42,298 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 26215173446ns > 50ns
> Apr 03 05:57:53 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:57:53,035 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 10736485907ns > 50ns
> Apr 03 05:58:30 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:58:30,790 Gossiper.java:814 - Gossip stage has 28 pending 
> tasks; skipping status check (no nodes will be marked down)
> Apr 03 05:58:33 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:58:33,060 Gossiper.java:814 - Gossip stage has 20 pending 
> tasks; skipping status check (no nodes will be marked down)
> Apr 03 06:04:33 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 06:04:33,826 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 400790432954ns > 50ns
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 06:04:49,133 Gossiper.java:814 - Gossip stage has 2 pending tasks; 
> skipping status check (no nodes will be marked down)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: ERROR 
> [StreamConnectionEstablisher:1] 2018-04-03 06:04:49,138 
> StreamSession.java:524 - [Stream #d4cd6420-3703-11e8-a6a5-e51ddc10cfe6] 
> Streaming error occurred on session with peer 52.88.241.181:7000
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: java.io.IOException: failed 
> to connect to 52.88.241.181:7000 (STREAM) for streaming data
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:98)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:57)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.createChannel(NettyStreamingMessageSender.java:183)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.setupControlMessageChannel(NettyStreamingMessageSender.java:165)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.sendMessage(NettyStreamingMessageSender.java:222)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.initialize(NettyStreamingMessageSender.java:146)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:271)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:273)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]:  

[jira] [Assigned] (CASSANDRA-14362) Bind to correct local address in 4.0 streaming

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown reassigned CASSANDRA-14362:
---

Assignee: Lerh Chuan Low  (was: Jason Brown)

> Bind to correct local address in 4.0 streaming
> --
>
> Key: CASSANDRA-14362
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14362
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Fix For: 4.0
>
>
> I'm not sure if anybody has tried using {{trunk}} and starting an actual EC2 
> cluster, but with a 3 node setup that works on 3.11, it fails to work on 
> {{trunk}}. I mistakenly stumbled into it while testing my own code changes. 
> My setup is as follows:
> broadcast_rpc_address: publicIP
> broadcast_address: publicIP
> listen_address: omitted. Ends up as privateIP. 
> Works on 3.11 just fine. 
> On {{trunk}} though, it never works. My node is never able to join the 
> cluster:
> {code}
> Apr 03 05:57:06 ip-10-0-47-122 cassandra[13914]: INFO  [main] 2018-04-03 
> 05:57:05,895 RangeStreamer.java:195 - Bootstrap: range 
> (-128373781239966537,-122439194129870521] exists on 52.88.241.181:7000 for 
> keyspace system_traces
> Apr 03 05:57:06 ip-10-0-47-122 cassandra[13914]: INFO  [main] 2018-04-03 
> 05:57:05,895 RangeStreamer.java:195 - Bootstrap: range 
> (6968670424536541270,6973888347502882935] exists on 52.88.241.181:7000 for 
> keyspace system_traces
> Apr 03 05:57:42 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:57:42,298 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 26215173446ns > 50ns
> Apr 03 05:57:53 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:57:53,035 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 10736485907ns > 50ns
> Apr 03 05:58:30 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:58:30,790 Gossiper.java:814 - Gossip stage has 28 pending 
> tasks; skipping status check (no nodes will be marked down)
> Apr 03 05:58:33 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:58:33,060 Gossiper.java:814 - Gossip stage has 20 pending 
> tasks; skipping status check (no nodes will be marked down)
> Apr 03 06:04:33 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 06:04:33,826 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 400790432954ns > 50ns
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 06:04:49,133 Gossiper.java:814 - Gossip stage has 2 pending tasks; 
> skipping status check (no nodes will be marked down)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: ERROR 
> [StreamConnectionEstablisher:1] 2018-04-03 06:04:49,138 
> StreamSession.java:524 - [Stream #d4cd6420-3703-11e8-a6a5-e51ddc10cfe6] 
> Streaming error occurred on session with peer 52.88.241.181:7000
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: java.io.IOException: failed 
> to connect to 52.88.241.181:7000 (STREAM) for streaming data
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:98)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:57)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.createChannel(NettyStreamingMessageSender.java:183)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.setupControlMessageChannel(NettyStreamingMessageSender.java:165)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.sendMessage(NettyStreamingMessageSender.java:222)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.initialize(NettyStreamingMessageSender.java:146)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:271)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:273)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> Apr 03 06:04:49 

[jira] [Updated] (CASSANDRA-14362) Bind to correct local address in 4.0 streaming

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-14362:

Resolution: Fixed
Status: Resolved  (was: Patch Available)

[~Lerh Low]'s patch is correct and fixes a bug I introduced with 
CASSANDRA-12229. Further, there is something wrong with my handling of 
CASSANDRA-12673, but I don't want to hold up Lerh or anyone else looking at 4.0 
in ec2. Thus, I've decided to commit Lerh's patch and will open a followup 
ticket to address the local address binding issue.

Committed as sha {{stF4fFtT9pMYpabtyomEEFKARMeB8A7H}}. Nice find and fix, Lehr.

> Bind to correct local address in 4.0 streaming
> --
>
> Key: CASSANDRA-14362
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14362
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Lerh Chuan Low
>Assignee: Jason Brown
>Priority: Major
> Fix For: 4.0
>
>
> I'm not sure if anybody has tried using {{trunk}} and starting an actual EC2 
> cluster, but with a 3 node setup that works on 3.11, it fails to work on 
> {{trunk}}. I mistakenly stumbled into it while testing my own code changes. 
> My setup is as follows:
> broadcast_rpc_address: publicIP
> broadcast_address: publicIP
> listen_address: omitted. Ends up as privateIP. 
> Works on 3.11 just fine. 
> On {{trunk}} though, it never works. My node is never able to join the 
> cluster:
> {code}
> Apr 03 05:57:06 ip-10-0-47-122 cassandra[13914]: INFO  [main] 2018-04-03 
> 05:57:05,895 RangeStreamer.java:195 - Bootstrap: range 
> (-128373781239966537,-122439194129870521] exists on 52.88.241.181:7000 for 
> keyspace system_traces
> Apr 03 05:57:06 ip-10-0-47-122 cassandra[13914]: INFO  [main] 2018-04-03 
> 05:57:05,895 RangeStreamer.java:195 - Bootstrap: range 
> (6968670424536541270,6973888347502882935] exists on 52.88.241.181:7000 for 
> keyspace system_traces
> Apr 03 05:57:42 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:57:42,298 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 26215173446ns > 50ns
> Apr 03 05:57:53 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:57:53,035 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 10736485907ns > 50ns
> Apr 03 05:58:30 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:58:30,790 Gossiper.java:814 - Gossip stage has 28 pending 
> tasks; skipping status check (no nodes will be marked down)
> Apr 03 05:58:33 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:58:33,060 Gossiper.java:814 - Gossip stage has 20 pending 
> tasks; skipping status check (no nodes will be marked down)
> Apr 03 06:04:33 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 06:04:33,826 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 400790432954ns > 50ns
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 06:04:49,133 Gossiper.java:814 - Gossip stage has 2 pending tasks; 
> skipping status check (no nodes will be marked down)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: ERROR 
> [StreamConnectionEstablisher:1] 2018-04-03 06:04:49,138 
> StreamSession.java:524 - [Stream #d4cd6420-3703-11e8-a6a5-e51ddc10cfe6] 
> Streaming error occurred on session with peer 52.88.241.181:7000
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: java.io.IOException: failed 
> to connect to 52.88.241.181:7000 (STREAM) for streaming data
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:98)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:57)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.createChannel(NettyStreamingMessageSender.java:183)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.setupControlMessageChannel(NettyStreamingMessageSender.java:165)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.sendMessage(NettyStreamingMessageSender.java:222)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.initialize(NettyStreamingMessageSender.java:146)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:271)
> Apr 03 06:04:49 ip-10-0-47-122 

cassandra git commit: Bind to correct local address in 4.0 streaming

[jasobrown]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 05d7661d0 -> 70d95359d


Bind to correct local address in 4.0 streaming

patch by Lerh Chaun Low; reviewed by jasobrown for CASSANDRA-14362


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/70d95359
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/70d95359
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/70d95359

Branch: refs/heads/trunk
Commit: 70d95359d2dca1c35f573776d11ed87bb9b4b441
Parents: 05d7661
Author: Lerh Chuan Low 
Authored: Tue Apr 3 16:39:11 2018 +1000
Committer: Jason Brown 
Committed: Tue Apr 17 05:44:03 2018 -0700

--
 CHANGES.txt| 1 +
 src/java/org/apache/cassandra/streaming/StreamSession.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/70d95359/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 4c557b4..5e36674 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0
+ * Bind to correct local address in 4.0 streaming (CASSANDRA-14362)
  * Use standard Amazon naming for datacenter and rack in Ec2Snitch 
(CASSANDRA-7839)
  * Fix junit failure for SSTableReaderTest (CASSANDRA-14387)
  * Abstract write path for pluggable storage (CASSANDRA-14118)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/70d95359/src/java/org/apache/cassandra/streaming/StreamSession.java
--
diff --git a/src/java/org/apache/cassandra/streaming/StreamSession.java 
b/src/java/org/apache/cassandra/streaming/StreamSession.java
index adf5d76..42d1d97 100644
--- a/src/java/org/apache/cassandra/streaming/StreamSession.java
+++ b/src/java/org/apache/cassandra/streaming/StreamSession.java
@@ -183,7 +183,7 @@ public class StreamSession implements 
IEndpointStateChangeSubscriber
 this.connecting = connecting;
 this.index = index;
 
-OutboundConnectionIdentifier id = 
OutboundConnectionIdentifier.stream(InetAddressAndPort.getByAddressOverrideDefaults(FBUtilities.getBroadcastAddressAndPort().address,
 0),
+OutboundConnectionIdentifier id = 
OutboundConnectionIdentifier.stream(InetAddressAndPort.getByAddressOverrideDefaults(FBUtilities.getJustLocalAddress(),
 0),
   
InetAddressAndPort.getByAddressOverrideDefaults(connecting.address, 
MessagingService.instance().portFor(connecting)));
 this.messageSender = new NettyStreamingMessageSender(this, id, 
factory, StreamMessage.CURRENT_VERSION, previewKind.isPreview());
 this.metrics = StreamingMetrics.get(connecting);


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14362) Bind to correct local address in 4.0 streaming

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-14362:

Summary: Bind to correct local address in 4.0 streaming  (was: Node unable 
to join cluster in trunk)

> Bind to correct local address in 4.0 streaming
> --
>
> Key: CASSANDRA-14362
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14362
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Lerh Chuan Low
>Assignee: Jason Brown
>Priority: Major
> Fix For: 4.0
>
>
> I'm not sure if anybody has tried using {{trunk}} and starting an actual EC2 
> cluster, but with a 3 node setup that works on 3.11, it fails to work on 
> {{trunk}}. I mistakenly stumbled into it while testing my own code changes. 
> My setup is as follows:
> broadcast_rpc_address: publicIP
> broadcast_address: publicIP
> listen_address: omitted. Ends up as privateIP. 
> Works on 3.11 just fine. 
> On {{trunk}} though, it never works. My node is never able to join the 
> cluster:
> {code}
> Apr 03 05:57:06 ip-10-0-47-122 cassandra[13914]: INFO  [main] 2018-04-03 
> 05:57:05,895 RangeStreamer.java:195 - Bootstrap: range 
> (-128373781239966537,-122439194129870521] exists on 52.88.241.181:7000 for 
> keyspace system_traces
> Apr 03 05:57:06 ip-10-0-47-122 cassandra[13914]: INFO  [main] 2018-04-03 
> 05:57:05,895 RangeStreamer.java:195 - Bootstrap: range 
> (6968670424536541270,6973888347502882935] exists on 52.88.241.181:7000 for 
> keyspace system_traces
> Apr 03 05:57:42 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:57:42,298 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 26215173446ns > 50ns
> Apr 03 05:57:53 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:57:53,035 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 10736485907ns > 50ns
> Apr 03 05:58:30 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:58:30,790 Gossiper.java:814 - Gossip stage has 28 pending 
> tasks; skipping status check (no nodes will be marked down)
> Apr 03 05:58:33 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 05:58:33,060 Gossiper.java:814 - Gossip stage has 20 pending 
> tasks; skipping status check (no nodes will be marked down)
> Apr 03 06:04:33 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 06:04:33,826 FailureDetector.java:324 - Not marking nodes down due 
> to local pause of 400790432954ns > 50ns
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: WARN  [GossipTasks:1] 
> 2018-04-03 06:04:49,133 Gossiper.java:814 - Gossip stage has 2 pending tasks; 
> skipping status check (no nodes will be marked down)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: ERROR 
> [StreamConnectionEstablisher:1] 2018-04-03 06:04:49,138 
> StreamSession.java:524 - [Stream #d4cd6420-3703-11e8-a6a5-e51ddc10cfe6] 
> Streaming error occurred on session with peer 52.88.241.181:7000
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: java.io.IOException: failed 
> to connect to 52.88.241.181:7000 (STREAM) for streaming data
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:98)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:57)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.createChannel(NettyStreamingMessageSender.java:183)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.setupControlMessageChannel(NettyStreamingMessageSender.java:165)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.sendMessage(NettyStreamingMessageSender.java:222)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.async.NettyStreamingMessageSender.initialize(NettyStreamingMessageSender.java:146)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:271)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:273)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> Apr 03 06:04:49 ip-10-0-47-122 cassandra[13914]: at 
> 

[jira] [Resolved] (CASSANDRA-14286) IndexOutOfBoundsException with SELECT JSON using IN and ORDER BY

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer resolved CASSANDRA-14286.

Resolution: Fixed

Committed into 2.2 at 594cde7937de6f848998bac35d35591f8a18aa10 and merged into 
3.0, 3.11 and trunk.

> IndexOutOfBoundsException with SELECT JSON using IN and ORDER BY
> 
>
> Key: CASSANDRA-14286
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14286
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Kubernetes cluster using cassandra:3.11.1 Docker image.
>Reporter: Szymon Acedański
>Assignee: Francisco Fernandez
>Priority: Major
> Attachments: orderbug-traceback.txt
>
>
> When running the following code:
> {code}
> public class CassandraJsonOrderingBug {
> public static void main(String[] args) {
> Session session = CassandraFactory.getSession();
> session.execute("CREATE TABLE thebug ( PRIMARY KEY (a, b), a INT, b 
> INT)");
> try {
> session.execute("INSERT INTO thebug (a, b) VALUES (20, 30)");
> session.execute("INSERT INTO thebug (a, b) VALUES (100, 200)");
> Statement statement = new SimpleStatement("SELECT JSON a, b FROM 
> thebug WHERE a IN (20, 100) ORDER BY b");
> statement.setFetchSize(Integer.MAX_VALUE);
> for (Row w: session.execute(statement)) {
> System.out.println(w.toString());
> }
> } finally {
> session.execute("DROP TABLE thebug");
> }
> }
> }
> {code}
> The following exception is thrown server-side:
> {noformat}
> java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
>   at java.util.Collections$SingletonList.get(Collections.java:4815) 
> ~[na:1.8.0_151]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement$SingleColumnComparator.compare(SelectStatement.java:1297)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement$SingleColumnComparator.compare(SelectStatement.java:1284)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
>   at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355) 
> ~[na:1.8.0_151]
>   at java.util.TimSort.sort(TimSort.java:220) ~[na:1.8.0_151]
>   at java.util.Arrays.sort(Arrays.java:1512) ~[na:1.8.0_151]
>   at java.util.ArrayList.sort(ArrayList.java:1460) ~[na:1.8.0_151]
>   at java.util.Collections.sort(Collections.java:175) ~[na:1.8.0_151]
> {noformat}
> (full traceback attached)
> The accessed index is the index of the sorted column in the SELECT JSON 
> fields list.
> Similarly, if the select clause is changed to
> SELECT JSON b, a FROM thebug WHERE a IN (20, 100) ORDER BY b
> then the query finishes, but the output is sorted incorrectly (by textual 
> JSON representation):
> {noformat}
> Row[{"b": 200, "a": 100}]
> Row[{"b": 30, "a": 20}]
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[05/10] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0

[blerer]

Merge branch cassandra-2.2 into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/598008d4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/598008d4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/598008d4

Branch: refs/heads/cassandra-3.11
Commit: 598008d43ce88ea276974ec0f247faa0d79c
Parents: 22bb413 594cde7
Author: Benjamin Lerer 
Authored: Tue Apr 17 12:12:48 2018 +0200
Committer: Benjamin Lerer 
Committed: Tue Apr 17 12:12:48 2018 +0200

--
 CHANGES.txt |  2 +
 .../org/apache/cassandra/cql3/ResultSet.java|  5 ++
 .../cassandra/cql3/selection/Selection.java | 69 +---
 .../cql3/statements/SelectStatement.java| 19 +++---
 .../cql3/validation/entities/JsonTest.java  | 21 ++
 5 files changed, 97 insertions(+), 19 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/598008d4/CHANGES.txt
--
diff --cc CHANGES.txt
index 9012f8c,967ee05..d3d8036
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,20 -1,7 +1,22 @@@
 -2.2.13
 +3.0.17
 + * Avoid deadlock when running nodetool refresh before node is fully up 
(CASSANDRA-14310)
 + * Handle all exceptions when opening sstables (CASSANDRA-14202)
 + * Handle incompletely written hint descriptors during startup 
(CASSANDRA-14080)
 + * Handle repeat open bound from SRP in read repair (CASSANDRA-14330)
 + * Use zero as default score in DynamicEndpointSnitch (CASSANDRA-14252)
 + * Respect max hint window when hinting for LWT (CASSANDRA-14215)
 + * Adding missing WriteType enum values to v3, v4, and v5 spec 
(CASSANDRA-13697)
 + * Don't regenerate bloomfilter and summaries on startup (CASSANDRA-11163)
 + * Fix NPE when performing comparison against a null frozen in LWT 
(CASSANDRA-14087)
 + * Log when SSTables are deleted (CASSANDRA-14302)
 + * Fix batch commitlog sync regression (CASSANDRA-14292)
 + * Write to pending endpoint when view replica is also base replica 
(CASSANDRA-14251)
 + * Chain commit log marker potential performance regression in batch commit 
mode (CASSANDRA-14194)
 + * Fully utilise specified compaction threads (CASSANDRA-14210)
 + * Pre-create deletion log records to finish compactions quicker 
(CASSANDRA-12763)
 +Merged from 2.2:
+  * Fix JSON queries with IN restrictions and ORDER BY clause (CASSANDRA-14286)
+  * CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)
 - * Fix query pager DEBUG log leak causing hit in paged reads throughput 
(CASSANDRA-14318)
   * Backport circleci yaml (CASSANDRA-14240)
  Merged from 2.1:
   * Check checksum before decompressing data (CASSANDRA-14284)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/598008d4/src/java/org/apache/cassandra/cql3/ResultSet.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/598008d4/src/java/org/apache/cassandra/cql3/selection/Selection.java
--
diff --cc src/java/org/apache/cassandra/cql3/selection/Selection.java
index 406f849,5385fc6..6227158
--- a/src/java/org/apache/cassandra/cql3/selection/Selection.java
+++ b/src/java/org/apache/cassandra/cql3/selection/Selection.java
@@@ -399,8 -434,17 +450,10 @@@ public abstract class Selectio
  sb.append(spec.type.toJSONString(buffer, 
protocolVersion));
  }
  sb.append("}");
- return 
Collections.singletonList(UTF8Type.instance.getSerializer().serialize(sb.toString()));
+ List jsonRow = new ArrayList<>();
+ 
jsonRow.add(UTF8Type.instance.getSerializer().serialize(sb.toString()));
+ return jsonRow;
  }
 -
 -private ByteBuffer value(Cell c)
 -{
 -return (c instanceof CounterCell)
 -? 
ByteBufferUtil.bytes(CounterContext.instance().total(c.value()))
 -: c.value();
 -}
  }
  
  private static interface Selectors

http://git-wip-us.apache.org/repos/asf/cassandra/blob/598008d4/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --cc src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 1e867bc,729cf83..a5e6254
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@@ -911,14 -841,14 +911,14 @@@ public class SelectStatement implement
  
  if (!parameters.orderings.isEmpty())
  {
 +assert !forView;
  

[08/10] cassandra git commit: Merge branch cassandra-3.0 into cassandra-3.11

[blerer]

Merge branch cassandra-3.0 into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/95cfee62
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/95cfee62
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/95cfee62

Branch: refs/heads/cassandra-3.11
Commit: 95cfee623456f1df51ce548cc6cf42fe5a78050c
Parents: 845243d 598008d
Author: Benjamin Lerer 
Authored: Tue Apr 17 12:14:58 2018 +0200
Committer: Benjamin Lerer 
Committed: Tue Apr 17 12:27:25 2018 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/cql3/ResultSet.java|  5 ++
 .../cassandra/cql3/selection/Selection.java | 70 +---
 .../cql3/statements/SelectStatement.java| 20 +++---
 .../cql3/validation/entities/JsonTest.java  | 41 +++-
 5 files changed, 116 insertions(+), 21 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/95cfee62/CHANGES.txt
--
diff --cc CHANGES.txt
index 4c513d7,d3d8036..42ea3b4
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -25,6 -15,8 +25,7 @@@ Merged from 3.0
   * Fully utilise specified compaction threads (CASSANDRA-14210)
   * Pre-create deletion log records to finish compactions quicker 
(CASSANDRA-12763)
  Merged from 2.2:
+  * Fix JSON queries with IN restrictions and ORDER BY clause (CASSANDRA-14286)
 - * CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)
   * Backport circleci yaml (CASSANDRA-14240)
  Merged from 2.1:
   * Check checksum before decompressing data (CASSANDRA-14284)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/95cfee62/src/java/org/apache/cassandra/cql3/ResultSet.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/95cfee62/src/java/org/apache/cassandra/cql3/selection/Selection.java
--
diff --cc src/java/org/apache/cassandra/cql3/selection/Selection.java
index 7b4e80c,6227158..fb0b60c
--- a/src/java/org/apache/cassandra/cql3/selection/Selection.java
+++ b/src/java/org/apache/cassandra/cql3/selection/Selection.java
@@@ -271,41 -288,13 +311,43 @@@ public abstract class Selectio
  @Override
  public String toString()
  {
 -return Objects.toStringHelper(this)
 -.add("columns", columns)
 -.add("columnMapping", columnMapping)
 -.add("metadata", metadata)
 -.add("collectTimestamps", collectTimestamps)
 -.add("collectTTLs", collectTTLs)
 -.toString();
 +return MoreObjects.toStringHelper(this)
 +  .add("columns", columns)
 +  .add("columnMapping", columnMapping)
 +  .add("metadata", metadata)
 +  .add("collectTimestamps", collectTimestamps)
 +  .add("collectTTLs", collectTTLs)
 +  .toString();
 +}
 +
 +public static List rowToJson(List row, 
ProtocolVersion protocolVersion, ResultSet.ResultMetadata metadata)
 +{
 +StringBuilder sb = new StringBuilder("{");
- for (int i = 0; i < metadata.names.size(); i++)
++for (int i = 0; i < metadata.getColumnCount(); i++)
 +{
 +if (i > 0)
 +sb.append(", ");
 +
 +ColumnSpecification spec = metadata.names.get(i);
 +String columnName = spec.name.toString();
 +if (!columnName.equals(columnName.toLowerCase(Locale.US)))
 +columnName = "\"" + columnName + "\"";
 +
 +ByteBuffer buffer = row.get(i);
 +sb.append('"');
 +sb.append(Json.quoteAsJsonString(columnName));
 +sb.append("\": ");
 +if (buffer == null)
 +sb.append("null");
 +else if (!buffer.hasRemaining())
 +sb.append("\"\"");
 +else
 +sb.append(spec.type.toJSONString(buffer, protocolVersion));
 +}
 +sb.append("}");
- return 
Collections.singletonList(UTF8Type.instance.getSerializer().serialize(sb.toString()));
++List jsonRow = new ArrayList<>();
++
jsonRow.add(UTF8Type.instance.getSerializer().serialize(sb.toString()));
++return jsonRow;
  }
  
  public class ResultSetBuilder
@@@ -445,12 -409,51 +487,24 @@@
  return resultSet;
  }
  
 -private List getOutputRow(int protocolVersion)
 +private List getOutputRow()
  {
  List outputRow = 
selectors.getOutputRow(protocolVersion);
- return isJson ? 

[06/10] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0

[blerer]

Merge branch cassandra-2.2 into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/598008d4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/598008d4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/598008d4

Branch: refs/heads/cassandra-3.0
Commit: 598008d43ce88ea276974ec0f247faa0d79c
Parents: 22bb413 594cde7
Author: Benjamin Lerer 
Authored: Tue Apr 17 12:12:48 2018 +0200
Committer: Benjamin Lerer 
Committed: Tue Apr 17 12:12:48 2018 +0200

--
 CHANGES.txt |  2 +
 .../org/apache/cassandra/cql3/ResultSet.java|  5 ++
 .../cassandra/cql3/selection/Selection.java | 69 +---
 .../cql3/statements/SelectStatement.java| 19 +++---
 .../cql3/validation/entities/JsonTest.java  | 21 ++
 5 files changed, 97 insertions(+), 19 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/598008d4/CHANGES.txt
--
diff --cc CHANGES.txt
index 9012f8c,967ee05..d3d8036
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,20 -1,7 +1,22 @@@
 -2.2.13
 +3.0.17
 + * Avoid deadlock when running nodetool refresh before node is fully up 
(CASSANDRA-14310)
 + * Handle all exceptions when opening sstables (CASSANDRA-14202)
 + * Handle incompletely written hint descriptors during startup 
(CASSANDRA-14080)
 + * Handle repeat open bound from SRP in read repair (CASSANDRA-14330)
 + * Use zero as default score in DynamicEndpointSnitch (CASSANDRA-14252)
 + * Respect max hint window when hinting for LWT (CASSANDRA-14215)
 + * Adding missing WriteType enum values to v3, v4, and v5 spec 
(CASSANDRA-13697)
 + * Don't regenerate bloomfilter and summaries on startup (CASSANDRA-11163)
 + * Fix NPE when performing comparison against a null frozen in LWT 
(CASSANDRA-14087)
 + * Log when SSTables are deleted (CASSANDRA-14302)
 + * Fix batch commitlog sync regression (CASSANDRA-14292)
 + * Write to pending endpoint when view replica is also base replica 
(CASSANDRA-14251)
 + * Chain commit log marker potential performance regression in batch commit 
mode (CASSANDRA-14194)
 + * Fully utilise specified compaction threads (CASSANDRA-14210)
 + * Pre-create deletion log records to finish compactions quicker 
(CASSANDRA-12763)
 +Merged from 2.2:
+  * Fix JSON queries with IN restrictions and ORDER BY clause (CASSANDRA-14286)
+  * CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)
 - * Fix query pager DEBUG log leak causing hit in paged reads throughput 
(CASSANDRA-14318)
   * Backport circleci yaml (CASSANDRA-14240)
  Merged from 2.1:
   * Check checksum before decompressing data (CASSANDRA-14284)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/598008d4/src/java/org/apache/cassandra/cql3/ResultSet.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/598008d4/src/java/org/apache/cassandra/cql3/selection/Selection.java
--
diff --cc src/java/org/apache/cassandra/cql3/selection/Selection.java
index 406f849,5385fc6..6227158
--- a/src/java/org/apache/cassandra/cql3/selection/Selection.java
+++ b/src/java/org/apache/cassandra/cql3/selection/Selection.java
@@@ -399,8 -434,17 +450,10 @@@ public abstract class Selectio
  sb.append(spec.type.toJSONString(buffer, 
protocolVersion));
  }
  sb.append("}");
- return 
Collections.singletonList(UTF8Type.instance.getSerializer().serialize(sb.toString()));
+ List jsonRow = new ArrayList<>();
+ 
jsonRow.add(UTF8Type.instance.getSerializer().serialize(sb.toString()));
+ return jsonRow;
  }
 -
 -private ByteBuffer value(Cell c)
 -{
 -return (c instanceof CounterCell)
 -? 
ByteBufferUtil.bytes(CounterContext.instance().total(c.value()))
 -: c.value();
 -}
  }
  
  private static interface Selectors

http://git-wip-us.apache.org/repos/asf/cassandra/blob/598008d4/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --cc src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 1e867bc,729cf83..a5e6254
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@@ -911,14 -841,14 +911,14 @@@ public class SelectStatement implement
  
  if (!parameters.orderings.isEmpty())
  {
 +assert !forView;
  

[02/10] cassandra git commit: Fix JSON queries with IN restrictions and ORDER BY clause

[blerer]

Fix JSON queries with IN restrictions and ORDER BY clause

patch by Francisco Fernandez; reviewed by Benjamin Lerer for CASSANDRA-14286


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/594cde79
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/594cde79
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/594cde79

Branch: refs/heads/cassandra-3.0
Commit: 594cde7937de6f848998bac35d35591f8a18aa10
Parents: b3ac793
Author: Francisco Fernandez 
Authored: Tue Apr 17 12:02:25 2018 +0200
Committer: Benjamin Lerer 
Committed: Tue Apr 17 12:04:33 2018 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/cql3/ResultSet.java|  5 ++
 .../cassandra/cql3/selection/Selection.java | 69 +---
 .../cql3/statements/SelectStatement.java| 19 +++---
 .../cql3/validation/entities/JsonTest.java  | 21 ++
 5 files changed, 96 insertions(+), 19 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/594cde79/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5221b1e..967ee05 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.13
+ * Fix JSON queries with IN restrictions and ORDER BY clause (CASSANDRA-14286)
  * CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)
  * Fix query pager DEBUG log leak causing hit in paged reads throughput 
(CASSANDRA-14318)
  * Backport circleci yaml (CASSANDRA-14240)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/594cde79/src/java/org/apache/cassandra/cql3/ResultSet.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ResultSet.java 
b/src/java/org/apache/cassandra/cql3/ResultSet.java
index 028691f..16f0d1b 100644
--- a/src/java/org/apache/cassandra/cql3/ResultSet.java
+++ b/src/java/org/apache/cassandra/cql3/ResultSet.java
@@ -254,6 +254,11 @@ public class ResultSet
 return new ResultMetadata(EnumSet.copyOf(flags), names, 
columnCount, pagingState);
 }
 
+public int getColumnCount()
+{
+return columnCount;
+}
+
 /**
  * Return only the column names requested by the user, excluding those 
added for post-query re-orderings,
  * see definition of names and columnCount.

http://git-wip-us.apache.org/repos/asf/cassandra/blob/594cde79/src/java/org/apache/cassandra/cql3/selection/Selection.java
--
diff --git a/src/java/org/apache/cassandra/cql3/selection/Selection.java 
b/src/java/org/apache/cassandra/cql3/selection/Selection.java
index dc4e240..5385fc6 100644
--- a/src/java/org/apache/cassandra/cql3/selection/Selection.java
+++ b/src/java/org/apache/cassandra/cql3/selection/Selection.java
@@ -24,6 +24,7 @@ import com.google.common.base.Objects;
 import com.google.common.base.Predicate;
 import com.google.common.collect.Iterables;
 import com.google.common.collect.Iterators;
+import com.google.common.collect.Lists;
 
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.ColumnDefinition;
@@ -56,6 +57,8 @@ public abstract class Selection
 private final ResultSet.ResultMetadata metadata;
 private final boolean collectTimestamps;
 private final boolean collectTTLs;
+// Columns used to order the result set for multi-partition queries
+private Map orderingIndex;
 
 protected Selection(CFMetaData cfm,
 List columns,
@@ -130,6 +133,24 @@ public abstract class Selection
 return false;
 }
 
+public Map getOrderingIndex(boolean isJson)
+{
+if (!isJson)
+return orderingIndex;
+
+// If we order post-query in json, the first and only column that we 
ship to the client is the json column.
+// In that case, we should keep ordering columns around to perform the 
ordering, then these columns will
+// be placed after the json column. As a consequence of where the 
colums are placed, we should give the
+// ordering index a value based on their position in the json encoding 
and discard the original index.
+// (CASSANDRA-14286)
+int columnIndex = 1;
+Map jsonOrderingIndex = new 
LinkedHashMap<>(orderingIndex.size());
+for (ColumnDefinition column : orderingIndex.keySet())
+jsonOrderingIndex.put(column, columnIndex++);
+
+return jsonOrderingIndex;
+}
+
 public ResultSet.ResultMetadata getResultMetadata(boolean isJson)
 {
 if 

[04/10] cassandra git commit: Fix JSON queries with IN restrictions and ORDER BY clause

[blerer]

Fix JSON queries with IN restrictions and ORDER BY clause

patch by Francisco Fernandez; reviewed by Benjamin Lerer for CASSANDRA-14286


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/594cde79
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/594cde79
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/594cde79

Branch: refs/heads/trunk
Commit: 594cde7937de6f848998bac35d35591f8a18aa10
Parents: b3ac793
Author: Francisco Fernandez 
Authored: Tue Apr 17 12:02:25 2018 +0200
Committer: Benjamin Lerer 
Committed: Tue Apr 17 12:04:33 2018 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/cql3/ResultSet.java|  5 ++
 .../cassandra/cql3/selection/Selection.java | 69 +---
 .../cql3/statements/SelectStatement.java| 19 +++---
 .../cql3/validation/entities/JsonTest.java  | 21 ++
 5 files changed, 96 insertions(+), 19 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/594cde79/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5221b1e..967ee05 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.13
+ * Fix JSON queries with IN restrictions and ORDER BY clause (CASSANDRA-14286)
  * CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)
  * Fix query pager DEBUG log leak causing hit in paged reads throughput 
(CASSANDRA-14318)
  * Backport circleci yaml (CASSANDRA-14240)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/594cde79/src/java/org/apache/cassandra/cql3/ResultSet.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ResultSet.java 
b/src/java/org/apache/cassandra/cql3/ResultSet.java
index 028691f..16f0d1b 100644
--- a/src/java/org/apache/cassandra/cql3/ResultSet.java
+++ b/src/java/org/apache/cassandra/cql3/ResultSet.java
@@ -254,6 +254,11 @@ public class ResultSet
 return new ResultMetadata(EnumSet.copyOf(flags), names, 
columnCount, pagingState);
 }
 
+public int getColumnCount()
+{
+return columnCount;
+}
+
 /**
  * Return only the column names requested by the user, excluding those 
added for post-query re-orderings,
  * see definition of names and columnCount.

http://git-wip-us.apache.org/repos/asf/cassandra/blob/594cde79/src/java/org/apache/cassandra/cql3/selection/Selection.java
--
diff --git a/src/java/org/apache/cassandra/cql3/selection/Selection.java 
b/src/java/org/apache/cassandra/cql3/selection/Selection.java
index dc4e240..5385fc6 100644
--- a/src/java/org/apache/cassandra/cql3/selection/Selection.java
+++ b/src/java/org/apache/cassandra/cql3/selection/Selection.java
@@ -24,6 +24,7 @@ import com.google.common.base.Objects;
 import com.google.common.base.Predicate;
 import com.google.common.collect.Iterables;
 import com.google.common.collect.Iterators;
+import com.google.common.collect.Lists;
 
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.ColumnDefinition;
@@ -56,6 +57,8 @@ public abstract class Selection
 private final ResultSet.ResultMetadata metadata;
 private final boolean collectTimestamps;
 private final boolean collectTTLs;
+// Columns used to order the result set for multi-partition queries
+private Map orderingIndex;
 
 protected Selection(CFMetaData cfm,
 List columns,
@@ -130,6 +133,24 @@ public abstract class Selection
 return false;
 }
 
+public Map getOrderingIndex(boolean isJson)
+{
+if (!isJson)
+return orderingIndex;
+
+// If we order post-query in json, the first and only column that we 
ship to the client is the json column.
+// In that case, we should keep ordering columns around to perform the 
ordering, then these columns will
+// be placed after the json column. As a consequence of where the 
colums are placed, we should give the
+// ordering index a value based on their position in the json encoding 
and discard the original index.
+// (CASSANDRA-14286)
+int columnIndex = 1;
+Map jsonOrderingIndex = new 
LinkedHashMap<>(orderingIndex.size());
+for (ColumnDefinition column : orderingIndex.keySet())
+jsonOrderingIndex.put(column, columnIndex++);
+
+return jsonOrderingIndex;
+}
+
 public ResultSet.ResultMetadata getResultMetadata(boolean isJson)
 {
 if (!isJson)
@@ 

[10/10] cassandra git commit: Merge branch cassandra-3.11 into trunk

[blerer]

Merge branch cassandra-3.11 into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/05d7661d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/05d7661d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/05d7661d

Branch: refs/heads/trunk
Commit: 05d7661d04b2cd47b29a767986d4b6d283419db2
Parents: 09f3c96 95cfee6
Author: Benjamin Lerer 
Authored: Tue Apr 17 12:28:16 2018 +0200
Committer: Benjamin Lerer 
Committed: Tue Apr 17 12:30:53 2018 +0200

--
 CHANGES.txt |  6 +-
 .../cassandra/cql3/selection/Selection.java | 66 +---
 .../cql3/statements/SelectStatement.java| 23 +--
 .../cql3/validation/entities/JsonTest.java  | 39 
 4 files changed, 103 insertions(+), 31 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/05d7661d/CHANGES.txt
--
diff --cc CHANGES.txt
index 71d947a,42ea3b4..4c557b4
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,229 -1,7 +1,228 @@@
 +4.0
 + * Use standard Amazon naming for datacenter and rack in Ec2Snitch 
(CASSANDRA-7839)
 + * Fix junit failure for SSTableReaderTest (CASSANDRA-14387)
 + * Abstract write path for pluggable storage (CASSANDRA-14118)
 + * nodetool describecluster should be more informative (CASSANDRA-13853)
 + * Compaction performance improvements (CASSANDRA-14261) 
 + * Refactor Pair usage to avoid boxing ints/longs (CASSANDRA-14260)
 + * Add options to nodetool tablestats to sort and limit output 
(CASSANDRA-13889)
 + * Rename internals to reflect CQL vocabulary (CASSANDRA-14354)
 + * Add support for hybrid MIN(), MAX() speculative retry policies
 +   (CASSANDRA-14293, CASSANDRA-14338, CASSANDRA-14352)
 + * Fix some regressions caused by 14058 (CASSANDRA-14353)
 + * Abstract repair for pluggable storage (CASSANDRA-14116)
 + * Add meaningful toString() impls (CASSANDRA-13653)
 + * Add sstableloader option to accept target keyspace name (CASSANDRA-13884)
 + * Move processing of EchoMessage response to gossip stage (CASSANDRA-13713)
 + * Add coordinator write metric per CF (CASSANDRA-14232)
 + * Correct and clarify SSLFactory.getSslContext method and call sites 
(CASSANDRA-14314)
 + * Handle static and partition deletion properly on 
ThrottledUnfilteredIterator (CASSANDRA-14315)
 + * NodeTool clientstats should show SSL Cipher (CASSANDRA-14322)
 + * Add ability to specify driver name and version (CASSANDRA-14275)
 + * Abstract streaming for pluggable storage (CASSANDRA-14115)
 + * Forced incremental repairs should promote sstables if they can 
(CASSANDRA-14294)
 + * Use Murmur3 for validation compactions (CASSANDRA-14002)
 + * Comma at the end of the seed list is interpretated as localhost 
(CASSANDRA-14285)
 + * Refactor read executor and response resolver, abstract read repair 
(CASSANDRA-14058)
 + * Add optional startup delay to wait until peers are ready (CASSANDRA-13993)
 + * Add a few options to nodetool verify (CASSANDRA-14201)
 + * CVE-2017-5929 Security vulnerability and redefine default log rotation 
policy (CASSANDRA-14183)
 + * Use JVM default SSL validation algorithm instead of custom default 
(CASSANDRA-13259)
 + * Better document in code InetAddressAndPort usage post 7544, incorporate 
port into UUIDGen node (CASSANDRA-14226)
 + * Fix sstablemetadata date string for minLocalDeletionTime (CASSANDRA-14132)
 + * Make it possible to change neverPurgeTombstones during runtime 
(CASSANDRA-14214)
 + * Remove GossipDigestSynVerbHandler#doSort() (CASSANDRA-14174)
 + * Add nodetool clientlist (CASSANDRA-13665)
 + * Revert ProtocolVersion changes from CASSANDRA-7544 (CASSANDRA-14211)
 + * Non-disruptive seed node list reload (CASSANDRA-14190)
 + * Nodetool tablehistograms to print statics for all the tables 
(CASSANDRA-14185)
 + * Migrate dtests to use pytest and python3 (CASSANDRA-14134)
 + * Allow storage port to be configurable per node (CASSANDRA-7544)
 + * Make sub-range selection for non-frozen collections return null instead of 
empty (CASSANDRA-14182)
 + * BloomFilter serialization format should not change byte ordering 
(CASSANDRA-9067)
 + * Remove unused on-heap BloomFilter implementation (CASSANDRA-14152)
 + * Delete temp test files on exit (CASSANDRA-14153)
 + * Make PartitionUpdate and Mutation immutable (CASSANDRA-13867)
 + * Fix CommitLogReplayer exception for CDC data (CASSANDRA-14066)
 + * Fix cassandra-stress startup failure (CASSANDRA-14106)
 + * Remove initialDirectories from CFS (CASSANDRA-13928)
 + * Fix trivial log format error (CASSANDRA-14015)
 + * Allow sstabledump to do a json object per partition (CASSANDRA-13848)
 + * Add option to optimise merkle tree comparison across replicas 
(CASSANDRA-3200)
 + * 

[09/10] cassandra git commit: Merge branch cassandra-3.0 into cassandra-3.11

[blerer]

Merge branch cassandra-3.0 into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/95cfee62
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/95cfee62
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/95cfee62

Branch: refs/heads/trunk
Commit: 95cfee623456f1df51ce548cc6cf42fe5a78050c
Parents: 845243d 598008d
Author: Benjamin Lerer 
Authored: Tue Apr 17 12:14:58 2018 +0200
Committer: Benjamin Lerer 
Committed: Tue Apr 17 12:27:25 2018 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/cql3/ResultSet.java|  5 ++
 .../cassandra/cql3/selection/Selection.java | 70 +---
 .../cql3/statements/SelectStatement.java| 20 +++---
 .../cql3/validation/entities/JsonTest.java  | 41 +++-
 5 files changed, 116 insertions(+), 21 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/95cfee62/CHANGES.txt
--
diff --cc CHANGES.txt
index 4c513d7,d3d8036..42ea3b4
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -25,6 -15,8 +25,7 @@@ Merged from 3.0
   * Fully utilise specified compaction threads (CASSANDRA-14210)
   * Pre-create deletion log records to finish compactions quicker 
(CASSANDRA-12763)
  Merged from 2.2:
+  * Fix JSON queries with IN restrictions and ORDER BY clause (CASSANDRA-14286)
 - * CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)
   * Backport circleci yaml (CASSANDRA-14240)
  Merged from 2.1:
   * Check checksum before decompressing data (CASSANDRA-14284)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/95cfee62/src/java/org/apache/cassandra/cql3/ResultSet.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/95cfee62/src/java/org/apache/cassandra/cql3/selection/Selection.java
--
diff --cc src/java/org/apache/cassandra/cql3/selection/Selection.java
index 7b4e80c,6227158..fb0b60c
--- a/src/java/org/apache/cassandra/cql3/selection/Selection.java
+++ b/src/java/org/apache/cassandra/cql3/selection/Selection.java
@@@ -271,41 -288,13 +311,43 @@@ public abstract class Selectio
  @Override
  public String toString()
  {
 -return Objects.toStringHelper(this)
 -.add("columns", columns)
 -.add("columnMapping", columnMapping)
 -.add("metadata", metadata)
 -.add("collectTimestamps", collectTimestamps)
 -.add("collectTTLs", collectTTLs)
 -.toString();
 +return MoreObjects.toStringHelper(this)
 +  .add("columns", columns)
 +  .add("columnMapping", columnMapping)
 +  .add("metadata", metadata)
 +  .add("collectTimestamps", collectTimestamps)
 +  .add("collectTTLs", collectTTLs)
 +  .toString();
 +}
 +
 +public static List rowToJson(List row, 
ProtocolVersion protocolVersion, ResultSet.ResultMetadata metadata)
 +{
 +StringBuilder sb = new StringBuilder("{");
- for (int i = 0; i < metadata.names.size(); i++)
++for (int i = 0; i < metadata.getColumnCount(); i++)
 +{
 +if (i > 0)
 +sb.append(", ");
 +
 +ColumnSpecification spec = metadata.names.get(i);
 +String columnName = spec.name.toString();
 +if (!columnName.equals(columnName.toLowerCase(Locale.US)))
 +columnName = "\"" + columnName + "\"";
 +
 +ByteBuffer buffer = row.get(i);
 +sb.append('"');
 +sb.append(Json.quoteAsJsonString(columnName));
 +sb.append("\": ");
 +if (buffer == null)
 +sb.append("null");
 +else if (!buffer.hasRemaining())
 +sb.append("\"\"");
 +else
 +sb.append(spec.type.toJSONString(buffer, protocolVersion));
 +}
 +sb.append("}");
- return 
Collections.singletonList(UTF8Type.instance.getSerializer().serialize(sb.toString()));
++List jsonRow = new ArrayList<>();
++
jsonRow.add(UTF8Type.instance.getSerializer().serialize(sb.toString()));
++return jsonRow;
  }
  
  public class ResultSetBuilder
@@@ -445,12 -409,51 +487,24 @@@
  return resultSet;
  }
  
 -private List getOutputRow(int protocolVersion)
 +private List getOutputRow()
  {
  List outputRow = 
selectors.getOutputRow(protocolVersion);
- return isJson ? 

[07/10] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0

[blerer]

Merge branch cassandra-2.2 into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/598008d4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/598008d4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/598008d4

Branch: refs/heads/trunk
Commit: 598008d43ce88ea276974ec0f247faa0d79c
Parents: 22bb413 594cde7
Author: Benjamin Lerer 
Authored: Tue Apr 17 12:12:48 2018 +0200
Committer: Benjamin Lerer 
Committed: Tue Apr 17 12:12:48 2018 +0200

--
 CHANGES.txt |  2 +
 .../org/apache/cassandra/cql3/ResultSet.java|  5 ++
 .../cassandra/cql3/selection/Selection.java | 69 +---
 .../cql3/statements/SelectStatement.java| 19 +++---
 .../cql3/validation/entities/JsonTest.java  | 21 ++
 5 files changed, 97 insertions(+), 19 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/598008d4/CHANGES.txt
--
diff --cc CHANGES.txt
index 9012f8c,967ee05..d3d8036
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,20 -1,7 +1,22 @@@
 -2.2.13
 +3.0.17
 + * Avoid deadlock when running nodetool refresh before node is fully up 
(CASSANDRA-14310)
 + * Handle all exceptions when opening sstables (CASSANDRA-14202)
 + * Handle incompletely written hint descriptors during startup 
(CASSANDRA-14080)
 + * Handle repeat open bound from SRP in read repair (CASSANDRA-14330)
 + * Use zero as default score in DynamicEndpointSnitch (CASSANDRA-14252)
 + * Respect max hint window when hinting for LWT (CASSANDRA-14215)
 + * Adding missing WriteType enum values to v3, v4, and v5 spec 
(CASSANDRA-13697)
 + * Don't regenerate bloomfilter and summaries on startup (CASSANDRA-11163)
 + * Fix NPE when performing comparison against a null frozen in LWT 
(CASSANDRA-14087)
 + * Log when SSTables are deleted (CASSANDRA-14302)
 + * Fix batch commitlog sync regression (CASSANDRA-14292)
 + * Write to pending endpoint when view replica is also base replica 
(CASSANDRA-14251)
 + * Chain commit log marker potential performance regression in batch commit 
mode (CASSANDRA-14194)
 + * Fully utilise specified compaction threads (CASSANDRA-14210)
 + * Pre-create deletion log records to finish compactions quicker 
(CASSANDRA-12763)
 +Merged from 2.2:
+  * Fix JSON queries with IN restrictions and ORDER BY clause (CASSANDRA-14286)
+  * CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)
 - * Fix query pager DEBUG log leak causing hit in paged reads throughput 
(CASSANDRA-14318)
   * Backport circleci yaml (CASSANDRA-14240)
  Merged from 2.1:
   * Check checksum before decompressing data (CASSANDRA-14284)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/598008d4/src/java/org/apache/cassandra/cql3/ResultSet.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/598008d4/src/java/org/apache/cassandra/cql3/selection/Selection.java
--
diff --cc src/java/org/apache/cassandra/cql3/selection/Selection.java
index 406f849,5385fc6..6227158
--- a/src/java/org/apache/cassandra/cql3/selection/Selection.java
+++ b/src/java/org/apache/cassandra/cql3/selection/Selection.java
@@@ -399,8 -434,17 +450,10 @@@ public abstract class Selectio
  sb.append(spec.type.toJSONString(buffer, 
protocolVersion));
  }
  sb.append("}");
- return 
Collections.singletonList(UTF8Type.instance.getSerializer().serialize(sb.toString()));
+ List jsonRow = new ArrayList<>();
+ 
jsonRow.add(UTF8Type.instance.getSerializer().serialize(sb.toString()));
+ return jsonRow;
  }
 -
 -private ByteBuffer value(Cell c)
 -{
 -return (c instanceof CounterCell)
 -? 
ByteBufferUtil.bytes(CounterContext.instance().total(c.value()))
 -: c.value();
 -}
  }
  
  private static interface Selectors

http://git-wip-us.apache.org/repos/asf/cassandra/blob/598008d4/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --cc src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 1e867bc,729cf83..a5e6254
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@@ -911,14 -841,14 +911,14 @@@ public class SelectStatement implement
  
  if (!parameters.orderings.isEmpty())
  {
 +assert !forView;
  

[03/10] cassandra git commit: Fix JSON queries with IN restrictions and ORDER BY clause

[blerer]

Fix JSON queries with IN restrictions and ORDER BY clause

patch by Francisco Fernandez; reviewed by Benjamin Lerer for CASSANDRA-14286


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/594cde79
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/594cde79
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/594cde79

Branch: refs/heads/cassandra-3.11
Commit: 594cde7937de6f848998bac35d35591f8a18aa10
Parents: b3ac793
Author: Francisco Fernandez 
Authored: Tue Apr 17 12:02:25 2018 +0200
Committer: Benjamin Lerer 
Committed: Tue Apr 17 12:04:33 2018 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/cql3/ResultSet.java|  5 ++
 .../cassandra/cql3/selection/Selection.java | 69 +---
 .../cql3/statements/SelectStatement.java| 19 +++---
 .../cql3/validation/entities/JsonTest.java  | 21 ++
 5 files changed, 96 insertions(+), 19 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/594cde79/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5221b1e..967ee05 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.13
+ * Fix JSON queries with IN restrictions and ORDER BY clause (CASSANDRA-14286)
  * CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)
  * Fix query pager DEBUG log leak causing hit in paged reads throughput 
(CASSANDRA-14318)
  * Backport circleci yaml (CASSANDRA-14240)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/594cde79/src/java/org/apache/cassandra/cql3/ResultSet.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ResultSet.java 
b/src/java/org/apache/cassandra/cql3/ResultSet.java
index 028691f..16f0d1b 100644
--- a/src/java/org/apache/cassandra/cql3/ResultSet.java
+++ b/src/java/org/apache/cassandra/cql3/ResultSet.java
@@ -254,6 +254,11 @@ public class ResultSet
 return new ResultMetadata(EnumSet.copyOf(flags), names, 
columnCount, pagingState);
 }
 
+public int getColumnCount()
+{
+return columnCount;
+}
+
 /**
  * Return only the column names requested by the user, excluding those 
added for post-query re-orderings,
  * see definition of names and columnCount.

http://git-wip-us.apache.org/repos/asf/cassandra/blob/594cde79/src/java/org/apache/cassandra/cql3/selection/Selection.java
--
diff --git a/src/java/org/apache/cassandra/cql3/selection/Selection.java 
b/src/java/org/apache/cassandra/cql3/selection/Selection.java
index dc4e240..5385fc6 100644
--- a/src/java/org/apache/cassandra/cql3/selection/Selection.java
+++ b/src/java/org/apache/cassandra/cql3/selection/Selection.java
@@ -24,6 +24,7 @@ import com.google.common.base.Objects;
 import com.google.common.base.Predicate;
 import com.google.common.collect.Iterables;
 import com.google.common.collect.Iterators;
+import com.google.common.collect.Lists;
 
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.ColumnDefinition;
@@ -56,6 +57,8 @@ public abstract class Selection
 private final ResultSet.ResultMetadata metadata;
 private final boolean collectTimestamps;
 private final boolean collectTTLs;
+// Columns used to order the result set for multi-partition queries
+private Map orderingIndex;
 
 protected Selection(CFMetaData cfm,
 List columns,
@@ -130,6 +133,24 @@ public abstract class Selection
 return false;
 }
 
+public Map getOrderingIndex(boolean isJson)
+{
+if (!isJson)
+return orderingIndex;
+
+// If we order post-query in json, the first and only column that we 
ship to the client is the json column.
+// In that case, we should keep ordering columns around to perform the 
ordering, then these columns will
+// be placed after the json column. As a consequence of where the 
colums are placed, we should give the
+// ordering index a value based on their position in the json encoding 
and discard the original index.
+// (CASSANDRA-14286)
+int columnIndex = 1;
+Map jsonOrderingIndex = new 
LinkedHashMap<>(orderingIndex.size());
+for (ColumnDefinition column : orderingIndex.keySet())
+jsonOrderingIndex.put(column, columnIndex++);
+
+return jsonOrderingIndex;
+}
+
 public ResultSet.ResultMetadata getResultMetadata(boolean isJson)
 {
 if 

[01/10] cassandra git commit: Fix JSON queries with IN restrictions and ORDER BY clause

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 b3ac7937e -> 594cde793
  refs/heads/cassandra-3.0 22bb413ba -> 598008d43
  refs/heads/cassandra-3.11 845243db9 -> 95cfee623
  refs/heads/trunk 09f3c969b -> 05d7661d0


Fix JSON queries with IN restrictions and ORDER BY clause

patch by Francisco Fernandez; reviewed by Benjamin Lerer for CASSANDRA-14286


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/594cde79
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/594cde79
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/594cde79

Branch: refs/heads/cassandra-2.2
Commit: 594cde7937de6f848998bac35d35591f8a18aa10
Parents: b3ac793
Author: Francisco Fernandez 
Authored: Tue Apr 17 12:02:25 2018 +0200
Committer: Benjamin Lerer 
Committed: Tue Apr 17 12:04:33 2018 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/cql3/ResultSet.java|  5 ++
 .../cassandra/cql3/selection/Selection.java | 69 +---
 .../cql3/statements/SelectStatement.java| 19 +++---
 .../cql3/validation/entities/JsonTest.java  | 21 ++
 5 files changed, 96 insertions(+), 19 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/594cde79/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5221b1e..967ee05 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.13
+ * Fix JSON queries with IN restrictions and ORDER BY clause (CASSANDRA-14286)
  * CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)
  * Fix query pager DEBUG log leak causing hit in paged reads throughput 
(CASSANDRA-14318)
  * Backport circleci yaml (CASSANDRA-14240)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/594cde79/src/java/org/apache/cassandra/cql3/ResultSet.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ResultSet.java 
b/src/java/org/apache/cassandra/cql3/ResultSet.java
index 028691f..16f0d1b 100644
--- a/src/java/org/apache/cassandra/cql3/ResultSet.java
+++ b/src/java/org/apache/cassandra/cql3/ResultSet.java
@@ -254,6 +254,11 @@ public class ResultSet
 return new ResultMetadata(EnumSet.copyOf(flags), names, 
columnCount, pagingState);
 }
 
+public int getColumnCount()
+{
+return columnCount;
+}
+
 /**
  * Return only the column names requested by the user, excluding those 
added for post-query re-orderings,
  * see definition of names and columnCount.

http://git-wip-us.apache.org/repos/asf/cassandra/blob/594cde79/src/java/org/apache/cassandra/cql3/selection/Selection.java
--
diff --git a/src/java/org/apache/cassandra/cql3/selection/Selection.java 
b/src/java/org/apache/cassandra/cql3/selection/Selection.java
index dc4e240..5385fc6 100644
--- a/src/java/org/apache/cassandra/cql3/selection/Selection.java
+++ b/src/java/org/apache/cassandra/cql3/selection/Selection.java
@@ -24,6 +24,7 @@ import com.google.common.base.Objects;
 import com.google.common.base.Predicate;
 import com.google.common.collect.Iterables;
 import com.google.common.collect.Iterators;
+import com.google.common.collect.Lists;
 
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.ColumnDefinition;
@@ -56,6 +57,8 @@ public abstract class Selection
 private final ResultSet.ResultMetadata metadata;
 private final boolean collectTimestamps;
 private final boolean collectTTLs;
+// Columns used to order the result set for multi-partition queries
+private Map orderingIndex;
 
 protected Selection(CFMetaData cfm,
 List columns,
@@ -130,6 +133,24 @@ public abstract class Selection
 return false;
 }
 
+public Map getOrderingIndex(boolean isJson)
+{
+if (!isJson)
+return orderingIndex;
+
+// If we order post-query in json, the first and only column that we 
ship to the client is the json column.
+// In that case, we should keep ordering columns around to perform the 
ordering, then these columns will
+// be placed after the json column. As a consequence of where the 
colums are placed, we should give the
+// ordering index a value based on their position in the json encoding 
and discard the original index.
+// (CASSANDRA-14286)
+int columnIndex = 1;
+Map jsonOrderingIndex = new 
LinkedHashMap<>(orderingIndex.size());
+for (ColumnDefinition 

[jira] [Commented] (CASSANDRA-14286) IndexOutOfBoundsException with SELECT JSON using IN and ORDER BY

[Francisco Fernandez (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440645#comment-16440645
 ] 

Francisco Fernandez commented on CASSANDRA-14286:
-

I see the problem now, thanks for pointing out. The code seems less complicated 
with your approach, so +1. Only one nit: there is a {{println}} on 
{{Selection.java}}.

> IndexOutOfBoundsException with SELECT JSON using IN and ORDER BY
> 
>
> Key: CASSANDRA-14286
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14286
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Kubernetes cluster using cassandra:3.11.1 Docker image.
>Reporter: Szymon Acedański
>Assignee: Francisco Fernandez
>Priority: Major
> Attachments: orderbug-traceback.txt
>
>
> When running the following code:
> {code}
> public class CassandraJsonOrderingBug {
> public static void main(String[] args) {
> Session session = CassandraFactory.getSession();
> session.execute("CREATE TABLE thebug ( PRIMARY KEY (a, b), a INT, b 
> INT)");
> try {
> session.execute("INSERT INTO thebug (a, b) VALUES (20, 30)");
> session.execute("INSERT INTO thebug (a, b) VALUES (100, 200)");
> Statement statement = new SimpleStatement("SELECT JSON a, b FROM 
> thebug WHERE a IN (20, 100) ORDER BY b");
> statement.setFetchSize(Integer.MAX_VALUE);
> for (Row w: session.execute(statement)) {
> System.out.println(w.toString());
> }
> } finally {
> session.execute("DROP TABLE thebug");
> }
> }
> }
> {code}
> The following exception is thrown server-side:
> {noformat}
> java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
>   at java.util.Collections$SingletonList.get(Collections.java:4815) 
> ~[na:1.8.0_151]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement$SingleColumnComparator.compare(SelectStatement.java:1297)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement$SingleColumnComparator.compare(SelectStatement.java:1284)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
>   at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355) 
> ~[na:1.8.0_151]
>   at java.util.TimSort.sort(TimSort.java:220) ~[na:1.8.0_151]
>   at java.util.Arrays.sort(Arrays.java:1512) ~[na:1.8.0_151]
>   at java.util.ArrayList.sort(ArrayList.java:1460) ~[na:1.8.0_151]
>   at java.util.Collections.sort(Collections.java:175) ~[na:1.8.0_151]
> {noformat}
> (full traceback attached)
> The accessed index is the index of the sorted column in the SELECT JSON 
> fields list.
> Similarly, if the select clause is changed to
> SELECT JSON b, a FROM thebug WHERE a IN (20, 100) ORDER BY b
> then the query finishes, but the output is sorted incorrectly (by textual 
> JSON representation):
> {noformat}
> Row[{"b": 200, "a": 100}]
> Row[{"b": 30, "a": 20}]
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14299) cqlsh: ssl setting not read from cqlshrc in 3.11

[Stefan Podkowinski (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Podkowinski updated CASSANDRA-14299:
---
   Resolution: Fixed
Fix Version/s: (was: 3.11.x)
   (was: 4.x)
   3.11.3
   Status: Resolved  (was: Patch Available)

Merged as 845243db93a5add273f6e on 3.11

Thx Jay!

> cqlsh: ssl setting not read from cqlshrc in 3.11 
> -
>
> Key: CASSANDRA-14299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14299
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Christian Becker
>Assignee: Stefan Podkowinski
>Priority: Major
> Fix For: 3.11.3
>
>
> With CASSANDRA-10458 an option was added to read the {{--ssl}} flag from 
> cqlshrc, however the commit seems to have been incorrectly merged or the 
> changes were dropped somehow.
> Currently adding the following has no effect:
> {code:java}
> [connection]
> ssl = true{code}
> When looking at the current tree it's obvious that the flag is not read: 
> [https://github.com/apache/cassandra/blame/cassandra-3.11/bin/cqlsh.py#L2247]
> However it should have been added with 
> [https://github.com/apache/cassandra/commit/70649a8d65825144fcdbde136d9b6354ef1fb911]
> The values like {{DEFAULT_SSL = False}}  are present, but the 
> {{option_with_default()}} call is missing.
> Git blame also shows no change to that line which would have reverted the 
> change.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[1/3] cassandra git commit: Fix cqlsh.py ssl flag regression

[spod]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.11 75a932087 -> 845243db9
  refs/heads/trunk fd44a69fc -> 09f3c969b


Fix cqlsh.py ssl flag regression

patch by Stefan Podkowinski; reviewed by Jay Zhuang for CASSANDRA-14299


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/845243db
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/845243db
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/845243db

Branch: refs/heads/cassandra-3.11
Commit: 845243db93a5add273f6edcf3cd0361c5f23cf0b
Parents: 75a9320
Author: Stefan Podkowinski 
Authored: Fri Mar 9 14:15:38 2018 +0100
Committer: Stefan Podkowinski 
Committed: Tue Apr 17 10:46:57 2018 +0200

--
 CHANGES.txt  | 1 +
 bin/cqlsh.py | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/845243db/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index e55ae28..4c513d7 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.11.3
+ * Fix cqlsh to read connection.ssl cqlshrc option again (CASSANDRA-14299)
  * Downgrade log level to trace for CommitLogSegmentManager (CASSANDRA-14370)
  * CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)
  * Serialize empty buffer as empty string for json output format 
(CASSANDRA-14245)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/845243db/bin/cqlsh.py
--
diff --git a/bin/cqlsh.py b/bin/cqlsh.py
index 3abb3ba..801a8f7 100644
--- a/bin/cqlsh.py
+++ b/bin/cqlsh.py
@@ -2244,7 +2244,7 @@ def read_options(cmdlineargs, environment):
 
 optvalues.debug = False
 optvalues.file = None
-optvalues.ssl = False
+optvalues.ssl = option_with_default(configs.getboolean, 'connection', 
'ssl', DEFAULT_SSL)
 optvalues.no_compact = False
 optvalues.encoding = option_with_default(configs.get, 'ui', 'encoding', 
UTF8)
 


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[3/3] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[spod]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/09f3c969
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/09f3c969
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/09f3c969

Branch: refs/heads/trunk
Commit: 09f3c969bc7a77f5e78a6d0cc28751c4b664f72e
Parents: fd44a69 845243d
Author: Stefan Podkowinski 
Authored: Tue Apr 17 10:48:00 2018 +0200
Committer: Stefan Podkowinski 
Committed: Tue Apr 17 10:48:00 2018 +0200

--

--



-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org