date:20181120

[jira] [Comment Edited] (CASSANDRA-14904) SSTableloader doesn't understand listening for CQL connections on multiple ports

2018-11-20 Thread Ian Cleasby (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692493#comment-16692493
 ] 

Ian Cleasby edited comment on CASSANDRA-14904 at 11/21/18 1:52 AM:
---

Patches:
[trunk|https://github.com/apache/cassandra/compare/trunk...PenguinRage:OST-129-Update-sstabletableloader-to-understand-CQL-listening-on-multiple-ports]
 
[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...PenguinRage:OST-129-Update-sstabletableloader-to-understand-CQL-listening-on-multiple-ports-3.11]


was (Author: icleasby):
Patches:
 
[trunk|https://github.com/apache/cassandra/compare/cassandra-3.11...PenguinRage:OST-129-Update-sstabletableloader-to-understand-CQL-listening-on-multiple-ports-3.11]

> SSTableloader doesn't understand listening for CQL connections on multiple 
> ports
> 
>
> Key: CASSANDRA-14904
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14904
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
>Reporter: Kurt Greaves
>Assignee: Ian Cleasby
>Priority: Minor
> Fix For: 4.0, 3.11.x
>
>
> sstableloader only searches the yaml for native_transport_port, so if 
> native_transport_port_ssl is set and encryption is enabled sstableloader will 
> fail to connect as it will use the non-SSL port for the connection.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14904) SSTableloader doesn't understand listening for CQL connections on multiple ports

2018-11-20 Thread Ian Cleasby (JIRA)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ian Cleasby updated CASSANDRA-14904:

Status: Patch Available  (was: Open)

> SSTableloader doesn't understand listening for CQL connections on multiple 
> ports
> 
>
> Key: CASSANDRA-14904
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14904
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
>Reporter: Kurt Greaves
>Assignee: Ian Cleasby
>Priority: Minor
> Fix For: 4.0, 3.11.x
>
>
> sstableloader only searches the yaml for native_transport_port, so if 
> native_transport_port_ssl is set and encryption is enabled sstableloader will 
> fail to connect as it will use the non-SSL port for the connection.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14904) SSTableloader doesn't understand listening for CQL connections on multiple ports

2018-11-20 Thread Ian Cleasby (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692493#comment-16692493
 ] 

Ian Cleasby edited comment on CASSANDRA-14904 at 11/21/18 1:49 AM:
---

Patches:
 
[trunk|https://github.com/apache/cassandra/compare/cassandra-3.11...PenguinRage:OST-129-Update-sstabletableloader-to-understand-CQL-listening-on-multiple-ports-3.11]


was (Author: icleasby):
Patches:
 [trunk
[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...PenguinRage:OST-129-Update-sstabletableloader-to-understand-CQL-listening-on-multiple-ports-3.11]
|https://github.com/apache/cassandra/compare/trunk...PenguinRage:OST-129-Update-sstabletableloader-to-understand-CQL-listening-on-multiple-ports]

> SSTableloader doesn't understand listening for CQL connections on multiple 
> ports
> 
>
> Key: CASSANDRA-14904
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14904
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
>Reporter: Kurt Greaves
>Assignee: Ian Cleasby
>Priority: Minor
> Fix For: 4.0, 3.11.x
>
>
> sstableloader only searches the yaml for native_transport_port, so if 
> native_transport_port_ssl is set and encryption is enabled sstableloader will 
> fail to connect as it will use the non-SSL port for the connection.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14904) SSTableloader doesn't understand listening for CQL connections on multiple ports

2018-11-20 Thread Ian Cleasby (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692493#comment-16692493
 ] 

Ian Cleasby edited comment on CASSANDRA-14904 at 11/21/18 1:48 AM:
---

Patches:
 [trunk
[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...PenguinRage:OST-129-Update-sstabletableloader-to-understand-CQL-listening-on-multiple-ports-3.11]
|https://github.com/apache/cassandra/compare/trunk...PenguinRage:OST-129-Update-sstabletableloader-to-understand-CQL-listening-on-multiple-ports]


was (Author: icleasby):
Patches:
[trunk|https://github.com/apache/cassandra/compare/trunk...PenguinRage:OST-129-Update-sstabletableloader-to-understand-CQL-listening-on-multiple-ports]

> SSTableloader doesn't understand listening for CQL connections on multiple 
> ports
> 
>
> Key: CASSANDRA-14904
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14904
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
>Reporter: Kurt Greaves
>Assignee: Ian Cleasby
>Priority: Minor
> Fix For: 4.0, 3.11.x
>
>
> sstableloader only searches the yaml for native_transport_port, so if 
> native_transport_port_ssl is set and encryption is enabled sstableloader will 
> fail to connect as it will use the non-SSL port for the connection.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14880) drop table and materialized view frequently get error over time

2018-11-20 Thread Jeremy Hanna (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693867#comment-16693867
 ] 

Jeremy Hanna commented on CASSANDRA-14880:
--

Also jhon, can you populate the "Reproduced In" version field of the ticket?  
There have been several fixes over time, so just to see if it has already been 
addressed directly or indirectly.

> drop table and materialized view frequently get error over time
> ---
>
> Key: CASSANDRA-14880
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14880
> Project: Cassandra
>  Issue Type: Bug
>  Components: Materialized Views
>Reporter: jhon
>Assignee: jhon
>Priority: Major
>
>     when i create table and materialized view,then i drop it, if i drop it 
> frequently it will got error: "no response received from cassandra within 
> timeout period",for example：
>  for i=0; i<100; i++ {            
>           create table;            
>           create materialized view;          
>           drop materialized view;          
>           drop table;
> how can i solve it ? 
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14346) Scheduled Repair in Cassandra

2018-11-20 Thread Joseph Lynch (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693844#comment-16693844
 ] 

Joseph Lynch commented on CASSANDRA-14346:
--

Quick update for those watching this ticket.

Looks like during the Reaper donation discussion this sort of took a different 
turn and we don't have a shepherd to get this in any time soon. As such, I 
believe the plan is to start collaborating in the management process ticket 
(CASSANDRA-14395) on something more basic like health checks instead of going 
straight to solving the hard problems of robust repair scheduling, and then 
once we build up that management process and it becomes a mature part of the 
project we will re-visit robust repair scheduling that works out of the box for 
everyone; or Reaper will have been donated / integrated at that point.

For those that need repair scheduling in production right now, I'm not sure 
what the path forward is yet. We at Netflix would like to re-work the attached 
patch and include it with our releases of Priam instead of Cassandra as a 
"distributed execution engine" for executing distributed maintenance tasks 
(repair, restart, upgrade, etc ...), but I'm not sure what the timeline there 
is and the relative prioritization vs the management process.

> Scheduled Repair in Cassandra
> -
>
> Key: CASSANDRA-14346
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14346
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Repair
>Reporter: Joseph Lynch
>Assignee: Joseph Lynch
>Priority: Major
>  Labels: 4.0-feature-freeze-review-requested, 
> CommunityFeedbackRequested
> Fix For: 4.x
>
> Attachments: ScheduledRepairV1_20180327.pdf
>
>
> There have been many attempts to automate repair in Cassandra, which makes 
> sense given that it is necessary to give our users eventual consistency. Most 
> recently CASSANDRA-10070, CASSANDRA-8911 and CASSANDRA-13924 have all looked 
> for ways to solve this problem.
> At Netflix we've built a scheduled repair service within Priam (our sidecar), 
> which we spoke about last year at NGCC. Given the positive feedback at NGCC 
> we focussed on getting it production ready and have now been using it in 
> production to repair hundreds of clusters, tens of thousands of nodes, and 
> petabytes of data for the past six months. Also based on feedback at NGCC we 
> have invested effort in figuring out how to integrate this natively into 
> Cassandra rather than open sourcing it as an external service (e.g. in Priam).
> As such, [~vinaykumarcse] and I would like to re-work and merge our 
> implementation into Cassandra, and have created a [design 
> document|https://docs.google.com/document/d/1RV4rOrG1gwlD5IljmrIq_t45rz7H3xs9GbFSEyGzEtM/edit?usp=sharing]
>  showing how we plan to make it happen, including the the user interface.
> As we work on the code migration from Priam to Cassandra, any feedback would 
> be greatly appreciated about the interface or v1 implementation features. I 
> have tried to call out in the document features which we explicitly consider 
> future work (as well as a path forward to implement them in the future) 
> because I would very much like to get this done before the 4.0 merge window 
> closes, and to do that I think aggressively pruning scope is going to be a 
> necessity.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14829) Make stop-server.bat wait for Cassandra to terminate

2018-11-20 Thread Georg Dietrich (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693813#comment-16693813
 ] 

Georg Dietrich commented on CASSANDRA-14829:


Hi [~djoshi3], I have successfully tried out this change on Cassandra 3.11.3 
and on the current trunk. I also see that avoiding a regression by having a 
test would be good, however, it may not be worth the effort getting things to 
work on Windows. (And to avoid a regression, the dtests would have to be 
executed regularly on Windows, too...)

So I'll leave everything as it is now. Thanks for review and all.

> Make stop-server.bat wait for Cassandra to terminate
> 
>
> Key: CASSANDRA-14829
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14829
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Packaging
> Environment: Windows 10
>Reporter: Georg Dietrich
>Assignee: Georg Dietrich
>Priority: Minor
>  Labels: easyfix, windows
> Fix For: 3.11.x, 4.x, 4.0.x
>
>
> While administering a single node Cassandra on Windows, I noticed that the 
> stop-server.bat script returns before the cassandra process has actually 
> terminated. For use cases like creating a script "shut down & create backup 
> of data directory without having to worry about open files, then restart", it 
> would be good to make stop-server.bat wait for Cassandra to terminate.
> All that is needed for that is to change in 
> apache-cassandra-3.11.3\bin\stop-server.bat "start /B powershell /file ..." 
> to "start /WAIT /B powershell /file ..." (additional /WAIT parameter).
> Does this sound reasonable?
> Here is the pull request: https://github.com/apache/cassandra/pull/287



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14905) if SizeEstimatesRecorder misses a 'onDropTable' notification, the size_estimates table will never be cleared for that table.

2018-11-20 Thread C. Scott Andreas (JIRA)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-14905:
-
Component/s: Metrics

> if SizeEstimatesRecorder misses a 'onDropTable' notification, the 
> size_estimates table will never be cleared for that table.
> 
>
> Key: CASSANDRA-14905
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14905
> Project: Cassandra
>  Issue Type: Bug
>  Components: Metrics
>Reporter: Aleksandr Sorokoumov
>Assignee: Aleksandr Sorokoumov
>Priority: Minor
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> if a node is down when a keyspace/table is dropped, it will receive the 
> schema notification before the size estimates listener is registered, so the 
> entries for the dropped keyspace/table will never be cleaned from the table. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14906) FastByteOperations.UnsafeOperations fails to handle read only byte buffers correctly

2018-11-20 Thread C. Scott Andreas (JIRA)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-14906:
-
Component/s: Core

> FastByteOperations.UnsafeOperations fails to handle read only byte buffers 
> correctly
> 
>
> Key: CASSANDRA-14906
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14906
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Aleksandr Sorokoumov
>Assignee: Aleksandr Sorokoumov
>Priority: Minor
> Fix For: 4.x
>
>
> If a buffer is read only it reports having no array, though it may well be 
> backed by an array. Code in UnsafeOperation works under the impression that a 
> buffer for which hasArray() == false is necessarily a direct buffer. Which is 
> not the case.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14904) SSTableloader doesn't understand listening for CQL connections on multiple ports

2018-11-20 Thread C. Scott Andreas (JIRA)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-14904:
-
Component/s: Configuration

> SSTableloader doesn't understand listening for CQL connections on multiple 
> ports
> 
>
> Key: CASSANDRA-14904
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14904
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
>Reporter: Kurt Greaves
>Assignee: Ian Cleasby
>Priority: Minor
> Fix For: 4.0, 3.11.x
>
>
> sstableloader only searches the yaml for native_transport_port, so if 
> native_transport_port_ssl is set and encryption is enabled sstableloader will 
> fail to connect as it will use the non-SSL port for the connection.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14855) Message Flusher scheduling fell off the event loop, resulting in out of memory

2018-11-20 Thread Sumanth Pasupuleti (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693502#comment-16693502
 ] 

Sumanth Pasupuleti commented on CASSANDRA-14855:


[~benedict] Here is the updated patch for review - 
[https://github.com/apache/cassandra/pull/293/files], Passing UTs: 
[https://circleci.com/gh/sumanth-pasupuleti/cassandra/185]

> Message Flusher scheduling fell off the event loop, resulting in out of memory
> --
>
> Key: CASSANDRA-14855
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14855
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.0.17
>
> Attachments: blocked_thread_pool.png, cpu.png, 
> eventloop_scheduledtasks.png, flusher running state.png, heap.png, 
> heap_dump.png, read_latency.png
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> We recently had a production issue where about 10 nodes in a 96 node cluster 
> ran out of heap. 
> From heap dump analysis, I believe there is enough evidence to indicate 
> `queued` data member of the Flusher got too big, resulting in out of memory.
> Below are specifics on what we found from the heap dump (relevant screenshots 
> attached):
> * non-empty "queued" data member of Flusher having retaining heap of 0.5GB, 
> and multiple such instances.
> * "running" data member of Flusher having "true" value
> * Size of scheduledTasks on the eventloop was 0.
> We suspect something (maybe an exception) caused the Flusher running state to 
> continue to be true, but was not able to schedule itself with the event loop.
> Could not find any ERROR in the system.log, except for following INFO logs 
> around the incident time.
> {code:java}
> INFO [epollEventLoopGroup-2-4] 2018-xx-xx xx:xx:xx,592 Message.java:619 - 
> Unexpected exception during request; channel = [id: 0x8d288811, 
> L:/xxx.xx.xxx.xxx:7104 - R:/xxx.xx.x.xx:18886]
> io.netty.channel.unix.Errors$NativeIoException: readAddress() failed: 
> Connection timed out
>  at io.netty.channel.unix.Errors.newIOException(Errors.java:117) 
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at io.netty.channel.unix.Errors.ioResult(Errors.java:138) 
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at io.netty.channel.unix.FileDescriptor.readAddress(FileDescriptor.java:175) 
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at 
> io.netty.channel.epoll.AbstractEpollChannel.doReadBytes(AbstractEpollChannel.java:238)
>  ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at 
> io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:926)
>  ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at 
> io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:397) 
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:302) 
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at 
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
>  [netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at 
> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
>  [netty-all-4.0.44.Final.jar:4.0.44.Final]
> {code}
> I would like to pursue the following proposals to fix this issue:
> # ImmediateFlusher: Backport trunk's ImmediateFlusher ( 
> [CASSANDRA-13651|https://issues.apache.org/jira/browse/CASSANDRA-13651] 
> https://github.com/apache/cassandra/commit/96ef514917e5a4829dbe864104dbc08a7d0e0cec)
>   to 3.0.x and maybe to other versions as well, since ImmediateFlusher seems 
> to be more robust than the existing Flusher as it does not depend on any 
> running state/scheduling.
> # Make "queued" data member of the Flusher bounded to avoid any potential of 
> causing out of memory due to otherwise unbounded nature.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14855) Message Flusher scheduling fell off the event loop, resulting in out of memory

2018-11-20 Thread ASF GitHub Bot (JIRA)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated CASSANDRA-14855:
---
Labels: pull-request-available  (was: )

> Message Flusher scheduling fell off the event loop, resulting in out of memory
> --
>
> Key: CASSANDRA-14855
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14855
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.0.17
>
> Attachments: blocked_thread_pool.png, cpu.png, 
> eventloop_scheduledtasks.png, flusher running state.png, heap.png, 
> heap_dump.png, read_latency.png
>
>
> We recently had a production issue where about 10 nodes in a 96 node cluster 
> ran out of heap. 
> From heap dump analysis, I believe there is enough evidence to indicate 
> `queued` data member of the Flusher got too big, resulting in out of memory.
> Below are specifics on what we found from the heap dump (relevant screenshots 
> attached):
> * non-empty "queued" data member of Flusher having retaining heap of 0.5GB, 
> and multiple such instances.
> * "running" data member of Flusher having "true" value
> * Size of scheduledTasks on the eventloop was 0.
> We suspect something (maybe an exception) caused the Flusher running state to 
> continue to be true, but was not able to schedule itself with the event loop.
> Could not find any ERROR in the system.log, except for following INFO logs 
> around the incident time.
> {code:java}
> INFO [epollEventLoopGroup-2-4] 2018-xx-xx xx:xx:xx,592 Message.java:619 - 
> Unexpected exception during request; channel = [id: 0x8d288811, 
> L:/xxx.xx.xxx.xxx:7104 - R:/xxx.xx.x.xx:18886]
> io.netty.channel.unix.Errors$NativeIoException: readAddress() failed: 
> Connection timed out
>  at io.netty.channel.unix.Errors.newIOException(Errors.java:117) 
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at io.netty.channel.unix.Errors.ioResult(Errors.java:138) 
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at io.netty.channel.unix.FileDescriptor.readAddress(FileDescriptor.java:175) 
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at 
> io.netty.channel.epoll.AbstractEpollChannel.doReadBytes(AbstractEpollChannel.java:238)
>  ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at 
> io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:926)
>  ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at 
> io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:397) 
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:302) 
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at 
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
>  [netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at 
> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
>  [netty-all-4.0.44.Final.jar:4.0.44.Final]
> {code}
> I would like to pursue the following proposals to fix this issue:
> # ImmediateFlusher: Backport trunk's ImmediateFlusher ( 
> [CASSANDRA-13651|https://issues.apache.org/jira/browse/CASSANDRA-13651] 
> https://github.com/apache/cassandra/commit/96ef514917e5a4829dbe864104dbc08a7d0e0cec)
>   to 3.0.x and maybe to other versions as well, since ImmediateFlusher seems 
> to be more robust than the existing Flusher as it does not depend on any 
> running state/scheduling.
> # Make "queued" data member of the Flusher bounded to avoid any potential of 
> causing out of memory due to otherwise unbounded nature.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14898) Key cache loading is very slow when there are many SSTables

2018-11-20 Thread Joseph Lynch (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693413#comment-16693413
 ] 

Joseph Lynch edited comment on CASSANDRA-14898 at 11/20/18 4:18 PM:


Started exploring caching the {{getSSTables}} call in [a 3.0 
branch|https://github.com/apache/cassandra/compare/cassandra-3.0...jolynch:CASSANDRA-14898_30x].
 The strategy I ended up using was to add an optional 
{{cleanupAfterDeserialize}} method to the {{CacheSerializer}} interface that 
cleans up any cached state after deserializations because I felt it was the 
least invasive change that allowed us to fix this problem. I also considered 
splitting the {{CacheSerializer}} interface into a {{CacheSerializer}} and 
{{CacheDeserializer}} class since that way the Deserializers wouldn't have to 
be static and could just throw away their state naturally at the end of a 
deserialization (since we only use them once afaik at startup) but I felt it 
was a more invasive change.

I also wrote a quick 
[microbenchmark|https://github.com/apache/cassandra/compare/cassandra-3.0...jolynch:CASSANDRA-14898_30x#diff-51c5d5cb6842a1aedbff457ba77dd424]
 that creates 1000 small sstables and then does a keycache load and as expected 
my branch is about two orders of magnitude faster. I'd expect it to reduce by 
the number of sstables which it did so that's good. Benchmark results:

status quo:
{noformat}
Benchmark Mode  Cnt
ScoreError  Units
CacheLoaderBench.keyCacheLoadTest   sample   25  
167.458 ± 24.430  ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.00sample   
125.436   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.50sample   
164.364   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.90sample   
216.583   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.95sample   
231.106   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.99sample   
236.454   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.999   sample   
236.454   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.  sample   
236.454   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p1.00sample   
236.454   ms/op
{noformat}
Pre-allocating the Hashmap:
{noformat}
Benchmark Mode  Cnt
ScoreError  Units
CacheLoaderBench.keyCacheLoadTest   sample   28  
149.801 ± 11.483  ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.00sample   
114.426   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.50sample   
147.980   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.90sample   
168.401   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.95sample   
181.941   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.99sample   
190.317   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.999   sample   
190.317   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.  sample   
190.317   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p1.00sample   
190.317   ms/op
{noformat}
Caching the entire {{getSSTables}} call and using a generation map for O(1) 
lookup:
{noformat}
Benchmark Mode   Cnt   
Score   Error  Units
CacheLoaderBench.keyCacheLoadTest   sample  1101   
3.190 ± 0.108  ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.00sample 
1.876  ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.50sample 
2.863  ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.90sample 
4.615  ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.95sample 
5.161  ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.99sample 
6.750  ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.999   sample
14.178  ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.  sample
14.205  ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p1.00sample
14.205  ms/op
{noformat}
So this would seem to indicate that the production traces are correct and the 
{{O\(n\)}} traversal of the {{getSSTables}} is the dominating factor.


was (Author: jolynch):
Started exploring caching the {{getSSTables}} call in [a 3.0 
branch|https://github.com/apache/cassandra/compare/cassandra-3.0...jolynch:CASSANDRA-14898_30x].
 The strategy I ended up using was to add an optional

[jira] [Commented] (CASSANDRA-14898) Key cache loading is very slow when there are many SSTables

2018-11-20 Thread Joseph Lynch (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693413#comment-16693413
 ] 

Joseph Lynch commented on CASSANDRA-14898:
--

Started exploring caching the {{getSSTables}} call in [a 3.0 
branch|https://github.com/apache/cassandra/compare/cassandra-3.0...jolynch:CASSANDRA-14898_30x].
 The strategy I ended up using was to add an optional 
{{cleanupAfterDeserialize}} method to the {{CacheSerializer}} interface that 
cleans up any cached state after deserializations because I felt it was the 
least invasive change that allowed us to fix this problem. I also considered 
splitting the {{CacheSerializer}} interface into a {{CacheSerializer}} and 
{{CacheDeserializer}} class since that way the Deserializers wouldn't have to 
be static and could just throw away their state naturally at the end of a 
deserialization (since we only use them once afaik at startup) but I felt it 
was a more invasive change.

I also wrote a quick 
[microbenchmark|https://github.com/apache/cassandra/compare/cassandra-3.0...jolynch:CASSANDRA-14898_30x#diff-51c5d5cb6842a1aedbff457ba77dd424]
 that creates 1000 small sstables and then does a keycache load and as expected 
my branch is about two orders of magnitude faster. I'd expect it to reduce by 
the number of sstables which it did so that's good. Benchmark results:

status quo:
{noformat}
Benchmark Mode  Cnt
ScoreError  Units
CacheLoaderBench.keyCacheLoadTest   sample   25  
167.458 ± 24.430  ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.00sample   
125.436   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.50sample   
164.364   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.90sample   
216.583   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.95sample   
231.106   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.99sample   
236.454   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.999   sample   
236.454   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.  sample   
236.454   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p1.00sample   
236.454   ms/op
{noformat}
Pre-allocating the Hashmap:
{noformat}
Benchmark Mode  Cnt
ScoreError  Units
CacheLoaderBench.keyCacheLoadTest   sample   28  
149.801 ± 11.483  ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.00sample   
114.426   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.50sample   
147.980   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.90sample   
168.401   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.95sample   
181.941   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.99sample   
190.317   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.999   sample   
190.317   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.  sample   
190.317   ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p1.00sample   
190.317   ms/op
{noformat}
Caching the entire {{getSSTables}} call and using a generation map for O(1) 
lookup:
{noformat}
Benchmark Mode   Cnt   
Score   Error  Units
CacheLoaderBench.keyCacheLoadTest   sample  1101   
3.190 ± 0.108  ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.00sample 
1.876  ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.50sample 
2.863  ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.90sample 
4.615  ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.95sample 
5.161  ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.99sample 
6.750  ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.999   sample
14.178  ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p0.  sample
14.205  ms/op
CacheLoaderBench.keyCacheLoadTest:keyCacheLoadTest·p1.00sample
14.205  ms/op
{noformat}
So this would seem to indicate that the production traces are correct and the 
{{O(n)}} traversal of the {{getSSTables}} is the dominating factor.

> Key cache loading is very slow when there are many SSTables
> ---
>
> Key: CASSANDRA-14898
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14898
> Project: Cassandra
>  Issue Type:

[jira] [Commented] (CASSANDRA-14899) Cannot perform slice reads in reverse direction against tables with clustering columns in mixed order

2018-11-20 Thread Aleksey Yeschenko (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693379#comment-16693379
 ] 

Aleksey Yeschenko commented on CASSANDRA-14899:
---

Committed as 
[cf6f7920f7742bb9a17a23ad37499d9213807d81|https://github.com/apache/cassandra/commit/cf6f7920f7742bb9a17a23ad37499d9213807d81]
 to 2.2, and merged upwards (just the unit tests bit). Cheers.

> Cannot perform slice reads in reverse direction against tables with 
> clustering columns in mixed order
> -
>
> Key: CASSANDRA-14899
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14899
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 2.2.14
>
>
> CASSANDRA-11196 accidentally broke reading from tables with mixed clustering 
> column order in the opposite direction.
> {{ReversedPrimaryKeyRestrictions::boundsAsComposites}} method attempts to 
> reverse the list returned from 
> {{PrimaryKeyRestrictionSet::boundsAsComposites}} and fails, as Guava’s 
> {{Lists::transform}} method returns a {{List}} that doesn’t support {{set()}}.
> Reproduction:
> {code}
> CREATE TABLE test.test (
> a int,
> b int,
> c int,
> PRIMARY KEY (a, b, c)
> ) WITH CLUSTERING ORDER BY (b ASC, c DESC);
> SELECT * FROM test.test WHERE a = 0 AND (b, c) > (0, 0) ORDER BY b DESC, c 
> ASC;
> > ServerError: java.lang.UnsupportedOperationException
> {code}
> {code}
> java.lang.UnsupportedOperationException: null
>   at java.util.AbstractList.set(AbstractList.java:132) ~[na:1.8.0_181]
>   at java.util.Collections.swap(Collections.java:497) ~[na:1.8.0_181]
>   at java.util.Collections.reverse(Collections.java:378) ~[na:1.8.0_181]
>   at 
> org.apache.cassandra.cql3.restrictions.ReversedPrimaryKeyRestrictions.boundsAsComposites(ReversedPrimaryKeyRestrictions.java:63)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.restrictions.StatementRestrictions.getClusteringColumnsBoundsAsComposites(StatementRestrictions.java:580)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.makeFilter(SelectStatement.java:418)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.getSliceCommands(SelectStatement.java:359)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.getPageableCommand(SelectStatement.java:191)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:172)
>  ~[main/:na]
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14899) Cannot perform slice reads in reverse direction against tables with clustering columns in mixed order

2018-11-20 Thread Aleksey Yeschenko (JIRA)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-14899:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Cannot perform slice reads in reverse direction against tables with 
> clustering columns in mixed order
> -
>
> Key: CASSANDRA-14899
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14899
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 2.2.14
>
>
> CASSANDRA-11196 accidentally broke reading from tables with mixed clustering 
> column order in the opposite direction.
> {{ReversedPrimaryKeyRestrictions::boundsAsComposites}} method attempts to 
> reverse the list returned from 
> {{PrimaryKeyRestrictionSet::boundsAsComposites}} and fails, as Guava’s 
> {{Lists::transform}} method returns a {{List}} that doesn’t support {{set()}}.
> Reproduction:
> {code}
> CREATE TABLE test.test (
> a int,
> b int,
> c int,
> PRIMARY KEY (a, b, c)
> ) WITH CLUSTERING ORDER BY (b ASC, c DESC);
> SELECT * FROM test.test WHERE a = 0 AND (b, c) > (0, 0) ORDER BY b DESC, c 
> ASC;
> > ServerError: java.lang.UnsupportedOperationException
> {code}
> {code}
> java.lang.UnsupportedOperationException: null
>   at java.util.AbstractList.set(AbstractList.java:132) ~[na:1.8.0_181]
>   at java.util.Collections.swap(Collections.java:497) ~[na:1.8.0_181]
>   at java.util.Collections.reverse(Collections.java:378) ~[na:1.8.0_181]
>   at 
> org.apache.cassandra.cql3.restrictions.ReversedPrimaryKeyRestrictions.boundsAsComposites(ReversedPrimaryKeyRestrictions.java:63)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.restrictions.StatementRestrictions.getClusteringColumnsBoundsAsComposites(StatementRestrictions.java:580)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.makeFilter(SelectStatement.java:418)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.getSliceCommands(SelectStatement.java:359)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.getPageableCommand(SelectStatement.java:191)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:172)
>  ~[main/:na]
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14899) Cannot perform slice reads in reverse direction against tables with clustering columns in mixed order

2018-11-20 Thread Aleksey Yeschenko (JIRA)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-14899:
--
Fix Version/s: (was: 2.2.x)
   2.2.14
   Status: Patch Available  (was: Open)

> Cannot perform slice reads in reverse direction against tables with 
> clustering columns in mixed order
> -
>
> Key: CASSANDRA-14899
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14899
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 2.2.14
>
>
> CASSANDRA-11196 accidentally broke reading from tables with mixed clustering 
> column order in the opposite direction.
> {{ReversedPrimaryKeyRestrictions::boundsAsComposites}} method attempts to 
> reverse the list returned from 
> {{PrimaryKeyRestrictionSet::boundsAsComposites}} and fails, as Guava’s 
> {{Lists::transform}} method returns a {{List}} that doesn’t support {{set()}}.
> Reproduction:
> {code}
> CREATE TABLE test.test (
> a int,
> b int,
> c int,
> PRIMARY KEY (a, b, c)
> ) WITH CLUSTERING ORDER BY (b ASC, c DESC);
> SELECT * FROM test.test WHERE a = 0 AND (b, c) > (0, 0) ORDER BY b DESC, c 
> ASC;
> > ServerError: java.lang.UnsupportedOperationException
> {code}
> {code}
> java.lang.UnsupportedOperationException: null
>   at java.util.AbstractList.set(AbstractList.java:132) ~[na:1.8.0_181]
>   at java.util.Collections.swap(Collections.java:497) ~[na:1.8.0_181]
>   at java.util.Collections.reverse(Collections.java:378) ~[na:1.8.0_181]
>   at 
> org.apache.cassandra.cql3.restrictions.ReversedPrimaryKeyRestrictions.boundsAsComposites(ReversedPrimaryKeyRestrictions.java:63)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.restrictions.StatementRestrictions.getClusteringColumnsBoundsAsComposites(StatementRestrictions.java:580)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.makeFilter(SelectStatement.java:418)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.getSliceCommands(SelectStatement.java:359)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.getPageableCommand(SelectStatement.java:191)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:172)
>  ~[main/:na]
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[04/10] cassandra git commit: Cannot perform slice reads in reverse direction against tables with clustering columns in mixed order

2018-11-20 Thread aleksey

Cannot perform slice reads in reverse direction against tables with clustering 
columns in mixed order

patch by Aleksey Yeschenko; reviewed by Alex Petrov for CASSANDRA-14899


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cf6f7920
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cf6f7920
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cf6f7920

Branch: refs/heads/trunk
Commit: cf6f7920f7742bb9a17a23ad37499d9213807d81
Parents: dcd92a9
Author: Aleksey Yeshchenko 
Authored: Thu Nov 15 14:54:05 2018 +
Committer: Aleksey Yeshchenko 
Committed: Tue Nov 20 15:19:28 2018 +

--
 CHANGES.txt |  5 
 .../restrictions/PrimaryKeyRestrictionSet.java  | 24 +++
 .../SelectMultiColumnRelationTest.java  | 25 
 3 files changed, 44 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf6f7920/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 71c57ea..bca036d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,7 +1,10 @@
 2.2.14
+ * Cannot perform slice reads in reverse direction against tables with 
clustering columns
+   in mixed order (CASSANDRA-14899)
  * Fix incorrect cqlsh results when selecting same columns multiple times 
(CASSANDRA-13262)
  * Returns null instead of NaN or Infinity in JSON strings (CASSANDRA-14377)
 
+
 2.2.13
  * Fix bug that prevented compaction of SSTables after full repairs 
(CASSANDRA-14423)
  * Incorrect counting of pending messages in OutboundTcpConnection 
(CASSANDRA-11551)
@@ -15,6 +18,7 @@ Merged from 2.1:
  * Check checksum before decompressing data (CASSANDRA-14284)
  * CVE-2017-5929 Security vulnerability in Logback warning in NEWS.txt 
(CASSANDRA-14183)
 
+
 2.2.12
  * Fix the inspectJvmOptions startup check (CASSANDRA-14112)
  * Fix race that prevents submitting compaction for a table when executor is 
full (CASSANDRA-13801)
@@ -25,6 +29,7 @@ Merged from 2.1:
  * More PEP8 compliance for cqlsh (CASSANDRA-14021)
  * RPM package spec: fix permissions for installed jars and config files 
(CASSANDRA-14181)
 
+
 2.2.11
  * Safely handle empty buffers when outputting to JSON (CASSANDRA-13868)
  * Copy session properties on cqlsh.py do_login (CASSANDRA-13847)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf6f7920/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
--
diff --git 
a/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java 
b/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
index 2549bdf..5136fee 100644
--- 
a/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
+++ 
b/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
@@ -20,8 +20,6 @@ package org.apache.cassandra.cql3.restrictions;
 import java.nio.ByteBuffer;
 import java.util.*;
 
-import com.google.common.collect.Lists;
-
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.ColumnDefinition;
 import org.apache.cassandra.cql3.QueryOptions;
@@ -195,14 +193,12 @@ final class PrimaryKeyRestrictionSet extends 
AbstractPrimaryKeyRestrictions impl
 // It is clearly a hack but it does not make a lot of sense to 
refactor 2.2 for that as the problem is
 // already solved in 3.0.
 List composites = filterAndSort(setEocs(r, bound, 
builder.build()));
-return Lists.transform(composites, new 
com.google.common.base.Function()
-{
-@Override
-public Composite apply(Composite composite)
-{
-return composite.isEmpty() ? Composites.EMPTY: 
composite;
-}
-});
+
+for (Composite c : composites)
+if (c.isEmpty())
+return normalizeEmptyComposites(composites);
+
+return composites;
 }
 
 r.appendBoundTo(cfm, builder, bound, options);
@@ -239,6 +235,14 @@ final class PrimaryKeyRestrictionSet extends 
AbstractPrimaryKeyRestrictions impl
 return new ArrayList<>(set);
 }
 
+private List normalizeEmptyComposites(List 
composites)
+{
+List transformed = new ArrayList<>(composites.size());
+for (Composite c : composites)
+transformed.add(c.isEmpty() ? Composites.EMPTY : c);
+return transformed;
+}
+
 /**
  * Sets EOCs for the composites returned by the specified slice 
restriction for the given bound.
  *

[10/10] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

2018-11-20 Thread aleksey

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/325d70d5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/325d70d5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/325d70d5

Branch: refs/heads/trunk
Commit: 325d70d5a62e92c64aa5e0331e24390604a45d8f
Parents: 9944d9e 54be1fa
Author: Aleksey Yeshchenko 
Authored: Tue Nov 20 15:25:38 2018 +
Committer: Aleksey Yeshchenko 
Committed: Tue Nov 20 15:25:38 2018 +

--
 .../SelectMultiColumnRelationTest.java  | 25 
 1 file changed, 25 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/325d70d5/test/unit/org/apache/cassandra/cql3/validation/operations/SelectMultiColumnRelationTest.java
--
diff --cc 
test/unit/org/apache/cassandra/cql3/validation/operations/SelectMultiColumnRelationTest.java
index 52a7f47,30c727b..5062448
--- 
a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectMultiColumnRelationTest.java
+++ 
b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectMultiColumnRelationTest.java
@@@ -1561,317 -1603,346 +1561,342 @@@ public class SelectMultiColumnRelationT
  @Test
  public void testMixedOrderColumns4() throws Throwable
  {
 -for (String compactOption : new String[]{"", " COMPACT STORAGE AND "})
 -{
 -createTable("CREATE TABLE %s (a int, b int, c int, d int, e int, 
PRIMARY KEY (a, b, c, d, e)) WITH " +
 -compactOption +
 -"CLUSTERING ORDER BY (b ASC, c DESC, d DESC, e ASC)");
 -
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, 2, 0, -1, 0);
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, 2, 0, -1, 1);
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, 2, 0, 1, 1);
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, 2, -1, 1, 1);
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, 2, -3, 1, 1);
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, 1, -1, 0, 0);
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, 1, -1, 1, 1);
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, 1, -1, 1, 0);
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, 1, 0, 1, -1);
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, 1, 0, 1, 1);
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, 1, 0, 0, -1);
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, 1, 0, 0, 0);
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, 1, 0, 0, 1);
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, 1, 0, -1, -1);
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, 1, 1, 0, -1);
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, 1, 1, 0, 0);
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, 1, 1, 0, -1);
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, 1, 1, 0, 0);
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, 1, 1, 0, 1);
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, 1, 1, -1, 0);
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, 0, 0, 0, 0);
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, -1, 0, -1, 0);
 -execute("INSERT INTO %s (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", 
0, -1, 0, 0, 0);
 -
 -assertRows(execute(
 -"SELECT * FROM %s" +
 -" WHERE a = ? " +
 -"AND (b,c,d,e)<(?,?,?,?) " +
 -"AND (b,c,d,e)>(?,?,?,?)", 0, 2, 0, 1, 1, -1, 0, -1, -1),
 -
 -   row(0, -1, 0, 0, 0),
 -   row(0, -1, 0, -1, 0),
 -   row(0, 0, 0, 0, 0),
 -   row(0, 1, 1, 0, -1),
 -   row(0, 1, 1, 0, 0),
 -   row(0, 1, 1, 0, 1),
 -   row(0, 1, 1, -1, 0),
 -   row(0, 1, 0, 1, -1),
 -   row(0, 1, 0, 1, 1),
 -   row(0, 1, 0, 0, -1),
 -   row(0, 1, 0, 0, 0),
 -   row(0, 1, 0, 0, 1),
 -   row(0, 1, 0, -1, -1),
 -   row(0, 1, -1, 1, 0),
 -   row(0, 1, -1, 1,

[09/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

2018-11-20 Thread aleksey

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/54be1fa8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/54be1fa8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/54be1fa8

Branch: refs/heads/trunk
Commit: 54be1fa8f6ebf59ca33da6b558e440c708a061e5
Parents: 78c7d57 c9dc69d
Author: Aleksey Yeshchenko 
Authored: Tue Nov 20 15:23:55 2018 +
Committer: Aleksey Yeshchenko 
Committed: Tue Nov 20 15:23:55 2018 +

--
 .../SelectMultiColumnRelationTest.java  | 25 
 1 file changed, 25 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/54be1fa8/test/unit/org/apache/cassandra/cql3/validation/operations/SelectMultiColumnRelationTest.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[05/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

2018-11-20 Thread aleksey

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c9dc69dc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c9dc69dc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c9dc69dc

Branch: refs/heads/cassandra-3.11
Commit: c9dc69dce61618834c0a43ded53e95f2d0480e1b
Parents: c6f822c cf6f792
Author: Aleksey Yeshchenko 
Authored: Tue Nov 20 15:23:42 2018 +
Committer: Aleksey Yeshchenko 
Committed: Tue Nov 20 15:23:42 2018 +

--
 .../SelectMultiColumnRelationTest.java  | 25 
 1 file changed, 25 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c9dc69dc/test/unit/org/apache/cassandra/cql3/validation/operations/SelectMultiColumnRelationTest.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[03/10] cassandra git commit: Cannot perform slice reads in reverse direction against tables with clustering columns in mixed order

2018-11-20 Thread aleksey

Cannot perform slice reads in reverse direction against tables with clustering 
columns in mixed order

patch by Aleksey Yeschenko; reviewed by Alex Petrov for CASSANDRA-14899


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cf6f7920
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cf6f7920
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cf6f7920

Branch: refs/heads/cassandra-3.11
Commit: cf6f7920f7742bb9a17a23ad37499d9213807d81
Parents: dcd92a9
Author: Aleksey Yeshchenko 
Authored: Thu Nov 15 14:54:05 2018 +
Committer: Aleksey Yeshchenko 
Committed: Tue Nov 20 15:19:28 2018 +

--
 CHANGES.txt |  5 
 .../restrictions/PrimaryKeyRestrictionSet.java  | 24 +++
 .../SelectMultiColumnRelationTest.java  | 25 
 3 files changed, 44 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf6f7920/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 71c57ea..bca036d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,7 +1,10 @@
 2.2.14
+ * Cannot perform slice reads in reverse direction against tables with 
clustering columns
+   in mixed order (CASSANDRA-14899)
  * Fix incorrect cqlsh results when selecting same columns multiple times 
(CASSANDRA-13262)
  * Returns null instead of NaN or Infinity in JSON strings (CASSANDRA-14377)
 
+
 2.2.13
  * Fix bug that prevented compaction of SSTables after full repairs 
(CASSANDRA-14423)
  * Incorrect counting of pending messages in OutboundTcpConnection 
(CASSANDRA-11551)
@@ -15,6 +18,7 @@ Merged from 2.1:
  * Check checksum before decompressing data (CASSANDRA-14284)
  * CVE-2017-5929 Security vulnerability in Logback warning in NEWS.txt 
(CASSANDRA-14183)
 
+
 2.2.12
  * Fix the inspectJvmOptions startup check (CASSANDRA-14112)
  * Fix race that prevents submitting compaction for a table when executor is 
full (CASSANDRA-13801)
@@ -25,6 +29,7 @@ Merged from 2.1:
  * More PEP8 compliance for cqlsh (CASSANDRA-14021)
  * RPM package spec: fix permissions for installed jars and config files 
(CASSANDRA-14181)
 
+
 2.2.11
  * Safely handle empty buffers when outputting to JSON (CASSANDRA-13868)
  * Copy session properties on cqlsh.py do_login (CASSANDRA-13847)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf6f7920/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
--
diff --git 
a/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java 
b/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
index 2549bdf..5136fee 100644
--- 
a/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
+++ 
b/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
@@ -20,8 +20,6 @@ package org.apache.cassandra.cql3.restrictions;
 import java.nio.ByteBuffer;
 import java.util.*;
 
-import com.google.common.collect.Lists;
-
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.ColumnDefinition;
 import org.apache.cassandra.cql3.QueryOptions;
@@ -195,14 +193,12 @@ final class PrimaryKeyRestrictionSet extends 
AbstractPrimaryKeyRestrictions impl
 // It is clearly a hack but it does not make a lot of sense to 
refactor 2.2 for that as the problem is
 // already solved in 3.0.
 List composites = filterAndSort(setEocs(r, bound, 
builder.build()));
-return Lists.transform(composites, new 
com.google.common.base.Function()
-{
-@Override
-public Composite apply(Composite composite)
-{
-return composite.isEmpty() ? Composites.EMPTY: 
composite;
-}
-});
+
+for (Composite c : composites)
+if (c.isEmpty())
+return normalizeEmptyComposites(composites);
+
+return composites;
 }
 
 r.appendBoundTo(cfm, builder, bound, options);
@@ -239,6 +235,14 @@ final class PrimaryKeyRestrictionSet extends 
AbstractPrimaryKeyRestrictions impl
 return new ArrayList<>(set);
 }
 
+private List normalizeEmptyComposites(List 
composites)
+{
+List transformed = new ArrayList<>(composites.size());
+for (Composite c : composites)
+transformed.add(c.isEmpty() ? Composites.EMPTY : c);
+return transformed;
+}
+
 /**
  * Sets EOCs for the composites returned by the specified slice 
restriction for the given bound.
  *

[07/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

2018-11-20 Thread aleksey

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c9dc69dc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c9dc69dc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c9dc69dc

Branch: refs/heads/trunk
Commit: c9dc69dce61618834c0a43ded53e95f2d0480e1b
Parents: c6f822c cf6f792
Author: Aleksey Yeshchenko 
Authored: Tue Nov 20 15:23:42 2018 +
Committer: Aleksey Yeshchenko 
Committed: Tue Nov 20 15:23:42 2018 +

--
 .../SelectMultiColumnRelationTest.java  | 25 
 1 file changed, 25 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c9dc69dc/test/unit/org/apache/cassandra/cql3/validation/operations/SelectMultiColumnRelationTest.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[08/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

2018-11-20 Thread aleksey

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/54be1fa8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/54be1fa8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/54be1fa8

Branch: refs/heads/cassandra-3.11
Commit: 54be1fa8f6ebf59ca33da6b558e440c708a061e5
Parents: 78c7d57 c9dc69d
Author: Aleksey Yeshchenko 
Authored: Tue Nov 20 15:23:55 2018 +
Committer: Aleksey Yeshchenko 
Committed: Tue Nov 20 15:23:55 2018 +

--
 .../SelectMultiColumnRelationTest.java  | 25 
 1 file changed, 25 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/54be1fa8/test/unit/org/apache/cassandra/cql3/validation/operations/SelectMultiColumnRelationTest.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[02/10] cassandra git commit: Cannot perform slice reads in reverse direction against tables with clustering columns in mixed order

2018-11-20 Thread aleksey

Cannot perform slice reads in reverse direction against tables with clustering 
columns in mixed order

patch by Aleksey Yeschenko; reviewed by Alex Petrov for CASSANDRA-14899


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cf6f7920
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cf6f7920
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cf6f7920

Branch: refs/heads/cassandra-3.0
Commit: cf6f7920f7742bb9a17a23ad37499d9213807d81
Parents: dcd92a9
Author: Aleksey Yeshchenko 
Authored: Thu Nov 15 14:54:05 2018 +
Committer: Aleksey Yeshchenko 
Committed: Tue Nov 20 15:19:28 2018 +

--
 CHANGES.txt |  5 
 .../restrictions/PrimaryKeyRestrictionSet.java  | 24 +++
 .../SelectMultiColumnRelationTest.java  | 25 
 3 files changed, 44 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf6f7920/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 71c57ea..bca036d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,7 +1,10 @@
 2.2.14
+ * Cannot perform slice reads in reverse direction against tables with 
clustering columns
+   in mixed order (CASSANDRA-14899)
  * Fix incorrect cqlsh results when selecting same columns multiple times 
(CASSANDRA-13262)
  * Returns null instead of NaN or Infinity in JSON strings (CASSANDRA-14377)
 
+
 2.2.13
  * Fix bug that prevented compaction of SSTables after full repairs 
(CASSANDRA-14423)
  * Incorrect counting of pending messages in OutboundTcpConnection 
(CASSANDRA-11551)
@@ -15,6 +18,7 @@ Merged from 2.1:
  * Check checksum before decompressing data (CASSANDRA-14284)
  * CVE-2017-5929 Security vulnerability in Logback warning in NEWS.txt 
(CASSANDRA-14183)
 
+
 2.2.12
  * Fix the inspectJvmOptions startup check (CASSANDRA-14112)
  * Fix race that prevents submitting compaction for a table when executor is 
full (CASSANDRA-13801)
@@ -25,6 +29,7 @@ Merged from 2.1:
  * More PEP8 compliance for cqlsh (CASSANDRA-14021)
  * RPM package spec: fix permissions for installed jars and config files 
(CASSANDRA-14181)
 
+
 2.2.11
  * Safely handle empty buffers when outputting to JSON (CASSANDRA-13868)
  * Copy session properties on cqlsh.py do_login (CASSANDRA-13847)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf6f7920/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
--
diff --git 
a/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java 
b/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
index 2549bdf..5136fee 100644
--- 
a/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
+++ 
b/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
@@ -20,8 +20,6 @@ package org.apache.cassandra.cql3.restrictions;
 import java.nio.ByteBuffer;
 import java.util.*;
 
-import com.google.common.collect.Lists;
-
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.ColumnDefinition;
 import org.apache.cassandra.cql3.QueryOptions;
@@ -195,14 +193,12 @@ final class PrimaryKeyRestrictionSet extends 
AbstractPrimaryKeyRestrictions impl
 // It is clearly a hack but it does not make a lot of sense to 
refactor 2.2 for that as the problem is
 // already solved in 3.0.
 List composites = filterAndSort(setEocs(r, bound, 
builder.build()));
-return Lists.transform(composites, new 
com.google.common.base.Function()
-{
-@Override
-public Composite apply(Composite composite)
-{
-return composite.isEmpty() ? Composites.EMPTY: 
composite;
-}
-});
+
+for (Composite c : composites)
+if (c.isEmpty())
+return normalizeEmptyComposites(composites);
+
+return composites;
 }
 
 r.appendBoundTo(cfm, builder, bound, options);
@@ -239,6 +235,14 @@ final class PrimaryKeyRestrictionSet extends 
AbstractPrimaryKeyRestrictions impl
 return new ArrayList<>(set);
 }
 
+private List normalizeEmptyComposites(List 
composites)
+{
+List transformed = new ArrayList<>(composites.size());
+for (Composite c : composites)
+transformed.add(c.isEmpty() ? Composites.EMPTY : c);
+return transformed;
+}
+
 /**
  * Sets EOCs for the composites returned by the specified slice 
restriction for the given bound.
  *

[01/10] cassandra git commit: Cannot perform slice reads in reverse direction against tables with clustering columns in mixed order

2018-11-20 Thread aleksey

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 dcd92a93a -> cf6f7920f
  refs/heads/cassandra-3.0 c6f822c2a -> c9dc69dce
  refs/heads/cassandra-3.11 78c7d57eb -> 54be1fa8f
  refs/heads/trunk 9944d9e24 -> 325d70d5a


Cannot perform slice reads in reverse direction against tables with clustering 
columns in mixed order

patch by Aleksey Yeschenko; reviewed by Alex Petrov for CASSANDRA-14899


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cf6f7920
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cf6f7920
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cf6f7920

Branch: refs/heads/cassandra-2.2
Commit: cf6f7920f7742bb9a17a23ad37499d9213807d81
Parents: dcd92a9
Author: Aleksey Yeshchenko 
Authored: Thu Nov 15 14:54:05 2018 +
Committer: Aleksey Yeshchenko 
Committed: Tue Nov 20 15:19:28 2018 +

--
 CHANGES.txt |  5 
 .../restrictions/PrimaryKeyRestrictionSet.java  | 24 +++
 .../SelectMultiColumnRelationTest.java  | 25 
 3 files changed, 44 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf6f7920/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 71c57ea..bca036d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,7 +1,10 @@
 2.2.14
+ * Cannot perform slice reads in reverse direction against tables with 
clustering columns
+   in mixed order (CASSANDRA-14899)
  * Fix incorrect cqlsh results when selecting same columns multiple times 
(CASSANDRA-13262)
  * Returns null instead of NaN or Infinity in JSON strings (CASSANDRA-14377)
 
+
 2.2.13
  * Fix bug that prevented compaction of SSTables after full repairs 
(CASSANDRA-14423)
  * Incorrect counting of pending messages in OutboundTcpConnection 
(CASSANDRA-11551)
@@ -15,6 +18,7 @@ Merged from 2.1:
  * Check checksum before decompressing data (CASSANDRA-14284)
  * CVE-2017-5929 Security vulnerability in Logback warning in NEWS.txt 
(CASSANDRA-14183)
 
+
 2.2.12
  * Fix the inspectJvmOptions startup check (CASSANDRA-14112)
  * Fix race that prevents submitting compaction for a table when executor is 
full (CASSANDRA-13801)
@@ -25,6 +29,7 @@ Merged from 2.1:
  * More PEP8 compliance for cqlsh (CASSANDRA-14021)
  * RPM package spec: fix permissions for installed jars and config files 
(CASSANDRA-14181)
 
+
 2.2.11
  * Safely handle empty buffers when outputting to JSON (CASSANDRA-13868)
  * Copy session properties on cqlsh.py do_login (CASSANDRA-13847)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf6f7920/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
--
diff --git 
a/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java 
b/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
index 2549bdf..5136fee 100644
--- 
a/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
+++ 
b/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
@@ -20,8 +20,6 @@ package org.apache.cassandra.cql3.restrictions;
 import java.nio.ByteBuffer;
 import java.util.*;
 
-import com.google.common.collect.Lists;
-
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.ColumnDefinition;
 import org.apache.cassandra.cql3.QueryOptions;
@@ -195,14 +193,12 @@ final class PrimaryKeyRestrictionSet extends 
AbstractPrimaryKeyRestrictions impl
 // It is clearly a hack but it does not make a lot of sense to 
refactor 2.2 for that as the problem is
 // already solved in 3.0.
 List composites = filterAndSort(setEocs(r, bound, 
builder.build()));
-return Lists.transform(composites, new 
com.google.common.base.Function()
-{
-@Override
-public Composite apply(Composite composite)
-{
-return composite.isEmpty() ? Composites.EMPTY: 
composite;
-}
-});
+
+for (Composite c : composites)
+if (c.isEmpty())
+return normalizeEmptyComposites(composites);
+
+return composites;
 }
 
 r.appendBoundTo(cfm, builder, bound, options);
@@ -239,6 +235,14 @@ final class PrimaryKeyRestrictionSet extends 
AbstractPrimaryKeyRestrictions impl
 return new ArrayList<>(set);
 }
 
+private List normalizeEmptyComposites(List 
composites)
+{
+List transformed = new ArrayList<>(composites.size());
+for (Composite c :

[06/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

2018-11-20 Thread aleksey

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c9dc69dc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c9dc69dc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c9dc69dc

Branch: refs/heads/cassandra-3.0
Commit: c9dc69dce61618834c0a43ded53e95f2d0480e1b
Parents: c6f822c cf6f792
Author: Aleksey Yeshchenko 
Authored: Tue Nov 20 15:23:42 2018 +
Committer: Aleksey Yeshchenko 
Committed: Tue Nov 20 15:23:42 2018 +

--
 .../SelectMultiColumnRelationTest.java  | 25 
 1 file changed, 25 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c9dc69dc/test/unit/org/apache/cassandra/cql3/validation/operations/SelectMultiColumnRelationTest.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-10540) RangeAwareCompaction

2018-11-20 Thread Marcus Eriksson (JIRA)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-10540:

Labels: compaction lcs vnodes  (was: 4.0-feature-freeze-review-requested 
compaction lcs vnodes)

removing tag 4.0-feature-freeze-review-requested as this is not close to being 
ready for review

> RangeAwareCompaction
> 
>
> Key: CASSANDRA-10540
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10540
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Compaction
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
>  Labels: compaction, lcs, vnodes
> Fix For: 4.x
>
>
> Broken out from CASSANDRA-6696, we should split sstables based on ranges 
> during compaction.
> Requirements;
> * dont create tiny sstables - keep them bunched together until a single vnode 
> is big enough (configurable how big that is)
> * make it possible to run existing compaction strategies on the per-range 
> sstables
> We should probably add a global compaction strategy parameter that states 
> whether this should be enabled or not.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14899) Cannot perform slice reads in reverse direction against tables with clustering columns in mixed order

2018-11-20 Thread Alex Petrov (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693303#comment-16693303
 ] 

Alex Petrov commented on CASSANDRA-14899:
-

+1 with two minor optional nits for your consideration: 

  * {{ArrayList -> List}} 
[here|https://github.com/iamaleksey/cassandra/commit/29b7f064d69aff55123dcfc282c3c44ab733734f#diff-f2feb48017f94269b5fbb96ee38b5816R204]
  * you can short-circuit 
[here|https://github.com/iamaleksey/cassandra/commit/29b7f064d69aff55123dcfc282c3c44ab733734f#diff-f2feb48017f94269b5fbb96ee38b5816R199]

If you think these modifications are unnecessary, please feel free to commit 
as-is.

> Cannot perform slice reads in reverse direction against tables with 
> clustering columns in mixed order
> -
>
> Key: CASSANDRA-14899
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14899
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 2.2.x
>
>
> CASSANDRA-11196 accidentally broke reading from tables with mixed clustering 
> column order in the opposite direction.
> {{ReversedPrimaryKeyRestrictions::boundsAsComposites}} method attempts to 
> reverse the list returned from 
> {{PrimaryKeyRestrictionSet::boundsAsComposites}} and fails, as Guava’s 
> {{Lists::transform}} method returns a {{List}} that doesn’t support {{set()}}.
> Reproduction:
> {code}
> CREATE TABLE test.test (
> a int,
> b int,
> c int,
> PRIMARY KEY (a, b, c)
> ) WITH CLUSTERING ORDER BY (b ASC, c DESC);
> SELECT * FROM test.test WHERE a = 0 AND (b, c) > (0, 0) ORDER BY b DESC, c 
> ASC;
> > ServerError: java.lang.UnsupportedOperationException
> {code}
> {code}
> java.lang.UnsupportedOperationException: null
>   at java.util.AbstractList.set(AbstractList.java:132) ~[na:1.8.0_181]
>   at java.util.Collections.swap(Collections.java:497) ~[na:1.8.0_181]
>   at java.util.Collections.reverse(Collections.java:378) ~[na:1.8.0_181]
>   at 
> org.apache.cassandra.cql3.restrictions.ReversedPrimaryKeyRestrictions.boundsAsComposites(ReversedPrimaryKeyRestrictions.java:63)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.restrictions.StatementRestrictions.getClusteringColumnsBoundsAsComposites(StatementRestrictions.java:580)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.makeFilter(SelectStatement.java:418)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.getSliceCommands(SelectStatement.java:359)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.getPageableCommand(SelectStatement.java:191)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:172)
>  ~[main/:na]
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14848) When upgrading 3.11.3->4.0 using SSL 4.0 nodes does not connect to old non seed nodes

2018-11-20 Thread Tommy Stendahl (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693242#comment-16693242
 ] 

Tommy Stendahl edited comment on CASSANDRA-14848 at 11/20/18 1:39 PM:
--

I have done some more troubleshooting on the new exception I got and I don't 
think its related to my patch, I get the same issue without my patch also. Its 
probably related to CASSANDRA-14896.
 My patch do solve the issue I reported in this jira and with it the new node 
selects the correct port to all old nodes.
 Patch is here: 
[cassandra-14848|https://github.com/tommystendahl/cassandra/commit/d2a9cfe87c0e41e20ea43a75bab76f2cba8e293c]


was (Author: tommy_s):
I have done some more troubleshooting on the new exception I got and I don't 
think its related to my patch, I get the same issue without  my patch also. Its 
probably related to CASSANDRA-14896.
My patch do solve the issue I reported in this jira and with it the new node 
selects the correct port to all old nodes.
Patch is here: 
[cassandra-14848|]https://github.com/tommystendahl/cassandra/commit/d2a9cfe87c0e41e20ea43a75bab76f2cba8e293c]

> When upgrading 3.11.3->4.0 using SSL 4.0 nodes does not connect to old non 
> seed nodes
> -
>
> Key: CASSANDRA-14848
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14848
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Tommy Stendahl
>Priority: Major
>  Labels: security
>
> When upgrading from 3.11.3 to 4.0 with server encryption enabled the new 4.0 
> node only connects to 3.11.3 seed node, there are no connection established 
> to non-seed nodes on the old version.
> I have four nodes, *.242 is upgraded to 4.0, *.243 and *.244 are 3.11.3 
> non-seed and *.246 are 3.11.3 seed. After starting the 4.0 node I get this 
> nodetool status on the different nodes:
> {noformat}
> *.242
> -- Address Load Tokens Owns (effective) Host ID Rack
> UN 10.216.193.242 1017.77 KiB 256 75,1% 7d278e14-d549-42f3-840d-77cfd852fbf4 
> RAC1
> DN 10.216.193.243 743.32 KiB 256 74,8% 5586243a-ca74-4125-8e7e-09e82e23c4e5 
> RAC1
> DN 10.216.193.244 711.54 KiB 256 75,2% c155e262-b898-4e86-9e1d-d4d0f97e88f6 
> RAC1
> UN 10.216.193.246 659.81 KiB 256 74,9% 502dd00f-fc02-4024-b65f-b98ba3808291 
> RAC1
> *.243 and *.244
> -- Address Load Tokens Owns (effective) Host ID Rack
> DN 10.216.193.242 657.4 KiB 256 75,1% 7d278e14-d549-42f3-840d-77cfd852fbf4 
> RAC1
> UN 10.216.193.243 471 KiB 256 74,8% 5586243a-ca74-4125-8e7e-09e82e23c4e5 RAC1
> UN 10.216.193.244 471.71 KiB 256 75,2% c155e262-b898-4e86-9e1d-d4d0f97e88f6 
> RAC1
> UN 10.216.193.246 388.54 KiB 256 74,9% 502dd00f-fc02-4024-b65f-b98ba3808291 
> RAC1
> *.246
> -- Address Load Tokens Owns (effective) Host ID Rack
> UN 10.216.193.242 657.4 KiB 256 75,1% 7d278e14-d549-42f3-840d-77cfd852fbf4 
> RAC1
> UN 10.216.193.243 471 KiB 256 74,8% 5586243a-ca74-4125-8e7e-09e82e23c4e5 RAC1
> UN 10.216.193.244 471.71 KiB 256 75,2% c155e262-b898-4e86-9e1d-d4d0f97e88f6 
> RAC1
> UN 10.216.193.246 388.54 KiB 256 74,9% 502dd00f-fc02-4024-b65f-b98ba3808291 
> RAC1
> {noformat}
>  
> I have built 4.0 with wire tracing activated and in my config the 
> storage_port=12700 and ssl_storage_port=12701. In the log I can see that the 
> 4.0 node start to connect to the 3.11.3 seed node on the storage_port but 
> quickly switch to the ssl_storage_port, but when connecting to the non-seed 
> nodes it never switch to the ssl_storage_port.
> {noformat}
> >grep 193.246 system.log | grep Outbound
> 2018-10-25T10:57:36.799+0200 [MessagingService-NettyOutbound-Thread-4-1] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x2f0e5e55] CONNECT: 
> /10.216.193.246:12700
> 2018-10-25T10:57:36.902+0200 [MessagingService-NettyOutbound-Thread-4-2] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x9e81f62c] CONNECT: 
> /10.216.193.246:12701
> 2018-10-25T10:57:36.905+0200 [MessagingService-NettyOutbound-Thread-4-2] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x9e81f62c, 
> L:/10.216.193.242:37252 - R:10.216.193.246/10.216.193.246:12701] ACTIVE
> 2018-10-25T10:57:36.906+0200 [MessagingService-NettyOutbound-Thread-4-2] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x9e81f62c, 
> L:/10.216.193.242:37252 - R:10.216.193.246/10.216.193.246:12701] WRITE: 8B
> >grep 193.243 system.log | grep Outbound
> 2018-10-25T10:57:38.438+0200 [MessagingService-NettyOutbound-Thread-4-3] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0xd8f1d6c4] CONNECT: 
> /10.216.193.243:12700
> 2018-10-25T10:57:38.540+0200 [MessagingService-NettyOutbound-Thread-4-4] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0xfde6cc9f] CONNECT: 
> /10.216.193.243:12700
>

[jira] [Updated] (CASSANDRA-14848) When upgrading 3.11.3->4.0 using SSL 4.0 nodes does not connect to old non seed nodes

2018-11-20 Thread Tommy Stendahl (JIRA)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tommy Stendahl updated CASSANDRA-14848:
---
Assignee: Tommy Stendahl
  Status: Patch Available  (was: Open)

> When upgrading 3.11.3->4.0 using SSL 4.0 nodes does not connect to old non 
> seed nodes
> -
>
> Key: CASSANDRA-14848
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14848
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Tommy Stendahl
>Assignee: Tommy Stendahl
>Priority: Major
>  Labels: security
>
> When upgrading from 3.11.3 to 4.0 with server encryption enabled the new 4.0 
> node only connects to 3.11.3 seed node, there are no connection established 
> to non-seed nodes on the old version.
> I have four nodes, *.242 is upgraded to 4.0, *.243 and *.244 are 3.11.3 
> non-seed and *.246 are 3.11.3 seed. After starting the 4.0 node I get this 
> nodetool status on the different nodes:
> {noformat}
> *.242
> -- Address Load Tokens Owns (effective) Host ID Rack
> UN 10.216.193.242 1017.77 KiB 256 75,1% 7d278e14-d549-42f3-840d-77cfd852fbf4 
> RAC1
> DN 10.216.193.243 743.32 KiB 256 74,8% 5586243a-ca74-4125-8e7e-09e82e23c4e5 
> RAC1
> DN 10.216.193.244 711.54 KiB 256 75,2% c155e262-b898-4e86-9e1d-d4d0f97e88f6 
> RAC1
> UN 10.216.193.246 659.81 KiB 256 74,9% 502dd00f-fc02-4024-b65f-b98ba3808291 
> RAC1
> *.243 and *.244
> -- Address Load Tokens Owns (effective) Host ID Rack
> DN 10.216.193.242 657.4 KiB 256 75,1% 7d278e14-d549-42f3-840d-77cfd852fbf4 
> RAC1
> UN 10.216.193.243 471 KiB 256 74,8% 5586243a-ca74-4125-8e7e-09e82e23c4e5 RAC1
> UN 10.216.193.244 471.71 KiB 256 75,2% c155e262-b898-4e86-9e1d-d4d0f97e88f6 
> RAC1
> UN 10.216.193.246 388.54 KiB 256 74,9% 502dd00f-fc02-4024-b65f-b98ba3808291 
> RAC1
> *.246
> -- Address Load Tokens Owns (effective) Host ID Rack
> UN 10.216.193.242 657.4 KiB 256 75,1% 7d278e14-d549-42f3-840d-77cfd852fbf4 
> RAC1
> UN 10.216.193.243 471 KiB 256 74,8% 5586243a-ca74-4125-8e7e-09e82e23c4e5 RAC1
> UN 10.216.193.244 471.71 KiB 256 75,2% c155e262-b898-4e86-9e1d-d4d0f97e88f6 
> RAC1
> UN 10.216.193.246 388.54 KiB 256 74,9% 502dd00f-fc02-4024-b65f-b98ba3808291 
> RAC1
> {noformat}
>  
> I have built 4.0 with wire tracing activated and in my config the 
> storage_port=12700 and ssl_storage_port=12701. In the log I can see that the 
> 4.0 node start to connect to the 3.11.3 seed node on the storage_port but 
> quickly switch to the ssl_storage_port, but when connecting to the non-seed 
> nodes it never switch to the ssl_storage_port.
> {noformat}
> >grep 193.246 system.log | grep Outbound
> 2018-10-25T10:57:36.799+0200 [MessagingService-NettyOutbound-Thread-4-1] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x2f0e5e55] CONNECT: 
> /10.216.193.246:12700
> 2018-10-25T10:57:36.902+0200 [MessagingService-NettyOutbound-Thread-4-2] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x9e81f62c] CONNECT: 
> /10.216.193.246:12701
> 2018-10-25T10:57:36.905+0200 [MessagingService-NettyOutbound-Thread-4-2] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x9e81f62c, 
> L:/10.216.193.242:37252 - R:10.216.193.246/10.216.193.246:12701] ACTIVE
> 2018-10-25T10:57:36.906+0200 [MessagingService-NettyOutbound-Thread-4-2] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x9e81f62c, 
> L:/10.216.193.242:37252 - R:10.216.193.246/10.216.193.246:12701] WRITE: 8B
> >grep 193.243 system.log | grep Outbound
> 2018-10-25T10:57:38.438+0200 [MessagingService-NettyOutbound-Thread-4-3] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0xd8f1d6c4] CONNECT: 
> /10.216.193.243:12700
> 2018-10-25T10:57:38.540+0200 [MessagingService-NettyOutbound-Thread-4-4] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0xfde6cc9f] CONNECT: 
> /10.216.193.243:12700
> 2018-10-25T10:57:38.694+0200 [MessagingService-NettyOutbound-Thread-4-5] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x7e87fc4e] CONNECT: 
> /10.216.193.243:12700
> 2018-10-25T10:57:38.741+0200 [MessagingService-NettyOutbound-Thread-4-7] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x39395296] CONNECT: 
> /10.216.193.243:12700{noformat}
>  
> When I had the dbug log activated and started the 4.0 node I can see that it 
> switch port for *.246 but not for *.243 and *.244.
> {noformat}
> >grep DEBUG system.log| grep OutboundMessagingConnection | grep 
> >maybeUpdateConnectionId
> 2018-10-25T13:12:56.095+0200 [ScheduledFastTasks:1] DEBUG 
> o.a.c.n.a.OutboundMessagingConnection:314 maybeUpdateConnectionId changing 
> connectionId to 10.216.193.246:12701 (GOSSIP), with a different port for 
> secure communication, because peer version is 11
> 2018-10-25T13:12:58.100+0200 [ReadStage-1]

[jira] [Comment Edited] (CASSANDRA-14848) When upgrading 3.11.3->4.0 using SSL 4.0 nodes does not connect to old non seed nodes

2018-11-20 Thread Tommy Stendahl (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693242#comment-16693242
 ] 

Tommy Stendahl edited comment on CASSANDRA-14848 at 11/20/18 1:39 PM:
--

I have done some more troubleshooting on the new exception I got and I don't 
think its related to my patch, I get the same issue without  my patch also. Its 
probably related to CASSANDRA-14896.
My patch do solve the issue I reported in this jira and with it the new node 
selects the correct port to all old nodes.
Patch is here: 
[cassandra-14848|]https://github.com/tommystendahl/cassandra/commit/d2a9cfe87c0e41e20ea43a75bab76f2cba8e293c]


was (Author: tommy_s):
I have done some more troubleshooting on the new exception I got and I don't 
think its related to my patch, I get the same issue without  my patch also. Its 
probably related to CASSANDRA-14896.
My patch do solve the issue I reported in this jira and with it the new node 
selects the correct port to all old nodes.
Patch is here: 
[cassandra-14848|]https://github.com/tommystendahl/cassandra/commit/d2a9cfe87c0e41e20ea43a75bab76f2cba8e293c

> When upgrading 3.11.3->4.0 using SSL 4.0 nodes does not connect to old non 
> seed nodes
> -
>
> Key: CASSANDRA-14848
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14848
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Tommy Stendahl
>Priority: Major
>  Labels: security
>
> When upgrading from 3.11.3 to 4.0 with server encryption enabled the new 4.0 
> node only connects to 3.11.3 seed node, there are no connection established 
> to non-seed nodes on the old version.
> I have four nodes, *.242 is upgraded to 4.0, *.243 and *.244 are 3.11.3 
> non-seed and *.246 are 3.11.3 seed. After starting the 4.0 node I get this 
> nodetool status on the different nodes:
> {noformat}
> *.242
> -- Address Load Tokens Owns (effective) Host ID Rack
> UN 10.216.193.242 1017.77 KiB 256 75,1% 7d278e14-d549-42f3-840d-77cfd852fbf4 
> RAC1
> DN 10.216.193.243 743.32 KiB 256 74,8% 5586243a-ca74-4125-8e7e-09e82e23c4e5 
> RAC1
> DN 10.216.193.244 711.54 KiB 256 75,2% c155e262-b898-4e86-9e1d-d4d0f97e88f6 
> RAC1
> UN 10.216.193.246 659.81 KiB 256 74,9% 502dd00f-fc02-4024-b65f-b98ba3808291 
> RAC1
> *.243 and *.244
> -- Address Load Tokens Owns (effective) Host ID Rack
> DN 10.216.193.242 657.4 KiB 256 75,1% 7d278e14-d549-42f3-840d-77cfd852fbf4 
> RAC1
> UN 10.216.193.243 471 KiB 256 74,8% 5586243a-ca74-4125-8e7e-09e82e23c4e5 RAC1
> UN 10.216.193.244 471.71 KiB 256 75,2% c155e262-b898-4e86-9e1d-d4d0f97e88f6 
> RAC1
> UN 10.216.193.246 388.54 KiB 256 74,9% 502dd00f-fc02-4024-b65f-b98ba3808291 
> RAC1
> *.246
> -- Address Load Tokens Owns (effective) Host ID Rack
> UN 10.216.193.242 657.4 KiB 256 75,1% 7d278e14-d549-42f3-840d-77cfd852fbf4 
> RAC1
> UN 10.216.193.243 471 KiB 256 74,8% 5586243a-ca74-4125-8e7e-09e82e23c4e5 RAC1
> UN 10.216.193.244 471.71 KiB 256 75,2% c155e262-b898-4e86-9e1d-d4d0f97e88f6 
> RAC1
> UN 10.216.193.246 388.54 KiB 256 74,9% 502dd00f-fc02-4024-b65f-b98ba3808291 
> RAC1
> {noformat}
>  
> I have built 4.0 with wire tracing activated and in my config the 
> storage_port=12700 and ssl_storage_port=12701. In the log I can see that the 
> 4.0 node start to connect to the 3.11.3 seed node on the storage_port but 
> quickly switch to the ssl_storage_port, but when connecting to the non-seed 
> nodes it never switch to the ssl_storage_port.
> {noformat}
> >grep 193.246 system.log | grep Outbound
> 2018-10-25T10:57:36.799+0200 [MessagingService-NettyOutbound-Thread-4-1] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x2f0e5e55] CONNECT: 
> /10.216.193.246:12700
> 2018-10-25T10:57:36.902+0200 [MessagingService-NettyOutbound-Thread-4-2] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x9e81f62c] CONNECT: 
> /10.216.193.246:12701
> 2018-10-25T10:57:36.905+0200 [MessagingService-NettyOutbound-Thread-4-2] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x9e81f62c, 
> L:/10.216.193.242:37252 - R:10.216.193.246/10.216.193.246:12701] ACTIVE
> 2018-10-25T10:57:36.906+0200 [MessagingService-NettyOutbound-Thread-4-2] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x9e81f62c, 
> L:/10.216.193.242:37252 - R:10.216.193.246/10.216.193.246:12701] WRITE: 8B
> >grep 193.243 system.log | grep Outbound
> 2018-10-25T10:57:38.438+0200 [MessagingService-NettyOutbound-Thread-4-3] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0xd8f1d6c4] CONNECT: 
> /10.216.193.243:12700
> 2018-10-25T10:57:38.540+0200 [MessagingService-NettyOutbound-Thread-4-4] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0xfde6cc9f] CONNECT: 
> /10.216.193.243:12700
>

[jira] [Commented] (CASSANDRA-14848) When upgrading 3.11.3->4.0 using SSL 4.0 nodes does not connect to old non seed nodes

2018-11-20 Thread Tommy Stendahl (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693242#comment-16693242
 ] 

Tommy Stendahl commented on CASSANDRA-14848:


I have done some more troubleshooting on the new exception I got and I don't 
think its related to my patch, I get the same issue without  my patch also. Its 
probably related to CASSANDRA-14896.
My patch do solve the issue I reported in this jira and with it the new node 
selects the correct port to all old nodes.
Patch is here: 
[cassandra-14848|https://github.com/tommystendahl/cassandra/tree/cassandra-14848]

> When upgrading 3.11.3->4.0 using SSL 4.0 nodes does not connect to old non 
> seed nodes
> -
>
> Key: CASSANDRA-14848
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14848
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Tommy Stendahl
>Priority: Major
>  Labels: security
>
> When upgrading from 3.11.3 to 4.0 with server encryption enabled the new 4.0 
> node only connects to 3.11.3 seed node, there are no connection established 
> to non-seed nodes on the old version.
> I have four nodes, *.242 is upgraded to 4.0, *.243 and *.244 are 3.11.3 
> non-seed and *.246 are 3.11.3 seed. After starting the 4.0 node I get this 
> nodetool status on the different nodes:
> {noformat}
> *.242
> -- Address Load Tokens Owns (effective) Host ID Rack
> UN 10.216.193.242 1017.77 KiB 256 75,1% 7d278e14-d549-42f3-840d-77cfd852fbf4 
> RAC1
> DN 10.216.193.243 743.32 KiB 256 74,8% 5586243a-ca74-4125-8e7e-09e82e23c4e5 
> RAC1
> DN 10.216.193.244 711.54 KiB 256 75,2% c155e262-b898-4e86-9e1d-d4d0f97e88f6 
> RAC1
> UN 10.216.193.246 659.81 KiB 256 74,9% 502dd00f-fc02-4024-b65f-b98ba3808291 
> RAC1
> *.243 and *.244
> -- Address Load Tokens Owns (effective) Host ID Rack
> DN 10.216.193.242 657.4 KiB 256 75,1% 7d278e14-d549-42f3-840d-77cfd852fbf4 
> RAC1
> UN 10.216.193.243 471 KiB 256 74,8% 5586243a-ca74-4125-8e7e-09e82e23c4e5 RAC1
> UN 10.216.193.244 471.71 KiB 256 75,2% c155e262-b898-4e86-9e1d-d4d0f97e88f6 
> RAC1
> UN 10.216.193.246 388.54 KiB 256 74,9% 502dd00f-fc02-4024-b65f-b98ba3808291 
> RAC1
> *.246
> -- Address Load Tokens Owns (effective) Host ID Rack
> UN 10.216.193.242 657.4 KiB 256 75,1% 7d278e14-d549-42f3-840d-77cfd852fbf4 
> RAC1
> UN 10.216.193.243 471 KiB 256 74,8% 5586243a-ca74-4125-8e7e-09e82e23c4e5 RAC1
> UN 10.216.193.244 471.71 KiB 256 75,2% c155e262-b898-4e86-9e1d-d4d0f97e88f6 
> RAC1
> UN 10.216.193.246 388.54 KiB 256 74,9% 502dd00f-fc02-4024-b65f-b98ba3808291 
> RAC1
> {noformat}
>  
> I have built 4.0 with wire tracing activated and in my config the 
> storage_port=12700 and ssl_storage_port=12701. In the log I can see that the 
> 4.0 node start to connect to the 3.11.3 seed node on the storage_port but 
> quickly switch to the ssl_storage_port, but when connecting to the non-seed 
> nodes it never switch to the ssl_storage_port.
> {noformat}
> >grep 193.246 system.log | grep Outbound
> 2018-10-25T10:57:36.799+0200 [MessagingService-NettyOutbound-Thread-4-1] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x2f0e5e55] CONNECT: 
> /10.216.193.246:12700
> 2018-10-25T10:57:36.902+0200 [MessagingService-NettyOutbound-Thread-4-2] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x9e81f62c] CONNECT: 
> /10.216.193.246:12701
> 2018-10-25T10:57:36.905+0200 [MessagingService-NettyOutbound-Thread-4-2] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x9e81f62c, 
> L:/10.216.193.242:37252 - R:10.216.193.246/10.216.193.246:12701] ACTIVE
> 2018-10-25T10:57:36.906+0200 [MessagingService-NettyOutbound-Thread-4-2] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x9e81f62c, 
> L:/10.216.193.242:37252 - R:10.216.193.246/10.216.193.246:12701] WRITE: 8B
> >grep 193.243 system.log | grep Outbound
> 2018-10-25T10:57:38.438+0200 [MessagingService-NettyOutbound-Thread-4-3] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0xd8f1d6c4] CONNECT: 
> /10.216.193.243:12700
> 2018-10-25T10:57:38.540+0200 [MessagingService-NettyOutbound-Thread-4-4] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0xfde6cc9f] CONNECT: 
> /10.216.193.243:12700
> 2018-10-25T10:57:38.694+0200 [MessagingService-NettyOutbound-Thread-4-5] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x7e87fc4e] CONNECT: 
> /10.216.193.243:12700
> 2018-10-25T10:57:38.741+0200 [MessagingService-NettyOutbound-Thread-4-7] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x39395296] CONNECT: 
> /10.216.193.243:12700{noformat}
>  
> When I had the dbug log activated and started the 4.0 node I can see that it 
> switch port for *.246 but not for *.243 and *.244.
> {noformat}
> >grep DEBUG system.log| grep OutboundMessagingConnection

[jira] [Comment Edited] (CASSANDRA-14848) When upgrading 3.11.3->4.0 using SSL 4.0 nodes does not connect to old non seed nodes

2018-11-20 Thread Tommy Stendahl (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693242#comment-16693242
 ] 

Tommy Stendahl edited comment on CASSANDRA-14848 at 11/20/18 1:38 PM:
--

I have done some more troubleshooting on the new exception I got and I don't 
think its related to my patch, I get the same issue without  my patch also. Its 
probably related to CASSANDRA-14896.
My patch do solve the issue I reported in this jira and with it the new node 
selects the correct port to all old nodes.
Patch is here: 
[cassandra-14848|]https://github.com/tommystendahl/cassandra/commit/d2a9cfe87c0e41e20ea43a75bab76f2cba8e293c


was (Author: tommy_s):
I have done some more troubleshooting on the new exception I got and I don't 
think its related to my patch, I get the same issue without  my patch also. Its 
probably related to CASSANDRA-14896.
My patch do solve the issue I reported in this jira and with it the new node 
selects the correct port to all old nodes.
Patch is here: 
[cassandra-14848|https://github.com/tommystendahl/cassandra/tree/cassandra-14848]

> When upgrading 3.11.3->4.0 using SSL 4.0 nodes does not connect to old non 
> seed nodes
> -
>
> Key: CASSANDRA-14848
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14848
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Tommy Stendahl
>Priority: Major
>  Labels: security
>
> When upgrading from 3.11.3 to 4.0 with server encryption enabled the new 4.0 
> node only connects to 3.11.3 seed node, there are no connection established 
> to non-seed nodes on the old version.
> I have four nodes, *.242 is upgraded to 4.0, *.243 and *.244 are 3.11.3 
> non-seed and *.246 are 3.11.3 seed. After starting the 4.0 node I get this 
> nodetool status on the different nodes:
> {noformat}
> *.242
> -- Address Load Tokens Owns (effective) Host ID Rack
> UN 10.216.193.242 1017.77 KiB 256 75,1% 7d278e14-d549-42f3-840d-77cfd852fbf4 
> RAC1
> DN 10.216.193.243 743.32 KiB 256 74,8% 5586243a-ca74-4125-8e7e-09e82e23c4e5 
> RAC1
> DN 10.216.193.244 711.54 KiB 256 75,2% c155e262-b898-4e86-9e1d-d4d0f97e88f6 
> RAC1
> UN 10.216.193.246 659.81 KiB 256 74,9% 502dd00f-fc02-4024-b65f-b98ba3808291 
> RAC1
> *.243 and *.244
> -- Address Load Tokens Owns (effective) Host ID Rack
> DN 10.216.193.242 657.4 KiB 256 75,1% 7d278e14-d549-42f3-840d-77cfd852fbf4 
> RAC1
> UN 10.216.193.243 471 KiB 256 74,8% 5586243a-ca74-4125-8e7e-09e82e23c4e5 RAC1
> UN 10.216.193.244 471.71 KiB 256 75,2% c155e262-b898-4e86-9e1d-d4d0f97e88f6 
> RAC1
> UN 10.216.193.246 388.54 KiB 256 74,9% 502dd00f-fc02-4024-b65f-b98ba3808291 
> RAC1
> *.246
> -- Address Load Tokens Owns (effective) Host ID Rack
> UN 10.216.193.242 657.4 KiB 256 75,1% 7d278e14-d549-42f3-840d-77cfd852fbf4 
> RAC1
> UN 10.216.193.243 471 KiB 256 74,8% 5586243a-ca74-4125-8e7e-09e82e23c4e5 RAC1
> UN 10.216.193.244 471.71 KiB 256 75,2% c155e262-b898-4e86-9e1d-d4d0f97e88f6 
> RAC1
> UN 10.216.193.246 388.54 KiB 256 74,9% 502dd00f-fc02-4024-b65f-b98ba3808291 
> RAC1
> {noformat}
>  
> I have built 4.0 with wire tracing activated and in my config the 
> storage_port=12700 and ssl_storage_port=12701. In the log I can see that the 
> 4.0 node start to connect to the 3.11.3 seed node on the storage_port but 
> quickly switch to the ssl_storage_port, but when connecting to the non-seed 
> nodes it never switch to the ssl_storage_port.
> {noformat}
> >grep 193.246 system.log | grep Outbound
> 2018-10-25T10:57:36.799+0200 [MessagingService-NettyOutbound-Thread-4-1] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x2f0e5e55] CONNECT: 
> /10.216.193.246:12700
> 2018-10-25T10:57:36.902+0200 [MessagingService-NettyOutbound-Thread-4-2] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x9e81f62c] CONNECT: 
> /10.216.193.246:12701
> 2018-10-25T10:57:36.905+0200 [MessagingService-NettyOutbound-Thread-4-2] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x9e81f62c, 
> L:/10.216.193.242:37252 - R:10.216.193.246/10.216.193.246:12701] ACTIVE
> 2018-10-25T10:57:36.906+0200 [MessagingService-NettyOutbound-Thread-4-2] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0x9e81f62c, 
> L:/10.216.193.242:37252 - R:10.216.193.246/10.216.193.246:12701] WRITE: 8B
> >grep 193.243 system.log | grep Outbound
> 2018-10-25T10:57:38.438+0200 [MessagingService-NettyOutbound-Thread-4-3] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0xd8f1d6c4] CONNECT: 
> /10.216.193.243:12700
> 2018-10-25T10:57:38.540+0200 [MessagingService-NettyOutbound-Thread-4-4] INFO 
> i.n.u.internal.logging.Slf4JLogger:101 info [id: 0xfde6cc9f] CONNECT: 
> /10.216.193.243:12700
> 2018-10-25T10:57:38.694+0200

[jira] [Commented] (CASSANDRA-14812) Multiget Thrift query processor skips records in case of digest mismatch

2018-11-20 Thread Sivukhin Nikita (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693207#comment-16693207
 ] 

Sivukhin Nikita commented on CASSANDRA-14812:
-

Thank you, [~benedict] . We tested your patch against provided integration 
tests and also try more tricky scenarios (for example, when digest mismatch 
happened in the middle of the sequence of requested keys). In all tested cases 
patch is working and the bug is not reproducing. When you plan to patch the 
master?

Also, I want to clarify one issue, just for my confidence. Is it true, that the 
machinery of request processing for Thrift queries is totally different than 
for CQL queries? I'm asking about it because the patch is very specific and if 
the same bug also contained somewhere in the CQL processing pipeline then the 
patch will not help. The code surrounding response transformation and 
filtration is slightly entangled for me, so I can't be sure that there is no 
subtly dependency between CQL and Thrift processing pipelines that can also be 
under attack of a bug.

 

> Multiget Thrift query processor skips records in case of digest mismatch
> 
>
> Key: CASSANDRA-14812
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14812
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination, Core
>Reporter: Sivukhin Nikita
>Priority: Critical
>  Labels: bug
> Attachments: repro_script.py, requirements.txt, small_repro_script.py
>
>
> It seems that in Cassandra 3.0.0 a nasty bug was introduced in {{multiget}} 
> Thrift query processing logic. When one tries to read data from several 
> partitions with a single {{multiget}} query and {{DigestMismatch}} exception 
> is raised during this query processing, request coordinator prematurely 
> terminates response stream right at the point where the first 
> \{{DigestMismatch}} error is occurring. This leads to situation where clients 
> "do not see" some data contained in the database.
> We managed to reproduce this bug in all versions of Cassandra starting with 
> v3.0.0. The pre-release version 3.0.0-rc2 works correctly. It looks like 
> [refactoring of iterator transformation 
> hierarchy|https://github.com/apache/cassandra/commit/609497471441273367013c09a1e0e1c990726ec7]
>  related to CASSANDRA-9975 triggers incorrect behaviour.
> When concatenated iterator is returned from the 
> [StorageProxy.fetchRows(...)|https://github.com/apache/cassandra/blob/a05785d82c621c9cd04d8a064c38fd2012ef981c/src/java/org/apache/cassandra/service/StorageProxy.java#L1770],
>  Cassandra starts to consume this combined iterator. Because of 
> {{DigestMismatch}} exception some elements of this combined iterator contain 
> additional {{ThriftCounter}}, that was added during 
> [DataResolver.resolve(...)|https://github.com/apache/cassandra/blob/ee9e06b5a75c0be954694b191ea4170456015b98/src/java/org/apache/cassandra/service/reads/DataResolver.java#L120]
>  execution. While consuming iterator for many partitions Cassandra calls 
> [BaseIterator.tryGetMoreContents(...)|https://github.com/apache/cassandra/blob/a05785d82c621c9cd04d8a064c38fd2012ef981c/src/java/org/apache/cassandra/db/transform/BaseIterator.java#L115]
>  method that must switch from one partition iterator to another in case of 
> exhaustion of the former. In this case all Transformations contained in the 
> next iterator are applied to the combined BaseIterator that enumerates 
> partitions sequence which is wrong. This behaviour causes BaseIterator to 
> stop enumeration after it fully consumes partition with {{DigestMismatch}} 
> error, because this partition iterator has additional {{ThriftCounter}} data 
> limit.
> The attachment contains the python2 script [^small_repro_script.py] that 
> reproduces this bug within 3-nodes ccmlib controlled cluster. Also, there is 
> an extended version of this script - [^repro_script.py] - that contains more 
> logging information and provides the ability to test behavior for many 
> Cassandra versions (to run all test cases from repro_script.py you can call 
> {{python -m unittest2 -v repro_script.ThriftMultigetTestCase}}). All the 
> necessary dependencies contained in the [^requirements.txt]
>  
> This bug is critical in our production environment because we can't permit 
> any data skip.
> Any ideas about a patch for this issue?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14906) FastByteOperations.UnsafeOperations fails to handle read only byte buffers correctly

2018-11-20 Thread Aleksandr Sorokoumov (JIRA)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksandr Sorokoumov updated CASSANDRA-14906:
-
Status: Patch Available  (was: Open)

Patch: [4.0 | https://github.com/Ge/cassandra/tree/14906-4.0]

> FastByteOperations.UnsafeOperations fails to handle read only byte buffers 
> correctly
> 
>
> Key: CASSANDRA-14906
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14906
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Aleksandr Sorokoumov
>Assignee: Aleksandr Sorokoumov
>Priority: Minor
> Fix For: 4.x
>
>
> If a buffer is read only it reports having no array, though it may well be 
> backed by an array. Code in UnsafeOperation works under the impression that a 
> buffer for which hasArray() == false is necessarily a direct buffer. Which is 
> not the case.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14855) Message Flusher scheduling fell off the event loop, resulting in out of memory

2018-11-20 Thread Benedict (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693043#comment-16693043
 ] 

Benedict commented on CASSANDRA-14855:
--

That seems like a reasonable path forward to me, yes.  I'll review the patch 
once you have it prepared.

I'd propose using a more descriptive name for the property, though, that's 
prefixed by 'native_transport'

> Message Flusher scheduling fell off the event loop, resulting in out of memory
> --
>
> Key: CASSANDRA-14855
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14855
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Major
> Fix For: 3.0.17
>
> Attachments: blocked_thread_pool.png, cpu.png, 
> eventloop_scheduledtasks.png, flusher running state.png, heap.png, 
> heap_dump.png, read_latency.png
>
>
> We recently had a production issue where about 10 nodes in a 96 node cluster 
> ran out of heap. 
> From heap dump analysis, I believe there is enough evidence to indicate 
> `queued` data member of the Flusher got too big, resulting in out of memory.
> Below are specifics on what we found from the heap dump (relevant screenshots 
> attached):
> * non-empty "queued" data member of Flusher having retaining heap of 0.5GB, 
> and multiple such instances.
> * "running" data member of Flusher having "true" value
> * Size of scheduledTasks on the eventloop was 0.
> We suspect something (maybe an exception) caused the Flusher running state to 
> continue to be true, but was not able to schedule itself with the event loop.
> Could not find any ERROR in the system.log, except for following INFO logs 
> around the incident time.
> {code:java}
> INFO [epollEventLoopGroup-2-4] 2018-xx-xx xx:xx:xx,592 Message.java:619 - 
> Unexpected exception during request; channel = [id: 0x8d288811, 
> L:/xxx.xx.xxx.xxx:7104 - R:/xxx.xx.x.xx:18886]
> io.netty.channel.unix.Errors$NativeIoException: readAddress() failed: 
> Connection timed out
>  at io.netty.channel.unix.Errors.newIOException(Errors.java:117) 
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at io.netty.channel.unix.Errors.ioResult(Errors.java:138) 
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at io.netty.channel.unix.FileDescriptor.readAddress(FileDescriptor.java:175) 
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at 
> io.netty.channel.epoll.AbstractEpollChannel.doReadBytes(AbstractEpollChannel.java:238)
>  ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at 
> io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:926)
>  ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at 
> io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:397) 
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:302) 
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at 
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
>  [netty-all-4.0.44.Final.jar:4.0.44.Final]
>  at 
> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
>  [netty-all-4.0.44.Final.jar:4.0.44.Final]
> {code}
> I would like to pursue the following proposals to fix this issue:
> # ImmediateFlusher: Backport trunk's ImmediateFlusher ( 
> [CASSANDRA-13651|https://issues.apache.org/jira/browse/CASSANDRA-13651] 
> https://github.com/apache/cassandra/commit/96ef514917e5a4829dbe864104dbc08a7d0e0cec)
>   to 3.0.x and maybe to other versions as well, since ImmediateFlusher seems 
> to be more robust than the existing Flusher as it does not depend on any 
> running state/scheduling.
> # Make "queued" data member of the Flusher bounded to avoid any potential of 
> causing out of memory due to otherwise unbounded nature.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14906) FastByteOperations.UnsafeOperations fails to handle read only byte buffers correctly

2018-11-20 Thread Aleksandr Sorokoumov (JIRA)

Aleksandr Sorokoumov created CASSANDRA-14906:


 Summary: FastByteOperations.UnsafeOperations fails to handle read 
only byte buffers correctly
 Key: CASSANDRA-14906
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14906
 Project: Cassandra
  Issue Type: Bug
Reporter: Aleksandr Sorokoumov
Assignee: Aleksandr Sorokoumov
 Fix For: 4.x


If a buffer is read only it reports having no array, though it may well be 
backed by an array. Code in UnsafeOperation works under the impression that a 
buffer for which hasArray() == false is necessarily a direct buffer. Which is 
not the case.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14905) if SizeEstimatesRecorder misses a 'onDropTable' notification, the size_estimates table will never be cleared for that table.

2018-11-20 Thread Aleksandr Sorokoumov (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692979#comment-16692979
 ] 

Aleksandr Sorokoumov edited comment on CASSANDRA-14905 at 11/20/18 10:00 AM:
-

Patches:
* [3.0|https://github.com/Ge/cassandra/tree/14905-3.0]
* [3.11|https://github.com/Ge/cassandra/tree/14905-3.11]
* [4.0|https://github.com/Ge/cassandra/tree/14905-4.0]
* [dtest|https://github.com/Ge/cassandra-dtest/tree/14905]


was (Author: ge):
Patches:
* [3.0|https://github.com/Ge/cassandra/tree/14905-3.0]
* [3.11|https://github.com/Ge/cassandra/tree/14905-3.11]
* [4.0|https://github.com/Ge/cassandra/tree/14905-4.0]

> if SizeEstimatesRecorder misses a 'onDropTable' notification, the 
> size_estimates table will never be cleared for that table.
> 
>
> Key: CASSANDRA-14905
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14905
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Aleksandr Sorokoumov
>Assignee: Aleksandr Sorokoumov
>Priority: Minor
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> if a node is down when a keyspace/table is dropped, it will receive the 
> schema notification before the size estimates listener is registered, so the 
> entries for the dropped keyspace/table will never be cleaned from the table. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14829) Make stop-server.bat wait for Cassandra to terminate

2018-11-20 Thread Dinesh Joshi (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692981#comment-16692981
 ] 

Dinesh Joshi commented on CASSANDRA-14829:
--

Hi [~georg], I am not a Windows user. If dtest cannot execute on Windows then 
perhaps we can skip it. Getting dtest working on Windows is beyond the scope of 
this ticket. I just wanted to ensure that we can avoid a regression in the 
future. For now, I'll try to test this on my Windows VM and if it works, we can 
find a committer to merge your change. Have you tried this fix on 3.x and trunk?

> Make stop-server.bat wait for Cassandra to terminate
> 
>
> Key: CASSANDRA-14829
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14829
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Packaging
> Environment: Windows 10
>Reporter: Georg Dietrich
>Assignee: Georg Dietrich
>Priority: Minor
>  Labels: easyfix, windows
> Fix For: 3.11.x, 4.x, 4.0.x
>
>
> While administering a single node Cassandra on Windows, I noticed that the 
> stop-server.bat script returns before the cassandra process has actually 
> terminated. For use cases like creating a script "shut down & create backup 
> of data directory without having to worry about open files, then restart", it 
> would be good to make stop-server.bat wait for Cassandra to terminate.
> All that is needed for that is to change in 
> apache-cassandra-3.11.3\bin\stop-server.bat "start /B powershell /file ..." 
> to "start /WAIT /B powershell /file ..." (additional /WAIT parameter).
> Does this sound reasonable?
> Here is the pull request: https://github.com/apache/cassandra/pull/287



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14905) if SizeEstimatesRecorder misses a 'onDropTable' notification, the size_estimates table will never be cleared for that table.

2018-11-20 Thread Aleksandr Sorokoumov (JIRA)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksandr Sorokoumov updated CASSANDRA-14905:
-
 Assignee: Aleksandr Sorokoumov
Fix Version/s: 4.0.x
   3.11.x
   3.0.x
   Status: Patch Available  (was: Open)

Patches:
* [3.0|https://github.com/Ge/cassandra/tree/14905-3.0]
* [3.11|https://github.com/Ge/cassandra/tree/14905-3.11]
* [4.0|https://github.com/Ge/cassandra/tree/14905-4.0]

> if SizeEstimatesRecorder misses a 'onDropTable' notification, the 
> size_estimates table will never be cleared for that table.
> 
>
> Key: CASSANDRA-14905
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14905
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Aleksandr Sorokoumov
>Assignee: Aleksandr Sorokoumov
>Priority: Minor
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> if a node is down when a keyspace/table is dropped, it will receive the 
> schema notification before the size estimates listener is registered, so the 
> entries for the dropped keyspace/table will never be cleaned from the table. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14905) if SizeEstimatesRecorder misses a 'onDropTable' notification, the size_estimates table will never be cleared for that table.

2018-11-20 Thread Aleksandr Sorokoumov (JIRA)

Aleksandr Sorokoumov created CASSANDRA-14905:


 Summary: if SizeEstimatesRecorder misses a 'onDropTable' 
notification, the size_estimates table will never be cleared for that table.
 Key: CASSANDRA-14905
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14905
 Project: Cassandra
  Issue Type: Bug
Reporter: Aleksandr Sorokoumov


if a node is down when a keyspace/table is dropped, it will receive the schema 
notification before the size estimates listener is registered, so the entries 
for the dropped keyspace/table will never be cleaned from the table. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

42 matches

Mail list logo