[
https://issues.apache.org/jira/browse/CASSANDRA-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281320#comment-15281320
]
Stefania commented on CASSANDRA-11521:
--------------------------------------
Just a quick update.
I have a tentative implementation
[here|https://github.com/apache/cassandra/compare/trunk...stef1927:11521#diff-468c74b80d7a1d5b21948217659af747R49]
based on a queue where a worker thread puts at most 3 pages and, if the queue
stays full for a specified number of seconds, then the worker thread releases
the resources and gives up. If the client keeps up, the worker continues to put
pages on the queue without releasing resources (well the pagers still create a
new partition iterator for each page but we can fix that later if it shows up
in JFR).
Unfortunately, the initial results from the [unit
tests|https://github.com/apache/cassandra/compare/trunk...stef1927:11521#diff-04e8835163e2a326515d61f448a8ebbcR75]
show that pure streaming is still about 3 times faster.
If I've read the profiles correctly, this is due to the fact that the encoding
of a {{ResultSet}} into a Netty {{ByteBuffer}} is still done synchronously when
the client requests a page. Therefore, I plan to encode the results directly
into a Netty BB ant put this, not a {{ResultSet}}, in the queue. Once this is
done, the difference between the two approaches should just be the client
request messages and the user of the pagers.
> Implement streaming for bulk read requests
> ------------------------------------------
>
> Key: CASSANDRA-11521
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11521
> Project: Cassandra
> Issue Type: Sub-task
> Components: Local Write-Read Paths
> Reporter: Stefania
> Assignee: Stefania
> Fix For: 3.x
>
>
> Allow clients to stream data from a C* host, bypassing the coordination layer
> and eliminating the need to query individual pages one by one.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
