[
https://issues.apache.org/jira/browse/CASSANDRA-19979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17886974#comment-17886974
]
Jeremiah Jordan commented on CASSANDRA-19979:
---------------------------------------------
I don't know where such a change would be made for either ticket, but given
CASSANDRA-15452 is still in OPEN would it make more sense to update that to
include streaming as well? VS having an entirely different ticket tracking the
change for the streaming path? I can only assume the change is basically the
same in both places?
> Use internal buffer on streaming slow path
> ------------------------------------------
>
> Key: CASSANDRA-19979
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19979
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Jon Haddad
> Priority: Normal
> Attachments: image-2024-10-04-12-40-26-727.png
>
>
> CASSANDRA-15452 is introducing an internal buffer to compaction in order to
> increase throughput while reducing IOPS. We can do the same thing with our
> streaming slow path. There's a common misconception that the overhead comes
> from serde overhead, but I've found on a lot of devices the overhead is due
> to our read patterns. This is most commonly found on non-NVMe drives,
> especially disaggregated storage such as EBS where the latency is higher and
> more variable.
> Attached is a perf profile showing the cost of streaming is dominated by
> pread. The team I was working with was seeing they could stream only 12MB
> per streaming session. Reducing the number of read operations by using
> internal buffered reads should improve this by at least 3-5x, as well as
> reduce CPU overhead from reduced system calls.
>
>
>
>
>
> !image-2024-10-04-12-40-26-727.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
