[
https://issues.apache.org/jira/browse/CASSANDRA-15932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153255#comment-17153255
]
Yadong Chen commented on CASSANDRA-15932:
-----------------------------------------
I create a pull request here:
[https://github.com/apache/cassandra/pull/668]
Since it's my very first involved in this project. Advise is more than welcome.
[~benedict]
[~djoshi]
[~jwest]
Could you spare a moment for this one
> CommitLogSegmentManager#sync is doing sync one by one need to be improved
> -------------------------------------------------------------------------
>
> Key: CASSANDRA-15932
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15932
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Commit Log
> Reporter: Yadong Chen
> Assignee: Yadong Chen
> Priority: Normal
> Attachments: image-2020-07-08-12-05-07-746.png,
> image-2020-07-08-12-21-10-369.png
>
>
> test machine settings:
> # 24 cores
> # 45G memory
> # maximum disk thoughput can over 500MB/s
>
> I was running PeriodicCommitLogStressTest#testFixedSize in that test machine
> and notice both cpu and io can't be full utilized
> PeriodicCommitLogStressTest#testFixedSize was running with
> CommitLogSyncPeriod set to 10000 (I changed it to match production
> environment setting) and no compression and no encryption
> !image-2020-07-08-12-05-07-746.png!
> After some digging I find out it's the sync process can't keep up and when
> lastSyncedAt is far behind all thread doing CommitLog#add will wait until it
> catch up.
> Once catch up, all threads go back continue writing and so on.
> The overall IO utilization I guess is about 60% more or less. After introduce
> a thread pool to do sync parallel instead of doing it one by one. The IO
> utilization is improved
> !image-2020-07-08-12-21-10-369.png!
> the IO throughput is more steady (almost always above 400MB) and need less
> time to reach that throughput
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
