[
https://issues.apache.org/jira/browse/FLINK-36526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zakelly Lan resolved FLINK-36526.
---------------------------------
Fix Version/s: 2.0.0
Resolution: Fixed
> Optimize the overhead of writing with direct buffer in ForSt
> -------------------------------------------------------------
>
> Key: FLINK-36526
> URL: https://issues.apache.org/jira/browse/FLINK-36526
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / State Backends
> Reporter: Zakelly Lan
> Assignee: Zakelly Lan
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.0.0
>
> Attachments: image-2024-10-14-15-52-41-457.png
>
>
> Currently, the ForSt gives a direct buffer to
> {{{}ByteBufferWritableFSDataOutputStream{}}}, where the data will be written
> one byte by byte. According our perf, the statistics of hadoop based fs will
> be updated once for each byte, which takes a lot of CPU. Below is a
> flamegraph, where the statistics part is marked as purple (taking 8.14% of
> the overall CPU).
> !image-2024-10-14-15-52-41-457.png|width=1296,height=616!
>
> It might be better to copy to a heap buffer before invoking write.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
