[
https://issues.apache.org/jira/browse/HDDS-11894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tanvi Penumudy updated HDDS-11894:
----------------------------------
Description:
Whenever a write operation occurs, OM calls into SCM for allocateBlock. This
adds overhead to each create request, including added latency to reach the
create and allocateBlock request, as well as resource starvation as the thread
in OM is blocked for the duration SCM responds to the call. Frequent RPC calls
to SCM can become a performance bottleneck under heavy workloads. To address
this, what if we could prefetch (preallocate) blocks from SCM asynchronously
and cache them within OM to reduce this dependency on SCM?
Design Proposal:
[https://docs.google.com/document/d/1hQAIBClitlVbZRetSPNat9Z-6qZzra8QmWLo1I2WL3k/]
was:
Whenever a write operation occurs, OM calls into SCM for allocateBlock.
Frequent RPC calls to SCM can become a performance bottleneck under heavy
workloads. To address this, what if we can prefetch (preallocate) blocks from
SCM asynchronously and cache them within OM to reduce this dependency on SCM?
Design Proposal:
https://docs.google.com/document/d/1hQAIBClitlVbZRetSPNat9Z-6qZzra8QmWLo1I2WL3k/
> PreAllocate and Cache Blocks in OM
> ----------------------------------
>
> Key: HDDS-11894
> URL: https://issues.apache.org/jira/browse/HDDS-11894
> Project: Apache Ozone
> Issue Type: New Feature
> Components: Ozone Manager, SCM
> Reporter: Tanvi Penumudy
> Assignee: Tanvi Penumudy
> Priority: Major
>
> Whenever a write operation occurs, OM calls into SCM for allocateBlock. This
> adds overhead to each create request, including added latency to reach the
> create and allocateBlock request, as well as resource starvation as the
> thread in OM is blocked for the duration SCM responds to the call. Frequent
> RPC calls to SCM can become a performance bottleneck under heavy workloads.
> To address this, what if we could prefetch (preallocate) blocks from SCM
> asynchronously and cache them within OM to reduce this dependency on SCM?
> Design Proposal:
> [https://docs.google.com/document/d/1hQAIBClitlVbZRetSPNat9Z-6qZzra8QmWLo1I2WL3k/]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
