[
https://issues.apache.org/jira/browse/HDDS-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-8329:
-----------------------------------
Summary: Performance Impact during container close when still Client is
writing (was: Performance Impact during container close when still Client is
writting)
> Performance Impact during container close when still Client is writing
> ----------------------------------------------------------------------
>
> Key: HDDS-8329
> URL: https://issues.apache.org/jira/browse/HDDS-8329
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Sumit Agrawal
> Assignee: Sumit Agrawal
> Priority: Major
> Labels: proton, pull-request-available
> Fix For: 1.4.0
>
>
> Steps for container closure:
> # SCM allocates blocks to client for write
> # Client perform write to blocks at DN pipeline
> # DN when identify container is about to full (default 90% of 5GB container
> size)
> # Then DN send notification to SCM for close
> # SCM move state from Open to closing and send notification to DN to close
> # At DN on success close, notify SCM for closure
> During Step 5, If client still writing to DN for blocks allocated before SCM
> receive close, those write will be rejected when DN close container.
> This causes retry for client to allocate new block and retry write.
> As solution,
> * SCM will move state to closing, so that it will not allocate new blocks to
> client
> * SCM will wait for a configured time before sending to DN close (as a
> buffer time), where Client can perform write.
> This buffer time is configurable and initially set as "2.5 minutes" which is
> 5 times DN notification interval (i.e. 30 sec * 5).
>
> Problem not intended to solve:
> * over subscription and container size - This is best approach to complete
> the write from client within configured time. Container size can cross the
> 5GB default limit, which does not much impact to system.
> * Slow client writer - If client is writing slowly, it may cross buffer time
> and can fail. This solution will provide optimal time as configured and will
> not handle slow client writer.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
