[
https://issues.apache.org/jira/browse/HDDS-14355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Andika updated HDDS-14355:
-------------------------------
Description:
Currently, it seems that OMKeyRequest#allocateBlock calls SCM's allocateBlock
RPC even if the requested size is 0, we can skip this call for createKey to
save redundant SCM calls.
This might be useful if we only want to test the write OM metadata performance
without involving HDDS layer (i.e. SCM and DN). Additionally, S3 users might
also benefit from this since from S3G logs, there are quite a number of
requests with 0 requested data size.
Note that OFS / O3FS always use a dataSize = 0 (since FS users do not know how
much data is going to be uploaded). For that reason, {{createFile}} will not
enable the feature. Note: since {{OmKeyArgs.dataSize}} is optional, it should
be more appropriate to not specify the data size if we don't know final data
size of the key. This would make the API simpler, but since {{dataSize}} seems
to always be sent in {{{}createFile{}}}, we have to work around this.
Test should validate that SCM allocateBlock should never be called when
uploading a zero-sized key (either by OMKeyCreateRequest or
OMAllocateBlockRequest (from the BlockOutputStreamEntryPool#allocateNewBlock)).
This might be useful if we only want to test the write OM metadata performance
without involving HDDS layer (i.e. SCM and DN).
was:
Currently, it seems that OMKeyRequest#allocateBlock calls SCM's allocateBlock
RPC even if the requested size is 0, we can skip this call since the
allocateBlock is redundant. However, for Ozone FS use case
(OMFileCreateRequest), the dataSize is always 0 since Hadoop FileSystem#create
does not specify the data size in advance.
Test should validate that SCM allocateBlock should never be called when
uploading a zero-sized key (either by OMKeyCreateRequest or
OMAllocateBlockRequest (from the BlockOutputStreamEntryPool#allocateNewBlock)).
This might be useful if we only want to test the write OM metadata performance
without involving HDDS layer (i.e. SCM and DN).
> Allow OM to skip allocateBlock in createKey for empty key
> ---------------------------------------------------------
>
> Key: HDDS-14355
> URL: https://issues.apache.org/jira/browse/HDDS-14355
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Ivan Andika
> Assignee: Ivan Andika
> Priority: Major
>
> Currently, it seems that OMKeyRequest#allocateBlock calls SCM's allocateBlock
> RPC even if the requested size is 0, we can skip this call for createKey to
> save redundant SCM calls.
> This might be useful if we only want to test the write OM metadata
> performance without involving HDDS layer (i.e. SCM and DN). Additionally, S3
> users might also benefit from this since from S3G logs, there are quite a
> number of requests with 0 requested data size.
> Note that OFS / O3FS always use a dataSize = 0 (since FS users do not know
> how much data is going to be uploaded). For that reason, {{createFile}} will
> not enable the feature. Note: since {{OmKeyArgs.dataSize}} is optional, it
> should be more appropriate to not specify the data size if we don't know
> final data size of the key. This would make the API simpler, but since
> {{dataSize}} seems to always be sent in {{{}createFile{}}}, we have to work
> around this.
> Test should validate that SCM allocateBlock should never be called when
> uploading a zero-sized key (either by OMKeyCreateRequest or
> OMAllocateBlockRequest (from the
> BlockOutputStreamEntryPool#allocateNewBlock)).
> This might be useful if we only want to test the write OM metadata
> performance without involving HDDS layer (i.e. SCM and DN).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
