[
https://issues.apache.org/jira/browse/HDDS-5461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Rose updated HDDS-5461:
-----------------------------
Target Version/s: 1.3.0 (was: 1.2.0)
I am managing the 1.2.0 release and we currently have more than 600 issues
targeted for 1.2.0. I am moving the target field to 1.3.0.
If you are actively working on this jira and believe this should be targeted
for the 1.2.0 release, Please reach out to me via Apache email or Slack.
> Move old objects to delete table on overwrite
> ---------------------------------------------
>
> Key: HDDS-5461
> URL: https://issues.apache.org/jira/browse/HDDS-5461
> Project: Apache Ozone
> Issue Type: Improvement
> Components: OM
> Affects Versions: 1.1.0
> Reporter: UENISHI Kota
> Assignee: UENISHI Kota
> Priority: Major
> Labels: pull-request-available
>
> HDDS-5243 was a patch for omitting key locations for clients on reading. But
> the same warning of large response size observed in our cluster for putting
> data. This is harmful in terms of retry storm, as hadoop-rpc handles this
> large-response-exception as retry-able exception. Thus,
> RetryInvocationHandler retries, despite it cannot be recovered by retry, for
> 15 times, receiving large response message exceeding default limit of RPC
> message size 128MB as follows.
> {quote}
> 2021-06-21 19:23:10,717 [IPC Server handler 65 on default port 9862] WARN
> org.apache.hadoop.ipc.Server: Large response size 134538349 for call
> Call#2037538 Retry#15
> org.apache.hadoop.ozone.om.protocol.OzoneManagerProtocol.submitRequest from
> 10.192.17.172:34070
> 2021-06-21 19:23:10,722 [IPC Server handler 65 on default port 9862] WARN
> org.apache.hadoop.ipc.Server: IPC Server handler 65 on default port 9862,
> call Call#2037538 Retry#15
> org.apache.hadoop.ozone.om.protocol.OzoneManagerProtocol.submitRequest from
> 10.192.17.172:34070: output error
> 2021-06-21 19:23:10,722 [IPC Server handler 65 on default port 9862] INFO
> org.apache.hadoop.ipc.Server: IPC Server handler 65 on default port 9862
> caught an exception
> java.nio.channels.AsynchronousCloseException
> at
> java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:205)
> at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:478)
> at org.apache.hadoop.ipc.Server.channelIO(Server.java:3642)
> at org.apache.hadoop.ipc.Server.channelWrite(Server.java:3594)
> at org.apache.hadoop.ipc.Server.access$1700(Server.java:139)
> at
> org.apache.hadoop.ipc.Server$Responder.processResponse(Server.java:1657)
> at org.apache.hadoop.ipc.Server$Responder.doRespond(Server.java:1727)
> {quote}
> Suggestion in HDDS-5393 was wrong and it shall be fixed by making old blocks
> eligible for deletion service, moving to deletion table. It is only needed
> for normal object-put, while not needed for MultipartUpload objects, if I
> understand correctly.
> Keeping old blocks and key locations after overwrite might be intended for
> supporting [object versioning
> API|https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectVersions.html],
> but IMO current design will not scale more than, say, thousands of objects.
> The order of the size of value in key table will be in O( n(version) *
> n(blocks) ), which might easily exceed current limit of RPC message (128MB by
> default) or intended value size in RocksDB. Although current implementation
> is effective for concurrency control, object versioning should be implemented
> in some different way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
