[
https://issues.apache.org/jira/browse/HDDS-10832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846246#comment-17846246
]
Attila Doroszlai commented on HDDS-10832:
-----------------------------------------
In fact the problem is not specific to S3 Gateway. Client first decides
whether to stream, then creates the key. It should first open the key, then
choose streaming only if the key's actual replication is not EC.
{{ozone sh key put}} works around the problem locally by always choosing
replication at client-side, even if not configured.
> Client should switch to streaming based on OpenKey replication
> --------------------------------------------------------------
>
> Key: HDDS-10832
> URL: https://issues.apache.org/jira/browse/HDDS-10832
> Project: Apache Ozone
> Issue Type: Bug
> Components: s3gateway
> Reporter: Attila Doroszlai
> Assignee: Attila Doroszlai
> Priority: Major
>
> {{ObjectEndpoint}} allows Ratis streaming only if bucket replication is not
> EC. However, it ignores default (global) server-side replication, so it may
> still end up using Ratis streaming with EC.
> Steps to reproduce:
> # Change server-side default replication to EC:
> {code}
> OZONE-SITE.XML_ozone.server.default.replication.type: EC
> OZONE-SITE.XML_ozone.server.default.replication: RS-3-2-1024k
> {code}
> # Create bucket without setting bucket-level replication
> {code}
> ozone sh bucket create -l OBJECT_STORE /s3v/bucket2
> {code}
> # Put large enough key to trigger Ratis streaming
> {code}
> aws s3 put-object --endpoint http://s3g:9878 --bucket bucket2 --body
> share/ozone/lib/rocksdbjni-7.7.3.jar --key key1
> {code}
> Result:
> {code}
> s3g_1 | 2024-05-08 19:51:16,885 [qtp932257672-36] WARN
> server.HttpChannel: handleException /bucket2/key1 java.io.IOException:
> java.lang.ClassCastException: class
> org.apache.hadoop.hdds.scm.ECXceiverClientGrpc cannot be cast to class
> org.apache.hadoop.hdds.scm.XceiverClientRatis
> (org.apache.hadoop.hdds.scm.ECXceiverClientGrpc and
> org.apache.hadoop.hdds.scm.XceiverClientRatis are in unnamed module of loader
> 'app')
> ...
> s3g_1 | Caused by: java.io.IOException: java.lang.ClassCastException:
> class org.apache.hadoop.hdds.scm.ECXceiverClientGrpc cannot be cast to class
> org.apache.hadoop.hdds.scm.XceiverClientRatis
> s3g_1 | at
> org.apache.hadoop.ozone.client.io.KeyDataStreamOutput.handleWrite(KeyDataStreamOutput.java:200)
> s3g_1 | at
> org.apache.hadoop.ozone.client.io.KeyDataStreamOutput.write(KeyDataStreamOutput.java:170)
> s3g_1 | at
> org.apache.hadoop.ozone.client.io.OzoneDataStreamOutput.write(OzoneDataStreamOutput.java:47)
> s3g_1 | at
> org.apache.hadoop.hdds.scm.storage.ByteBufferStreamOutput.write(ByteBufferStreamOutput.java:39)
> s3g_1 | at
> org.apache.hadoop.ozone.s3.endpoint.ObjectEndpointStreaming.writeToStreamOutput(ObjectEndpointStreaming.java:153)
> s3g_1 | at
> org.apache.hadoop.ozone.s3.endpoint.ObjectEndpointStreaming.putKeyWithStream(ObjectEndpointStreaming.java:108)
> s3g_1 | at
> org.apache.hadoop.ozone.s3.endpoint.ObjectEndpointStreaming.put(ObjectEndpointStreaming.java:68)
> s3g_1 | at
> org.apache.hadoop.ozone.s3.endpoint.ObjectEndpoint.put(ObjectEndpoint.java:307)
> ...
> s3g_1 | Caused by: java.lang.ClassCastException: class
> org.apache.hadoop.hdds.scm.ECXceiverClientGrpc cannot be cast to class
> org.apache.hadoop.hdds.scm.XceiverClientRatis
> (org.apache.hadoop.hdds.scm.ECXceiverClientGrpc and
> org.apache.hadoop.hdds.scm.XceiverClientRatis are in unnamed module of loader
> 'app')
> s3g_1 | at
> org.apache.hadoop.hdds.scm.storage.BlockDataStreamOutput.<init>(BlockDataStreamOutput.java:169)
> s3g_1 | at
> org.apache.hadoop.ozone.client.io.BlockDataStreamOutputEntry.checkStream(BlockDataStreamOutputEntry.java:99)
> s3g_1 | at
> org.apache.hadoop.ozone.client.io.BlockDataStreamOutputEntry.write(BlockDataStreamOutputEntry.java:107)
> s3g_1 | at
> org.apache.hadoop.ozone.client.io.KeyDataStreamOutput.writeToDataStreamOutput(KeyDataStreamOutput.java:212)
> s3g_1 | at
> org.apache.hadoop.ozone.client.io.KeyDataStreamOutput.handleWrite(KeyDataStreamOutput.java:190)
> s3g_1 | ... 79 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
