Soumitra Sulav created HDDS-10571:
-------------------------------------
Summary: [Ozone-s3api] Chinese characters written via s3api is
breaking sh and fs operations
Key: HDDS-10571
URL: https://issues.apache.org/jira/browse/HDDS-10571
Project: Apache Ozone
Issue Type: Bug
Components: S3, s3gateway
Affects Versions: 1.5.0
Reporter: Soumitra Sulav
Assignee: Tanvi Penumudy
Read/write chinese characters works well with S3 Gateway but if we do create
such keys with Chinese characters in the name, further interoperable cli like
sh and fs breaks.
Tested with both OBS and FSO bucket layout. Delete / List fail with error on
such bucket or path.
{code:java}
Malformed input or input contains unmappable characters:{code}
Error in detail :
{code:java}
[[email protected] multi_upload]# ozone sh bucket
info /s3v/chinese-bucket
{
"metadata" : { },
"volumeName" : "s3v",
"name" : "chinese-bucket",
"storageType" : "DISK",
"versioning" : false,
"usedBytes" : 18000000000,
"usedNamespace" : 1,
"creationTime" : "2024-03-21T10:19:54.719Z",
"modificationTime" : "2024-03-21T10:19:54.719Z",
"sourcePathExist" : true,
"quotaInBytes" : -1,
"quotaInNamespace" : -1,
"bucketLayout" : "OBJECT_STORE",
"owner" : "om",
"link" : false
}
[[email protected] multi_upload]# ozone sh key
list /s3v/chinese-bucket
Malformed input or input contains unmappable characters: ??
[[email protected] multi_upload]# aws s3api
--endpoint https://ccycloud-4.quasar-jqpbgq.root.comops.site:9879/ list-objects
--bucket chinese-bucket --no-verify-ssl
/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py:1061:
InsecureRequestWarning: Unverified HTTPS request is being made to host
'ccycloud-4.quasar-jqpbgq.root.comops.site'. Adding certificate verification is
strongly advised. See:
https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
warnings.warn(
{
"Contents": [
{
"Key": "客客",
"LastModified": "2024-03-21T10:31:10.512Z",
"ETag": "2024-03-21T10:31:10.512Z",
"Size": 6000000000,
"StorageClass": "STANDARD"
}
],
"RequestCharged": null
}
[[email protected] multi_upload]# ozone sh key
delete /s3v/chinese-bucket/客客
KEY_NOT_FOUND Key not found[[email protected]
multi_upload]# ozone sh bucket info /linkvol1710602076/gkpznymu
{
"metadata" : { },
"volumeName" : "linkvol1710602076",
"name" : "gkpznymu",
"storageType" : "DISK",
"versioning" : false,
"usedBytes" : 36094371864,
"usedNamespace" : 7,
"creationTime" : "2024-03-21T08:01:57.656Z",
"modificationTime" : "2024-03-21T09:03:41.356Z",
"sourcePathExist" : true,
"quotaInBytes" : -1,
"quotaInNamespace" : -1,
"bucketLayout" : "FILE_SYSTEM_OPTIMIZED",
"owner" : "om",
"link" : false
}
[[email protected] multi_upload]# ozone fs -ls -R
ofs://ozone1710999924/linkvol1710602076/gkpznymu
-ls: Malformed input or input contains unmappable characters: ?
Usage: ozone fs [generic options] {code}
Steps to reproduce :
{code:java}
aws s3 cp 客客 s3://chi --endpoint
https://ccycloud-4.quasar-jqpbgq.root.comops.site:9879/ --no-verify-ssl
# To validate
aws s3api list-objects --bucket chi --endpoint
https://ccycloud-4.quasar-jqpbgq.root.comops.site:9879/ --no-verify-ssl
ozone sh key list /s3v/chi
## Malformed input or input contains unmappable characters: ??
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
