Neil Joshi created HDDS-7594:
--------------------------------
Summary: [FSO] Folders created through S3G are created on file
system as "files".
Key: HDDS-7594
URL: https://issues.apache.org/jira/browse/HDDS-7594
Project: Apache Ozone
Issue Type: Bug
Reporter: Neil Joshi
With FSO bucket layout buckets, folders created with the s3-gateway create 0
byte files on file system. Unlike Object Store bucket layout buckets that
create folders correctly when created through the s3-gateway, folder creation
on FSO buckets with s3 gateway results in 0 byte files. Expected is the same
behaviour. This has caused problems with interoperability for applications
accessing ozone through both ofs and the s3 gateway, such as Trino.
Problem can be produced issuing aws-cli folder creation to the s3 gateway on an
ozone cluster layout {*}FILE_SYSTEM_OPTIMIZED{*}:
{code:java}
bash-4.2$ ozone sh bucket create --layout=FILE_SYSTEM_OPTIMIZED /s3v/bucketfso
ozone$ aws s3api --profile ozone --endpoint http://localhost:9878 put-object
--bucket bucket3 --key test1/
ozone$ aws s3api --profile ozone --endpoint http://localhost:9878 put-object
--bucket bucket3 --key testdir/test.txt --body ./README.md
An error occurred (InvalidRequest) when calling the PutObject operation: An
error occurred (InvalidRequest) when calling the PutObject/MPU PartUpload
operation: ozone.om.enable.filesystem.paths is enabled Keys are considered as
Unix Paths. Path has Violated FS Semantics which caused put operation to fail.
bash-4.2$ ozone fs -ls ofs://om/s3v/bucketfso Found 2 items -rw-rw-rw- 1
hadoop hadoop 0 2022-12-05 22:34 ofs://om/s3v/bucketfso/test1
{code}
File is created _when_ a folder is expected. If it were correctly set it would
result in seeing _*drwxrwxrwx*_ for the test1 bucket as in,
{code:java}
drwxrwxrwx - hadoop hadoop 0 2022-12-05 22:40
ofs://om/s3v/bucketfos/test1
{code}
In contrast, the same aws command issued on a *OBJECT_STORE* layout bucket
results in :
{code:java}
bash-4.2$ ozone sh key list /s3v/bucket2
[ {
"volumeName" : "s3v",
"bucketName" : "bucketobs",
"name" : "test1",
"dataSize" : 0,
"creationTime" : "2022-12-05T22:33:21.737Z",
"modificationTime" : "2022-12-05T22:33:21.746Z",
"replicationConfig" : {
"replicationFactor" : "ONE",
"requiredNodes" : 1,
"replicationType" : "RATIS"
},
"metadata" : { }
}, {
"volumeName" : "s3v",
"bucketName" : "bucketobs",
"name" : "test1/README.md",
"dataSize" : 3879,
"creationTime" : "2022-12-05T22:34:02.402Z",
"modificationTime" : "2022-12-05T22:37:00.758Z",
"replicationConfig" : {
"replicationFactor" : "ONE",
"requiredNodes" : 1,
"replicationType" : "RATIS"
},
"metadata" : { }
} ]{code}
As is expected.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
