[
https://issues.apache.org/jira/browse/HDDS-9229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sammi Chen updated HDDS-9229:
-----------------------------
Fix Version/s: 1.4.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> [FSO] S3A compatibility - dfs -put creates dir and a file
> ---------------------------------------------------------
>
> Key: HDDS-9229
> URL: https://issues.apache.org/jira/browse/HDDS-9229
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Mladjan Gadzic
> Assignee: Ashish Kumar
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.4.0
>
>
> Bucket with FSO layout (i.e. /s3v/fso) is created on Ozone side (unsecured
> Ozone Docker cluster).
> On Hadoop side (official 3.3.4 release) hdfs dfs -put command is issued (i.e
> README.txt file to be put to s3a://fso/).
> What happens is: 2 keys are created (dir /fso/README.txt/ and file
> README.txt/README.txt._COPYING_)
> Expected behavior: 1 key is created, file /fso/README.txt
>
> Logs from Ozone:
> {code:java}
> bash-4.2$ ozone sh key list /s3v/fso
> [ {
> "volumeName" : "s3v",
> "bucketName" : "fso",
> "name" : "README.txt/",
> "dataSize" : 0,
> "creationTime" : "2023-08-30T14:56:49.675Z",
> "modificationTime" : "2023-08-30T14:56:49.675Z",
> "replicationConfig" : {
> "replicationFactor" : "THREE",
> "requiredNodes" : 3,
> "replicationType" : "RATIS"
> },
> "metadata" : { },
> "file" : false
> }, {
> "volumeName" : "s3v",
> "bucketName" : "fso",
> "name" : "README.txt/EADME.txt._COPYING_",
> "dataSize" : 175,
> "creationTime" : "2023-08-30T14:56:49.675Z",
> "modificationTime" : "2023-08-30T14:56:50.295Z",
> "replicationConfig" : {
> "replicationFactor" : "ONE",
> "requiredNodes" : 1,
> "replicationType" : "RATIS"
> },
> "metadata" : { },
> "file" : true
> } ]{code}
>
> Logs from Hadoop:
> {code:java}
> bin/hdfs dfs -Dfs.s3a.access.key=1 -Dfs.s3a.secret.key=1
> -Dfs.s3a.endpoint=http://localhost:9878 -Dfs.s3a.path.style.access=true -put
> README.txt s3a://fso/
> 2023-08-30 14:56:41,596 INFO impl.MetricsConfig: Loaded properties from
> hadoop-metrics2.properties
> 2023-08-30 14:56:41,747 INFO impl.MetricsSystemImpl: Scheduled Metric
> snapshot period at 10 second(s).
> 2023-08-30 14:56:41,747 INFO impl.MetricsSystemImpl: s3a-file-system metrics
> system started
> 2023-08-30 14:56:43,663 INFO impl.DirectoryPolicyImpl: Directory markers will
> be kept
> 2023-08-30 14:56:50,503 INFO impl.MetricsSystemImpl: Stopping s3a-file-system
> metrics system...
> 2023-08-30 14:56:50,503 INFO impl.MetricsSystemImpl: s3a-file-system metrics
> system stopped.
> 2023-08-30 14:56:50,504 INFO impl.MetricsSystemImpl: s3a-file-system metrics
> system shutdown complete.{code}
> Hadoop configuration (core-site.xml)
>
> {code:java}
> <configuration>
> <property>
> <name>fs.s3a.access.key</name>
> <value>1</value>
> </property>
> <property>
> <name>fs.s3a.secret.key</name>
> <value>1</value>
> </property><property>
> <name>fs.s3a.capability.directory.marker.policy.delete</name>
> <value>keep</value>
> </property>
> <property>
> <name>fs.s3a.capability.directory.marker.action.delete</name>
> <value>keep</value>
> </property>
> <property>
> <name>fs.s3a.directory.marker.retention</name>
> <value>keep</value>
> </property>
> </configuration> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
