[
https://issues.apache.org/jira/browse/HADOOP-19072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835555#comment-17835555
]
ASF GitHub Bot commented on HADOOP-19072:
-----------------------------------------
virajjasani commented on PR #6543:
URL: https://github.com/apache/hadoop/pull/6543#issuecomment-2046222247
> I'm wondering if we should provide a list of options to "optimise", e.g
"create, mkdir, delete" and the specific optimisations those get turned on.
Yes, it is suspiciously like those option sets we have for databases.
I believe this would be good optimization tuning overall, and yes it does
remind of database optimizations. In fact, HBase can use these optimizations
for HFile creation, deletion etc too (as long as it knowns when and what it is
doing). Since we already have `fs.s3a.create.performance`, how about we file
separate jiras for `fs.s3a.delete.performance` etc new modes?
or you meant, we provide a generic config `fs.s3a.optimizations` with values
including `create` and/or `delete`, accordingly we skip HEAD requests?
> S3A: expand optimisations on stores with "fs.s3a.create.performance"
> --------------------------------------------------------------------
>
> Key: HADOOP-19072
> URL: https://issues.apache.org/jira/browse/HADOOP-19072
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Steve Loughran
> Assignee: Viraj Jasani
> Priority: Major
> Labels: pull-request-available
>
> on an s3a store with fs.s3a.create.performance set, speed up other operations
> * mkdir to skip parent directory check: just do a HEAD to see if there's a
> file at the target location
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
