[
https://issues.apache.org/jira/browse/HADOOP-13261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15705095#comment-15705095
]
Steve Loughran commented on HADOOP-13261:
-----------------------------------------
it'd be expensive to query though, as you do not get this information back on a
LIST; you'd need to do a HEAD on the file —way too expensive to use in split
calculation.
Where it could be used is in copy and perhaps an s3-specific distcp, where the
partition size could be propagated
> save partition split size on multipart uploads
> ----------------------------------------------
>
> Key: HADOOP-13261
> URL: https://issues.apache.org/jira/browse/HADOOP-13261
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 2.8.0
> Reporter: Steve Loughran
> Priority: Minor
>
> On multipart uploads, save the split size as a metadata value. This would
> allow split calculation optimized for the partitions to be performed in some
> bulk operation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
