[
https://issues.apache.org/jira/browse/HADOOP-16117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16857457#comment-16857457
]
Steve Loughran commented on HADOOP-16117:
-----------------------------------------
thanks. committed.
I cut one of the testing.md command line tests before pushing up the patch, as
I realised it was one I'd tried which didn't actually work
{code}
echo hello | bin/hadoop fs -cat $BUCKET/file
{code}
Seems to me (and I couldn't find it) we need a way on the CLI to actually echo
something on the command line into a file. I know HDFS people hate files of a
few bytes, but if its to a cloud object store, the cost of metadata compared to
that of the (billable) data is not my problem.
I also think we need an {{listfiles}} command to do the recursive list way more
efficiently. I've got one up as a PoC elsewhere for others who need it:
https://github.com/steveloughran/cloudstore
> Upgrade to latest AWS SDK
> -------------------------
>
> Key: HADOOP-16117
> URL: https://issues.apache.org/jira/browse/HADOOP-16117
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Upgrade to the most recent AWS SDK. That's 1.11; even though there's a 2.0
> out it'll be more significant an upgrade, with impact downstream.
> The new [AWS SDK update
> process|https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md#-qualifying-an-aws-sdk-update]
> *must* be followed, and we should plan for 1-2 surprises afterwards anyway.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
