[
https://issues.apache.org/jira/browse/HADOOP-12756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15128180#comment-15128180
]
Steve Loughran commented on HADOOP-12756:
-----------------------------------------
# would need its own module under hadoop-tools
# testability is always a problem with object stores —can those of in the EU/US
test against it?
# Have a look at the filesystem specification docs to see what to do,
especially in things object stores don't do well (rename, delete)
# you'll need to implement all the FS contract tests
# and help verify everything still works before any release.
Object stores are the under-supported bit of the hadoop codebase. Be advised
that there's generally little enthusiasm for adding another one; we don't
spend enough time looking after the s3 one.
Note also that as spark 2.0is going to load lib/*, which is needed to pick up
the hadoop-aws and amazon-aws-s3 JARs, you don't need to get your code into
Hadoop 2.9+ for it to be supported by spark. write something which builds
against Hadoop 2.6+ (or earlier), and get it into that dir and it'll
automatically be picked up by hadoop and spark. This is your fastest way to
getting into people's hands
finally, don't be afraid to subscribe to hadoop's common dev list and talk
about this
> Incorporate Aliyun OSS file system implementation
> -------------------------------------------------
>
> Key: HADOOP-12756
> URL: https://issues.apache.org/jira/browse/HADOOP-12756
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
> Reporter: shimingfei
> Assignee: shimingfei
> Attachments: OSS integration.pdf
>
>
> Aliyun OSS is widely used among China’s cloud users, but currently it is not
> easy to access data laid on OSS storage from user’s Hadoop/Spark application,
> because of no original support for OSS in Hadoop.
> This work aims to integrate Aliyun OSS with Hadoop. By simple configuration,
> Spark/Hadoop applications can read/write data from OSS without any code
> change. Narrowing the gap between user’s APP and data storage, like what have
> been done for S3 in Hadoop
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
