[
https://issues.apache.org/jira/browse/HADOOP-15616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16550230#comment-16550230
]
Steve Loughran commented on HADOOP-15616:
-----------------------------------------
bq. I noticed most previous work like s3, adl, oss, etc. are all put under
hadoop-tools. Do we have plan to move them all to under
hadoop-cloud-storage-project?
usual issue: makes backporting patches to older versions harder, and
cherrypicking is all too common.
One thing you will want to do is make sure that the hadoop-cloud-storage POM
picks you your artifacts, *excluding hadoop-common*, which you can downgrade to
provided. But do make sure that you sort our your declared dependencies
properly first (always the pain). Why this? Spark and Tez now depend on this
artifact, so will automatically get the new stuff when built with the right
profile. But you will still need to carefully work with the dependencies there.
See also https://github.com/hortonworks-spark/cloud-integration for some
integration tests downstream of spark itself
> Incorporate Tencent Cloud COS File System Implementation
> --------------------------------------------------------
>
> Key: HADOOP-15616
> URL: https://issues.apache.org/jira/browse/HADOOP-15616
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs/cos
> Reporter: Junping Du
> Assignee: YangY
> Priority: Major
>
> Tencent cloud is top 2 cloud vendors in China market and the object store COS
> （https://intl.cloud.tencent.com/product/cos） is widely used among China’s
> cloud users but now it is hard for hadoop user to access data laid on COS
> storage as no native support for COS in Hadoop.
> This work aims to integrate Tencent cloud COS with Hadoop/Spark/Hive, just
> like what we do before for S3, ADL, OSS, etc. With simple configuration,
> Hadoop applications can read/write data from COS without any code change.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
