[
https://issues.apache.org/jira/browse/HADOOP-19236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868688#comment-17868688
]
Jinglun commented on HADOOP-19236:
----------------------------------
Thanks very much [[email protected]] for your patient and detailed comments.
Especially for point out the right direction. Let me answer the questions.
> How far have you come along with this?
We have implemented a version of hadoop-tos integration and some users are
using it. We need some extra efforts to make it satisfy the Hadoop
requirements, including design, code style, dependency issue etc.
> One issue for incorporating into the hadoop as long-term maintenance and
> testing. We do have people working on cos/oss maintenance and the reason an
> expectation that this would continue for tos.
Totally agree long-term maintenance is very important. I'm a hadoop
committer(mostly work on hdfs) and glad to maintain the hadoop-tos module.
> I've had a quick look at your new dependency and awhile it is a licensed
> appropriately; you're going to have to cut out that okio class. There are
> also going to be problems with transitive dependencies, especially jackson.
Thanks for your guide. I will fix the dependency problem.
> Now the bad news: I cannot personally commit to doing any reviewing of this
> work, or testing. I'm sorry but I am behind with review PR related to S3A and
> ABFS and any commitment I make you will be unrealistic. It would be good if
> you could actually get support for anyone working in one of the other cloud
> connect modules to see if they would assist.
That indeed is bad news. You are the most experienced expert in the object
storage filesystem, the work will go much more smoothly with your help. Again
thanks a lot for point out the right direction. I'll try to push forward the
follow-up works.
> Integration of Volcano Engine TOS in Hadoop.
> --------------------------------------------
>
> Key: HADOOP-19236
> URL: https://issues.apache.org/jira/browse/HADOOP-19236
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs, tools
> Affects Versions: 3.4.0
> Reporter: Jinglun
> Assignee: Jinglun
> Priority: Major
> Attachments: Integration of Volcano Engine TOS in Hadoop.pdf
>
>
> Volcano Engine is a fast growing cloud vendor launched by ByteDance, and TOS
> is the object storage service of Volcano Engine. A common way is to store
> data into TOS and run Hadoop/Spark/Flink applications to access TOS. But
> there is no original support for TOS in hadoop, thus it is not easy for users
> to build their Big Data System based on TOS.
>
> This work aims to integrate TOS with Hadoop to help users run their
> applications on TOS. Users only need to do some simple configuration, then
> their applications can read/write TOS without any code change. This work is
> similar to AWS S3, AzureBlob, AliyunOSS, Tencnet COS and HuaweiCloud Object
> Storage in Hadoop.
>
> Please see the attached document "Integration of Volcano Engine TOS in
> Hadoop" for more details.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
