[
https://issues.apache.org/jira/browse/HADOOP-15027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16250933#comment-16250933
]
wujinhu commented on HADOOP-15027:
----------------------------------
Yes, agree with [~uncleGen] . We could optimize random IO step by step.
As we all known, Hadoop 2.7.5 will be released soon. We hope this patch can be
released so that we can solve the sequential IO issue(single thread read) in
current implementation. Random IO remains the same, and we can solve this later.
I have read class _*SemaphoredDelegatingExecutor*_ and it's good enough. I
think [~uncleGen] you can do this job, thanks.
Besides, I will provide more detailed test results later.
> Improvements for Hadoop read from AliyunOSS
> -------------------------------------------
>
> Key: HADOOP-15027
> URL: https://issues.apache.org/jira/browse/HADOOP-15027
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/oss
> Affects Versions: 3.0.0
> Reporter: wujinhu
> Assignee: wujinhu
> Attachments: HADOOP-15027.001.patch, HADOOP-15027.002.patch,
> HADOOP-15027.003.patch
>
>
> Currently, read performance is poor when Hadoop reads from AliyunOSS. It
> needs about 1min to read 1GB from OSS.
> Class AliyunOSSInputStream uses single thread to read data from AliyunOSS,
> so we can refactor this by using multi-thread pre read to improve this.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
