[
https://issues.apache.org/jira/browse/HADOOP-17038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177430#comment-17177430
]
Anoop Sam John commented on HADOOP-17038:
-----------------------------------------
bq.The option should be going into hbase-site, not hadoop-site, but I fear
people doing the tuning may miss that
Oh ya u said that.. Ya exactly this is what needed. Even reducing the value
for fs.azure.read.request.size , we make sure that is added not in
core-site.xml.. For Hive like workload, this read ahead (just calling that
way) is really helpful. Fully agree to ur concern.. Actually because of this
concern, ie. reduce fs.azure.read.request.size to some thing like 512 KB for
HBase gets which affects the HBase long range scans and specially compaction
issue many many reads to Azure FS, only I came up with this patch itself.
> Support positional read in AbfsInputStream
> ------------------------------------------
>
> Key: HADOOP-17038
> URL: https://issues.apache.org/jira/browse/HADOOP-17038
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Priority: Major
> Labels: HBase, abfsactive
> Attachments: HBase Perf Test Report.xlsx, screenshot-1.png
>
>
> Right now it will do a seek to the position , read and then seek back to the
> old position. (As per the impl in the super class)
> In HBase kind of workloads we rely mostly on short preads. (like 64 KB size
> by default). So would be ideal to support a pure pos read API which will not
> even keep the data in a buffer but will only read the required data as what
> is asked for by the caller. (Not reading ahead more data as per the read size
> config)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
