[
https://issues.apache.org/jira/browse/HADOOP-16791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17008747#comment-17008747
]
Steve Loughran commented on HADOOP-16791:
-----------------------------------------
Sounds good. We've just added some speedups to DurationInfo; we should be back
porting that.
Some of my colleagues (hello Hive team) have been expressing concern to me
about how long it takes object store filesystems to be created once their
initialize() method starts making remote HTTP calls to multiple Service
endpoints. This matters because FileSystem.get() Will create multiple instances
of the same FS endpoint in parallel if the initialisation is ongoing while
separate thread also call the get() method. The discussion has focused around
doing async init in S3A; abfs could add that too. You've have a separate thread
doing the token init etc -but then every public API call would need to
potentially block awaiting that init to complete.
+[~rbalamohan] [~gabor.bota]
> ABFS: Have all external dependent module execution tracked with DurationInfo
> ----------------------------------------------------------------------------
>
> Key: HADOOP-16791
> URL: https://issues.apache.org/jira/browse/HADOOP-16791
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Reporter: Sneha Vijayarajan
> Assignee: Sneha Vijayarajan
> Priority: Major
> Fix For: 3.3.0
>
>
> To be able to break down the perf impact of the external module executions
> within ABFS Driver, add execution time computation using DurationInfo in all
> the relative places.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
