[jira] [Commented] (HADOOP-17020) Improve RawFileSystem Performance
[
https://issues.apache.org/jira/browse/HADOOP-17020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140555#comment-17140555
]
Steve Loughran commented on HADOOP-17020:
-
BTW [~rajesh.balamohan] [~mehakmeetSingh] -do you know that
{{FilterFileSystem}} doesn't pass exists() through to the wrapped FS? so the
ChecksumFS wrap around local FS doesn't get a speedup?
someone should (probably) review the list of unimplemented methods in
org.apache.hadoop.fs.TestFilterFileSystem.MustNotImplement to see if there are
methods in there which do need to be passed down for max performance. Though as
has been shown elsewhere...that needs to be done carefully.
> Improve RawFileSystem Performance
> -
>
> Key: HADOOP-17020
> URL: https://issues.apache.org/jira/browse/HADOOP-17020
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
>Affects Versions: 3.3.0
>Reporter: Rajesh Balamohan
>Assignee: Mehakmeet Singh
>Priority: Minor
> Fix For: 3.3.1
>
> Attachments: HADOOP-17020.1.patch, Screenshot 2020-04-29 at 5.24.53
> PM.png, Screenshot 2020-05-01 at 7.12.06 AM.png
>
>
> Improving RawFileSystem performance.
> Changes:
> * RawLocalFileSystem could localize the default block size to avoid sync
> bottleneck with a Configuration object.
> [https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java#L666]
> * Exists() override method for optimization.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17020) Improve RawFileSystem Performance
[ https://issues.apache.org/jira/browse/HADOOP-17020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138570#comment-17138570 ] Hudson commented on HADOOP-17020: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18358 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/18358/]) HADOOP-17020. Improve RawFileSystem Performance (#2063) (github: rev 2bfb22840acc9f96a8bdec1ef82da37d06937da8) * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java > Improve RawFileSystem Performance > - > > Key: HADOOP-17020 > URL: https://issues.apache.org/jira/browse/HADOOP-17020 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.3.0 >Reporter: Rajesh Balamohan >Assignee: Mehakmeet Singh >Priority: Minor > Fix For: 3.3.1 > > Attachments: HADOOP-17020.1.patch, Screenshot 2020-04-29 at 5.24.53 > PM.png, Screenshot 2020-05-01 at 7.12.06 AM.png > > > Improving RawFileSystem performance. > Changes: > * RawLocalFileSystem could localize the default block size to avoid sync > bottleneck with a Configuration object. > [https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java#L666] > * Exists() override method for optimization. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
