[jira] [Commented] (HDDS-3692) Consider avoiding stream/map/sum in write hotpath
[ https://issues.apache.org/jira/browse/HDDS-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17123342#comment-17123342 ] Nanda kumar commented on HDDS-3692: --- Thanks [~rajesh.balamohan] for the contribution. Added you as a contributor in HDDS. > Consider avoiding stream/map/sum in write hotpath > - > > Key: HDDS-3692 > URL: https://issues.apache.org/jira/browse/HDDS-3692 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Filesystem >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Critical > Labels: Triaged, performance, pull-request-available > Fix For: 0.6.0 > > Attachments: HDDS-3692.1.patch, image-2020-06-01-11-08-08-711.png > > > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BufferPool.java#L113] > > {noformat} > public long computeBufferData() { > return bufferList.stream().mapToInt(ChunkBuffer::position) > .sum(); > } > {noformat} > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L256] > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L264] > Getting back to regular iteration should fix the issue. This causes 3x CPU > expensive when compared to HDFS flow. > > !image-2020-06-01-11-08-08-711.png|width=777,height=564! > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-3692) Consider avoiding stream/map/sum in write hotpath
[ https://issues.apache.org/jira/browse/HDDS-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121225#comment-17121225 ] Dinesh Chitlangia commented on HDDS-3692: - [~rajesh.balamohan] - LGTM, thank you for filing the issue and the patch. Could you please submit your changes via a GitHub Pull Request ? > Consider avoiding stream/map/sum in write hotpath > - > > Key: HDDS-3692 > URL: https://issues.apache.org/jira/browse/HDDS-3692 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Filesystem >Reporter: Rajesh Balamohan >Priority: Critical > Labels: performance > Attachments: HDDS-3692.1.patch, image-2020-06-01-11-08-08-711.png > > > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BufferPool.java#L113] > > {noformat} > public long computeBufferData() { > return bufferList.stream().mapToInt(ChunkBuffer::position) > .sum(); > } > {noformat} > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L256] > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L264] > Getting back to regular iteration should fix the issue. This causes 3x CPU > expensive when compared to HDFS flow. > > !image-2020-06-01-11-08-08-711.png|width=777,height=564! > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org