[jira] [Updated] (HDDS-3692) Consider avoiding stream/map/sum in write hotpath
[ https://issues.apache.org/jira/browse/HDDS-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Chitlangia updated HDDS-3692: Fix Version/s: 0.6.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Consider avoiding stream/map/sum in write hotpath > - > > Key: HDDS-3692 > URL: https://issues.apache.org/jira/browse/HDDS-3692 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Filesystem >Reporter: Rajesh Balamohan >Priority: Critical > Labels: Triaged, performance, pull-request-available > Fix For: 0.6.0 > > Attachments: HDDS-3692.1.patch, image-2020-06-01-11-08-08-711.png > > > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BufferPool.java#L113] > > {noformat} > public long computeBufferData() { > return bufferList.stream().mapToInt(ChunkBuffer::position) > .sum(); > } > {noformat} > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L256] > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L264] > Getting back to regular iteration should fix the issue. This causes 3x CPU > expensive when compared to HDFS flow. > > !image-2020-06-01-11-08-08-711.png|width=777,height=564! > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-3692) Consider avoiding stream/map/sum in write hotpath
[ https://issues.apache.org/jira/browse/HDDS-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3692: - Labels: Triaged performance pull-request-available (was: Triaged performance) > Consider avoiding stream/map/sum in write hotpath > - > > Key: HDDS-3692 > URL: https://issues.apache.org/jira/browse/HDDS-3692 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Filesystem >Reporter: Rajesh Balamohan >Priority: Critical > Labels: Triaged, performance, pull-request-available > Attachments: HDDS-3692.1.patch, image-2020-06-01-11-08-08-711.png > > > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BufferPool.java#L113] > > {noformat} > public long computeBufferData() { > return bufferList.stream().mapToInt(ChunkBuffer::position) > .sum(); > } > {noformat} > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L256] > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L264] > Getting back to regular iteration should fix the issue. This causes 3x CPU > expensive when compared to HDFS flow. > > !image-2020-06-01-11-08-08-711.png|width=777,height=564! > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-3692) Consider avoiding stream/map/sum in write hotpath
[ https://issues.apache.org/jira/browse/HDDS-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDDS-3692: Labels: Triaged performance (was: performance) > Consider avoiding stream/map/sum in write hotpath > - > > Key: HDDS-3692 > URL: https://issues.apache.org/jira/browse/HDDS-3692 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Filesystem >Reporter: Rajesh Balamohan >Priority: Critical > Labels: Triaged, performance > Attachments: HDDS-3692.1.patch, image-2020-06-01-11-08-08-711.png > > > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BufferPool.java#L113] > > {noformat} > public long computeBufferData() { > return bufferList.stream().mapToInt(ChunkBuffer::position) > .sum(); > } > {noformat} > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L256] > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L264] > Getting back to regular iteration should fix the issue. This causes 3x CPU > expensive when compared to HDFS flow. > > !image-2020-06-01-11-08-08-711.png|width=777,height=564! > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-3692) Consider avoiding stream/map/sum in write hotpath
[ https://issues.apache.org/jira/browse/HDDS-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDDS-3692: Priority: Critical (was: Major) > Consider avoiding stream/map/sum in write hotpath > - > > Key: HDDS-3692 > URL: https://issues.apache.org/jira/browse/HDDS-3692 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Filesystem >Reporter: Rajesh Balamohan >Priority: Critical > Labels: performance > Attachments: HDDS-3692.1.patch, image-2020-06-01-11-08-08-711.png > > > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BufferPool.java#L113] > > {noformat} > public long computeBufferData() { > return bufferList.stream().mapToInt(ChunkBuffer::position) > .sum(); > } > {noformat} > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L256] > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L264] > Getting back to regular iteration should fix the issue. This causes 3x CPU > expensive when compared to HDFS flow. > > !image-2020-06-01-11-08-08-711.png|width=777,height=564! > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-3692) Consider avoiding stream/map/sum in write hotpath
[ https://issues.apache.org/jira/browse/HDDS-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDDS-3692: Target Version/s: 0.6.0 > Consider avoiding stream/map/sum in write hotpath > - > > Key: HDDS-3692 > URL: https://issues.apache.org/jira/browse/HDDS-3692 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Filesystem >Reporter: Rajesh Balamohan >Priority: Major > Labels: performance > Attachments: HDDS-3692.1.patch, image-2020-06-01-11-08-08-711.png > > > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BufferPool.java#L113] > > {noformat} > public long computeBufferData() { > return bufferList.stream().mapToInt(ChunkBuffer::position) > .sum(); > } > {noformat} > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L256] > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L264] > Getting back to regular iteration should fix the issue. This causes 3x CPU > expensive when compared to HDFS flow. > > !image-2020-06-01-11-08-08-711.png|width=777,height=564! > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-3692) Consider avoiding stream/map/sum in write hotpath
[ https://issues.apache.org/jira/browse/HDDS-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HDDS-3692: --- Attachment: HDDS-3692.1.patch > Consider avoiding stream/map/sum in write hotpath > - > > Key: HDDS-3692 > URL: https://issues.apache.org/jira/browse/HDDS-3692 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Filesystem >Reporter: Rajesh Balamohan >Priority: Major > Labels: performance > Attachments: HDDS-3692.1.patch, image-2020-06-01-11-08-08-711.png > > > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BufferPool.java#L113] > > {noformat} > public long computeBufferData() { > return bufferList.stream().mapToInt(ChunkBuffer::position) > .sum(); > } > {noformat} > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L256] > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L264] > Getting back to regular iteration should fix the issue. This causes 3x CPU > expensive when compared to HDFS flow. > > !image-2020-06-01-11-08-08-711.png|width=777,height=564! > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-3692) Consider avoiding stream/map/sum in write hotpath
[ https://issues.apache.org/jira/browse/HDDS-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HDDS-3692: --- Status: Patch Available (was: Open) > Consider avoiding stream/map/sum in write hotpath > - > > Key: HDDS-3692 > URL: https://issues.apache.org/jira/browse/HDDS-3692 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Filesystem >Reporter: Rajesh Balamohan >Priority: Major > Labels: performance > Attachments: HDDS-3692.1.patch, image-2020-06-01-11-08-08-711.png > > > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BufferPool.java#L113] > > {noformat} > public long computeBufferData() { > return bufferList.stream().mapToInt(ChunkBuffer::position) > .sum(); > } > {noformat} > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L256] > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L264] > Getting back to regular iteration should fix the issue. This causes 3x CPU > expensive when compared to HDFS flow. > > !image-2020-06-01-11-08-08-711.png|width=777,height=564! > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-3692) Consider avoiding stream/map/sum in write hotpath
[ https://issues.apache.org/jira/browse/HDDS-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HDDS-3692: --- Labels: performance (was: ) > Consider avoiding stream/map/sum in write hotpath > - > > Key: HDDS-3692 > URL: https://issues.apache.org/jira/browse/HDDS-3692 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Filesystem >Reporter: Rajesh Balamohan >Priority: Major > Labels: performance > Attachments: image-2020-06-01-11-08-08-711.png > > > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BufferPool.java#L113] > > {noformat} > public long computeBufferData() { > return bufferList.stream().mapToInt(ChunkBuffer::position) > .sum(); > } > {noformat} > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L256] > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L264] > Getting back to regular iteration should fix the issue. This causes 3x CPU > expensive when compared to HDFS flow. > > !image-2020-06-01-11-08-08-711.png|width=777,height=564! > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-3692) Consider avoiding stream/map/sum in write hotpath
[ https://issues.apache.org/jira/browse/HDDS-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HDDS-3692: --- Description: [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BufferPool.java#L113] {noformat} public long computeBufferData() { return bufferList.stream().mapToInt(ChunkBuffer::position) .sum(); } {noformat} [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L256] [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L264] Getting back to regular iteration should fix the issue. This causes 3x CPU expensive when compared to HDFS flow. !image-2020-06-01-11-08-08-711.png|width=777,height=564! was: [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BufferPool.java#L113] {noformat} public long computeBufferData() { return bufferList.stream().mapToInt(ChunkBuffer::position) .sum(); } {noformat} [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L256] [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L264] Getting back to regular iteration should fix the issue. !image-2020-06-01-11-08-08-711.png|width=777,height=564! > Consider avoiding stream/map/sum in write hotpath > - > > Key: HDDS-3692 > URL: https://issues.apache.org/jira/browse/HDDS-3692 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Filesystem >Reporter: Rajesh Balamohan >Priority: Major > Attachments: image-2020-06-01-11-08-08-711.png > > > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BufferPool.java#L113] > > {noformat} > public long computeBufferData() { > return bufferList.stream().mapToInt(ChunkBuffer::position) > .sum(); > } > {noformat} > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L256] > [https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L264] > Getting back to regular iteration should fix the issue. This causes 3x CPU > expensive when compared to HDFS flow. > > !image-2020-06-01-11-08-08-711.png|width=777,height=564! > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org