[jira] [Updated] (HDFS-14402) Use FileChannel.transferTo() method for transferring block to SCM cache
[ https://issues.apache.org/jira/browse/HDFS-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-14402: --- Description: We will consider to use transferTo API to improve SCM's cach performace. Discussion: 5th comment in [HDFS-14355|https://issues.apache.org/jira/browse/HDFS-14355?focusedCommentId=16797627&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16797627] Comment: FileMappableBlockLoader : Currently you are reading data from input stream and verifying checksum and writing that buffer to MBB. One thought here is: how about using FileChannel#transferTo API for transferring data from one channel to other natively. and then do mmap on destination file(assuming mmap may be faster in target file) and do checksum verification on it? was: We will consider to use transferTo API to improve SCM's cach performace. Discussion: 5th comment in HDFS-14355 Comment: FileMappableBlockLoader : Currently you are reading data from input stream and verifying checksum and writing that buffer to MBB. One thought here is: how about using FileChannel#transferTo API for transferring data from one channel to other natively. and then do mmap on destination file(assuming mmap may be faster in target file) and do checksum verification on it? > Use FileChannel.transferTo() method for transferring block to SCM cache > --- > > Key: HDFS-14402 > URL: https://issues.apache.org/jira/browse/HDFS-14402 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: caching, datanode >Reporter: Feilong He >Assignee: Feilong He >Priority: Major > Labels: SCM > Fix For: 3.3.0 > > Attachments: HDFS-14402.000.patch, HDFS-14402.001.patch, > HDFS-14402.002.patch, With-Cache-Improvement-Patch.png, > Without-Cache-Improvement-Patch.png > > > We will consider to use transferTo API to improve SCM's cach performace. > Discussion: 5th comment in > [HDFS-14355|https://issues.apache.org/jira/browse/HDFS-14355?focusedCommentId=16797627&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16797627] > Comment: FileMappableBlockLoader : Currently you are reading data from input > stream and verifying checksum and writing that buffer to MBB. One thought > here is: how about using FileChannel#transferTo API for transferring data > from one channel to other natively. and then do mmap on destination > file(assuming mmap may be faster in target file) and do checksum verification > on it? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14402) Use FileChannel.transferTo() method for transferring block to SCM cache
[ https://issues.apache.org/jira/browse/HDFS-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-14402: --- Description: We will consider to use transferTo API to improve SCM's cach performace. Discussion: 5th comment in HDFS-14355 Comment: FileMappableBlockLoader : Currently you are reading data from input stream and verifying checksum and writing that buffer to MBB. One thought here is: how about using FileChannel#transferTo API for transferring data from one channel to other natively. and then do mmap on destination file(assuming mmap may be faster in target file) and do checksum verification on it? was: We will consider to use transferTo API to improve SCM's cach performace. Discussion: 5th comment in HDFS-14355 > Use FileChannel.transferTo() method for transferring block to SCM cache > --- > > Key: HDFS-14402 > URL: https://issues.apache.org/jira/browse/HDFS-14402 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: caching, datanode >Reporter: Feilong He >Assignee: Feilong He >Priority: Major > Labels: SCM > Fix For: 3.3.0 > > Attachments: HDFS-14402.000.patch, HDFS-14402.001.patch, > HDFS-14402.002.patch, With-Cache-Improvement-Patch.png, > Without-Cache-Improvement-Patch.png > > > We will consider to use transferTo API to improve SCM's cach performace. > Discussion: 5th comment in HDFS-14355 > Comment: FileMappableBlockLoader : Currently you are reading data from input > stream and verifying checksum and writing that buffer to MBB. One thought > here is: how about using FileChannel#transferTo API for transferring data > from one channel to other natively. and then do mmap on destination > file(assuming mmap may be faster in target file) and do checksum verification > on it? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14402) Use FileChannel.transferTo() method for transferring block to SCM cache
[ https://issues.apache.org/jira/browse/HDFS-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-14402: --- Description: We will consider to use transferTo API to improve SCM's cach performace. Discussion: 5th comment in HDFS-14355 was:We will consider to use transferTo API to improve SCM's cach performace. > Use FileChannel.transferTo() method for transferring block to SCM cache > --- > > Key: HDFS-14402 > URL: https://issues.apache.org/jira/browse/HDFS-14402 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: caching, datanode >Reporter: Feilong He >Assignee: Feilong He >Priority: Major > Labels: SCM > Fix For: 3.3.0 > > Attachments: HDFS-14402.000.patch, HDFS-14402.001.patch, > HDFS-14402.002.patch, With-Cache-Improvement-Patch.png, > Without-Cache-Improvement-Patch.png > > > We will consider to use transferTo API to improve SCM's cach performace. > Discussion: 5th comment in HDFS-14355 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14402) Use FileChannel.transferTo() method for transferring block to SCM cache
[ https://issues.apache.org/jira/browse/HDFS-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-14402: Labels: SCM (was: ) > Use FileChannel.transferTo() method for transferring block to SCM cache > --- > > Key: HDFS-14402 > URL: https://issues.apache.org/jira/browse/HDFS-14402 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: caching, datanode >Reporter: Feilong He >Assignee: Feilong He >Priority: Major > Labels: SCM > Fix For: 3.3.0 > > Attachments: HDFS-14402.000.patch, HDFS-14402.001.patch, > HDFS-14402.002.patch, With-Cache-Improvement-Patch.png, > Without-Cache-Improvement-Patch.png > > > We will consider to use transferTo API to improve SCM's cach performace. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14402) Use FileChannel.transferTo() method for transferring block to SCM cache
[ https://issues.apache.org/jira/browse/HDFS-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-14402: Resolution: Fixed Fix Version/s: 3.3.0 Status: Resolved (was: Patch Available) I have committed latest patch to trunk. Thanks [~PhiloHe] for the contribution! > Use FileChannel.transferTo() method for transferring block to SCM cache > --- > > Key: HDFS-14402 > URL: https://issues.apache.org/jira/browse/HDFS-14402 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: caching, datanode >Reporter: Feilong He >Assignee: Feilong He >Priority: Major > Fix For: 3.3.0 > > Attachments: HDFS-14402.000.patch, HDFS-14402.001.patch, > HDFS-14402.002.patch, With-Cache-Improvement-Patch.png, > Without-Cache-Improvement-Patch.png > > > We will consider to use transferTo API to improve SCM's cach performace. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14402) Use FileChannel.transferTo() method for transferring block to SCM cache
[ https://issues.apache.org/jira/browse/HDFS-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feilong He updated HDFS-14402: -- Attachment: (was: With-Cache-Improvement) > Use FileChannel.transferTo() method for transferring block to SCM cache > --- > > Key: HDFS-14402 > URL: https://issues.apache.org/jira/browse/HDFS-14402 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: caching, datanode >Reporter: Feilong He >Assignee: Feilong He >Priority: Major > Attachments: HDFS-14402.000.patch, HDFS-14402.001.patch, > HDFS-14402.002.patch > > > We will consider to use transferTo API to improve SCM's cach performace. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14402) Use FileChannel.transferTo() method for transferring block to SCM cache
[ https://issues.apache.org/jira/browse/HDFS-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feilong He updated HDFS-14402: -- Attachment: Without-Cache-Improvement-Patch.png > Use FileChannel.transferTo() method for transferring block to SCM cache > --- > > Key: HDFS-14402 > URL: https://issues.apache.org/jira/browse/HDFS-14402 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: caching, datanode >Reporter: Feilong He >Assignee: Feilong He >Priority: Major > Attachments: HDFS-14402.000.patch, HDFS-14402.001.patch, > HDFS-14402.002.patch, With-Cache-Improvement-Patch.png, > Without-Cache-Improvement-Patch.png > > > We will consider to use transferTo API to improve SCM's cach performace. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14402) Use FileChannel.transferTo() method for transferring block to SCM cache
[ https://issues.apache.org/jira/browse/HDFS-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feilong He updated HDFS-14402: -- Attachment: With-Cache-Improvement-Patch.png > Use FileChannel.transferTo() method for transferring block to SCM cache > --- > > Key: HDFS-14402 > URL: https://issues.apache.org/jira/browse/HDFS-14402 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: caching, datanode >Reporter: Feilong He >Assignee: Feilong He >Priority: Major > Attachments: HDFS-14402.000.patch, HDFS-14402.001.patch, > HDFS-14402.002.patch, With-Cache-Improvement-Patch.png, > Without-Cache-Improvement-Patch.png > > > We will consider to use transferTo API to improve SCM's cach performace. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14402) Use FileChannel.transferTo() method for transferring block to SCM cache
[ https://issues.apache.org/jira/browse/HDFS-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feilong He updated HDFS-14402: -- Attachment: (was: No-Cache-Improvement) > Use FileChannel.transferTo() method for transferring block to SCM cache > --- > > Key: HDFS-14402 > URL: https://issues.apache.org/jira/browse/HDFS-14402 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: caching, datanode >Reporter: Feilong He >Assignee: Feilong He >Priority: Major > Attachments: HDFS-14402.000.patch, HDFS-14402.001.patch, > HDFS-14402.002.patch > > > We will consider to use transferTo API to improve SCM's cach performace. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14402) Use FileChannel.transferTo() method for transferring block to SCM cache
[ https://issues.apache.org/jira/browse/HDFS-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feilong He updated HDFS-14402: -- Attachment: With-Cache-Improvement > Use FileChannel.transferTo() method for transferring block to SCM cache > --- > > Key: HDFS-14402 > URL: https://issues.apache.org/jira/browse/HDFS-14402 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: caching, datanode >Reporter: Feilong He >Assignee: Feilong He >Priority: Major > Attachments: HDFS-14402.000.patch, HDFS-14402.001.patch, > HDFS-14402.002.patch, No-Cache-Improvement, With-Cache-Improvement > > > We will consider to use transferTo API to improve SCM's cach performace. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14402) Use FileChannel.transferTo() method for transferring block to SCM cache
[ https://issues.apache.org/jira/browse/HDFS-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feilong He updated HDFS-14402: -- Attachment: No-Cache-Improvement > Use FileChannel.transferTo() method for transferring block to SCM cache > --- > > Key: HDFS-14402 > URL: https://issues.apache.org/jira/browse/HDFS-14402 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: caching, datanode >Reporter: Feilong He >Assignee: Feilong He >Priority: Major > Attachments: HDFS-14402.000.patch, HDFS-14402.001.patch, > HDFS-14402.002.patch, No-Cache-Improvement > > > We will consider to use transferTo API to improve SCM's cach performace. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14402) Use FileChannel.transferTo() method for transferring block to SCM cache
[ https://issues.apache.org/jira/browse/HDFS-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feilong He updated HDFS-14402: -- Attachment: HDFS-14402.001.patch > Use FileChannel.transferTo() method for transferring block to SCM cache > --- > > Key: HDFS-14402 > URL: https://issues.apache.org/jira/browse/HDFS-14402 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: caching, datanode >Reporter: Feilong He >Assignee: Feilong He >Priority: Major > Attachments: HDFS-14402.000.patch, HDFS-14402.001.patch, > HDFS-14402.002.patch > > > We will consider to use transferTo API to improve SCM's cach performace. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14402) Use FileChannel.transferTo() method for transferring block to SCM cache
[ https://issues.apache.org/jira/browse/HDFS-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feilong He updated HDFS-14402: -- Attachment: (was: HFDS-14402.001.patch) > Use FileChannel.transferTo() method for transferring block to SCM cache > --- > > Key: HDFS-14402 > URL: https://issues.apache.org/jira/browse/HDFS-14402 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: caching, datanode >Reporter: Feilong He >Assignee: Feilong He >Priority: Major > Attachments: HDFS-14402.000.patch, HDFS-14402.002.patch > > > We will consider to use transferTo API to improve SCM's cach performace. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14402) Use FileChannel.transferTo() method for transferring block to SCM cache
[ https://issues.apache.org/jira/browse/HDFS-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feilong He updated HDFS-14402: -- Attachment: HDFS-14402.002.patch > Use FileChannel.transferTo() method for transferring block to SCM cache > --- > > Key: HDFS-14402 > URL: https://issues.apache.org/jira/browse/HDFS-14402 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: caching, datanode >Reporter: Feilong He >Assignee: Feilong He >Priority: Major > Attachments: HDFS-14402.000.patch, HDFS-14402.002.patch, > HFDS-14402.001.patch > > > We will consider to use transferTo API to improve SCM's cach performace. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14402) Use FileChannel.transferTo() method for transferring block to SCM cache
[ https://issues.apache.org/jira/browse/HDFS-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-14402: Summary: Use FileChannel.transferTo() method for transferring block to SCM cache (was: Improve the implementation for HDFS cache on SCM) > Use FileChannel.transferTo() method for transferring block to SCM cache > --- > > Key: HDFS-14402 > URL: https://issues.apache.org/jira/browse/HDFS-14402 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: caching, datanode >Reporter: Feilong He >Assignee: Feilong He >Priority: Major > Attachments: HDFS-14402.000.patch, HFDS-14402.001.patch > > > We will consider to use transferTo API to improve SCM's cach performace. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
