[jira] [Updated] (HDFS-8203) Erasure Coding: Seek and other Ops in DFSStripedInputStream.
[
https://issues.apache.org/jira/browse/HDFS-8203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liu updated HDFS-8203:
-
Attachment: HDFS-8203.003.patch
Thanks Jing for the review! I update the patch to address all your comments
and all related tests run successfully in my local env.
In last patch, I changed {{readOneStripe}} not need to fetch all striped group
cells and just start from the cell contains the target pos.
But I didn't modify {{StripeRange#offsetInBlock}}, it still was {{n \*
stripeLen}}, but if the *pos* is at middle of stripe cell group if there is
{{seekToNewSource}} or {{seek}}, it's not correct if we seek back and address
your #1 commet.
In this patch, I do a bit more change in {{readOneStripe}}: change the
{{StripeRange#offsetInBlock}} to be {{offsetInBlockGroup}}.
Erasure Coding: Seek and other Ops in DFSStripedInputStream.
Key: HDFS-8203
URL: https://issues.apache.org/jira/browse/HDFS-8203
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Yi Liu
Assignee: Yi Liu
Attachments: HDFS-8203.001.patch, HDFS-8203.002.patch,
HDFS-8203.003.patch
In HDFS-7782 and HDFS-8033, we handle pread and stateful read for
{{DFSStripedInputStream}}, we also need handle other operations, such as
{{seek}}, zerocopy read ...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-8203) Erasure Coding: Seek and other Ops in DFSStripedInputStream.
[
https://issues.apache.org/jira/browse/HDFS-8203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liu updated HDFS-8203:
-
Attachment: (was: HDFS-8203.003.patch)
Erasure Coding: Seek and other Ops in DFSStripedInputStream.
Key: HDFS-8203
URL: https://issues.apache.org/jira/browse/HDFS-8203
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Yi Liu
Assignee: Yi Liu
Attachments: HDFS-8203.001.patch, HDFS-8203.002.patch
In HDFS-7782 and HDFS-8033, we handle pread and stateful read for
{{DFSStripedInputStream}}, we also need handle other operations, such as
{{seek}}, zerocopy read ...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-8203) Erasure Coding: Seek and other Ops in DFSStripedInputStream.
[
https://issues.apache.org/jira/browse/HDFS-8203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liu updated HDFS-8203:
-
Attachment: HDFS-8203.003.patch
Erasure Coding: Seek and other Ops in DFSStripedInputStream.
Key: HDFS-8203
URL: https://issues.apache.org/jira/browse/HDFS-8203
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Yi Liu
Assignee: Yi Liu
Attachments: HDFS-8203.001.patch, HDFS-8203.002.patch,
HDFS-8203.003.patch
In HDFS-7782 and HDFS-8033, we handle pread and stateful read for
{{DFSStripedInputStream}}, we also need handle other operations, such as
{{seek}}, zerocopy read ...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-8203) Erasure Coding: Seek and other Ops in DFSStripedInputStream.
[
https://issues.apache.org/jira/browse/HDFS-8203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liu updated HDFS-8203:
-
Attachment: HDFS-8203.002.patch
Update the patch.
Current we have the {{curStripedBuf}}, so {{seek}} could be simpler. I update
the logic of {{readOneStripe}}.
In {{readOneStripe}}, we try to read striped group cells to fill the buffer.
But if seek happens, the target pos can be at the middle of striped group cells:
# We don't need to fetch all striped group cells, just need to start from the
cell contains the target pos.
The tests passed in my local env.
Erasure Coding: Seek and other Ops in DFSStripedInputStream.
Key: HDFS-8203
URL: https://issues.apache.org/jira/browse/HDFS-8203
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Yi Liu
Assignee: Yi Liu
Attachments: HDFS-8203.001.patch, HDFS-8203.002.patch
In HDFS-7782 and HDFS-8033, we handle pread and stateful read for
{{DFSStripedInputStream}}, we also need handle other operations, such as
{{seek}}, zerocopy read ...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-8203) Erasure Coding: Seek and other Ops in DFSStripedInputStream.
[
https://issues.apache.org/jira/browse/HDFS-8203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liu updated HDFS-8203:
-
Attachment: HDFS-8203.001.patch
Will add some test for the seek tomorrow.
Erasure Coding: Seek and other Ops in DFSStripedInputStream.
Key: HDFS-8203
URL: https://issues.apache.org/jira/browse/HDFS-8203
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Yi Liu
Assignee: Yi Liu
Attachments: HDFS-8203.001.patch
In HDFS-7782 and HDFS-8033, we handle pread and stateful read for
{{DFSStripedInputStream}}, we also need handle other operations, such as
{{seek}}, zerocopy read ...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
