[
https://issues.apache.org/jira/browse/HDFS-8411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15727514#comment-15727514
]
SammiChen commented on HDFS-8411:
---------------------------------
Hi [~andrew.wang], thanks for reviewing the patch!
bq. •In StripedReader, bytesRead is incremented when the future is submitted.
However, we don't know if the future has succeeded until later on. Seems like
we should increment only for successful requests?
I'm do aware of that the future may fails. My current implementation bases on
1. If the reconstruction task succeeds finally, no matter how many future
fails, the actual successful byte read account is the same.
2. If the final reconstruction task fails, the actual byte read account is
inaccurate in this case. Given that current future task doesn't carry how many
data is read, I trade off a bit of accuracy with the complexity introduced by
refactor the future task information. I assume that the possibility is very
low. If that's not the case, I will refine the patch for a more accurate
account.
bq. •Why don't we increment the metrics directly in StripedReader /
StripedWriter rather than doing it in StripedBlockReconstructor?
Because StripedReader/StripedWriter is not only used in reconstructor work, but
also other tasks, such as striped block checksum recalculation, that should be
excluded from account into the metrics.
JIRA HDFS-11216 created for follow-on.
> Add bytes count metrics to datanode for ECWorker
> ------------------------------------------------
>
> Key: HDFS-8411
> URL: https://issues.apache.org/jira/browse/HDFS-8411
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Li Bo
> Assignee: SammiChen
> Attachments: HDFS-8411-001.patch, HDFS-8411-002.patch,
> HDFS-8411-003.patch, HDFS-8411-004.patch, HDFS-8411-005.patch,
> HDFS-8411-006.patch, HDFS-8411-007.patch, HDFS-8411-008.patch,
> HDFS-8411-009.patch
>
>
> This is a sub task of HDFS-7674. It calculates the amount of data that is
> read from local or remote to attend decoding work, and also the amount of
> data that is written to local or remote datanodes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
