[
https://issues.apache.org/jira/browse/HDFS-9521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wellington Chevreuil updated HDFS-9521:
---------------------------------------
Attachment: HDFS-9521.patch.1
New patch version including logging for combined time for download + fsyncs to
all disks.
Also fixed minor checkstyles issues.
> TransferFsImage.receiveFile should account and log separate times for image
> download and fsync to disk
> -------------------------------------------------------------------------------------------------------
>
> Key: HDFS-9521
> URL: https://issues.apache.org/jira/browse/HDFS-9521
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Wellington Chevreuil
> Assignee: Wellington Chevreuil
> Priority: Minor
> Attachments: HDFS-9521.patch, HDFS-9521.patch.1
>
>
> Currently, TransferFsImage.receiveFile is logging total transfer time as
> below:
> {noformat}
> double xferSec = Math.max(
> ((float)(Time.monotonicNow() - startTime)) / 1000.0, 0.001);
> long xferKb = received / 1024;
> LOG.info(String.format("Transfer took %.2fs at %.2f KB/s",xferSec, xferKb /
> xferSec))
> {noformat}
> This is really useful, but it just measures the total method execution time,
> which includes time taken to download the image and do an fsync to all the
> namenode metadata directories.
> Sometime when troubleshooting these imager transfer problems, it's
> interesting to know which part of the process is being the bottleneck
> (whether network or disk write).
> This patch accounts time for image download and fsync to each disk
> separately, logging how much time did it take on each operation.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
