Wellington Chevreuil created HDFS-9521:
------------------------------------------
Summary: TransferFsImage.receiveFile should account and log
separate times for image download and fsync to disk
Key: HDFS-9521
URL: https://issues.apache.org/jira/browse/HDFS-9521
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Wellington Chevreuil
Assignee: Wellington Chevreuil
Currently, TransferFsImage.receiveFile is logging total transfer time as below:
{noformat}
double xferSec = Math.max(
((float)(Time.monotonicNow() - startTime)) / 1000.0, 0.001);
long xferKb = received / 1024;
LOG.info(String.format("Transfer took %.2fs at %.2f KB/s",xferSec, xferKb /
xferSec))
{noformat}
This is really useful, but it just measures the total method execution time,
which includes time taken to download the image and do an fsync to all the
namenode metadata directories.
Sometime when troubleshooting these imager transfer problems, it's interesting
to know which part of the process is being the bottleneck (whether network or
disk write).
This patch accounts time for image download and fsync to each disk separately,
logging how much time did it take on each operation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
