[
https://issues.apache.org/jira/browse/HDFS-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16720183#comment-16720183
]
Steve Loughran commented on HDFS-14084:
---------------------------------------
h3. code
RpcDetailedMetrics should be private, not protected.
Time.monotonicNow() is only guaranteed to be monotonic on a single core in a
single server socket; it is potentially different across sockets
[http://steveloughran.blogspot.com/2015/09/time-on-multi-core-multi-socket-servers.html]
. I think you may as well go for currentTimeMillis and have special handling
for the case where time comes out negative. Because for safe execution in a
multisocket system, you need to handle that case too
h3. tests
can you make the GenericTestUtils.setLogLevel setup in a @BeforeClass and reset
in an @AfterClass method? Otherwise the logging will be at debug for the rest
of all tests run in the same JVM
h3. imports
import ordering is all wrong, please do it in the (unenforced yet expected)
order of:
{code}
java.*
---
(non-org.apache)
---
(org.apache)
--
all static
and within each block: sorted. You can set a rule for this in the IDE -but do
disable any automatic reordering, as it makes merging impossible.
> Need for more stats in DFSClient
> --------------------------------
>
> Key: HDFS-14084
> URL: https://issues.apache.org/jira/browse/HDFS-14084
> Project: Hadoop HDFS
> Issue Type: Improvement
> Affects Versions: 3.0.0
> Reporter: Pranay Singh
> Assignee: Pranay Singh
> Priority: Minor
> Attachments: HDFS-14084.001.patch, HDFS-14084.002.patch,
> HDFS-14084.003.patch, HDFS-14084.004.patch, HDFS-14084.005.patch,
> HDFS-14084.006.patch
>
>
> The usage of HDFS has changed from being used as a map-reduce filesystem, now
> it's becoming more of like a general purpose filesystem. In most of the cases
> there are issues with the Namenode so we have metrics to know the workload or
> stress on Namenode.
> However, there is a need to have more statistics collected for different
> operations/RPCs in DFSClient to know which RPC operations are taking longer
> time or to know what is the frequency of the operation.These statistics can
> be exposed to the users of DFS Client and they can periodically log or do
> some sort of flow control if the response is slow. This will also help to
> isolate HDFS issue in a mixed environment where on a node say we have Spark,
> HBase and Impala running together. We can check the throughput of different
> operation across client and isolate the problem caused because of noisy
> neighbor or network congestion or shared JVM.
> We have dealt with several problems from the field for which there is no
> conclusive evidence as to what caused the problem. If we had metrics or stats
> in DFSClient we would be better equipped to solve such complex problems.
> List of jiras for reference:
> -------------------------
> HADOOP-15538 HADOOP-15530 ( client side deadlock)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
