Ahmed Hussein created HDFS-15707:
------------------------------------
Summary: NNTop counts don't add up as expected
Key: HDFS-15707
URL: https://issues.apache.org/jira/browse/HDFS-15707
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs, metrics, namenode
Reporter: Ahmed Hussein
Assignee: Ahmed Hussein
When we queried
https://host-name:port/jmx?qry=Hadoop:service=NameNode,name=FSNamesystemState
earlier today, we received surprising results. Below is a subset of the data.
The notable thing here is that the count for {{opType='\*'}} from user
{{user1}} is {{66652}}, but the sum of counts for other {{optype}} values by
{{user1}} is actually larger: {{77253}}. I'd expect that {{opType='\*'}} to
always be greater than or equal to the sum for individual operation types.
{code}
{
"windows": [
{
"windowLenMs": 300000,
"ops": [
{
"totalCount": 24158,
"opType": "rpc.complete",
"topUsers": [{ "count": 2944, "user": "user1" }]
},
{
"totalCount": 15921,
"opType": "rpc.rename",
"topUsers": [{ "count": 2891, "user": "user1" }]
},
{
"totalCount": 3015834,
"opType": "*",
"topUsers": [{ "count": 66652, "user": "user1" }]
},
{
"totalCount": 2086,
"opType": "rpc.abandonBlock",
"topUsers": [{ "count": 603, "user": "user1" }]
},
{
"totalCount": 30258,
"opType": "rpc.addBlock",
"topUsers": [{ "count": 3182, "user": "user1" }]
},
{
"totalCount": 101440,
"opType": "rpc.getServerDefaults",
"topUsers": [{ "count": 3521, "user": "user1" }]
},
{
"totalCount": 25258,
"opType": "rpc.create",
"topUsers": [{ "count": 1864, "user": "user1" }]
},
{
"totalCount": 1377563,
"opType": "rpc.getFileInfo",
"topUsers": [{ "count": 56541, "user": "user1" }]
},
{
"totalCount": 60836,
"opType": "rpc.renewLease",
"topUsers": [{ "count": 3783, "user": "user1" }]
},
{
"totalCount": 182212,
"opType": "rpc.getListing",
"topUsers": [{ "count": 1848, "user": "user1" }]
},
{
"totalCount": 380,
"opType": "rpc.updateBlockForPipeline",
"topUsers": [{ "count": 58, "user": "user1" }]
},
{
"totalCount": 215,
"opType": "rpc.updatePipeline",
"topUsers": [{ "count": 18, "user": "user1" }]
}
]
}
],
"timestamp": "2017-07-11T17:20:03+0000"
}
{code}
The full {{TopUserOpCounts}} results from the NNTop endpoint results are
attached to this JIRA as {{nntop.json}}. The full endpoint output is attached
as {{endpoint.json}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
