[
https://issues.apache.org/jira/browse/HADOOP-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12624635#action_12624635
]
Amar Kamat commented on HADOOP-3970:
------------------------------------
It seems that the jobhistory uses the display names instead of actual names in
the stringified representation.
One such example of actual name and display name is
_actual-name_ = {{HDFS_WRITE}} and _display-name_ = {{HDFS bytes written}}.
Note that the mapping from the actual name to the display name is mentioned in
the _.properties_ file.
Looks like its difficult to get the (actual) name from the display name because
of the following reasons
1) Display names can be same across counters.
2) There are two ways to add/increment a counter
- pass the _group_, _counter_ and the _value_
- pass the (enum) _key_ and the _value_. For the (enum) key, the name is the
classname for that (enum) _key_.
The display name is obtained from the _property_ file. In case there is no
property file corresponding to the enum, the actual name is used. In either
case, figuring out the actual name seems difficult.
So, I suggest we also add the actual name to jobhistory along with the display
name and provide api to set the display-name of the counter/group.
> Counters written to the job history cannot be recovered back
> ------------------------------------------------------------
>
> Key: HADOOP-3970
> URL: https://issues.apache.org/jira/browse/HADOOP-3970
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Attachments: HADOOP-3970-v1.patch
>
>
> Counters that are written to the JobHistory are stringified using
> {{Counters.makeCompactString()}}. The format in which this api converts the
> counter into a string is _groupname.countername:value_. The problem is that
> _groupname_ and _countername_ can contain a '.' and hence recovering the
> counter becomes difficult. Since JobHistory can be used for various purposes,
> reconstructing the counter object back might be useful. One such usecase is
> HADOOP-3245. There should be some way to recover the counter object back from
> its string representation and also to keep the string version readable.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
