[
https://issues.apache.org/jira/browse/HADOOP-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12623776#action_12623776
]
Amar Kamat commented on HADOOP-3970:
------------------------------------
Currently the way counters are converted to string (in jobhistory) is as follows
_groupname.countername:value_
So for a counter which has the following contents
||groupname||countername||value||
|g1|c1|v1|
|g1|c2|v2|
|g1|c3|v3|
|g2|c4|v4|
|g2|c5|v5|
We get
{{g1.c1:v1, g1.c2:v2, g1.c3:v3, g2.c4:v4, g2.c5:v5}}
One way to overcome the problem stated above is to use the length of the names
present in the counter
So the above counter might look like
{{[|g1|] g1 {[|c1|] c1:v1, [|c2|] c2:v2, [|c3|] c3:v3 }, [|g2|] g2 {[|c4|]
c4:v4, [|c5|] c5:v5 }}}
Here |s| means length of s.
Hence the length of the name in the counter helps to correctly identify the
name and helps in counter recovery.
----
Thoughts?
> Counters written to the job history cannot be recovered back
> ------------------------------------------------------------
>
> Key: HADOOP-3970
> URL: https://issues.apache.org/jira/browse/HADOOP-3970
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
>
> Counters that are written to the JobHistory are stringified using
> {{Counters.makeCompactString()}}. The format in which this api converts the
> counter into a string is _groupname.countername:value_. The problem is that
> _groupname_ and _countername_ can contain a '.' and hence recovering the
> counter becomes difficult. Since JobHistory can be used for various purposes,
> reconstructing the counter object back might be useful. One such usecase is
> HADOOP-3245. There should be some way to recover the counter object back from
> its string representation and also to keep the string version readable.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
