[
https://issues.apache.org/jira/browse/HADOOP-14219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15938235#comment-15938235
]
Julien Vaudour edited comment on HADOOP-14219 at 3/23/17 1:04 PM:
------------------------------------------------------------------
The proposed patched consider all timestamp as the more generic class
java.lang.Number
It also ignores task attempts having null hostName
was (Author: seneque):
The proposed patched consider all as the more generic class java.lang.Number
It also ignores task attempts having null hostName
> RumenToSLS: parsing problem with crashed attempts
> -------------------------------------------------
>
> Key: HADOOP-14219
> URL: https://issues.apache.org/jira/browse/HADOOP-14219
> Project: Hadoop Common
> Issue Type: Bug
> Components: tools
> Affects Versions: 2.6.0
> Reporter: Julien Vaudour
> Priority: Minor
> Attachments: HADOOP-14219.patch
>
>
> In case of crashed task attempts, we may have in rumen logs task attempts
> with null hostName and finishTime defined to -1
> for example
> {code}
> {
> "resourceUsageMetrics": {
> "heapUsage": 0,
> "physicalMemoryUsage": 0,
> "virtualMemoryUsage": 0,
> "cumulativeCpuUsage": 0
> },
> "vmemKbytes": [],
> "physMemKbytes": [],
> "cpuUsages": [],
> "clockSplits": [],
> "location": null,
> "sortFinished": -1,
> "shuffleFinished": -1,
> "spilledRecords": -1,
> "reduceOutputRecords": -1,
> "reduceShuffleBytes": -1,
> "fileBytesRead": -1,
> "hdfsBytesWritten": -1,
> "hdfsBytesRead": -1,
> "hostName": null,
> "finishTime": -1,
> "startTime": 1489619193378,
> "result": null,
> "attemptID": "attempt_1488896259152_410442_r_000015_1",
> "fileBytesWritten": -1,
> "mapInputRecords": -1,
> "mapInputBytes": -1,
> "mapOutputBytes": -1,
> "mapOutputRecords": -1,
> "combineInputRecords": -1,
> "reduceInputGroups": -1,
> "reduceInputRecords": -1
> }
> {code}
> Jackson parser will automatically consider -1 as a java.lang.Integer. However
> RumenToSLSConverter make the assumption than jackson has deserialize all
> timstamp as instance of java.lang.Long, resulting in a ClassCastException.
> RumenToSLSConverter also make the assumption that hostName is not null, so we
> can also have a NullPointerException.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
