[ 
https://issues.apache.org/jira/browse/YARN-3706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589255#comment-14589255
 ] 

Joep Rottinghuis commented on YARN-3706:
----------------------------------------

Good call on the testing.

Interesting restults. Loading actual history files works fine in local pseudo 
distributed mode.
However, running the load tool errors out. I'll have to dive into what's going 
on there.

{noformat}
15/06/16 20:24:21 ERROR mapred.SimpleEntityWriter: writing to the timeline 
service failed
org.codehaus.jackson.JsonParseException: Unrecognized token 'foo_event_id': was 
expecting 'null', 'true', 'false' or NaN
 at [Source: [B@7bd9729f; line: 1, column: 14]
        at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:1433)
        at 
org.codehaus.jackson.impl.JsonParserMinimalBase._reportError(JsonParserMinimalBase.java:521)
        at 
org.codehaus.jackson.impl.Utf8StreamParser._reportInvalidToken(Utf8StreamParser.java:2274)
        at 
org.codehaus.jackson.impl.Utf8StreamParser._matchToken(Utf8StreamParser.java:2232)
        at 
org.codehaus.jackson.impl.Utf8StreamParser._nextTokenNotInObject(Utf8StreamParser.java:584)
        at 
org.codehaus.jackson.impl.Utf8StreamParser.nextToken(Utf8StreamParser.java:492)
        at 
org.codehaus.jackson.map.ObjectReader._initForReading(ObjectReader.java:828)
        at 
org.codehaus.jackson.map.ObjectReader._bindAndClose(ObjectReader.java:752)
        at 
org.codehaus.jackson.map.ObjectReader.readValue(ObjectReader.java:486)
        at 
org.apache.hadoop.yarn.server.timeline.GenericObjectMapper.read(GenericObjectMapper.java:93)
        at 
org.apache.hadoop.yarn.server.timeline.GenericObjectMapper.read(GenericObjectMapper.java:77)
        at 
org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineWriterImpl.storeEvents(HBaseTimelineWriterImpl.java:197)
        at 
org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineWriterImpl.write(HBaseTimelineWriterImpl.java:99)
        at 
org.apache.hadoop.yarn.server.timelineservice.collector.TimelineCollector.putEntities(TimelineCollector.java:93)
        at 
org.apache.hadoop.mapred.SimpleEntityWriter.writeEntities(SimpleEntityWriter.java:118)
        at 
org.apache.hadoop.mapred.TimelineServicePerformanceV2$EntityWriter.map(TimelineServicePerformanceV2.java:220)
        at 
org.apache.hadoop.mapred.TimelineServicePerformanceV2$EntityWriter.map(TimelineServicePerformanceV2.java:1)
        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
        at 
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:244)
        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
{noformat}

It appears that the value that the GenericObjectMapper.read is reading is 
interpreted as a string, which probably depends on the value of the key 
created. I'll try to repro this in a unit test.
I'm not even quite sure why I'm using a GenericObjectMapper there and not 
simply Bytes.readString which is certainly the correct thing to do. Seems that 
indeed the test case didn't contain enough values.

I think once the reader is there it should be easier to crank through a bunch 
of entities through the combined write/read path and assert that they are equal.

> Generalize native HBase writer for additional tables
> ----------------------------------------------------
>
>                 Key: YARN-3706
>                 URL: https://issues.apache.org/jira/browse/YARN-3706
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Joep Rottinghuis
>            Assignee: Joep Rottinghuis
>            Priority: Minor
>         Attachments: YARN-3706-YARN-2928.001.patch, 
> YARN-3706-YARN-2928.010.patch, YARN-3706-YARN-2928.011.patch, 
> YARN-3706-YARN-2928.012.patch, YARN-3706-YARN-2928.013.patch, 
> YARN-3706-YARN-2928.014.patch, YARN-3726-YARN-2928.002.patch, 
> YARN-3726-YARN-2928.003.patch, YARN-3726-YARN-2928.004.patch, 
> YARN-3726-YARN-2928.005.patch, YARN-3726-YARN-2928.006.patch, 
> YARN-3726-YARN-2928.007.patch, YARN-3726-YARN-2928.008.patch, 
> YARN-3726-YARN-2928.009.patch
>
>
> When reviewing YARN-3411 we noticed that we could change the class hierarchy 
> a little in order to accommodate additional tables easily.
> In order to get ready for benchmark testing we left the original layout in 
> place, as performance would not be impacted by the code hierarchy.
> Here is a separate jira to address the hierarchy.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to