[ https://issues.apache.org/jira/browse/YARN-3706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589255#comment-14589255 ]
Joep Rottinghuis commented on YARN-3706: ---------------------------------------- Good call on the testing. Interesting restults. Loading actual history files works fine in local pseudo distributed mode. However, running the load tool errors out. I'll have to dive into what's going on there. {noformat} 15/06/16 20:24:21 ERROR mapred.SimpleEntityWriter: writing to the timeline service failed org.codehaus.jackson.JsonParseException: Unrecognized token 'foo_event_id': was expecting 'null', 'true', 'false' or NaN at [Source: [B@7bd9729f; line: 1, column: 14] at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:1433) at org.codehaus.jackson.impl.JsonParserMinimalBase._reportError(JsonParserMinimalBase.java:521) at org.codehaus.jackson.impl.Utf8StreamParser._reportInvalidToken(Utf8StreamParser.java:2274) at org.codehaus.jackson.impl.Utf8StreamParser._matchToken(Utf8StreamParser.java:2232) at org.codehaus.jackson.impl.Utf8StreamParser._nextTokenNotInObject(Utf8StreamParser.java:584) at org.codehaus.jackson.impl.Utf8StreamParser.nextToken(Utf8StreamParser.java:492) at org.codehaus.jackson.map.ObjectReader._initForReading(ObjectReader.java:828) at org.codehaus.jackson.map.ObjectReader._bindAndClose(ObjectReader.java:752) at org.codehaus.jackson.map.ObjectReader.readValue(ObjectReader.java:486) at org.apache.hadoop.yarn.server.timeline.GenericObjectMapper.read(GenericObjectMapper.java:93) at org.apache.hadoop.yarn.server.timeline.GenericObjectMapper.read(GenericObjectMapper.java:77) at org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineWriterImpl.storeEvents(HBaseTimelineWriterImpl.java:197) at org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineWriterImpl.write(HBaseTimelineWriterImpl.java:99) at org.apache.hadoop.yarn.server.timelineservice.collector.TimelineCollector.putEntities(TimelineCollector.java:93) at org.apache.hadoop.mapred.SimpleEntityWriter.writeEntities(SimpleEntityWriter.java:118) at org.apache.hadoop.mapred.TimelineServicePerformanceV2$EntityWriter.map(TimelineServicePerformanceV2.java:220) at org.apache.hadoop.mapred.TimelineServicePerformanceV2$EntityWriter.map(TimelineServicePerformanceV2.java:1) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:244) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) {noformat} It appears that the value that the GenericObjectMapper.read is reading is interpreted as a string, which probably depends on the value of the key created. I'll try to repro this in a unit test. I'm not even quite sure why I'm using a GenericObjectMapper there and not simply Bytes.readString which is certainly the correct thing to do. Seems that indeed the test case didn't contain enough values. I think once the reader is there it should be easier to crank through a bunch of entities through the combined write/read path and assert that they are equal. > Generalize native HBase writer for additional tables > ---------------------------------------------------- > > Key: YARN-3706 > URL: https://issues.apache.org/jira/browse/YARN-3706 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver > Reporter: Joep Rottinghuis > Assignee: Joep Rottinghuis > Priority: Minor > Attachments: YARN-3706-YARN-2928.001.patch, > YARN-3706-YARN-2928.010.patch, YARN-3706-YARN-2928.011.patch, > YARN-3706-YARN-2928.012.patch, YARN-3706-YARN-2928.013.patch, > YARN-3706-YARN-2928.014.patch, YARN-3726-YARN-2928.002.patch, > YARN-3726-YARN-2928.003.patch, YARN-3726-YARN-2928.004.patch, > YARN-3726-YARN-2928.005.patch, YARN-3726-YARN-2928.006.patch, > YARN-3726-YARN-2928.007.patch, YARN-3726-YARN-2928.008.patch, > YARN-3726-YARN-2928.009.patch > > > When reviewing YARN-3411 we noticed that we could change the class hierarchy > a little in order to accommodate additional tables easily. > In order to get ready for benchmark testing we left the original layout in > place, as performance would not be impacted by the code hierarchy. > Here is a separate jira to address the hierarchy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)