Zhijie Shen commented on YARN-3046:

Junping, thanks for the patch. Overall it looks good to me. Some comments about 
the patch details.

1. I'm not sure if we should have a MR config to determine is new or old 
timeline service. If this MR config is set to true, but YARN cluster is still 
setup with old timeline service. It still doesn't work.
433       public static final String MAPREDUCE_JOB_NEW_TIMELINE_SERVICE_ENABLED 
434           "mapreduce.job.new-timeline-service.enabled";

2. Node need to have JobHistoryEventUtils, you can move util method to 
JobHistoryUtils if you want.

3. {{TimelineClient getTimelineClient();}} shouldn't belong to AppContext, 
which will be presented to the history server too. It's better to put it into 

4. In the current way of shutting down the threadpool, is it guaranteed that 
the pending entity is going to be published before shutting down?

5. It's not documented well, but can we treat parent -> child relationship as 
the outbound edge from parent and inbound edge to child. Therefore,{{ 
entity.addRelatesToEntity(relatedJobEntity, jobId.toString());

6. In {{TaskAttemptStartedEvent}}, TASK_TYPE is written twice.

7. createTimelineEntityFromEvent -> createJobEntity and 
createTimelineEntityFromEventWithRelatedJobEntity -> createTaskEntity?

8. Use {{IOException | YarnException e}}?
1122        try {
1123          timelineClient.putEntities(tEntity);
1124        } catch (IOException ex) {
1125          LOG.error("Error putting entity " + tEntity.getId() + " to new 
1126          + "Service", ex);
1127        } catch (YarnException ex) {

> [Event producers] Implement MapReduce AM writing some MR metrics to ATS
> -----------------------------------------------------------------------
>                 Key: YARN-3046
>                 URL: https://issues.apache.org/jira/browse/YARN-3046
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Junping Du
>         Attachments: YARN-3046-no-test-v2.patch, YARN-3046-no-test.patch, 
> YARN-3046-v1-rebase.patch, YARN-3046-v1.patch
> Per design in YARN-2928, select a handful of MR metrics (e.g. HDFS bytes 
> written) and have the MR AM write the framework-specific metrics to ATS.

This message was sent by Atlassian JIRA

Reply via email to