[
https://issues.apache.org/jira/browse/YARN-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14394437#comment-14394437
]
Junping Du commented on YARN-3437:
----------------------------------
Thanks [~sjlee0] for delivering a patch here!
Just quickly go through the patch, looks like we are generating one app
collector per map task. I think this is good for scalability test on backend
storage which can be a bottleneck in mainstream cases. In addition, do we want
to address some extreme cases, e.g. a huge applications will have hundreds of
thousands or even millions tasks? If so, then may be we want to know a single
app collector's bottleneck as well for accepting/forwarding messages from
hundreds of thousands maps. Also, in a real cluster, the mapping from cluster
to app, and app to tasks are all 1-N mapping. May be making app aggregator
number configurable (just like map task number, and byte per map, etc.) is
something we can do for next step?
BTW, it has some duplicated code with YARN-2556 (like
TimelineServerPerformance.java). Looks like YARN-2556 is in pretty good shape
and possible to go to trunk and branch-2 quickly. I would remind to keep
watching that JIRA status and do necessary rebase work if that patch go in and
we may want to merge it into YARN-2928 branch soon.
> convert load test driver to timeline service v.2
> ------------------------------------------------
>
> Key: YARN-3437
> URL: https://issues.apache.org/jira/browse/YARN-3437
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Sangjin Lee
> Assignee: Sangjin Lee
> Attachments: YARN-3437.001.patch
>
>
> This subtask covers the work for converting the proposed patch for the load
> test driver (YARN-2556) to work with the timeline service v.2.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
