[
https://issues.apache.org/jira/browse/TEZ-3357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386400#comment-15386400
]
Hitesh Shah commented on TEZ-3357:
----------------------------------
Comments:
- TEZ_HISTORY_LOGGING_USED_NUM_DAGS_PER_GROUP - can be set to scope AM for
now. It ideally does not belong in TezConfig as it is an ATS v1.5 specific
setting.
- Tez users reading javadocs are not aware of timeline groupIds. Javadocs
need more clarification on this setting.
- TEZ_HISTORY_LOGGING_NUM_DAGS_PER_GROUP_DEFAULT is an incompatible change
to today's behavior of one file/group per dag.
- "if (groupIds != null) {" - shouldn't this check now change to not null
and not empty?
{code}
203 // Warn for performance impact
204 if (allNumGroupsPerDag.size() > 3) {
205 LOG.warn("Too many entries in " +
TezConfiguration.TEZ_HISTORY_LOGGING_USED_NUM_DAGS_PER_GROUP +
206 ", you may see reduced performace for ATS/TezUI.");
207 }
{code}
- might be good to make the message more clear on where there will be a perf
issue. Also, ATS should be changed to YARN Timeline Server.
{code}
TimelineCachePluginImpl plugin =
ReflectionUtils.newInstance(TimelineCachePluginImpl.class, new Configuration());
{code}
- this should not be changed to verify that the old code path works
correctly ( i.e. where conf was not being invoked ).
Test will need changing given that
TEZ_HISTORY_LOGGING_NUM_DAGS_PER_GROUP_DEFAULT cannot be set to "50" by
default.
> Change TimelineCachePlugin to return grouped entity ids.
> --------------------------------------------------------
>
> Key: TEZ-3357
> URL: https://issues.apache.org/jira/browse/TEZ-3357
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Harish Jaiprakash
> Assignee: Harish Jaiprakash
> Attachments: TEZ-3357.01.patch, TEZ-3357.03.patch, TEZ-3357.04.patch
>
>
> TimelineCachePlugin has to return TimelineEntityGroupId grouped based on dag
> id.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
