[
https://issues.apache.org/jira/browse/TEZ-2226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14486198#comment-14486198
]
Hitesh Shah commented on TEZ-2226:
----------------------------------
bq. However I am having some difficulties to write unit tests to test those
scenarios.
Try taking a look at overridden the historyACLPolicyManager class used in
TezClient. It could either be provided via config or through a diff ctor ( that
is only for testing ). You can create a mock acl policy manager which can then
be used to behave in different ways to throw errors as needed.
Other comments on the patch:
{code}
TezConfiguration conf = amConfig.getTezConfiguration();
388 conf.setBoolean(YarnConfiguration.TIMELINE_SERVICE_ENABLED,
loggingDefault);
389 amConfig.setTezConfiguration(conf);
{code}
This will override the original value. The same change could be done by just
amConfig.getTezConfiguration().set(...)
{code}
} catch (TezUncheckedException e) {
if (e.getMessage().contains("Fail to create domain due to fail
to post domain")) {
{code}
The above is quite error prone especially if the message changes later. Maybe
change the policy manager to throw an explicit exception that can be caught and
handled correctly?
When domain creation fails for a DAG, can you use "public DAG setConf(String
property, String value)" instead to disable logging on a per dag basis? Also,
this might be a bit tricky as there is a single logging service per AM but
there are multiple dags. Furthermore, events can get backlogged so while DAG
1's events are still in the queue waiting to be pushed to ATS, dag 2 can come
in ( with a flag saying history disabled ). This also needs to be accounted
for. WIth this in mind, you will really need to consider having a flag per dag
which can then control whether logging is enabled or not on a per dag basis.
> Disable writing history to timeline if domain creation fails.
> -------------------------------------------------------------
>
> Key: TEZ-2226
> URL: https://issues.apache.org/jira/browse/TEZ-2226
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Hitesh Shah
> Assignee: Chang Li
> Priority: Blocker
> Attachments: TEZ-2226.2.patch, TEZ-2226.3.patch, TEZ-2226.4.patch,
> TEZ-2226.patch, TEZ-2226.wip.2.patch, TEZ-2226.wip.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
