[
https://issues.apache.org/jira/browse/TEZ-3404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15435903#comment-15435903
]
Hitesh Shah edited comment on TEZ-3404 at 8/24/16 11:15 PM:
------------------------------------------------------------
Comments:
- the non-session dag plan code path seems a bit hacky. How about merging the
am acls and the dag access controls into the config at the client-side itself
for the non-session case? This would make the AM codepath more straight forward.
- better to delete "// tezClient.setUpHistoryAclManager(myAclPolicyManager);"
instead of commenting out.
{code}
if (sessionDomainId == null) {
172 sessionDomainId = createSessionDomain();
173 if (sessionDomainId == null) {
174 return;
175 }
176 }
{code}
- could use some logging and doc comments on what is happening here.
{code}
String dagDomainId =
dagConf.get(TezConfiguration.YARN_ATS_ACL_DAG_DOMAIN_ID);
{code}
- if the config has the value set, this means that we are not creating the
domain Id? Seems like a regression to earlier behavior.
- I believe the config based approach allows a user to override the domain id
in use but the framework should always try to create the domain id? If the
domain id is not provided via config, then the framework generates a domain id.
Was the earlier code path doing something different?
Has the patch been tested with acls disabled?
To clarify, the use of TEZ_AM_ALLOW_DISABLED_TIMELINE_DOMAINS is no longer
supported?
{code}
} catch (IOException | HistoryACLPolicyException e) {
465 LOG.warn("Could not setup ACLs for DAG. Disabling history
logging", e);
466 }
{code}
- with the exception caught, how exactly is logging being disabled?
{code}
2 if (acls != null) {
443 return
acls.get(TezConfiguration.YARN_ATS_ACL_SESSION_DOMAIN_ID);
444 }
445 } catch (HistoryACLPolicyException | IOException e) {
446 LOG.warn("Could not setup history acls, disabling history
logging.", e);
447 }
448 historyLoggingEnabled = false;
449 return null;
{code}
- I am not sure I understand the disable of logging here. If acls are
disabled, why is history logging being disabled?
ATS logger comments apply to both v1 and v15 versions.
was (Author: hitesh):
Comments:
- the non-session dag plan code path seems a bit hacky. How about merging the
am acls and the dag access controls into the config at the client-side itself
for the non-session case? This would make the AM codepath more straight forward.
- better to delete "// tezClient.setUpHistoryAclManager(myAclPolicyManager);"
instead of commenting out.
{code}
if (sessionDomainId == null) {
172 sessionDomainId = createSessionDomain();
173 if (sessionDomainId == null) {
174 return;
175 }
176 }
{code}
- could use some logging and doc comments on what is happening here.
{code}
String dagDomainId =
dagConf.get(TezConfiguration.YARN_ATS_ACL_DAG_DOMAIN_ID);
{code}
- if the config has the value set, this means that we are not creating the
domain Id? Seems like a regression to earlier behavior.
- I believe the config based approach allows a user to override the domain id
in use but the framework should always try to create the domain id? If the
domain id is not provided via config, then the framework generates a domain id.
Was the earlier code path doing something different?
Has the codepath been tested with acls disabled?
To clarify, the use of TEZ_AM_ALLOW_DISABLED_TIMELINE_DOMAINS is no longer
supported?
{code}
} catch (IOException | HistoryACLPolicyException e) {
465 LOG.warn("Could not setup ACLs for DAG. Disabling history
logging", e);
466 }
{code}
- with the exception caught, how exactly is logging being disabled?
{code}
2 if (acls != null) {
443 return
acls.get(TezConfiguration.YARN_ATS_ACL_SESSION_DOMAIN_ID);
444 }
445 } catch (HistoryACLPolicyException | IOException e) {
446 LOG.warn("Could not setup history acls, disabling history
logging.", e);
447 }
448 historyLoggingEnabled = false;
449 return null;
{code}
- I am not sure I understand the disable of logging here. If acls are
disabled, why is history logging being disabled?
ATS logger comments apply to both v1 and v15 versions.
> Remove blocking call for ATS domain creation from client side to AM
> -------------------------------------------------------------------
>
> Key: TEZ-3404
> URL: https://issues.apache.org/jira/browse/TEZ-3404
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Harish Jaiprakash
> Assignee: Harish Jaiprakash
> Attachments: TEZ-3404-WIP-01.patch, TEZ-3404-WIP-02.patch,
> TEZ-3404-WIP-03.patch
>
>
> Today, there is a blocking call for creating the domain on the client side
> which blocks hive query throughput. We need to move this to the AM and still
> retain the same security guarantees.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
