[
https://issues.apache.org/jira/browse/TEZ-3404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423782#comment-15423782
]
Harish Jaiprakash edited comment on TEZ-3404 at 8/17/16 2:59 AM:
-----------------------------------------------------------------
* Currently ApplicationId is being passed to history acl manager class, but
ATSv1.5 needs an app attempt id, which is as of now is a dummy id, to fix this
I though we have to change the API to take appAttemptId instead of
ApplicationId.
* I'll cleanup prepareAndCreateDAGPlan, which used the parameter name
additionalConfigs, instead of aclConfigs so I thought it might have other uses.
I've removed it where it was called aclConfigs.
* If I move out the code to historyLoggingService then this is not required,
else I do the changes. I guess I was lazy and copied the old code, sorry for
that. But this will require an interface change too, hope that is fine.
* I tried quite a bit to keep it in HistoryLoggingService, but
HistoryLoggingService does not know if its running in a session mode or non
session mode, it cannot make different calls to historyAclPolicyManager in
different context. Is it ok to just create different domains for DAG and AM,
like we do for session mode for non session mode too? Or do we have to keep it
compatible with the existing code? And dag object is not available in the start
for it to setup AM acls like before for non session mode.
* Will move the conversion code out and use it in DAGImpl too.
* dagAccessControls was already being serialized into the conf object, so I
decided to use the same. I can leave at it is, or change it and remove the old
serialization code.
* I thought ATSHistoryLoggingService will be started before DAGAppMaster
serviceStart, I guess I'm wrong I'll move the code out (if its ok to keep it
DAGAppMaster).
* Sorry, I'll test this piece out, how do I ensure that I do not recreate
domain on recovery, in this case? Is there a storage available for me to use?
was (Author: harishjp):
* Currently ApplicationId is being passed to history acl manager class, but
ATSv1.5 needs and app attempt id, which is as of now is a dummy id, to fix this
I though we have to change the API to take appAttemptId instead of
ApplicationId.
* I'll cleanup prepareAndCreateDAGPlan, which used the parameter name
additionalConfigs, instead of aclConfigs so I thought it might have other uses.
I've removed it where it was called aclConfigs.
* If I move out the code to historyLoggingService then this is not required,
else I do the changes. I guess I was lazy and copied the old code, sorry for
that. But this will require an interface change too, hope that is fine.
* I tried quite a bit to keep it in HistoryLoggingService, but
HistoryLoggingService does not know if its running in a session mode or non
session mode, it cannot make different calls to historyAclPolicyManager in
different context. Is it ok to just create different domains for DAG and AM,
like we do for session mode for non session mode too? Or do we have to keep it
compatible with the existing code?
* Will move the conversion code out and use it in DAGImpl too.
* dagAccessControls was already being serialized into the conf object, so I
decided to use the same. I can leave at it is, or change it and remove the old
serialization code.
* I thought ATSHistoryLoggingService will be started before DAGAppMaster
serviceStart, I guess I'm wrong I'll move the code out (if its ok to keep it
DAGAppMaster).
* Sorry, I'll test this piece out, how do I ensure that I do not recreate
domain on recovery, in this case? Is there a storage available for me to use?
> Remove blocking call for ATS domain creation from client side to AM
> -------------------------------------------------------------------
>
> Key: TEZ-3404
> URL: https://issues.apache.org/jira/browse/TEZ-3404
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Harish Jaiprakash
> Assignee: Harish Jaiprakash
> Attachments: TEZ-3404-WIP-01.patch, TEZ-3404-WIP-02.patch
>
>
> Today, there is a blocking call for creating the domain on the client side
> which blocks hive query throughput. We need to move this to the AM and still
> retain the same security guarantees.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
