[jira] [Comment Edited] (MAPREDUCE-6838) [ATSv2 Security] Add timeline delegation token received in allocate response to UGI

2017-08-21 Thread Varun Saxena (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136206#comment-16136206
 ] 

Varun Saxena edited comment on MAPREDUCE-6838 at 8/22/17 3:41 AM:
--

Thanks [~jianhe] for the comments.

bq. The comment says is OR condition where as the code is AND, which one is 
true?
The code condition is correct. Will change the comment.

bq. Also, when will the "delegationToken.getService()" be empty ?
These are just checks for sanity. As NodeTimelineCollectorManager belongs to 
timelineservice module and this to yarn-common. So added these checks because 
change elsewhere should not break code here. 

bq. it uses "SecurityUtil.getTokenServiceAddr(timelineToken)" to set the token 
service. Then next time collectorAddr is not null because 
timelineServiceAddress is not null, it always call 
"NetUtils.createSocketAddr(collectorAddr) " to set the token service. Is my 
understanding correct? why not just consistently use one of them to make it 
look simpler?
So this is because we are polling on timelineservice address in another 
thread(entity dispatcher) and as soon as it is found, we go on to publish 
existing entities sitting in queue. As there can be a potential race, so I 
first update the token and then the timeline address. I can write a comment in 
code to make this clear.

bq. Does the collector address change if NM restarts? If so, we may have two 
keys(different address) for two tokens in the UGI.
Yes, that's true but the token will be picked up by 
DelegationTokenAuthenticatedURL based on current collector address. Could not 
find any API to remove the token from UGI. Not sure why. Should we add one?



was (Author: varun_saxena):
Thanks [~jianhe] for the comments.

bq. The comment says is OR condition where as the code is AND, which one is 
true?
The code condition is correct. Will change the comment.

bq. Also, when will the "delegationToken.getService()" be empty ?
These are just checks for sanity. As NodeTimelineCollectorManager belongs to 
timelineservice module and this to yarn-common. So added these checks because 
change elsewhere should not break code here. 

bq. it uses "SecurityUtil.getTokenServiceAddr(timelineToken)" to set the token 
service. Then next time collectorAddr is not null because 
timelineServiceAddress is not null, it always call 
"NetUtils.createSocketAddr(collectorAddr) " to set the token service. Is my 
understanding correct? why not just consistently use one of them to make it 
look simpler?
So this is because we are polling on timelineservice address in another 
thread(entity dispatcher) and as soon as it is found, we go on to publish. So 
there can be a potential race so I first update the token and then the timeline 
address. I can write a comment in code to make this clear.

bq. Does the collector address change if NM restarts? If so, we may have two 
keys(different address) for two tokens in the UGI.
Yes, that's true but the token will be picked up by 
DelegationTokenAuthenticatedURL based on current collector address. Could not 
find any API to remove the token from UGI. Not sure why. Should we add one?


> [ATSv2 Security] Add timeline delegation token received in allocate response 
> to UGI
> ---
>
> Key: MAPREDUCE-6838
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6838
> Project: Hadoop Map/Reduce
>  Issue Type: Sub-task
>Reporter: Varun Saxena
>Assignee: Varun Saxena
>  Labels: yarn-5355-merge-blocker
> Fix For: YARN-5355
>
> Attachments: MAPREDUCE-6838-YARN-5355.01.patch, 
> MAPREDUCE-6838-YARN-5355.02.patch, MAPREDUCE-6838-YARN-5355.03.patch, 
> MAPREDUCE-6838-YARN-5355.04.patch, MAPREDUCE-6838-YARN-5355.05.patch, 
> MAPREDUCE-6838-YARN-5355.06.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Comment Edited] (MAPREDUCE-6838) [ATSv2 Security] Add timeline delegation token received in allocate response to UGI

2017-08-21 Thread Varun Saxena (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136206#comment-16136206
 ] 

Varun Saxena edited comment on MAPREDUCE-6838 at 8/22/17 3:37 AM:
--

Thanks [~jianhe] for the comments.

bq. The comment says is OR condition where as the code is AND, which one is 
true?
The code condition is correct. Will change the comment.

bq. Also, when will the "delegationToken.getService()" be empty ?
These are just checks for sanity. As NodeTimelineCollectorManager belongs to 
timelineservice module and this to yarn-common. So added these checks because 
change elsewhere should not break code here. 

bq. it uses "SecurityUtil.getTokenServiceAddr(timelineToken)" to set the token 
service. Then next time collectorAddr is not null because 
timelineServiceAddress is not null, it always call 
"NetUtils.createSocketAddr(collectorAddr) " to set the token service. Is my 
understanding correct? why not just consistently use one of them to make it 
look simpler?
So this is because we are polling on timelineservice address in another 
thread(entity dispatcher) and as soon as it is found, we go on to publish. So 
there can be a potential race so I first update the token and then the timeline 
address. I can write a comment in code to make this clear.

bq. Does the collector address change if NM restarts? If so, we may have two 
keys(different address) for two tokens in the UGI.
Yes, that's true but the token will be picked up by 
DelegationTokenAuthenticatedURL based on current collector address. Could not 
find any API to remove the token from UGI. Not sure why. Should we add one?



was (Author: varun_saxena):
Thanks [~jianhe] for the comments.

bq. The comment says is OR condition where as the code is AND, which one is 
true?
The code condition is correct. Will change the comment.

bq. Also, when will the "delegationToken.getService()" be empty ?
These are just checks for sanity. As NodeTimelineCollectorManager belongs to 
timelineservice module and this to yarn-common. So added these checks because 
change elsewhere should not break code here. 

bq. it uses "SecurityUtil.getTokenServiceAddr(timelineToken)" to set the token 
service. Then next time collectorAddr is not null because 
timelineServiceAddress is not null, it always call 
"NetUtils.createSocketAddr(collectorAddr) " to set the token service. Is my 
understanding correct? why not just consistently use one of them to make it 
look simpler?
So this is because we are polling on timelineservice address in another 
thread(entity dispatcher) and as soon as it is found, we go on to publish. So 
there can be a potential race so I first update the token and then the timeline 
address. I can write a comment in code to make this clear.

bq. Does the collector address change if NM restarts? If so, we may have two 
keys(different address) for two tokens in the UGI.
Yes, that's true but the token will be picked up by 
DelegationTokenAuthenticatedURL based on current collector address. Could not 
find any API to remove the token from UGI.


> [ATSv2 Security] Add timeline delegation token received in allocate response 
> to UGI
> ---
>
> Key: MAPREDUCE-6838
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6838
> Project: Hadoop Map/Reduce
>  Issue Type: Sub-task
>Reporter: Varun Saxena
>Assignee: Varun Saxena
>  Labels: yarn-5355-merge-blocker
> Fix For: YARN-5355
>
> Attachments: MAPREDUCE-6838-YARN-5355.01.patch, 
> MAPREDUCE-6838-YARN-5355.02.patch, MAPREDUCE-6838-YARN-5355.03.patch, 
> MAPREDUCE-6838-YARN-5355.04.patch, MAPREDUCE-6838-YARN-5355.05.patch, 
> MAPREDUCE-6838-YARN-5355.06.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Comment Edited] (MAPREDUCE-6838) [ATSv2 Security] Add timeline delegation token received in allocate response to UGI

2017-08-19 Thread Varun Saxena (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134241#comment-16134241
 ] 

Varun Saxena edited comment on MAPREDUCE-6838 at 8/19/17 9:23 PM:
--

Thanks [~rohithsharma] for the review.

bq. Need to log a WARN message if collector info is null.
Ok.

bq. currentTimelineToken should be volatile
Need not be. Atleast in MR AM. This is used only while AM is updating the token 
and that happens only from RMContainer Allocator thread so only one thread sees 
and updates it. While using token, it is picked from UGI. Will it be likely 
that token will be updated from 2 separate threads?
Address is volatile and its different because the thread publishing the entity 
and using the address would be different from the allocator thread which would 
communicate with RM and update the address. However, making it volatile doesn't 
cost us anything. Infact it helps us handle cases when token is updated from 2 
threads(if any such situation arises). We can make it volatile I guess.

bq. Creating Token does not required to check service==null. Internally 
constructor does. And we can ignore token service passed by delegationToken 
always and set it up collector address.
You mean the constructor inside setTimelineDelegationToken method i.e. at L203? 
Actually the constructor takes service as Text and not String. The check I am 
making is for service as String. If I do not make the check and call new 
Text(service), a null service would throw NPE.

bq. 
!delegationToken.getKind().equals(TimelineDelegationTokenIdentifier.KIND_NAME.toString())
 check is not required since equals does this comparrission too.
Didn't quite get you. This is to avoid updating token for another kind. This is 
to avoid updating a token altogether i.e. even if we do not have a previous 
token. The equals check is for not updating the token if it is equal to cached 
token. If I remove this check, a token of another kind will be added in UGI.

bq. In CollectorInfo object, If collector address is null and Token is 
non-null. Do not add that token into ugi.
Check like this required? If token service exists and timeline service address 
is already updated, should we not update the token, if we look at this piece of 
code independently. Currently we send both together but the protocol doesn't 
enforce it. The proto definition of CollectorInfo marks collector address field 
as optional.
These checks are primarily for robustness if we consider the TimelineV2Client 
code in isolation and not merely go by what we currently know RM sends. If we 
make assumptions based on current implementation, we are tightly coupling the 
RM/NM logic with logic here and it is not enforced by protocol either. It is 
likely to work just fine as developers would take care but I would suggest that 
ideally if we assume that collector address is carried always, we enforce it in 
proto definition of CollectorInfo i.e. make collector address as "required" 
instead of "optional" in it. And we will have to see if address should be 
"required" in AppCollectorData too.
Please note that RM may not have access to collector address initially when AM 
container is launched. So this change would also mean change in RM to not send 
collector info at all if address is null.

The last comment i.e. suggested refactoring depends on comments above i.e. 
whether to update the token or not if address is not carried in collector info. 
Thoughts?


was (Author: varun_saxena):
Thanks [~rohithsharma] for the review.

bq. Need to log a WARN message if collector info is null.
Ok.

bq. currentTimelineToken should be volatile
Need not be. Atleast in MR AM. This is used only while AM is updating the token 
and that happens only from RMContainer Allocator thread so only one thread sees 
and updates it. While using token, it is picked from UGI. Will it be likely 
that token will be updated from 2 separate threads? We anyways do not claim any 
thread safety for timeline client. 
Address is volatile and its different because the thread publishing the entity 
and using the address would be different from the allocator thread which would 
communicate with RM and update the address. However, making it volatile doesn't 
cost us anything. As you say. I do not have a strong opinion on this. Thoughts?

bq. Creating Token does not required to check service==null. Internally 
constructor does. And we can ignore token service passed by delegationToken 
always and set it up collector address.
You mean the constructor inside setTimelineDelegationToken method i.e. at L203? 
Actually the constructor takes service as Text and not String. The check I am 
making is for service as String. If I do not make the check and call new 
Text(service), a null service would throw NPE.

bq. 
!delegationToken.getKind().equals(TimelineDelegationTokenIdentifier.KIND_NAME.toString())
 check is not 

[jira] [Comment Edited] (MAPREDUCE-6838) [ATSv2 Security] Add timeline delegation token received in allocate response to UGI

2017-08-19 Thread Varun Saxena (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134241#comment-16134241
 ] 

Varun Saxena edited comment on MAPREDUCE-6838 at 8/19/17 8:36 PM:
--

Thanks [~rohithsharma] for the review.

bq. Need to log a WARN message if collector info is null.
Ok.

bq. currentTimelineToken should be volatile
Need not be. Atleast in MR AM. This is used only while AM is updating the token 
and that happens only from RMContainer Allocator thread so only one thread sees 
and updates it. While using token, it is picked from UGI. Will it be likely 
that token will be updated from 2 separate threads? We anyways do not claim any 
thread safety for timeline client. 
Address is volatile and its different because the thread publishing the entity 
and using the address would be different from the allocator thread which would 
communicate with RM and update the address. However, making it volatile doesn't 
cost us anything. As you say. I do not have a strong opinion on this. Thoughts?

bq. Creating Token does not required to check service==null. Internally 
constructor does. And we can ignore token service passed by delegationToken 
always and set it up collector address.
You mean the constructor inside setTimelineDelegationToken method i.e. at L203? 
Actually the constructor takes service as Text and not String. The check I am 
making is for service as String. If I do not make the check and call new 
Text(service), a null service would throw NPE.

bq. 
!delegationToken.getKind().equals(TimelineDelegationTokenIdentifier.KIND_NAME.toString())
 check is not required since equals does this comparrission too.
Didn't quite get you. This is to avoid updating token for another kind. This is 
to avoid updating a token altogether i.e. even if we do not have a previous 
token. The equals check is for not updating the token if it is equal to cached 
token. If I remove this check, a token of another kind will be added in UGI.

bq. In CollectorInfo object, If collector address is null and Token is 
non-null. Do not add that token into ugi.
Check like this required? If token service exists and timeline service address 
is already updated, should we not update the token, if we look at this piece of 
code independently. Currently we send both together but the protocol doesn't 
enforce it. The proto definition of CollectorInfo marks collector address field 
as optional.
These checks are primarily for robustness if we consider the TimelineV2Client 
code in isolation and not merely go by what we currently know RM sends. If we 
make assumptions based on current implementation, we are tightly coupling the 
RM/NM logic with logic here and it is not enforced by protocol either. It is 
likely to work just fine as developers would take care but I would suggest that 
ideally if we assume that collector address is carried always, we enforce it in 
proto definition of CollectorInfo i.e. make collector address as "required" 
instead of "optional" in it. And we will have to see if address should be 
"required" in AppCollectorData too.
Please note that RM may not have access to collector address initially when AM 
container is launched. So this change would also mean change in RM to not send 
collector info at all if address is null.

The last comment i.e. suggested refactoring depends on comments above i.e. 
whether to update the token or not if address is not carried in collector info. 
Thoughts?


was (Author: varun_saxena):
Thanks [~rohithsharma] for the review.

bq. Need to log a WARN message if collector info is null.
Ok.

bq. currentTimelineToken should be volatile
Need not be. Atleast in MR AM. This is used only while AM is updating the token 
and that happens only from RMContainer Allocator thread so only one thread sees 
and updates it. While using token is picked from UGI. Will it be likely that 
token will be updated from 2 separate threads? We anyways do not claim any 
thread safety for timeline client. 
Address is volatile and its different because the thread publishing the entity 
and using the address would be different from the allocator thread which would 
communicate with RM and update the address. However, making it volatile doesn't 
cost us anything. As you say. I do not have a strong opinion on this. Thoughts?

bq. Creating Token does not required to check service==null. Internally 
constructor does. And we can ignore token service passed by delegationToken 
always and set it up collector address.
You mean the constructor inside setTimelineDelegationToken method i.e. at L203? 
Actually the constructor takes service as Text and not String. The check I am 
making is for service as String. If I do not make the check and call new 
Text(service), a null service would throw NPE.

bq. 
!delegationToken.getKind().equals(TimelineDelegationTokenIdentifier.KIND_NAME.toString())
 check is not required since 

[jira] [Comment Edited] (MAPREDUCE-6838) [ATSv2 Security] Add timeline delegation token received in allocate response to UGI

2017-08-19 Thread Varun Saxena (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134021#comment-16134021
 ] 

Varun Saxena edited comment on MAPREDUCE-6838 at 8/19/17 10:20 AM:
---

Maybe what we can do is that provide another API in TimelineV2Client, say, 
setTimelineToken and that does the job of sanitizing the service and setting 
the token service. That is, move the code from RMContainerAllocator to 
TimelineV2ClientImpl and set the token in UGI there.
Thoughts?


was (Author: varun_saxena):
Maybe what we can do is that provide another API in TimelineV2Client, say, 
setTimelineToken and that does the job of sanitizing the service and setting 
the token. That is, move the code from RMContainerAllocator to 
TimelineV2ClientImpl and set the token in UGI there.
Thoughts?

> [ATSv2 Security] Add timeline delegation token received in allocate response 
> to UGI
> ---
>
> Key: MAPREDUCE-6838
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6838
> Project: Hadoop Map/Reduce
>  Issue Type: Sub-task
>Reporter: Varun Saxena
>Assignee: Varun Saxena
>  Labels: yarn-5355-merge-blocker
> Fix For: YARN-5355
>
> Attachments: MAPREDUCE-6838-YARN-5355.01.patch, 
> MAPREDUCE-6838-YARN-5355.02.patch, MAPREDUCE-6838-YARN-5355.03.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Comment Edited] (MAPREDUCE-6838) [ATSv2 Security] Add timeline delegation token received in allocate response to UGI

2017-08-19 Thread Varun Saxena (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134021#comment-16134021
 ] 

Varun Saxena edited comment on MAPREDUCE-6838 at 8/19/17 8:34 AM:
--

Maybe what we can do is that provide another API in TimelineV2Client, say, 
setTimelineToken and that does the job of sanitizing the service and setting 
the token. That is, move the code from RMContainerAllocator to 
TimelineV2ClientImpl and set the token in UGI there.
Thoughts?


was (Author: varun_saxena):
Maybe what we can do is that provide another API in TimelineV2Client, say, 
setTimelineToken and that does the job of sanitizing the service and setting 
the token. That is, move the code from RMContainerAllocator to 
TimelineV2ClientImpl and set the token in UGI there.
Thoughts?

> [ATSv2 Security] Add timeline delegation token received in allocate response 
> to UGI
> ---
>
> Key: MAPREDUCE-6838
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6838
> Project: Hadoop Map/Reduce
>  Issue Type: Sub-task
>Reporter: Varun Saxena
>Assignee: Varun Saxena
>  Labels: yarn-5355-merge-blocker
> Fix For: YARN-5355
>
> Attachments: MAPREDUCE-6838-YARN-5355.01.patch, 
> MAPREDUCE-6838-YARN-5355.02.patch, MAPREDUCE-6838-YARN-5355.03.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Comment Edited] (MAPREDUCE-6838) [ATSv2 Security] Add timeline delegation token received in allocate response to UGI

2017-08-19 Thread Jian He (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134009#comment-16134009
 ] 

Jian He edited comment on MAPREDUCE-6838 at 8/19/17 8:08 AM:
-

today, for other delegation tokens RMDelegationToken, the old ATSv1 
DelegationToken, the token service is not set at server side, it is set at 
client side - the client call the SecurityUtils#buildTokenService and then set 
the token service. I don't know why it was done like that - maybe because it 
avoids the use_ip config inconsistency between client and serve ?

Should we follow the same ? The client can construct the tokenService based on 
the collector address info ? (One caveat is to make sure the old token gets 
replaced properly - in case ip changes on restart?)
The CollectorInfo#getCollectorAddr right now is a string, should it be an 
address type ?





was (Author: jianhe):
today, for other delegation tokens RMDelegationToken, the old ATSv1 
DelegationToken, the token service is not set at server side, it is set at 
client side - the client call the SecurityUtils#buildTokenService and then set 
the token service. I don't know why it was done like that - maybe because it 
avoids the use_ip config inconsistency between client and serve ?

Should we follow the same ? The client can construct the tokenService based on 
the collector address info ? (One caveat is to make sure the old token gets 
replaced properly - in case ip changes ?)
The CollectorInfo#getCollectorAddr right now is a string, should it be an 
address type ?




> [ATSv2 Security] Add timeline delegation token received in allocate response 
> to UGI
> ---
>
> Key: MAPREDUCE-6838
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6838
> Project: Hadoop Map/Reduce
>  Issue Type: Sub-task
>Reporter: Varun Saxena
>Assignee: Varun Saxena
>  Labels: yarn-5355-merge-blocker
> Fix For: YARN-5355
>
> Attachments: MAPREDUCE-6838-YARN-5355.01.patch, 
> MAPREDUCE-6838-YARN-5355.02.patch, MAPREDUCE-6838-YARN-5355.03.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Comment Edited] (MAPREDUCE-6838) [ATSv2 Security] Add timeline delegation token received in allocate response to UGI

2017-08-19 Thread Jian He (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134009#comment-16134009
 ] 

Jian He edited comment on MAPREDUCE-6838 at 8/19/17 7:57 AM:
-

today, for other delegation tokens RMDelegationToken, the old ATSv1 
DelegationToken, the token service is not set at server side, it is set at 
client side - the client call the SecurityUtils#buildTokenService and then set 
the token service. I don't know why it was done like that - maybe because it 
avoids the use_ip config inconsistency between client and serve ?

Should we follow the same ? The client can construct the tokenService based on 
the collector address info ? (One caveat is to make sure the old token gets 
replaced properly - in case ip changes ?)
The CollectorInfo#getCollectorAddr right now is a string, should it be an 
address type ?





was (Author: jianhe):
today, for other delegation tokens RMDelegationToken, the old ATSv1 
DelegationToken, the token service is not set at server side, it is set at 
client side - the client call the SecurityUtils#buildTokenService and then set 
the token service. I don't know what it was done like that - maybe because it 
avoids the use_ip config inconsistency between client and serve ?

Should we follow the same ? The client can construct the tokenService based on 
the collector address info ? (One caveat is to make sure the old token gets 
replaced properly - in case ip changes ?)
The CollectorInfo#getCollectorAddr right now is a string, should it be an 
address type ?




> [ATSv2 Security] Add timeline delegation token received in allocate response 
> to UGI
> ---
>
> Key: MAPREDUCE-6838
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6838
> Project: Hadoop Map/Reduce
>  Issue Type: Sub-task
>Reporter: Varun Saxena
>Assignee: Varun Saxena
>  Labels: yarn-5355-merge-blocker
> Fix For: YARN-5355
>
> Attachments: MAPREDUCE-6838-YARN-5355.01.patch, 
> MAPREDUCE-6838-YARN-5355.02.patch, MAPREDUCE-6838-YARN-5355.03.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Comment Edited] (MAPREDUCE-6838) [ATSv2 Security] Add timeline delegation token received in allocate response to UGI

2017-08-19 Thread Jian He (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134009#comment-16134009
 ] 

Jian He edited comment on MAPREDUCE-6838 at 8/19/17 7:56 AM:
-

today, for other delegation tokens RMDelegationToken, the old ATSv1 
DelegationToken, the token service is not set at server side, it is set at 
client side - the client call the SecurityUtils#buildTokenService and then set 
the token service. I don't know what it was done like that - maybe because it 
avoids the use_ip config inconsistency between client and serve ?

Should we follow the same ? The client can construct the tokenService based on 
the collector address info ? (One caveat is to make sure the old token gets 
replaced properly - in case ip changes ?)
The CollectorInfo#getCollectorAddr right now is a string, should it be an 
address type ?





was (Author: jianhe):
today, for other delegation tokens RMDelegationToken, the old ATSv1 
DelegationToken, the token service is not set at server side, it is set at 
client side - the client call the SecurityUtils#buildTokenService and then set 
the token service. I don't know what it was done like that - maybe because it 
avoids the use_ip config inconsistency between client and serve ?

Should we follow the same ? The client can construct the tokenService based on 
the collector address info ? (One caveat is to make sure the old token gets 
probably replaced properly - in case ip changes ?)
The CollectorInfo#getCollectorAddr right now is a string, should it be an 
address type ?




> [ATSv2 Security] Add timeline delegation token received in allocate response 
> to UGI
> ---
>
> Key: MAPREDUCE-6838
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6838
> Project: Hadoop Map/Reduce
>  Issue Type: Sub-task
>Reporter: Varun Saxena
>Assignee: Varun Saxena
>  Labels: yarn-5355-merge-blocker
> Fix For: YARN-5355
>
> Attachments: MAPREDUCE-6838-YARN-5355.01.patch, 
> MAPREDUCE-6838-YARN-5355.02.patch, MAPREDUCE-6838-YARN-5355.03.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Comment Edited] (MAPREDUCE-6838) [ATSv2 Security] Add timeline delegation token received in allocate response to UGI

2017-08-16 Thread Varun Saxena (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128398#comment-16128398
 ] 

Varun Saxena edited comment on MAPREDUCE-6838 at 8/16/17 4:07 PM:
--

TestSecureMRTimelineEventHandling failure is strange. It passes for me 
everytime.
Here, AM container is exiting with exit code 1. Ideallly LCE should be used but 
will be difficult to ensure a E2E test with it I guess.
[~rohithsharma], can you reproduce the failure?


was (Author: varun_saxena):
TestSecureMRTimelineEventHandling failure is strange. It passes for me 
everytime.
Here, AM container is exiting with exit code 1. Ideallly LCE should be used but 
will be difficult to ensure a E2E test with it I guess.
[~rohithsharma], can you simulate the failure?

> [ATSv2 Security] Add timeline delegation token received in allocate response 
> to UGI
> ---
>
> Key: MAPREDUCE-6838
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6838
> Project: Hadoop Map/Reduce
>  Issue Type: Sub-task
>Reporter: Varun Saxena
>Assignee: Varun Saxena
>  Labels: yarn-5355-merge-blocker
> Fix For: YARN-5355
>
> Attachments: MAPREDUCE-6838-YARN-5355.01.patch, 
> MAPREDUCE-6838-YARN-5355.02.patch, MAPREDUCE-6838-YARN-5355.03.patch, 
> MAPREDUCE-6838-YARN-5355.03.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org