[jira] [Commented] (YARN-874) Tracking YARN/MR test failures after HADOOP-9421 and YARN-827

2013-06-25 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13693186#comment-13693186 ] Daryn Sharp commented on YARN-874: -- +1! Tracking YARN/MR test failures

[jira] [Updated] (YARN-960) TestMRCredentials and TestBinaryTokenFile are failing on trunk

2013-07-24 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-960: - Attachment: YARN-960.patch All tokens but the AMRM token are being lost are lost if security is disabled.

[jira] [Commented] (YARN-945) AM register failing after AMRMToken

2013-07-24 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718975#comment-13718975 ] Daryn Sharp commented on YARN-945: -- Please be sure I get a chance to look at the patch.

[jira] [Commented] (YARN-707) Add user info in the YARN ClientToken

2013-08-23 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13748999#comment-13748999 ] Daryn Sharp commented on YARN-707: -- It almost seems like it would be better to invert the

[jira] [Commented] (YARN-707) Add user info in the YARN ClientToken

2013-08-28 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13752443#comment-13752443 ] Daryn Sharp commented on YARN-707: -- Technically you should be bumping the token ident's

[jira] [Commented] (YARN-707) Add user info in the YARN ClientToken

2013-09-04 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13757904#comment-13757904 ] Daryn Sharp commented on YARN-707: -- Ug, the RM and AM are abusing the same secret manager

[jira] [Commented] (YARN-707) Add user info in the YARN ClientToken

2013-09-04 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13757890#comment-13757890 ] Daryn Sharp commented on YARN-707: -- Still reviewing, but an initial observation is

[jira] [Commented] (YARN-707) Add user info in the YARN ClientToken

2013-09-04 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13757926#comment-13757926 ] Daryn Sharp commented on YARN-707: -- Minor: # {{ClientToAMTokenIdentifier#getUser()}}

[jira] [Commented] (YARN-1146) RM DTSM and RMStateStore mismanage sequence number

2013-09-04 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13757983#comment-13757983 ] Daryn Sharp commented on YARN-1146: --- Note that bug #2 will not self-correct if the

[jira] [Created] (YARN-1146) RM DTSM and RMStateStore mismanage sequence number

2013-09-04 Thread Daryn Sharp (JIRA)
Daryn Sharp created YARN-1146: - Summary: RM DTSM and RMStateStore mismanage sequence number Key: YARN-1146 URL: https://issues.apache.org/jira/browse/YARN-1146 Project: Hadoop YARN Issue Type:

[jira] [Commented] (YARN-1146) RM DTSM and RMStateStore mismanage sequence number

2013-09-04 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13758184#comment-13758184 ] Daryn Sharp commented on YARN-1146: --- [~vinodkv] I'm desynch'ing the ADTSM on HADOOP-9930.

[jira] [Commented] (YARN-707) Add user info in the YARN ClientToken

2013-09-04 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13758375#comment-13758375 ] Daryn Sharp commented on YARN-707: -- +1 Looks good enough to me. Add user

[jira] [Commented] (YARN-1189) NMTokenSecretManagerInNM is not being told when applications have finished

2013-09-13 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766738#comment-13766738 ] Daryn Sharp commented on YARN-1189: --- Oops, I thought the .1 patch was the latest so I

[jira] [Commented] (YARN-1189) NMTokenSecretManagerInNM is not being told when applications have finished

2013-09-13 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766728#comment-13766728 ] Daryn Sharp commented on YARN-1189: --- +1 but a test, even a mock that spies appFinished

[jira] [Commented] (YARN-986) YARN should have a ClusterId/ServiceId

2013-09-25 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13777523#comment-13777523 ] Daryn Sharp commented on YARN-986: -- This sounds like NN HA tokens which IMHO are rather

[jira] [Commented] (YARN-1915) ClientToAMTokenMasterKey should be provided to AM at launch time

2014-08-13 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095471#comment-14095471 ] Daryn Sharp commented on YARN-1915: --- +1 But since I had a hand in the design, we should

[jira] [Commented] (YARN-1915) ClientToAMTokenMasterKey should be provided to AM at launch time

2014-08-13 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095820#comment-14095820 ] Daryn Sharp commented on YARN-1915: --- I suspect it's because it removes the burden from

[jira] [Commented] (YARN-1915) ClientToAMTokenMasterKey should be provided to AM at launch time

2014-08-13 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096319#comment-14096319 ] Daryn Sharp commented on YARN-1915: --- Yes, I thought the ugi mangling was gone, but the

[jira] [Commented] (YARN-60) NMs rejects all container tokens after secret key rolls

2012-08-31 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-60?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13446319#comment-13446319 ] Daryn Sharp commented on YARN-60: - +1 23 patch changes look trivial. Thanks!

[jira] [Commented] (YARN-241) Node Manager fails to launch containers after NM restart in secure mode

2012-11-26 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13503836#comment-13503836 ] Daryn Sharp commented on YARN-241: -- Is this maybe caused by a race condition where the NM

[jira] [Commented] (YARN-50) Implement renewal / cancellation of Delegation Tokens

2012-12-20 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13537075#comment-13537075 ] Daryn Sharp commented on YARN-50: - A few minor questions/comments: # Should the ctors for

[jira] [Created] (YARN-279) Generalize RM token management

2012-12-20 Thread Daryn Sharp (JIRA)
Daryn Sharp created YARN-279: Summary: Generalize RM token management Key: YARN-279 URL: https://issues.apache.org/jira/browse/YARN-279 Project: Hadoop YARN Issue Type: Improvement

[jira] [Commented] (YARN-279) Generalize RM token management

2012-12-20 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13537148#comment-13537148 ] Daryn Sharp commented on YARN-279: -- The current token management (renew/cancel) has

[jira] [Created] (YARN-280) RM does not reject app submission with invalid tokens

2012-12-20 Thread Daryn Sharp (JIRA)
Daryn Sharp created YARN-280: Summary: RM does not reject app submission with invalid tokens Key: YARN-280 URL: https://issues.apache.org/jira/browse/YARN-280 Project: Hadoop YARN Issue Type:

[jira] [Updated] (YARN-280) RM does not reject app submission with invalid tokens

2012-12-20 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-280: - Attachment: YARN-280.patch Instead of scheduling tokens for immediate renewal in the background, the first

[jira] [Commented] (YARN-50) Implement renewal / cancellation of Delegation Tokens

2012-12-20 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13537329#comment-13537329 ] Daryn Sharp commented on YARN-50: - I think it looks like my concerns are addressed. Only

[jira] [Commented] (YARN-50) Implement renewal / cancellation of Delegation Tokens

2012-12-20 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13537335#comment-13537335 ] Daryn Sharp commented on YARN-50: - Actually, it looks like

[jira] [Commented] (YARN-279) Generalize RM token management

2012-12-21 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13538123#comment-13538123 ] Daryn Sharp commented on YARN-279: -- The problem with the passport idea is how would you

[jira] [Commented] (YARN-279) Generalize RM token management

2012-12-21 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13538199#comment-13538199 ] Daryn Sharp commented on YARN-279: -- I'd have concerns that a 6 month expiration provides a

[jira] [Updated] (YARN-320) RM should always be able to renew its own tokens

2013-01-07 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-320: - Attachment: YARN-320.branch-23.patch Proposed minimal change that fakes out the renewer. If the RM is

[jira] [Commented] (YARN-320) RM should always be able to renew its own tokens

2013-01-08 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13546988#comment-13546988 ] Daryn Sharp commented on YARN-320: -- Admittedly, extracting the renewer is a bit dubious.

[jira] [Updated] (YARN-320) RM should always be able to renew its own tokens

2013-01-08 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-320: - Attachment: YARN-320.branch-23.patch Add unit tests, trunk patch forthcoming. RM should

[jira] [Updated] (YARN-320) RM should always be able to renew its own tokens

2013-01-08 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-320: - Attachment: YARN-320.patch Patch for trunk and branch-2. RM should always be able to

[jira] [Created] (YARN-338) RM renews tokens even when maxDate will soon be exceeded

2013-01-14 Thread Daryn Sharp (JIRA)
Daryn Sharp created YARN-338: Summary: RM renews tokens even when maxDate will soon be exceeded Key: YARN-338 URL: https://issues.apache.org/jira/browse/YARN-338 Project: Hadoop YARN Issue Type:

[jira] [Commented] (YARN-135) ClientTokens should be per app-attempt and be unregistered on App-finish.

2013-01-15 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13554123#comment-13554123 ] Daryn Sharp commented on YARN-135: -- It may be a good idea to not reuse the {{Token}}

[jira] [Created] (YARN-355) RM app submission jams under load

2013-01-23 Thread Daryn Sharp (JIRA)
Daryn Sharp created YARN-355: Summary: RM app submission jams under load Key: YARN-355 URL: https://issues.apache.org/jira/browse/YARN-355 Project: Hadoop YARN Issue Type: Bug

[jira] [Commented] (YARN-355) RM app submission jams under load

2013-01-23 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13560808#comment-13560808 ] Daryn Sharp commented on YARN-355: -- The RM needs to short-circuit the RPC loopback

[jira] [Updated] (YARN-355) RM app submission jams under load

2013-01-23 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-355: - Attachment: YARN-320.branch-23.patch Proposed patch that works on 23. The RM token renewer will short out

[jira] [Updated] (YARN-355) RM app submission jams under load

2013-01-23 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-355: - Attachment: (was: YARN-320.branch-23.patch) RM app submission jams under load

[jira] [Updated] (YARN-355) RM app submission jams under load

2013-01-23 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-355: - Attachment: YARN-355.branch-23.patch Renaming patch with correct jira id! RM app

[jira] [Updated] (YARN-357) App submission should not be synchronized

2013-01-24 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-357: - Attachment: YARN-357.patch Existing tests for MAPREDUCE-2953 demonstrate the race is fixed w/o

[jira] [Created] (YARN-360) Allow apps to concurrently register tokens for renewal

2013-01-25 Thread Daryn Sharp (JIRA)
Daryn Sharp created YARN-360: Summary: Allow apps to concurrently register tokens for renewal Key: YARN-360 URL: https://issues.apache.org/jira/browse/YARN-360 Project: Hadoop YARN Issue Type:

[jira] [Updated] (YARN-360) Allow apps to concurrently register tokens for renewal

2013-01-25 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-360: - Attachment: YARN-360.patch Allow apps to concurrently register tokens for renewal

[jira] [Commented] (YARN-360) Allow apps to concurrently register tokens for renewal

2013-01-28 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13564354#comment-13564354 ] Daryn Sharp commented on YARN-360: -- Findbugs is false positive. {{Timer}} class is

[jira] [Updated] (YARN-357) App submission should not be synchronized

2013-01-28 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-357: - Attachment: YARN-357.patch Add tests that both better verify MAPREDUCE-2953 by ensuring

[jira] [Commented] (YARN-357) App submission should not be synchronized

2013-01-29 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13565499#comment-13565499 ] Daryn Sharp commented on YARN-357: -- Yes, the test failed on a non-patched trunk build.

[jira] [Updated] (YARN-355) RM app submission jams under load

2013-01-30 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-355: - Attachment: YARN-355.patch Things got shuffled on main and caused dependency woes. Removed private

[jira] [Commented] (YARN-357) App submission should not be synchronized

2013-02-05 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13571361#comment-13571361 ] Daryn Sharp commented on YARN-357: -- Thanks for the review! I struggled quite a bit with

[jira] [Commented] (YARN-355) RM app submission jams under load

2013-02-05 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13571362#comment-13571362 ] Daryn Sharp commented on YARN-355: -- I did somewhat intentionally neglect checking the port

[jira] [Updated] (YARN-355) RM app submission jams under load

2013-02-05 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-355: - Attachment: YARN-355.patch Check token service addr against RM addr and correctly handles RM addr being a

[jira] [Updated] (YARN-355) RM app submission jams under load

2013-02-06 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-355: - Attachment: YARN-355.patch YARN-355.branch-23.patch Thanks Sid. Updated patches to set

[jira] [Created] (YARN-494) RM should be able to hard stop a lingering app on a NM

2013-03-21 Thread Daryn Sharp (JIRA)
Daryn Sharp created YARN-494: Summary: RM should be able to hard stop a lingering app on a NM Key: YARN-494 URL: https://issues.apache.org/jira/browse/YARN-494 Project: Hadoop YARN Issue Type:

[jira] [Commented] (YARN-494) RM should be able to hard stop a lingering app on a NM

2013-03-22 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13611077#comment-13611077 ] Daryn Sharp commented on YARN-494: -- Yes, log aggregation is a yarn service provided by the

[jira] [Moved] (YARN-503) DelegationTokens will be renewed forever if multiple jobs share tokens and the first one sets JOB_CANCEL_DELEGATION_TOKEN to false

2013-03-22 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp moved MAPREDUCE-3979 to YARN-503: - Component/s: (was: resourcemanager)

[jira] [Updated] (YARN-503) DelegationTokens will be renewed forever if multiple jobs share tokens and the first one sets JOB_CANCEL_DELEGATION_TOKEN to false

2013-03-22 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-503: - Attachment: YARN-503.patch Overhaul and simplify the RM's token renewer. Apps now track their tokens, and

[jira] [Commented] (YARN-503) DelegationTokens will be renewed forever if multiple jobs share tokens and the first one sets JOB_CANCEL_DELEGATION_TOKEN to false

2013-03-26 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13614502#comment-13614502 ] Daryn Sharp commented on YARN-503: -- bq. Will the case of MR actions be an issue? that the

[jira] [Updated] (YARN-503) DelegationTokens will be renewed forever if multiple jobs share tokens and the first one sets JOB_CANCEL_DELEGATION_TOKEN to false

2013-04-09 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-503: - Attachment: YARN-503.patch Fixed findbugs issues. Upon review of those issues, I realized I was trying

[jira] [Commented] (YARN-503) DelegationTokens will be renewed forever if multiple jobs share tokens and the first one sets JOB_CANCEL_DELEGATION_TOKEN to false

2013-04-12 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13630619#comment-13630619 ] Daryn Sharp commented on YARN-503: -- bq. There's likely a race between the RenewalTask and

[jira] [Reopened] (YARN-579) Make ApplicationToken part of Container's token list to help RM-restart

2013-04-29 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp reopened YARN-579: -- This has broken secure clusters. The AM is unable to find the token to register with the RM. I've

[jira] [Commented] (YARN-575) ContainerManager APIs should be user accessible

2013-04-29 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13644793#comment-13644793 ] Daryn Sharp commented on YARN-575: -- I agree with your 2nd point, I think allowing users to

[jira] [Commented] (YARN-617) In unsercure mode, AM can fake resource requirements

2013-04-29 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13644795#comment-13644795 ] Daryn Sharp commented on YARN-617: -- Does there really need to be different NM behavior?

[jira] [Commented] (YARN-582) Restore appToken and clientToken for app attempt after RM restart

2013-04-29 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13644805#comment-13644805 ] Daryn Sharp commented on YARN-582: -- I've only glanced over the patch, but do these tokens

[jira] [Commented] (YARN-613) Create NM proxy per NM instead of per container

2013-04-29 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13644809#comment-13644809 ] Daryn Sharp commented on YARN-613: -- Question: How do you plan for NMs to authenticate the

[jira] [Commented] (YARN-617) In unsercure mode, AM can fake resource requirements

2013-04-30 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13645817#comment-13645817 ] Daryn Sharp commented on YARN-617: -- If a token is available, the RPC client will attempt

[jira] [Commented] (YARN-613) Create NM proxy per NM instead of per container

2013-04-30 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13645852#comment-13645852 ] Daryn Sharp commented on YARN-613: -- I assumed that was the implementation. Does a global

[jira] [Commented] (YARN-613) Create NM proxy per NM instead of per container

2013-05-01 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13646558#comment-13646558 ] Daryn Sharp commented on YARN-613: -- I just have general concerns with assuming the entire

[jira] [Commented] (YARN-617) In unsercure mode, AM can fake resource requirements

2013-05-01 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13646572#comment-13646572 ] Daryn Sharp commented on YARN-617: -- bq. we are trying to change the auth to use AMTokens

[jira] [Commented] (YARN-338) RM renews tokens even when maxDate will soon be exceeded

2013-05-07 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13651085#comment-13651085 ] Daryn Sharp commented on YARN-338: -- I think I may have filed a jira about this a long time

[jira] [Commented] (YARN-617) In unsercure mode, AM can fake resource requirements

2013-05-10 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13654919#comment-13654919 ] Daryn Sharp commented on YARN-617: -- In addition to Vinod: * {{NodeManager}}: Minor, does

[jira] [Commented] (YARN-613) Create NM proxy per NM instead of per container

2013-05-10 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13655014#comment-13655014 ] Daryn Sharp commented on YARN-613: -- bq. That is true in general. And I am not sure how we

[jira] [Commented] (YARN-613) Create NM proxy per NM instead of per container

2013-05-13 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13656029#comment-13656029 ] Daryn Sharp commented on YARN-613: -- Agreed that a rogue NM is impossible to prevent

[jira] [Created] (YARN-690) RM exits on token cancel/renew problems

2013-05-16 Thread Daryn Sharp (JIRA)
Daryn Sharp created YARN-690: Summary: RM exits on token cancel/renew problems Key: YARN-690 URL: https://issues.apache.org/jira/browse/YARN-690 Project: Hadoop YARN Issue Type: Bug

[jira] [Updated] (YARN-690) RM exits on token cancel/renew problems

2013-05-16 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-690: - Attachment: YARN-690.patch 1-line change to a catch. No test added due to difficulty of testing calls to

[jira] [Updated] (YARN-690) RM exits on token cancel/renew problems

2013-05-16 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-690: - Attachment: YARN-690.patch Doh, you're right. It was a test, and you passed! RM exits

[jira] [Commented] (YARN-1841) YARN ignores/overrides explicit security settings

2014-03-17 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13937933#comment-13937933 ] Daryn Sharp commented on YARN-1841: --- The reason the custom AM in the related user@hadoop

[jira] [Resolved] (YARN-1841) YARN ignores/overrides explicit security settings

2014-03-17 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp resolved YARN-1841. --- Resolution: Not A Problem Oleg, the authentication config setting specifies the _external

[jira] [Commented] (YARN-1841) YARN ignores/overrides explicit security settings

2014-03-18 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13939272#comment-13939272 ] Daryn Sharp commented on YARN-1841: --- bq. The fact that it behaves differently once

[jira] [Commented] (YARN-1841) YARN ignores/overrides explicit security settings

2014-03-18 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13939353#comment-13939353 ] Daryn Sharp commented on YARN-1841: --- I thought you were having issues talking to other

[jira] [Commented] (YARN-2156) ApplicationMasterService#serviceStart() method has hardcoded AuthMethod.TOKEN as security configuration

2014-06-12 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14029913#comment-14029913 ] Daryn Sharp commented on YARN-2156: --- Yes, this is by design. Yarn uses tokens regardless

[jira] [Resolved] (YARN-2156) ApplicationMasterService#serviceStart() method has hardcoded AuthMethod.TOKEN as security configuration

2014-06-12 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp resolved YARN-2156. --- Resolution: Not a Problem ApplicationMasterService#serviceStart() method has hardcoded

[jira] [Commented] (YARN-2156) ApplicationMasterService#serviceStart() method has hardcoded AuthMethod.TOKEN as security configuration

2014-06-13 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030639#comment-14030639 ] Daryn Sharp commented on YARN-2156: --- A warning doesn't make sense because it implies

[jira] [Commented] (YARN-2147) client lacks delegation token exception details when application submit fails

2014-06-17 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14034612#comment-14034612 ] Daryn Sharp commented on YARN-2147: --- I don't think the patch handles the use case it's

[jira] [Commented] (YARN-2147) client lacks delegation token exception details when application submit fails

2014-06-30 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14047946#comment-14047946 ] Daryn Sharp commented on YARN-2147: --- Code looks fine. Currently the test verifies the

[jira] [Assigned] (YARN-691) Invalid NaN values in Hadoop REST API JSON response

2013-11-07 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp reassigned YARN-691: Assignee: Daryn Sharp Invalid NaN values in Hadoop REST API JSON response

[jira] [Updated] (YARN-691) Invalid NaN values in Hadoop REST API JSON response

2013-11-07 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-691: - Assignee: (was: Daryn Sharp) Invalid NaN values in Hadoop REST API JSON response

[jira] [Commented] (YARN-1628) TestContainerManagerSecurity fails on trunk

2014-01-29 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13885677#comment-13885677 ] Daryn Sharp commented on YARN-1628: --- +1. Will check in later today. Thanks!

[jira] [Commented] (YARN-2964) RM prematurely cancels tokens for jobs that submit jobs (oozie)

2014-12-15 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14247400#comment-14247400 ] Daryn Sharp commented on YARN-2964: --- [~vinodkv], can you take a look at this? RM

[jira] [Commented] (YARN-2971) RM uses conf instead of token service address to renew timeline delegation tokens

2015-01-27 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14293559#comment-14293559 ] Daryn Sharp commented on YARN-2971: --- I think it's because the job cannot assume that the

[jira] [Commented] (YARN-2971) RM uses conf instead of token service address to renew timeline delegation tokens

2015-02-09 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14313090#comment-14313090 ] Daryn Sharp commented on YARN-2971: --- +1 Looks good RM uses conf instead of token

[jira] [Commented] (YARN-3055) The token is not renewed properly if it's shared by jobs (oozie) in DelegationTokenRenewer

2015-04-06 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481225#comment-14481225 ] Daryn Sharp commented on YARN-3055: --- Correctly handling the don't cancel setting for jobs

[jira] [Commented] (YARN-3055) The token is not renewed properly if it's shared by jobs (oozie) in DelegationTokenRenewer

2015-04-07 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14484117#comment-14484117 ] Daryn Sharp commented on YARN-3055: --- On cursory glance, are you sure this isn't going to

[jira] [Commented] (YARN-3055) The token is not renewed properly if it's shared by jobs (oozie) in DelegationTokenRenewer

2015-04-07 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14484201#comment-14484201 ] Daryn Sharp commented on YARN-3055: --- This appears to go back to the really old days of

[jira] [Updated] (YARN-3055) The token is not renewed properly if it's shared by jobs (oozie) in DelegationTokenRenewer

2015-04-08 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-3055: -- Attachment: YARN-3055.patch Haven't had a chance to run findbugs. Might grumble about sync

[jira] [Commented] (YARN-3055) The token is not renewed properly if it's shared by jobs (oozie) in DelegationTokenRenewer

2015-04-08 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485461#comment-14485461 ] Daryn Sharp commented on YARN-3055: --- bq. It does seem odd to get the expiration date by

[jira] [Commented] (YARN-3055) The token is not renewed properly if it's shared by jobs (oozie) in DelegationTokenRenewer

2015-04-08 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485890#comment-14485890 ] Daryn Sharp commented on YARN-3055: --- The renew at job submission isn't the problem. It's

[jira] [Commented] (YARN-3055) The token is not renewed properly if it's shared by jobs (oozie) in DelegationTokenRenewer

2015-04-08 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14486104#comment-14486104 ] Daryn Sharp commented on YARN-3055: --- I believe you are describing the behavior of 2.6's

[jira] [Commented] (YARN-3055) The token is not renewed properly if it's shared by jobs (oozie) in DelegationTokenRenewer

2015-04-09 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487620#comment-14487620 ] Daryn Sharp commented on YARN-3055: --- Two apps could double renew tokens (completely

[jira] [Updated] (YARN-3055) The token is not renewed properly if it's shared by jobs (oozie) in DelegationTokenRenewer

2015-04-09 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-3055: -- Attachment: YARN-3055.patch The token is not renewed properly if it's shared by jobs (oozie) in

[jira] [Commented] (YARN-3055) The token is not renewed properly if it's shared by jobs (oozie) in DelegationTokenRenewer

2015-04-09 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487482#comment-14487482 ] Daryn Sharp commented on YARN-3055: --- Thanks Vinod, I'll revise this morning. The ignores

[jira] [Commented] (YARN-3760) Log aggregation failures

2015-06-02 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14569289#comment-14569289 ] Daryn Sharp commented on YARN-3760: --- Cancelled tokens trigger the retry proxy bug. Log

  1   2   >