Jason Lowe updated YARN-1915:

    Attachment: YARN-1915.patch

We're starting to see this as well in our rollout of 2.x.  Attaching a patch 
that works around the issue by having the AM secret manager wait around for a 
bit before trying to validate a token if the master key isn't set yet.

Another approach we could try is to have the RM not advertise to clients where 
the AM is (i.e.: hide the host, port, and tracking URL) until the RM has seen 
at least one heartbeat after the AM registered.  The approach in this patch was 
easy to implement and probably just as effective in practice.

> ClientToAMTokenMasterKey should be provided to AM at launch time
> ----------------------------------------------------------------
>                 Key: YARN-1915
>                 URL: https://issues.apache.org/jira/browse/YARN-1915
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>    Affects Versions: 2.2.0
>            Reporter: Hitesh Shah
>            Priority: Critical
>         Attachments: YARN-1915.patch
> Currently, the AM receives the key as part of registration. This introduces a 
> race where a client can connect to the AM when the AM has not received the 
> key. 
> Current Flow:
> 1) AM needs to start the client listening service in order to get host:port 
> and send it to the RM as part of registration
> 2) RM gets the port info in register() and transitions the app to RUNNING. 
> Responds back with client secret to AM.
> 3) User asks RM for client token. Gets it and pings the AM. AM hasn't 
> received client secret from RM and so RPC itself rejects the request.

This message was sent by Atlassian JIRA

Reply via email to