[ 
https://issues.apache.org/jira/browse/YARN-9920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16970758#comment-16970758
 ] 

Prabhu Joseph commented on YARN-9920:
-------------------------------------

[~wilfreds] Thanks for reviewing the Jira.

1. {{AccessRequest#getRemoteAddress()}} is used by Ranger Authorization for 
Audit Purpose. The *Client IP* field is null as per the ranger audit screenshot

!AccessAudist_yarn_clientIPempty.png|height=200!

 

2. {{Server.getRemoteAddress()}} will return the right Client IP address from 
thread local {{CurCall}} to any method which is executed as part of the IPC 
Server Thread, else will return Null. We have used 
{{Server.getRemoteAddress()}} at below place which is not part of IPC Server 
thread and hence returned Null.

*EventDispatcher Thread -> FairScheduler#addApplication -> FSQueue.hasAccess -> 
Server.getRemoteAddress returns null*

To fix this, have stored the Client IP Address inside {{RMAppImpl}} while 
{{createAndPopulateNewRMApp}} which is called as part of IPC Server thread. 
This will be used later by FairScheduler when checking queue access.

*IPC Server -> RMAppManager#createAndPopulateNewRMApp -> AppAddedSchedulerEvent*
{code:java}
FairScheduler.java:

+      RMApp rmApp = rmContext.getRMApps().get(applicationId);
+      String remoteAddress = (rmApp != null) ?
+          rmApp.getRemoteAddress() : Server.getRemoteAddress();
+
+      if (!queue.hasAccess(QueueACL.SUBMIT_APPLICATIONS, userUgi,
+          remoteAddress, null) &&
{code}
 

3. In {{RMWebServices}}, there are certain places it directly uses 
{{checkAccess(}}) where {{HttpServletRequest#getRemoteAddr()}} is passed. But 
when using ClientRMService for submit app, move app, need to figure out a way 
to get the ClientIPAddress.

> YarnAuthorizationProvider AccessRequest gets Null RemoteAddress from 
> FairScheduler
> ----------------------------------------------------------------------------------
>
>                 Key: YARN-9920
>                 URL: https://issues.apache.org/jira/browse/YARN-9920
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler, security
>    Affects Versions: 3.3.0
>            Reporter: Prabhu Joseph
>            Assignee: Prabhu Joseph
>            Priority: Major
>         Attachments: AccessAudist_yarn_clientIPempty.png, 
> YARN-9920-001.patch, YARN-9920-002.patch, YARN-9920-003.patch
>
>
> YarnAuthorizationProvider AccessRequest has null RemoteAddress in case of 
> FairScheduler. FSQueue#hasAccess uses Server.getRemoteAddress() which will be 
> null when the call is from RMWebServices and EventDispatcher. It works fine 
> when called by IPC Server Handler.
> FSQueue#hasAccess is called at three places where (2) and (3) returns null.
> *1. IPC Server -> RMAppManager#createAndPopulateNewRMApp -> FSQueue#hasAccess 
> -> Server.getRemoteAddress returns correct Remote IP.*
>  
> *2. IPC Server -> RMAppManager#createAndPopulateNewRMApp -> 
> AppAddedSchedulerEvent*
>     *EventDispatcher -> FairScheduler#addApplication -> FSQueue.hasAccess -> 
> Server.getRemoteAddress returns null*
>   
> {code:java}
> org.apache.hadoop.yarn.security.ConfiguredYarnAuthorizer.checkPermission(ConfiguredYarnAuthorizer.java:101)
>         at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue.hasAccess(FSQueue.java:316)
>         at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplication(FairScheduler.java:509)
>         at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1268)
>         at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:133)
>         at 
> org.apache.hadoop.yarn.event.EventDispatcher$EventProcessor.run(EventDispatcher.java:66)
> {code}
>  
> *3. RMWebServices -> QueueACLsManager#checkAccess -> FSQueue.hasAccess -> 
> Server.getRemoteAddress returns null.*
> {code:java}
> org.apache.hadoop.yarn.security.ConfiguredYarnAuthorizer.checkPermission(ConfiguredYarnAuthorizer.java:101)
>         at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue.hasAccess(FSQueue.java:316)
>         at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.checkAccess(FairScheduler.java:1610)
>         at 
> org.apache.hadoop.yarn.server.resourcemanager.security.QueueACLsManager.checkAccess(QueueACLsManager.java:84)
>         at 
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.hasAccess(RMWebServices.java:270)
>         at 
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.getApps(RMWebServices.java:553)
> {code}
>  
> Have verified with CapacityScheduler and it works fine.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to