[jira] [Commented] (HDFS-13293) RBF: The RouterRPCServer should transfer client IP via CallerContext to NamenodeRpcServer

2023-02-22 Thread Takanobu Asanuma (Jira) 


[ 
https://issues.apache.org/jira/browse/HDFS-13293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17692551#comment-17692551
 ] 

Takanobu Asanuma commented on HDFS-13293:
-

I added 3.3.5 to fix versions.

> RBF: The RouterRPCServer should transfer client IP via CallerContext to 
> NamenodeRpcServer
> -
>
> Key: HDFS-13293
> URL: https://issues.apache.org/jira/browse/HDFS-13293
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: rbf
>Reporter: Baolong Mao
>Assignee: Hui Fei
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0, 3.3.5
>
> Attachments: HDFS-13293.001.patch
>
>  Time Spent: 4h 40m
>  Remaining Estimate: 0h
>
> Otherwise, the namenode don't know the client's callerContext
> This jira focuses on audit log which logs real client ip. Leave locality to 
> HDFS-13248



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13293) RBF: The RouterRPCServer should transfer client IP via CallerContext to NamenodeRpcServer

2020-10-13 Thread Chengwei Wang (Jira) 


[ 
https://issues.apache.org/jira/browse/HDFS-13293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213026#comment-17213026
 ] 

Chengwei Wang commented on HDFS-13293:
--

Hi [~ferhui], I have filed a new jira HDFS-15630 and would try to fix the issue 
soon.

> RBF: The RouterRPCServer should transfer client IP via CallerContext to 
> NamenodeRpcServer
> -
>
> Key: HDFS-13293
> URL: https://issues.apache.org/jira/browse/HDFS-13293
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: rbf
>Reporter: maobaolong
>Assignee: Hui Fei
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
> Attachments: HDFS-13293.001.patch
>
>  Time Spent: 4h 40m
>  Remaining Estimate: 0h
>
> Otherwise, the namenode don't know the client's callerContext
> This jira focuses on audit log which logs real client ip. Leave locality to 
> HDFS-13248



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13293) RBF: The RouterRPCServer should transfer client IP via CallerContext to NamenodeRpcServer

2020-10-13 Thread Hui Fei (Jira) 


[ 
https://issues.apache.org/jira/browse/HDFS-13293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17212944#comment-17212944
 ] 

Hui Fei commented on HDFS-13293:


[~smarthan] Thanks for report! Would you please file a new jira and Upload your 
improve fix? Thanks

> RBF: The RouterRPCServer should transfer client IP via CallerContext to 
> NamenodeRpcServer
> -
>
> Key: HDFS-13293
> URL: https://issues.apache.org/jira/browse/HDFS-13293
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Reporter: maobaolong
>Assignee: Hui Fei
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
> Attachments: HDFS-13293.001.patch
>
>  Time Spent: 4h 40m
>  Remaining Estimate: 0h
>
> Otherwise, the namenode don't know the client's callerContext
> This jira focuses on audit log which logs real client ip. Leave locality to 
> HDFS-13248



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13293) RBF: The RouterRPCServer should transfer client IP via CallerContext to NamenodeRpcServer

2020-10-13 Thread Chengwei Wang (Jira) 


[ 
https://issues.apache.org/jira/browse/HDFS-13293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17212912#comment-17212912
 ] 

Chengwei Wang commented on HDFS-13293:
--

Hi [~ferhui], did you test this patch while a mount table with 
multi-destinations?

In my test cases, there were two issues when a mount  table contains 
multi-destinations.

1. The clientIp duplicated in CallerContext when 
RouterRpcClient#invokeSequential. It caused by  that current thread would 
append the clientIp to current CallerContext  when it called 
RouterRpcClient#invokeMethod every time.
{code:java}
  2020-10-13 14:46:17,083 [IPC Server handler 6 on default port 13189] INFO  
FSNamesystem.audit (FSNamesystem.java:logAuditMessage(8680)) - allowed=true  
ugi=root (auth:SIMPLE)  ip=/127.0.0.1   cmd=getfileinfo   
src=/test_dir_with_callercontextdst=nullperm=null   proto=rpc   
callerContext=clientContext,clientIp:10.145.43.214  

  2020-10-13 14:46:17,084 [IPC Server handler 1 on default port 14269] INFO  
FSNamesystem.audit (FSNamesystem.java:logAuditMessage(8680)) - allowed=true  
ugi=root (auth:SIMPLE)  ip=/127.0.0.1   cmd=getfileinfo   
src=/test_dir_with_callercontextdst=nullperm=null   proto=rpc 
callerContext=clientContext,clientIp:10.145.43.214,clientIp:10.145.43.214{code}
 

2. the clientIp missed in CallerContext when RouterRpcClient#invokeConcurrent. 
It caused by that  RouterRpcClient#invokeConcurrent used a ThreadPoolExecutor 
to execute request concurrent, however the worker threads of ThreadPoolExecutor 
can't get clientIp by Server#getRemoteAddress (thread local value only 
initialized of current thread).

 
{code:java}
  2020-10-13 14:46:18,305 [IPC Server handler 7 on default port 13189] INFO 
FSNamesystem.audit (FSNamesystem.java:logAuditMessage(8680)) - allowed=true 
ugi=root (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/ dst=null perm=null 
proto=rpc callerContext=clientContext
  2020-10-13 14:46:18,305 [IPC Server handler 0 on default port 14269] INFO 
FSNamesystem.audit (FSNamesystem.java:logAuditMessage(8680)) - allowed=true 
ugi=root (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/ dst=null perm=null 
proto=rpc callerContext=clientContext
{code}
 

 

Please help to check the issues, and I would try to fix these  if necessary.

> RBF: The RouterRPCServer should transfer client IP via CallerContext to 
> NamenodeRpcServer
> -
>
> Key: HDFS-13293
> URL: https://issues.apache.org/jira/browse/HDFS-13293
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Reporter: maobaolong
>Assignee: Hui Fei
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
> Attachments: HDFS-13293.001.patch
>
>  Time Spent: 4h 40m
>  Remaining Estimate: 0h
>
> Otherwise, the namenode don't know the client's callerContext
> This jira focuses on audit log which logs real client ip. Leave locality to 
> HDFS-13248



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13293) RBF: The RouterRPCServer should transfer client IP via CallerContext to NamenodeRpcServer

2020-10-09 Thread Hui Fei (Jira) 


[ 
https://issues.apache.org/jira/browse/HDFS-13293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17210722#comment-17210722
 ] 

Hui Fei commented on HDFS-13293:


Merged. [~maobaolong] Thanks for report, [~aajisaka] [~elgoiri] Thanks for 
review!

> RBF: The RouterRPCServer should transfer client IP via CallerContext to 
> NamenodeRpcServer
> -
>
> Key: HDFS-13293
> URL: https://issues.apache.org/jira/browse/HDFS-13293
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Reporter: maobaolong
>Assignee: Hui Fei
>Priority: Major
>  Labels: pull-request-available
> Attachments: HDFS-13293.001.patch
>
>  Time Spent: 3h 50m
>  Remaining Estimate: 0h
>
> Otherwise, the namenode don't know the client's callerContext
> This jira focuses on audit log which logs real client ip. Leave locality to 
> HDFS-13248



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13293) RBF: The RouterRPCServer should transfer client ip via CallerContext to NamenodeRpcServer

2020-10-07 Thread Hui Fei (Jira) 


[ 
https://issues.apache.org/jira/browse/HDFS-13293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209490#comment-17209490
 ] 

Hui Fei commented on HDFS-13293:


Modify the caption and the description, focus on audit log

> RBF: The RouterRPCServer should transfer client ip via CallerContext to 
> NamenodeRpcServer
> -
>
> Key: HDFS-13293
> URL: https://issues.apache.org/jira/browse/HDFS-13293
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Reporter: maobaolong
>Assignee: Hui Fei
>Priority: Major
>  Labels: pull-request-available
> Attachments: HDFS-13293.001.patch
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> Otherwise, the namenode don't know the client's callerContext
> This jira focuses on audit log which logs real client ip. Leave locality to 
> HDFS-13248



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org