[
https://issues.apache.org/jira/browse/HDFS-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17410334#comment-17410334
]
lei w edited comment on HDFS-16196 at 9/6/21, 3:28 AM:
-------------------------------------------------------
Thanks [~hexiaoqiao] comment. I checked the logic of trunk was same like this.
In RouterClientProtocol#complete logic is as follow:
RouterClientProtocol#complete(... ...) --> RouterRpcClient#invokeSingle(...
...) --> RouterRpcClient#invokeMethod(..) --> RPC to NN.
And in RouterClientProtocol#complete(... ...) we will generate RemoteMethod
as follows:
{code:java}
RemoteMethod method = new RemoteMethod("complete",
new Class<?>[] {String.class, String.class, ExtendedBlock.class,
long.class},
// params[0] params[1] params[2] params[3]
new RemoteParam(), clientName, last, fileId);
{code}
RemoteMethod uses an array of object type named params to store parameters ,
and params[0] will store class RemoteParam instance.
Then in RouterRpcClient#invokeSingle(... ...) we will generate path by
RemoteLocation instance as follows :
{code:java}
List<? extends FederationNamenodeContext> nns =
getNamenodesForNameservice(nsId);
// create RemoteLocation with src path "/" , destpath "/"
RemoteLocationContext loc = new RemoteLocation(nsId, "/", "/");
Class<?> proto = method.getProtocol();
Method m = method.getMethod();
//Because RemoteMethod params[0] store class RemoteParam instance. we
will use RemoteLocation dest "/" as method path. please look
router.RemoteMethod#getParams(... ...)
Object[] params = method.getParams(loc);
return invokeMethod(ugi, nns, proto, m, params);
{code}
As mentioned above, NameNodeRpcServer will receive "/" as complete method path
information and will log "/" in audit log. This is my simple analysis. If you
have any questions, thank you for your correction.
was (Author: lei w):
Thanks [~hexiaoqiao] comment. I checked the logic of trunk was same like this.
In RouterClientProtocol#complete logic is as follow:
RouterClientProtocol#complete(... ...) --> RouterRpcClient#invokeSingle(...
...) --> RouterRpcClient#invokeMethod(..) --> RPC to NN.
And in RouterClientProtocol#complete(... ...) we will generate RemoteMethod
as follows:
{code:java}
RemoteMethod method = new RemoteMethod("complete",
new Class<?>[] {String.class, String.class, ExtendedBlock.class,
long.class},
// params[0] params[1] params[2] params[3]
new RemoteParam(), clientName, last, fileId);
{code}
RemoteMethod uses an array of object type named params to store parameters ,
and params[0] will store class RemoteParam instance.
Then in RouterRpcClient#invokeSingle(... ...) we will generate path by
RemoteLocation instance as follows :
{code:java}
List<? extends FederationNamenodeContext> nns =
getNamenodesForNameservice(nsId);
// create RemoteLocation with src path "/" , destpath "/"
RemoteLocationContext loc = new RemoteLocation(nsId, "/", "/");
Class<?> proto = method.getProtocol();
Method m = method.getMethod();
//Because RemoteMethod params[0] store class RemoteParam instance. we
will use RemoteLocation dest "/" as method path. please look
router.RemoteMethod#getParams(... ...)
Object[] params = method.getParams(loc);
return invokeMethod(ugi, nns, proto, m, params);
{code}
As mentioned above, NameNodeRpcServer will receive "/" as complete method path
information and will log "/" in audit log. This is my simple analysis. If you
have any questions, thank you for your correction.
> Namesystem#completeFile method will log incorrect path information when
> router to access
> ----------------------------------------------------------------------------------------
>
> Key: HDFS-16196
> URL: https://issues.apache.org/jira/browse/HDFS-16196
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: lei w
> Priority: Minor
>
> Router not send entire path information to namenode because
> ClientProtocol#complete method`s parameter with fileId. Then NameNode will
> log incorrect path information. This is very confusing, should we let the
> router pass the path information or modify the log path on namenode?
> completeFile log as fllow:
> StateChange: DIR* completeFile: / is closed by DFSClient_NONMAPREDUC_*****
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
