abhishek sahani created HDFS-14929:
--------------------------------------
Summary: Hadoop 2.9.1 rename functionality infrequent breaking
Key: HDFS-14929
URL: https://issues.apache.org/jira/browse/HDFS-14929
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 2.9.1
Reporter: abhishek sahani
We are infrequently seeing rename functionality not working properly , in logs
it seems file is rename is success but in ui and when listing the files using
./hdfs dfs -ls-R file is not present .
DEBUG hdfs.StateChange: *DIR* NameNode.rename:
/topics/+tmp/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_15/year=2019/month=10/day=08/hour=19/894901ae-5913-4ad9-8d65-7071655c2db0_tmp.parquet
to
/topics/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_15/year=2019/month=10/day=08/hour=19/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic+10+0000000000+0000000099.parquet19/10/23
19:06:41 DEBUG hdfs.StateChange: *DIR* NameNode.rename:
/topics/+tmp/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_15/year=2019/month=10/day=08/hour=19/894901ae-5913-4ad9-8d65-7071655c2db0_tmp.parquet
to
/topics/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_15/year=2019/month=10/day=08/hour=19/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic+10+0000000000+0000000099.parquet
19/10/23 19:06:41DEBUG hdfs.StateChange: DIR* NameSystem.renameTo:
/topics/+tmp/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_15/year=2019/month=10/day=08/hour=19/894901ae-5913-4ad9-8d65-7071655c2db0_tmp.parquet
to
/topics/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_15/year=2019/month=10/day=08/hour=19/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic+10+0000000000+0000000099.parquet19/10/23
19:06:41DEBUG hdfs.StateChange: DIR* FSDirectory.renameTo:
/topics/+tmp/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_15/year=2019/month=10/day=08/hour=19/894901ae-5913-4ad9-8d65-7071655c2db0_tmp.parquet
to
/topics/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_15/year=2019/month=10/day=08/hour=19/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic+10+0000000000+0000000099.parquet19/10/23
19:06:41 DEBUG hdfs.StateChange: DIR* FSDirectory.unprotectedRenameTo:
/topics/+tmp/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_15/year=2019/month=10/day=08/hour=19/894901ae-5913-4ad9-8d65-7071655c2db0_tmp.parquet
is renamed to
/topics/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_15/year=2019/month=10/day=08/hour=19/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic+10+0000000000+0000000099.parquet19/10/23
19:06:41 DEBUG namenode.FSEditLog: logEdit [RpcEdit op:RenameOldOp [length=0,
src=/topics/+tmp/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_15/year=2019/month=10/day=08/hour=19/894901ae-5913-4ad9-8d65-7071655c2db0_tmp.parquet,
dst=/topics/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_15/year=2019/month=10/day=08/hour=19/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic+10+0000000000+0000000099.parquet,
timestamp=1571837801360, RpcClientId=bfde99e8-a523-47c0-ab94-b14f96122d02,
RpcCallId=17790, opCode=OP_RENAME_OLD, txid=-12345] call:Call#17790 Retry#0
org.apache.hadoop.hdfs.protocol.ClientProtocol.rename from
19/10/23 19:06:41 DEBUG namenode.FSEditLog: logSync [RpcEdit op:RenameOldOp
[length=0,
src=/topics/+tmp/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_15/year=2019/month=10/day=08/hour=19/894901ae-5913-4ad9-8d65-7071655c2db0_tmp.parquet,
dst=/topics/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_15/year=2019/month=10/day=08/hour=19/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic+10+0000000000+0000000099.parquet,
timestamp=1571837801360, RpcClientId=bfde99e8-a523-47c0-ab94-b14f96122d02,
RpcCallId=17790, opCode=OP_RENAME_OLD, txid=-12345] call:Call#17790 Retry#0
org.apache.hadoop.hdfs.protocol.ClientProtocol.rename from
19/10/23 19:06:41 DEBUG top.TopAuditLogger: ------------------- logged event
for top service: allowed=true ugi=root (auth:SIMPLE) ip=/*.*.*.* cmd=rename
src=/topics/+tmp/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_15/year=2019/month=10/day=08/hour=19/894901ae-5913-4ad9-8d65-7071655c2db0_tmp.parquet
dst=/topics/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_15/year=2019/month=10/day=08/hour=19/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic+10+0000000000+0000000099.parquet
perm=root:supergroup:rw-r--r--19/10/23 19:10:40 DEBUG top.TopAuditLogger:
------------------- logged event for top service: allowed=true ugi=root
(auth:SIMPLE) ip=/*.*.*.* cmd=getfileinfo
src=/topics/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_15/year=2019/month=10/day=08/hour=19/datapipelinefinaltest2.5da59e664cedfd00090d3757.dataPipeLineEvent_15.topic+10+0000000000+0000000099.parquet
dst=null perm=null
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
