Surendra Singh Lilhore created HDFS-14581: ---------------------------------------------
Summary: EC: Append validation should be done before getting lease on file Key: HDFS-14581 URL: https://issues.apache.org/jira/browse/HDFS-14581 Project: Hadoop HDFS Issue Type: Bug Components: erasure-coding Affects Versions: 3.3.0 Reporter: Surendra Singh Lilhore Assignee: Surendra Singh Lilhore *org.apache.hadoop.hdfs.server.namenode.FSDirAppendOp.prepareFileForAppend(..)@189* {noformat} file.recordModification(iip.getLatestSnapshotId()); file.toUnderConstruction(leaseHolder, clientMachine); fsn.getLeaseManager().addLease( file.getFileUnderConstructionFeature().getClientName(), file.getId()); LocatedBlock ret = null; if (!newBlock) { if (file.isStriped()) { throw new UnsupportedOperationException( "Append on EC file without new block is not supported."); }{noformat} In this code "UnsupportedOperationException" exception thows after marking file toUnderConstruction. In this case file is opened without any "Open" editlogs, after some time lease manager close this file and add close edit log. When SBN tail this edit log, it will fail with this exception. {noformat} 2019-06-13 19:17:51,513 ERROR org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader: Encountered exception on operation CloseOp [length=0, inodeId=0, path=/ECtest/********, replication=1, mtime=1560261947480, atime=1560258249117, blockSize=134217728, blocks=[blk_-9223372036854775792_1005], permissions=root:hadoop:rw-r--r--, aclEntries=null, clientName=, clientMachine=, overwrite=false, storagePolicyId=0, erasureCodingPolicyId=0, opCode=OP_CLOSE, txid=1363] java.io.IOException: File is not under construction: /ECtest/container-executor at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.applyEditLogOp(FSEditLogLoader.java:504) at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:286) at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:181) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:924) at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:329) at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:460) at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$300(EditLogTailer.java:410) at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:427) at org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:485) at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:423) {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org