[
https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584629#comment-14584629
]
Akira AJISAKA commented on MAPREDUCE-6363:
------------------------------------------
Thanks [~brahmareddy] for updating the patch.
{code}
...
- doCreateWriteOp("file_" + hostName + "_", reporter);
+ doCreateWriteOp("file_" + taskId + "_", reporter);
} else if (op.equals(OP_OPEN_READ)) {
startTimeTPmS = System.currentTimeMillis();
- doOpenReadOp("file_" + hostName + "_", reporter);
+ doOpenReadOp("file_" + taskId + "_", reporter);
} else if (op.equals(OP_RENAME)) {
startTimeTPmS = System.currentTimeMillis();
- doRenameOp("file_" + hostName + "_", reporter);
+ doRenameOp("file_" + taskId + "_", reporter);
} else if (op.equals(OP_DELETE)) {
...
{code}
Would you create a variable such as {{filePrefix}} and reuse it as follows?
That way we can avoid inconsistent file names by typo.
{code}
if (barrier()) {
String filePrefix = "file_" + taskId + "_";
startTimeTPmS = System.currentTimeMillis();
if (op.equals(OP_CREATE_WRITE)) {
doCreateWriteOp(filePrefix, reporter);
} else if (op.equals(OP_OPEN_READ)) {
doOpenReadOp(filePrefix, reporter);
...
{code}
> [NNBench mark] Lease mismatch error when running with multiple mappers
> ----------------------------------------------------------------------
>
> Key: MAPREDUCE-6363
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
> Reporter: Brahma Reddy Battula
> Assignee: Brahma Reddy Battula
> Priority: Critical
> Labels: BB2015-05-TBR
> Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch
>
>
> Command :
> ./yarn jar
> ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar
> nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456
> -bytesToWrite 1024000000 -baseDir /benchmarks/NNBench`hostname -s`
> -replicationFactorPerFile 3 -maps 100 -reduces 10
> Trace :
> 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler
> 7 on 9005, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from
> 192.168.105.214:36320: error:
> org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch
> on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by
> DFSClient_attempt_1371782327901_0001_m_000048_0_1383437860_1 but is accessed
> by DFSClient_attempt_1371782327901_0001_m_000084_0_1880545303_1
> org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch
> on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by
> DFSClient_attempt_1371782327901_0001_m_000048_0_1383437860_1 but is accessed
> by DFSClient_attempt_1371782327901_0001_m_000084_0_1880545303_1
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
