[
https://issues.apache.org/jira/browse/HDFS-14604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17269046#comment-17269046
]
Jiahao Dai commented on HDFS-14604:
-----------------------------------
Hi all,
We met this problem recently, we would like to reopen that this issue is not
resolved in apache hadoop 3.3.0 release as following
{code:java}
// [root@cm hadoop-3.3.0]# java -version
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)
[root@cm hadoop-3.3.0]#
[root@cm hadoop-3.3.0]# JAVA_HOME=/usr/java/jdk1.8.0_181-oracle/ ./bin/hdfs oev
-i ~/wb/edits_0000000000559774991-0000000000559786960 -o ~/wb/out.xml
[root@cm hadoop-3.3.0]# echo $?
0
[root@cm hadoop-3.3.0]# JAVA_HOME=/usr/java/jdk1.8.0_181-oracle/ ./bin/hdfs oev
-p binary -i ~/wb/out.xml -o ~/wb/out.binary
fromXml error decoding opcode null
{<BLOCK>{<BLOCK_ID>{"1146070648"}, <GENSTAMP>{"73083476"}, <NUM_BYTES>{"7"}},
<CLIENTMACHINE>{"172.23.4.28"},
<CLIENTNAME>{"DFSClient_NONMAPREDUCE_1383910112_346"}, <NEWLENGTH>{"2"},
<SRC>{"/xx/xx"}, <TIMESTAMP>{"1611150764878"}, <TXID>{"559775003"}}
Encountered exception. Exiting: no entry found for BLOCK_ID
org.apache.hadoop.hdfs.util.XMLUtils$InvalidXmlException: no entry found for
BLOCK_ID
at
org.apache.hadoop.hdfs.util.XMLUtils$Stanza.getValue(XMLUtils.java:314)
at
org.apache.hadoop.hdfs.server.namenode.FSEditLogOp.blockFromXml(FSEditLogOp.java:5380)
at
org.apache.hadoop.hdfs.server.namenode.FSEditLogOp$TruncateOp.fromXml(FSEditLogOp.java:2937)
at
org.apache.hadoop.hdfs.server.namenode.FSEditLogOp.decodeXml(FSEditLogOp.java:5363)
at
org.apache.hadoop.hdfs.tools.offlineEditsViewer.OfflineEditsXmlLoader.endElement(OfflineEditsXmlLoader.java:224)
at
com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
at
com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:842)
at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:771)
at
com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
at
com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
at
org.apache.hadoop.hdfs.tools.offlineEditsViewer.OfflineEditsXmlLoader.loadEdits(OfflineEditsXmlLoader.java:92)
at
org.apache.hadoop.hdfs.tools.offlineEditsViewer.OfflineEditsViewer.go(OfflineEditsViewer.java:158)
at
org.apache.hadoop.hdfs.tools.offlineEditsViewer.OfflineEditsViewer.run(OfflineEditsViewer.java:251)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at
org.apache.hadoop.hdfs.tools.offlineEditsViewer.OfflineEditsViewer.main(OfflineEditsViewer.java:260)
[root@cm hadoop-3.3.0]#
{code}
> OEV tool fails to read edit xml file if OP_TRUNCATE has BLOCK tag
> ------------------------------------------------------------------
>
> Key: HDFS-14604
> URL: https://issues.apache.org/jira/browse/HDFS-14604
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: tools
> Affects Versions: 3.0.2, 3.1.1, 2.9.2, 3.1.2
> Reporter: huowang
> Priority: Major
> Labels: patch
> Attachments: image-2019-06-25-14-56-53-151.png,
> image-2019-06-25-14-58-39-681.png
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> 使用 HDFS OEV 命令处理 edits 文件,从xml格式的文件转成成binary格式的文件的时候,当xml文件中存在
> OP_TRUNCATE 操作并且含有子 BLOCK的时候报错
> 我追踪了源码发现 blockFromXml 方法需要的是一个block 对象但是被传递的是一个
> data对象,我已经解决了这个问题,并且验证了可用性,
> When edits files are processed with HDFS OEV command and converted from
> XML files to binary files, errors are reported when OP_TRUNCATE operation
> exists in XML files and BLOCKs are included.
> I tracked the source code and found that the blockFromXml method needed a
> block object but was passed a data object. I have solved this problem and
> verified its availability.
> I will submit a patch for this as soon as possible.
> !image-2019-06-25-14-56-53-151.png!
> !image-2019-06-25-14-58-39-681.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
