[
https://issues.apache.org/jira/browse/HDFS-10572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15350932#comment-15350932
]
Yiqun Lin edited comment on HDFS-10572 at 6/27/16 12:46 PM:
------------------------------------------------------------
It seemed that the content was different in file {{edits}} and
{{editsReparsed}}. I parsed the binary file {{editsReparsed}} back to xml
again. And I found there were some differences. Here are the differences, from
txid 84 to 86.
Content in {{edits}}:
{code}
<RECORD>
<OPCODE>OP_REASSIGN_LEASE</OPCODE>
<DATA>
<TXID>84</TXID>
<LEASEHOLDER>DFSClient_NONMAPREDUCE_428966708_1</LEASEHOLDER>
<PATH>/hard-lease-recovery-test</PATH>
<NEWHOLDER>HDFS_NameNode</NEWHOLDER>
</DATA>
</RECORD>
<RECORD>
<OPCODE>OP_CLOSE</OPCODE>
<DATA>
<TXID>85</TXID>
<LENGTH>0</LENGTH>
<INODEID>0</INODEID>
<PATH>/hard-lease-recovery-test</PATH>
<REPLICATION>1</REPLICATION>
<MTIME>1467028963024</MTIME>
<ATIME>1467028960848</ATIME>
<BLOCKSIZE>512</BLOCKSIZE>
<CLIENT_NAME></CLIENT_NAME>
<CLIENT_MACHINE></CLIENT_MACHINE>
<OVERWRITE>false</OVERWRITE>
<BLOCK>
<BLOCK_ID>1073741837</BLOCK_ID>
<NUM_BYTES>11</NUM_BYTES>
<GENSTAMP>1014</GENSTAMP>
</BLOCK>
<PERMISSION_STATUS>
<USERNAME>zhexuan</USERNAME>
<GROUPNAME>supergroup</GROUPNAME>
<MODE>420</MODE>
</PERMISSION_STATUS>
</DATA>
</RECORD>
<RECORD>
<OPCODE>OP_ADD_CACHE_POOL</OPCODE>
<DATA>
<TXID>86</TXID>
<POOLNAME>pool1</POOLNAME>
<OWNERNAME>zhexuan</OWNERNAME>
<GROUPNAME>staff</GROUPNAME>
<MODE>493</MODE>
<LIMIT>9223372036854775807</LIMIT>
<MAXRELATIVEEXPIRY>2305843009213693951</MAXRELATIVEEXPIRY>
<RPC_CLIENTID>03f2daa2-e04f-4b8f-aa09-5d21e14024bd</RPC_CLIENTID>
<RPC_CALLID>81</RPC_CALLID>
</DATA>
</RECORD>
{code}
Content parsed from file {{editsReparsed}}:
{code}
<RECORD>
<OPCODE>OP_REASSIGN_LEASE</OPCODE>
<DATA>
<TXID>84</TXID>
<LEASEHOLDER>DFSClient_NONMAPREDUCE_929984910_1</LEASEHOLDER>
<PATH>/hard-lease-recovery-test</PATH>
<NEWHOLDER>HDFS_NameNode</NEWHOLDER>
</DATA>
</RECORD>
<RECORD>
<OPCODE>OP_SET_GENSTAMP_V2</OPCODE>
<DATA>
<TXID>85</TXID>
<GENSTAMPV2>1015</GENSTAMPV2>
</DATA>
</RECORD>
<RECORD>
<OPCODE>OP_REASSIGN_LEASE</OPCODE>
<DATA>
<TXID>86</TXID>
<LEASEHOLDER>HDFS_NameNode</LEASEHOLDER>
<PATH>/hard-lease-recovery-test</PATH>
<NEWHOLDER>HDFS_NameNode</NEWHOLDER>
</DATA>
</RECORD>
{code}
We can make use of these infos, it will help us to fix this issue.
was (Author: linyiqun):
It seemed that the content was different in file {{edits}} and
{{editsReparsed}}. I parsed the binary file {{editsReparsed}} back to xml
again. And I found there were some differences. Here are the differences, from
txid 84 to 86.
Content in {{edits}}:
{no-format}
<RECORD>
<OPCODE>OP_REASSIGN_LEASE</OPCODE>
<DATA>
<TXID>84</TXID>
<LEASEHOLDER>DFSClient_NONMAPREDUCE_428966708_1</LEASEHOLDER>
<PATH>/hard-lease-recovery-test</PATH>
<NEWHOLDER>HDFS_NameNode</NEWHOLDER>
</DATA>
</RECORD>
<RECORD>
<OPCODE>OP_CLOSE</OPCODE>
<DATA>
<TXID>85</TXID>
<LENGTH>0</LENGTH>
<INODEID>0</INODEID>
<PATH>/hard-lease-recovery-test</PATH>
<REPLICATION>1</REPLICATION>
<MTIME>1467028963024</MTIME>
<ATIME>1467028960848</ATIME>
<BLOCKSIZE>512</BLOCKSIZE>
<CLIENT_NAME></CLIENT_NAME>
<CLIENT_MACHINE></CLIENT_MACHINE>
<OVERWRITE>false</OVERWRITE>
<BLOCK>
<BLOCK_ID>1073741837</BLOCK_ID>
<NUM_BYTES>11</NUM_BYTES>
<GENSTAMP>1014</GENSTAMP>
</BLOCK>
<PERMISSION_STATUS>
<USERNAME>zhexuan</USERNAME>
<GROUPNAME>supergroup</GROUPNAME>
<MODE>420</MODE>
</PERMISSION_STATUS>
</DATA>
</RECORD>
<RECORD>
<OPCODE>OP_ADD_CACHE_POOL</OPCODE>
<DATA>
<TXID>86</TXID>
<POOLNAME>pool1</POOLNAME>
<OWNERNAME>zhexuan</OWNERNAME>
<GROUPNAME>staff</GROUPNAME>
<MODE>493</MODE>
<LIMIT>9223372036854775807</LIMIT>
<MAXRELATIVEEXPIRY>2305843009213693951</MAXRELATIVEEXPIRY>
<RPC_CLIENTID>03f2daa2-e04f-4b8f-aa09-5d21e14024bd</RPC_CLIENTID>
<RPC_CALLID>81</RPC_CALLID>
</DATA>
</RECORD>
{no-format}
Content parsed from file {{editsReparsed}}:
{no-format}
<RECORD>
<OPCODE>OP_REASSIGN_LEASE</OPCODE>
<DATA>
<TXID>84</TXID>
<LEASEHOLDER>DFSClient_NONMAPREDUCE_929984910_1</LEASEHOLDER>
<PATH>/hard-lease-recovery-test</PATH>
<NEWHOLDER>HDFS_NameNode</NEWHOLDER>
</DATA>
</RECORD>
<RECORD>
<OPCODE>OP_SET_GENSTAMP_V2</OPCODE>
<DATA>
<TXID>85</TXID>
<GENSTAMPV2>1015</GENSTAMPV2>
</DATA>
</RECORD>
<RECORD>
<OPCODE>OP_REASSIGN_LEASE</OPCODE>
<DATA>
<TXID>86</TXID>
<LEASEHOLDER>HDFS_NameNode</LEASEHOLDER>
<PATH>/hard-lease-recovery-test</PATH>
<NEWHOLDER>HDFS_NameNode</NEWHOLDER>
</DATA>
</RECORD>
{no-format}
We can make use of these infos, it will help us to fix this issue.
> Fix TestOfflineEditsViewer#testGenerated
> ----------------------------------------
>
> Key: HDFS-10572
> URL: https://issues.apache.org/jira/browse/HDFS-10572
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: newbie, test
> Reporter: Xiaoyu Yao
> Assignee: Hanisha Koneru
>
> The test has been failing consistently on trunk recently. This ticket is open
> to fix this test to avoid false alarm on Jenkins. Figure out which recent
> commit caused this failure can be a good start.
>
> {code}
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Running org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
> Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 15.646 sec
> <<< FAILURE! - in
> org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
> testGenerated(org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer)
> Time elapsed: 3.623 sec <<< FAILURE!
> java.lang.AssertionError: Generated edits and reparsed (bin to XML to bin)
> should be same
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer.testGenerated(TestOfflineEditsViewer.java:125)
> Results :
> Failed tests:
> TestOfflineEditsViewer.testGenerated:125 Generated edits and reparsed (bin
> to XML to bin) should be same
> Tests run: 5, Failures: 1, Errors: 0, Skipped: 0
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
