[
https://issues.apache.org/jira/browse/HDFS-4298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron T. Myers updated HDFS-4298:
---------------------------------
Attachment: HDFS-4298.patch
Here's a patch which addresses the issue by plumbing through a "forReading"
parameter to selectInputStreams/getEditLogManifest so that the NN edit log
purger can indicate to the JNs that it won't be reading from the transaction ID
it's asking for, and thus the JNs shouldn't error out if the txid isn't on an
edit log segment boundary.
This patch also takes the liberty of fixing a GenericTestUtils where we were
passing (actual, expected) instead of (expected, actual).
> StorageRetentionManager spews warnings when used with QJM
> ---------------------------------------------------------
>
> Key: HDFS-4298
> URL: https://issues.apache.org/jira/browse/HDFS-4298
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 3.0.0, 2.0.3-alpha
> Reporter: Todd Lipcon
> Assignee: Aaron T. Myers
> Attachments: HDFS-4298.patch
>
>
> When the NN is configured with a QJM, we see the following warning message
> every time a checkpoint is made or uploaded:
> 12/12/10 16:07:52 WARN namenode.FSEditLog: Unable to determine input streams
> from QJM to [127.0.0.1:13001, 127.0.0.1:13002, 127.0.0.1:13003]. Skipping.
> org.apache.hadoop.hdfs.qjournal.client.QuorumException: Got too many
> exceptions to achieve quorum size 2/3. 3 exceptions thrown:
> 127.0.0.1:13002: Asked for firstTxId 114837 which is in the middle of file
> /tmp/jn-2/myjournal/current/edits_0000000000000095185-0000000000000114846
> ...
> This is because, since HDFS-2946, the NN calls {{selectInputStreams}} to
> determine the number of log segments and put a cap on the number. This API
> throws an exception in the case of QJM if the argument falls in the middle of
> an edit log boundary.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
