[
https://issues.apache.org/jira/browse/HDFS-13610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524657#comment-16524657
]
Erik Krogen commented on HDFS-13610:
------------------------------------
Looking through the test failures, I noticed some testing issues, as well as
some bugs with the existing code. Given that this is meant to be a "cleanup"
JIRA I plan to fix the bugs here; if anyone disagrees let me know and I can
file separate JIRAs.
* Due to changes in HDFS-13609 to enable use of the RPC mechanism whenever
inProgressOK is true, there were codepaths which would use the RPC mechanism
even when dfs.ha.tail-edits.in-progress was false, meaning that the JNs did not
enable the cache. I have updated the QJM logic to only use
{{selectRpcInputStreams}} if this config is true.
* Fixed a bug where there was a false error logged when the layout version
changed. Also fixed the logging when a layout version change occurs to avoid
printing out a placeholder value ({{Integer.MAX_VALUE}}).
* Fixed a test in {{TestQuorumJournalManager}}.
* The tests in {{TestStandbyInProgressTail}}, now that they use the RPC
mechanism, were subject to the failure scenario described below. I updated the
tests to be resilient to this by allowing them to wait briefly for the expected
edit to be tailed.
** JNs A, B, and C are at txID 1.
** A write is persisted to a quorum of JNs, say A and B. They are now at txID
2, and C is at txID 1.
** The edit tailing succeeds from a different quorum of JNs, say B and C. Only
transactions up to txID 1 will be tailed.
** An assertion requiring that the standby sees txID 2 fails.
I have uploaded a v002 patch incorporating these changes.
> [Edit Tail Fast Path Pt 4] Cleanup: integration test, documentation, remove
> unnecessary dummy sync
> --------------------------------------------------------------------------------------------------
>
> Key: HDFS-13610
> URL: https://issues.apache.org/jira/browse/HDFS-13610
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha, journal-node, namenode
> Reporter: Erik Krogen
> Assignee: Erik Krogen
> Priority: Major
> Attachments: HDFS-13610-HDFS-12943.000.patch,
> HDFS-13610-HDFS-12943.001.patch, HDFS-13610-HDFS-12943.002.patch
>
>
> See HDFS-13150 for full design.
> This JIRA is targeted at cleanup tasks:
> * Add in integration testing. We can expand {{TestStandbyInProgressTail}}
> * Documentation in HDFSHighAvailabilityWithQJM
> * Remove the dummy sync added as part of HDFS-10519; it is unnecessary since
> now in-progress tailing does not rely as heavily on the JN committedTxnId
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
