[
https://issues.apache.org/jira/browse/HDFS-13609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17277005#comment-17277005
]
xuzq commented on HDFS-13609:
-----------------------------
Hi [~xkrogen] and [~linyiqun] , recently I am learning *Consistent Reads from
Standby Node*.
{code:java}
private void selectRpcInputStreams(Collection<EditLogInputStream> streams,
long fromTxnId, boolean onlyDurableTxns) throws IOException {
QuorumCall<AsyncLogger, GetJournaledEditsResponseProto> q =
loggers.getJournaledEdits(fromTxnId, maxTxnsPerRpc);
Map<AsyncLogger, GetJournaledEditsResponseProto> responseMap =
loggers.waitForWriteQuorum(q, selectInputStreamsTimeoutMs,
"selectRpcInputStreams");
assert responseMap.size() >= loggers.getMajoritySize() :
"Quorum call returned without a majority";
List<Integer> responseCounts = new ArrayList<>();
for (GetJournaledEditsResponseProto resp : responseMap.values()) {
responseCounts.add(resp.getTxnCount());
}
Collections.sort(responseCounts);
int highestTxnCount = responseCounts.get(responseCounts.size() - 1);
...
// Cancel any outstanding calls to JN's.
q.cancelCalls();
int maxAllowedTxns = !onlyDurableTxns ? highestTxnCount :
responseCounts.get(responseCounts.size() - loggers.getMajoritySize());
if (maxAllowedTxns == 0) {
LOG.debug("No new edits available in logs; requested starting from " +
"ID " + fromTxnId);
return;
}
...
}
{code}
Maybe somethings wrong in
{code:java}
int maxAllowedTxns = !onlyDurableTxns ? highestTxnCount :
responseCounts.get(responseCounts.size() - loggers.getMajoritySize());{code}
* Let's say we have 5 JournalNodes, and loggers.getMajoritySize() is 3.
* _loggers.getJournaledEdits(fromTxnId, maxTxnsPerRpc)_ just need quorum
result, so responseCounts.size() maybe is 3.
* when _onlyDurableTxns_ is false, _maxAllowedTxns_ = responseCounts.get(0)
* _responseCounts.get(0)_ maybe not expect Quorum Result, and it maybe even
doesn't have any results from _fromTxnId_
** maybe one journal disk is wrong, and only write into cache for _fromTxnId_
[~xkrogen] and [~linyiqun], if have time, please look at this question, thanks.
> [Edit Tail Fast Path Pt 3] NameNode-side changes to support tailing edits via
> RPC
> ---------------------------------------------------------------------------------
>
> Key: HDFS-13609
> URL: https://issues.apache.org/jira/browse/HDFS-13609
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha, namenode
> Reporter: Erik Krogen
> Assignee: Erik Krogen
> Priority: Major
> Fix For: HDFS-12943, 3.3.0
>
> Attachments: HDFS-13609-HDFS-12943.000.patch,
> HDFS-13609-HDFS-12943.001.patch, HDFS-13609-HDFS-12943.002.patch,
> HDFS-13609-HDFS-12943.003.patch, HDFS-13609-HDFS-12943.004.patch
>
>
> See HDFS-13150 for the full design.
> This JIRA is targetted at the NameNode-side changes to enable tailing
> in-progress edits via the RPC mechanism added in HDFS-13608. Most changes are
> in the QuorumJournalManager.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
