[
https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777382#comment-16777382
]
Konstantin Shvachko commented on HDFS-14272:
--------------------------------------------
??If msync() is called on an observer or standby NN, the node will reject the
request.??
Exactly right, so {{msync()}} will try to connect to all NNs until it finds the
active node. More precisely, msync() exploits the failover mechanism to loop
over the available NameNodes, which is a correct way of doing things, but not
the most efficient.
What I suggest is to loop explicitly through available NNs using
{{getHAServiceState()}} on the first RPC call. It is equivalent to what msync()
will be doing in this case, but with half of RPC calls. Since proxies to
individual NNs have not been created yet, {{changeProxy()}} will create proxy
for the corresponding NN, then call {{getHAServiceState()}} and cache the
state. Then it will proceed with the msync() call, which is not needed because
the client already knows the state of that NN.
> [SBN read] ObserverReadProxyProvider should sync with active txnID on startup
> -----------------------------------------------------------------------------
>
> Key: HDFS-14272
> URL: https://issues.apache.org/jira/browse/HDFS-14272
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: tools
> Environment: CDH6.1 (Hadoop 3.0.x) + Consistency Reads from Standby +
> SSL + Kerberos + RPC encryption
> Reporter: Wei-Chiu Chuang
> Assignee: Erik Krogen
> Priority: Major
> Attachments: HDFS-14272.000.patch, HDFS-14272.001.patch
>
>
> It is typical for integration tests to create some files and then check their
> existence. For example, like the following simple bash script:
> {code:java}
> # hdfs dfs -touchz /tmp/abc
> # hdfs dfs -ls /tmp/abc
> {code}
> The test executes HDFS bash command sequentially, but it may fail with
> Consistent Standby Read because the -ls does not find the file.
> Analysis: the second bash command, while launched sequentially after the
> first one, is not aware of the state id returned from the first bash command.
> So ObserverNode wouldn't wait for the the edits to get propagated, and thus
> fails.
> I've got a cluster where the Observer has tens of seconds of RPC latency, and
> this becomes very annoying. (I am still trying to figure out why this
> Observer has such a long RPC latency. But that's another story.)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
