[
https://issues.apache.org/jira/browse/HDFS-10702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426044#comment-15426044
]
Andrew Wang commented on HDFS-10702:
------------------------------------
Hi Jiayi, thanks for working on this. I didn't get through all of it, notably
the StaleReadProxyProvider or the unit tests, but I did do a pass over the
other stuff. This is mostly nitty, but a few bigger questions fell out too:
* Typo: "chekc"
* Client#getStaleBound is unused
* In the proto, I'd prefer we add a new struct rather than just an int64, that
way we can add more fields later if necessary. Like how we have
RPCCallerContext, this is essentially also a form of context. Same for how we
have a CallerContext object rather than fields directly.
* Do you have any thoughts about an HDFS-specific RPC header? I doubt staleness
is useful to other users of the RPC subsystem, and really we only need it for
ClientNameNodeProtocol.
* StandbyState, we can unpack the if statements to simplify the logic.
* Please don't use a wildcard imports (e.g. ClientNNTranslatorPB, HdfsAdmin)
* DFSClient.java, more accurate javadoc would be to say "returns SyncInfo from
the..." rather than saying transaction ID.
* EditLogTailer has some unnecessary whitespace changes
* The new FSNamesystem methods, these are just for testing? It'd be better if
we did this by reaching into the Server instead, or found some way to test this
via mocking. Whitespace could also be improved.
* FSN#checkOperation, the comments don't match up anymore
* Do we really need a SyncInfo? There's a NamenodeProtocol#getTransactionID
which might be sufficient, if we also expose it to clients. The HA state only
seems to be used in a test.
* Related, in NNRpcServer, getSyncInfo is checkOperation WRITE. If this is so,
won't this always return "active"? Unless we're calling getSyncInfo on standby
NNs too?
* Also related, since getSyncInfo calls getTransactionID, which in turn checks
for superuser permissions, I don't think this will work as a normal user. Could
you add a unit test?
* ProtoUtil, RemoteEditLogManifest have some unnecessary whitespace changes
* Server: typo requet
* MethodCategory, is there no way to do this with Java annotations instead?
People are likely not going to remember to update this file when appropriate.
> Add a Client API and Proxy Provider to enable stale read from Standby
> ---------------------------------------------------------------------
>
> Key: HDFS-10702
> URL: https://issues.apache.org/jira/browse/HDFS-10702
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Jiayi Zhou
> Assignee: Jiayi Zhou
> Priority: Minor
> Attachments: HDFS-10702.001.patch, HDFS-10702.002.patch,
> HDFS-10702.003.patch, StaleReadfromStandbyNN.pdf
>
>
> Currently, clients must always talk to the active NameNode when performing
> any metadata operation, which means active NameNode could be a bottleneck for
> scalability. One way to solve this problem is to send read-only operations to
> Standby NameNode. The disadvantage is that it might be a stale read.
> Here, I'm thinking of adding a Client API to enable/disable stale read from
> Standby which gives Client the power to set the staleness restriction.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
