[ https://issues.apache.org/jira/browse/HDFS-13399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16464160#comment-16464160 ]
Plamen Jeliazkov commented on HDFS-13399: ----------------------------------------- Yes I propose to remove it from {{DFSClient}}. I think for now I will create a new {{ProxyProvider}} that is only used in tests and makes use of {{AlignmentContext}}. I will be able to pull it because I'll have access to the instance within my tests. Similar to how others created their own {{RpcEngine}} implementations within unit tests. This should be enough to showcase the stateId transfer. We can remove my class if we want after we have the {{StandbyReadsProxyProvider}} working. Regarding the issue about the transactionId – I want to clear up below what I am talking about: Imagine a fresh HA-enabled DFS, at transactionId 0, is initialized. A client connects and makes a single directory. We should now expect to be at transactionId 1 and expect that the client received, in the RPC response header, a stateId of 1. However this is not the case. The reason it is not the case is because HA-enabled NameNodes utilize {{FSEditLogAsync}} which updates the txid field, the field we rely on in {{FSNamesystem.getLastWrittenTransactionId}}, asynchronously from the client call. The result is that in the RPC response header the client receives they get a stateId of 0. Not 1. This is clearly incorrect. We do not want a client to connect to a NameNode that is behind in state. Clearly this is just a race condition but it has already appeared in my unit tests. One idea is to modify {{FSEditLogAsync}} like so: {code:java} @Override long getLastWrittenTxIdWithoutLock() { return super.getLastWrittenTxIdWithoutLock() + editPendingQ.size() + syncWaitQ.size(); } {code} However I am unsure if this would be correct / safe to do. Input from others would be desired here. > Make Client field AlignmentContext non-static. > ---------------------------------------------- > > Key: HDFS-13399 > URL: https://issues.apache.org/jira/browse/HDFS-13399 > Project: Hadoop HDFS > Issue Type: Sub-task > Affects Versions: HDFS-12943 > Reporter: Plamen Jeliazkov > Assignee: Plamen Jeliazkov > Priority: Major > Attachments: HDFS-13399-HDFS-12943.000.patch, > HDFS-13399-HDFS-12943.001.patch, HDFS-13399-HDFS-12943.002.patch, > HDFS-13399-HDFS-12943.003.patch, HDFS-13399-HDFS-12943.004.patch, > HDFS-13399-HDFS-12943.005.patch, HDFS-13399-HDFS-12943.006.patch > > > In HDFS-12977, DFSClient's constructor was altered to make use of a new > static method in Client that allowed one to set an AlignmentContext. This > work is to remove that static field and make each DFSClient pass it's > AlignmentContext down to the proxy Call level. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org