[ https://issues.apache.org/jira/browse/HDFS-13767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16568674#comment-16568674 ]
Plamen Jeliazkov edited comment on HDFS-13767 at 8/3/18 7:57 PM: ----------------------------------------------------------------- Hey Chen, Change looks pretty good so far; couple things. (1) I like the idea of splitting AlignmentContext into client and server interfaces. I think originally Konstantin wanted just a single interface but I think its clear there is a difference. I would not mind picking up that JIRA if we want to go ahead with it. This would let us allow the server-side version of AlignmentContext to better handle defering / re-queue'ing calls. (2) A concern about the unit test -- I tried to add a 10 second sleep between the thread start and the assert: {code:java} assertFalse(readSucceed.get()); {code} However I found that the test would fail if I did. Which makes me question the unit test. I would expect the 'getFileStatus' to basically hang until the _rollEditLogAndTail_ call but seems it is updating anyway. I tried also stopping the _EditLogTailer_ on the ObserverNode but it still updated anyway. Please let me know if I am missing something about this test; I will look further. (3) So far these patches don't actually implement _NameNodeRpcServer.msync()_. Should we rename it? (4) [~xkrogen], [~shv], and others -- I was curious about making msync pause at the server level. What if we had msync hang at the client side instead? This way we aren't impacting Observer queues with many deferred calls. We could simply have the client check up with the Observers' states and so rather than re-queue'ing we throw a StandbyException (or something) and force the client to maybe pause and retry the msync call on the same or a different Observer. If the msync call succeeds then we know we can use the particular Observer that it succeeded on. Thoughts? was (Author: zero45): Hey Chen, Change looks pretty good so far; couple things. (1) I like the idea of splitting AlignmentContext into client and server interfaces. I think originally Konstantin wanted just a single interface but I think its clear there is a difference. I would not mind picking up that JIRA if we want to go ahead with it. This would let us allow the server-side version of AlignmentContext to better handle defering / re-queue'ing calls. (2) A concern about the unit test -- I tried to add a 10 second sleep between the thread start and the assert: {code:java} assertFalse(readSucceed.get()); {code} However I found that the test would fail if I did. Which makes me question the unit test. I would expect the 'getFileStatus' to basically hang until the _rollEditLogAndTail_ call but seems it is updating anyway. I tried also stopping the _EditLogTailer_ on the ObserverNode but it still updated anyway. Please let me know if I am missing something about this test; I will look further. (3) So far these patches don't actually implement _NameNodeRpcServer.msync()_. Should we rename it? (4) [~xkrogen], [~shv], and others -- I was curios about making msync pause at the server level. What if we had msync hang at the client side instead? This way we aren't impacting Observer queues with many deferred calls. We could simply have the client check up with the Observers' states and so rather than re-queue'ing we throw a StandbyException (or something) and force the client to maybe pause and retry the msync call on the same or a different Observer. If the msync call succeeds then we know we can use the particular Observer that it succeeded on. Thoughts? > Add msync server implementation. > -------------------------------- > > Key: HDFS-13767 > URL: https://issues.apache.org/jira/browse/HDFS-13767 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: namenode > Reporter: Chen Liang > Assignee: Chen Liang > Priority: Major > Attachments: HDFS-13767-HDFS-12943.001.patch, > HDFS-13767-HDFS-12943.002.patch, HDFS-13767.WIP.001.patch, > HDFS-13767.WIP.002.patch, HDFS-13767.WIP.003.patch, HDFS-13767.WIP.004.patch > > > This is a followup on HDFS-13688, where msync API is introduced to > {{ClientProtocol}} but the server side implementation is missing. This is > Jira is to implement the server side logic. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org