[
https://issues.apache.org/jira/browse/HDFS-13767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556255#comment-16556255
]
Chen Liang commented on HDFS-13767:
-----------------------------------
Post WIP.001 patch for early reviews.
Some notes to ease the review:
The idea of the patch is to have Observer Handler threads defer processing a
call until it's state id has caught up. This applies to every call to Observer.
The key goals are:
1. the calls from the same client should maintain order,
2. the interference of calls from different client should be minimized.
To do this, the WIP.001 patch introduces a per client deferred queue, created
and destroyed at runtime.
For example:
Say we have call with id 1, 2, A, B, 3 in the queue, where 1, 2, 3 are from
one client, A, B are from another. Now 1 does not have state id ready. it gets
added to client specific deferred call queue, making it [1], and call 1 gets
added back to call queue, making it 2, A, B, 3, 1.
Now say when it comes to 2, the id has caught up. But 2 can nothandled because
1 is not handled yet. This is identified by
[1]'s head is 1 != 2. So 2 is also added back to call queue, making it A, B,
3, 1, 2. id 2 is added to the client deferred queue, making it [1, 2].
A, B are from a different client, so they are handled separately. Let's say
they both have state id caught up, so they are handled. Now call queue is 3, 1,
2.
Now, say another call 4, gets added, and the call queue becomes 3, 1, 2, 4, all
from same client.
For 3, if state id hasn't caught up, it would be same case as 1, if state id
has caught up, it would be the same case as 2. Either way, it gets added to
call queue again, making it 1, 2, 4, 3; as well as client deferred queue,
making it [1, 2, 3]
Now call 1 gets picked up again, which is the same as the head client deferred
queue. So it gets processed because state id has caught up, and 1 is removed
from client deferred queue, now client's deferred call queue is [2, 3].
The same process applies to 2. So the call queue becomes 4, 3, and the deferred
call queue is [3].
Then call 4 gets handled, since it does not match the head of deferred call
queue (which is 3), it gets added to deferred queue, making [3, 4]. And gets
inserted back to call queue, making it 3,4.
Then call 3 gets picked up, and it gets processed, because it matches the head
of deferred queue. Later, the same applies to call 4. All calls are handled at
this point.
> Add msync server implementation.
> --------------------------------
>
> Key: HDFS-13767
> URL: https://issues.apache.org/jira/browse/HDFS-13767
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Reporter: Chen Liang
> Assignee: Chen Liang
> Priority: Major
> Attachments: HDFS-13767.WIP.001.patch
>
>
> This is a followup on HDFS-13688, where msync API is introduced to
> {{ClientProtocol}} but the server side implementation is missing. This is
> Jira is to implement the server side logic.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
