[jira] [Commented] (HDFS-13872) Only some protocol methods should perform msync wait
[
https://issues.apache.org/jira/browse/HDFS-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16599057#comment-16599057
]
Erik Krogen commented on HDFS-13872:
Got it... Will move discussion to HDFS-13880, thanks for the heads up
[~vagarychen]
> Only some protocol methods should perform msync wait
>
>
> Key: HDFS-13872
> URL: https://issues.apache.org/jira/browse/HDFS-13872
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HDFS-13872-HDFS-12943.000.patch
>
>
> Currently the implementation of msync added in HDFS-13767 waits until the
> server has caught up to the client-specified transaction ID regardless of
> what the inbound RPC is. This particularly causes problems for
> ObserverReadProxyProvider (see HDFS-13779) when we try to fetch the state
> from an observer/standby; this should be a quick operation, but it has to
> wait for the node to catch up to the most current state. I initially thought
> all {{HAServiceProtocol}} methods should thus be excluded from the wait
> period, but actually I think the right approach is that _only_
> {{ClientProtocol}} methods should be subjected to the wait period. I propose
> that we can do this via an annotation on client protocol which can then be
> checked within {{ipc.Server}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13872) Only some protocol methods should perform msync wait
[
https://issues.apache.org/jira/browse/HDFS-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16599035#comment-16599035
]
Chen Liang commented on HDFS-13872:
---
Somehow I missed this Jira completely...so I filed HDFS-13880 and submitted a
patch there too...Sorry my bad!
I was taking a very similar approach at the beginning, I added a method in
ReadOnly annotation to indicate whether this method should go through msync.
But then I ran into an issue, which was that ReadOnly annotation is only
applied to ClientProtocol. But when it comes down to {{ProtobufRpcEngine}}
layer, it actually changes from {{ClientProtocol}} to
{{ClientNamenodeProtocol}} and the annotation could no longer be found. And
{{ClientNamenodeProtocol}} class is actually a protobuf-generated class so we
can not annotate there...Also, chatted with Konstantin, seems a more desired
approach to do the check on server side.
So the approach I take in HDFS-13880 is that on server side when receiving a
RPC call, it looks up method name in the RPC call in ClientProtocol, if the
same name method exist, then the annotation of this method in ClientProtocol
will be used to check if msync should be bypassed.
Again, sorry I missed this Jira earlier...
> Only some protocol methods should perform msync wait
>
>
> Key: HDFS-13872
> URL: https://issues.apache.org/jira/browse/HDFS-13872
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HDFS-13872-HDFS-12943.000.patch
>
>
> Currently the implementation of msync added in HDFS-13767 waits until the
> server has caught up to the client-specified transaction ID regardless of
> what the inbound RPC is. This particularly causes problems for
> ObserverReadProxyProvider (see HDFS-13779) when we try to fetch the state
> from an observer/standby; this should be a quick operation, but it has to
> wait for the node to catch up to the most current state. I initially thought
> all {{HAServiceProtocol}} methods should thus be excluded from the wait
> period, but actually I think the right approach is that _only_
> {{ClientProtocol}} methods should be subjected to the wait period. I propose
> that we can do this via an annotation on client protocol which can then be
> checked within {{ipc.Server}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13872) Only some protocol methods should perform msync wait
[
https://issues.apache.org/jira/browse/HDFS-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598047#comment-16598047
]
Erik Krogen commented on HDFS-13872:
I like the idea of marking only certain methods. I've attached a v000 patch
which only adds the {{AlignmentContext}} if the provided method is annotated
with a new annotation, {{AlignmentContext.NeedsAlignment}}. This can't be a
part of {{ReadOnly}} since that is HDFS-specific and {{AlignmentContext}} is
generic.
v000 patch doesn't contain tests or actually annotate any methods with this
yet, just posting for comments.
> Only some protocol methods should perform msync wait
>
>
> Key: HDFS-13872
> URL: https://issues.apache.org/jira/browse/HDFS-13872
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Erik Krogen
>Priority: Major
> Attachments: HDFS-13872-HDFS-12943.000.patch
>
>
> Currently the implementation of msync added in HDFS-13767 waits until the
> server has caught up to the client-specified transaction ID regardless of
> what the inbound RPC is. This particularly causes problems for
> ObserverReadProxyProvider (see HDFS-13779) when we try to fetch the state
> from an observer/standby; this should be a quick operation, but it has to
> wait for the node to catch up to the most current state. I initially thought
> all {{HAServiceProtocol}} methods should thus be excluded from the wait
> period, but actually I think the right approach is that _only_
> {{ClientProtocol}} methods should be subjected to the wait period. I propose
> that we can do this via an annotation on client protocol which can then be
> checked within {{ipc.Server}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
