[jira] [Comment Edited] (HDFS-12976) Introduce ObserverReadProxyProvider
[
https://issues.apache.org/jira/browse/HDFS-12976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525574#comment-16525574
]
Chao Sun edited comment on HDFS-12976 at 6/27/18 8:28 PM:
--
Thanks for the comments [~xkrogen] and [~shv]!
[~xkrogen]: yes we can potentially use {{stateId}} to achieve the same purpose
- totally agree with you on the concern about adding another flag which
potentially is only used for talking to NameNode.
[~shv]: I like the idea of {{T extends HAServiceProtocol}}, however it may need
more change as {{NameNodeProxiesClient#createProxyWithClientProtocol}} only
create the {{T}} with {{ClientProtocol}}, which may cause exception like:
{code}
Caused by: java.lang.ClassCastException:
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB cannot be
cast to org.apache.hadoop.ha.HAServiceProtocol
{code}
Sorry I totally forgot about the alignment changes. Thanks for doing that.
Attached the v004 patch after rebasing to the latest HDFS-12943.
was (Author: csun):
Thanks for the comments [~xkrogen] and [~shv]!
[~xkrogen]: yes we can potentially use {{stateId}} to achieve the same purpose
- totally agree with you on the concern about adding another flag which
potentially is only used for talking to NameNode.
[~shv]: I like the idea of {{T extends HAServiceProtocol}}, however it may need
more change as {{NameNodeProxiesClient#createProxyWithClientProtocol}} only
create the {{T}} with {{ClientProtocol}}. Otherwise you may get exception like:
{code}
Caused by: java.lang.ClassCastException:
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB cannot be
cast to org.apache.hadoop.ha.HAServiceProtocol
{code}
Sorry I totally forgot about the alignment changes. Thanks for doing that.
Attached the v004 patch after rebasing to the latest HDFS-12943.
> Introduce ObserverReadProxyProvider
> ---
>
> Key: HDFS-12976
> URL: https://issues.apache.org/jira/browse/HDFS-12976
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Reporter: Konstantin Shvachko
>Assignee: Chao Sun
>Priority: Major
> Attachments: HDFS-12976-HDFS-12943.000.patch,
> HDFS-12976-HDFS-12943.001.patch, HDFS-12976-HDFS-12943.002.patch,
> HDFS-12976-HDFS-12943.003.patch, HDFS-12976-HDFS-12943.004.patch,
> HDFS-12976-HDFS-12943.005.patch, HDFS-12976.WIP.patch
>
>
> {{StandbyReadProxyProvider}} should implement {{FailoverProxyProvider}}
> interface and be able to submit read requests to ANN and SBN(s).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-12976) Introduce ObserverReadProxyProvider
[
https://issues.apache.org/jira/browse/HDFS-12976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522689#comment-16522689
]
Chao Sun edited comment on HDFS-12976 at 6/25/18 7:03 PM:
--
Another idea I'm thinking is to add a flag in {{RpcRequestHeaderProto}} and
then check this flag on the NameNode server side, using {{AlignmentContext}}.
With this approach, no change on {{ConfiguredFailoverProxyProvider}} will be
required. Let me know your opinion on this.
was (Author: csun):
Another thought I'm thinking is to add a flag in {{RpcRequestHeaderProto}} and
then check this flag on the NameNode server side, using {{AlignmentContext}}.
With this approach, no change on {{ConfiguredFailoverProxyProvider}} will be
required. Let me know your opinion on this.
> Introduce ObserverReadProxyProvider
> ---
>
> Key: HDFS-12976
> URL: https://issues.apache.org/jira/browse/HDFS-12976
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Reporter: Konstantin Shvachko
>Assignee: Chao Sun
>Priority: Major
> Attachments: HDFS-12976-HDFS-12943.000.patch,
> HDFS-12976-HDFS-12943.001.patch, HDFS-12976-HDFS-12943.002.patch,
> HDFS-12976-HDFS-12943.003.patch, HDFS-12976.WIP.patch
>
>
> {{StandbyReadProxyProvider}} should implement {{FailoverProxyProvider}}
> interface and be able to submit read requests to ANN and SBN(s).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-12976) Introduce ObserverReadProxyProvider
[
https://issues.apache.org/jira/browse/HDFS-12976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514055#comment-16514055
]
Erik Krogen edited comment on HDFS-12976 at 6/15/18 4:31 PM:
-
Hey [~csun], I don't see why the changes to
{{ConfiguredFailoverProxyProvider#performFailover()}} are necessary. The intent
as I understand it is that you increment the proxy index, then try it, and if
the proxy is in the wrong state it will throw an exception and
{{performFailover()}} will be called again. This process should work fine
without the {{isObserverState}} check.
was (Author: xkrogen):
Hey [~csun], I don't see why the changes to
{{ConfiguredFailoverProxyProvider#performFailover()}} are necessary. The intent
is that you increment the proxy index, then try it, and if the proxy is in the
wrong state it will throw an exception and {{performFailover()}} will be called
again. This process should work fine without the {{isObserverState}} check.
> Introduce ObserverReadProxyProvider
> ---
>
> Key: HDFS-12976
> URL: https://issues.apache.org/jira/browse/HDFS-12976
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Reporter: Konstantin Shvachko
>Assignee: Chao Sun
>Priority: Major
> Attachments: HDFS-12976-HDFS-12943.000.patch,
> HDFS-12976-HDFS-12943.001.patch, HDFS-12976-HDFS-12943.002.patch,
> HDFS-12976-HDFS-12943.003.patch, HDFS-12976.WIP.patch
>
>
> {{StandbyReadProxyProvider}} should implement {{FailoverProxyProvider}}
> interface and be able to submit read requests to ANN and SBN(s).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-12976) Introduce ObserverReadProxyProvider
[
https://issues.apache.org/jira/browse/HDFS-12976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16510679#comment-16510679
]
Chao Sun edited comment on HDFS-12976 at 6/13/18 6:27 AM:
--
Thanks [~shv]. Addressed the issues (should have enabled the parameterized
class check in my IDE..) and uploaded patch v2.
Do you know why the jenkins wasn't triggered for my previous patches?
was (Author: csun):
Thanks [~shv]. Addressed the issues (should have enabled the parameterized
class check in my IDE..) and uploaded patch v2.
> Introduce ObserverReadProxyProvider
> ---
>
> Key: HDFS-12976
> URL: https://issues.apache.org/jira/browse/HDFS-12976
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Reporter: Konstantin Shvachko
>Assignee: Chao Sun
>Priority: Major
> Attachments: HDFS-12976-HDFS-12943.000.patch,
> HDFS-12976-HDFS-12943.001.patch, HDFS-12976-HDFS-12943.002.patch,
> HDFS-12976.WIP.patch
>
>
> {{StandbyReadProxyProvider}} should implement {{FailoverProxyProvider}}
> interface and be able to submit read requests to ANN and SBN(s).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-12976) Introduce ObserverReadProxyProvider
[
https://issues.apache.org/jira/browse/HDFS-12976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16510679#comment-16510679
]
Chao Sun edited comment on HDFS-12976 at 6/13/18 6:27 AM:
--
Thanks [~shv]. Addressed the issues (should have enabled the parameterized
class check in my IDE..) and uploaded patch v2.
Do you know why the jenkins wasn't triggered for my previous patches?
was (Author: csun):
Thanks [~shv]. Addressed the issues (should have enabled the parameterized
class check in my IDE..) and uploaded patch v2.
Do you know why the jenkins wasn't triggered for my previous patches?
> Introduce ObserverReadProxyProvider
> ---
>
> Key: HDFS-12976
> URL: https://issues.apache.org/jira/browse/HDFS-12976
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Reporter: Konstantin Shvachko
>Assignee: Chao Sun
>Priority: Major
> Attachments: HDFS-12976-HDFS-12943.000.patch,
> HDFS-12976-HDFS-12943.001.patch, HDFS-12976-HDFS-12943.002.patch,
> HDFS-12976.WIP.patch
>
>
> {{StandbyReadProxyProvider}} should implement {{FailoverProxyProvider}}
> interface and be able to submit read requests to ANN and SBN(s).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-12976) Introduce ObserverReadProxyProvider
[
https://issues.apache.org/jira/browse/HDFS-12976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508270#comment-16508270
]
Erik Krogen edited comment on HDFS-12976 at 6/11/18 4:11 PM:
-
[~vagarychen]:
{quote}
The thing is that, the current patch looks at ReadOnly to determine if a call
should go to observer, meaning if a method is annotated with ReadOnly, it
always go to Observer, is this the case? Because there may be situations where
we want to make a read call to active, which becomes impossible. I guess a more
ideal way would probably be having new annotations for observer operation and
active options, which is also for better readability.
{quote}
The ReadOnly annotation added in HDFS-13578 supports a flag {{activeOnly}} to
indicate that, although it is a read operation, it should only be served by the
active. Does this meet your requirement or are you saying there should be a way
to dynamically decide?
was (Author: xkrogen):
{quote}
The thing is that, the current patch looks at ReadOnly to determine if a call
should go to observer, meaning if a method is annotated with ReadOnly, it
always go to Observer, is this the case? Because there may be situations where
we want to make a read call to active, which becomes impossible. I guess a more
ideal way would probably be having new annotations for observer operation and
active options, which is also for better readability.
{quote}
The ReadOnly annotation added in HDFS-13578 supports a flag {{activeOnly}} to
indicate that, although it is a read operation, it should only be served by the
active. Does this meet your requirement or are you saying there should be a way
to dynamically decide?
> Introduce ObserverReadProxyProvider
> ---
>
> Key: HDFS-12976
> URL: https://issues.apache.org/jira/browse/HDFS-12976
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Reporter: Konstantin Shvachko
>Assignee: Chao Sun
>Priority: Major
> Attachments: HDFS-12976-HDFS-12943.000.patch, HDFS-12976.WIP.patch
>
>
> {{StandbyReadProxyProvider}} should implement {{FailoverProxyProvider}}
> interface and be able to submit read requests to ANN and SBN(s).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
