[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[
https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815039#comment-17815039
]
ASF GitHub Bot commented on HDFS-17362:
---
simbadzina commented on code in PR #6510:
URL: https://github.com/apache/hadoop/pull/6510#discussion_r1480692356
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/RouterObserverReadProxyProvider.java:
##
@@ -84,7 +84,8 @@ public class RouterObserverReadProxyProvider extends
AbstractNNFailoverProxyP
public RouterObserverReadProxyProvider(Configuration conf, URI uri, Class
xface,
HAProxyFactory factory) {
-this(conf, uri, xface, factory, new IPFailoverProxyProvider<>(conf, uri,
xface, factory));
+this(conf, uri, xface, factory,
+new ConfiguredFailoverProxyProvider<>(conf, uri, xface, factory));
Review Comment:
Cool. That sounds good to me.
> RBF: RouterObserverReadProxyProvider should use
> ConfiguredFailoverProxyProvider internally
> --
>
> Key: HDFS-17362
> URL: https://issues.apache.org/jira/browse/HDFS-17362
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Takanobu Asanuma
>Assignee: Takanobu Asanuma
>Priority: Major
> Labels: pull-request-available
>
> Currently, RouterObserverReadProxyProvider is using IPFailoverProxyProvider,
> while ObserverReadProxyProvider is using ConfiguredFailoverProxyProvider. If
> we are to align RouterObserverReadProxyProvider with
> ObserverReadProxyProvider, RouterObserverReadProxyProvider should internally
> use ConfiguredFailoverProxyProvider. Moreover, IPFailoverProxyProvider has
> an issue with resolving HA configurations. (For example,
> IPFailoverProxyProvider cannot resolve hdfs://router-service.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[
https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17814863#comment-17814863
]
ASF GitHub Bot commented on HDFS-17362:
---
hadoop-yetus commented on PR #6510:
URL: https://github.com/apache/hadoop/pull/6510#issuecomment-1930035843
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 34s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 25s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 30m 57s | | trunk passed |
| +1 :green_heart: | compile | 5m 24s | | trunk passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 5m 7s | | trunk passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 22s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 36s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 26s | | trunk passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 13s | | trunk passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 52s | | trunk passed |
| +1 :green_heart: | shadedclient | 32m 32s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 20s | | the patch passed |
| +1 :green_heart: | compile | 5m 17s | | the patch passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 5m 17s | | the patch passed |
| +1 :green_heart: | compile | 5m 2s | | the patch passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 5m 2s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 12s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 22s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 6s | | the patch passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 0s | | the patch passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 53s | | the patch passed |
| +1 :green_heart: | shadedclient | 32m 42s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 27s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | unit | 13m 49s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not
generate ASF License warnings. |
| | | 173m 40s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.44 ServerAPI=1.44 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6510/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6510 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 746ecb197a58 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / aca8c4ec079570299ae30f3bdf3e443b8a5ff22e |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6510/6/testReport/ |
| Max. process+thread count | 3009 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client
hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project |
| Console output |
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[
https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17814754#comment-17814754
]
ASF GitHub Bot commented on HDFS-17362:
---
hadoop-yetus commented on PR #6510:
URL: https://github.com/apache/hadoop/pull/6510#issuecomment-1929344958
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 32s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 30m 43s | | trunk passed |
| +1 :green_heart: | compile | 5m 27s | | trunk passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 5m 12s | | trunk passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 23s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 25s | | trunk passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 15s | | trunk passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 53s | | trunk passed |
| +1 :green_heart: | shadedclient | 32m 59s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 33m 20s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 19s | | the patch passed |
| +1 :green_heart: | compile | 5m 15s | | the patch passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 5m 15s | | the patch passed |
| +1 :green_heart: | compile | 5m 5s | | the patch passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 5m 5s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 12s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 22s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 5s | | the patch passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 0s | | the patch passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 57s | | the patch passed |
| +1 :green_heart: | shadedclient | 32m 32s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 26s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | unit | 13m 49s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 173m 55s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.44 ServerAPI=1.44 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6510/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6510 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux a93b561fa761 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 279c68805e8e9da5467d7e981d2cebc5ae53ec79 |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6510/5/testReport/ |
| Max. process+thread count | 2988 (vs. ulimit of 5500) |
| modules | C:
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[
https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17814741#comment-17814741
]
ASF GitHub Bot commented on HDFS-17362:
---
hadoop-yetus commented on PR #6510:
URL: https://github.com/apache/hadoop/pull/6510#issuecomment-1929328194
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 33s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 13s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 30m 56s | | trunk passed |
| +1 :green_heart: | compile | 5m 25s | | trunk passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 5m 10s | | trunk passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 23s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 26s | | trunk passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 13s | | trunk passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 55s | | trunk passed |
| +1 :green_heart: | shadedclient | 32m 45s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 33m 6s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 32s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 21s | | the patch passed |
| +1 :green_heart: | compile | 5m 16s | | the patch passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 5m 16s | | the patch passed |
| +1 :green_heart: | compile | 5m 4s | | the patch passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 5m 4s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 12s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 22s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 4s | | the patch passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 0s | | the patch passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 59s | | the patch passed |
| +1 :green_heart: | shadedclient | 32m 44s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 27s | | hadoop-hdfs-client in the patch
passed. |
| -1 :x: | unit | 13m 53s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6510/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 173m 56s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.hdfs.server.federation.router.TestObserverWithRouter |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.44 ServerAPI=1.44 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6510/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6510 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 28e884691938 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 543645c16730c0faf66d0d65dbab28299815f87a |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions |
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[
https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17814683#comment-17814683
]
ASF GitHub Bot commented on HDFS-17362:
---
tasanuma commented on code in PR #6510:
URL: https://github.com/apache/hadoop/pull/6510#discussion_r1479405963
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/RouterObserverReadProxyProvider.java:
##
@@ -84,7 +84,8 @@ public class RouterObserverReadProxyProvider extends
AbstractNNFailoverProxyP
public RouterObserverReadProxyProvider(Configuration conf, URI uri, Class
xface,
HAProxyFactory factory) {
-this(conf, uri, xface, factory, new IPFailoverProxyProvider<>(conf, uri,
xface, factory));
+this(conf, uri, xface, factory,
+new ConfiguredFailoverProxyProvider<>(conf, uri, xface, factory));
Review Comment:
I have implemented `RouterObserverReadConfiguredFailoverProxyProvider` and
updated `TestObserverWithRouter` accordingly.
> RBF: RouterObserverReadProxyProvider should use
> ConfiguredFailoverProxyProvider internally
> --
>
> Key: HDFS-17362
> URL: https://issues.apache.org/jira/browse/HDFS-17362
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Takanobu Asanuma
>Assignee: Takanobu Asanuma
>Priority: Major
> Labels: pull-request-available
>
> Currently, RouterObserverReadProxyProvider is using IPFailoverProxyProvider,
> while ObserverReadProxyProvider is using ConfiguredFailoverProxyProvider. If
> we are to align RouterObserverReadProxyProvider with
> ObserverReadProxyProvider, RouterObserverReadProxyProvider should internally
> use ConfiguredFailoverProxyProvider. Moreover, IPFailoverProxyProvider has
> an issue with resolving HA configurations. (For example,
> IPFailoverProxyProvider cannot resolve hdfs://router-service.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[
https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17814577#comment-17814577
]
ASF GitHub Bot commented on HDFS-17362:
---
tasanuma commented on code in PR #6510:
URL: https://github.com/apache/hadoop/pull/6510#discussion_r1479162410
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/RouterObserverReadProxyProvider.java:
##
@@ -84,7 +84,8 @@ public class RouterObserverReadProxyProvider extends
AbstractNNFailoverProxyP
public RouterObserverReadProxyProvider(Configuration conf, URI uri, Class
xface,
HAProxyFactory factory) {
-this(conf, uri, xface, factory, new IPFailoverProxyProvider<>(conf, uri,
xface, factory));
+this(conf, uri, xface, factory,
+new ConfiguredFailoverProxyProvider<>(conf, uri, xface, factory));
Review Comment:
I'd prefer to create a new class. I'd like to keep the existing class as is
and create a new class named
`RouterObserverReadConfiguredFailoverProxyProvider` using
`ConfiguredFailoverProxyProvider` internally. I will update the PR soon.
> RBF: RouterObserverReadProxyProvider should use
> ConfiguredFailoverProxyProvider internally
> --
>
> Key: HDFS-17362
> URL: https://issues.apache.org/jira/browse/HDFS-17362
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Takanobu Asanuma
>Assignee: Takanobu Asanuma
>Priority: Major
> Labels: pull-request-available
>
> Currently, RouterObserverReadProxyProvider is using IPFailoverProxyProvider,
> while ObserverReadProxyProvider is using ConfiguredFailoverProxyProvider. If
> we are to align RouterObserverReadProxyProvider with
> ObserverReadProxyProvider, RouterObserverReadProxyProvider should internally
> use ConfiguredFailoverProxyProvider. Moreover, IPFailoverProxyProvider has
> an issue with resolving HA configurations. (For example,
> IPFailoverProxyProvider cannot resolve hdfs://router-service.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[
https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17814518#comment-17814518
]
ASF GitHub Bot commented on HDFS-17362:
---
simbadzina commented on code in PR #6510:
URL: https://github.com/apache/hadoop/pull/6510#discussion_r1478942086
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/RouterObserverReadProxyProvider.java:
##
@@ -84,7 +84,8 @@ public class RouterObserverReadProxyProvider extends
AbstractNNFailoverProxyP
public RouterObserverReadProxyProvider(Configuration conf, URI uri, Class
xface,
HAProxyFactory factory) {
-this(conf, uri, xface, factory, new IPFailoverProxyProvider<>(conf, uri,
xface, factory));
+this(conf, uri, xface, factory,
+new ConfiguredFailoverProxyProvider<>(conf, uri, xface, factory));
Review Comment:
Yes, I was suggesting adding a new parameter, like
`dfs.client.failover.router.internal.proxy.provider` you named.
Creating a new class is also a good solution. I'm a bit worried though about
the update story for clients who are already using the existing class.
The new parameter approach makes the update backward compatible with
existing client configs.
> RBF: RouterObserverReadProxyProvider should use
> ConfiguredFailoverProxyProvider internally
> --
>
> Key: HDFS-17362
> URL: https://issues.apache.org/jira/browse/HDFS-17362
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Takanobu Asanuma
>Assignee: Takanobu Asanuma
>Priority: Major
> Labels: pull-request-available
>
> Currently, RouterObserverReadProxyProvider is using IPFailoverProxyProvider,
> while ObserverReadProxyProvider is using ConfiguredFailoverProxyProvider. If
> we are to align RouterObserverReadProxyProvider with
> ObserverReadProxyProvider, RouterObserverReadProxyProvider should internally
> use ConfiguredFailoverProxyProvider. Moreover, IPFailoverProxyProvider has
> an issue with resolving HA configurations. (For example,
> IPFailoverProxyProvider cannot resolve hdfs://router-service.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[ https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17813684#comment-17813684 ] ASF GitHub Bot commented on HDFS-17362: --- tasanuma commented on PR #6510: URL: https://github.com/apache/hadoop/pull/6510#issuecomment-1923910581 There is an issue that we cannot run unit tests in TestObserverWithRouter using `mvn test -Dtest=TestObserverWithRouter`. I have created a PR to fix it in #6522. I would appreciate it if you could also review it when you have time. > RBF: RouterObserverReadProxyProvider should use > ConfiguredFailoverProxyProvider internally > -- > > Key: HDFS-17362 > URL: https://issues.apache.org/jira/browse/HDFS-17362 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Major > Labels: pull-request-available > > Currently, RouterObserverReadProxyProvider is using IPFailoverProxyProvider, > while ObserverReadProxyProvider is using ConfiguredFailoverProxyProvider. If > we are to align RouterObserverReadProxyProvider with > ObserverReadProxyProvider, RouterObserverReadProxyProvider should internally > use ConfiguredFailoverProxyProvider. Moreover, IPFailoverProxyProvider has > an issue with resolving HA configurations. (For example, > IPFailoverProxyProvider cannot resolve hdfs://router-service.) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[
https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17813673#comment-17813673
]
ASF GitHub Bot commented on HDFS-17362:
---
tasanuma commented on code in PR #6510:
URL: https://github.com/apache/hadoop/pull/6510#discussion_r1476061151
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/RouterObserverReadProxyProvider.java:
##
@@ -84,7 +84,8 @@ public class RouterObserverReadProxyProvider extends
AbstractNNFailoverProxyP
public RouterObserverReadProxyProvider(Configuration conf, URI uri, Class
xface,
HAProxyFactory factory) {
-this(conf, uri, xface, factory, new IPFailoverProxyProvider<>(conf, uri,
xface, factory));
+this(conf, uri, xface, factory,
+new ConfiguredFailoverProxyProvider<>(conf, uri, xface, factory));
Review Comment:
As an alternative approach, what if we created a
`RouterObserverReadProxyProviderWithIPFailover` using `IPFailoverProxyProvider`
to align it with `ObserverReadProxyProviderWithIPFailover`?
```java
public class RouterObserverReadProxyProviderWithIPFailover
extends RouterObserverReadProxyProvider {
@VisibleForTesting
static final Logger LOG =
LoggerFactory.getLogger(RouterObserverReadProxyProviderWithIPFailover.class);
public RouterObserverReadProxyProviderWithIPFailover(Configuration conf,
URI uri, Class xface,
HAProxyFactory factory) {
super(conf, uri, xface, factory, new IPFailoverProxyProvider<>(conf,
uri, xface, factory));
}
}
```
> RBF: RouterObserverReadProxyProvider should use
> ConfiguredFailoverProxyProvider internally
> --
>
> Key: HDFS-17362
> URL: https://issues.apache.org/jira/browse/HDFS-17362
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Takanobu Asanuma
>Assignee: Takanobu Asanuma
>Priority: Major
> Labels: pull-request-available
>
> Currently, RouterObserverReadProxyProvider is using IPFailoverProxyProvider,
> while ObserverReadProxyProvider is using ConfiguredFailoverProxyProvider. If
> we are to align RouterObserverReadProxyProvider with
> ObserverReadProxyProvider, RouterObserverReadProxyProvider should internally
> use ConfiguredFailoverProxyProvider. Moreover, IPFailoverProxyProvider has
> an issue with resolving HA configurations. (For example,
> IPFailoverProxyProvider cannot resolve hdfs://router-service.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[
https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17813671#comment-17813671
]
ASF GitHub Bot commented on HDFS-17362:
---
tasanuma commented on code in PR #6510:
URL: https://github.com/apache/hadoop/pull/6510#discussion_r1476059897
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/RouterObserverReadProxyProvider.java:
##
@@ -84,7 +84,8 @@ public class RouterObserverReadProxyProvider extends
AbstractNNFailoverProxyP
public RouterObserverReadProxyProvider(Configuration conf, URI uri, Class
xface,
HAProxyFactory factory) {
-this(conf, uri, xface, factory, new IPFailoverProxyProvider<>(conf, uri,
xface, factory));
+this(conf, uri, xface, factory,
+new ConfiguredFailoverProxyProvider<>(conf, uri, xface, factory));
Review Comment:
@simbadzina Thanks for your review and sharing your thoughts. I understand
that you intentionally chose `IPFailoverProxyProvider`.
> Can we make the proxy provider configurable? To support both usecases.
Are you suggesting that we create another configuration like
`dfs.client.failover.router.internal.proxy.provider`?
> RBF: RouterObserverReadProxyProvider should use
> ConfiguredFailoverProxyProvider internally
> --
>
> Key: HDFS-17362
> URL: https://issues.apache.org/jira/browse/HDFS-17362
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Takanobu Asanuma
>Assignee: Takanobu Asanuma
>Priority: Major
> Labels: pull-request-available
>
> Currently, RouterObserverReadProxyProvider is using IPFailoverProxyProvider,
> while ObserverReadProxyProvider is using ConfiguredFailoverProxyProvider. If
> we are to align RouterObserverReadProxyProvider with
> ObserverReadProxyProvider, RouterObserverReadProxyProvider should internally
> use ConfiguredFailoverProxyProvider. Moreover, IPFailoverProxyProvider has
> an issue with resolving HA configurations. (For example,
> IPFailoverProxyProvider cannot resolve hdfs://router-service.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[
https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17813553#comment-17813553
]
ASF GitHub Bot commented on HDFS-17362:
---
simbadzina commented on code in PR #6510:
URL: https://github.com/apache/hadoop/pull/6510#discussion_r1475695932
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/RouterObserverReadProxyProvider.java:
##
@@ -84,7 +84,8 @@ public class RouterObserverReadProxyProvider extends
AbstractNNFailoverProxyP
public RouterObserverReadProxyProvider(Configuration conf, URI uri, Class
xface,
HAProxyFactory factory) {
-this(conf, uri, xface, factory, new IPFailoverProxyProvider<>(conf, uri,
xface, factory));
+this(conf, uri, xface, factory,
+new ConfiguredFailoverProxyProvider<>(conf, uri, xface, factory));
Review Comment:
For a different internal proxyprovider we have code like below.
```
this(conf, uri, xface, proxyFactory, getFailoverProxyProvider(conf, uri,
xface, proxyFactory));
```
and
```
@VisibleForTesting
protected static AbstractNNFailoverProxyProvider
getFailoverProxyProvider(
Configuration conf, URI uri, Class xface, HAProxyFactory
proxyFactory) {
String configKey = FAILOVER_PROXY_PROVIDER + "." + uri.getHost();
Class> fppClass =
(Class>) conf.getClass(configKey,
(Class)
DEFAULT_FAILOVER_PROXY_PROVIDER,
AbstractNNFailoverProxyProvider.class);
try {
Constructor> ctor =
fppClass.getConstructor(
Configuration.class, URI.class, Class.class, HAProxyFactory.class);
return ctor.newInstance(conf, uri, xface, proxyFactory);
} catch (Exception e) {
LOG.error("Failed to create failover proxy provider", e);
throw new IllegalArgumentException(e);
}
}
```
> RBF: RouterObserverReadProxyProvider should use
> ConfiguredFailoverProxyProvider internally
> --
>
> Key: HDFS-17362
> URL: https://issues.apache.org/jira/browse/HDFS-17362
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Takanobu Asanuma
>Assignee: Takanobu Asanuma
>Priority: Major
> Labels: pull-request-available
>
> Currently, RouterObserverReadProxyProvider is using IPFailoverProxyProvider,
> while ObserverReadProxyProvider is using ConfiguredFailoverProxyProvider. If
> we are to align RouterObserverReadProxyProvider with
> ObserverReadProxyProvider, RouterObserverReadProxyProvider should internally
> use ConfiguredFailoverProxyProvider. Moreover, IPFailoverProxyProvider has
> an issue with resolving HA configurations. (For example,
> IPFailoverProxyProvider cannot resolve hdfs://router-service.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[
https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17813548#comment-17813548
]
ASF GitHub Bot commented on HDFS-17362:
---
simbadzina commented on code in PR #6510:
URL: https://github.com/apache/hadoop/pull/6510#discussion_r1475691426
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/RouterObserverReadProxyProvider.java:
##
@@ -84,7 +84,8 @@ public class RouterObserverReadProxyProvider extends
AbstractNNFailoverProxyP
public RouterObserverReadProxyProvider(Configuration conf, URI uri, Class
xface,
HAProxyFactory factory) {
-this(conf, uri, xface, factory, new IPFailoverProxyProvider<>(conf, uri,
xface, factory));
+this(conf, uri, xface, factory,
+new ConfiguredFailoverProxyProvider<>(conf, uri, xface, factory));
Review Comment:
~Actually, `IPFailoverProxyProvider` can still be set on
`routerContext.getFileSystemURI().toString()` even when we use the
ConfiguredFailoverProxyProvider.
We should have a test case for this.~
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/RouterObserverReadProxyProvider.java:
##
@@ -84,7 +84,8 @@ public class RouterObserverReadProxyProvider extends
AbstractNNFailoverProxyP
public RouterObserverReadProxyProvider(Configuration conf, URI uri, Class
xface,
HAProxyFactory factory) {
-this(conf, uri, xface, factory, new IPFailoverProxyProvider<>(conf, uri,
xface, factory));
+this(conf, uri, xface, factory,
+new ConfiguredFailoverProxyProvider<>(conf, uri, xface, factory));
Review Comment:
~Actually, `IPFailoverProxyProvider` can still be set on
`routerContext.getFileSystemURI().toString()` even when we use the
ConfiguredFailoverProxyProvider.
We should have a test case for this.~
> RBF: RouterObserverReadProxyProvider should use
> ConfiguredFailoverProxyProvider internally
> --
>
> Key: HDFS-17362
> URL: https://issues.apache.org/jira/browse/HDFS-17362
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Takanobu Asanuma
>Assignee: Takanobu Asanuma
>Priority: Major
> Labels: pull-request-available
>
> Currently, RouterObserverReadProxyProvider is using IPFailoverProxyProvider,
> while ObserverReadProxyProvider is using ConfiguredFailoverProxyProvider. If
> we are to align RouterObserverReadProxyProvider with
> ObserverReadProxyProvider, RouterObserverReadProxyProvider should internally
> use ConfiguredFailoverProxyProvider. Moreover, IPFailoverProxyProvider has
> an issue with resolving HA configurations. (For example,
> IPFailoverProxyProvider cannot resolve hdfs://router-service.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[
https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17813545#comment-17813545
]
ASF GitHub Bot commented on HDFS-17362:
---
simbadzina commented on code in PR #6510:
URL: https://github.com/apache/hadoop/pull/6510#discussion_r1475654924
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/RouterObserverReadProxyProvider.java:
##
@@ -84,7 +84,8 @@ public class RouterObserverReadProxyProvider extends
AbstractNNFailoverProxyP
public RouterObserverReadProxyProvider(Configuration conf, URI uri, Class
xface,
HAProxyFactory factory) {
-this(conf, uri, xface, factory, new IPFailoverProxyProvider<>(conf, uri,
xface, factory));
+this(conf, uri, xface, factory,
+new ConfiguredFailoverProxyProvider<>(conf, uri, xface, factory));
Review Comment:
In the original code I assumed the routers would all sit behind a single
host name and traffic would be split via DNS. The IpFailoverProxyProvider is
then needed incase one router is down but the IP is still in the DNS record.
I can see how in other setups users may need to list all the routers
explicitly in the client configurations.
Can we make the proxy provider configurable? To support both usecases.
> RBF: RouterObserverReadProxyProvider should use
> ConfiguredFailoverProxyProvider internally
> --
>
> Key: HDFS-17362
> URL: https://issues.apache.org/jira/browse/HDFS-17362
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Takanobu Asanuma
>Assignee: Takanobu Asanuma
>Priority: Major
> Labels: pull-request-available
>
> Currently, RouterObserverReadProxyProvider is using IPFailoverProxyProvider,
> while ObserverReadProxyProvider is using ConfiguredFailoverProxyProvider. If
> we are to align RouterObserverReadProxyProvider with
> ObserverReadProxyProvider, RouterObserverReadProxyProvider should internally
> use ConfiguredFailoverProxyProvider. Moreover, IPFailoverProxyProvider has
> an issue with resolving HA configurations. (For example,
> IPFailoverProxyProvider cannot resolve hdfs://router-service.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[
https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17813544#comment-17813544
]
ASF GitHub Bot commented on HDFS-17362:
---
simbadzina commented on code in PR #6510:
URL: https://github.com/apache/hadoop/pull/6510#discussion_r1475695932
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/RouterObserverReadProxyProvider.java:
##
@@ -84,7 +84,8 @@ public class RouterObserverReadProxyProvider extends
AbstractNNFailoverProxyP
public RouterObserverReadProxyProvider(Configuration conf, URI uri, Class
xface,
HAProxyFactory factory) {
-this(conf, uri, xface, factory, new IPFailoverProxyProvider<>(conf, uri,
xface, factory));
+this(conf, uri, xface, factory,
+new ConfiguredFailoverProxyProvider<>(conf, uri, xface, factory));
Review Comment:
What do you think about making it configurable? For a different internal
proxyprovider we have code like below.
```
this(conf, uri, xface, proxyFactory, getFailoverProxyProvider(conf, uri,
xface, proxyFactory));
```
and
```
@VisibleForTesting
protected static AbstractNNFailoverProxyProvider
getFailoverProxyProvider(
Configuration conf, URI uri, Class xface, HAProxyFactory
proxyFactory) {
String configKey = FAILOVER_PROXY_PROVIDER + "." + uri.getHost();
Class> fppClass =
(Class>) conf.getClass(configKey,
(Class)
DEFAULT_FAILOVER_PROXY_PROVIDER,
AbstractNNFailoverProxyProvider.class);
try {
Constructor> ctor =
fppClass.getConstructor(
Configuration.class, URI.class, Class.class, HAProxyFactory.class);
return ctor.newInstance(conf, uri, xface, proxyFactory);
} catch (Exception e) {
LOG.error("Failed to create failover proxy provider", e);
throw new IllegalArgumentException(e);
}
}
```
> RBF: RouterObserverReadProxyProvider should use
> ConfiguredFailoverProxyProvider internally
> --
>
> Key: HDFS-17362
> URL: https://issues.apache.org/jira/browse/HDFS-17362
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Takanobu Asanuma
>Assignee: Takanobu Asanuma
>Priority: Major
> Labels: pull-request-available
>
> Currently, RouterObserverReadProxyProvider is using IPFailoverProxyProvider,
> while ObserverReadProxyProvider is using ConfiguredFailoverProxyProvider. If
> we are to align RouterObserverReadProxyProvider with
> ObserverReadProxyProvider, RouterObserverReadProxyProvider should internally
> use ConfiguredFailoverProxyProvider. Moreover, IPFailoverProxyProvider has
> an issue with resolving HA configurations. (For example,
> IPFailoverProxyProvider cannot resolve hdfs://router-service.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[
https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17813543#comment-17813543
]
ASF GitHub Bot commented on HDFS-17362:
---
simbadzina commented on code in PR #6510:
URL: https://github.com/apache/hadoop/pull/6510#discussion_r1475691426
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/RouterObserverReadProxyProvider.java:
##
@@ -84,7 +84,8 @@ public class RouterObserverReadProxyProvider extends
AbstractNNFailoverProxyP
public RouterObserverReadProxyProvider(Configuration conf, URI uri, Class
xface,
HAProxyFactory factory) {
-this(conf, uri, xface, factory, new IPFailoverProxyProvider<>(conf, uri,
xface, factory));
+this(conf, uri, xface, factory,
+new ConfiguredFailoverProxyProvider<>(conf, uri, xface, factory));
Review Comment:
Actually, `IPFailoverProxyProvider` can still be set on
`routerContext.getFileSystemURI().toString()` even when we use the
ConfiguredFailoverProxyProvider.
We should have a test case for this.
> RBF: RouterObserverReadProxyProvider should use
> ConfiguredFailoverProxyProvider internally
> --
>
> Key: HDFS-17362
> URL: https://issues.apache.org/jira/browse/HDFS-17362
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Takanobu Asanuma
>Assignee: Takanobu Asanuma
>Priority: Major
> Labels: pull-request-available
>
> Currently, RouterObserverReadProxyProvider is using IPFailoverProxyProvider,
> while ObserverReadProxyProvider is using ConfiguredFailoverProxyProvider. If
> we are to align RouterObserverReadProxyProvider with
> ObserverReadProxyProvider, RouterObserverReadProxyProvider should internally
> use ConfiguredFailoverProxyProvider. Moreover, IPFailoverProxyProvider has
> an issue with resolving HA configurations. (For example,
> IPFailoverProxyProvider cannot resolve hdfs://router-service.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[
https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17813541#comment-17813541
]
ASF GitHub Bot commented on HDFS-17362:
---
simbadzina commented on code in PR #6510:
URL: https://github.com/apache/hadoop/pull/6510#discussion_r1475691426
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/RouterObserverReadProxyProvider.java:
##
@@ -84,7 +84,8 @@ public class RouterObserverReadProxyProvider extends
AbstractNNFailoverProxyP
public RouterObserverReadProxyProvider(Configuration conf, URI uri, Class
xface,
HAProxyFactory factory) {
-this(conf, uri, xface, factory, new IPFailoverProxyProvider<>(conf, uri,
xface, factory));
+this(conf, uri, xface, factory,
+new ConfiguredFailoverProxyProvider<>(conf, uri, xface, factory));
Review Comment:
Actually, `IPFailoverProxyProvider` can still be set on
`routerContext.getFileSystemURI().toString()` even when we use the
ConfiguredFailoverProxyProvider.
> RBF: RouterObserverReadProxyProvider should use
> ConfiguredFailoverProxyProvider internally
> --
>
> Key: HDFS-17362
> URL: https://issues.apache.org/jira/browse/HDFS-17362
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Takanobu Asanuma
>Assignee: Takanobu Asanuma
>Priority: Major
> Labels: pull-request-available
>
> Currently, RouterObserverReadProxyProvider is using IPFailoverProxyProvider,
> while ObserverReadProxyProvider is using ConfiguredFailoverProxyProvider. If
> we are to align RouterObserverReadProxyProvider with
> ObserverReadProxyProvider, RouterObserverReadProxyProvider should internally
> use ConfiguredFailoverProxyProvider. Moreover, IPFailoverProxyProvider has
> an issue with resolving HA configurations. (For example,
> IPFailoverProxyProvider cannot resolve hdfs://router-service.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[
https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17813538#comment-17813538
]
ASF GitHub Bot commented on HDFS-17362:
---
simbadzina commented on code in PR #6510:
URL: https://github.com/apache/hadoop/pull/6510#discussion_r1475654924
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/RouterObserverReadProxyProvider.java:
##
@@ -84,7 +84,8 @@ public class RouterObserverReadProxyProvider extends
AbstractNNFailoverProxyP
public RouterObserverReadProxyProvider(Configuration conf, URI uri, Class
xface,
HAProxyFactory factory) {
-this(conf, uri, xface, factory, new IPFailoverProxyProvider<>(conf, uri,
xface, factory));
+this(conf, uri, xface, factory,
+new ConfiguredFailoverProxyProvider<>(conf, uri, xface, factory));
Review Comment:
In my original code I assumed the routers would all sit behind a single host
name and traffic would be split via DNS. The IpFailoverProxyProvider is then
needed incase one router is down but the IP is still in the DNS record.
I can see how in other setups users may need to list all the routers
explicitly in the client configurations.
Can we make the proxy provider configurable? To support both usecases.
> RBF: RouterObserverReadProxyProvider should use
> ConfiguredFailoverProxyProvider internally
> --
>
> Key: HDFS-17362
> URL: https://issues.apache.org/jira/browse/HDFS-17362
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Takanobu Asanuma
>Assignee: Takanobu Asanuma
>Priority: Major
> Labels: pull-request-available
>
> Currently, RouterObserverReadProxyProvider is using IPFailoverProxyProvider,
> while ObserverReadProxyProvider is using ConfiguredFailoverProxyProvider. If
> we are to align RouterObserverReadProxyProvider with
> ObserverReadProxyProvider, RouterObserverReadProxyProvider should internally
> use ConfiguredFailoverProxyProvider. Moreover, IPFailoverProxyProvider has
> an issue with resolving HA configurations. (For example,
> IPFailoverProxyProvider cannot resolve hdfs://router-service.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[
https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812684#comment-17812684
]
ASF GitHub Bot commented on HDFS-17362:
---
hadoop-yetus commented on PR #6510:
URL: https://github.com/apache/hadoop/pull/6510#issuecomment-1918979261
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 8s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 33m 59s | | trunk passed |
| +1 :green_heart: | compile | 5m 24s | | trunk passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 5m 17s | | trunk passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 20s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 37s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 22s | | trunk passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 11s | | trunk passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 53s | | trunk passed |
| +1 :green_heart: | shadedclient | 36m 3s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 21s | | the patch passed |
| +1 :green_heart: | compile | 6m 1s | | the patch passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 6m 1s | | the patch passed |
| +1 :green_heart: | compile | 5m 59s | | the patch passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 5m 59s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 25s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 26s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 6s | | the patch passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 0s | | the patch passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 4m 17s | | the patch passed |
| +1 :green_heart: | shadedclient | 36m 9s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 26s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | unit | 22m 8s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not
generate ASF License warnings. |
| | | 194m 10s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.44 ServerAPI=1.44 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6510/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6510 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 9ea8f1ee342d 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0e66511214db118df601b41670c1446e63f97f49 |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6510/3/testReport/ |
| Max. process+thread count | 2574 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client
hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project |
| Console
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[ https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812585#comment-17812585 ] ASF GitHub Bot commented on HDFS-17362: --- tasanuma commented on code in PR #6510: URL: https://github.com/apache/hadoop/pull/6510#discussion_r1472501788 ## hadoop-hdfs-project/hadoop-hdfs-rbf/pom.xml: ## @@ -177,6 +177,11 @@ https://maven.apache.org/xsd/maven-4.0.0.xsd;> junit-jupiter-params test + + org.junit.jupiter + junit-jupiter-engine + test + Review Comment: Agreed, I deleted it by 0e66511214db118df601b41670c1446e63f97f49. I will create another ticket. > RBF: RouterObserverReadProxyProvider should use > ConfiguredFailoverProxyProvider internally > -- > > Key: HDFS-17362 > URL: https://issues.apache.org/jira/browse/HDFS-17362 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Major > Labels: pull-request-available > > Currently, RouterObserverReadProxyProvider is using IPFailoverProxyProvider, > while ObserverReadProxyProvider is using ConfiguredFailoverProxyProvider. If > we are to align RouterObserverReadProxyProvider with > ObserverReadProxyProvider, RouterObserverReadProxyProvider should internally > use ConfiguredFailoverProxyProvider. Moreover, IPFailoverProxyProvider has > an issue with resolving HA configurations. (For example, > IPFailoverProxyProvider cannot resolve hdfs://router-service.) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[ https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812582#comment-17812582 ] ASF GitHub Bot commented on HDFS-17362: --- tasanuma commented on PR #6510: URL: https://github.com/apache/hadoop/pull/6510#issuecomment-1918665891 @ayushtkn Thanks for the review. I've updated the PR to use `RouterObserverReadProxyProvider.class` in the logger. It seems that `RouterObserverReadProxyProvider` is simpler than `ObserverReadProxyProvider` as Router is handling some tasks for it. So, extending `ObserverReadProxyProvider` might not be the best approach. Indeed, there are some common fields and methods such as `autoMsyncIfNecessary()`, so creating an abstract class or an interface could be a good idea. I would like to hear @simbadzina 's opinion on this. By the way, @chunyiyang is actually a woman. :) > RBF: RouterObserverReadProxyProvider should use > ConfiguredFailoverProxyProvider internally > -- > > Key: HDFS-17362 > URL: https://issues.apache.org/jira/browse/HDFS-17362 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Major > Labels: pull-request-available > > Currently, RouterObserverReadProxyProvider is using IPFailoverProxyProvider, > while ObserverReadProxyProvider is using ConfiguredFailoverProxyProvider. If > we are to align RouterObserverReadProxyProvider with > ObserverReadProxyProvider, RouterObserverReadProxyProvider should internally > use ConfiguredFailoverProxyProvider. Moreover, IPFailoverProxyProvider has > an issue with resolving HA configurations. (For example, > IPFailoverProxyProvider cannot resolve hdfs://router-service.) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[
https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17811785#comment-17811785
]
ASF GitHub Bot commented on HDFS-17362:
---
hadoop-yetus commented on PR #6510:
URL: https://github.com/apache/hadoop/pull/6510#issuecomment-1914273925
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 33s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 26s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 33m 48s | | trunk passed |
| +1 :green_heart: | compile | 6m 6s | | trunk passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 5m 43s | | trunk passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 35s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 21s | | trunk passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 8s | | trunk passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 4m 1s | | trunk passed |
| +1 :green_heart: | shadedclient | 35m 42s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 21s | | the patch passed |
| +1 :green_heart: | compile | 5m 45s | | the patch passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 5m 45s | | the patch passed |
| +1 :green_heart: | compile | 5m 41s | | the patch passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 5m 41s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 14s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 25s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 5s | | the patch passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 57s | | the patch passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 4m 14s | | the patch passed |
| +1 :green_heart: | shadedclient | 37m 18s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 28s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | unit | 9m 58s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not
generate ASF License warnings. |
| | | 183m 3s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.44 ServerAPI=1.44 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6510/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6510 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 30022d11c9c6 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7b605dad4a164e73c36cf54efa49d68718e8e2e3 |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6510/2/testReport/ |
| Max. process+thread count | 2636 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client
hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project |
| Console
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[
https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17811729#comment-17811729
]
ASF GitHub Bot commented on HDFS-17362:
---
hadoop-yetus commented on PR #6510:
URL: https://github.com/apache/hadoop/pull/6510#issuecomment-1913950036
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 32s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 3s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 31m 12s | | trunk passed |
| +1 :green_heart: | compile | 5m 27s | | trunk passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 5m 19s | | trunk passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 25s | | trunk passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 14s | | trunk passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 58s | | trunk passed |
| +1 :green_heart: | shadedclient | 32m 50s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 19s | | the patch passed |
| +1 :green_heart: | compile | 5m 14s | | the patch passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 5m 14s | | the patch passed |
| +1 :green_heart: | compile | 5m 4s | | the patch passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 5m 4s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 11s |
[/results-checkstyle-hadoop-hdfs-project.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6510/1/artifact/out/results-checkstyle-hadoop-hdfs-project.txt)
| hadoop-hdfs-project: The patch generated 4 new + 0 unchanged - 0 fixed = 4
total (was 0) |
| +1 :green_heart: | mvnsite | 1m 20s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 6s | | the patch passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 0s | | the patch passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 4m 1s | | the patch passed |
| +1 :green_heart: | shadedclient | 32m 43s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 26s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | unit | 9m 49s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not
generate ASF License warnings. |
| | | 171m 5s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.44 ServerAPI=1.44 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6510/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6510 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 3434721bf1a2 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 77d9bbc7ece50b49c9801318669b92ce96950d62 |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results |
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[ https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17811719#comment-17811719 ] ASF GitHub Bot commented on HDFS-17362: --- tasanuma commented on code in PR #6510: URL: https://github.com/apache/hadoop/pull/6510#discussion_r1469042435 ## hadoop-hdfs-project/hadoop-hdfs-rbf/pom.xml: ## @@ -177,6 +177,11 @@ https://maven.apache.org/xsd/maven-4.0.0.xsd;> junit-jupiter-params test + + org.junit.jupiter + junit-jupiter-engine + test + Review Comment: `mvn test -Dtest=TestObserverWithRouter` skips ParameterizedTest if we don't add junit-jupiter-engine here. > RBF: RouterObserverReadProxyProvider should use > ConfiguredFailoverProxyProvider internally > -- > > Key: HDFS-17362 > URL: https://issues.apache.org/jira/browse/HDFS-17362 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Major > Labels: pull-request-available > > Currently, RouterObserverReadProxyProvider is using IPFailoverProxyProvider, > while ObserverReadProxyProvider is using ConfiguredFailoverProxyProvider. If > we are to align RouterObserverReadProxyProvider with > ObserverReadProxyProvider, RouterObserverReadProxyProvider should internally > use ConfiguredFailoverProxyProvider. Moreover, IPFailoverProxyProvider has > an issue with resolving HA configurations. (For example, > IPFailoverProxyProvider cannot resolve hdfs://router-service.) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17362) RBF: RouterObserverReadProxyProvider should use ConfiguredFailoverProxyProvider internally
[ https://issues.apache.org/jira/browse/HDFS-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17811706#comment-17811706 ] ASF GitHub Bot commented on HDFS-17362: --- tasanuma opened a new pull request, #6510: URL: https://github.com/apache/hadoop/pull/6510 ### Description of PR Currently, RouterObserverReadProxyProvider is using IPFailoverProxyProvider, while ObserverReadProxyProvider is using ConfiguredFailoverProxyProvider. If we are to align RouterObserverReadProxyProvider with ObserverReadProxyProvider, RouterObserverReadProxyProvider should internally use ConfiguredFailoverProxyProvider. Moreover, IPFailoverProxyProvider has an issue with resolving HA configurations. (For example, IPFailoverProxyProvider cannot resolve hdfs://router-service.) ### How was this patch tested? `mvn test -Dtest=TestObserverWithRouter` succeeded in my local computer. ### For code changes: - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? > RBF: RouterObserverReadProxyProvider should use > ConfiguredFailoverProxyProvider internally > -- > > Key: HDFS-17362 > URL: https://issues.apache.org/jira/browse/HDFS-17362 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Major > > Currently, RouterObserverReadProxyProvider is using IPFailoverProxyProvider, > while ObserverReadProxyProvider is using ConfiguredFailoverProxyProvider. If > we are to align RouterObserverReadProxyProvider with > ObserverReadProxyProvider, RouterObserverReadProxyProvider should internally > use ConfiguredFailoverProxyProvider. Moreover, IPFailoverProxyProvider has > an issue with resolving HA configurations. (For example, > IPFailoverProxyProvider cannot resolve hdfs://router-service.) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
