[jira] [Commented] (HDFS-13653) Make dfs.client.failover.random.order a per nameservice configuration
[ https://issues.apache.org/jira/browse/HDFS-13653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16501410#comment-16501410 ] Ekanth S commented on HDFS-13653: - Fixed default value in the documentation. Updated patch [^HDFS-13653.003.patch]. > Make dfs.client.failover.random.order a per nameservice configuration > - > > Key: HDFS-13653 > URL: https://issues.apache.org/jira/browse/HDFS-13653 > Project: Hadoop HDFS > Issue Type: Improvement > Components: federation >Reporter: Ekanth S >Assignee: Ekanth S >Priority: Major > Attachments: HDFS-13653.001.patch, HDFS-13653.002.patch, > HDFS-13653.003.patch > > > Currently the dfs.client.failover.random.order is applied globally. If we > have a combination of router and non-router nameservice, the random order > should ideally be enabled only for the router based nameservice. This Jira is > to make this configuration per-nameservice so that this can be configured > independently for each nameservice. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13653) Make dfs.client.failover.random.order a per nameservice configuration
[ https://issues.apache.org/jira/browse/HDFS-13653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekanth S updated HDFS-13653: Attachment: HDFS-13653.003.patch > Make dfs.client.failover.random.order a per nameservice configuration > - > > Key: HDFS-13653 > URL: https://issues.apache.org/jira/browse/HDFS-13653 > Project: Hadoop HDFS > Issue Type: Improvement > Components: federation >Reporter: Ekanth S >Assignee: Ekanth S >Priority: Major > Attachments: HDFS-13653.001.patch, HDFS-13653.002.patch, > HDFS-13653.003.patch > > > Currently the dfs.client.failover.random.order is applied globally. If we > have a combination of router and non-router nameservice, the random order > should ideally be enabled only for the router based nameservice. This Jira is > to make this configuration per-nameservice so that this can be configured > independently for each nameservice. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13653) Make dfs.client.failover.random.order a per nameservice configuration
[ https://issues.apache.org/jira/browse/HDFS-13653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16501372#comment-16501372 ] Ekanth S commented on HDFS-13653: - Good point [~csun]. I have updated the patch with docs about configuration in the new patch [^HDFS-13653.002.patch]. > Make dfs.client.failover.random.order a per nameservice configuration > - > > Key: HDFS-13653 > URL: https://issues.apache.org/jira/browse/HDFS-13653 > Project: Hadoop HDFS > Issue Type: Improvement > Components: federation >Reporter: Ekanth S >Assignee: Ekanth S >Priority: Major > Attachments: HDFS-13653.001.patch, HDFS-13653.002.patch > > > Currently the dfs.client.failover.random.order is applied globally. If we > have a combination of router and non-router nameservice, the random order > should ideally be enabled only for the router based nameservice. This Jira is > to make this configuration per-nameservice so that this can be configured > independently for each nameservice. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13653) Make dfs.client.failover.random.order a per nameservice configuration
[ https://issues.apache.org/jira/browse/HDFS-13653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekanth S updated HDFS-13653: Attachment: HDFS-13653.002.patch > Make dfs.client.failover.random.order a per nameservice configuration > - > > Key: HDFS-13653 > URL: https://issues.apache.org/jira/browse/HDFS-13653 > Project: Hadoop HDFS > Issue Type: Improvement > Components: federation >Reporter: Ekanth S >Assignee: Ekanth S >Priority: Major > Attachments: HDFS-13653.001.patch, HDFS-13653.002.patch > > > Currently the dfs.client.failover.random.order is applied globally. If we > have a combination of router and non-router nameservice, the random order > should ideally be enabled only for the router based nameservice. This Jira is > to make this configuration per-nameservice so that this can be configured > independently for each nameservice. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13653) Make dfs.client.failover.random.order a per nameservice configuration
[ https://issues.apache.org/jira/browse/HDFS-13653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16501302#comment-16501302 ] Ekanth S commented on HDFS-13653: - [~elgoiri], thanks for the quick review. I did think about unit tests when writing the patch. Since this is a private method that is about randomness, it isn't easy to unit test this as any expectation would be unpredictable (say from what getProxy would return for example). There are tests in TestDFSClientFailover.java that test this code path to make sure the basic functionality works as in calling getRandomOrder() as part of constructing ConfiguredFailoverProxyProvider. However, I am open to ideas to test this. Let me know if you have specific guidance here. On the style check, it appears okay after the yetus run. I did notice other methods with params that do not have a period at the end. So I guess it is okay as it is? > Make dfs.client.failover.random.order a per nameservice configuration > - > > Key: HDFS-13653 > URL: https://issues.apache.org/jira/browse/HDFS-13653 > Project: Hadoop HDFS > Issue Type: Improvement > Components: federation >Reporter: Ekanth S >Assignee: Ekanth S >Priority: Major > Attachments: HDFS-13653.001.patch > > > Currently the dfs.client.failover.random.order is applied globally. If we > have a combination of router and non-router nameservice, the random order > should ideally be enabled only for the router based nameservice. This Jira is > to make this configuration per-nameservice so that this can be configured > independently for each nameservice. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13653) [RBF] Make dfs.client.failover.random.order a per nameservice configuration
[ https://issues.apache.org/jira/browse/HDFS-13653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16500923#comment-16500923 ] Ekanth S commented on HDFS-13653: - The patch [^HDFS-13653.001.patch] handles backwards compatibility with current configs that use it as global value and gives preference to a specific configuration if present. > [RBF] Make dfs.client.failover.random.order a per nameservice configuration > --- > > Key: HDFS-13653 > URL: https://issues.apache.org/jira/browse/HDFS-13653 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: federation >Reporter: Ekanth S >Assignee: Ekanth S >Priority: Major > Attachments: HDFS-13653.001.patch > > > Currently the dfs.client.failover.random.order is applied globally. If we > have a combination of router and non-router nameservice, the random order > should ideally be enabled only for the router based nameservice. This Jira is > to make this configuration per-nameservice so that this can be configured > independently for each nameservice. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13653) [RBF] Make dfs.client.failover.random.order a per nameservice configuration
[ https://issues.apache.org/jira/browse/HDFS-13653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16500917#comment-16500917 ] Ekanth S commented on HDFS-13653: - Agree with you [~goiri] on the non exclusivity to RBF. Attached a patch. [~crh], [~csun], [~goiri], would appreciate your review. > [RBF] Make dfs.client.failover.random.order a per nameservice configuration > --- > > Key: HDFS-13653 > URL: https://issues.apache.org/jira/browse/HDFS-13653 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: federation >Reporter: Ekanth S >Assignee: Ekanth S >Priority: Major > Attachments: HDFS-13653.001.patch > > > Currently the dfs.client.failover.random.order is applied globally. If we > have a combination of router and non-router nameservice, the random order > should ideally be enabled only for the router based nameservice. This Jira is > to make this configuration per-nameservice so that this can be configured > independently for each nameservice. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13653) [RBF] Make dfs.client.failover.random.order a per nameservice configuration
[ https://issues.apache.org/jira/browse/HDFS-13653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekanth S updated HDFS-13653: Attachment: HDFS-13653.001.patch > [RBF] Make dfs.client.failover.random.order a per nameservice configuration > --- > > Key: HDFS-13653 > URL: https://issues.apache.org/jira/browse/HDFS-13653 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: federation >Reporter: Ekanth S >Assignee: Ekanth S >Priority: Major > Attachments: HDFS-13653.001.patch > > > Currently the dfs.client.failover.random.order is applied globally. If we > have a combination of router and non-router nameservice, the random order > should ideally be enabled only for the router based nameservice. This Jira is > to make this configuration per-nameservice so that this can be configured > independently for each nameservice. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-13653) [RBF] Make dfs.client.failover.random.order a per nameservice configuration
Ekanth S created HDFS-13653: --- Summary: [RBF] Make dfs.client.failover.random.order a per nameservice configuration Key: HDFS-13653 URL: https://issues.apache.org/jira/browse/HDFS-13653 Project: Hadoop HDFS Issue Type: Sub-task Components: federation Reporter: Ekanth S Assignee: Ekanth S Currently the dfs.client.failover.random.order is applied globally. If we have a combination of router and non-router nameservice, the random order should ideally be enabled only for the router based nameservice. This Jira is to make this configuration per-nameservice so that this can be configured independently for each nameservice. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-13637) RBF: Router fails when threadIndex (in ConnectionPool) wraps around Integer.MIN_VALUE
[
https://issues.apache.org/jira/browse/HDFS-13637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekanth S reassigned HDFS-13637:
---
Assignee: CR Hota
> RBF: Router fails when threadIndex (in ConnectionPool) wraps around
> Integer.MIN_VALUE
> -
>
> Key: HDFS-13637
> URL: https://issues.apache.org/jira/browse/HDFS-13637
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: federation
>Reporter: CR Hota
>Assignee: CR Hota
>Priority: Major
>
> {code:java}
> int threadIndex = this.clientIndex.getAndIncrement();
> for (int i=0; i int index = (threadIndex + i) % size;
> conn = tmpConnections.get(index);
> if (conn != null && conn.isUsable()) {
> return conn;
> }
> }
> {code}
> The above code in ConnectionPool.java getConnection method throws
> java.lang.ArrayIndexOutOfBoundsException when clientIndex wraps to
> Integer.MIN_VALUE and makes router reject all requests. threadIndex should be
> reset to 0.
> {code:java}
> if (threadIndex < 0) {
> // Wrap around 0 to keep array lookup index positive
> this.clientIndex.set(0);
> threadIndex = this.clientIndex.getAndIncrement();
> }
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13508) RBF: Normalize paths (automatically) when adding, updating, removing or listing mount table entries
[ https://issues.apache.org/jira/browse/HDFS-13508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekanth S updated HDFS-13508: Attachment: HDFS-13508.003.patch > RBF: Normalize paths (automatically) when adding, updating, removing or > listing mount table entries > --- > > Key: HDFS-13508 > URL: https://issues.apache.org/jira/browse/HDFS-13508 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Ekanth S >Assignee: Ekanth S >Priority: Minor > Attachments: HDFS-13508.001.patch, HDFS-13508.002.patch, > HDFS-13508.003.patch > > > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move > Mount Table Entries: > Source Destinations Owner Group Mode > /home/move hdfs-oahu->/home/move me hadoop rwxr-xr-x > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move/ > Mount Table Entries: > Source Destinations Owner Group Mode > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -rm /home/move/ > Cannot remove mount point /home/move/ > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -add /home/move/ > hdfs-oahu /home/move/ -readonly > Cannot add mount point /home/move/ > The slash '/' at the end should be normalized before calling the API from the > CLI. > Note: add command fails with a terminating '/' . when it is an existing entry > (it checks the not-normalized value with the normalized value in the > mount-table). Adding a new mount point with '/' at the end works because the > CLI normalizes the mount before calling the API. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13508) RBF: Normalize paths (automatically) when adding, removing or listing mount table entries
[ https://issues.apache.org/jira/browse/HDFS-13508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454845#comment-16454845 ] Ekanth S commented on HDFS-13508: - Added a new patch [^HDFS-13508.002.patch] after rebasing and adding support for update command as well. > RBF: Normalize paths (automatically) when adding, removing or listing mount > table entries > - > > Key: HDFS-13508 > URL: https://issues.apache.org/jira/browse/HDFS-13508 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Ekanth S >Assignee: Ekanth S >Priority: Minor > Attachments: HDFS-13508.001.patch, HDFS-13508.002.patch > > > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move > Mount Table Entries: > Source Destinations Owner Group Mode > /home/move hdfs-oahu->/home/move me hadoop rwxr-xr-x > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move/ > Mount Table Entries: > Source Destinations Owner Group Mode > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -rm /home/move/ > Cannot remove mount point /home/move/ > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -add /home/move/ > hdfs-oahu /home/move/ -readonly > Cannot add mount point /home/move/ > The slash '/' at the end should be normalized before calling the API from the > CLI. > Note: add command fails with a terminating '/' . when it is an existing entry > (it checks the not-normalized value with the normalized value in the > mount-table). Adding a new mount point with '/' at the end works because the > CLI normalizes the mount before calling the API. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13508) RBF: Normalize paths (automatically) when adding, updating, removing or listing mount table entries
[ https://issues.apache.org/jira/browse/HDFS-13508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekanth S updated HDFS-13508: Summary: RBF: Normalize paths (automatically) when adding, updating, removing or listing mount table entries (was: RBF: Normalize paths (automatically) when adding, removing or listing mount table entries) > RBF: Normalize paths (automatically) when adding, updating, removing or > listing mount table entries > --- > > Key: HDFS-13508 > URL: https://issues.apache.org/jira/browse/HDFS-13508 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Ekanth S >Assignee: Ekanth S >Priority: Minor > Attachments: HDFS-13508.001.patch, HDFS-13508.002.patch > > > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move > Mount Table Entries: > Source Destinations Owner Group Mode > /home/move hdfs-oahu->/home/move me hadoop rwxr-xr-x > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move/ > Mount Table Entries: > Source Destinations Owner Group Mode > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -rm /home/move/ > Cannot remove mount point /home/move/ > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -add /home/move/ > hdfs-oahu /home/move/ -readonly > Cannot add mount point /home/move/ > The slash '/' at the end should be normalized before calling the API from the > CLI. > Note: add command fails with a terminating '/' . when it is an existing entry > (it checks the not-normalized value with the normalized value in the > mount-table). Adding a new mount point with '/' at the end works because the > CLI normalizes the mount before calling the API. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13508) RBF: Normalize paths (automatically) when adding, removing or listing mount table entries
[ https://issues.apache.org/jira/browse/HDFS-13508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekanth S updated HDFS-13508: Attachment: HDFS-13508.002.patch > RBF: Normalize paths (automatically) when adding, removing or listing mount > table entries > - > > Key: HDFS-13508 > URL: https://issues.apache.org/jira/browse/HDFS-13508 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Ekanth S >Assignee: Ekanth S >Priority: Minor > Attachments: HDFS-13508.001.patch, HDFS-13508.002.patch > > > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move > Mount Table Entries: > Source Destinations Owner Group Mode > /home/move hdfs-oahu->/home/move me hadoop rwxr-xr-x > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move/ > Mount Table Entries: > Source Destinations Owner Group Mode > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -rm /home/move/ > Cannot remove mount point /home/move/ > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -add /home/move/ > hdfs-oahu /home/move/ -readonly > Cannot add mount point /home/move/ > The slash '/' at the end should be normalized before calling the API from the > CLI. > Note: add command fails with a terminating '/' . when it is an existing entry > (it checks the not-normalized value with the normalized value in the > mount-table). Adding a new mount point with '/' at the end works because the > CLI normalizes the mount before calling the API. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13508) RBF: Normalize paths (automatically) when adding, removing or listing mount table entries
[ https://issues.apache.org/jira/browse/HDFS-13508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454817#comment-16454817 ] Ekanth S commented on HDFS-13508: - Thanks [~goiri] for taking a look at the patch. Let me update the patch. > RBF: Normalize paths (automatically) when adding, removing or listing mount > table entries > - > > Key: HDFS-13508 > URL: https://issues.apache.org/jira/browse/HDFS-13508 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Ekanth S >Assignee: Ekanth S >Priority: Minor > Attachments: HDFS-13508.001.patch > > > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move > Mount Table Entries: > Source Destinations Owner Group Mode > /home/move hdfs-oahu->/home/move me hadoop rwxr-xr-x > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move/ > Mount Table Entries: > Source Destinations Owner Group Mode > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -rm /home/move/ > Cannot remove mount point /home/move/ > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -add /home/move/ > hdfs-oahu /home/move/ -readonly > Cannot add mount point /home/move/ > The slash '/' at the end should be normalized before calling the API from the > CLI. > Note: add command fails with a terminating '/' . when it is an existing entry > (it checks the not-normalized value with the normalized value in the > mount-table). Adding a new mount point with '/' at the end works because the > CLI normalizes the mount before calling the API. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13508) RBF: Normalize paths (automatically) when adding, removing or listing mount table entries
[ https://issues.apache.org/jira/browse/HDFS-13508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454641#comment-16454641 ] Ekanth S commented on HDFS-13508: - [~goiri] [~ywskycn] [~gangli2384], appreciate if you could take a look at the patch. > RBF: Normalize paths (automatically) when adding, removing or listing mount > table entries > - > > Key: HDFS-13508 > URL: https://issues.apache.org/jira/browse/HDFS-13508 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Ekanth S >Assignee: Ekanth S >Priority: Minor > Attachments: HDFS-13508.001.patch > > > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move > Mount Table Entries: > Source Destinations Owner Group Mode > /home/move hdfs-oahu->/home/move me hadoop rwxr-xr-x > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move/ > Mount Table Entries: > Source Destinations Owner Group Mode > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -rm /home/move/ > Cannot remove mount point /home/move/ > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -add /home/move/ > hdfs-oahu /home/move/ -readonly > Cannot add mount point /home/move/ > The slash '/' at the end should be normalized before calling the API from the > CLI. > Note: add command fails with a terminating '/' . when it is an existing entry > (it checks the not-normalized value with the normalized value in the > mount-table). Adding a new mount point with '/' at the end works because the > CLI normalizes the mount before calling the API. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13508) RBF: Normalize paths (automatically) when adding, removing or listing mount table entries
[ https://issues.apache.org/jira/browse/HDFS-13508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekanth S updated HDFS-13508: Status: Patch Available (was: Open) > RBF: Normalize paths (automatically) when adding, removing or listing mount > table entries > - > > Key: HDFS-13508 > URL: https://issues.apache.org/jira/browse/HDFS-13508 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Ekanth S >Assignee: Ekanth S >Priority: Minor > Attachments: HDFS-13508.001.patch > > > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move > Mount Table Entries: > Source Destinations Owner Group Mode > /home/move hdfs-oahu->/home/move me hadoop rwxr-xr-x > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move/ > Mount Table Entries: > Source Destinations Owner Group Mode > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -rm /home/move/ > Cannot remove mount point /home/move/ > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -add /home/move/ > hdfs-oahu /home/move/ -readonly > Cannot add mount point /home/move/ > The slash '/' at the end should be normalized before calling the API from the > CLI. > Note: add command fails with a terminating '/' . when it is an existing entry > (it checks the not-normalized value with the normalized value in the > mount-table). Adding a new mount point with '/' at the end works because the > CLI normalizes the mount before calling the API. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13508) RBF: Normalize paths (automatically) when adding, removing or listing mount table entries
[ https://issues.apache.org/jira/browse/HDFS-13508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekanth S updated HDFS-13508: Flags: (was: Patch) > RBF: Normalize paths (automatically) when adding, removing or listing mount > table entries > - > > Key: HDFS-13508 > URL: https://issues.apache.org/jira/browse/HDFS-13508 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Ekanth S >Assignee: Ekanth S >Priority: Minor > Attachments: HDFS-13508.001.patch > > > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move > Mount Table Entries: > Source Destinations Owner Group Mode > /home/move hdfs-oahu->/home/move me hadoop rwxr-xr-x > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move/ > Mount Table Entries: > Source Destinations Owner Group Mode > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -rm /home/move/ > Cannot remove mount point /home/move/ > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -add /home/move/ > hdfs-oahu /home/move/ -readonly > Cannot add mount point /home/move/ > The slash '/' at the end should be normalized before calling the API from the > CLI. > Note: add command fails with a terminating '/' . when it is an existing entry > (it checks the not-normalized value with the normalized value in the > mount-table). Adding a new mount point with '/' at the end works because the > CLI normalizes the mount before calling the API. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13508) RBF: Normalize paths (automatically) when adding, removing or listing mount table entries
[ https://issues.apache.org/jira/browse/HDFS-13508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekanth S updated HDFS-13508: Flags: Patch > RBF: Normalize paths (automatically) when adding, removing or listing mount > table entries > - > > Key: HDFS-13508 > URL: https://issues.apache.org/jira/browse/HDFS-13508 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Ekanth S >Assignee: Ekanth S >Priority: Minor > Attachments: HDFS-13508.001.patch > > > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move > Mount Table Entries: > Source Destinations Owner Group Mode > /home/move hdfs-oahu->/home/move me hadoop rwxr-xr-x > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move/ > Mount Table Entries: > Source Destinations Owner Group Mode > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -rm /home/move/ > Cannot remove mount point /home/move/ > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -add /home/move/ > hdfs-oahu /home/move/ -readonly > Cannot add mount point /home/move/ > The slash '/' at the end should be normalized before calling the API from the > CLI. > Note: add command fails with a terminating '/' . when it is an existing entry > (it checks the not-normalized value with the normalized value in the > mount-table). Adding a new mount point with '/' at the end works because the > CLI normalizes the mount before calling the API. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13508) RBF: Normalize paths (automatically) when adding, removing or listing mount table entries
[ https://issues.apache.org/jira/browse/HDFS-13508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekanth S updated HDFS-13508: Attachment: HDFS-13508.001.patch > RBF: Normalize paths (automatically) when adding, removing or listing mount > table entries > - > > Key: HDFS-13508 > URL: https://issues.apache.org/jira/browse/HDFS-13508 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Ekanth S >Assignee: Ekanth S >Priority: Minor > Attachments: HDFS-13508.001.patch > > > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move > Mount Table Entries: > Source Destinations Owner Group Mode > /home/move hdfs-oahu->/home/move me hadoop rwxr-xr-x > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move/ > Mount Table Entries: > Source Destinations Owner Group Mode > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -rm /home/move/ > Cannot remove mount point /home/move/ > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -add /home/move/ > hdfs-oahu /home/move/ -readonly > Cannot add mount point /home/move/ > The slash '/' at the end should be normalized before calling the API from the > CLI. > Note: add command fails with a terminating '/' . when it is an existing entry > (it checks the not-normalized value with the normalized value in the > mount-table). Adding a new mount point with '/' at the end works because the > CLI normalizes the mount before calling the API. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-13508) Normalize paths (automatically) when adding, removing or listing mount table entries
[ https://issues.apache.org/jira/browse/HDFS-13508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekanth S reassigned HDFS-13508: --- Assignee: Ekanth S > Normalize paths (automatically) when adding, removing or listing mount table > entries > > > Key: HDFS-13508 > URL: https://issues.apache.org/jira/browse/HDFS-13508 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Ekanth S >Assignee: Ekanth S >Priority: Minor > > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move > Mount Table Entries: > Source Destinations Owner Group Mode > /home/move hdfs-oahu->/home/move me hadoop rwxr-xr-x > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move/ > Mount Table Entries: > Source Destinations Owner Group Mode > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -rm /home/move/ > Cannot remove mount point /home/move/ > me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -add /home/move/ > hdfs-oahu /home/move/ -readonly > Cannot add mount point /home/move/ > The slash '/' at the end should be normalized before calling the API from the > CLI. > Note: add command fails with a terminating '/' . when it is an existing entry > (it checks the not-normalized value with the normalized value in the > mount-table). Adding a new mount point with '/' at the end works because the > CLI normalizes the mount before calling the API. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-13508) Normalize paths (automatically) when adding, removing or listing mount table entries
Ekanth S created HDFS-13508: --- Summary: Normalize paths (automatically) when adding, removing or listing mount table entries Key: HDFS-13508 URL: https://issues.apache.org/jira/browse/HDFS-13508 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Ekanth S me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move Mount Table Entries: Source Destinations Owner Group Mode /home/move hdfs-oahu->/home/move me hadoop rwxr-xr-x me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -ls /home/move/ Mount Table Entries: Source Destinations Owner Group Mode me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -rm /home/move/ Cannot remove mount point /home/move/ me@gateway-hawaii-all:/mnt/host/bin$ hdfs dfsrouteradmin -add /home/move/ hdfs-oahu /home/move/ -readonly Cannot add mount point /home/move/ The slash '/' at the end should be normalized before calling the API from the CLI. Note: add command fails with a terminating '/' . when it is an existing entry (it checks the not-normalized value with the normalized value in the mount-table). Adding a new mount point with '/' at the end works because the CLI normalizes the mount before calling the API. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13289) RBF: TestConnectionManager#testCleanup() test case need correction
[
https://issues.apache.org/jira/browse/HDFS-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16420801#comment-16420801
]
Ekanth S commented on HDFS-13289:
-
Thanks [~dibyendu_hadoop] for addressing the comments. [^HDFS-13289.005.patch]
looks good (apart from [~goiri]'s comment). +1 from me.
> RBF: TestConnectionManager#testCleanup() test case need correction
> --
>
> Key: HDFS-13289
> URL: https://issues.apache.org/jira/browse/HDFS-13289
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Dibyendu Karmakar
>Assignee: Dibyendu Karmakar
>Priority: Minor
> Attachments: HDFS-13289.001.patch, HDFS-13289.002.patch,
> HDFS-13289.003.patch, HDFS-13289.004.patch, HDFS-13289.005.patch
>
>
> In TestConnectionManager#testCleanup()
>
> {code:java}
> // Make sure the number of connections doesn't go below minSize
> ConnectionPool pool3 = new ConnectionPool(
> conf, TEST_NN_ADDRESS, TEST_USER3, 2, 10);
> addConnectionsToPool(pool3, 10, 0);
> poolMap.put(new ConnectionPoolId(TEST_USER2, TEST_NN_ADDRESS), pool3);
> connManager.cleanup(pool3);
> checkPoolConnections(TEST_USER3, 2, 0);
> {code}
> this part need correction
> Here new ConnectionPoolId is created with TEST_USER2 but checkPoolConnections
> is done using TEST_USER3.
> In checkPoolConnections method
> {code:java}
> if (e.getKey().getUgi() == ugi)
> {code}
> then only it will validate numOfConns and numOfActiveConns. In this case for
> TEST_USER3 ' *if* 'condition is returning *false* and if you pass any value
> to the checkPoolConnections method, the test case will pass.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-13289) RBF: TestConnectionManager#testCleanup() test case need correction
[
https://issues.apache.org/jira/browse/HDFS-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419348#comment-16419348
]
Ekanth S edited comment on HDFS-13289 at 3/29/18 4:53 PM:
--
Thanks for the patch [~dibyendu_hadoop].
Nitpick for readability: Could we rename isValidUser to connPoolFoundForUser?
And along with that the error message to "Connection pool for user not found"?
was (Author: ekanth):
Thanks for the patch [~dibyendu_hadoop].
Nitpick for readability: Could we rename isUserFound to connPoolFoundForUser?
And along with that the error message to "Connection pool for user not found"?
> RBF: TestConnectionManager#testCleanup() test case need correction
> --
>
> Key: HDFS-13289
> URL: https://issues.apache.org/jira/browse/HDFS-13289
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Dibyendu Karmakar
>Assignee: Dibyendu Karmakar
>Priority: Minor
> Attachments: HDFS-13289.001.patch, HDFS-13289.002.patch,
> HDFS-13289.003.patch, HDFS-13289.004.patch
>
>
> In TestConnectionManager#testCleanup()
>
> {code:java}
> // Make sure the number of connections doesn't go below minSize
> ConnectionPool pool3 = new ConnectionPool(
> conf, TEST_NN_ADDRESS, TEST_USER3, 2, 10);
> addConnectionsToPool(pool3, 10, 0);
> poolMap.put(new ConnectionPoolId(TEST_USER2, TEST_NN_ADDRESS), pool3);
> connManager.cleanup(pool3);
> checkPoolConnections(TEST_USER3, 2, 0);
> {code}
> this part need correction
> Here new ConnectionPoolId is created with TEST_USER2 but checkPoolConnections
> is done using TEST_USER3.
> In checkPoolConnections method
> {code:java}
> if (e.getKey().getUgi() == ugi)
> {code}
> then only it will validate numOfConns and numOfActiveConns. In this case for
> TEST_USER3 ' *if* 'condition is returning *false* and if you pass any value
> to the checkPoolConnections method, the test case will pass.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13289) RBF: TestConnectionManager#testCleanup() test case need correction
[
https://issues.apache.org/jira/browse/HDFS-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419348#comment-16419348
]
Ekanth S commented on HDFS-13289:
-
Thanks for the patch [~dibyendu_hadoop].
Nitpick for readability: Could we rename isUserFound to connPoolFoundForUser?
And along with that the error message to "Connection pool for user not found"?
> RBF: TestConnectionManager#testCleanup() test case need correction
> --
>
> Key: HDFS-13289
> URL: https://issues.apache.org/jira/browse/HDFS-13289
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Dibyendu Karmakar
>Assignee: Dibyendu Karmakar
>Priority: Minor
> Attachments: HDFS-13289.001.patch, HDFS-13289.002.patch,
> HDFS-13289.003.patch, HDFS-13289.004.patch
>
>
> In TestConnectionManager#testCleanup()
>
> {code:java}
> // Make sure the number of connections doesn't go below minSize
> ConnectionPool pool3 = new ConnectionPool(
> conf, TEST_NN_ADDRESS, TEST_USER3, 2, 10);
> addConnectionsToPool(pool3, 10, 0);
> poolMap.put(new ConnectionPoolId(TEST_USER2, TEST_NN_ADDRESS), pool3);
> connManager.cleanup(pool3);
> checkPoolConnections(TEST_USER3, 2, 0);
> {code}
> this part need correction
> Here new ConnectionPoolId is created with TEST_USER2 but checkPoolConnections
> is done using TEST_USER3.
> In checkPoolConnections method
> {code:java}
> if (e.getKey().getUgi() == ugi)
> {code}
> then only it will validate numOfConns and numOfActiveConns. In this case for
> TEST_USER3 ' *if* 'condition is returning *false* and if you pass any value
> to the checkPoolConnections method, the test case will pass.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13318) RBF: Fix FindBugs in hadoop-hdfs-rbf
[ https://issues.apache.org/jira/browse/HDFS-13318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408634#comment-16408634 ] Ekanth S commented on HDFS-13318: - [~goiri], [~ywskycn], added a patch that resolves the FindBugs warnings. Please review and let me know. > RBF: Fix FindBugs in hadoop-hdfs-rbf > > > Key: HDFS-13318 > URL: https://issues.apache.org/jira/browse/HDFS-13318 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Ekanth S >Priority: Minor > Attachments: HDFS-13318.001.patch > > > hadoop-hdfs-rbf has 3 FindBug warnings: > * NamenodePriorityComparator should be serializable > * RemoteMethod.getTypes() may expose internal representation > * RemoteMethod may store mutable objects -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13318) RBF: Fix FindBugs in hadoop-hdfs-rbf
[ https://issues.apache.org/jira/browse/HDFS-13318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekanth S updated HDFS-13318: Attachment: HDFS-13318.001.patch > RBF: Fix FindBugs in hadoop-hdfs-rbf > > > Key: HDFS-13318 > URL: https://issues.apache.org/jira/browse/HDFS-13318 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Ekanth S >Priority: Minor > Attachments: HDFS-13318.001.patch > > > hadoop-hdfs-rbf has 3 FindBug warnings: > * NamenodePriorityComparator should be serializable > * RemoteMethod.getTypes() may expose internal representation > * RemoteMethod may store mutable objects -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-13318) RBF: Fix FindBugs in hadoop-hdfs-rbf
[ https://issues.apache.org/jira/browse/HDFS-13318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekanth S reassigned HDFS-13318: --- Assignee: Ekanth S (was: Íñigo Goiri) > RBF: Fix FindBugs in hadoop-hdfs-rbf > > > Key: HDFS-13318 > URL: https://issues.apache.org/jira/browse/HDFS-13318 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Ekanth S >Priority: Minor > > hadoop-hdfs-rbf has 3 FindBug warnings: > * NamenodePriorityComparator should be serializable > * RemoteMethod.getTypes() may expose internal representation > * RemoteMethod may store mutable objects -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13318) RBF: Fix FindBugs in hadoop-hdfs-rbf
[ https://issues.apache.org/jira/browse/HDFS-13318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408184#comment-16408184 ] Ekanth S commented on HDFS-13318: - I'll give this a shot. > RBF: Fix FindBugs in hadoop-hdfs-rbf > > > Key: HDFS-13318 > URL: https://issues.apache.org/jira/browse/HDFS-13318 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Íñigo Goiri >Priority: Minor > > hadoop-hdfs-rbf has 3 FindBug warnings: > * NamenodePriorityComparator should be serializable > * RemoteMethod.getTypes() may expose internal representation > * RemoteMethod may store mutable objects -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12505) Extend TestFileStatusWithECPolicy with a random EC policy
[ https://issues.apache.org/jira/browse/HDFS-12505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16395735#comment-16395735 ] Ekanth S commented on HDFS-12505: - Thanks [~tasanuma0829] for the patch. It looks good to me. > Extend TestFileStatusWithECPolicy with a random EC policy > - > > Key: HDFS-12505 > URL: https://issues.apache.org/jira/browse/HDFS-12505 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding, test >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Major > Labels: hdfs-ec-3.0-nice-to-have > Attachments: HDFS-12505.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-12677) Extend TestReconstructStripedFile with a random EC policy
[ https://issues.apache.org/jira/browse/HDFS-12677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16395677#comment-16395677 ] Ekanth S edited comment on HDFS-12677 at 3/12/18 6:52 PM: -- The first patch failed for me in a couple of other places (where the default policy was set/enabled and not the randomly chosen policy). I have uploaded a new patch that fixes this issue and I also updated the patch to change the assertTrue to assumeTrue to make sure tests (for XOR-2-1-1024k or a future EC policy that matches this condition) do not skip the check. was (Author: ekanth): The first patch failed for me in a couple of other places (where the default policy was set/enabled and not the randomly chosen policy). I have uploaded a new patch that fixes this issue and I also updated the patch to change the assertTrue to assumeTrue to make sure tests do not skip the check. > Extend TestReconstructStripedFile with a random EC policy > - > > Key: HDFS-12677 > URL: https://issues.apache.org/jira/browse/HDFS-12677 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding, test >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Major > Attachments: HDFS-12677.002.patch, HDFS-12677.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12677) Extend TestReconstructStripedFile with a random EC policy
[ https://issues.apache.org/jira/browse/HDFS-12677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16395677#comment-16395677 ] Ekanth S commented on HDFS-12677: - The first patch failed for me in a couple of other places (where the default policy was set/enabled and not the randomly chosen policy). I have uploaded a new patch that fixes this issue and I also updated the patch to change the assertTrue to assumeTrue to make sure tests do not skip the check. > Extend TestReconstructStripedFile with a random EC policy > - > > Key: HDFS-12677 > URL: https://issues.apache.org/jira/browse/HDFS-12677 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding, test >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Major > Attachments: HDFS-12677.002.patch, HDFS-12677.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12677) Extend TestReconstructStripedFile with a random EC policy
[ https://issues.apache.org/jira/browse/HDFS-12677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekanth S updated HDFS-12677: Attachment: HDFS-12677.002.patch > Extend TestReconstructStripedFile with a random EC policy > - > > Key: HDFS-12677 > URL: https://issues.apache.org/jira/browse/HDFS-12677 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding, test >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Major > Attachments: HDFS-12677.002.patch, HDFS-12677.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13232) RBF: ConnectionPool should return first usable connection
[
https://issues.apache.org/jira/browse/HDFS-13232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393915#comment-16393915
]
Ekanth S commented on HDFS-13232:
-
Sure. Fixed the white line change with the new patch.
> RBF: ConnectionPool should return first usable connection
> -
>
> Key: HDFS-13232
> URL: https://issues.apache.org/jira/browse/HDFS-13232
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Wei Yan
>Assignee: Ekanth S
>Priority: Minor
> Attachments: HDFS-13232.001.patch, HDFS-13232.002.patch,
> HDFS-13232.003.patch
>
>
> In current ConnectionPool.getConnection(), it will return the first active
> connection:
> {code:java}
> for (int i=0; i int index = (threadIndex + i) % size;
> conn = tmpConnections.get(index);
> if (conn != null && !conn.isUsable()) {
> return conn;
> }
> }
> {code}
> Here "!conn.isUsable()" should be "conn.isUsable()".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13232) RBF: ConnectionPool should return first usable connection
[
https://issues.apache.org/jira/browse/HDFS-13232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekanth S updated HDFS-13232:
Attachment: HDFS-13232.003.patch
> RBF: ConnectionPool should return first usable connection
> -
>
> Key: HDFS-13232
> URL: https://issues.apache.org/jira/browse/HDFS-13232
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Wei Yan
>Assignee: Ekanth S
>Priority: Minor
> Attachments: HDFS-13232.001.patch, HDFS-13232.002.patch,
> HDFS-13232.003.patch
>
>
> In current ConnectionPool.getConnection(), it will return the first active
> connection:
> {code:java}
> for (int i=0; i int index = (threadIndex + i) % size;
> conn = tmpConnections.get(index);
> if (conn != null && !conn.isUsable()) {
> return conn;
> }
> }
> {code}
> Here "!conn.isUsable()" should be "conn.isUsable()".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-13232) RBF: ConnectionPool should return first usable connection
[
https://issues.apache.org/jira/browse/HDFS-13232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393719#comment-16393719
]
Ekanth S edited comment on HDFS-13232 at 3/9/18 10:51 PM:
--
Thanks [~ywskycn] and [~elgoiri] for the quick review. Updated the patch with
the missing test, extra check for number of connections and formatting.
was (Author: ekanth):
Thanks Wei and Inigo for the quick review. Updated the patch with the missing
test, extra check for number of connections and formatting.
> RBF: ConnectionPool should return first usable connection
> -
>
> Key: HDFS-13232
> URL: https://issues.apache.org/jira/browse/HDFS-13232
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Wei Yan
>Assignee: Ekanth S
>Priority: Minor
> Attachments: HDFS-13232.001.patch, HDFS-13232.002.patch
>
>
> In current ConnectionPool.getConnection(), it will return the first active
> connection:
> {code:java}
> for (int i=0; i int index = (threadIndex + i) % size;
> conn = tmpConnections.get(index);
> if (conn != null && !conn.isUsable()) {
> return conn;
> }
> }
> {code}
> Here "!conn.isUsable()" should be "conn.isUsable()".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13232) RBF: ConnectionPool should return first usable connection
[
https://issues.apache.org/jira/browse/HDFS-13232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393719#comment-16393719
]
Ekanth S commented on HDFS-13232:
-
Thanks Wei and Inigo for the quick review. Updated the patch with the missing
test, extra check for number of connections and formatting.
> RBF: ConnectionPool should return first usable connection
> -
>
> Key: HDFS-13232
> URL: https://issues.apache.org/jira/browse/HDFS-13232
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Wei Yan
>Assignee: Ekanth S
>Priority: Minor
> Attachments: HDFS-13232.001.patch, HDFS-13232.002.patch
>
>
> In current ConnectionPool.getConnection(), it will return the first active
> connection:
> {code:java}
> for (int i=0; i int index = (threadIndex + i) % size;
> conn = tmpConnections.get(index);
> if (conn != null && !conn.isUsable()) {
> return conn;
> }
> }
> {code}
> Here "!conn.isUsable()" should be "conn.isUsable()".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13232) RBF: ConnectionPool should return first usable connection
[
https://issues.apache.org/jira/browse/HDFS-13232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekanth S updated HDFS-13232:
Attachment: HDFS-13232.002.patch
> RBF: ConnectionPool should return first usable connection
> -
>
> Key: HDFS-13232
> URL: https://issues.apache.org/jira/browse/HDFS-13232
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Wei Yan
>Assignee: Ekanth S
>Priority: Minor
> Attachments: HDFS-13232.001.patch, HDFS-13232.002.patch
>
>
> In current ConnectionPool.getConnection(), it will return the first active
> connection:
> {code:java}
> for (int i=0; i int index = (threadIndex + i) % size;
> conn = tmpConnections.get(index);
> if (conn != null && !conn.isUsable()) {
> return conn;
> }
> }
> {code}
> Here "!conn.isUsable()" should be "conn.isUsable()".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13232) RBF: ConnectionPool should return first usable connection
[
https://issues.apache.org/jira/browse/HDFS-13232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393636#comment-16393636
]
Ekanth S commented on HDFS-13232:
-
I have attached a patch for this issue with a test case. Let me know if you
have any suggestions.
> RBF: ConnectionPool should return first usable connection
> -
>
> Key: HDFS-13232
> URL: https://issues.apache.org/jira/browse/HDFS-13232
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Wei Yan
>Assignee: Ekanth S
>Priority: Minor
> Attachments: HDFS-13232.001.patch
>
>
> In current ConnectionPool.getConnection(), it will return the first active
> connection:
> {code:java}
> for (int i=0; i int index = (threadIndex + i) % size;
> conn = tmpConnections.get(index);
> if (conn != null && !conn.isUsable()) {
> return conn;
> }
> }
> {code}
> Here "!conn.isUsable()" should be "conn.isUsable()".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13232) RBF: ConnectionPool should return first usable connection
[
https://issues.apache.org/jira/browse/HDFS-13232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekanth S updated HDFS-13232:
Attachment: HDFS-13232.001.patch
> RBF: ConnectionPool should return first usable connection
> -
>
> Key: HDFS-13232
> URL: https://issues.apache.org/jira/browse/HDFS-13232
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Wei Yan
>Assignee: Ekanth S
>Priority: Minor
> Attachments: HDFS-13232.001.patch
>
>
> In current ConnectionPool.getConnection(), it will return the first active
> connection:
> {code:java}
> for (int i=0; i int index = (threadIndex + i) % size;
> conn = tmpConnections.get(index);
> if (conn != null && !conn.isUsable()) {
> return conn;
> }
> }
> {code}
> Here "!conn.isUsable()" should be "conn.isUsable()".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-13232) RBF: ConnectionPool should return first usable connection
[
https://issues.apache.org/jira/browse/HDFS-13232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekanth S reassigned HDFS-13232:
---
Assignee: Ekanth S (was: Wei Yan)
> RBF: ConnectionPool should return first usable connection
> -
>
> Key: HDFS-13232
> URL: https://issues.apache.org/jira/browse/HDFS-13232
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Wei Yan
>Assignee: Ekanth S
>Priority: Minor
>
> In current ConnectionPool.getConnection(), it will return the first active
> connection:
> {code:java}
> for (int i=0; i int index = (threadIndex + i) % size;
> conn = tmpConnections.get(index);
> if (conn != null && !conn.isUsable()) {
> return conn;
> }
> }
> {code}
> Here "!conn.isUsable()" should be "conn.isUsable()".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
