[jira] [Work logged] (HDDS-1891) Ozone fs shell command should work with default port when port number is not specified
[ https://issues.apache.org/jira/browse/HDDS-1891?focusedWorklogId=294259&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-294259 ] ASF GitHub Bot logged work on HDDS-1891: Author: ASF GitHub Bot Created on: 13/Aug/19 22:44 Start Date: 13/Aug/19 22:44 Worklog Time Spent: 10m Work Description: anuengineer commented on issue #1218: HDDS-1891. Ozone fs shell command should work with default port when port number is not specified URL: https://github.com/apache/hadoop/pull/1218#issuecomment-521036865 @smengcl Thanks for the contribution. @bharatviswa504 Thanks for the review, I have committed this to the trunk. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 294259) Time Spent: 3.5h (was: 3h 20m) > Ozone fs shell command should work with default port when port number is not > specified > -- > > Key: HDDS-1891 > URL: https://issues.apache.org/jira/browse/HDDS-1891 > Project: Hadoop Distributed Data Store > Issue Type: Improvement >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Time Spent: 3.5h > Remaining Estimate: 0h > > {code:bash|title=Without port number -> Error} > $ ozone fs -ls o3fs://bucket.volume.localhost/ > -ls: Ozone file system url should be either one of the two forms: > o3fs://bucket.volume/key OR o3fs://bucket.volume.om-host.example.com:5678/key > ... > {code} > {code:bash|title=With port number -> Success} > $ ozone fs -ls o3fs://bucket.volume.localhost:9862/ > Found 1 items > -rw-rw-rw- 1 hadoop hadoop 1485 2019-08-01 21:14 > o3fs://bucket.volume.localhost:9862/README.txt > {code} > We expect the first command to attempt port 9862 by default. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1891) Ozone fs shell command should work with default port when port number is not specified
[ https://issues.apache.org/jira/browse/HDDS-1891?focusedWorklogId=294260&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-294260 ] ASF GitHub Bot logged work on HDDS-1891: Author: ASF GitHub Bot Created on: 13/Aug/19 22:44 Start Date: 13/Aug/19 22:44 Worklog Time Spent: 10m Work Description: anuengineer commented on pull request #1218: HDDS-1891. Ozone fs shell command should work with default port when port number is not specified URL: https://github.com/apache/hadoop/pull/1218 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 294260) Time Spent: 3h 40m (was: 3.5h) > Ozone fs shell command should work with default port when port number is not > specified > -- > > Key: HDDS-1891 > URL: https://issues.apache.org/jira/browse/HDDS-1891 > Project: Hadoop Distributed Data Store > Issue Type: Improvement >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Time Spent: 3h 40m > Remaining Estimate: 0h > > {code:bash|title=Without port number -> Error} > $ ozone fs -ls o3fs://bucket.volume.localhost/ > -ls: Ozone file system url should be either one of the two forms: > o3fs://bucket.volume/key OR o3fs://bucket.volume.om-host.example.com:5678/key > ... > {code} > {code:bash|title=With port number -> Success} > $ ozone fs -ls o3fs://bucket.volume.localhost:9862/ > Found 1 items > -rw-rw-rw- 1 hadoop hadoop 1485 2019-08-01 21:14 > o3fs://bucket.volume.localhost:9862/README.txt > {code} > We expect the first command to attempt port 9862 by default. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1891) Ozone fs shell command should work with default port when port number is not specified
[ https://issues.apache.org/jira/browse/HDDS-1891?focusedWorklogId=292023&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-292023 ] ASF GitHub Bot logged work on HDDS-1891: Author: ASF GitHub Bot Created on: 09/Aug/19 12:15 Start Date: 09/Aug/19 12:15 Worklog Time Spent: 10m Work Description: smengcl commented on issue #1218: HDDS-1891. Ozone fs shell command should work with default port when port number is not specified URL: https://github.com/apache/hadoop/pull/1218#issuecomment-519896335 Thanks for the comment @adoroszlai ! Just posted a new commit to address both checkstyle issues. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 292023) Time Spent: 3h 20m (was: 3h 10m) > Ozone fs shell command should work with default port when port number is not > specified > -- > > Key: HDDS-1891 > URL: https://issues.apache.org/jira/browse/HDDS-1891 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Time Spent: 3h 20m > Remaining Estimate: 0h > > {code:bash|title=Without port number -> Error} > $ ozone fs -ls o3fs://bucket.volume.localhost/ > -ls: Ozone file system url should be either one of the two forms: > o3fs://bucket.volume/key OR o3fs://bucket.volume.om-host.example.com:5678/key > ... > {code} > {code:bash|title=With port number -> Success} > $ ozone fs -ls o3fs://bucket.volume.localhost:9862/ > Found 1 items > -rw-rw-rw- 1 hadoop hadoop 1485 2019-08-01 21:14 > o3fs://bucket.volume.localhost:9862/README.txt > {code} > We expect the first command to attempt port 9862 by default. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1891) Ozone fs shell command should work with default port when port number is not specified
[ https://issues.apache.org/jira/browse/HDDS-1891?focusedWorklogId=292021&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-292021 ] ASF GitHub Bot logged work on HDDS-1891: Author: ASF GitHub Bot Created on: 09/Aug/19 12:14 Start Date: 09/Aug/19 12:14 Worklog Time Spent: 10m Work Description: smengcl commented on pull request #1218: HDDS-1891. Ozone fs shell command should work with default port when port number is not specified URL: https://github.com/apache/hadoop/pull/1218#discussion_r312451547 ## File path: hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystemWithMocks.java ## @@ -78,6 +79,42 @@ public void testFSUriWithHostPortOverrides() throws Exception { OzoneClientFactory.getRpcClient("local.host", 5899, conf); } + @Test + public void testFSUriWithHostPortUnspecified() throws Exception { +Configuration conf = new OzoneConfiguration(); +final int OM_PORT_DEFAULT = OmUtils.getOmRpcPort(conf); Review comment: Done. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 292021) Time Spent: 3h 10m (was: 3h) > Ozone fs shell command should work with default port when port number is not > specified > -- > > Key: HDDS-1891 > URL: https://issues.apache.org/jira/browse/HDDS-1891 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Time Spent: 3h 10m > Remaining Estimate: 0h > > {code:bash|title=Without port number -> Error} > $ ozone fs -ls o3fs://bucket.volume.localhost/ > -ls: Ozone file system url should be either one of the two forms: > o3fs://bucket.volume/key OR o3fs://bucket.volume.om-host.example.com:5678/key > ... > {code} > {code:bash|title=With port number -> Success} > $ ozone fs -ls o3fs://bucket.volume.localhost:9862/ > Found 1 items > -rw-rw-rw- 1 hadoop hadoop 1485 2019-08-01 21:14 > o3fs://bucket.volume.localhost:9862/README.txt > {code} > We expect the first command to attempt port 9862 by default. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1891) Ozone fs shell command should work with default port when port number is not specified
[ https://issues.apache.org/jira/browse/HDDS-1891?focusedWorklogId=292020&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-292020 ] ASF GitHub Bot logged work on HDDS-1891: Author: ASF GitHub Bot Created on: 09/Aug/19 12:14 Start Date: 09/Aug/19 12:14 Worklog Time Spent: 10m Work Description: smengcl commented on pull request #1218: HDDS-1891. Ozone fs shell command should work with default port when port number is not specified URL: https://github.com/apache/hadoop/pull/1218#discussion_r312451506 ## File path: hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/BasicOzoneFileSystem.java ## @@ -54,6 +55,8 @@ import static org.apache.hadoop.fs.ozone.Constants.OZONE_USER_DIR; import static org.apache.hadoop.ozone.OzoneConsts.OZONE_URI_DELIMITER; import static org.apache.hadoop.ozone.OzoneConsts.OZONE_URI_SCHEME; +import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_PORT_DEFAULT; Review comment: Done. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 292020) Time Spent: 3h (was: 2h 50m) > Ozone fs shell command should work with default port when port number is not > specified > -- > > Key: HDDS-1891 > URL: https://issues.apache.org/jira/browse/HDDS-1891 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Time Spent: 3h > Remaining Estimate: 0h > > {code:bash|title=Without port number -> Error} > $ ozone fs -ls o3fs://bucket.volume.localhost/ > -ls: Ozone file system url should be either one of the two forms: > o3fs://bucket.volume/key OR o3fs://bucket.volume.om-host.example.com:5678/key > ... > {code} > {code:bash|title=With port number -> Success} > $ ozone fs -ls o3fs://bucket.volume.localhost:9862/ > Found 1 items > -rw-rw-rw- 1 hadoop hadoop 1485 2019-08-01 21:14 > o3fs://bucket.volume.localhost:9862/README.txt > {code} > We expect the first command to attempt port 9862 by default. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1891) Ozone fs shell command should work with default port when port number is not specified
[ https://issues.apache.org/jira/browse/HDDS-1891?focusedWorklogId=291969&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-291969 ] ASF GitHub Bot logged work on HDDS-1891: Author: ASF GitHub Bot Created on: 09/Aug/19 10:06 Start Date: 09/Aug/19 10:06 Worklog Time Spent: 10m Work Description: adoroszlai commented on pull request #1218: HDDS-1891. Ozone fs shell command should work with default port when port number is not specified URL: https://github.com/apache/hadoop/pull/1218#discussion_r312382889 ## File path: hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/BasicOzoneFileSystem.java ## @@ -54,6 +55,8 @@ import static org.apache.hadoop.fs.ozone.Constants.OZONE_USER_DIR; import static org.apache.hadoop.ozone.OzoneConsts.OZONE_URI_DELIMITER; import static org.apache.hadoop.ozone.OzoneConsts.OZONE_URI_SCHEME; +import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_PORT_DEFAULT; Review comment: Unused import, checkstyle will complain. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 291969) Time Spent: 2h 40m (was: 2.5h) > Ozone fs shell command should work with default port when port number is not > specified > -- > > Key: HDDS-1891 > URL: https://issues.apache.org/jira/browse/HDDS-1891 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > > {code:bash|title=Without port number -> Error} > $ ozone fs -ls o3fs://bucket.volume.localhost/ > -ls: Ozone file system url should be either one of the two forms: > o3fs://bucket.volume/key OR o3fs://bucket.volume.om-host.example.com:5678/key > ... > {code} > {code:bash|title=With port number -> Success} > $ ozone fs -ls o3fs://bucket.volume.localhost:9862/ > Found 1 items > -rw-rw-rw- 1 hadoop hadoop 1485 2019-08-01 21:14 > o3fs://bucket.volume.localhost:9862/README.txt > {code} > We expect the first command to attempt port 9862 by default. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1891) Ozone fs shell command should work with default port when port number is not specified
[ https://issues.apache.org/jira/browse/HDDS-1891?focusedWorklogId=291970&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-291970 ] ASF GitHub Bot logged work on HDDS-1891: Author: ASF GitHub Bot Created on: 09/Aug/19 10:06 Start Date: 09/Aug/19 10:06 Worklog Time Spent: 10m Work Description: adoroszlai commented on pull request #1218: HDDS-1891. Ozone fs shell command should work with default port when port number is not specified URL: https://github.com/apache/hadoop/pull/1218#discussion_r312386384 ## File path: hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystemWithMocks.java ## @@ -78,6 +79,42 @@ public void testFSUriWithHostPortOverrides() throws Exception { OzoneClientFactory.getRpcClient("local.host", 5899, conf); } + @Test + public void testFSUriWithHostPortUnspecified() throws Exception { +Configuration conf = new OzoneConfiguration(); +final int OM_PORT_DEFAULT = OmUtils.getOmRpcPort(conf); Review comment: Also from checkstyle: `Name 'OM_PORT_DEFAULT' must match pattern '^[a-z][a-zA-Z0-9]*$'` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 291970) Time Spent: 2h 50m (was: 2h 40m) > Ozone fs shell command should work with default port when port number is not > specified > -- > > Key: HDDS-1891 > URL: https://issues.apache.org/jira/browse/HDDS-1891 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Time Spent: 2h 50m > Remaining Estimate: 0h > > {code:bash|title=Without port number -> Error} > $ ozone fs -ls o3fs://bucket.volume.localhost/ > -ls: Ozone file system url should be either one of the two forms: > o3fs://bucket.volume/key OR o3fs://bucket.volume.om-host.example.com:5678/key > ... > {code} > {code:bash|title=With port number -> Success} > $ ozone fs -ls o3fs://bucket.volume.localhost:9862/ > Found 1 items > -rw-rw-rw- 1 hadoop hadoop 1485 2019-08-01 21:14 > o3fs://bucket.volume.localhost:9862/README.txt > {code} > We expect the first command to attempt port 9862 by default. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1891) Ozone fs shell command should work with default port when port number is not specified
[ https://issues.apache.org/jira/browse/HDDS-1891?focusedWorklogId=291735&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-291735 ] ASF GitHub Bot logged work on HDDS-1891: Author: ASF GitHub Bot Created on: 09/Aug/19 03:06 Start Date: 09/Aug/19 03:06 Worklog Time Spent: 10m Work Description: smengcl commented on issue #1218: HDDS-1891. Ozone fs shell command should work with default port when port number is not specified URL: https://github.com/apache/hadoop/pull/1218#issuecomment-519760099 /retest This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 291735) Time Spent: 2.5h (was: 2h 20m) > Ozone fs shell command should work with default port when port number is not > specified > -- > > Key: HDDS-1891 > URL: https://issues.apache.org/jira/browse/HDDS-1891 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Time Spent: 2.5h > Remaining Estimate: 0h > > {code:bash|title=Without port number -> Error} > $ ozone fs -ls o3fs://bucket.volume.localhost/ > -ls: Ozone file system url should be either one of the two forms: > o3fs://bucket.volume/key OR o3fs://bucket.volume.om-host.example.com:5678/key > ... > {code} > {code:bash|title=With port number -> Success} > $ ozone fs -ls o3fs://bucket.volume.localhost:9862/ > Found 1 items > -rw-rw-rw- 1 hadoop hadoop 1485 2019-08-01 21:14 > o3fs://bucket.volume.localhost:9862/README.txt > {code} > We expect the first command to attempt port 9862 by default. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1891) Ozone fs shell command should work with default port when port number is not specified
[ https://issues.apache.org/jira/browse/HDDS-1891?focusedWorklogId=291734&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-291734 ] ASF GitHub Bot logged work on HDDS-1891: Author: ASF GitHub Bot Created on: 09/Aug/19 03:06 Start Date: 09/Aug/19 03:06 Worklog Time Spent: 10m Work Description: smengcl commented on issue #1218: HDDS-1891. Ozone fs shell command should work with default port when port number is not specified URL: https://github.com/apache/hadoop/pull/1218#issuecomment-519760075 Fixed the code and the unit test. The following fs commands all work as I tested in docker: ```bash ozone fs -put README.txt o3fs:/// ozone fs -ls / ozone fs -ls o3fs://bucket.volume.om/ ozone fs -ls o3fs://bucket.volume.om:9862/ ozone fs -ls o3fs://bucket.volume/ ozone fs -get /README.txt R.txt ozone fs -get o3fs://bucket.volume.om/README.txt R2.txt ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 291734) Time Spent: 2h 20m (was: 2h 10m) > Ozone fs shell command should work with default port when port number is not > specified > -- > > Key: HDDS-1891 > URL: https://issues.apache.org/jira/browse/HDDS-1891 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > {code:bash|title=Without port number -> Error} > $ ozone fs -ls o3fs://bucket.volume.localhost/ > -ls: Ozone file system url should be either one of the two forms: > o3fs://bucket.volume/key OR o3fs://bucket.volume.om-host.example.com:5678/key > ... > {code} > {code:bash|title=With port number -> Success} > $ ozone fs -ls o3fs://bucket.volume.localhost:9862/ > Found 1 items > -rw-rw-rw- 1 hadoop hadoop 1485 2019-08-01 21:14 > o3fs://bucket.volume.localhost:9862/README.txt > {code} > We expect the first command to attempt port 9862 by default. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1891) Ozone fs shell command should work with default port when port number is not specified
[ https://issues.apache.org/jira/browse/HDDS-1891?focusedWorklogId=291684&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-291684 ] ASF GitHub Bot logged work on HDDS-1891: Author: ASF GitHub Bot Created on: 09/Aug/19 00:51 Start Date: 09/Aug/19 00:51 Worklog Time Spent: 10m Work Description: smengcl commented on issue #1218: HDDS-1891. Ozone fs shell command should work with default port when port number is not specified URL: https://github.com/apache/hadoop/pull/1218#issuecomment-519737497 > > One more comment, I think we should make a similar change in BasicOzoneFileSystem.java. > > See, can we have a some utility method, which can be used across 2 classes. > > OzoneFileSystem extends BasicOzoneFileSystem, so this has been taken care. Just test it out locally. While I'm testing it in a docker-compose cluster, it seems that the change of `authority` [here](https://github.com/apache/hadoop/pull/1218/commits/ab93f4bc3fd8d2acf31dac5bbf79c49546eccacd#diff-e48c4ce6b86d4a33c3f38ba8d6d06ea3R129) might have broken something. It throws error: ``` bash-4.2$ ozone fs -ls o3fs://bucket.volume.om/ -ls: Wrong FS: o3fs://bucket.volume.om/, expected: o3fs://bucket.volume.om:9862 ... ``` After I commented out this line, compiled and re-tested, it works again: ``` bash-4.2$ ozone fs -ls o3fs://bucket.volume.om/ Found 1 items -rw-rw-rw- 1 hadoop hadoop 1485 1970-01-01 00:46 o3fs://bucket.volume.om/README.txt ``` FYI my test prep steps: ```bash mvn clean install -f pom.ozone.xml -DskipTests=true -Dmaven.javadoc.skip=true cd hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/compose/ozone docker-compose up -d --scale datanode=3 docker-compose exec om /bin/bash ozone sh volume create /volume ozone sh bucket create /volume/bucket vi /etc/hadoop/core-site.xml # Add fs.o3fs.impl and fs.defaultFS config and save # Ref: https://hadoop.apache.org/ozone/docs/0.4.0-alpha/ozonefs.html ozone fs -put README.txt o3fs:/// ozone fs -ls / ``` Investigating. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 291684) Time Spent: 2h 10m (was: 2h) > Ozone fs shell command should work with default port when port number is not > specified > -- > > Key: HDDS-1891 > URL: https://issues.apache.org/jira/browse/HDDS-1891 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > {code:bash|title=Without port number -> Error} > $ ozone fs -ls o3fs://bucket.volume.localhost/ > -ls: Ozone file system url should be either one of the two forms: > o3fs://bucket.volume/key OR o3fs://bucket.volume.om-host.example.com:5678/key > ... > {code} > {code:bash|title=With port number -> Success} > $ ozone fs -ls o3fs://bucket.volume.localhost:9862/ > Found 1 items > -rw-rw-rw- 1 hadoop hadoop 1485 2019-08-01 21:14 > o3fs://bucket.volume.localhost:9862/README.txt > {code} > We expect the first command to attempt port 9862 by default. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1891) Ozone fs shell command should work with default port when port number is not specified
[ https://issues.apache.org/jira/browse/HDDS-1891?focusedWorklogId=291683&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-291683 ] ASF GitHub Bot logged work on HDDS-1891: Author: ASF GitHub Bot Created on: 09/Aug/19 00:50 Start Date: 09/Aug/19 00:50 Worklog Time Spent: 10m Work Description: smengcl commented on issue #1218: HDDS-1891. Ozone fs shell command should work with default port when port number is not specified URL: https://github.com/apache/hadoop/pull/1218#issuecomment-519737497 > > One more comment, I think we should make a similar change in BasicOzoneFileSystem.java. > > See, can we have a some utility method, which can be used across 2 classes. > > OzoneFileSystem extends BasicOzoneFileSystem, so this has been taken care. Just test it out locally. While I'm testing it in a docker-compose cluster, it seems that the change of `authority` [here](https://github.com/apache/hadoop/pull/1218/commits/ab93f4bc3fd8d2acf31dac5bbf79c49546eccacd#diff-e48c4ce6b86d4a33c3f38ba8d6d06ea3R129) might have broken something. It throws error: ``` bash-4.2$ ozone fs -ls o3fs://bucket.volume.om/ -ls: Wrong FS: o3fs://bucket.volume.om/, expected: o3fs://bucket.volume.om:9862 ... ``` After I commented out this line, compile and re-test, it works again: ``` bash-4.2$ ozone fs -ls o3fs://bucket.volume.om/ Found 1 items -rw-rw-rw- 1 hadoop hadoop 1485 1970-01-01 00:46 o3fs://bucket.volume.om/README.txt ``` Investigating. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 291683) Time Spent: 2h (was: 1h 50m) > Ozone fs shell command should work with default port when port number is not > specified > -- > > Key: HDDS-1891 > URL: https://issues.apache.org/jira/browse/HDDS-1891 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Time Spent: 2h > Remaining Estimate: 0h > > {code:bash|title=Without port number -> Error} > $ ozone fs -ls o3fs://bucket.volume.localhost/ > -ls: Ozone file system url should be either one of the two forms: > o3fs://bucket.volume/key OR o3fs://bucket.volume.om-host.example.com:5678/key > ... > {code} > {code:bash|title=With port number -> Success} > $ ozone fs -ls o3fs://bucket.volume.localhost:9862/ > Found 1 items > -rw-rw-rw- 1 hadoop hadoop 1485 2019-08-01 21:14 > o3fs://bucket.volume.localhost:9862/README.txt > {code} > We expect the first command to attempt port 9862 by default. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1891) Ozone fs shell command should work with default port when port number is not specified
[ https://issues.apache.org/jira/browse/HDDS-1891?focusedWorklogId=291682&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-291682 ] ASF GitHub Bot logged work on HDDS-1891: Author: ASF GitHub Bot Created on: 09/Aug/19 00:50 Start Date: 09/Aug/19 00:50 Worklog Time Spent: 10m Work Description: smengcl commented on issue #1218: HDDS-1891. Ozone fs shell command should work with default port when port number is not specified URL: https://github.com/apache/hadoop/pull/1218#issuecomment-519737497 > > One more comment, I think we should make a similar change in BasicOzoneFileSystem.java. > > See, can we have a some utility method, which can be used across 2 classes. > > OzoneFileSystem extends BasicOzoneFileSystem, so this has been taken care. Just test it out locally. While I'm testing it in a docker-compose cluster, it seems that the change of `authority` [here](https://github.com/apache/hadoop/pull/1218/commits/ab93f4bc3fd8d2acf31dac5bbf79c49546eccacd#diff-e48c4ce6b86d4a33c3f38ba8d6d06ea3R129) might have broken something. It throws error: {code} bash-4.2$ ozone fs -ls o3fs://bucket.volume.om/ -ls: Wrong FS: o3fs://bucket.volume.om/, expected: o3fs://bucket.volume.om:9862 ... {code} After I commented out this line, compile and re-test, it works again: {code} bash-4.2$ ozone fs -ls o3fs://bucket.volume.om/ Found 1 items -rw-rw-rw- 1 hadoop hadoop 1485 1970-01-01 00:46 o3fs://bucket.volume.om/README.txt {code} Investigating. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 291682) Time Spent: 1h 50m (was: 1h 40m) > Ozone fs shell command should work with default port when port number is not > specified > -- > > Key: HDDS-1891 > URL: https://issues.apache.org/jira/browse/HDDS-1891 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > {code:bash|title=Without port number -> Error} > $ ozone fs -ls o3fs://bucket.volume.localhost/ > -ls: Ozone file system url should be either one of the two forms: > o3fs://bucket.volume/key OR o3fs://bucket.volume.om-host.example.com:5678/key > ... > {code} > {code:bash|title=With port number -> Success} > $ ozone fs -ls o3fs://bucket.volume.localhost:9862/ > Found 1 items > -rw-rw-rw- 1 hadoop hadoop 1485 2019-08-01 21:14 > o3fs://bucket.volume.localhost:9862/README.txt > {code} > We expect the first command to attempt port 9862 by default. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1891) Ozone fs shell command should work with default port when port number is not specified
[ https://issues.apache.org/jira/browse/HDDS-1891?focusedWorklogId=291651&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-291651 ] ASF GitHub Bot logged work on HDDS-1891: Author: ASF GitHub Bot Created on: 08/Aug/19 23:18 Start Date: 08/Aug/19 23:18 Worklog Time Spent: 10m Work Description: smengcl commented on pull request #1218: HDDS-1891. Ozone fs shell command should work with default port when port number is not specified URL: https://github.com/apache/hadoop/pull/1218#discussion_r312280716 ## File path: hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/BasicOzoneFileSystem.java ## @@ -113,11 +115,14 @@ public void initialize(URI name, Configuration conf) throws IOException { String omPort = String.valueOf(-1); if (!isEmpty(remaining)) { String[] parts = remaining.split(":"); - if (parts.length != 2) { + // Array length should only be 1 or 2 + if (parts.length > 2) { throw new IllegalArgumentException(URI_EXCEPTION_TEXT); Review comment: Done. Updated `URI_EXCEPTION_TEXT`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 291651) Time Spent: 1h 40m (was: 1.5h) > Ozone fs shell command should work with default port when port number is not > specified > -- > > Key: HDDS-1891 > URL: https://issues.apache.org/jira/browse/HDDS-1891 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > {code:bash|title=Without port number -> Error} > $ ozone fs -ls o3fs://bucket.volume.localhost/ > -ls: Ozone file system url should be either one of the two forms: > o3fs://bucket.volume/key OR o3fs://bucket.volume.om-host.example.com:5678/key > ... > {code} > {code:bash|title=With port number -> Success} > $ ozone fs -ls o3fs://bucket.volume.localhost:9862/ > Found 1 items > -rw-rw-rw- 1 hadoop hadoop 1485 2019-08-01 21:14 > o3fs://bucket.volume.localhost:9862/README.txt > {code} > We expect the first command to attempt port 9862 by default. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1891) Ozone fs shell command should work with default port when port number is not specified
[ https://issues.apache.org/jira/browse/HDDS-1891?focusedWorklogId=291640&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-291640 ] ASF GitHub Bot logged work on HDDS-1891: Author: ASF GitHub Bot Created on: 08/Aug/19 23:13 Start Date: 08/Aug/19 23:13 Worklog Time Spent: 10m Work Description: smengcl commented on pull request #1218: HDDS-1891. Ozone fs shell command should work with default port when port number is not specified URL: https://github.com/apache/hadoop/pull/1218#discussion_r312279650 ## File path: hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/BasicOzoneFileSystem.java ## @@ -113,11 +115,14 @@ public void initialize(URI name, Configuration conf) throws IOException { String omPort = String.valueOf(-1); if (!isEmpty(remaining)) { String[] parts = remaining.split(":"); - if (parts.length != 2) { + // Array length should only be 1 or 2 + if (parts.length > 2) { throw new IllegalArgumentException(URI_EXCEPTION_TEXT); Review comment: On it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 291640) Time Spent: 1.5h (was: 1h 20m) > Ozone fs shell command should work with default port when port number is not > specified > -- > > Key: HDDS-1891 > URL: https://issues.apache.org/jira/browse/HDDS-1891 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > {code:bash|title=Without port number -> Error} > $ ozone fs -ls o3fs://bucket.volume.localhost/ > -ls: Ozone file system url should be either one of the two forms: > o3fs://bucket.volume/key OR o3fs://bucket.volume.om-host.example.com:5678/key > ... > {code} > {code:bash|title=With port number -> Success} > $ ozone fs -ls o3fs://bucket.volume.localhost:9862/ > Found 1 items > -rw-rw-rw- 1 hadoop hadoop 1485 2019-08-01 21:14 > o3fs://bucket.volume.localhost:9862/README.txt > {code} > We expect the first command to attempt port 9862 by default. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1891) Ozone fs shell command should work with default port when port number is not specified
[ https://issues.apache.org/jira/browse/HDDS-1891?focusedWorklogId=291639&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-291639 ] ASF GitHub Bot logged work on HDDS-1891: Author: ASF GitHub Bot Created on: 08/Aug/19 23:10 Start Date: 08/Aug/19 23:10 Worklog Time Spent: 10m Work Description: smengcl commented on pull request #1218: HDDS-1891. Ozone fs shell command should work with default port when port number is not specified URL: https://github.com/apache/hadoop/pull/1218#discussion_r312278971 ## File path: hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/BasicOzoneFileSystem.java ## @@ -113,11 +115,14 @@ public void initialize(URI name, Configuration conf) throws IOException { String omPort = String.valueOf(-1); if (!isEmpty(remaining)) { String[] parts = remaining.split(":"); - if (parts.length != 2) { + // Array length should only be 1 or 2 + if (parts.length > 2) { throw new IllegalArgumentException(URI_EXCEPTION_TEXT); } omHost = parts[0]; - omPort = parts[1]; + // If port number is not specified, try default OM port + omPort = parts.length == 2 ? + parts[1] : String.valueOf(OZONE_OM_PORT_DEFAULT); Review comment: Updated. I'm using `OmUtils.getOmRpcPort(conf)` instead. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 291639) Time Spent: 1h 20m (was: 1h 10m) > Ozone fs shell command should work with default port when port number is not > specified > -- > > Key: HDDS-1891 > URL: https://issues.apache.org/jira/browse/HDDS-1891 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > {code:bash|title=Without port number -> Error} > $ ozone fs -ls o3fs://bucket.volume.localhost/ > -ls: Ozone file system url should be either one of the two forms: > o3fs://bucket.volume/key OR o3fs://bucket.volume.om-host.example.com:5678/key > ... > {code} > {code:bash|title=With port number -> Success} > $ ozone fs -ls o3fs://bucket.volume.localhost:9862/ > Found 1 items > -rw-rw-rw- 1 hadoop hadoop 1485 2019-08-01 21:14 > o3fs://bucket.volume.localhost:9862/README.txt > {code} > We expect the first command to attempt port 9862 by default. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1891) Ozone fs shell command should work with default port when port number is not specified
[ https://issues.apache.org/jira/browse/HDDS-1891?focusedWorklogId=291576&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-291576 ] ASF GitHub Bot logged work on HDDS-1891: Author: ASF GitHub Bot Created on: 08/Aug/19 21:06 Start Date: 08/Aug/19 21:06 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on issue #1218: HDDS-1891. Ozone fs shell command should work with default port when port number is not specified URL: https://github.com/apache/hadoop/pull/1218#issuecomment-519687588 > One more comment, I think we should make a similar change in BasicOzoneFileSystem.java. > > See, can we have a some utility method, which can be used across 2 classes. OzoneFileSystem extends BasicOzoneFileSystem, so this has been taken care. Just test it out locally. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 291576) Time Spent: 1h 10m (was: 1h) > Ozone fs shell command should work with default port when port number is not > specified > -- > > Key: HDDS-1891 > URL: https://issues.apache.org/jira/browse/HDDS-1891 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > {code:bash|title=Without port number -> Error} > $ ozone fs -ls o3fs://bucket.volume.localhost/ > -ls: Ozone file system url should be either one of the two forms: > o3fs://bucket.volume/key OR o3fs://bucket.volume.om-host.example.com:5678/key > ... > {code} > {code:bash|title=With port number -> Success} > $ ozone fs -ls o3fs://bucket.volume.localhost:9862/ > Found 1 items > -rw-rw-rw- 1 hadoop hadoop 1485 2019-08-01 21:14 > o3fs://bucket.volume.localhost:9862/README.txt > {code} > We expect the first command to attempt port 9862 by default. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1891) Ozone fs shell command should work with default port when port number is not specified
[ https://issues.apache.org/jira/browse/HDDS-1891?focusedWorklogId=291161&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-291161 ] ASF GitHub Bot logged work on HDDS-1891: Author: ASF GitHub Bot Created on: 08/Aug/19 10:53 Start Date: 08/Aug/19 10:53 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1218: HDDS-1891. Ozone fs shell command should work with default port when port number is not specified URL: https://github.com/apache/hadoop/pull/1218#issuecomment-519469774 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 74 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 624 | trunk passed | | +1 | compile | 387 | trunk passed | | +1 | checkstyle | 78 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 945 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 174 | trunk passed | | 0 | spotbugs | 494 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 722 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 576 | the patch passed | | +1 | compile | 381 | the patch passed | | +1 | javac | 381 | the patch passed | | +1 | checkstyle | 78 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 1 | The patch has no whitespace issues. | | +1 | shadedclient | 749 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 178 | the patch passed | | +1 | findbugs | 696 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 333 | hadoop-hdds in the patch passed. | | -1 | unit | 2173 | hadoop-ozone in the patch failed. | | +1 | asflicense | 43 | The patch does not generate ASF License warnings. | | | | 8355 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.client.rpc.TestOzoneRpcClient | | | hadoop.ozone.client.rpc.TestMultiBlockWritesWithDnFailures | | | hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures | | | hadoop.ozone.client.rpc.TestOzoneAtRestEncryption | | | hadoop.ozone.client.rpc.TestOzoneClientRetriesOnException | | | hadoop.ozone.client.rpc.TestSecureOzoneRpcClient | | | hadoop.ozone.om.TestKeyManagerImpl | | | hadoop.ozone.om.TestScmSafeMode | | | hadoop.hdds.scm.pipeline.TestRatisPipelineProvider | | | hadoop.ozone.client.rpc.TestCommitWatcher | | | hadoop.hdds.scm.pipeline.TestRatisPipelineCreateAndDestory | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1218/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1218 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux ff2368d64ba7 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 00b5a27 | | Default Java | 1.8.0_222 | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1218/2/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1218/2/testReport/ | | Max. process+thread count | 5222 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/ozonefs U: hadoop-ozone/ozonefs | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1218/2/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 291161) Time Spent: 1h (was: 50m) > Ozone fs shell command should work with default port when port number is not > specified >
[jira] [Work logged] (HDDS-1891) Ozone fs shell command should work with default port when port number is not specified
[ https://issues.apache.org/jira/browse/HDDS-1891?focusedWorklogId=290916&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-290916 ] ASF GitHub Bot logged work on HDDS-1891: Author: ASF GitHub Bot Created on: 08/Aug/19 01:29 Start Date: 08/Aug/19 01:29 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1218: HDDS-1891. Ozone fs shell command should work with default port when port number is not specified URL: https://github.com/apache/hadoop/pull/1218#issuecomment-519327221 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 44 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 594 | trunk passed | | +1 | compile | 377 | trunk passed | | +1 | checkstyle | 74 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 777 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 146 | trunk passed | | 0 | spotbugs | 415 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 620 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 526 | the patch passed | | +1 | compile | 350 | the patch passed | | +1 | javac | 350 | the patch passed | | +1 | checkstyle | 69 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 645 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 164 | the patch passed | | -1 | findbugs | 353 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | +1 | unit | 295 | hadoop-hdds in the patch passed. | | -1 | unit | 340 | hadoop-ozone in the patch failed. | | +1 | asflicense | 34 | The patch does not generate ASF License warnings. | | | | 5750 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1218/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1218 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux de9c20ca4927 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 3cc0ace | | Default Java | 1.8.0_212 | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1218/1/artifact/out/patch-findbugs-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1218/1/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1218/1/testReport/ | | Max. process+thread count | 1302 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/ozonefs U: hadoop-ozone/ozonefs | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1218/1/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 290916) Time Spent: 50m (was: 40m) > Ozone fs shell command should work with default port when port number is not > specified > -- > > Key: HDDS-1891 > URL: https://issues.apache.org/jira/browse/HDDS-1891 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > {code:bash|title=Without port number -> Error} > $ ozone fs -ls o3fs://bucket.volume.localhost/ > -ls: Ozone file s
[jira] [Work logged] (HDDS-1891) Ozone fs shell command should work with default port when port number is not specified
[ https://issues.apache.org/jira/browse/HDDS-1891?focusedWorklogId=288367&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-288367 ] ASF GitHub Bot logged work on HDDS-1891: Author: ASF GitHub Bot Created on: 03/Aug/19 02:30 Start Date: 03/Aug/19 02:30 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on issue #1218: HDDS-1891. Ozone fs shell command should work with default port when port number is not specified URL: https://github.com/apache/hadoop/pull/1218#issuecomment-517887376 One more comment, I think we should make a similar change in BasicOzoneFileSystem.java. See, can we have a some utility method, which can be used across 2 classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 288367) Time Spent: 40m (was: 0.5h) > Ozone fs shell command should work with default port when port number is not > specified > -- > > Key: HDDS-1891 > URL: https://issues.apache.org/jira/browse/HDDS-1891 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > {code:bash|title=Without port number -> Error} > $ ozone fs -ls o3fs://bucket.volume.localhost/ > -ls: Ozone file system url should be either one of the two forms: > o3fs://bucket.volume/key OR o3fs://bucket.volume.om-host.example.com:5678/key > ... > {code} > {code:bash|title=With port number -> Success} > $ ozone fs -ls o3fs://bucket.volume.localhost:9862/ > Found 1 items > -rw-rw-rw- 1 hadoop hadoop 1485 2019-08-01 21:14 > o3fs://bucket.volume.localhost:9862/README.txt > {code} > We expect the first command to attempt port 9862 by default. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1891) Ozone fs shell command should work with default port when port number is not specified
[ https://issues.apache.org/jira/browse/HDDS-1891?focusedWorklogId=288322&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-288322 ] ASF GitHub Bot logged work on HDDS-1891: Author: ASF GitHub Bot Created on: 02/Aug/19 23:09 Start Date: 02/Aug/19 23:09 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on pull request #1218: HDDS-1891. Ozone fs shell command should work with default port when port number is not specified URL: https://github.com/apache/hadoop/pull/1218#discussion_r310322479 ## File path: hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/BasicOzoneFileSystem.java ## @@ -113,11 +115,14 @@ public void initialize(URI name, Configuration conf) throws IOException { String omPort = String.valueOf(-1); if (!isEmpty(remaining)) { String[] parts = remaining.split(":"); - if (parts.length != 2) { + // Array length should only be 1 or 2 + if (parts.length > 2) { throw new IllegalArgumentException(URI_EXCEPTION_TEXT); } omHost = parts[0]; - omPort = parts[1]; + // If port number is not specified, try default OM port + omPort = parts.length == 2 ? + parts[1] : String.valueOf(OZONE_OM_PORT_DEFAULT); Review comment: Here do we need to use configuration.get(OZONE_OM_PORT, OZONE_OM_PORT_DEFAULT). To support a case for reading default value from configuration. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 288322) Time Spent: 0.5h (was: 20m) > Ozone fs shell command should work with default port when port number is not > specified > -- > > Key: HDDS-1891 > URL: https://issues.apache.org/jira/browse/HDDS-1891 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > {code:bash|title=Without port number -> Error} > $ ozone fs -ls o3fs://bucket.volume.localhost/ > -ls: Ozone file system url should be either one of the two forms: > o3fs://bucket.volume/key OR o3fs://bucket.volume.om-host.example.com:5678/key > ... > {code} > {code:bash|title=With port number -> Success} > $ ozone fs -ls o3fs://bucket.volume.localhost:9862/ > Found 1 items > -rw-rw-rw- 1 hadoop hadoop 1485 2019-08-01 21:14 > o3fs://bucket.volume.localhost:9862/README.txt > {code} > We expect the first command to attempt port 9862 by default. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1891) Ozone fs shell command should work with default port when port number is not specified
[ https://issues.apache.org/jira/browse/HDDS-1891?focusedWorklogId=288321&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-288321 ] ASF GitHub Bot logged work on HDDS-1891: Author: ASF GitHub Bot Created on: 02/Aug/19 23:08 Start Date: 02/Aug/19 23:08 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on pull request #1218: HDDS-1891. Ozone fs shell command should work with default port when port number is not specified URL: https://github.com/apache/hadoop/pull/1218#discussion_r310322305 ## File path: hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/BasicOzoneFileSystem.java ## @@ -113,11 +115,14 @@ public void initialize(URI name, Configuration conf) throws IOException { String omPort = String.valueOf(-1); if (!isEmpty(remaining)) { String[] parts = remaining.split(":"); - if (parts.length != 2) { + // Array length should only be 1 or 2 + if (parts.length > 2) { throw new IllegalArgumentException(URI_EXCEPTION_TEXT); Review comment: We need to update the URI_EXCEPTION_TEXT to add about the new support of o3fs://bucket.volume:om-host. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 288321) Time Spent: 20m (was: 10m) > Ozone fs shell command should work with default port when port number is not > specified > -- > > Key: HDDS-1891 > URL: https://issues.apache.org/jira/browse/HDDS-1891 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > {code:bash|title=Without port number -> Error} > $ ozone fs -ls o3fs://bucket.volume.localhost/ > -ls: Ozone file system url should be either one of the two forms: > o3fs://bucket.volume/key OR o3fs://bucket.volume.om-host.example.com:5678/key > ... > {code} > {code:bash|title=With port number -> Success} > $ ozone fs -ls o3fs://bucket.volume.localhost:9862/ > Found 1 items > -rw-rw-rw- 1 hadoop hadoop 1485 2019-08-01 21:14 > o3fs://bucket.volume.localhost:9862/README.txt > {code} > We expect the first command to attempt port 9862 by default. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1891) Ozone fs shell command should work with default port when port number is not specified
[ https://issues.apache.org/jira/browse/HDDS-1891?focusedWorklogId=288209&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-288209 ] ASF GitHub Bot logged work on HDDS-1891: Author: ASF GitHub Bot Created on: 02/Aug/19 20:02 Start Date: 02/Aug/19 20:02 Worklog Time Spent: 10m Work Description: smengcl commented on pull request #1218: HDDS-1891. Ozone fs shell command should work with default port when port number is not specified URL: https://github.com/apache/hadoop/pull/1218 Tested manually with ozone fs: ```bash $ ozone fs -ls o3fs://bucket.volume.localhost/ 2019-08-02 12:57:13,223 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9862. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS) ... $ ozone fs -ls o3fs://bucket.volume.localhost:9861/ 2019-08-02 12:57:21,604 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9861. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS) ... ``` Do we want to add a unit test for this? If yes, existing `TestOzoneShell` won't suit the need since `getOmAddress()` always returns the full address with port number. And the port number is almost always not default OM port 9862 when the running the test. So I might need to come up with a new unit test suite. e.g. test the parser only. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 288209) Time Spent: 10m Remaining Estimate: 0h > Ozone fs shell command should work with default port when port number is not > specified > -- > > Key: HDDS-1891 > URL: https://issues.apache.org/jira/browse/HDDS-1891 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > {code:bash|title=Without port number -> Error} > $ ozone fs -ls o3fs://bucket.volume.localhost/ > -ls: Ozone file system url should be either one of the two forms: > o3fs://bucket.volume/key OR o3fs://bucket.volume.om-host.example.com:5678/key > ... > {code} > {code:bash|title=With port number -> Success} > $ ozone fs -ls o3fs://bucket.volume.localhost:9862/ > Found 1 items > -rw-rw-rw- 1 hadoop hadoop 1485 2019-08-01 21:14 > o3fs://bucket.volume.localhost:9862/README.txt > {code} > We expect the first command to attempt port 9862 by default. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org