[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[ https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-10892: --- Fix Version/s: 3.0.0-alpha2 2.8.0 > Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat' > --- > > Key: HDFS-10892 > URL: https://issues.apache.org/jira/browse/HDFS-10892 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs, shell, test >Reporter: Mingliang Liu >Assignee: Mingliang Liu > Fix For: 2.8.0, 3.0.0-alpha2 > > Attachments: HDFS-10892.000.patch, HDFS-10892.001.patch, > HDFS-10892.002.patch, HDFS-10892.003.patch, HDFS-10892.004.patch, > HDFS-10892.005.patch > > > I did not find unit test in {{trunk}} code for following cases: > - HDFS command {{dfs -tail}} > - HDFS command {{dfs -stat}} > I think it still merits to have one though the commands have served us for > years. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[ https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10892: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to {{trunk}} through {{branch-2.8}}. Thanks [~jnp] for review. > Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat' > --- > > Key: HDFS-10892 > URL: https://issues.apache.org/jira/browse/HDFS-10892 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs, shell, test >Reporter: Mingliang Liu >Assignee: Mingliang Liu > Attachments: HDFS-10892.000.patch, HDFS-10892.001.patch, > HDFS-10892.002.patch, HDFS-10892.003.patch, HDFS-10892.004.patch, > HDFS-10892.005.patch > > > I did not find unit test in {{trunk}} code for following cases: > - HDFS command {{dfs -tail}} > - HDFS command {{dfs -stat}} > I think it still merits to have one though the commands have served us for > years. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[ https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10892: - Attachment: HDFS-10892.005.patch > Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat' > --- > > Key: HDFS-10892 > URL: https://issues.apache.org/jira/browse/HDFS-10892 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs, shell, test >Reporter: Mingliang Liu >Assignee: Mingliang Liu > Attachments: HDFS-10892.000.patch, HDFS-10892.001.patch, > HDFS-10892.002.patch, HDFS-10892.003.patch, HDFS-10892.004.patch, > HDFS-10892.005.patch > > > I did not find unit test in {{trunk}} code for following cases: > - HDFS command {{dfs -tail}} > - HDFS command {{dfs -stat}} > I think it still merits to have one though the commands have served us for > years. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[ https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10892: - Attachment: HDFS-10892.004.patch > Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat' > --- > > Key: HDFS-10892 > URL: https://issues.apache.org/jira/browse/HDFS-10892 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs, shell, test >Reporter: Mingliang Liu >Assignee: Mingliang Liu > Attachments: HDFS-10892.000.patch, HDFS-10892.001.patch, > HDFS-10892.002.patch, HDFS-10892.003.patch, HDFS-10892.004.patch > > > I did not find unit test in {{trunk}} code for following cases: > - HDFS command {{dfs -tail}} > - HDFS command {{dfs -stat}} > I think it still merits to have one though the commands have served us for > years. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[ https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10892: - Description: I did not find unit test in {{trunk}} code for following cases: - HDFS command {{dfs -tail}} - HDFS command {{dfs -stat}} I think it still merits to have one though the commands have served us for years. was: I did not find unit test in {{trunk}} code for following cases: - HDFS command {{dfs -tail}} - HDFS command {{dfs -stat}} - file name or content with UTF-8 characters I think it still merits to have one though the commands have served us for years. The {{hadoop.hdfs.TestDFSShell#testUtf8Encoding}} was not able to run in Jenkins. I was able to pass the test locally without any problem. Perhaps it has something to with the {{LANG}} environment variable according to discussions on the Internet. I can confirm that my local test machine is using following settings. I also think Yetus is setting the local correctly. [~aw] Do you have any idea about this? Thanks. {code} $ locale LANG="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_CTYPE="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_ALL= {code} As this test case is not tightly related to the commands "-tail" or "-stat", I remove this test case in the latest patch. Just for what it's worth, the test code is: {code} /** * Test that the file name and content can have UTF-8 chars. */ @Test (timeout = 3) public void testUtf8Encoding() throws Exception { final int blockSize = 1024; final Configuration conf = new HdfsConfiguration(); conf.setInt(DFSConfigKeys.DFS_BLOCK_SIZE_KEY, blockSize); try (MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf).numDataNodes(3).build()) { cluster.waitActive(); final DistributedFileSystem dfs = cluster.getFileSystem(); final Path workDir= new Path("/testUtf8Encoding"); dfs.mkdirs(workDir); System.setProperty("sun.jnu.encoding", "UTF-8"); System.setProperty("file.encoding", "UTF-8"); final String chineseStr = "哈杜普.txt"; final File testFile = new File(TEST_ROOT_DIR, chineseStr); // create a local file; its content contains the Chinese file name createLocalFile(testFile); dfs.copyFromLocalFile(new Path(testFile.getPath()), workDir); assertTrue(dfs.exists(new Path(workDir, testFile.getName(; final ByteArrayOutputStream out = new ByteArrayOutputStream(); System.setOut(new PrintStream(out)); final String argv[] = new String[]{ "-cat", workDir + "/" + testFile.getName()}; final int ret = ToolRunner.run(new FsShell(conf), argv); assertEquals(Arrays.toString(argv) + " returned non-zero status " + ret, 0, ret); assertTrue("Unexpected -cat output: " + out, out.toString().contains(chineseStr)); } } {code} If anyone suggests a working approach, I'd like to submit separate JIRA for tracking this test. Otherwise, we test it elsewhere (say, nightly system tests). > Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat' > --- > > Key: HDFS-10892 > URL: https://issues.apache.org/jira/browse/HDFS-10892 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs, shell, test >Reporter: Mingliang Liu >Assignee: Mingliang Liu > Attachments: HDFS-10892.000.patch, HDFS-10892.001.patch, > HDFS-10892.002.patch, HDFS-10892.003.patch > > > I did not find unit test in {{trunk}} code for following cases: > - HDFS command {{dfs -tail}} > - HDFS command {{dfs -stat}} > I think it still merits to have one though the commands have served us for > years. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[ https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10892: - Attachment: HDFS-10892.003.patch > Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat' > --- > > Key: HDFS-10892 > URL: https://issues.apache.org/jira/browse/HDFS-10892 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs, shell, test >Reporter: Mingliang Liu >Assignee: Mingliang Liu > Attachments: HDFS-10892.000.patch, HDFS-10892.001.patch, > HDFS-10892.002.patch, HDFS-10892.003.patch > > > I did not find unit test in {{trunk}} code for following cases: > - HDFS command {{dfs -tail}} > - HDFS command {{dfs -stat}} > - file name or content with UTF-8 characters > I think it still merits to have one though the commands have served us for > years. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[ https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10892: - Description: I did not find unit test in {{trunk}} code for following cases: - HDFS command {{dfs -tail}} - HDFS command {{dfs -stat}} - file name or content with UTF-8 characters I think it still merits to have one though the commands have served us for years. was:I did not find unit test for HDFS command 'dfs -tail' in {{trunk}} code. I think it still merits to have one though the command itself has been used for years. > Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat' > --- > > Key: HDFS-10892 > URL: https://issues.apache.org/jira/browse/HDFS-10892 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs, shell, test >Reporter: Mingliang Liu >Assignee: Mingliang Liu > Attachments: HDFS-10892.000.patch, HDFS-10892.001.patch, > HDFS-10892.002.patch > > > I did not find unit test in {{trunk}} code for following cases: > - HDFS command {{dfs -tail}} > - HDFS command {{dfs -stat}} > - file name or content with UTF-8 characters > I think it still merits to have one though the commands have served us for > years. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[ https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10892: - Attachment: HDFS-10892.002.patch > Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat' > --- > > Key: HDFS-10892 > URL: https://issues.apache.org/jira/browse/HDFS-10892 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs, shell, test >Reporter: Mingliang Liu >Assignee: Mingliang Liu > Attachments: HDFS-10892.000.patch, HDFS-10892.001.patch, > HDFS-10892.002.patch > > > I did not find unit test for HDFS command 'dfs -tail' in {{trunk}} code. I > think it still merits to have one though the command itself has been used for > years. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[ https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10892: - Attachment: HDFS-10892.001.patch > Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat' > --- > > Key: HDFS-10892 > URL: https://issues.apache.org/jira/browse/HDFS-10892 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs, shell, test >Reporter: Mingliang Liu >Assignee: Mingliang Liu > Attachments: HDFS-10892.000.patch, HDFS-10892.001.patch > > > I did not find unit test for HDFS command 'dfs -tail' in {{trunk}} code. I > think it still merits to have one though the command itself has been used for > years. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[ https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10892: - Status: Patch Available (was: Open) > Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat' > --- > > Key: HDFS-10892 > URL: https://issues.apache.org/jira/browse/HDFS-10892 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs, shell, test >Reporter: Mingliang Liu >Assignee: Mingliang Liu > Attachments: HDFS-10892.000.patch > > > I did not find unit test for HDFS command 'dfs -tail' in {{trunk}} code. I > think it still merits to have one though the command itself has been used for > years. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[ https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10892: - Attachment: HDFS-10892.000.patch The v0 patch added tests for 'dfs -tail' and 'dfs -stat'. Will probably add more to this single patch as the logic is very similar. > Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat' > --- > > Key: HDFS-10892 > URL: https://issues.apache.org/jira/browse/HDFS-10892 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs, shell, test >Reporter: Mingliang Liu >Assignee: Mingliang Liu > Attachments: HDFS-10892.000.patch > > > I did not find unit test for HDFS command 'dfs -tail' in {{trunk}} code. I > think it still merits to have one though the command itself has been used for > years. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[ https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10892: - Summary: Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat' (was: Add unit tests for HDFS command 'dfs -tail') > Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat' > --- > > Key: HDFS-10892 > URL: https://issues.apache.org/jira/browse/HDFS-10892 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs, shell, test >Reporter: Mingliang Liu >Assignee: Mingliang Liu > > I did not find unit test for HDFS command 'dfs -tail' in {{trunk}} code. I > think it still merits to have one though the command itself has been used for > years. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail'
[ https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10892: - Description: I did not find unit test for HDFS command 'dfs -tail' in {{trunk}} code. I think it still merits to have one though the command itself has been used for years. (was: I did not find unit test for HDFS command 'dfs -tail' in {{trunk}} code. I think it still merits to have one though the command itself has been used for years. It also happens to 'dfs -test'. Perhaps we can consolidate the patches here.) > Add unit tests for HDFS command 'dfs -tail' > --- > > Key: HDFS-10892 > URL: https://issues.apache.org/jira/browse/HDFS-10892 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs, shell, test >Reporter: Mingliang Liu >Assignee: Mingliang Liu > > I did not find unit test for HDFS command 'dfs -tail' in {{trunk}} code. I > think it still merits to have one though the command itself has been used for > years. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail'
[ https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10892: - Component/s: shell fs > Add unit tests for HDFS command 'dfs -tail' > --- > > Key: HDFS-10892 > URL: https://issues.apache.org/jira/browse/HDFS-10892 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs, shell, test >Reporter: Mingliang Liu >Assignee: Mingliang Liu > > I did not find unit test for HDFS command 'dfs -tail' in {{trunk}} code. I > think it still merits to have one though the command itself has been used for > years. > It also happens to 'dfs -test'. Perhaps we can consolidate the patches here. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail'
[ https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10892: - Component/s: (was: hdfs) test > Add unit tests for HDFS command 'dfs -tail' > --- > > Key: HDFS-10892 > URL: https://issues.apache.org/jira/browse/HDFS-10892 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs, shell, test >Reporter: Mingliang Liu >Assignee: Mingliang Liu > > I did not find unit test for HDFS command 'dfs -tail' in {{trunk}} code. I > think it still merits to have one though the command itself has been used for > years. > It also happens to 'dfs -test'. Perhaps we can consolidate the patches here. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail'
[ https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10892: - Description: I did not find unit test for HDFS command 'dfs -tail' in {{trunk}} code. I think it still merits to have one though the command itself has been used for years. It also happens to 'dfs -test'. Perhaps we can consolidate the patches here. was: I did not find unit test for HDFS command 'dfs -tail' in {{trunk}} code. I think it still merits to have one though the command itself has been used there for years. It also happens to 'dfs -test'. Perhaps we can consolidate the patches here. > Add unit tests for HDFS command 'dfs -tail' > --- > > Key: HDFS-10892 > URL: https://issues.apache.org/jira/browse/HDFS-10892 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Mingliang Liu >Assignee: Mingliang Liu > > I did not find unit test for HDFS command 'dfs -tail' in {{trunk}} code. I > think it still merits to have one though the command itself has been used for > years. > It also happens to 'dfs -test'. Perhaps we can consolidate the patches here. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org