[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[
https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang updated HDFS-10892:
---
Fix Version/s: 3.0.0-alpha2
2.8.0
> Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
> ---
>
> Key: HDFS-10892
> URL: https://issues.apache.org/jira/browse/HDFS-10892
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs, shell, test
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Fix For: 2.8.0, 3.0.0-alpha2
>
> Attachments: HDFS-10892.000.patch, HDFS-10892.001.patch,
> HDFS-10892.002.patch, HDFS-10892.003.patch, HDFS-10892.004.patch,
> HDFS-10892.005.patch
>
>
> I did not find unit test in {{trunk}} code for following cases:
> - HDFS command {{dfs -tail}}
> - HDFS command {{dfs -stat}}
> I think it still merits to have one though the commands have served us for
> years.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[
https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HDFS-10892:
-
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to {{trunk}} through {{branch-2.8}}. Thanks [~jnp] for review.
> Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
> ---
>
> Key: HDFS-10892
> URL: https://issues.apache.org/jira/browse/HDFS-10892
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs, shell, test
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-10892.000.patch, HDFS-10892.001.patch,
> HDFS-10892.002.patch, HDFS-10892.003.patch, HDFS-10892.004.patch,
> HDFS-10892.005.patch
>
>
> I did not find unit test in {{trunk}} code for following cases:
> - HDFS command {{dfs -tail}}
> - HDFS command {{dfs -stat}}
> I think it still merits to have one though the commands have served us for
> years.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[
https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HDFS-10892:
-
Attachment: HDFS-10892.005.patch
> Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
> ---
>
> Key: HDFS-10892
> URL: https://issues.apache.org/jira/browse/HDFS-10892
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs, shell, test
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-10892.000.patch, HDFS-10892.001.patch,
> HDFS-10892.002.patch, HDFS-10892.003.patch, HDFS-10892.004.patch,
> HDFS-10892.005.patch
>
>
> I did not find unit test in {{trunk}} code for following cases:
> - HDFS command {{dfs -tail}}
> - HDFS command {{dfs -stat}}
> I think it still merits to have one though the commands have served us for
> years.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[
https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HDFS-10892:
-
Attachment: HDFS-10892.004.patch
> Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
> ---
>
> Key: HDFS-10892
> URL: https://issues.apache.org/jira/browse/HDFS-10892
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs, shell, test
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-10892.000.patch, HDFS-10892.001.patch,
> HDFS-10892.002.patch, HDFS-10892.003.patch, HDFS-10892.004.patch
>
>
> I did not find unit test in {{trunk}} code for following cases:
> - HDFS command {{dfs -tail}}
> - HDFS command {{dfs -stat}}
> I think it still merits to have one though the commands have served us for
> years.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[
https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HDFS-10892:
-
Description:
I did not find unit test in {{trunk}} code for following cases:
- HDFS command {{dfs -tail}}
- HDFS command {{dfs -stat}}
I think it still merits to have one though the commands have served us for
years.
was:
I did not find unit test in {{trunk}} code for following cases:
- HDFS command {{dfs -tail}}
- HDFS command {{dfs -stat}}
- file name or content with UTF-8 characters
I think it still merits to have one though the commands have served us for
years.
The {{hadoop.hdfs.TestDFSShell#testUtf8Encoding}} was not able to run in
Jenkins. I was able to pass the test locally without any problem. Perhaps it
has something to with the {{LANG}} environment variable according to
discussions on the Internet. I can confirm that my local test machine is using
following settings. I also think Yetus is setting the local correctly. [~aw] Do
you have any idea about this? Thanks.
{code}
$ locale
LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL=
{code}
As this test case is not tightly related to the commands "-tail" or "-stat", I
remove this test case in the latest patch. Just for what it's worth, the test
code is:
{code}
/**
* Test that the file name and content can have UTF-8 chars.
*/
@Test (timeout = 3)
public void testUtf8Encoding() throws Exception {
final int blockSize = 1024;
final Configuration conf = new HdfsConfiguration();
conf.setInt(DFSConfigKeys.DFS_BLOCK_SIZE_KEY, blockSize);
try (MiniDFSCluster cluster =
new MiniDFSCluster.Builder(conf).numDataNodes(3).build()) {
cluster.waitActive();
final DistributedFileSystem dfs = cluster.getFileSystem();
final Path workDir= new Path("/testUtf8Encoding");
dfs.mkdirs(workDir);
System.setProperty("sun.jnu.encoding", "UTF-8");
System.setProperty("file.encoding", "UTF-8");
final String chineseStr = "哈杜普.txt";
final File testFile = new File(TEST_ROOT_DIR, chineseStr);
// create a local file; its content contains the Chinese file name
createLocalFile(testFile);
dfs.copyFromLocalFile(new Path(testFile.getPath()), workDir);
assertTrue(dfs.exists(new Path(workDir, testFile.getName(;
final ByteArrayOutputStream out = new ByteArrayOutputStream();
System.setOut(new PrintStream(out));
final String argv[] = new String[]{
"-cat", workDir + "/" + testFile.getName()};
final int ret = ToolRunner.run(new FsShell(conf), argv);
assertEquals(Arrays.toString(argv) + " returned non-zero status " + ret,
0, ret);
assertTrue("Unexpected -cat output: " + out,
out.toString().contains(chineseStr));
}
}
{code}
If anyone suggests a working approach, I'd like to submit separate JIRA for
tracking this test. Otherwise, we test it elsewhere (say, nightly system tests).
> Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
> ---
>
> Key: HDFS-10892
> URL: https://issues.apache.org/jira/browse/HDFS-10892
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs, shell, test
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-10892.000.patch, HDFS-10892.001.patch,
> HDFS-10892.002.patch, HDFS-10892.003.patch
>
>
> I did not find unit test in {{trunk}} code for following cases:
> - HDFS command {{dfs -tail}}
> - HDFS command {{dfs -stat}}
> I think it still merits to have one though the commands have served us for
> years.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[
https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HDFS-10892:
-
Attachment: HDFS-10892.003.patch
> Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
> ---
>
> Key: HDFS-10892
> URL: https://issues.apache.org/jira/browse/HDFS-10892
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs, shell, test
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-10892.000.patch, HDFS-10892.001.patch,
> HDFS-10892.002.patch, HDFS-10892.003.patch
>
>
> I did not find unit test in {{trunk}} code for following cases:
> - HDFS command {{dfs -tail}}
> - HDFS command {{dfs -stat}}
> - file name or content with UTF-8 characters
> I think it still merits to have one though the commands have served us for
> years.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[
https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HDFS-10892:
-
Description:
I did not find unit test in {{trunk}} code for following cases:
- HDFS command {{dfs -tail}}
- HDFS command {{dfs -stat}}
- file name or content with UTF-8 characters
I think it still merits to have one though the commands have served us for
years.
was:I did not find unit test for HDFS command 'dfs -tail' in {{trunk}} code.
I think it still merits to have one though the command itself has been used for
years.
> Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
> ---
>
> Key: HDFS-10892
> URL: https://issues.apache.org/jira/browse/HDFS-10892
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs, shell, test
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-10892.000.patch, HDFS-10892.001.patch,
> HDFS-10892.002.patch
>
>
> I did not find unit test in {{trunk}} code for following cases:
> - HDFS command {{dfs -tail}}
> - HDFS command {{dfs -stat}}
> - file name or content with UTF-8 characters
> I think it still merits to have one though the commands have served us for
> years.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[
https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HDFS-10892:
-
Attachment: HDFS-10892.002.patch
> Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
> ---
>
> Key: HDFS-10892
> URL: https://issues.apache.org/jira/browse/HDFS-10892
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs, shell, test
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-10892.000.patch, HDFS-10892.001.patch,
> HDFS-10892.002.patch
>
>
> I did not find unit test for HDFS command 'dfs -tail' in {{trunk}} code. I
> think it still merits to have one though the command itself has been used for
> years.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[
https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HDFS-10892:
-
Attachment: HDFS-10892.001.patch
> Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
> ---
>
> Key: HDFS-10892
> URL: https://issues.apache.org/jira/browse/HDFS-10892
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs, shell, test
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-10892.000.patch, HDFS-10892.001.patch
>
>
> I did not find unit test for HDFS command 'dfs -tail' in {{trunk}} code. I
> think it still merits to have one though the command itself has been used for
> years.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[
https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HDFS-10892:
-
Status: Patch Available (was: Open)
> Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
> ---
>
> Key: HDFS-10892
> URL: https://issues.apache.org/jira/browse/HDFS-10892
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs, shell, test
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-10892.000.patch
>
>
> I did not find unit test for HDFS command 'dfs -tail' in {{trunk}} code. I
> think it still merits to have one though the command itself has been used for
> years.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[
https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HDFS-10892:
-
Attachment: HDFS-10892.000.patch
The v0 patch added tests for 'dfs -tail' and 'dfs -stat'. Will probably add
more to this single patch as the logic is very similar.
> Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
> ---
>
> Key: HDFS-10892
> URL: https://issues.apache.org/jira/browse/HDFS-10892
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs, shell, test
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-10892.000.patch
>
>
> I did not find unit test for HDFS command 'dfs -tail' in {{trunk}} code. I
> think it still merits to have one though the command itself has been used for
> years.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10892) Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
[
https://issues.apache.org/jira/browse/HDFS-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HDFS-10892:
-
Summary: Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat' (was:
Add unit tests for HDFS command 'dfs -tail')
> Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'
> ---
>
> Key: HDFS-10892
> URL: https://issues.apache.org/jira/browse/HDFS-10892
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs, shell, test
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
>
> I did not find unit test for HDFS command 'dfs -tail' in {{trunk}} code. I
> think it still merits to have one though the command itself has been used for
> years.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
