[jira] [Updated] (HDFS-12938) TestErasureCodigCLI testAll failing consistently.

2017-12-21 Thread Lei (Eddy) Xu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-12938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lei (Eddy) Xu updated HDFS-12938:
-
  Resolution: Fixed
   Fix Version/s: 3.0.1
  3.1.0
Target Version/s: 3.0.0, 3.0.1
  Status: Resolved  (was: Patch Available)

Thanks for the quick response, [~ajayydv]! Committed to trunk and branch-3

> TestErasureCodigCLI testAll failing consistently.
> -
>
> Key: HDFS-12938
> URL: https://issues.apache.org/jira/browse/HDFS-12938
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: erasure-coding, hdfs
>Affects Versions: 3.1.0
>Reporter: Rushabh S Shah
>Assignee: Ajay Kumar
> Fix For: 3.1.0, 3.0.1
>
> Attachments: HDFS-12938.001.patch
>
>
> {{TestErasureCodingCLI#testAll}} is failing consistently.
> It failed in this precommit: 
> https://builds.apache.org/job/PreCommit-HDFS-Build/22435/testReport/org.apache.hadoop.cli/TestErasureCodingCLI/testAll/
> I ran locally on my laptop and it failed too.
> It failed with this stack trace:
> {noformat}
> java.lang.AssertionError: One of the tests failed. See the Detailed results 
> to identify the command that failed
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at 
> org.apache.hadoop.cli.CLITestHelper.displayResults(CLITestHelper.java:264)
>   at org.apache.hadoop.cli.CLITestHelper.tearDown(CLITestHelper.java:126)
>   at 
> org.apache.hadoop.cli.TestErasureCodingCLI.tearDown(TestErasureCodingCLI.java:77)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> {noformat}
> Below is the detailed report from 
> {{org.apache.hadoop.cli.TestErasureCodingCLI-output.txt}}.
> {noformat}
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(156)) - 
> ---
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(157)) - Test ID: [15]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(158)) -Test Description: 
> [setPolicy : set policy on non-empty directory]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(159)) - 
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(163)) -   Test Commands: [-fs 
> hdfs://localhost:52345 -mkdir /ecdir]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(163)) -   Test Commands: [-fs 
> hdfs://localhost:52345 -touchz /ecdir/file1]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(163)) -   Test Commands: [-fs 
> hdfs://localhost:52345 -setPolicy -policy RS-6-3-1024k -path /ecdir]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(167)) - 
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(170)) -Cleanup Commands: [-fs 
> hdfs://localhost:52345 -rm -R /ecdir]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(174)) - 
> 2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(178)) -  Comparator: 
> [SubstringComparator]
> 2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(180)) -  Comparision result:   
> [fail]
> 2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(182)) - Expected output:   
> [Warning: setting erasure coding policy on an non-empty directory will not 
> automatically convert existing data to RS-6-3-1024]
> 2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(184)) -   Actual output:   
> [Set erasure

[jira] [Updated] (HDFS-12938) TestErasureCodigCLI testAll failing consistently.

2017-12-21 Thread Ajay Kumar (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-12938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ajay Kumar updated HDFS-12938:
--
Status: Patch Available  (was: Open)

> TestErasureCodigCLI testAll failing consistently.
> -
>
> Key: HDFS-12938
> URL: https://issues.apache.org/jira/browse/HDFS-12938
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: erasure-coding, hdfs
>Affects Versions: 3.1.0
>Reporter: Rushabh S Shah
>Assignee: Ajay Kumar
> Attachments: HDFS-12938.001.patch
>
>
> {{TestErasureCodingCLI#testAll}} is failing consistently.
> It failed in this precommit: 
> https://builds.apache.org/job/PreCommit-HDFS-Build/22435/testReport/org.apache.hadoop.cli/TestErasureCodingCLI/testAll/
> I ran locally on my laptop and it failed too.
> It failed with this stack trace:
> {noformat}
> java.lang.AssertionError: One of the tests failed. See the Detailed results 
> to identify the command that failed
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at 
> org.apache.hadoop.cli.CLITestHelper.displayResults(CLITestHelper.java:264)
>   at org.apache.hadoop.cli.CLITestHelper.tearDown(CLITestHelper.java:126)
>   at 
> org.apache.hadoop.cli.TestErasureCodingCLI.tearDown(TestErasureCodingCLI.java:77)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> {noformat}
> Below is the detailed report from 
> {{org.apache.hadoop.cli.TestErasureCodingCLI-output.txt}}.
> {noformat}
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(156)) - 
> ---
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(157)) - Test ID: [15]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(158)) -Test Description: 
> [setPolicy : set policy on non-empty directory]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(159)) - 
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(163)) -   Test Commands: [-fs 
> hdfs://localhost:52345 -mkdir /ecdir]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(163)) -   Test Commands: [-fs 
> hdfs://localhost:52345 -touchz /ecdir/file1]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(163)) -   Test Commands: [-fs 
> hdfs://localhost:52345 -setPolicy -policy RS-6-3-1024k -path /ecdir]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(167)) - 
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(170)) -Cleanup Commands: [-fs 
> hdfs://localhost:52345 -rm -R /ecdir]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(174)) - 
> 2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(178)) -  Comparator: 
> [SubstringComparator]
> 2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(180)) -  Comparision result:   
> [fail]
> 2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(182)) - Expected output:   
> [Warning: setting erasure coding policy on an non-empty directory will not 
> automatically convert existing data to RS-6-3-1024]
> 2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(184)) -   Actual output:   
> [Set erasure coding policy RS-6-3-1024k on /ecdir
> Warning: setting erasure coding policy on a non-empty directory will not 
> automatically convert existing files to RS-6-3-1024k
> ]
> 2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper 
>

[jira] [Updated] (HDFS-12938) TestErasureCodigCLI testAll failing consistently.

2017-12-20 Thread Ajay Kumar (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-12938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ajay Kumar updated HDFS-12938:
--
Attachment: HDFS-12938.001.patch

Seems there is difference in warning message coming from  {{ECAdmin}} and one 
we are expecting in {{testErasureCodingConf.xml}}. Uploading patch for review.

> TestErasureCodigCLI testAll failing consistently.
> -
>
> Key: HDFS-12938
> URL: https://issues.apache.org/jira/browse/HDFS-12938
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: erasure-coding, hdfs
>Affects Versions: 3.1.0
>Reporter: Rushabh S Shah
>Assignee: Ajay Kumar
> Attachments: HDFS-12938.001.patch
>
>
> {{TestErasureCodingCLI#testAll}} is failing consistently.
> It failed in this precommit: 
> https://builds.apache.org/job/PreCommit-HDFS-Build/22435/testReport/org.apache.hadoop.cli/TestErasureCodingCLI/testAll/
> I ran locally on my laptop and it failed too.
> It failed with this stack trace:
> {noformat}
> java.lang.AssertionError: One of the tests failed. See the Detailed results 
> to identify the command that failed
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at 
> org.apache.hadoop.cli.CLITestHelper.displayResults(CLITestHelper.java:264)
>   at org.apache.hadoop.cli.CLITestHelper.tearDown(CLITestHelper.java:126)
>   at 
> org.apache.hadoop.cli.TestErasureCodingCLI.tearDown(TestErasureCodingCLI.java:77)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> {noformat}
> Below is the detailed report from 
> {{org.apache.hadoop.cli.TestErasureCodingCLI-output.txt}}.
> {noformat}
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(156)) - 
> ---
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(157)) - Test ID: [15]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(158)) -Test Description: 
> [setPolicy : set policy on non-empty directory]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(159)) - 
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(163)) -   Test Commands: [-fs 
> hdfs://localhost:52345 -mkdir /ecdir]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(163)) -   Test Commands: [-fs 
> hdfs://localhost:52345 -touchz /ecdir/file1]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(163)) -   Test Commands: [-fs 
> hdfs://localhost:52345 -setPolicy -policy RS-6-3-1024k -path /ecdir]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(167)) - 
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(170)) -Cleanup Commands: [-fs 
> hdfs://localhost:52345 -rm -R /ecdir]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(174)) - 
> 2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(178)) -  Comparator: 
> [SubstringComparator]
> 2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(180)) -  Comparision result:   
> [fail]
> 2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(182)) - Expected output:   
> [Warning: setting erasure coding policy on an non-empty directory will not 
> automatically convert existing data to RS-6-3-1024]
> 2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(184)) -   Actual output:   
> [Set erasure coding policy RS-6-3-1024k on /ecdir
> Warning: setting erasure coding policy on a non-empty directory

[jira] [Updated] (HDFS-12938) TestErasureCodigCLI testAll failing consistently.

2017-12-18 Thread Rushabh S Shah (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-12938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rushabh S Shah updated HDFS-12938:
--
Summary: TestErasureCodigCLI testAll failing consistently.  (was: 
TestErasureCodigCLI test failing consistently.)

> TestErasureCodigCLI testAll failing consistently.
> -
>
> Key: HDFS-12938
> URL: https://issues.apache.org/jira/browse/HDFS-12938
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: erasure-coding, hdfs
>Affects Versions: 3.1.0
>Reporter: Rushabh S Shah
>
> {{TestErasureCodingCLI#testAll}} is failing consistently.
> It failed in this precommit: 
> https://builds.apache.org/job/PreCommit-HDFS-Build/22435/testReport/org.apache.hadoop.cli/TestErasureCodingCLI/testAll/
> I ran locally on my laptop and it failed too.
> It failed with this stack trace:
> {noformat}
> java.lang.AssertionError: One of the tests failed. See the Detailed results 
> to identify the command that failed
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at 
> org.apache.hadoop.cli.CLITestHelper.displayResults(CLITestHelper.java:264)
>   at org.apache.hadoop.cli.CLITestHelper.tearDown(CLITestHelper.java:126)
>   at 
> org.apache.hadoop.cli.TestErasureCodingCLI.tearDown(TestErasureCodingCLI.java:77)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> {noformat}
> Below is the detailed report from 
> {{org.apache.hadoop.cli.TestErasureCodingCLI-output.txt}}.
> {noformat}
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(156)) - 
> ---
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(157)) - Test ID: [15]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(158)) -Test Description: 
> [setPolicy : set policy on non-empty directory]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(159)) - 
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(163)) -   Test Commands: [-fs 
> hdfs://localhost:52345 -mkdir /ecdir]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(163)) -   Test Commands: [-fs 
> hdfs://localhost:52345 -touchz /ecdir/file1]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(163)) -   Test Commands: [-fs 
> hdfs://localhost:52345 -setPolicy -policy RS-6-3-1024k -path /ecdir]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(167)) - 
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(170)) -Cleanup Commands: [-fs 
> hdfs://localhost:52345 -rm -R /ecdir]
> 2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(174)) - 
> 2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(178)) -  Comparator: 
> [SubstringComparator]
> 2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(180)) -  Comparision result:   
> [fail]
> 2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(182)) - Expected output:   
> [Warning: setting erasure coding policy on an non-empty directory will not 
> automatically convert existing data to RS-6-3-1024]
> 2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper 
> (CLITestHelper.java:displayResults(184)) -   Actual output:   
> [Set erasure coding policy RS-6-3-1024k on /ecdir
> Warning: setting erasure coding policy on a non-empty directory will not 
> automatically convert existing files to RS-6-3-1024k
> ]
> 2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper 
>