[
https://issues.apache.org/jira/browse/BIGTOP-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14001603#comment-14001603
]
Martin Bukatovic commented on BIGTOP-1307:
------------------------------------------
{quote}
You run tests via the maven itest incantation
{quote}
Not exactly, I use my own bash wrapper which runs the testcase instead of
maven. This allows me to prefer system hadoop libraries over ones from local
maven repository.
{quote}
few of them fail (what percent , do others pass?)
{quote}
136 test cases are failing because of this 'No such file' error as described in
this JIRA, but there are other 33 cases which are failing for other reasons
(eg. missing files in case #37 which check `fs -du` functionality) - I tried to
check few cases from this 2nd group and observed the same results - when done
manually, these cases worked as well. This makes me think that issues of both
groups shares the root cause. The rest of cases, 328 (66%), are passing without
any problems.
{quote}
For any individual failing test, calling from bash succeeds. Is that the "bug"
you are seeing?
{quote}
Yes.
{quote}
One cool/fun/interesting/and possibly useful way to debug this will be running
the groovy scripts directly, rather than compiling them.
{quote}
The TestCLI cases are written in java, so this is not directly possible. The
reason why I'm checking the java cases without modifications with HDFS first is
to make sure my environment works before I start applying further changes. And
given the nature of this issue, I'm not sure if the problem hides within my
test environment or TestCLI test code (which is not triggered in default
configuration).
I will try to check this on different clusters and let you know.
> Some TestCLI cases fail with 'No such file or directory'
> --------------------------------------------------------
>
> Key: BIGTOP-1307
> URL: https://issues.apache.org/jira/browse/BIGTOP-1307
> Project: Bigtop
> Issue Type: Bug
> Components: Tests
> Affects Versions: 0.8.0
> Environment: HDP 2.0.6
> Reporter: Martin Bukatovic
> Priority: Critical
> Attachments: filter-cases.sh, testcli.nosuchfile-cases.log
>
>
> I observe weird results of xml-defined test cases of TestCLI bigtop test:
> 136 test cases failed because of 'No such file or directory' error.
> To show what the problem is, see testcase #1:
> {noformat}
> 14/05/15 16:50:40 INFO cli.CLITestHelper:
> -------------------------------------------
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Test ID: [1]
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Test Description: [ls:
> file using absolute path]
> 14/05/15 16:50:40 INFO cli.CLITestHelper:
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Test Commands: [-fs
> hdfs://dhcp-lab-203.local:8020 -touchz /tmp/testcli_1400165386646/file1]
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Test Commands: [-fs
> hdfs://dhcp-lab-203.local:8020 -ls /tmp/testcli_1400165386646/file1]
> 14/05/15 16:50:40 INFO cli.CLITestHelper:
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Cleanup Commands: [-fs
> hdfs://dhcp-lab-203.local:8020 -rm /tmp/testcli_1400165386646/file1]
> 14/05/15 16:50:40 INFO cli.CLITestHelper:
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Comparator:
> [TokenComparator]
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Comparision result:
> [fail]
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Expected output:
> [Found 1 items]
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Actual output: [ls:
> `/tmp/testcli_1400165386646/file1': No such file or directory
> ]
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Comparator:
> [RegexpComparator]
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Comparision result:
> [fail]
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Expected output:
> [^-rw-r--r--( )*1( )*[a-z]*( )*hdfs( )*0( )*[0-9]{4,}-[0-9]{2,}-[0-9]{2,}
> [0-9]{2,}:[0-9]{2,}( )*/tmp/testcli_1400165386646/file1]
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Actual output: [ls:
> `/tmp/testcli_1400165386646/file1': No such file or directory
> ]
> 14/05/15 16:50:40 INFO cli.CLITestHelper:
> 14/05/15 16:50:40 INFO cli.CLITestHelper:
> -------------------------------------------
> {noformat}
> The results looks as if there were someting wrong with hadoop/hdfs.
> Nevertheless when I checked this particular case manually, it worked just
> fine:
> {noformat}
> [bigtop@dhcp-lab-203 testcli]$ hadoop fs -mkdir /tmp/testcli_1400165386646
> [bigtop@dhcp-lab-203 testcli]$ hadoop fs -fs hdfs://dhcp-lab-203.local:8020
> -touchz /tmp/testcli_1400165386646/file1
> [bigtop@dhcp-lab-203 testcli]$ hadoop fs -fs hdfs://dhcp-lab-203.local:8020
> -ls /tmp/testcli_1400165386646/file1
> Found 1 items
> -rw-r--r-- 3 bigtop hdfs 0 2014-05-15 17:08
> /tmp/testcli_1400165386646/file1
> [bigtop@dhcp-lab-203 testcli]$ hadoop fs -fs hdfs://dhcp-lab-203.local:8020
> -rm /tmp/testcli_1400165386646/file1
> 14/05/15 17:08:27 INFO fs.TrashPolicyDefault: Namenode trash configuration:
> Deletion interval = 21600000 minutes, Emptier interval = 0 minutes.
> Moved: 'hdfs://dhcp-lab-203.local:8020/tmp/testcli_1400165386646/file1' to
> trash at: hdfs://dhcp-lab-203.local:8020/user/bigtop/.Trash/Current
> [bigtop@dhcp-lab-203 testcli]$
> {noformat}
> I manually checked 5 other cases with the same result: when the testcase is
> done
> manually, it works without any problems.
> Moreover I rerun all TestCLI cases 5 times, and the set of failed cases
> was always the same.
> Have anybody seen similar behaviour? I have executed TestCLI cases via wrapper
> which sets system classpath instead of maven defined enviromnent. Can this
> caused the issue, or is it likely that the problem is the bigtop tests? Also
> feel free to propose a way to debug this further.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
