[
https://issues.apache.org/jira/browse/BIGTOP-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006061#comment-14006061
]
Martin Bukatovic commented on BIGTOP-1307:
------------------------------------------
That's true, TestCLI creates shell object with hdfs as a user, but this object
is used only for initialization of the test, all testcases are run directly via
java api (it uses protobuf under the hood) under the effective user of process
which executes the tests - which is bigtop in my environment.
So it would make sense to fail on startup when the effective user is not hdfs
(or root for glusterfs in hcfs compliant code), but I would wait with such
change a little until I understand where the problem with this JIRA really is.
Since it's likely that there are additional assumptions that are not checked
and it would be good to sum it up into a single patch. I will create new JIRA
for tracking these assumptions today (as I may just found another one).
> Some TestCLI cases fail with 'No such file or directory'
> --------------------------------------------------------
>
> Key: BIGTOP-1307
> URL: https://issues.apache.org/jira/browse/BIGTOP-1307
> Project: Bigtop
> Issue Type: Bug
> Components: Tests
> Affects Versions: 0.8.0
> Environment: HDP 2.0.6
> Reporter: Martin Bukatovic
> Priority: Critical
> Attachments: filter-cases.sh, testcli.nosuchfile-cases.log
>
>
> I observe weird results of xml-defined test cases of TestCLI bigtop test:
> 136 test cases failed because of 'No such file or directory' error.
> To show what the problem is, see testcase #1:
> {noformat}
> 14/05/15 16:50:40 INFO cli.CLITestHelper:
> -------------------------------------------
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Test ID: [1]
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Test Description: [ls:
> file using absolute path]
> 14/05/15 16:50:40 INFO cli.CLITestHelper:
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Test Commands: [-fs
> hdfs://dhcp-lab-203.local:8020 -touchz /tmp/testcli_1400165386646/file1]
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Test Commands: [-fs
> hdfs://dhcp-lab-203.local:8020 -ls /tmp/testcli_1400165386646/file1]
> 14/05/15 16:50:40 INFO cli.CLITestHelper:
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Cleanup Commands: [-fs
> hdfs://dhcp-lab-203.local:8020 -rm /tmp/testcli_1400165386646/file1]
> 14/05/15 16:50:40 INFO cli.CLITestHelper:
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Comparator:
> [TokenComparator]
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Comparision result:
> [fail]
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Expected output:
> [Found 1 items]
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Actual output: [ls:
> `/tmp/testcli_1400165386646/file1': No such file or directory
> ]
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Comparator:
> [RegexpComparator]
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Comparision result:
> [fail]
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Expected output:
> [^-rw-r--r--( )*1( )*[a-z]*( )*hdfs( )*0( )*[0-9]{4,}-[0-9]{2,}-[0-9]{2,}
> [0-9]{2,}:[0-9]{2,}( )*/tmp/testcli_1400165386646/file1]
> 14/05/15 16:50:40 INFO cli.CLITestHelper: Actual output: [ls:
> `/tmp/testcli_1400165386646/file1': No such file or directory
> ]
> 14/05/15 16:50:40 INFO cli.CLITestHelper:
> 14/05/15 16:50:40 INFO cli.CLITestHelper:
> -------------------------------------------
> {noformat}
> The results looks as if there were someting wrong with hadoop/hdfs.
> Nevertheless when I checked this particular case manually, it worked just
> fine:
> {noformat}
> [bigtop@dhcp-lab-203 testcli]$ hadoop fs -mkdir /tmp/testcli_1400165386646
> [bigtop@dhcp-lab-203 testcli]$ hadoop fs -fs hdfs://dhcp-lab-203.local:8020
> -touchz /tmp/testcli_1400165386646/file1
> [bigtop@dhcp-lab-203 testcli]$ hadoop fs -fs hdfs://dhcp-lab-203.local:8020
> -ls /tmp/testcli_1400165386646/file1
> Found 1 items
> -rw-r--r-- 3 bigtop hdfs 0 2014-05-15 17:08
> /tmp/testcli_1400165386646/file1
> [bigtop@dhcp-lab-203 testcli]$ hadoop fs -fs hdfs://dhcp-lab-203.local:8020
> -rm /tmp/testcli_1400165386646/file1
> 14/05/15 17:08:27 INFO fs.TrashPolicyDefault: Namenode trash configuration:
> Deletion interval = 21600000 minutes, Emptier interval = 0 minutes.
> Moved: 'hdfs://dhcp-lab-203.local:8020/tmp/testcli_1400165386646/file1' to
> trash at: hdfs://dhcp-lab-203.local:8020/user/bigtop/.Trash/Current
> [bigtop@dhcp-lab-203 testcli]$
> {noformat}
> I manually checked 5 other cases with the same result: when the testcase is
> done
> manually, it works without any problems.
> Moreover I rerun all TestCLI cases 5 times, and the set of failed cases
> was always the same.
> Have anybody seen similar behaviour? I have executed TestCLI cases via wrapper
> which sets system classpath instead of maven defined enviromnent. Can this
> caused the issue, or is it likely that the problem is the bigtop tests? Also
> feel free to propose a way to debug this further.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
