[
https://issues.apache.org/jira/browse/YARN-9098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16724900#comment-16724900
]
Szilard Nemeth edited comment on YARN-9098 at 12/19/18 11:26 AM:
-
Hi [~Jim_Brennan]!
Thanks for looking at my patch and taking steps on testing!
First of all: the static import is fixed with the new patch.
Actually, I understand your fear regarding this refactoring but this eliminates
some code duplication and improves the quality of the code significantly.
About the test errors:
These are very strange errors. At this level, they seem to be all test
framework issues, as somehow on your computer, the temp directory storing
cgroups and the cpu controller underneath have not created, which is very
strange as the tests are having assertions on the file and directory creations.
I re-built and ran the testcases with these commands again on my computer,
locally:
1.
{code:java}
mvn clean package -Pdist -DskipTests -Dmaven.javadoc.skip=true{code}
2.
{code:java}
mvn test -pl org.apache.hadoop:hadoop-yarn-server-nodemanager -fae | tee
~/maventest`date +%Y%m%d`
{code}
I don't have test failures on any of the classes where you had, this is the
excerpt of the output:
{noformat}
[INFO] Running
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.TestMtabFileParser
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.138 s
- in
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.TestMtabFileParser
...
[INFO] Running
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.TestCGroupsHandlerImpl
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.538 s
- in
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.TestCGroupsHandlerImpl
...
[INFO] Running
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.TestCGroupsControllerPaths
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.127 s
- in
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.TestCGroupsControllerPaths
{noformat}
This is the file listing of my target directory:
{noformat}
??-( szilardnemeth@snemeth-MBP[10:17:19] <0> @YARN-9098 )--(
~/development/apache/hadoop )--
└-$ cat
/Users/szilardnemeth/development/apache/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir/cgroups/025802ba-4abe-4862-942b-beef2d279ca7
none
/Users/szilardnemeth/development/apache/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir/cgroups/cp
cgroup rw,relatime,cpu 0 0
none
/Users/szilardnemeth/development/apache/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir/cgroups/cpu
cgroup rw,relatime,cpu 0 0
none
/Users/szilardnemeth/development/apache/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir/cgroups/blkio
cgroup rw,relatime,blkio 0 0{noformat}
I even tried to remove the test directory (with {{rm -rf
/Users/szilardnemeth/development/apache/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir}})
and re-execute the tests but they work fine.
Anyway, I added some code that better describes the assertion errors plus
added some logging of cgroup paths to 2 testcases.
Could you please rerun the tests with the new code and do a file listing like I
did?
This must be some platform issue I suppose.
The test failures for {{TestCGroupsHandlerImpl}}:
{{TestCGroupsHandlerImpl#createPremountedCgroups}} calls:
{noformat}
File cpuCgroup = new File(parentDir, "cpu");
//and later on...
assertTrue("Directory should be created", cpuCgroup.mkdirs());
{noformat}
This should create cgroups for cpu, and as you can see, it is even asserted
properly.
Could you please re-test?
Thanks!
was (Author: snemeth):
Hi [~Jim_Brennan]!
Thanks for looking at my patch and taking steps on testing!
First of all: the static import is fixed with the new patch.
Actually, I understand your fear regarding this refactoring but this eliminates
some code duplication and improves the quality of the code significantly.
About the test errors:
These are very strange errors. At this level, they seem to be all test
framework issues, as somehow on your computer, the temp directory storing
cgroups and the cpu controller underneath have not created, which is very
strange as the tests are having assertions on the file and directory creations.
I re-built and ran the testcases with these commands again on my computer,
locally:
1.
{code:java}
mvn clean package -Pdist -DskipTests -Dmaven.javadoc.skip=true{code}
2.
{code:java}
mvn test -pl org.apache.hadoop:ha