[
https://issues.apache.org/jira/browse/YARN-9561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965061#comment-16965061
]
Jim Brennan commented on YARN-9561:
-----------------------------------
Thanks for updating the patch [~ebadger]! I tested this along with the patches
for YARN-9562 and YARN-9564. Everything seems to be working well. I did run
into one issue with the container executor unit tests (cetest). I normally
compile with this option: -Dcontainer-executor.conf.dir=${HADOOP_CONF_DIR}
This causes some failures in cetest:
{noformat}
[----------] 7 tests from TestRunc
[ RUN ] TestRunc.test_parse_runc_launch_cmd_valid
Could not create /home/gs/hadoop/conf/container-executor.cfg
/home/jbrennan02/git/y-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/utils/test_runc_util.cc:47:
Failure
Expected: ret
Which is: 1
To be equal to: 0
Container executor cfg setup failed
[ FAILED ] TestRunc.test_parse_runc_launch_cmd_valid (1 ms)
[ RUN ] TestRunc.test_parse_runc_launch_cmd_bad_container_id
Could not create /home/gs/hadoop/conf/container-executor.cfg
/home/jbrennan02/git/y-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/utils/test_runc_util.cc:47:
Failure
Expected: ret
Which is: 1
To be equal to: 0
Container executor cfg setup failed
[ FAILED ] TestRunc.test_parse_runc_launch_cmd_bad_container_id (0 ms)
[ RUN ] TestRunc.test_parse_runc_launch_cmd_existing_pidfile
Could not create /home/gs/hadoop/conf/container-executor.cfg
/home/jbrennan02/git/y-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/utils/test_runc_util.cc:47:
Failure
Expected: ret
Which is: 1
To be equal to: 0
Container executor cfg setup failed
[ FAILED ] TestRunc.test_parse_runc_launch_cmd_existing_pidfile (0 ms)
[ RUN ] TestRunc.test_parse_runc_launch_cmd_invalid_media_type
Could not create /home/gs/hadoop/conf/container-executor.cfg
/home/jbrennan02/git/y-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/utils/test_runc_util.cc:47:
Failure
Expected: ret
Which is: 1
To be equal to: 0
Container executor cfg setup failed
[ FAILED ] TestRunc.test_parse_runc_launch_cmd_invalid_media_type (0 ms)
[ RUN ] TestRunc.test_parse_runc_launch_cmd_invalid_num_reap_layers_keep
Could not create /home/gs/hadoop/conf/container-executor.cfg
/home/jbrennan02/git/y-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/utils/test_runc_util.cc:47:
Failure
Expected: ret
Which is: 1
To be equal to: 0
Container executor cfg setup failed
[ FAILED ] TestRunc.test_parse_runc_launch_cmd_invalid_num_reap_layers_keep
(0 ms)
[ RUN ] TestRunc.test_parse_runc_launch_cmd_valid_mounts
Could not create /home/gs/hadoop/conf/container-executor.cfg
/home/jbrennan02/git/y-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/utils/test_runc_util.cc:47:
Failure
Expected: ret
Which is: 1
To be equal to: 0
Container executor cfg setup failed
[ FAILED ] TestRunc.test_parse_runc_launch_cmd_valid_mounts (0 ms)
[ RUN ] TestRunc.test_parse_runc_launch_cmd_invalid_mounts
Could not create /home/gs/hadoop/conf/container-executor.cfg
/home/jbrennan02/git/y-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/utils/test_runc_util.cc:47:
Failure
Expected: ret
Which is: 1
To be equal to: 0
Container executor cfg setup failed
[ FAILED ] TestRunc.test_parse_runc_launch_cmd_invalid_mounts (0 ms)
[----------] 7 tests from TestRunc (2 ms total)
[----------] Global test environment tear-down
[==========] 89 tests from 10 test cases ran. (82 ms total)
[ PASSED ] 82 tests.
[ FAILED ] 7 tests, listed below:
[ FAILED ] TestRunc.test_parse_runc_launch_cmd_valid
[ FAILED ] TestRunc.test_parse_runc_launch_cmd_bad_container_id
[ FAILED ] TestRunc.test_parse_runc_launch_cmd_existing_pidfile
[ FAILED ] TestRunc.test_parse_runc_launch_cmd_invalid_media_type
[ FAILED ] TestRunc.test_parse_runc_launch_cmd_invalid_num_reap_layers_keep
[ FAILED ] TestRunc.test_parse_runc_launch_cmd_valid_mounts
[ FAILED ] TestRunc.test_parse_runc_launch_cmd_invalid_mounts
{noformat}
It's failing because I already have a container-executor.cfg there and it is
owned by root.
If I run without defining {{container-executor.conf.dir}}, all of the tests
pass.
I was able to get this to work by modifying
test_runc_util.cc::create_ce_file():
{noformat}
int create_ce_file() {
int ret = 0;
const char *fname = HADOOP_CONF_DIR "/" CONF_FILENAME;
if (strcmp("../etc/hadoop/container-executor.cfg", fname) == 0) {
ret = mkdir("../etc", 0755);
if (ret == 0 || errno == EEXIST) {
ret = mkdir("../etc/hadoop", 0755);
if (ret == 0 || errno == EEXIST) {
write_file("../etc/hadoop/container-executor.cfg", "");
ret = 0;
} else {
std::cerr << "Could not create ../etc/hadoop, " <<
strerror(errno) << std::endl;
}
} else {
std::cerr << "Could not create ../etc, " << strerror(errno)
<< std::endl;
}
} else {
write_file(fname, "");
struct stat buffer;
if (stat(fname, &buffer) != 0) {
std::cerr << "Could not create " << fname <<
strerror(errno) << std::endl;
ret = 1;
}
}
return ret;
}{noformat}
> Add C changes for the new RuncContainerRuntime
> ----------------------------------------------
>
> Key: YARN-9561
> URL: https://issues.apache.org/jira/browse/YARN-9561
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Eric Badger
> Assignee: Eric Badger
> Priority: Major
> Attachments: YARN-9561.001.patch, YARN-9561.002.patch,
> YARN-9561.003.patch, YARN-9561.004.patch, YARN-9561.005.patch,
> YARN-9561.006.patch, YARN-9561.007.patch, YARN-9561.008.patch
>
>
> This JIRA will be used to add the C changes to the container-executor native
> binary that are necessary for the new RuncContainerRuntime. There should be
> no changes to existing code paths.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
