Hi,
This is the first LARGE project I'm dealing with, it is exciting but
also a bit complex to browse through even with cscope and ctags.
I'm trying to play with the part of the code that is responsible for
killing a job if it exceeds a memory limit, for instance via cgroups or so.
If I understood correctly, job/step would be given the signal SIG_OOM
and killed eventually with SIG_KILL/SIG_TERM.
My question is, who is responsible for killing the job? is it
src/slumd/slumstepd/mgr.c or is it the proctrack plugin?
Running through mgr.c, this is what I understood.
1290 /* Attach slurmstepd to system cgroups, if configured */
1291 attach_system_cgroup_pid(getpid()); *-> I assume this is where
we also attach the pid of the workload to a specified cgroup.*
1292
1293 /* if we are not polling then we need to make sure we get some
1294 * information here
1295 */
1296 if (!conf->job_acct_gather_freq)
1297 jobacct_gather_stat_task(0); *-> we still enter here
because we need information cgroups stats obtained _prec_extra of
/plugins/jobacct_gather/cgroup/jobacct_gather_cgroup.c*
1298
1299 /* Send job launch response with list of pids */
1300 _send_launch_resp(job, 0);
1301 _set_job_state(job, SLURMSTEPD_STEP_RUNNING); *-> i do not see
the SIG_OOM being set here...*
1302
1303 #ifdef PR_SET_DUMPABLE
1304 /* RHEL6 requires setting "dumpable" flag AGAIN; after euid
changes */
1305 if (prctl(PR_SET_DUMPABLE, 1) < 0)
1306 debug ("Unable to set dumpable to 1");
1307 #endif /* PR_SET_DUMPABLE */
1308
1309 _wait_for_all_tasks(job); *-> this is where we already have the
signal set to 9 already.. who sets it actually?*
1310 acct_gather_profile_endpoll();
1311 acct_gather_profile_g_node_step_end();
1312 acct_gather_profile_fini();
1313
1314 _set_job_state(job, SLURMSTEPD_STEP_ENDING);
Am I looking at the correct place?
Any help in this regard would be helpful. I look forward to your response.
Best wishes,
Rajiv
