> On June 20, 2016, 8:09 a.m., Qian Zhang wrote: > > @haosdent, I tried in my env, it seems when cgroup is destroyed, eventfd > > will not be triggered. What I did is, use `mesos-execute` to launch a > > container to do a simple task (sleep 5 seconds), and after 5 seconds, the > > task finishes and I see the cgroup of the container is destroyed, but I do > > not see the eventfd (`memory.oom_control`) is triggered.
>it seems when cgroup is destroyed, eventfd will not be triggered. It is because when the oom_control triggered (when we cleanup the hierarchy), `info->oomNotifier.discard()` would be executed first. To verify if oom would be triggered when cgroup is destroyed, we could use below code to do a quick test. ``` // Copy from http://harasou.github.io/2015/05/20/cgroup-%E3%81%AB%E3%82%88%E3%82%8B-oom-killer-%E3%81%AE%E7%8A%B6%E6%85%8B%E3%82%92-eventfd-%E7%B5%8C%E7%94%B1%E3%81%A7%E5%8F%97%E3%81%91%E5%8F%96%E3%82%8B/ #include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> #include <sys/eventfd.h> #include <errno.h> #include <string.h> #include <stdio.h> #include <stdlib.h> static inline void die(const char *msg) { fprintf(stderr, "error: %s: %s(%d)\n", msg, strerror(errno), errno); exit(EXIT_FAILURE); } static inline void usage(void) { fprintf(stderr, "usage: oom_eventfd_test <cgroup.event_control> <memory.oom_control>\n"); exit(EXIT_FAILURE); } #define BUFSIZE 256 int main(int argc, char *argv[]) { char buf[BUFSIZE]; int efd, cfd, ofd, rb, wb; uint64_t u; if (argc != 3) usage(); if ((efd = eventfd(0, 0)) == -1) die("eventfd"); if ((cfd = open(argv[1], O_WRONLY)) == -1) die("cgroup.event_control"); if ((ofd = open(argv[2], O_RDONLY)) == -1) die("memory.oom_control"); if ((wb = snprintf(buf, BUFSIZE, "%d %d", efd, ofd)) >= BUFSIZE) die("buffer too small"); if (write(cfd, buf, wb) == -1) die("write cgroup.event_control"); if (close(cfd) == -1) die("close cgroup.event_control"); for (;;) { if (read(efd, &u, sizeof(uint64_t)) != sizeof(uint64_t)) die("read eventfd"); printf("mem_cgroup oom event received \n"); } return 0; } ``` ``` $ make oom-notify $ mkdir /sys/fs/cgroup/memory/test $ ./oom-notify /sys/fs/cgroup/memory/test/cgroup.event_control /sys/fs/cgroup/memory/test/memory.oom_control & # Now we destroy the hierarchy. $ rmdir /sys/fs/cgroup/memory/test # Now we could see the oom event is triggered. mem_cgroup oom event received ``` We could find the similar logic in [runc](https://github.com/opencontainers/runc/blob/master/libcontainer/notify_linux.go#L55-L56) which docker based. ``` // When a cgroup is destroyed, an event is sent to eventfd. // So if the control path is gone, return instead of notifying. if _, err := os.Lstat(eventControlPath); os.IsNotExist(err) { return } ``` - haosdent ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46299/#review138548 ----------------------------------------------------------- On June 19, 2016, 10:30 a.m., haosdent huang wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46299/ > ----------------------------------------------------------- > > (Updated June 19, 2016, 10:30 a.m.) > > > Review request for mesos, Gilbert Song, Guangya Liu, Ian Downes, Jie Yu, > Kevin Klues, and Qian Zhang. > > > Bugs: MESOS-5045 > https://issues.apache.org/jira/browse/MESOS-5045 > > > Repository: mesos > > > Description > ------- > > Ignore eventfd caused by hierarchy destruction. > > > Diffs > ----- > > src/linux/cgroups.cpp 95ceb373ca4d961c402f9936f31cda1a25c60e87 > > Diff: https://reviews.apache.org/r/46299/diff/ > > > Testing > ------- > > > Thanks, > > haosdent huang > >
