On 10-Sep-20 5:24 PM, David Marchand wrote:
This is something we encountered while working in an OpenShift environment with SELinux enabled. In this environment, a DPDK application could create/write to hugepage files but removing them was refused. This resulted in dirty files being reused when starting a new DPDK application and triggered random crashes / erratic behavior.Getting a SELinux setup can be a challenge, and even more if you add containers to the picture :-). So here is a reproducer for the interested testers: # cat >wrap.c <<EOF #define _GNU_SOURCE #include <dlfcn.h> #include <errno.h> #include <stdio.h> #include <string.h> #include <sys/stat.h> #include <sys/types.h> #include <unistd.h> int unlink(const char *pathname) { static int (*orig)(const char *pathname) = NULL; struct stat st; if (orig == NULL) orig = dlsym(RTLD_NEXT, "unlink"); if (strstr(pathname, "rtemap_") != NULL && stat(pathname, &st) == 0) { fprintf(stderr, "### refused unlink for %s\n", pathname); errno = EACCES; return -1; } fprintf(stderr, "### called unlink for %s\n", pathname); return orig(pathname); } int unlinkat(int dirfd, const char *pathname, int flags) { static int (*orig)(int dirfd, const char *pathname, int flags) = NULL; struct stat st; if (orig == NULL) orig = dlsym(RTLD_NEXT, "unlinkat"); if (strstr(pathname, "rtemap_") != NULL && fstatat(dirfd, pathname, &st, flags) == 0) { fprintf(stderr, "### refused unlinkat for %s\n", pathname); errno = EACCES; return -1; } fprintf(stderr, "### called unlinkat for %s\n", pathname); return orig(dirfd, pathname, flags); } EOF # gcc -fPIC -shared -o libwrap.so wrap.c -ldl # \rm /dev/hugepages/rtemap* # # First run is fine # LD_PRELOAD=libwrap.so dpdk-testpmd -w 0000:01:00.0 -- -i [...] Configuring Port 0 (socket 0) Port 0: 24:6E:96:3C:52:D8 Checking link statuses... Done testpmd> # # Second run we have dirty memory # LD_PRELOAD=libwrap.so dpdk-testpmd -w 0000:01:00.0 -- -i [...] ### refused unlinkat for rtemap_0 [...] Port 0 is now not stopped Please stop the ports first Done testpmd> Removing hugepage files is done in multiple places and the memory allocation code is complex. This fix tries to do the minimum and avoids touching other paths. If trying to remove the hugepage file before allocating a page fails, the error is reported to the caller and the user will see a memory allocation error log. Fixes: 582bed1e1d1d ("mem: support mapping hugepages at runtime") Cc: [email protected] Signed-off-by: David Marchand <[email protected]> ---
I believe only the primary will try to allocate new pages, but this only covers the page-per-file scenario. There's also legacy mem option (which would attempt to remove files prior to creating new ones - not sure if it's refused in that case), and single file segments option (which will mostly fallocate holes rather than delete files, but may still attempt to delete files - by the way, how does fallocate work with SELinux?).
So, for the contents of the patch in question, it's good, but *might* be incomplete for the above reasons. I can't test this right this moment so i'll leave this up to you :P
-- Thanks, Anatoly
