Hi Stephen, Thanks for your reply!
I'm sure the protection bit is correctly OR-ed with PROT_EXEC. I use mprotect() on the entire hugepage on both primary and secondary processes, and /proc/pid/maps file can verify that. Thanks for your hint on libhugetlbfs. I skimmed through its HOWTO and found that it uses a custom linker script to load the text/data/bss segment on hugepages. However, I didn't see how it can cooperate with the shared objects. And I also found a SO question on how to do that: https://stackoverflow.com/questions/40285971/how-to-load-text-segments-of-shared-libraries-into-huge-pages-on-linux Also, as far as I know, the loading of shared objects is handled by ld-linux.so, and changing the behavior of ld-linux.so needs some source level modifications or symbol interpositioning. I compile libhugetlbfs but find no obvious sign of symbol interpositioning. -- Best, Fengkai On Tue, Jul 20, 2021 at 11:44 PM Stephen Hemminger < [email protected]> wrote: > On Tue, 20 Jul 2021 17:28:59 +0800 > Fengkai Sun <[email protected]> wrote: > > > Hi list, > > > > Recently I read about the multi-process support of dpdk and found the > > feature of keeping the hugepage mappings between primary and secondary > > processes very interesting. > > Therefore, I made the following experiments: > > > > 1. I modify the glibc to make dlopen() map shared libraries on hugepages. > > This is done by changing the underlying mapping strategy from mmap to > pread. > > 2. I allocate a continuous address by rte_malloc(), and instruct dlopen() > > to map a shared library there, let's call it lib1.so. > > 3. After dlopen() succeeds, I use dlsym() in the primary process to > locate > > the global variables and functions in lib1.so, and it turns out they are > > correctly mapped and functions can be called without a problem. > > 4. Because hugepage seems to get away with the effect of ASLR, I record > the > > address of those global variables and functions in lib1.so, and verify > them > > in the secondary process. > > The secondary can access the global variables at the same address, > but > > when it tries to call the function, a segfault occurs. > > 5. I try to use dlopen() with the same arguments as the primary process > in > > the secondary process, but it just gives a segfault. > > > > Unfortunately, gdb also gives a segfault when the program starts up, so I > > cannot give some useful debug info. > > > > My question is, does dpdk permit functions to be loaded on hugepages and > be > > called by multiple functions? Though the two processes see the exact same > > content on the hugepage, the secondary just cannot call the function on > it. > > > Sounds like no-exec bit. > You might want to look into other libraries that already handle huge > pages and libraries such as hugetlbfs > > >
