I looked at this code briefly and the main reason OSv fails many of these queries is lack of sysfs support in OSv - https://github.com/numactl/numactl/blob/1959c774398fa6859ff98dde76c01c7579be2731/libnuma.c#L1268-L1339.
It looks for files and directories like these: /sys/devices/system/node /sys/devices/system/node/node%d/meminfo /sys/devices/system/node/node%d/cpumap Does not seem to bad to create something similar to procfs that would implement sysfs. On Thursday, November 22, 2018 at 3:45:46 AM UTC-5, Nadav Har'El wrote: > > On Thu, Nov 22, 2018 at 12:30 AM Waldek Kozaczuk <[email protected] > <javascript:>> wrote: > >> Based on one old patch from 2015 it looks some basic single numa setup >> should be supported: >> > > It should be "supported" in the sense that whatever NUMA configuration you > try to set up, OSv will not know about it (it doesn't have the code to > query the VM for its NUMA configuration), and even if it did, malloc() > allocates memory from the entire memory range and doesn't make any attempt > to allocate from different areas depending on the current core. I thought > we had an open issue about that, but I can't find it now so I guess we > don't. > > So currently, OSv is supposed to tell libnuma users that there is just one > NUMA node. For example, get_mempolicy() works (we fixed it by implementing > syscall(__NR_get_mempolicy,...) - see linux.cc). It is possible, even > likely, we missed other system calls that libnuma relies on, because nobody > tested it heavily (it was only tested by the XLAB guys who ran OpenMPI on > OSv). > > >> So when I run ffmpeg with x265 codec trying to use NUMA (I have >> libnuma.so in the image): >> >> qemu-system-x86_64 -m 4G -smp 4 -vnc :1 \ >> -gdb tcp::1234,server,nowait -device >> virtio-blk-pci,id=blk0,drive=hd0,scsi=off,bootindex=0 \ >> -drive >> file=/home/wkozaczuk/projects/osv/build/last/usr.img,if=none,id=hd0,cache=none,aio=native >> >> \ >> -netdev user,id=un0,net=192.168.122.0/24,host=192.168.122.1 -device >> virtio-net-pci,netdev=un0 \ >> -device virtio-rng-pci \ >> * -numa node,mem=4G,cpus=0-3,nodeid=0 \* >> -enable-kvm -cpu host,+x2apic \ >> -chardev stdio,mux=on,id=stdio,signal=off \ >> -mon chardev=stdio,mode=readline -device isa-serial,chardev=stdio >> >> I get it correctly transcode video however I see this warning from x265 >> code (see corresponding code - >> https://github.com/videolan/x265/blob/aaa112abf66c88730e679e574c5b5be83bf5659b/source/common/threadpool.cpp#L272-L294 >> ): >> ... >> x265 [error]: Failed to genrate CPU mask >> > > So apparently numa_node_to_cpus() failed. You should look at libnuma's > source and see what this function does. Perhaps it is using a system call > (which is supposed to return the list of cpus, or number of cpus, or > something) that we forgot to implement. > > x265 [warning]: No thread pool allocated, --wpp disabled >> x265 [warning]: No thread pool allocated, --lookahead-slices disabled >> ... >> >> What am I missing? >> >> Waldek >> >> On Tuesday, November 20, 2018 at 12:51:49 AM UTC-5, Waldek Kozaczuk wrote: >>> >>> As I understand adding numa support to OSv is probably not a trivial >>> task (but maybe not as difficult?) - https://linux.die.net/man/3/numa. >>> One of the ffmpeg codec libraries wants to uses some of these numa >>> functions. It turns out that adding libnuma.so from Linux host make the >>> codec fail gracefully. >>> >>> But maybe the right and fairly easy way for now would be to add >>> >>> int numa_available(void) >>> { >>> return -1; >>> } >>> >>> to at least clearly indicate numa support is unavailable rather than >>> crash with symbol missing error? >>> >>> Waldek >>> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "OSv Development" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
