On Thu, Nov 22, 2018 at 12:30 AM Waldek Kozaczuk <[email protected]> wrote:
> Based on one old patch from 2015 it looks some basic single numa setup > should be supported: > It should be "supported" in the sense that whatever NUMA configuration you try to set up, OSv will not know about it (it doesn't have the code to query the VM for its NUMA configuration), and even if it did, malloc() allocates memory from the entire memory range and doesn't make any attempt to allocate from different areas depending on the current core. I thought we had an open issue about that, but I can't find it now so I guess we don't. So currently, OSv is supposed to tell libnuma users that there is just one NUMA node. For example, get_mempolicy() works (we fixed it by implementing syscall(__NR_get_mempolicy,...) - see linux.cc). It is possible, even likely, we missed other system calls that libnuma relies on, because nobody tested it heavily (it was only tested by the XLAB guys who ran OpenMPI on OSv). > So when I run ffmpeg with x265 codec trying to use NUMA (I have libnuma.so > in the image): > > qemu-system-x86_64 -m 4G -smp 4 -vnc :1 \ > -gdb tcp::1234,server,nowait -device > virtio-blk-pci,id=blk0,drive=hd0,scsi=off,bootindex=0 \ > -drive > file=/home/wkozaczuk/projects/osv/build/last/usr.img,if=none,id=hd0,cache=none,aio=native > \ > -netdev user,id=un0,net=192.168.122.0/24,host=192.168.122.1 -device > virtio-net-pci,netdev=un0 \ > -device virtio-rng-pci \ > * -numa node,mem=4G,cpus=0-3,nodeid=0 \* > -enable-kvm -cpu host,+x2apic \ > -chardev stdio,mux=on,id=stdio,signal=off \ > -mon chardev=stdio,mode=readline -device isa-serial,chardev=stdio > > I get it correctly transcode video however I see this warning from x265 > code (see corresponding code - > https://github.com/videolan/x265/blob/aaa112abf66c88730e679e574c5b5be83bf5659b/source/common/threadpool.cpp#L272-L294 > ): > ... > x265 [error]: Failed to genrate CPU mask > So apparently numa_node_to_cpus() failed. You should look at libnuma's source and see what this function does. Perhaps it is using a system call (which is supposed to return the list of cpus, or number of cpus, or something) that we forgot to implement. x265 [warning]: No thread pool allocated, --wpp disabled > x265 [warning]: No thread pool allocated, --lookahead-slices disabled > ... > > What am I missing? > > Waldek > > On Tuesday, November 20, 2018 at 12:51:49 AM UTC-5, Waldek Kozaczuk wrote: >> >> As I understand adding numa support to OSv is probably not a trivial task >> (but maybe not as difficult?) - https://linux.die.net/man/3/numa. One of >> the ffmpeg codec libraries wants to uses some of these numa functions. It >> turns out that adding libnuma.so from Linux host make the codec fail >> gracefully. >> >> But maybe the right and fairly easy way for now would be to add >> >> int numa_available(void) >> { >> return -1; >> } >> >> to at least clearly indicate numa support is unavailable rather than >> crash with symbol missing error? >> >> Waldek >> >> > -- > You received this message because you are subscribed to the Google Groups > "OSv Development" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
