Wow, awesome results! I will try to reproduce shortly from here! If you want to increase the problem size, add -n 16777216 (which is 64x bigger than the current problem size of 262144, something will take take a couple of seconds to run).
I am taking note of your commands to run the build and will be doing the same on my machine to see what I produce. I will post screenshots :) On Tuesday, February 25, 2020 at 1:09:08 PM UTC-5, Waldek Kozaczuk wrote: > > So I did try to build and run the radix test (please note my Ubuntu laptop > has only 4 cores and hyper-threading disabled). BTW it seems that > particular benchmark does not need read-write FS so I used ROFS): > > ./scripts/manifest_from_host.sh -w ../splash2-posix/kernels/radix/radix && > ./scripts/*build* fs=rofs --append-manifest -j4 > > Linux host 1 cpu: > > ./radix -p 1 -r4096 > > > Integer Radix Sort > > 262144 Keys > > 1 Processors > > Radix = 4096 > > Max key = 524288 > > > > PROCESS STATISTICS > > Total Rank Sort > > Proc Time Time Time > > 0 7335 2568 4765 > > > TIMING INFORMATION > > Start time : 1582652832386234 > > Initialization finish time : 1582652832444092 > > Overall finish time : 1582652832451427 > > Total time with initialization : 65193 > > Total time without initialization : 7335 > > > Linux host 2 cpus: > ./radix -p 2 -r4096 > > Integer Radix Sort > 262144 Keys > 2 Processors > Radix = 4096 > Max key = 524288 > > > PROCESS STATISTICS > Total Rank Sort > Proc Time Time Time > 0 4325 1571 2704 > > TIMING INFORMATION > Start time : 1582652821496771 > Initialization finish time : 1582652821531279 > Overall finish time : 1582652821535604 > Total time with initialization : 38833 > Total time without initialization : 4325 > > host 4 cpus: > ./radix -p 4 -r4096 > > Integer Radix Sort > 262144 Keys > 4 Processors > Radix = 4096 > Max key = 524288 > > > PROCESS STATISTICS > Total Rank Sort > Proc Time Time Time > 0 2599 1077 1470 > > TIMING INFORMATION > Start time : 1582653906150199 > Initialization finish time : 1582653906171932 > Overall finish time : 1582653906174531 > Total time with initialization : 24332 > Total time without initialization : 2599 > > > OSv 1 CPU > ./scripts/run.py -p qemu_microvm --qemu-path > /home/wkozaczuk/projects/qemu/bin/release/native/x86_64-softmmu/qemu-system-x86_64 > > --nics 0 --nogdb -m 64M -c 1 --block-device-cache writeback,aio=threads -e > '/radix -p 1 -r4096' > OSv v0.54.0-119-g4ee4b788 > Booted up in 3.75 ms > Cmdline: /radix -p 1 -r4096 > > Integer Radix Sort > 262144 Keys > 1 Processors > Radix = 4096 > Max key = 524288 > > > PROCESS STATISTICS > Total Rank Sort > Proc Time Time Time > 0 6060 2002 4049 > > TIMING INFORMATION > Start time : 1582652845450708 > Initialization finish time : 1582652845500348 > Overall finish time : 1582652845506408 > Total time with initialization : 55700 > Total time without initialization : 6060 > > OSv 2 CPUs: > ./scripts/run.py -p qemu_microvm --qemu-path > /home/wkozaczuk/projects/qemu/bin/release/native/x86_64-softmmu/qemu-system-x86_64 > > --nics 0 --nogdb -m 64M -c 2 --block-device-cache writeback,aio=threads -e > '/radix -p 2 -r4096' > OSv v0.54.0-119-g4ee4b788 > Booted up in 4.81 ms > Cmdline: /radix -p 2 -r4096 > > Integer Radix Sort > 262144 Keys > 2 Processors > Radix = 4096 > Max key = 524288 > > > PROCESS STATISTICS > Total Rank Sort > Proc Time Time Time > 0 5797 1702 4089 > > TIMING INFORMATION > Start time : 1582653305076852 > Initialization finish time : 1582653305129792 > Overall finish time : 1582653305135589 > Total time with initialization : 58737 > Total time without initialization : 5797 > > OSv 4 cpus > ./scripts/run.py -p qemu_microvm --qemu-path > /home/wkozaczuk/projects/qemu/bin/release/native/x86_64-softmmu/qemu-system-x86_64 > > --nics 0 --nogdb -m 64M -c 4 --block-device-cache writeback,aio=threads -e > '/radix -p 4 -r4096' > OSv v0.54.0-119-g4ee4b788 > Booted up in 5.26 ms > Cmdline: /radix -p 4 -r4096 > > Integer Radix Sort > 262144 Keys > 4 Processors > Radix = 4096 > Max key = 524288 > > > PROCESS STATISTICS > Total Rank Sort > Proc Time Time Time > 0 6498 2393 4099 > > TIMING INFORMATION > Start time : 1582653946823458 > Initialization finish time : 1582653946875522 > Overall finish time : 1582653946882020 > Total time with initialization : 58562 > Total time without initialization : 6498 > > > As you can see with single CPU the benchmark seems to be 10-15 % faster. > But with two and four CPUs OSv barely sees any improvements, whereas on > host the app runs 40% faster. So OSv does not seem to scale at all > (somebody mentioned it used to) so it would be nice to understand why. OSv > has many sophisticated tracing tools that can help here - > https://github.com/cloudius-systems/osv/wiki/Trace-analysis-using-trace.py > > Waldek > > BTW1. I tried to bump size of the matrix to something higher but with > -r8192 the app crashes on both Linux and OSv. > BTW2. It would be interestingly to compare OSv with Linux guest (vs host). > > On Tuesday, February 25, 2020 at 10:05:08 AM UTC-5, [email protected] > wrote: >> >> Thanks for the response! I will get this information to you after work >> with the few modifications you recommended! The application is essentially >> just testing CPU performance using multiprocessing. Nothing too fancy about >> it! The code I am using can be found at: >> >> https://www.github.com/ProfessorWest/splash2-posix >> >> In side of the kernels folder located at radix.c and I change the problem >> size to 16,777,206. >> >> If you happen to examine the code, do ignore the lacking cleanness of the >> code...we just smashed everything into one file for simplicity on our end. >> (Running the same code across all platforms being benchmarked). >> >> On Tuesday, February 25, 2020 at 8:52:48 AM UTC-5, Waldek Kozaczuk wrote: >>> >>> Hi, >>> >>> I am quite late to the party :-) Could you run OSv on single CPU with >>> verbose on (add -V to run.py) and send us the output so we can see a little >>> more what is happening. To disable networking you need to add '--nics=0' >>> (for all 50 options run.py supports run it with '--help'). I am not >>> familiar with that benchmark but I wonder if it needs read-write FS (ZFS in >>> OSv case), if not that you can build OSv images with read-only FS >>> (./scripts/build fs=rofs). Lastly, you can improve boot time by running OSv >>> on firecracker ( >>> https://github.com/cloudius-systems/osv/wiki/Running-OSv-on-Firecracker) >>> or on QEMU microvm (-p qemu_imcrovm - requires QEMU >= 4.1), with read-only >>> FS on both OSv should boot within 5ms, ZFS within 40ms). Last thing - >>> writing to console on OSv can be quite slow, I wonder how much this >>> benchmark does it. >>> >>> While I definitely agree with my colleague Nadav, where he essentially >>> says do not use OSv if the raw performance matters (database for example) >>> and Linux will beat it no matter what, OSv may have advantages in use cases >>> where pure performance does not matter (it still needs to be reasonable). I >>> think the best use cases for OSv are serverless or stateless apps >>> (microservices or web assembly) running on single CPU where all state >>> management is delegated to a remote persistent store (most custom-built >>> business apps are like that) and where high isolation matters. >>> >>> Relatedly, I think it might be more useful to think of OSv (and other >>> unikernels) as highly isolated processes. To that end, we still need to >>> optimize memory overhead (stacks for example) and improve virtio-fs support >>> (in this case to run the app on OSv you do not need full image, just kernel >>> to run a Linux app). >>> >>> Also, I think the lack of good tooling in unikernel space affects their >>> adoption. Compare it with docker - build, push, pull, run. OSv has its >>> equivalent - capstan - but at this point, we do not really have a registry >>> where one can pull the latest OSv kernel or push, pull images. Trying to >>> run an app on OSv is still quite painful to a business app developer - it >>> probably takes at least 30 minutes or so. >>> >>> Lastly, I think one of the main reasons for Docker adoption, was >>> repeatability (besides its fantastic ease of use) where one can create an >>> image and expect it to run almost the same way in production. Imagine you >>> can achieve that with OSv. >>> >>> Waldek >>> >>> On Tuesday, February 25, 2020 at 7:00:16 AM UTC-5, [email protected] >>> wrote: >>>> >>>> Very well explained. Thank you for that. That does make perfect sense >>>> as well. >>> >>> -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/86749b7a-c3da-45ad-ba3a-f3e4628e9197%40googlegroups.com.
