Hi guys, I've also tried on the Fedora 30 linux to execute the Java app with 400 threads and I must confirm that the machine memory does not go over 700 MB during the Java app execution.
It's a mistery why on OSv it requires much more memory with threads. Any hints? thanks a lot roberto Il giorno martedì 21 maggio 2019 00:49:56 UTC+2, Waldek Kozaczuk ha scritto: > > I obviously meant > cat /proc/self/maps > > Sent from my iPhone > > On May 20, 2019, at 17:55, Waldek Kozaczuk <jwkoz...@gmail.com > <javascript:>> wrote: > > I see you are adding html5 cli app. So you can open /cli console in your > favorite browser when your app is running and type this: > cat /proc/self.maps > > You should see something like this (obviously yours is going to show > different info): > > 0-0 ---p 00000000 00:00 0 > 100000000000-100000001000 r-xp 00000000 00:00 0 /libvdso.so > 100000200000-100000201000 r--p 00000000 00:00 0 /libvdso.so > 100000201000-100000202000 rw-p 00001000 00:00 0 /libvdso.so > 100000400000-100000408000 r-xp 00000000 00:00 0 /cat > 100000608000-100000609000 r--p 00008000 00:00 0 /cat > 100000609000-10000060a000 rw-p 00009000 00:00 0 /cat > 200000000000-200000001000 ---p 00000000 00:00 0 > 200000001000-200000100000 rw-p 00000000 00:00 0 > 200000100000-200000101000 ---p 00000000 00:00 0 > 200000101000-200000200000 rw-p 00000000 00:00 0 > 800000000000-800000000000 ---p 00000000 00:00 0 > > It should give you an idea of what has been mmap'ed. > > On Monday, May 20, 2019 at 12:09:49 PM UTC-4, Nadav Har'El wrote: >> >> >> On Mon, May 20, 2019 at 5:35 PM Waldek Kozaczuk <jwkoz...@gmail.com> >> wrote: >> >>> >>> >>> On Sunday, May 19, 2019 at 12:19:59 PM UTC-4, robertob wrote: >>>> >>>> Hi everybody, >>>> >>>> I have built an OSv image that run with QEMU and KVM acceleration. It >>>> executes a mine Java multi threading application. >>>> >>>> This is the command line I execute: >>>> >>>> capstan run unikure -n "bridge" -c 2 -m 4500M -e >>>> "--ip=eth0,192.168.122.101,255.255.255.0 >>>> --defaultgw=192.168.122.1 --nameserver=192.168.122.1 /java.so -Xmx2024m >>>> -jar /Uni.jar" >>>> >>>> The image require these libs: >>>> >>>> - openjdk8-zulu-compact3-with-java-beans >>>> - osv.httpserver-html5-gui-and-cli >>>> - osv.run-java >>>> - osv.cli >>>> >>>> My Java application allocates circa 400 threads and other services >>>> threads. The application has a strange behavior when it's executed in OSv: >>>> *despite >>>> the java app has set the Xmx to 1024 MB and It never* >>>> >>> >> 1024, or 2024 as in the above example? Looking at the printouts above, >> it's probably 2024. >> >> * reaches this maximum memory size, the execution in OSv halts and after >>>> some time I get a memory allocation error*. This seems so strange to >>>> me. Maybe Java threads occupy more memory when executed in OSv? >>>> >>> Are there 400 threads Java ones or all threads including OSv kernel >>> ones? Can you tell us roughly what this app does? Also when you run it on >>> Linux (same JRE, command line) does it use less memory? >>> >> >> Besides the 2024 MB used by Java, OSv uses additional memory for >> threads, the filesystem, and a lot of other stuff. >> Note in particular that Java will use 2024 MB for its heap, but can use >> additional memory depending on your application - some applications >> deliberately use mmap() or otherwise allocate memory outside the heap, and >> Java uses off-heap memory - unless I'm misremembering it has a 1MB stack >> per thread so if you have 400 Java threads, this adds up, there is a >> permanent generation, there is loaded class files (can be big in a big Java >> program), etc. And of course there are also OSv overheads. >> >> Unfortunately >> https://github.com/cloudius-systems/osv/wiki/Debugging-OSv >> doesn't include much explanations on how to debug what in OSv is taking >> memory, it should probably be added. >> But another thing you can try is to run the same code on Linux, and see >> there how small you can make the VM. If it's considerably smaller than what >> OSv can do, maybe you uncovered a bug. >> >> By the way, are you using the ZFS filesystem or something else (ROFS, >> etc.)? >> >> >>>> I've tried different memory sizes for the execution of the image until >>>> 4500 MB. With that size the run terminates correctly, otherwise It halts. >>>> >>> >>>> Follows the log of my application with Heap memory of Java application >>>> and at the end the OSv image error. >>>> >>>> >>>> >>>> 2019-05-19 09:17:42,508 DEBUG [g279TC#192.168.122.101:45830-ServerThread] >>>> ==> [HEAP] 1800M (max);118M (tot);90M (free); >>>> >>>> Out of memory: could not reclaim any further. Current memory: 2036 Kb >>>> [backtrace] >>>> 0x00000000003ebe2c <memory::oom()+28> >>>> 0x00000000003ecd4c <memory::reclaimer::_do_reclaim()+380> >>>> 0x00000000003ecd9b <???+4115867> >>>> 0x0000000000400aa6 <thread_main_c+38> >>>> 0x00000000003a60a2 <???+3825826> >>>> >>>> Any suggestions? >>>> >>> >>> Have you tried to connect with something like VisualVM and profile it >>> this way? >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "OSv Development" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to osv...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/osv-dev/8af6dd05-e1bb-4547-b8db-8ccef73cde4a%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/osv-dev/8af6dd05-e1bb-4547-b8db-8ccef73cde4a%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- > You received this message because you are subscribed to a topic in the > Google Groups "OSv Development" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/osv-dev/LqUAWjzVpSc/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > osv...@googlegroups.com <javascript:>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/osv-dev/d644a8bf-b506-4167-af73-25f6a7c0322f%40googlegroups.com > > <https://groups.google.com/d/msgid/osv-dev/d644a8bf-b506-4167-af73-25f6a7c0322f%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > > -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to osv-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/07a92e7b-0963-4cd2-b12f-4acea518f848%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
