On Mon, Feb 11, 2019 at 1:33 AM Waldek Kozaczuk <[email protected]>
wrote:
> Here is the scenario:
> 1) Build image with some app X and httpserver API module.
> 2) Start app X using REST API /app call with new_program=false (same
> namespace)
> 3) OSv loads app library and any dependent ones from disk and executes it.
> 4) Start app X again same way as in 2) over and over again (unlimited
> times) BUT OSv skips load any libraries from disk instead re-uses relevant
> ELF objects already loaded/mmaped in memory.
>
> This actually works using simple native-example. But fails on a second run
> with ffmpeg app where I get this type of stack trace:
> 0x00000000003a9482 in processor::cli_hlt () at arch/x64/processor.hh:247
> 247 asm volatile ("cli; hlt" : : : "memory");
> (gdb) bt
> #0 0x00000000003a9482 in processor::cli_hlt () at
> arch/x64/processor.hh:247
> #1 arch::halt_no_interrupts () at arch/x64/arch.hh:48
> #2 osv::halt () at arch/x64/power.cc:24
> #3 0x000000000023f144 in abort (fmt=fmt@entry=0x63335b "Aborted\n") at
> runtime.cc:132
> #4 0x0000000000202765 in abort () at runtime.cc:98
> #5 0x000000000045dc9c in osv::generate_signal (siginfo=...,
> ef=0xffff800003099068) at libc/signal.cc:124
> #6 0x000000000045dd0b in osv::handle_mmap_fault (addr=<optimized out>,
> sig=<optimized out>, ef=<optimized out>) at libc/signal.cc:139
> #7 0x0000000000347bf3 in mmu::vm_fault (addr=0, addr@entry=56,
> ef=ef@entry=0xffff800003099068) at core/mmu.cc:1338
> #8 0x00000000003a32de in page_fault (ef=0xffff800003099068) at
> arch/x64/mmu.cc:38
> #9 <signal handler called>
> #10 0x0000100003d39d6c in ?? ()
> #11 0x00002000003ff220 in ?? ()
> #12 0xffffa00002ce7ada in ?? ()
> #13 0x0000000000000000 in ?? ()
>
> If I set new_program=true ffmpeg runs fine every time up to 32 times.
>
> So my motivation is to be able to run given application on OSv
> sequentially one by one without having to restart OSv and only pay price
> for loading code in memory once on the first run.
>
osv::application:run() and osv::run() have accumulated so many layers over
the year that I forget all their details and you may need to look at the
actual code (the REST API calls osv::application:run() to do its work). I
do believe that after the application finishes running, it indeed unloads
the library by default.
One thing you could do is after app= osv::application::run(...), you can do
something like
auto lib = app.lib();
// as long as this "lib" exists, the library will not be unloaded.
You can even create an eternal copy with something like:
new std::shared_ptr<elf::object>(lib);
However, *not* unloading the library after the run finishes may not make it
easier to run it again - it might even be the opposite, e.g., if the
program needs some global variable to be set on load to some default
values. Some programs may not be possible to run a second time if they
leave behind too stuff they don't clean, etc, but this is probably not your
case. I wonder if in your case somehow the library did *not* unload, and
that caused the change from the "new_program" case.
Can you please try to look at the backtrace with gdb, instead of the above,
maybe we'll get more information?
>
> I have not analyzed OSv code well enough to understand if that is what
> already happens. But as the ffmpeg example shows it comes with some quirks.
> Also this would be best effort and only work if app cleans itself after
> completion which I think is the case for ffmpeg but may not the case for
> all kinds of apps.
>
Indeed.
> Looking at elf.cc it looks like the elf segments get unmapped in
> remove_objects() that is setup to be called once they are no longer
> referenced. So ideally we would want to have an option of skipping
> unmapping the ELF so that when app is re-executed again we skip the mmaping
> part as code would be still in memory.
>
> I do not understand ELF lifecycle well enough to know if that is even
> possible.
>
> --
> You received this message because you are subscribed to the Google Groups
> "OSv Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups "OSv
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
