On Thu, Jan 23, 2020 at 2:23 PM Waldek Kozaczuk <[email protected]>
wrote:
> Can you send us the full output? I wonder if there are any warnings before
> it?
>
> I see you are running this on firecracker. Can you run it under qemu and
> connect to it with gdb and see if you get better stacktrace?
>
> You can add another debug statement like that:
>
> if (strcmp(" /lib/python3.6/google/protobuf/pyext/_message.
> cpython-36m-x86_64-linux-gnu.so",pathname)==0 && i == 28) {
> ...
> // Put breakpoint here
> }
>
> and try to see what statement causes the fault. Make sure to do 'osv syms"
> to get as much debug info resolved as possible. (see
> https://github.com/cloudius-systems/osv/wiki/Debugging-OSv).
>
I hope that "osv syms" will find the
newly-loaded-but-not-yet-completely-loaded libraries. If it doesn't, maybe
we can fix the order of when the array that "osv syms" uses gets written
during loaded.
>
> I wonder if this has to with the order of initializing the ELF objects
> when called by dlopen().
>
> Waldek
>
> PS. If no more clue next step would be to add an app so we can build it
> and reproduce it?
>
> On Wednesday, January 22, 2020 at 6:23:30 PM UTC-5, zhiting zhu wrote:
>>
>> Hey,
>>
>> I'm hitting this error:
>>
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing DT_INIT function
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Finished executing DT_INIT
>> function
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 56 DT_INIT_ARRAYSZ
>> functions
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 0 init func
>> 0x1000388ad500
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 1 init func
>> 0x1000388ac8a0
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 2 init func
>> 0x1000388ac8d0
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 3 init func
>> 0x1000388ac900
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 4 init func
>> 0x1000388ac930
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 5 init func
>> 0x1000388ac960
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 6 init func
>> 0x1000388ac990
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 7 init func
>> 0x1000388ac9c0
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 8 init func
>> 0x1000388ac9f0
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 9 init func
>> 0x1000388aca20
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 10 init func
>> 0x1000388aca50
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 11 init func
>> 0x1000388aca80
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 12 init func
>> 0x1000388acab0
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 13 init func
>> 0x1000388acae0
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 14 init func
>> 0x1000388acb10
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 15 init func
>> 0x1000388acb40
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 16 init func
>> 0x1000388acb70
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 17 init func
>> 0x1000388acc50
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 18 init func
>> 0x1000388acc80
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 19 init func
>> 0x1000388accb0
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 20 init func
>> 0x1000388acce0
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 21 init func
>> 0x1000388acd10
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 22 init func
>> 0x1000388acd40
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 23 init func
>> 0x1000388acd70
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 24 init func
>> 0x1000388acda0
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 25 init func
>> 0x1000388acdd0
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 26 init func
>> 0x1000388ace00
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 27 init func
>> 0x1000388ace50
>> ELF [tid:51, /lib/python3.6/google/protobuf/pyext/_
>> message.cpython-36m-x86_64-linux-gnu.so]: Executing 28 init func
>> 0x1000388ace80
>> Aborted
>>
>> [backtrace]
>> 0x0000000040463abb <osv::generate_signal(siginfo&, exception_frame*)+59>
>> 0x0000000040463b2a <osv::handle_mmap_fault(unsigned long, int,
>> exception_frame*)+26>
>> 0x000000004032f3e9 <mmu::vm_fault(unsigned long, exception_frame*)+185>
>> 0x000000004038f7b6 <page_fault+166>
>> 0x000000004038e5f6 <???+1077470710>
>> 0x0000000040341042 <elf::program::init_library(int, char**)+402>
>> 0x00000000403473db
>> <elf::program::get_library(std::__cxx11::basic_string<char,
>> std::char_traits<char>, std::allocator<char> >,
>> std::vector<std::__cxx11::basic_string<char, std::char_traits<char>,
>> std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char,
>> std::char_traits<char>, std::allocator<char> > > >, bool)+715>
>> 0x0000000040462f19 <dlopen+153>
>> 0x0000100000937228 <_PyImport_FindSharedFuncptr+376>
>> 0x006567617373656c <???+1936942444>
>> 2020-01-22T17:13:51.345740567 [anonymous-instance:ERROR:vmm/src/
>> lib.rs:1658] Failed to log metrics: Logger was not initialized.
>>
>> Is there any clue how to debug this? The function pointer seems to point
>> to a valid address but I get a seg fault when executing it.
>>
>> Best,
>> Zhiting
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "OSv Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/osv-dev/ecdc93a7-2a7e-4d39-87e9-6de15578b7df%40googlegroups.com
> <https://groups.google.com/d/msgid/osv-dev/ecdc93a7-2a7e-4d39-87e9-6de15578b7df%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups "OSv
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/osv-dev/CANEVyjsYXNroc3%3DeXB7Z0cV-rfeO9yUGpkDGZP87LDNunXHocQ%40mail.gmail.com.
