After installing python3-grpcio-dbgsym, we see
gdb --args python3 -c 'import grpc; from concurrent import futures;
grpc.server(futures.ThreadPoolExecutor(max_workers=1))'
ctrl-C at the hang. The backtrace shows __wrap_memcpy in grpc's core
module:
(gdb) bt
#0 __wrap_memcpy (destination=0x558f655de140, source=0x7ffd4ba5d453, num=6)
at src/core/lib/gpr/wrap_memcpy.cc:33
#1 0x00007f652d8fe96b in memcpy (__len=6, __src=0x7ffd4ba5d453,
__dest=<optimized out>) at
/usr/include/x86_64-linux-gnu/bits/string_fortified.h:29
#2 gpr_strdup (src=0x7ffd4ba5d453 "DEBUG") at src/core/lib/gpr/string.cc:48
#3 0x00007f652d902b70 in grpc_core::GlobalConfigEnv::GetValue
(this=0x7f652dad15c0 <g_env_grpc_verbosity>) at
src/core/lib/gprpp/global_config_env.cc:75
#4 grpc_core::GlobalConfigEnvString::Get (this=0x7f652dad15c0
<g_env_grpc_verbosity>) at src/core/lib/gprpp/global_config_env.cc:126
#5 0x00007f652d946768 in gpr_global_config_get_grpc_verbosity () at
src/core/lib/gpr/log.cc:31
#6 gpr_log_verbosity_init () at src/core/lib/gpr/log.cc:78
#7 do_basic_init () at src/core/lib/surface/init.cc:69
#8 0x00007f652ee99ee8 in __pthread_once_slow (once_control=0x7f652daf08e8
<_ZL12g_basic_init.lto_priv.0>, init_routine=0x7f652d946740 <do_basic_init()>)
at ./nptl/pthread_once.c:116
...
#35 0x0000558f64a95d6d in Py_BytesMain ()
#36 0x00007f652ee29d90 in __libc_start_call_main
(main=main@entry=0x558f64a95d30, argc=argc@entry=3,
argv=argv@entry=0x7ffd4ba5b528) at ../sysdeps/nptl/libc_start_call_main.h:58
#37 0x00007f652ee29e40 in __libc_start_main_impl (main=0x558f64a95d30,
argc=3, argv=0x7ffd4ba5b528, init=<optimized out>, fini=<optimized out>,
rtld_fini=<optimized out>, stack_end=0x7ffd4ba5b518) at ../csu/libc-start.c:392
#38 0x0000558f64a95c65 in _start ()
The __wrap_memcpy definition is jumping back to itself
(gdb) disas
Dump of assembler code for function __wrap_memcpy(void*, void const*, size_t):
0x00007f652d901cf0 <+0>: endbr64
=> 0x00007f652d901cf4 <+4>: jmp 0x7f652d901cf0 <__wrap_memcpy(void*, void
const*, size_t)>
End of assembler dump.
The __wrap_memcpy function is defined to call memcpy, which goes to
__wrap_memcpy since the python bindings link with -Wl,-wrap,memcpy:
- memcpy definition:
https://github.com/grpc/grpc/blob/v1.30.2/src/core/lib/gpr/wrap_memcpy.cc#L32-L35
- python linker config:
https://github.com/grpc/grpc/blob/v1.30.2/setup.py#L203-L204
Note: It looks like the ruby bindings also use that option. If there are
server-creation hangs reported on any deb packages for the ruby gem, it
may be the same underlying issue.
The link-time optimizer probably sees that this resolves to calling
itself so it just replaces the recursive call with that infinite loop.
It seems unexpected that disabling LTO fixes the issue since the
recursive calls still exist.
Probably what needs to happen is to replace the memcpy() call in the
body of __wrap_memcpy with a call to __real_memcpy(). Then a definition
of __real_memcpy() (that calls memcpy) is probably needed /in a separate
file/ to account for builds where --wrap is not used (per the note about
defining __real in https://ftp.gnu.org/pub/old-
gnu/Manuals/ld-2.9.1/html_node/ld_3.html).
It looks like the wrapper no longer exists in the project's latest
versions, which may be why people are finding that pip installed
versions work okay.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1971114
Title:
Ubuntu 22.04 python3-grpcio causes some apps to use 100% of the CPU
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/grpc/+bug/1971114/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
