Quanlong Huang created IMPALA-13202:
---------------------------------------
Summary: KRPC flags used by libkudu_client.so can't be configured
Key: IMPALA-13202
URL: https://issues.apache.org/jira/browse/IMPALA-13202
Project: IMPALA
Issue Type: Bug
Components: Backend
Reporter: Quanlong Huang
The way Impala integrates with KRPC is porting the KRPC codes into the Impala
code base. Flags and methods of KRPC are defined as GLOBAL in the impalad
executable. libkudu_client.so also compiles from the same KRPC codes and have
duplicate flags and methods defined as HIDDEN.
To be specifit, both the impalad executable and libkudu_client.so have the
symbol for kudu::rpc::InboundTransfer::ReceiveBuffer()
{noformat}
$ readelf -s --wide be/build/latest/service/impalad | grep ReceiveBuffer
11118: 00000000022f5c88 1936 FUNC GLOBAL DEFAULT 13
_ZN4kudu3rpc15InboundTransfer13ReceiveBufferEPNS_6SocketEPNS_10faststringE
81380: 00000000022f5c88 1936 FUNC GLOBAL DEFAULT 13
_ZN4kudu3rpc15InboundTransfer13ReceiveBufferEPNS_6SocketEPNS_10faststringE
$ readelf -s --wide
toolchain/toolchain-packages-gcc10.4.0/kudu-e742f86f6d/debug/lib/libkudu_client.so
| grep ReceiveBuffer
1601: 0000000000086e4a 108 FUNC LOCAL DEFAULT 12
_ZN4kudu3rpc15InboundTransfer13ReceiveBufferEPNS_6SocketEPNS_10faststringE.cold
11905: 00000000001fec60 2076 FUNC LOCAL HIDDEN 12
_ZN4kudu3rpc15InboundTransfer13ReceiveBufferEPNS_6SocketEPNS_10faststringE
$ c++filt
_ZN4kudu3rpc15InboundTransfer13ReceiveBufferEPNS_6SocketEPNS_10faststringE
kudu::rpc::InboundTransfer::ReceiveBuffer(kudu::Socket*, kudu::faststring*)
{noformat}
KRPC flags like rpc_max_message_size are also defined in both the impalad
executable and libkudu_client.so:
{noformat}
$ readelf -s --wide be/build/latest/service/impalad | grep
FLAGS_rpc_max_message_size
14380: 0000000006006738 8 OBJECT GLOBAL DEFAULT 30
_ZN5fLI6426FLAGS_rpc_max_message_sizeE
80396: 0000000006006741 1 OBJECT GLOBAL DEFAULT 30
_ZN3fLB44FLAGS_rpc_max_message_size_enable_validationE
81399: 0000000006006741 1 OBJECT GLOBAL DEFAULT 30
_ZN3fLB44FLAGS_rpc_max_message_size_enable_validationE
117873: 0000000006006738 8 OBJECT GLOBAL DEFAULT 30
_ZN5fLI6426FLAGS_rpc_max_message_sizeE
$ readelf -s --wide
toolchain/toolchain-packages-gcc10.4.0/kudu-e742f86f6d/debug/lib/libkudu_client.so
| grep FLAGS_rpc_max_message_size
11882: 00000000008d61e1 1 OBJECT LOCAL HIDDEN 27
_ZN3fLB44FLAGS_rpc_max_message_size_enable_validationE
11906: 00000000008d61d8 8 OBJECT LOCAL DEFAULT 27
_ZN5fLI6426FLAGS_rpc_max_message_sizeE
$ c++filt _ZN5fLI6426FLAGS_rpc_max_message_sizeE
fLI64::FLAGS_rpc_max_message_size {noformat}
libkudu_client.so uses its own methods and flags. The flags are HIDDEN so can't
be modified by Impala codes. E.g. IMPALA-4874 bumps FLAGS_rpc_max_message_size
to 2GB in RpcMgr::Init(), but the HIDDEN variable FLAGS_rpc_max_message_size
used in libkudu_client.so is still the default value 50MB (52428800). We've
seen error messages like this in the master branch:
{code:java}
I0708 10:23:31.784974 2943 meta_cache.cc:294]
c243bda4702a5ab9:0ba93d2400000001] tablet 0c8f3446538449ee9d3df5056afe775e:
replica e0e1db54dab74f208e37ea1b975595e5 (127.0.0.1:31202) has failed: Network
error: TS failed: RPC frame had a length of 53477464, but we only support
messages up to 52428800 bytes long.{code}
CC [~joemcdonnell] [~wzhou] [~aserbin]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
