This is an automated email from the ASF dual-hosted git repository. alexey pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/kudu.git
commit ee86f568cb405f8636ae56f3b69f8009f8f481da Author: Alexey Serbin <[email protected]> AuthorDate: Sat Mar 21 12:41:33 2020 -0700 [util] GetBindIpForDaemon() consumes PID wider than 18 bit This patch updates the logic of GetBindIpForDaemon() to accept process identifiers wider than 18 bits. Prior to this patch, maximum allowable PID was 262143, and feeding in a higher PID lead to a crash. The limitation was due to the mapping scheme of (PID, index) --> loopback IP address: it mapped 18-bit PID and 6-bit index into the 24-bit space of 127.0.0.0/8 loopback addresses. With this patch, GetBindIpForDaemon() stops imposing the limit on the maximum PID. Now the uniqueness of the generated IP address is not guaranteed in general if OS generates PIDs wider than 18 bits, but it still holds across all Kudu test processes if both of the following conditions are true: * PIDs are monotonically increasing with a delta D which is much smaller than 262143 for newly spawned processes (usually D is 1) * no more than 262144/D processes are spawned between the earliest still running process and the latest running one while executing Kudu tests involving external mini-cluster The former item is not true for hardened Linux distros where PID assignment is randomized, but it's unlikely that anybody is interested running Kudu tests on such systems. Essentially, the new scheme uses PID % 262144 as input for the PID part, keeping it in the 18-bit space even if the original PID address space is wider (it might be up to 22 bits wide for 64-bit Linux distros). This patch helps running Kudu tests on Docker containers where the host OS is configured to have PIDs higher than 262143 and updating maximum PID is not an option since /proc/sys/kernel/pid_max is mounted read-only. This patch relates to the following JIRA issues: KUDU-1334 KUDU-3044 Change-Id: Idf2162a2a976a0ce997f02a321f230e7f5878328 Reviewed-on: http://gerrit.cloudera.org:8080/15519 Reviewed-by: Adar Dembo <[email protected]> Tested-by: Kudu Jenkins --- src/kudu/util/net/net_util.cc | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/src/kudu/util/net/net_util.cc b/src/kudu/util/net/net_util.cc index 6599f5c..81835c1 100644 --- a/src/kudu/util/net/net_util.cc +++ b/src/kudu/util/net/net_util.cc @@ -492,11 +492,16 @@ string GetBindIpForDaemon(int index, BindMode bind_mode) { CHECK(0 < index && index <= kServersMaxNum) << Substitute( "server index $0 is not in range ($1, $2]", index, 0, kServersMaxNum); + static constexpr uint32_t kMaxPid = 1 << kPidBits; switch (bind_mode) { case BindMode::UNIQUE_LOOPBACK: { uint32_t pid = getpid(); - CHECK_LT(pid, 1 << kPidBits) << Substitute( - "PID $0 is more than $1 bits wide", pid, kPidBits); + if (pid >= kMaxPid) { + LOG(INFO) << Substitute( + "PID $0 is more than $1 bits wide, substituted with $2", + pid, kPidBits, pid % kMaxPid); + pid %= kMaxPid; + } uint32_t ip = (pid << kServerIdxBits) | static_cast<uint32_t>(index); uint8_t octets[] = { static_cast<uint8_t>((ip >> 16) & 0xff),
