[
https://issues.apache.org/jira/browse/KUDU-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16884162#comment-16884162
]
Alexey Serbin commented on KUDU-1334:
-------------------------------------
The idea behind the bit manipulations in
https://github.com/apache/kudu/blob/c1d16856bd4b3bb4b6242fa098d600b6ef321e6a/src/kudu/util/net/net_util.cc#L472
can be seen clearer if looking at the usage of the utility method
https://github.com/apache/kudu/blob/c1d16856bd4b3bb4b6242fa098d600b6ef321e6a/src/kudu/mini-cluster/mini_cluster.cc#L37
in {{internal_mini_cluster.cc}} and {{external_mini_cluster.cc}}, assuming
it's all about {{UNIQUE_LOOPBACK}} bind mode.
Basically, all you need is a set of unique loopback IP addresses to bind Kudu
masters and tservers, so multiple tests that use internal/external mini cluster
can run in parallel at the same machine without port conflicts. Current
implementation allows for running about 2^18 (262144) Kudu processes at one
machine that uses IPv4 addresses for its network interfaces, that's about 4K
Kudu mini-clusters at once (each mini-cluster in that scheme can run at most 62
indexed daemons, meaning master and tservers). Those bits manipulations is
just a simple mapping of a process identifier (PID) and an index of a daemon
into a 32-bit number (IPv4 address).
I think that in case of using IPv6 addresses the restrictions for maximum
allowable PID is less strict since the target address space is much larger.
> Support pid_max > 16 bits in the mini cluster
> ---------------------------------------------
>
> Key: KUDU-1334
> URL: https://issues.apache.org/jira/browse/KUDU-1334
> Project: Kudu
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.5.0
> Reporter: Jean-Daniel Cryans
> Assignee: Alexey Serbin
> Priority: Major
> Fix For: 1.6.0
>
>
> Pretty much anybody running on newer machines/platforms will hit this while
> running the unit tests:
> {noformat}
> I0216 11:27:23.617383 110702 external_mini_cluster.cc:582] Started
> /home/stack/apache-kudu-incubating-0.7.0/build/rc/bin/kudu-master as pid
> 110706
> F0216 11:27:23.617473 110702 external_mini_cluster.cc:258] Check failed: p
> <= MathLimits<uint16_t>::kMax (110702 vs. 65535) Cannot run on systems with
> >16-bit pid
> *** Check failure stack trace: ***
> {noformat}
> Having this limitation was fine but now it's something everybody hits.
> The workaround is running this:
> {noformat}
> echo "32768" > /proc/sys/kernel/pid_max
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
