Joe McDonnell created IMPALA-13329:
--------------------------------------
Summary: Add a more to bin/single_node_perf_run.py to use cpusets
to isolate impalads
Key: IMPALA-13329
URL: https://issues.apache.org/jira/browse/IMPALA-13329
Project: IMPALA
Issue Type: Task
Components: Infrastructure
Affects Versions: Impala 4.5.0
Reporter: Joe McDonnell
We want the perf-AB-test job to have as little noise and bias as possible to
get a clean read on performance differences. One way to reduce the noise would
be to use dedicated CPUs for each Impalad while they are running. Keeping the
minicluster components and perf script runner separate avoids interactions
between the two. Keeping the impalads separate from each other better simulates
different physical machines with dedicated CPUs.
One way to do this is to use cpusets via cgroups. Essentially, we can create a
separate cpuset cgroup for each impalad and another for everything else. For
example, on a 16 CPU machine, one could give 4 CPUs to each of the three
impalads and then have the remaining 4 CPUs available for everything else. This
aligns with the hyperthreading boundaries, so nothing is sharing anything.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
