Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20862
Change subject: IMPALA-12643 (part 1): Limit memory consumption for resolve_minidumps.py ...................................................................... IMPALA-12643 (part 1): Limit memory consumption for resolve_minidumps.py On some platforms (Centos 7), resolve_minidumps.py's call to minidump_stackwalk goes haywire and uses all the system memory until it gets OOM killed. Some library must have corrupt symbols, etc. As a workaround, this detects whether the prlimit utility is present and uses this to run minidump_stackwalk with a 4GB limit on virtual memory. This kills the process earlier and avoids using all system memory. Testing: - Verified that bin/jenkins/finalize.sh uses resolve_minidumps.py on a Redhat 8 Jenkins job (and it works) - Verified that bin/jenkins/finalize.sh works properly on my Ubuntu 20 box - Ran a Jenkins job on Centos 7 and verified that the prlimit code kills minidump_stackwalk when it uses 4GB of memory. Change-Id: I4db8facb8a037327228c3714e047e0d1f0fe1d94 --- M bin/resolve_minidumps.py 1 file changed, 15 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/62/20862/1 -- To view, visit http://gerrit.cloudera.org:8080/20862 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I4db8facb8a037327228c3714e047e0d1f0fe1d94 Gerrit-Change-Number: 20862 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell <[email protected]>
