[Impala-ASF-CR] IMPALA-6917: Limit impalad mem-limit to 12GB.
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/10051 ) Change subject: IMPALA-6917: Limit impalad mem-limit to 12GB. .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/10051/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/10051/3//COMMIT_MSG@7 PS3, Line 7: 6917 > Um... I know the horse is already gone, but this is actually IMPALA-5917. :/ We'll survive. -- To view, visit http://gerrit.cloudera.org:8080/10051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8024414c5c23bb42cce912d8f34cd0b787e0e39a Gerrit-Change-Number: 10051 Gerrit-PatchSet: 3 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 22:32:42 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6917: Limit impalad mem-limit to 12GB.
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/10051 ) Change subject: IMPALA-6917: Limit impalad mem-limit to 12GB. .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/10051/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/10051/3//COMMIT_MSG@7 PS3, Line 7: 6917 Um... I know the horse is already gone, but this is actually IMPALA-5917. -- To view, visit http://gerrit.cloudera.org:8080/10051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8024414c5c23bb42cce912d8f34cd0b787e0e39a Gerrit-Change-Number: 10051 Gerrit-PatchSet: 3 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 22:28:54 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6917: Limit impalad mem-limit to 12GB.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10051 ) Change subject: IMPALA-6917: Limit impalad mem-limit to 12GB. .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8024414c5c23bb42cce912d8f34cd0b787e0e39a Gerrit-Change-Number: 10051 Gerrit-PatchSet: 2 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 20:31:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6917: Limit impalad mem-limit to 12GB.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10051 ) Change subject: IMPALA-6917: Limit impalad mem-limit to 12GB. .. IMPALA-6917: Limit impalad mem-limit to 12GB. This changes the memlimit choosing code in start-impala-cluster to have a maximum of 12GB. In 68GB machines, this has the effect of reducing the memlimit from ~15.8GB to 12GB. On machines with less than 51.4 GB of RAM, this has no effect. I'm tinkering with this threshold because ASAN builds sometimes die at the hands of the OOM killer on m2.4xlarge machines (68GB). My theory for why it's only sometimes is that during the parallel tests, our memory usage could vary widely depending on which tests and queries execute. End-users don't see this code; this is only used by our minicluster tests to test Impala. I have run the ASAN build with this change successfully, though I've found this particular OOM seems to come and go. Change-Id: I8024414c5c23bb42cce912d8f34cd0b787e0e39a Reviewed-on: http://gerrit.cloudera.org:8080/10051 Reviewed-by: Philip Zeyliger Tested-by: Impala Public Jenkins --- M bin/start-impala-cluster.py 1 file changed, 8 insertions(+), 2 deletions(-) Approvals: Philip Zeyliger: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/10051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I8024414c5c23bb42cce912d8f34cd0b787e0e39a Gerrit-Change-Number: 10051 Gerrit-PatchSet: 3 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-6917: Limit impalad mem-limit to 12GB.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10051 ) Change subject: IMPALA-6917: Limit impalad mem-limit to 12GB. .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2332/ -- To view, visit http://gerrit.cloudera.org:8080/10051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8024414c5c23bb42cce912d8f34cd0b787e0e39a Gerrit-Change-Number: 10051 Gerrit-PatchSet: 2 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 16:37:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6917: Limit impalad mem-limit to 12GB.
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/10051 ) Change subject: IMPALA-6917: Limit impalad mem-limit to 12GB. .. Patch Set 2: Code-Review+2 Rebase. -- To view, visit http://gerrit.cloudera.org:8080/10051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8024414c5c23bb42cce912d8f34cd0b787e0e39a Gerrit-Change-Number: 10051 Gerrit-PatchSet: 2 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 16:37:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6917: Limit impalad mem-limit to 12GB.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10051 ) Change subject: IMPALA-6917: Limit impalad mem-limit to 12GB. .. Patch Set 1: Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2329/ -- To view, visit http://gerrit.cloudera.org:8080/10051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8024414c5c23bb42cce912d8f34cd0b787e0e39a Gerrit-Change-Number: 10051 Gerrit-PatchSet: 1 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 05:20:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6917: Limit impalad mem-limit to 12GB.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10051 ) Change subject: IMPALA-6917: Limit impalad mem-limit to 12GB. .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2329/ -- To view, visit http://gerrit.cloudera.org:8080/10051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8024414c5c23bb42cce912d8f34cd0b787e0e39a Gerrit-Change-Number: 10051 Gerrit-PatchSet: 1 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 18 Apr 2018 21:28:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6917: Limit impalad mem-limit to 12GB.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10051 ) Change subject: IMPALA-6917: Limit impalad mem-limit to 12GB. .. Patch Set 1: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2313/ -- To view, visit http://gerrit.cloudera.org:8080/10051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8024414c5c23bb42cce912d8f34cd0b787e0e39a Gerrit-Change-Number: 10051 Gerrit-PatchSet: 1 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Tue, 17 Apr 2018 00:58:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6917: Limit impalad mem-limit to 12GB.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10051 ) Change subject: IMPALA-6917: Limit impalad mem-limit to 12GB. .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2313/ -- To view, visit http://gerrit.cloudera.org:8080/10051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8024414c5c23bb42cce912d8f34cd0b787e0e39a Gerrit-Change-Number: 10051 Gerrit-PatchSet: 1 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Mon, 16 Apr 2018 18:14:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6917: Limit impalad mem-limit to 12GB.
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/10051 ) Change subject: IMPALA-6917: Limit impalad mem-limit to 12GB. .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8024414c5c23bb42cce912d8f34cd0b787e0e39a Gerrit-Change-Number: 10051 Gerrit-PatchSet: 1 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 12 Apr 2018 21:47:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6917: Limit impalad mem-limit to 12GB.
Philip Zeyliger has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10051 Change subject: IMPALA-6917: Limit impalad mem-limit to 12GB. .. IMPALA-6917: Limit impalad mem-limit to 12GB. This changes the memlimit choosing code in start-impala-cluster to have a maximum of 12GB. In 68GB machines, this has the effect of reducing the memlimit from ~15.8GB to 12GB. On machines with less than 51.4 GB of RAM, this has no effect. I'm tinkering with this threshold because ASAN builds sometimes die at the hands of the OOM killer on m2.4xlarge machines (68GB). My theory for why it's only sometimes is that during the parallel tests, our memory usage could vary widely depending on which tests and queries execute. End-users don't see this code; this is only used by our minicluster tests to test Impala. I have run the ASAN build with this change successfully, though I've found this particular OOM seems to come and go. Change-Id: I8024414c5c23bb42cce912d8f34cd0b787e0e39a --- M bin/start-impala-cluster.py 1 file changed, 8 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/51/10051/1 -- To view, visit http://gerrit.cloudera.org:8080/10051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I8024414c5c23bb42cce912d8f34cd0b787e0e39a Gerrit-Change-Number: 10051 Gerrit-PatchSet: 1 Gerrit-Owner: Philip Zeyliger