Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/16723 )
Change subject: IMPALA-10314: Optimize planning time for simple limits ...................................................................... Patch Set 15: > Patch Set 9: > > > Patch Set 8: > > > > > Patch Set 8: > > > > > > > Patch Set 8: > > > > > > > > > Patch Set 8: Verified-1 > > > > > > > > > > Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/6687/ > > > > > > > > There's 1 failure: > > > > [gw1] FAILED > > > > query_test/test_exprs.py::TestExprLimits::test_statement_expression_limit > > > > > > > > However, on my desktop I ran query_test/test_exprs.py and all tests > > > > under it passed. > > > > > > On Jenkins this test hit an OOM GC overhead limit exceeded: > > > gw1] linux2 -- Python 2.7.16 > > > /home/ubuntu/Impala/bin/../infra/python/env-gcc7.5.0/bin/python > > > query_test/test_exprs.py:176: in test_statement_expression_limit > > > assert re.search(expected_err_re, str(err)) > > > E assert None > > > E + where None = <function search at 0x7fc9ff90e848>('Exceeded the > > > statement expression limit \\(250000\\)\nStatement has .* expressions.', > > > "ImpalaBeeswaxException:\n INNER EXCEPTION: <class > > > 'beeswaxd.ttypes.BeeswaxException'>\n MESSAGE: OutOfMemoryError: GC > > > overhead limit exceeded\n") > > > E + where <function search at 0x7fc9ff90e848> = re.search > > > E + and "ImpalaBeeswaxException:\n INNER EXCEPTION: <class > > > 'beeswaxd.ttypes.BeeswaxException'>\n MESSAGE: OutOfMemoryError: GC > > > overhead limit exceeded\n" = str(ImpalaBeeswaxException()) > > > > Oh wait, there's actually another failure in FE: > > > > 23:47:52 [INFO] Running org.apache.impala.analysis.ParserTest > > 23:47:52 [ERROR] Tests run: 98, Failures: 1, Errors: 0, Skipped: 0, Time > > elapsed: 0.829 s <<< FAILURE! - in org.apache.impala.analysis.ParserTest > > 23:47:52 [ERROR] TestGetErrorMsg(org.apache.impala.analysis.ParserTest) > > Time elapsed: 0.005 s <<< FAILURE! > > at > > org.apache.impala.analysis.ParserTest.ParserError(ParserTest.java:77) > > at > > org.apache.impala.analysis.ParserTest.TestGetErrorMsg(ParserTest.java:3475) > > > > Will need to look into this one. > > PS9 fixes the ParserTest issue..just had to update the list of expected > keywords in the test after a WHERE clause. Not yet sure about the other OOM > failure since I cannot repro locally. One difference between my machine's execution vs the one on Jenkins is that the Jenkins one runs it under a dockerised container and the JVM Max heap is 1.78GB which is smaller than the 2GB on my machine. >From ubuntu-16.04-dockerised-tests-3539/logs/ee_tests/impalad_coord_exec-0: impalad.4069f4f171ef.ubuntu.log.INFO.20201126-001441.1: JVM: max heap size: Total=1.78 GB impalad.4069f4f171ef.ubuntu.log.INFO.20201126-001441.1: JVM: non-heap committed: Total=114.50 MB I looked closer at the 'GC overhead limit exceeded' issue with the test_statement_expression_limit() test and I suspect it happens because in my original patch the default construction of the Expr object also allocates an empty ArrayList for hints (this was consistent with how we handle other types of hints). However, this is not really necessary and it can be done on demand only when an actual WHERE clause hint is supplied. This particular test creates a huge number of Exprs and considering that the Dockerised run has a slightly smaller max heap, it might cause the GC to run more often without reclaiming sufficient free memory. In PS 15 I have made the change to allocate the arraylist only when a predicate hint was supplied. Hopefully it addresses the issue. -- To view, visit http://gerrit.cloudera.org:8080/16723 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9d6a79263bc092e0f3e9a1d72da5618f3cc35574 Gerrit-Change-Number: 16723 Gerrit-PatchSet: 15 Gerrit-Owner: Aman Sinha <amsi...@cloudera.com> Gerrit-Reviewer: Aman Sinha <amsi...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com> Gerrit-Reviewer: Shant Hovsepian <sh...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Comment-Date: Fri, 27 Nov 2020 18:06:40 +0000 Gerrit-HasComments: No
