Hello Kurt Deschler, Riza Suminto, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18602
to look at the new patch set (#6).
Change subject: IMPALA-9615: re2's max_mem opt configurable via an Impala
startup flag
......................................................................
IMPALA-9615: re2's max_mem opt configurable via an Impala startup flag
Some regex patterns require more memory to be compiled and pattern matched using
different string functions and like predicate available.
For more memory consuming patterns this can cause the following error:
"re2/re2.cc:667: DFA out of memory: size xxxxx, bytemap range xx, list count
xxxxx".
To avoid such errors in Impalad's ERROR log, a global flag can
be added to impala cluster startup. The re2_max_mem_usage flag will
accept an unsigned int64 which will set the re2 max_mem parameter for
memory used to store regexps in Bytes.
The testcase uses a long regex pattern to use up all the memory in the
case of allocating less or the same amount of memory as default for re2.
By using a greater value for re2_max_mem_usage flag, the regexp can be
consumed with no error.
Change-Id: Idf28d2f7217b1322ab8fdfb2c02fff0608078571
---
M be/src/common/global-flags.cc
M be/src/common/init.cc
M be/src/exprs/like-predicate.cc
M be/src/exprs/string-functions-ir.cc
M be/src/exprs/string-functions.h
A tests/custom_cluster/test_re2_max_mem.py
6 files changed, 122 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/02/18602/6
--
To view, visit http://gerrit.cloudera.org:8080/18602
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Idf28d2f7217b1322ab8fdfb2c02fff0608078571
Gerrit-Change-Number: 18602
Gerrit-PatchSet: 6
Gerrit-Owner: Omid Shahidi <oshah...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kdesc...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com>
