Michael Brown has uploaded a new change for review.
http://gerrit.cloudera.org:8080/3720
Change subject: IMPALA-3864: qgen: reduce likelihood of create_query()
exceptions
......................................................................
IMPALA-3864: qgen: reduce likelihood of create_query() exceptions
1. Fix a bug in which the computation to produce the string for an
exception was raising a TypeError. We fix the bug by changing how the
string is built.
2. Fix a bug in which we tried to choose a relational function (defined
as taking in more than one argument and returning a Boolean) and were
looking for its weight in QueryProfile.weights.RELATIONAL_FUNCS, but
the function wasn't defined in that dictionary. We fix the bug by
defining weights for those functions.
3. Fix a bug in which QueryProfile.choose_func_signatures() was choosing
a function without taking into account the set of functions in the
signatures given to it. We fix the bug by pruning off the weights of
functions that aren't included in provided signatures. We also add a
note explaining how the weights defined are "best effort", since
sometimes functions will be pruned.
4. Added Char signatures to LessThan, GreaterThan, LessThanOrEquals,
GreaterThanOrEquals
4. Misc small changes to aid in debugging or testing:
a. Added funcs.Signature representation
b. Increased the number of fake columns when calling
query_generator.__main__, which is useful for testing.
c. Renamed a few variables. For some reason in the query_generator
module there are a lot of overwritten variables, which makes
debugging difficult.
Testing:
1. impala-python tests/comparison/query_generator.py produces far fewer
exceptions. The ones for this bug are fixed, but see IMPALA-3890.
2. Full 3 and 6 hour runs of the query generator system don't show any
obvious regressions in behavior.
Change-Id: Idd9434a92973176aefb99e11e039209cac3cea65
---
M tests/comparison/funcs.py
M tests/comparison/query_generator.py
M tests/comparison/query_profile.py
3 files changed, 49 insertions(+), 18 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/20/3720/1
--
To view, visit http://gerrit.cloudera.org:8080/3720
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Idd9434a92973176aefb99e11e039209cac3cea65
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Michael Brown <[email protected]>
