Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11160 )
Change subject: IMPALA-7412: width_bucket() function overflows too easily ...................................................................... IMPALA-7412: width_bucket() function overflows too easily Running the tests of https://gerrit.cloudera.org/#/c/10859/ it turned out that the width_bucket() function overflows very often. A common problem is that the function tries to cast the 'num_buckets' parameter to the decimal determined by the Frontend. When the Frontend determined the precision and scale of this decimal it only considered the decimal arguments and ignored everything else. Therefore the determined precision and scale is often not suitable for the 'num_buckets' parameter. WidthBucketImpl() has three decimal arguments, all of them have the same byte size, precision, and scale. So it is possible to interpret them as plain integers and still calculate the proper bucket. I included the python test cases from IMPALA-7202 developed by Taras Bobrovytsky. For performance test I used the following query: SELECT sum(width_bucket(cast(l_orderkey AS DECIMAL(30, 10)), 0, 5500000, 1000000)) FROM tpch_parquet.lineitem; The new implementation executed it in ~0.3 seconds. The old implementation executed it in ~0.8 seconds. Change-Id: I728cc05d9aef8d081e6f2da66146f6d7b75dbb57 Reviewed-on: http://gerrit.cloudera.org:8080/11160 Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> --- M be/src/exprs/expr-test.cc M be/src/exprs/math-functions-ir.cc M be/src/util/bit-util.h M tests/query_test/test_decimal_fuzz.py 4 files changed, 175 insertions(+), 124 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/11160 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I728cc05d9aef8d081e6f2da66146f6d7b75dbb57 Gerrit-Change-Number: 11160 Gerrit-PatchSet: 10 Gerrit-Owner: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com>
