Gabor Kaszab has uploaded this change for review. (
http://gerrit.cloudera.org:8080/16267
Change subject: IMPALA-10017: Implement ds_kll_union() function
......................................................................
IMPALA-10017: Implement ds_kll_union() function
This function receives a set of serialized Apache DataSketches KLL
sketches produced by ds_kll_sketch() and merges them into a single
sketch.
An example usage is to create a sketch for each partition of a table,
write these sketches to a separate table and based on which partition
the user is interested of the relevant sketches can be union-ed
together to get an estimate. E.g.:
SELECT
ds_kll_quantile(ds_kll_union(sketch_col), 0.5)
FROM sketch_tbl
WHERE partition_col=1 OR partition_col=5;
Testing:
- Apart from the automated tests I added to this patch I also
tested ds_kll_union() on a bigger dataset to check that
serialization, deserialization and merging steps work well. I
took TPCH25.linelitem, created a number of sketches with grouping
by l_shipdate and called ds_kll_union() on those sketches.
Change-Id: I020aea28d36f9b6ef9fb57c08411f2170f5c24bf
---
M be/src/exprs/aggregate-functions-ir.cc
M be/src/exprs/aggregate-functions.h
M fe/src/main/java/org/apache/impala/catalog/BuiltinsDb.java
M testdata/data/README
M testdata/workloads/functional-query/queries/QueryTest/datasketches-kll.test
M tests/query_test/test_datasketches.py
6 files changed, 184 insertions(+), 37 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/67/16267/1
--
To view, visit http://gerrit.cloudera.org:8080/16267
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I020aea28d36f9b6ef9fb57c08411f2170f5c24bf
Gerrit-Change-Number: 16267
Gerrit-PatchSet: 1
Gerrit-Owner: Gabor Kaszab <[email protected]>
