[
https://issues.apache.org/jira/browse/IMPALA-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344718#comment-17344718
]
ASF subversion and git services commented on IMPALA-10687:
----------------------------------------------------------
Commit b1326f7eff62dd521989132d650fa8b53057085b in impala's branch
refs/heads/master from Fucun Chu
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b1326f7 ]
IMPALA-10687: Implement ds_cpc_union() function
This function receives a set of serialized Apache DataSketches CPC
sketches produced by ds_cpc_sketch() and merges them into a single
sketch.
An example usage is to create a sketch for each partition of a table,
write these sketches to a separate table and based on which partition
the user is interested of the relevant sketches can be union-ed
together to get an estimate. E.g.:
SELECT
ds_cpc_estimate(ds_cpc_union(sketch_col))
FROM sketch_tbl
WHERE partition_col=1 OR partition_col=5;
Testing:
- Apart from the automated tests I added to this patch I also
tested ds_cpc_union() on a bigger dataset to check that
serialization, deserialization and merging steps work well. I
took TPCH25.linelitem, created a number of sketches with grouping
by l_shipdate and called ds_cpc_union() on those sketches
Change-Id: Ib94b45ae79efcc11adc077dd9df9b9868ae82cb6
Reviewed-on: http://gerrit.cloudera.org:8080/17372
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Implement ds_cpc_union() function
> ---------------------------------
>
> Key: IMPALA-10687
> URL: https://issues.apache.org/jira/browse/IMPALA-10687
> Project: IMPALA
> Issue Type: New Feature
> Reporter: Fucun Chu
> Assignee: Fucun Chu
> Priority: Major
>
> Similarly to https://issues.apache.org/jira/browse/IMPALA-9633 DataSketches
> CPC also has union functionality.
> ds_cpc_union() expects a dataset of STRINGs (e.g. column of a table) that
> represents serialized CPC sketches and returns a single STRING that is the
> received sketches unioned and serialized.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
