jiangxintong created SPARK-57924:
------------------------------------
Summary: Add bitmap_xor_agg aggregation function
Key: SPARK-57924
URL: https://issues.apache.org/jira/browse/SPARK-57924
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 4.2.0
Reporter: jiangxintong
Introduce bitmap_xor_agg, a bitwise XOR (symmetric difference) aggregation
function for flat bitmaps, completing the function set alongside bitmap_or_agg
(3.5.0), bitmap_construct_agg (3.5.0), bitmap_count (3.5.0), and bitmap_and_agg
(4.1.0, SPARK-53877).
The bitmap_xor_agg function outputs a bitmap representing the bitwise XOR of
all bitmaps in the input column. The input column must contain bitmaps
generated from bitmap_construct_agg().
{code:sql}
> SELECT substring(hex(bitmap_xor_agg(col)), 0, 6) FROM VALUES (X '10'), (X
> '30'), (X '40') AS tab(col);
600000
> SELECT substring(hex(bitmap_xor_agg(col)), 0, 6) FROM VALUES (X '10'), (X
> '10') AS tab(col);
000000
{code}
(The second example demonstrates symmetric difference: two identical values XOR
to zero.)
h2. Use Cases
Symmetric difference is useful for:
* Finding users who belong to exactly one of two groups (not both)
* Detecting changes between two snapshots
* Any scenario requiring XOR semantics on bitmaps
h2. Implementation
Follow the exact pattern of BitmapAndAgg (SPARK-53877):
* Merge operation: buffer[i] ^= input[i]
* Initialize buffer to all zeros (same as BitmapOrAgg)
* Both update() and merge() paths must use XOR
Relates to: SPARK-53877
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
