ybapat opened a new pull request, #57567: URL: https://github.com/apache/spark/pull/57567
### What changes were proposed in this pull request? `bitmap_or_agg` and `bitmap_and_agg` silently accept input bitmaps of any size. When an input bitmap is larger than `BitmapExpressionUtils.NUM_BYTES` (4096 bytes), `bitmapMerge` / `bitmapAndMerge` silently reads only the first 4096 bytes, discarding the rest. This can produce incorrect results without any indication that data was dropped. This PR adds an explicit runtime check in the `update()` methods of both `BitmapOrAgg` and `BitmapAndAgg`. If the input exceeds 4096 bytes, a `SparkRuntimeException` with error class `BITMAP_INPUT_TOO_LARGE` is thrown. Changes: - Add `BITMAP_INPUT_TOO_LARGE` error condition to `error-conditions.json` (SQLSTATE `22001` — string/binary data right truncation). - Add `bitmapInputTooLargeError` factory method to `QueryExecutionErrors`. - Add size guard in `BitmapOrAgg.update()` and `BitmapAndAgg.update()`. - Update `@ExpressionDescription` for both aggregates to accurately document short-input behavior and the size limit. - Add two tests in `BitmapExpressionsQuerySuite` verifying the error is thrown. ### Why are the changes needed? Without validation, oversized inputs are silently truncated, producing incorrect aggregate results with no warning. Explicit validation follows Spark's design principle of failing fast with a structured error rather than silently producing wrong results. ### Does this PR introduce _any_ user-facing change? Yes. Queries that pass bitmaps larger than 4096 bytes to `bitmap_or_agg` or `bitmap_and_agg` will now fail with `BITMAP_INPUT_TOO_LARGE` instead of silently truncating. Inputs created by `bitmap_construct_agg()` are always exactly 4096 bytes and are unaffected. ### How was this patch tested? - New tests `"bitmap_or_agg rejects input larger than 4096 bytes"` and `"bitmap_and_agg rejects input larger than 4096 bytes"` in `BitmapExpressionsQuerySuite`. - Existing bitmap expression tests pass. Closes #57561 Co-Authored-By: Claude Sonnet 4.6 <[email protected]> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
