andygrove opened a new issue, #297:
URL: https://github.com/apache/datafusion-spark/issues/297
Part of #272 and apache/datafusion-comet#7300.
### What the function does
`try_aes_decrypt(expr, key[, mode[, padding[, aad]]])` behaves like
`aes_decrypt` but returns NULL instead of
raising an error. Arguments, defaults (`mode = 'GCM'`, `padding =
'DEFAULT'`, `aad = ''`), implicit casts and the
supported `(mode, padding)` pairs are the same as `aes_decrypt`: `ECB`/`CBC`
with `PKCS`/`DEFAULT`, `GCM` with
`NONE`/`DEFAULT`; 16/24/32-byte keys; CBC input `iv(16) || ct`, GCM input
`iv(12) || ct || tag(16)`.
Returns nullable `BINARY`. Since 3.5.0.
Spark implements it as `TryEval(AesDecrypt(...))` (`RuntimeReplaceable` with
`InheritAnalysisRules`). `TryEval`
catches every `Exception` thrown while evaluating the wrapped expression, so
all runtime errors become NULL.
Analysis-time errors (wrong argument count or types) are still raised.
### Examples
```sql
SELECT
cast(try_aes_decrypt(unhex('6E7CA17BBB468D3084B5744BCA729FB7B2B7BCB8E4472847D02670489D95FA97DBBA7D3210'),
'0000111122223333', 'GCM') AS STRING);
-- Spark SQL
SELECT
try_aes_decrypt(unhex('----------468D3084B5744BCA729FB7B2B7BCB8E4472847D02670489D95FA97DBBA7D3210'),
'0000111122223333', 'GCM');
-- NULL
SELECT
cast(try_aes_decrypt(unbase64('AAAAAAAAAAAAAAAAQiYi+sTLm7KD9UcZ2nlRdYDe/PX4'),
'abcdefghijklmnop12345678ABCDEFGH', 'GCM', 'DEFAULT', 'This is an AAD mixed
into the input') AS STRING);
-- Spark
SELECT
cast(try_aes_decrypt(unbase64('AAAAAAAAAAAAAAAAAAAAAPSd4mWyMZ5mhvjiAPQJnfg='),
'abcdefghijklmnop12345678ABCDEFGH', 'CBC') AS STRING);
-- Spark
SELECT
cast(try_aes_decrypt(unhex('83F16B2AA704794132802D248E6BFD4E380078182D1544813898AC97E709B28A94'),
'0000111122223333') AS STRING);
-- Spark
```
The first two come from the `ExpressionDescription`. In the second, `unhex`
of invalid hex already returns NULL, so
it does not exercise the error path. The others are `MiscFunctionsSuite`
inputs whose `aes_decrypt` results are in
the `aes_decrypt` examples. The NULL cases below follow from `TryEval` plus
the `aes_decrypt` error tests (code
reading):
```sql
SELECT try_aes_decrypt(unbase64('4Hv0UKCx6nfUeAoPZo1z+w=='),
'12345678901234567'); -- NULL (AES_KEY_LENGTH)
SELECT try_aes_decrypt(unbase64('4Hv0UKCx6nfUeAoPZo1z+w=='),
binary('1234567812345678'), 'ECB'); -- NULL (bad padding, wrong key)
SELECT try_aes_decrypt(unbase64('4Hv0UKCx6nfUeAoPZo1z+w=='),
'abcdefghijklmnop', 'GCM', 'PKCS'); -- NULL (AES_MODE)
SELECT
try_aes_decrypt(unbase64('AAAAAAAAAAAAAAAAQiYi+sTLm7KD9UcZ2nlRdYDe/PX4'),
'abcdefghijklmnop12345678ABCDEFGH', 'GCM', 'DEFAULT', 'The ciphertext is
valid, but the AAD is wrong'); -- NULL (tag mismatch)
```
### Edge cases to test
- Every `aes_decrypt` error becomes NULL:
`INVALID_PARAMETER_VALUE.AES_KEY_LENGTH`, `UNSUPPORTED_FEATURE.AES_MODE`,
`UNSUPPORTED_FEATURE.AES_MODE_AAD` (AAD with CBC),
`INVALID_PARAMETER_VALUE.AES_CRYPTO_ERROR` (wrong key, corrupted
or truncated ciphertext, empty GCM input, wrong AAD).
- Successful decrypts match `aes_decrypt` byte for byte, for all modes and
key sizes.
- NULL in any argument gives NULL.
- `TryEval` wraps the argument expressions too, so a runtime error inside an
argument (for example an ANSI cast
failure in a `CAST` with ANSI on, or `raise_error(...)` passed as `key`)
is also swallowed. In
DataFusion, arguments are evaluated before the UDF runs, so this cannot be
reproduced inside the UDF. Document the
difference or handle it at plan-conversion time.
- AAD with ECB is ignored, not an error (same as `aes_decrypt`).
- Literal and column for each argument, with a mix of good and bad rows in
one batch (bad rows NULL, good rows
decrypted).
- Deterministic.
### Differences across Spark versions
- Not available in 3.4. Added in 3.5.0 with CBC and AAD support already
present.
- 4.0: collated `mode` / `padding` accepted. No behavior change through 4.2.
### Implementation notes
- Spark:
`sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala`
(`TryAesDecrypt`),
`expressions/TryEval.scala`, and
`sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/ExpressionImplUtils.java`
(`aesInternal`).
- DataFusion core has no AES functions.
- Implement on top of the `aes_decrypt` kernel (see the `aes_decrypt`
issue). Per-row errors become nulls instead
of failing the batch. Existing `try_*` functions in datafusion-spark
(`try_to_binary`, `try_url_decode`,
`try_parse_url`) show the pattern.
- Module: `crates/spark/src/function/misc/`.
- Tests: add `crates/sqllogictest/test_files/misc/try_aes_decrypt.slt`;
document with `#[user_doc]`.
---
_This issue was generated by an LLM (Claude) from the Apache Spark source
code and tests. Examples come from Spark's `ExpressionDescription` examples,
test suites and `.sql.out` golden files, but please verify them against Spark
before relying on them._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]