andygrove opened a new issue, #296:
URL: https://github.com/apache/datafusion-spark/issues/296

   Part of #272 and apache/datafusion-comet#7300.
   
   ### What the function does
   
   `aes_decrypt(expr, key[, mode[, padding[, aad]]])` decrypts a value produced 
by `aes_encrypt` and returns `BINARY`.
   `expr`, `key` and `aad` are `BINARY`; `mode` and `padding` are `STRING` 
(collated strings accepted from 4.0).
   `ImplicitCastInputTypes` casts string arguments for `expr`, `key` and `aad` 
to their UTF-8 bytes. Defaults:
   `mode = 'GCM'`, `padding = 'DEFAULT'`, `aad = ''`. Since 3.3.0 (`aad` since 
3.5.0). NULL in any argument gives NULL.
   There is no `iv` argument: the IV is read from the input.
   
   Supported `(mode, padding)` pairs (case-insensitive): `ECB` + 
`PKCS`/`DEFAULT`, `CBC` + `PKCS`/`DEFAULT`,
   `GCM` + `NONE`/`DEFAULT`. The key must be 16, 24 or 32 bytes. Expected input 
layout: ECB is the ciphertext only;
   CBC is `iv (16) || ciphertext`; GCM is `iv (12) || ciphertext || tag (16)`, 
with the 128-bit tag verified.
   The result is the raw plaintext bytes. Use `CAST(... AS STRING)` to get text 
(the `ExpressionDescription` shows the
   bytes as text).
   
   ### Examples
   
   From the `ExpressionDescription`, `DataFrameFunctionsSuite` and 
`MiscFunctionsSuite`:
   
   ```sql
   SELECT 
cast(aes_decrypt(unhex('83F16B2AA704794132802D248E6BFD4E380078182D1544813898AC97E709B28A94'),
 '0000111122223333') AS STRING);
   -- Spark
   SELECT 
cast(aes_decrypt(unhex('6E7CA17BBB468D3084B5744BCA729FB7B2B7BCB8E4472847D02670489D95FA97DBBA7D3210'),
 '0000111122223333', 'GCM') AS STRING);
   -- Spark SQL
   SELECT cast(aes_decrypt(unbase64('3lmwu+Mw0H3fi5NDvcu9lg=='), 
'1234567890abcdef', 'ECB', 'PKCS') AS STRING);
   -- Spark SQL
   SELECT 
cast(aes_decrypt(unbase64('2NYmDCjgXTbbxGA3/SnJEfFC/JQ7olk2VQWReIAAFKo='), 
'1234567890abcdef', 'CBC') AS STRING);
   -- Apache Spark
   SELECT 
cast(aes_decrypt(unbase64('AAAAAAAAAAAAAAAAAAAAAPSd4mWyMZ5mhvjiAPQJnfg='), 
'abcdefghijklmnop12345678ABCDEFGH', 'CBC', 'DEFAULT') AS STRING);
   -- Spark
   SELECT 
cast(aes_decrypt(unbase64('AAAAAAAAAAAAAAAAQiYi+sTLm7KD9UcZ2nlRdYDe/PX4'), 
'abcdefghijklmnop12345678ABCDEFGH', 'GCM', 'DEFAULT', 'This is an AAD mixed 
into the input') AS STRING);
   -- Spark
   SELECT cast(aes_decrypt(unbase64('jmTOhz8XTbskI/zYFFgOFQ=='), 
'abcdefghijklmnop', 'ECB') AS STRING);
   -- (empty string)
   SELECT cast(aes_decrypt(unbase64('9J3iZbIxnmaG+OIA9Amd+A=='), 
binary('abcdefghijklmnop12345678ABCDEFGH'), 'ECB') AS STRING);
   -- Spark
   SELECT aes_decrypt(cast(null as binary), 'abcdefghijklmnop');
   -- NULL
   ```
   
   ### Edge cases to test
   
   - NULL in each argument gives NULL. String vs binary `key` give the same 
result.
   - Key length not 16/24/32 (0, 15, 17): 
`INVALID_PARAMETER_VALUE.AES_KEY_LENGTH` (sqlState 22023), parameters
     `` `key` ``, `` `aes_encrypt`/`aes_decrypt` ``, `actualLength`.
   - Unsupported pairs (`'GSM'` with the default padding reports `padding = 
DEFAULT`; `'GCM','PKCS'`; `'ECB','None'`;
     `'CBC','NoPadding'`): `UNSUPPORTED_FEATURE.AES_MODE` (sqlState 0A000).
   - Wrong key for ECB: `INVALID_PARAMETER_VALUE.AES_CRYPTO_ERROR`, parameter 
`` `expr`, `key` ``, `detailMessage`
     "Given final block not properly padded. Such issues can arise if a bad key 
is used during decryption."
   - Corrupted GCM ciphertext or wrong AAD: `AES_CRYPTO_ERROR` with "Tag 
mismatch" (JDK may append `!`).
   - Truncated CBC input: `AES_CRYPTO_ERROR` with a message starting "Input 
length". Empty GCM input:
     `AES_CRYPTO_ERROR` with "Invalid buffer arguments" (input shorter than the 
IV). The JCE messages are not
     reproducible in Rust; match the error condition and parameters, not 
`detailMessage`.
   - AAD with CBC: `UNSUPPORTED_FEATURE.AES_MODE_AAD`, and the message names `` 
`aes_encrypt` `` even when decrypting.
     AAD with ECB is silently ignored on decrypt (the ECB branch never looks at 
`aad`, by code reading).
   - Round trips through `aes_encrypt` for all modes and key sizes, empty 
plaintext, plaintext longer than one block,
     and multi-byte UTF-8 (results are bytes, not validated as UTF-8).
   - Literal and column for each argument, including per-row `mode` / `key`.
   - Deterministic.
   
   ### Differences across Spark versions
   
   - 3.4: only ECB and GCM, no `aad`. A too-short GCM input throws an unmapped 
`IllegalArgumentException`, because
     only `GeneralSecurityException` maps to `AES_CRYPTO_ERROR`.
   - 3.5: adds CBC, `aad`, `UNSUPPORTED_FEATURE.AES_MODE_AAD`, and maps 
`IllegalArgumentException` too.
   - 4.0: collated `mode` / `padding` accepted. No behavior change through 4.2.
   
   ### Implementation notes
   
   - Spark: 
`sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala`
 (`AesDecrypt`,
     `RuntimeReplaceable` over `StaticInvoke`) and
     
`sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/ExpressionImplUtils.java`
 (`aesDecrypt`,
     `aesInternal`). Errors in 
`sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala`.
   - DataFusion core has no AES functions.
   - Suggested crates: RustCrypto `aes`, `aes-gcm`, `cbc`, `ecb` + 
`block-padding` (PKCS7).
   - Put it in `crates/spark/src/function/misc/` with a helper module shared 
with `aes_encrypt` and `try_aes_decrypt`.
     The helper should return a typed error so `try_aes_decrypt` can turn every 
failure into NULL.
   - Tests: add `crates/sqllogictest/test_files/misc/aes_decrypt.slt`; document 
with `#[user_doc]`.
   
   ---
   _This issue was generated by an LLM (Claude) from the Apache Spark source 
code and tests. Examples come from Spark's `ExpressionDescription` examples, 
test suites and `.sql.out` golden files, but please verify them against Spark 
before relying on them._


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to