zhang-jianq opened a new issue, #13416:
URL: https://github.com/apache/gluten/issues/13416
### Description
## Motivation
The Flink hash expressions — MD5 and SHA1 — are core building blocks of
Flink SQL. Today gluten-flink does not map them, so any Calc projecting them
fails function offload ("Function not supported") and stays on the native path.
## Proposal
Offload them to the velox flinksql function set:
- velox side (companion PR): Flink-semantics implementations under
`velox/functions/flinksql/` — `MD5(string)` and `SHA1(string)` compute the
digest of the input's UTF-8 bytes and expand it to 32 / 40 lowercase hex
digits, mirroring Flink's `MessageDigest("MD5")` / `MessageDigest("SHA")` +
`EncodingUtils.hex` chain. The prestosql/sparksql `Md5Function`/`Sha1Function`
are not reused because they operate on VARBINARY and return raw digest bytes
(or VARBINARY), which does not match Flink's `VARCHAR -> lowercase-hex VARCHAR`
shape.
- planner side (this repo): `MD5 -> "md5"` and `SHA1 -> "sha1"` name
mappings via `DefaultRexCallConverter` (one-line passthrough; no dedicated
converter needed).
Semantics to preserve: lowercase-hex output (32 / 40 digits), digest over
UTF-8 bytes, multi-byte / empty / long-input correctness, and null propagation
(NULL in -> NULL out), with no hashLength argument.
Gluten version: main branch (gluten-flink)
### Gluten version
main branch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
