andygrove opened a new pull request, #107:
URL: https://github.com/apache/datafusion-spark/pull/107
Part of #55
## Rationale
#58 added `#[user_doc]` and documented `sha2` and `substring` as examples.
This PR applies the same approach to the remaining hash, bitwise, bitmap,
`equal_null` and `if` functions, so they appear in generated function
references.
## Changes
- Add `#[user_doc]` and `documentation()` to:
- hash: `sha1` (alias `sha`), `crc32`
- bitwise: `bit_get` (alias `getbit`), `bit_count`, `bitwise_not`,
`shiftleft`, `shiftright`, `shiftrightunsigned`
- bitmap: `bitmap_count`, `bitmap_bucket_number`, `bitmap_bit_position`
- misc: `equal_null`
- conditional: `if`
- Each description covers the Spark edge cases the tests check, for example:
- `bit_count` sign-extends to 64 bits.
- `bit_get`'s position limit depends on the input type. Integer literals
are BIGINT here and INT in Spark, so the limit for an uncast literal differs.
- Shift amounts are taken modulo the bit width.
- `equal_null` treats NaN as equal to NaN and 0.0 as equal to -0.0.
- `if` only raises errors from the branch it takes.
- `shiftleft`, `shiftright` and `shiftrightunsigned` share one struct,
`SparkBitShift`. A single `#[user_doc]` on it would give all three the same
syntax and examples. Instead, each function's docs sit on a private marker
type, and `documentation()` picks one based on the shift direction.
- Cut the structs' rustdoc down to a one-line summary and a link to the
Spark docs. Spark SQL has no `bitwise_not` function (it uses `~`), so that link
points to the PySpark docs.
- I didn't touch `sha2`, which is already documented, or `xxhash64`, which
#68 changes.
## Testing
- `cargo fmt --all`, `cargo clippy --all-targets --all-features -- -D
warnings`, `cargo test -p datafusion-spark` and `cargo test -p
datafusion-spark-sqllogictest` pass.
- I ran every SQL example in the docs (36 queries) through the sqllogictest
runner, and they all match.
- Everything the docs describe is covered by the existing SLT or unit tests
for these functions, or is visible in the code.
## User-facing changes
`documentation()` now returns docs for these functions. Behavior does not
change.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
