jubins opened a new pull request, #56851:
URL: https://github.com/apache/spark/pull/56851
### What is the purpose of the change
Fixes SPARK-57740 — two related doc-correctness issues in the `unix_*`
epoch-unit function family in `python/pyspark/sql/functions/builtin.py`:
1. **Missing truncation disclaimer in `unix_micros`.** `unix_millis` and
`unix_seconds` both include `"Truncates higher levels of precision."` in their
one-line description. `unix_micros` did not, despite silently dropping
sub-microsecond digits when given a nanosecond-precision input
(`TIMESTAMP_LTZ(p)` / `TIMESTAMP_NTZ(p)`, `p` in `[7, 9]`). This became a
practical concern once `unix_nanos` (SPARK-57579) made nanosecond-precision
timestamps usable in PySpark.
2. **Missing `unix_nanos` cross-references.** `unix_nanos` (SPARK-57579)
correctly links to all three siblings in its `See Also` section. None of the
three sibling functions were updated to link back to `unix_nanos`, breaking
discoverability of the function family from the existing functions.
### Brief change log
- `python/pyspark/sql/functions/builtin.py`:
- `unix_micros`: added `"Truncates higher levels of precision."` to the
one-line description
- `unix_micros`, `unix_millis`, `unix_seconds`: added
`:meth:\`pyspark.sql.functions.unix_nanos\`` to each `See Also` section
### Verifying this change
These are documentation-only changes with no logic impact. Verified by
inspection that:
- All four `unix_*` functions now consistently include a truncation
disclaimer in their description
- All four `unix_*` functions now cross-reference each other in their `See
Also` sections symmetrically
### Does this pull request potentially affect one of the following parts
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public`/`@Evolving`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: no
- The S3 file system connector: no
### Documentation
Does this pull request introduce a new feature? No — documentation fix only.
### Was generative AI tooling used to co-author this PR?
- [x] Yes — Claude Code was used as a pair-programming assistant. All code
was written, understood, and verified by the author.
Generated-by: Claude Opus 4.8
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
