This is an automated email from the ASF dual-hosted git repository.
zhengruifeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new cc88e6c778dd [SPARK-57367][PYTHON][DOC] Improve See Also
cross-references in pyspark.sql.functions
cc88e6c778dd is described below
commit cc88e6c778ddb400cf13d7cdd98307717e8e87ff
Author: Ruifeng Zheng <[email protected]>
AuthorDate: Wed Jun 10 19:31:19 2026 +0800
[SPARK-57367][PYTHON][DOC] Improve See Also cross-references in
pyspark.sql.functions
### What changes were proposed in this pull request?
Improve `See Also` cross-references in `pyspark/sql/functions/builtin.py`
across three categories:
**Fix broken links** (referenced functions that don't exist):
- `var_samp`: `std_samp` → `stddev_samp`
- `var_pop`: `std_pop` → `stddev_pop`
**Add new `See Also` sections** (functions that had none):
| Category | Functions |
|---|---|
| Math aliases | `ceil` ↔ `ceiling`, `sign` ↔ `signum`, `log` ↔ `ln` |
| String aliases | `lcase`, `ucase`, `length` / `char_length` /
`character_length`, `printf` |
| Try variants | `to_binary` ↔ `try_to_binary`, `to_number` ↔
`try_to_number` |
| Aggregates | `avg`, `sum`, `median`, `count_distinct`, `uniform` |
**Add missing symmetric cross-references** to existing `See Also` sections:
| Function | Added |
|---|---|
| `lower`, `upper` | `lcase`, `ucase` (alias back-links) |
| `variance` | `std` |
| `getbit` | `bit_count` |
| `get` | `try_element_at` |
| `substr` | `locate` |
| `day` | `weekday` |
| `dayofyear` | `dayofweek` |
| `weekday` | `dayofweek`, `dayofyear`, `dayofmonth` |
| `date_add`, `dateadd` | `add_months` |
| `date_diff`, `timestamp_diff` | `time_diff` |
| `to_date` ↔ `try_to_date` | each other |
| `to_timestamp_ltz`, `to_timestamp_ntz` | `try_to_timestamp` |
| `try_to_timestamp` | `try_to_date`, `try_to_time` |
| `bitmap_construct_agg` | `bitmap_and_agg` |
### Why are the changes needed?
The broken links produce dead references in the generated API docs (e.g.,
https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.var_samp.html).
The missing cross-references make it harder for users to discover related
functions when browsing the API documentation.
### Does this PR introduce _any_ user-facing change?
No. Documentation-only changes.
### How was this patch tested?
Verified programmatically that all referenced function names resolve to
actual definitions or re-exported names in `pyspark.sql.functions`.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Sonnet 4.6
Closes #56404 from zhengruifeng/pyspark-see-also-dev4.
Authored-by: Ruifeng Zheng <[email protected]>
Signed-off-by: Ruifeng Zheng <[email protected]>
---
python/pyspark/sql/functions/builtin.py | 120 ++++++++++++++++++++++++++++++++
1 file changed, 120 insertions(+)
diff --git a/python/pyspark/sql/functions/builtin.py
b/python/pyspark/sql/functions/builtin.py
index 2f48d1e1901f..59fc22cb43f1 100644
--- a/python/pyspark/sql/functions/builtin.py
+++ b/python/pyspark/sql/functions/builtin.py
@@ -1644,6 +1644,12 @@ def sum(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
the column for computed results.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.min`
+ :meth:`pyspark.sql.functions.max`
+ :meth:`pyspark.sql.functions.avg`
+
Examples
--------
Example 1: Calculating the sum of values in a column
@@ -1702,6 +1708,12 @@ def avg(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
the column for computed results.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.min`
+ :meth:`pyspark.sql.functions.max`
+ :meth:`pyspark.sql.functions.sum`
+
Examples
--------
Example 1: Calculating the average age
@@ -1802,6 +1814,12 @@ def median(col: "ColumnOrName") -> Column:
:meth:`pyspark.sql.functions.approx_percentile`
:meth:`pyspark.sql.functions.percentile_approx`
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.approx_percentile`
+ :meth:`pyspark.sql.functions.percentile`
+ :meth:`pyspark.sql.functions.percentile_approx`
+
Examples
--------
>>> from pyspark.sql import functions as sf
@@ -2642,6 +2660,10 @@ def ceil(col: "ColumnOrName", scale:
Optional[Union[Column, int]] = None) -> Col
:class:`~pyspark.sql.Column`
A column for the computed results.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.ceiling`
+
Examples
--------
Example 1: Compute the ceiling of a column value
@@ -2696,6 +2718,10 @@ def ceiling(col: "ColumnOrName", scale:
Optional[Union[Column, int]] = None) ->
:class:`~pyspark.sql.Column`
A column for the computed results.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.ceil`
+
Examples
--------
Example 1: Compute the ceiling of a column value
@@ -3408,6 +3434,10 @@ def signum(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
the column for computed results.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.sign`
+
Examples
--------
>>> import pyspark.sql.functions as sf
@@ -3446,6 +3476,10 @@ def sign(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
the column for computed results.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.signum`
+
Examples
--------
>>> import pyspark.sql.functions as sf
@@ -3889,6 +3923,7 @@ def getbit(col: "ColumnOrName", pos: "ColumnOrName") ->
Column:
See Also
--------
:meth:`pyspark.sql.functions.bit_get`
+ :meth:`pyspark.sql.functions.bit_count`
Examples
--------
@@ -4412,6 +4447,7 @@ def variance(col: "ColumnOrName") -> Column:
:meth:`pyspark.sql.functions.stddev`
:meth:`pyspark.sql.functions.skewness`
:meth:`pyspark.sql.functions.kurtosis`
+ :meth:`pyspark.sql.functions.std`
Examples
--------
@@ -6984,6 +7020,10 @@ def count_distinct(col: "ColumnOrName", *cols:
"ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
distinct values of these two column values.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.approx_count_distinct`
+
Examples
--------
Example 1: Counting distinct values of a single column
@@ -8455,6 +8495,10 @@ def log(arg1: Union["ColumnOrName", float], arg2:
Optional["ColumnOrName"] = Non
:class:`~pyspark.sql.Column`
logariphm of given value.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.ln`
+
Examples
--------
Example 1: Specify both base number and the input value
@@ -8522,6 +8566,10 @@ def ln(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
natural logarithm of given value.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.log`
+
Examples
--------
>>> from pyspark.sql import functions as sf
@@ -10113,6 +10161,7 @@ def day(col: "ColumnOrName") -> Column:
:meth:`pyspark.sql.functions.extract`
:meth:`pyspark.sql.functions.datepart`
:meth:`pyspark.sql.functions.date_part`
+ :meth:`pyspark.sql.functions.weekday`
Examples
--------
@@ -10199,6 +10248,7 @@ def dayofyear(col: "ColumnOrName") -> Column:
:meth:`pyspark.sql.functions.dayofyear`
:meth:`pyspark.sql.functions.dayofmonth`
:meth:`pyspark.sql.functions.weekofyear`
+ :meth:`pyspark.sql.functions.dayofweek`
Examples
--------
@@ -10614,6 +10664,9 @@ def weekday(col: "ColumnOrName") -> Column:
--------
:meth:`pyspark.sql.functions.day`
:meth:`pyspark.sql.functions.weekofyear`
+ :meth:`pyspark.sql.functions.dayofweek`
+ :meth:`pyspark.sql.functions.dayofyear`
+ :meth:`pyspark.sql.functions.dayofmonth`
Examples
--------
@@ -11075,6 +11128,7 @@ def date_add(start: "ColumnOrName", days:
Union["ColumnOrName", int]) -> Column:
:meth:`pyspark.sql.functions.datediff`
:meth:`pyspark.sql.functions.date_diff`
:meth:`pyspark.sql.functions.timestamp_add`
+ :meth:`pyspark.sql.functions.add_months`
Examples
--------
@@ -11134,6 +11188,7 @@ def dateadd(start: "ColumnOrName", days:
Union["ColumnOrName", int]) -> Column:
:meth:`pyspark.sql.functions.datediff`
:meth:`pyspark.sql.functions.date_diff`
:meth:`pyspark.sql.functions.timestamp_add`
+ :meth:`pyspark.sql.functions.add_months`
Examples
--------
@@ -11303,6 +11358,7 @@ def date_diff(end: "ColumnOrName", start:
"ColumnOrName") -> Column:
:meth:`pyspark.sql.functions.date_sub`
:meth:`pyspark.sql.functions.datediff`
:meth:`pyspark.sql.functions.timestamp_diff`
+ :meth:`pyspark.sql.functions.time_diff`
Examples
--------
@@ -11516,6 +11572,7 @@ def to_date(col: "ColumnOrName", format: Optional[str]
= None) -> Column:
:meth:`pyspark.sql.functions.to_utc_timestamp`
:meth:`pyspark.sql.functions.try_to_timestamp`
:meth:`pyspark.sql.functions.date_format`
+ :meth:`pyspark.sql.functions.try_to_date`
Examples
--------
@@ -11572,6 +11629,7 @@ def try_to_date(col: "ColumnOrName", format:
Optional[str] = None) -> Column:
:meth:`pyspark.sql.functions.to_utc_timestamp`
:meth:`pyspark.sql.functions.try_to_timestamp`
:meth:`pyspark.sql.functions.date_format`
+ :meth:`pyspark.sql.functions.to_date`
Examples
--------
@@ -12019,6 +12077,8 @@ def try_to_timestamp(col: "ColumnOrName", format:
Optional["ColumnOrName"] = Non
:meth:`pyspark.sql.functions.to_timestamp`
:meth:`pyspark.sql.functions.to_utc_timestamp`
:meth:`pyspark.sql.functions.date_format`
+ :meth:`pyspark.sql.functions.try_to_date`
+ :meth:`pyspark.sql.functions.try_to_time`
Examples
--------
@@ -13096,6 +13156,7 @@ def timestamp_diff(unit: str, start: "ColumnOrName",
end: "ColumnOrName") -> Col
--------
:meth:`pyspark.sql.functions.datediff`
:meth:`pyspark.sql.functions.date_diff`
+ :meth:`pyspark.sql.functions.time_diff`
Examples
--------
@@ -13615,6 +13676,7 @@ def to_timestamp_ltz(
:meth:`pyspark.sql.functions.to_utc_timestamp`
:meth:`pyspark.sql.functions.to_unix_timestamp`
:meth:`pyspark.sql.functions.date_format`
+ :meth:`pyspark.sql.functions.try_to_timestamp`
Examples
--------
@@ -13685,6 +13747,7 @@ def to_timestamp_ntz(
:meth:`pyspark.sql.functions.to_utc_timestamp`
:meth:`pyspark.sql.functions.to_unix_timestamp`
:meth:`pyspark.sql.functions.date_format`
+ :meth:`pyspark.sql.functions.try_to_timestamp`
Examples
--------
@@ -14357,6 +14420,7 @@ def upper(col: "ColumnOrName") -> Column:
See Also
--------
:meth:`pyspark.sql.functions.lower`
+ :meth:`pyspark.sql.functions.ucase`
Examples
--------
@@ -14397,6 +14461,7 @@ def lower(col: "ColumnOrName") -> Column:
See Also
--------
:meth:`pyspark.sql.functions.upper`
+ :meth:`pyspark.sql.functions.lcase`
Examples
--------
@@ -16826,6 +16891,11 @@ def uniform(
:class:`~pyspark.sql.Column`
The generated random number within the specified range.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.rand`
+ :meth:`pyspark.sql.functions.randn`
+
Examples
--------
>>> import pyspark.sql.functions as sf
@@ -16878,6 +16948,11 @@ def length(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
length of the value.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.char_length`
+ :meth:`pyspark.sql.functions.character_length`
+
Examples
--------
>>> from pyspark.sql import functions as sf
@@ -17024,6 +17099,10 @@ def to_binary(col: "ColumnOrName", format:
Optional["ColumnOrName"] = None) -> C
format : :class:`~pyspark.sql.Column` or str, optional
format to use to convert binary values.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.try_to_binary`
+
Examples
--------
Example 1: Convert string to a binary with encoding specified
@@ -17170,6 +17249,10 @@ def to_number(col: "ColumnOrName", format:
"ColumnOrName") -> Column:
format : :class:`~pyspark.sql.Column` or str, optional
format to use to convert number values.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.try_to_number`
+
Examples
--------
>>> df = spark.createDataFrame([("$78.12",)], ["e"])
@@ -17289,6 +17372,7 @@ def substr(
:meth:`pyspark.sql.functions.substring`
:meth:`pyspark.sql.functions.substring_index`
:meth:`pyspark.sql.Column.substr`
+ :meth:`pyspark.sql.functions.locate`
Examples
--------
@@ -17546,6 +17630,10 @@ def printf(format: "ColumnOrName", *cols:
"ColumnOrName") -> Column:
cols : :class:`~pyspark.sql.Column` or str
column names or :class:`~pyspark.sql.Column`\\s to be used in
formatting
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.format_string`
+
Examples
--------
>>> import pyspark.sql.functions as sf
@@ -17957,6 +18045,11 @@ def char_length(str: "ColumnOrName") -> Column:
str : :class:`~pyspark.sql.Column` or str
Input column or strings.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.character_length`
+ :meth:`pyspark.sql.functions.length`
+
Examples
--------
>>> import pyspark.sql.functions as sf
@@ -17984,6 +18077,11 @@ def character_length(str: "ColumnOrName") -> Column:
str : :class:`~pyspark.sql.Column` or str
Input column or strings.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.char_length`
+ :meth:`pyspark.sql.functions.length`
+
Examples
--------
>>> import pyspark.sql.functions as sf
@@ -18047,6 +18145,10 @@ def try_to_binary(col: "ColumnOrName", format:
Optional["ColumnOrName"] = None)
format : :class:`~pyspark.sql.Column` or str, optional
format to use to convert binary values.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.to_binary`
+
Examples
--------
Example 1: Convert string to a binary with encoding specified
@@ -18101,6 +18203,10 @@ def try_to_number(col: "ColumnOrName", format:
"ColumnOrName") -> Column:
format : :class:`~pyspark.sql.Column` or str, optional
format to use to convert number values.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.to_number`
+
Examples
--------
Example 1: Convert a string to a number with a format specified
@@ -18335,6 +18441,12 @@ def lcase(str: "ColumnOrName") -> Column:
str : :class:`~pyspark.sql.Column` or str
Input column or strings.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.lower`
+ :meth:`pyspark.sql.functions.ucase`
+ :meth:`pyspark.sql.functions.upper`
+
Examples
--------
>>> import pyspark.sql.functions as sf
@@ -18360,6 +18472,12 @@ def ucase(str: "ColumnOrName") -> Column:
str : :class:`~pyspark.sql.Column` or str
Input column or strings.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.upper`
+ :meth:`pyspark.sql.functions.lcase`
+ :meth:`pyspark.sql.functions.lower`
+
Examples
--------
>>> import pyspark.sql.functions as sf
@@ -19595,6 +19713,7 @@ def get(col: "ColumnOrName", index:
Union["ColumnOrName", int]) -> Column:
See Also
--------
:meth:`pyspark.sql.functions.element_at`
+ :meth:`pyspark.sql.functions.try_element_at`
Examples
--------
@@ -30442,6 +30561,7 @@ def bitmap_construct_agg(col: "ColumnOrName") -> Column:
:meth:`pyspark.sql.functions.bitmap_bucket_number`
:meth:`pyspark.sql.functions.bitmap_count`
:meth:`pyspark.sql.functions.bitmap_or_agg`
+ :meth:`pyspark.sql.functions.bitmap_and_agg`
Examples
--------
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
