This is an automated email from the ASF dual-hosted git repository.
zhengruifeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 978c9177b531 [SPARK-58219][PYTHON][DOC] Add More See Also sections to
PySpark functions
978c9177b531 is described below
commit 978c9177b531493578c991227a19f27f5edf8e51
Author: Ruifeng Zheng <[email protected]>
AuthorDate: Tue Jul 21 08:39:31 2026 +0800
[SPARK-58219][PYTHON][DOC] Add More See Also sections to PySpark functions
### What changes were proposed in this pull request?
Adds reciprocal `See Also` sections to related PySpark SQL built-in
function docstrings:
- Null and NaN predicates: `isnan`, `isnull`, and `isnotnull`
- Null-handling functions: `nvl` and `nvl2`
- SQL pattern and regular-expression matching functions: `like`, `ilike`,
`rlike`, `regexp`, and `regexp_like`
- Array helpers: every `array_*` function now links to the closest related
array or collection APIs, including search, mutation, set operations, ordering,
sizing, construction, and joining
- Trigonometric functions: direct and inverse pairs, reciprocal functions,
hyperbolic and inverse-hyperbolic pairs, angle conversions, and the
`atan2`/`hypot` polar-coordinate pair
- Window functions: ranking and distribution functions (`row_number`,
`rank`, `dense_rank`, `percent_rank`, `cume_dist`, and `ntile`), plus
value-selection functions (`nth_value`, `first_value`, and `last_value`)
### Why are the changes needed?
Related functions and aliases are spread across the generated PySpark API
reference. Cross-references make their different matching, null-handling,
array-processing, mathematical, and windowing semantics easier to discover and
compare.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
No tests were added because this changes only API docstrings. Verified each
diff with `git diff --check` and parsed the module to confirm the targeted
`array_*`, trigonometric, and window APIs have `See Also` sections.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Codex (GPT-5)
Closes #57374 from zhengruifeng/python-see-also-docs.
Authored-by: Ruifeng Zheng <[email protected]>
Signed-off-by: Ruifeng Zheng <[email protected]>
---
python/pyspark/sql/functions/builtin.py | 276 ++++++++++++++++++++++++++++++++
1 file changed, 276 insertions(+)
diff --git a/python/pyspark/sql/functions/builtin.py
b/python/pyspark/sql/functions/builtin.py
index 2c2345c1a58a..78847d7d625d 100644
--- a/python/pyspark/sql/functions/builtin.py
+++ b/python/pyspark/sql/functions/builtin.py
@@ -2292,6 +2292,10 @@ def acos(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
A new column object representing the inverse cosine of the input.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.cos`
+
Examples
--------
Example 1: Compute the inverse cosine
@@ -2347,6 +2351,10 @@ def acosh(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
A new column object representing the inverse hyperbolic cosine of the
input.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.cosh`
+
Examples
--------
Example 1: Compute the inverse hyperbolic cosine
@@ -2398,6 +2406,10 @@ def asin(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
inverse sine of `col`, as if computed by `java.lang.Math.asin()`
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.sin`
+
Examples
--------
Example 1: Compute the inverse sine
@@ -2450,6 +2462,10 @@ def asinh(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
the column for computed results.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.sinh`
+
Examples
--------
Example 1: Compute the inverse hyperbolic sine
@@ -2501,6 +2517,11 @@ def atan(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
inverse tangent of `col`, as if computed by `java.lang.Math.atan()`
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.atan2`
+ :meth:`pyspark.sql.functions.tan`
+
Examples
--------
Example 1: Compute the inverse tangent
@@ -2552,6 +2573,10 @@ def atanh(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
the column for computed results.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.tanh`
+
Examples
--------
Example 1: Compute the inverse hyperbolic tangent
@@ -2772,6 +2797,11 @@ def cos(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
cosine of the angle, as if computed by `java.lang.Math.cos()`.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.acos`
+ :meth:`pyspark.sql.functions.sec`
+
Examples
--------
Example 1: Compute the cosine
@@ -2824,6 +2854,10 @@ def cosh(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
hyperbolic cosine of the angle, as if computed by
`java.lang.Math.cosh()`
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.acosh`
+
Examples
--------
Example 1: Compute the cosine
@@ -2875,6 +2909,10 @@ def cot(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
cotangent of the angle.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.tan`
+
Examples
--------
Example 1: Compute the cotangent
@@ -2927,6 +2965,10 @@ def csc(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
cosecant of the angle.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.sin`
+
Examples
--------
Example 1: Compute the cosecant
@@ -3383,6 +3425,10 @@ def sec(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
Secant of the angle.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.cos`
+
Examples
--------
Example 1: Compute the secant
@@ -3518,6 +3564,11 @@ def sin(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
sine of the angle, as if computed by `java.lang.Math.sin()`
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.asin`
+ :meth:`pyspark.sql.functions.csc`
+
Examples
--------
Example 1: Compute the sine
@@ -3571,6 +3622,10 @@ def sinh(col: "ColumnOrName") -> Column:
hyperbolic sine of the given value,
as if computed by `java.lang.Math.sinh()`
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.asinh`
+
Examples
--------
Example 1: Compute the hyperbolic sine
@@ -3622,6 +3677,12 @@ def tan(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
tangent of the given value, as if computed by `java.lang.Math.tan()`
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.atan`
+ :meth:`pyspark.sql.functions.atan2`
+ :meth:`pyspark.sql.functions.cot`
+
Examples
--------
Example 1: Compute the tangent
@@ -3675,6 +3736,10 @@ def tanh(col: "ColumnOrName") -> Column:
hyperbolic tangent of the given value
as if computed by `java.lang.Math.tanh()`
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.atanh`
+
Examples
--------
Example 1: Compute the hyperbolic tangent sine
@@ -6103,6 +6168,10 @@ def degrees(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
angle in degrees, as if computed by `java.lang.Math.toDegrees()`
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.radians`
+
Examples
--------
>>> from pyspark.sql import functions as sf
@@ -6142,6 +6211,10 @@ def radians(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
angle in radians, as if computed by `java.lang.Math.toRadians()`
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.degrees`
+
Examples
--------
>>> from pyspark.sql import functions as sf
@@ -6187,6 +6260,11 @@ def atan2(col1: Union["ColumnOrName", float], col2:
Union["ColumnOrName", float]
(`x`, `y`) in Cartesian coordinates,
as if computed by `java.lang.Math.atan2()`
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.atan`
+ :meth:`pyspark.sql.functions.hypot`
+
Examples
--------
>>> from pyspark.sql import functions as sf
@@ -6395,6 +6473,14 @@ def row_number() -> Column:
:class:`~pyspark.sql.Column`
the column for calculating row numbers.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.cume_dist`
+ :meth:`pyspark.sql.functions.dense_rank`
+ :meth:`pyspark.sql.functions.ntile`
+ :meth:`pyspark.sql.functions.percent_rank`
+ :meth:`pyspark.sql.functions.rank`
+
Examples
--------
>>> from pyspark.sql import functions as sf
@@ -6436,6 +6522,14 @@ def dense_rank() -> Column:
:class:`~pyspark.sql.Column`
the column for calculating ranks.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.cume_dist`
+ :meth:`pyspark.sql.functions.ntile`
+ :meth:`pyspark.sql.functions.percent_rank`
+ :meth:`pyspark.sql.functions.rank`
+ :meth:`pyspark.sql.functions.row_number`
+
Examples
--------
>>> from pyspark.sql import functions as sf
@@ -6480,6 +6574,14 @@ def rank() -> Column:
:class:`~pyspark.sql.Column`
the column for calculating ranks.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.cume_dist`
+ :meth:`pyspark.sql.functions.dense_rank`
+ :meth:`pyspark.sql.functions.ntile`
+ :meth:`pyspark.sql.functions.percent_rank`
+ :meth:`pyspark.sql.functions.row_number`
+
Examples
--------
>>> from pyspark.sql import functions as sf
@@ -6595,6 +6697,14 @@ def cume_dist() -> Column:
:class:`~pyspark.sql.Column`
the column for calculating cumulative distribution.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.dense_rank`
+ :meth:`pyspark.sql.functions.ntile`
+ :meth:`pyspark.sql.functions.percent_rank`
+ :meth:`pyspark.sql.functions.rank`
+ :meth:`pyspark.sql.functions.row_number`
+
Examples
--------
>>> from pyspark.sql import functions as sf
@@ -6630,6 +6740,14 @@ def percent_rank() -> Column:
:class:`~pyspark.sql.Column`
the column for calculating relative rank.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.cume_dist`
+ :meth:`pyspark.sql.functions.dense_rank`
+ :meth:`pyspark.sql.functions.ntile`
+ :meth:`pyspark.sql.functions.rank`
+ :meth:`pyspark.sql.functions.row_number`
+
Examples
--------
>>> from pyspark.sql import functions as sf
@@ -7360,6 +7478,11 @@ def isnan(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
True if value is NaN and False otherwise.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.isnull`
+ :meth:`pyspark.sql.functions.isnotnull`
+
Examples
--------
>>> from pyspark.sql import functions as sf
@@ -7394,6 +7517,11 @@ def isnull(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
True if value is null and False otherwise.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.isnan`
+ :meth:`pyspark.sql.functions.isnotnull`
+
Examples
--------
>>> from pyspark.sql import functions as sf
@@ -8933,6 +9061,11 @@ def nth_value(col: "ColumnOrName", offset: int,
ignoreNulls: Optional[bool] = Fa
:class:`~pyspark.sql.Column`
value of nth row.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.first_value`
+ :meth:`pyspark.sql.functions.last_value`
+
Examples
--------
>>> from pyspark.sql import functions as sf
@@ -9047,6 +9180,7 @@ def first_value(col: "ColumnOrName", ignoreNulls:
Optional[Union[bool, Column]]
See Also
--------
:meth:`pyspark.sql.functions.last_value`
+ :meth:`pyspark.sql.functions.nth_value`
Examples
--------
@@ -9100,6 +9234,7 @@ def last_value(col: "ColumnOrName", ignoreNulls:
Optional[Union[bool, Column]] =
See Also
--------
:meth:`pyspark.sql.functions.first_value`
+ :meth:`pyspark.sql.functions.nth_value`
Examples
--------
@@ -9268,6 +9403,14 @@ def ntile(n: int) -> Column:
:class:`~pyspark.sql.Column`
portioned group id.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.cume_dist`
+ :meth:`pyspark.sql.functions.dense_rank`
+ :meth:`pyspark.sql.functions.percent_rank`
+ :meth:`pyspark.sql.functions.rank`
+ :meth:`pyspark.sql.functions.row_number`
+
Examples
--------
>>> from pyspark.sql import functions as sf
@@ -16215,6 +16358,13 @@ def rlike(str: "ColumnOrName", regexp: "ColumnOrName")
-> Column:
:class:`~pyspark.sql.Column`
true if `str` matches a Java regex, or false otherwise.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.regexp`
+ :meth:`pyspark.sql.functions.regexp_like`
+ :meth:`pyspark.sql.functions.like`
+ :meth:`pyspark.sql.functions.ilike`
+
Examples
--------
>>> import pyspark.sql.functions as sf
@@ -16268,6 +16418,13 @@ def regexp(str: "ColumnOrName", regexp:
"ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
true if `str` matches a Java regex, or false otherwise.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.rlike`
+ :meth:`pyspark.sql.functions.regexp_like`
+ :meth:`pyspark.sql.functions.like`
+ :meth:`pyspark.sql.functions.ilike`
+
Examples
--------
>>> import pyspark.sql.functions as sf
@@ -16321,6 +16478,13 @@ def regexp_like(str: "ColumnOrName", regexp:
"ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
true if `str` matches a Java regex, or false otherwise.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.rlike`
+ :meth:`pyspark.sql.functions.regexp`
+ :meth:`pyspark.sql.functions.like`
+ :meth:`pyspark.sql.functions.ilike`
+
Examples
--------
>>> import pyspark.sql.functions as sf
@@ -18511,6 +18675,13 @@ def like(
If an escape character precedes a special symbol or another escape
character, the
following character is matched literally. It is invalid to escape any
other character.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.ilike`
+ :meth:`pyspark.sql.functions.rlike`
+ :meth:`pyspark.sql.functions.regexp`
+ :meth:`pyspark.sql.functions.regexp_like`
+
Examples
--------
>>> df = spark.createDataFrame([("Spark", "_park")], ['a', 'b'])
@@ -18561,6 +18732,13 @@ def ilike(
If an escape character precedes a special symbol or another escape
character, the
following character is matched literally. It is invalid to escape any
other character.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.like`
+ :meth:`pyspark.sql.functions.rlike`
+ :meth:`pyspark.sql.functions.regexp`
+ :meth:`pyspark.sql.functions.regexp_like`
+
Examples
--------
>>> df = spark.createDataFrame([("Spark", "_park")], ['a', 'b'])
@@ -19127,6 +19305,10 @@ def array_contains(col: "ColumnOrName", value: Any) ->
Column:
A new Column of Boolean type, where each value indicates whether the
corresponding array
from the input column contains the specified value.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.array_position`
+
Examples
--------
Example 1: Basic usage of array_contains function.
@@ -19365,6 +19547,11 @@ def array_join(
A new column of string type, where each value is the result of joining
the corresponding
array from the input column.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.concat`
+ :meth:`pyspark.sql.functions.concat_ws`
+
Examples
--------
Example 1: Basic usage of array_join function.
@@ -19559,6 +19746,10 @@ def array_position(col: "ColumnOrName", value: Any) ->
Column:
:class:`~pyspark.sql.Column`
position of the value in the given array if found and 0 otherwise.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.array_contains`
+
Examples
--------
Example 1: Finding the position of a string in an array of strings
@@ -19950,6 +20141,11 @@ def array_prepend(col: "ColumnOrName", value: Any) ->
Column:
:class:`~pyspark.sql.Column`
an array with the given value prepended.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.array_append`
+ :meth:`pyspark.sql.functions.array_insert`
+
Examples
--------
Example 1: Prepending a column value to an array column
@@ -20043,6 +20239,10 @@ def array_remove(col: "ColumnOrName", element: Any) ->
Column:
:class:`~pyspark.sql.Column`
A new column that is an array excluding the given value from the input
column.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.array_compact`
+
Examples
--------
Example 1: Removing a specific value from a simple array
@@ -20139,6 +20339,12 @@ def array_distinct(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
A new column that is an array of unique values from the input column.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.array_except`
+ :meth:`pyspark.sql.functions.array_intersect`
+ :meth:`pyspark.sql.functions.array_union`
+
Examples
--------
Example 1: Removing duplicate values from a simple array
@@ -20233,6 +20439,11 @@ def array_insert(arr: "ColumnOrName", pos:
Union["ColumnOrName", int], value: An
-----
Supports Spark Connect.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.array_append`
+ :meth:`pyspark.sql.functions.array_prepend`
+
Examples
--------
Example 1: Inserting a value at a specific position
@@ -20325,6 +20536,12 @@ def array_intersect(col1: "ColumnOrName", col2:
"ColumnOrName") -> Column:
-----
This function does not preserve the order of the elements in the input
arrays.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.array_distinct`
+ :meth:`pyspark.sql.functions.array_except`
+ :meth:`pyspark.sql.functions.array_union`
+
Examples
--------
Example 1: Basic usage
@@ -20418,6 +20635,12 @@ def array_union(col1: "ColumnOrName", col2:
"ColumnOrName") -> Column:
-----
This function does not preserve the order of the elements in the input
arrays.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.array_distinct`
+ :meth:`pyspark.sql.functions.array_except`
+ :meth:`pyspark.sql.functions.array_intersect`
+
Examples
--------
Example 1: Basic usage
@@ -20511,6 +20734,12 @@ def array_except(col1: "ColumnOrName", col2:
"ColumnOrName") -> Column:
-----
This function does not preserve the order of the elements in the input
arrays.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.array_distinct`
+ :meth:`pyspark.sql.functions.array_intersect`
+ :meth:`pyspark.sql.functions.array_union`
+
Examples
--------
Example 1: Basic usage
@@ -20598,6 +20827,10 @@ def array_compact(col: "ColumnOrName") -> Column:
-----
Supports Spark Connect.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.array_remove`
+
Examples
--------
Example 1: Removing null values from a simple array
@@ -20690,6 +20923,11 @@ def array_append(col: "ColumnOrName", value: Any) ->
Column:
-----
Supports Spark Connect.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.array_insert`
+ :meth:`pyspark.sql.functions.array_prepend`
+
Examples
--------
Example 1: Appending a column value to an array column
@@ -22839,6 +23077,12 @@ def array_min(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
A new column that contains the minimum value of each array.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.array_max`
+ :meth:`pyspark.sql.functions.array_sort`
+ :meth:`pyspark.sql.functions.sort_array`
+
Examples
--------
Example 1: Basic usage with integer array
@@ -22924,6 +23168,12 @@ def array_max(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
A new column that contains the maximum value of each array.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.array_min`
+ :meth:`pyspark.sql.functions.array_sort`
+ :meth:`pyspark.sql.functions.sort_array`
+
Examples
--------
Example 1: Basic usage with integer array
@@ -23007,6 +23257,11 @@ def array_size(col: "ColumnOrName") -> Column:
:class:`~pyspark.sql.Column`
A new column that contains the size of each array.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.cardinality`
+ :meth:`pyspark.sql.functions.size`
+
Examples
--------
Example 1: Basic usage with integer array
@@ -23230,6 +23485,10 @@ def array_sort(
:class:`~pyspark.sql.Column`
sorted array.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.sort_array`
+
Examples
--------
>>> df = spark.createDataFrame([([2, 1, None, 3],),([1],),([],)], ['data'])
@@ -23857,6 +24116,10 @@ def array_repeat(col: "ColumnOrName", count:
Union["ColumnOrName", int]) -> Colu
:class:`~pyspark.sql.Column`
A new column that contains an array of repeated elements.
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.array`
+
Examples
--------
Example 1: Usage with string
@@ -29956,6 +30219,11 @@ def isnotnull(col: "ColumnOrName") -> Column:
----------
col : :class:`~pyspark.sql.Column` or column name
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.isnan`
+ :meth:`pyspark.sql.functions.isnull`
+
Examples
--------
>>> from pyspark.sql import functions as sf
@@ -30096,6 +30364,10 @@ def nvl(col1: "ColumnOrName", col2: "ColumnOrName") ->
Column:
col1 : :class:`~pyspark.sql.Column` or column name
col2 : :class:`~pyspark.sql.Column` or column name
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.nvl2`
+
Examples
--------
>>> import pyspark.sql.functions as sf
@@ -30132,6 +30404,10 @@ def nvl2(col1: "ColumnOrName", col2: "ColumnOrName",
col3: "ColumnOrName") -> Co
col2 : :class:`~pyspark.sql.Column` or column name
col3 : :class:`~pyspark.sql.Column` or column name
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.nvl`
+
Examples
--------
>>> import pyspark.sql.functions as sf
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
