This is an automated email from the ASF dual-hosted git repository.
HyukjinKwon pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new 0a27f736f2df [MINOR][PYTHON][DOC] Fixed some links in documentation.
0a27f736f2df is described below
commit 0a27f736f2df5a3391dee1276141214417147057
Author: art000109 <[email protected]>
AuthorDate: Wed Jul 22 07:58:32 2026 +0900
[MINOR][PYTHON][DOC] Fixed some links in documentation.
### What changes were proposed in this pull request?
Fixed some links in DataFrame and Pandas-On-Spark-DataFrame classes.
### Why are the changes needed?
The documentation will be more comprehensive.
### Does this PR introduce any user-facing change?
No. This is a documentation-only change.
### How was this patch tested?
Manual review.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #57359 from art000109/master.
Lead-authored-by: art000109 <[email protected]>
Co-authored-by: Artem Kozhin <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit c198adcf3268e934f14a1bb76504b84163ef3bb5)
Signed-off-by: Hyukjin Kwon <[email protected]>
---
.../source/reference/pyspark.sql/core_classes.rst | 3 +++
python/pyspark/pandas/frame.py | 24 ++++++++++---------
python/pyspark/sql/dataframe.py | 4 +++-
python/pyspark/sql/pandas/functions.py | 2 +-
python/pyspark/sql/tvf.py | 28 +++++++++++-----------
5 files changed, 34 insertions(+), 27 deletions(-)
diff --git a/python/docs/source/reference/pyspark.sql/core_classes.rst
b/python/docs/source/reference/pyspark.sql/core_classes.rst
index af38ef1c4f5e..1ba4c62994b7 100644
--- a/python/docs/source/reference/pyspark.sql/core_classes.rst
+++ b/python/docs/source/reference/pyspark.sql/core_classes.rst
@@ -38,6 +38,9 @@ Core Classes
DataFrameReader
DataFrameWriter
DataFrameWriterV2
+ MergeIntoWriter
+ DataStreamReader
+ DataStreamWriter
UDFRegistration
UDTFRegistration
udf.UserDefinedFunction
diff --git a/python/pyspark/pandas/frame.py b/python/pyspark/pandas/frame.py
index 4b9ce84b3b7a..5bdc665a15e7 100644
--- a/python/pyspark/pandas/frame.py
+++ b/python/pyspark/pandas/frame.py
@@ -390,6 +390,19 @@ class DataFrame(Frame, Generic[T]):
pandas-on-Spark DataFrame that corresponds to pandas DataFrame logically.
This holds Spark
DataFrame internally.
+ .. versionadded:: 3.2.0
+
+ .. versionchanged:: 3.4.0
+ Since 3.4.0, it deals with `data` and `index` in this approach:
+ 1, when `data` is a distributed dataset (Internal DataFrame/Spark
DataFrame/
+ pandas-on-Spark DataFrame/pandas-on-Spark Series), it will first
parallelize
+ the `index` if necessary, and then try to combine the `data` and
`index`;
+ Note that if `data` and `index` doesn't have the same anchor, then
+ `compute.ops_on_diff_frames` should be turned on;
+ 2, when `data` is a local dataset (Pandas DataFrame/numpy
ndarray/list/etc),
+ it will first collect the `index` to driver if necessary, and then
apply
+ the `pandas.DataFrame(...)` creation internally;
+
:ivar _internal: an internal immutable Frame to manage metadata.
:type _internal: InternalFrame
@@ -409,17 +422,6 @@ class DataFrame(Frame, Generic[T]):
copy : boolean, default False
Copy data from inputs. Only affects DataFrame / 2d ndarray input
- .. versionchanged:: 3.4.0
- Since 3.4.0, it deals with `data` and `index` in this approach:
- 1, when `data` is a distributed dataset (Internal DataFrame/Spark
DataFrame/
- pandas-on-Spark DataFrame/pandas-on-Spark Series), it will first
parallelize
- the `index` if necessary, and then try to combine the `data` and
`index`;
- Note that if `data` and `index` doesn't have the same anchor, then
- `compute.ops_on_diff_frames` should be turned on;
- 2, when `data` is a local dataset (Pandas DataFrame/numpy
ndarray/list/etc),
- it will first collect the `index` to driver if necessary, and then
apply
- the `pandas.DataFrame(...)` creation internally;
-
Examples
--------
Constructing DataFrame from a dictionary.
diff --git a/python/pyspark/sql/dataframe.py b/python/pyspark/sql/dataframe.py
index 6c4d32ea1797..508e83f84d4b 100644
--- a/python/pyspark/sql/dataframe.py
+++ b/python/pyspark/sql/dataframe.py
@@ -1206,6 +1206,8 @@ class DataFrame:
:class:`DataFrame`
Hinted DataFrame
+ .. note:: See also `Hints
<https://spark.apache.org/docs/latest/sql-ref-syntax-qry-select-hints.html>`_
+
Examples
--------
>>> df = spark.createDataFrame([(2, "Alice"), (5, "Bob")],
schema=["age", "name"])
@@ -6482,7 +6484,7 @@ class DataFrame:
See Also
--------
- pyspark.pandas.frame.DataFrame.to_spark
+ pyspark.pandas.DataFrame.to_spark
Examples
--------
diff --git a/python/pyspark/sql/pandas/functions.py
b/python/pyspark/sql/pandas/functions.py
index b14b10b44859..4b6699829996 100644
--- a/python/pyspark/sql/pandas/functions.py
+++ b/python/pyspark/sql/pandas/functions.py
@@ -74,7 +74,7 @@ def arrow_udf(f=None, returnType=None, functionType=None):
the return type of the user-defined function. The value can be either a
:class:`pyspark.sql.types.DataType` object or a DDL-formatted type
string.
functionType : int, optional
- an enum value in :class:`pyspark.sql.functions.ArrowUDFType`.
+ an enum value in :class:`~pyspark.sql.functions.ArrowUDFType`.
Default: SCALAR. This parameter exists for compatibility.
Using Python type hints is encouraged.
diff --git a/python/pyspark/sql/tvf.py b/python/pyspark/sql/tvf.py
index 90874578a356..57b0f525675f 100644
--- a/python/pyspark/sql/tvf.py
+++ b/python/pyspark/sql/tvf.py
@@ -59,7 +59,7 @@ class TableValuedFunction:
Returns
-------
- :class:`DataFrame`
+ :class:`~pyspark.sql.DataFrame`
Examples
--------
@@ -101,7 +101,7 @@ class TableValuedFunction:
Returns
-------
- :class:`DataFrame`
+ :class:`~pyspark.sql.DataFrame`
See Also
--------
@@ -185,7 +185,7 @@ class TableValuedFunction:
Returns
-------
- :class:`DataFrame`
+ :class:`~pyspark.sql.DataFrame`
See Also
--------
@@ -238,7 +238,7 @@ class TableValuedFunction:
Returns
-------
- :class:`DataFrame`
+ :class:`~pyspark.sql.DataFrame`
See Also
--------
@@ -301,7 +301,7 @@ class TableValuedFunction:
Returns
-------
- :class:`DataFrame`
+ :class:`~pyspark.sql.DataFrame`
See Also
--------
@@ -344,7 +344,7 @@ class TableValuedFunction:
Returns
-------
- :class:`DataFrame`
+ :class:`~pyspark.sql.DataFrame`
See Also
--------
@@ -394,7 +394,7 @@ class TableValuedFunction:
Returns
-------
- :class:`DataFrame`
+ :class:`~pyspark.sql.DataFrame`
See Also
--------
@@ -437,7 +437,7 @@ class TableValuedFunction:
Returns
-------
- :class:`DataFrame`
+ :class:`~pyspark.sql.DataFrame`
See Also
--------
@@ -490,7 +490,7 @@ class TableValuedFunction:
Returns
-------
- :class:`DataFrame`
+ :class:`~pyspark.sql.DataFrame`
See Also
--------
@@ -525,7 +525,7 @@ class TableValuedFunction:
Returns
-------
- :class:`DataFrame`
+ :class:`~pyspark.sql.DataFrame`
Examples
--------
@@ -546,7 +546,7 @@ class TableValuedFunction:
Returns
-------
- :class:`DataFrame`
+ :class:`~pyspark.sql.DataFrame`
Examples
--------
@@ -574,7 +574,7 @@ class TableValuedFunction:
Returns
-------
- :class:`DataFrame`
+ :class:`~pyspark.sql.DataFrame`
Examples
--------
@@ -639,7 +639,7 @@ class TableValuedFunction:
Returns
-------
- :class:`DataFrame`
+ :class:`~pyspark.sql.DataFrame`
Examples
--------
@@ -681,7 +681,7 @@ class TableValuedFunction:
Returns
-------
- :class:`DataFrame`
+ :class:`~pyspark.sql.DataFrame`
Examples
--------
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
