zhengruifeng commented on code in PR #42151:
URL: https://github.com/apache/spark/pull/42151#discussion_r1274257724
##########
python/pyspark/sql/dataframe.py:
##########
@@ -3831,42 +3856,57 @@ def unionByName(self, other: "DataFrame",
allowMissingColumns: bool = False) ->
allowMissingColumns : bool, optional, default False
Specify whether to allow missing columns.
- .. versionadded:: 3.1.0
-
Returns
-------
:class:`DataFrame`
- Combined DataFrame.
+ A new :class:`DataFrame` containing the combined rows with
corresponding
+ columns of the two given DataFrames.
+
+ Raises
+ ------
+ Py4JJavaError : If the column name conflict
Examples
--------
- The difference between this function and :func:`union` is that this
function
- resolves columns by name (not by position):
-
+ Example 1: Union of two DataFrames by Name
>>> df1 = spark.createDataFrame([[1, 2, 3]], ["col0", "col1", "col2"])
>>> df2 = spark.createDataFrame([[4, 5, 6]], ["col1", "col2", "col0"])
- >>> df1.unionByName(df2).show()
+ >>> df3 = df1.unionByName(df2)
+ >>> df3.show()
+----+----+----+
|col0|col1|col2|
+----+----+----+
| 1| 2| 3|
| 6| 4| 5|
+----+----+----+
- When the parameter `allowMissingColumns` is ``True``, the set of
column names
- in this and other :class:`DataFrame` can differ; missing columns will
be filled with null.
- Further, the missing columns of this :class:`DataFrame` will be added
at the end
- in the schema of the union result:
-
+ Example 2: Union of DataFrames with missing columns
>>> df1 = spark.createDataFrame([[1, 2, 3]], ["col0", "col1", "col2"])
>>> df2 = spark.createDataFrame([[4, 5, 6]], ["col1", "col2", "col3"])
- >>> df1.unionByName(df2, allowMissingColumns=True).show()
+ >>> df3 = df1.unionByName(df2, allowMissingColumns=True)
+ >>> df3.show()
+----+----+----+----+
|col0|col1|col2|col3|
+----+----+----+----+
- | 1| 2| 3|NULL|
- |NULL| 4| 5| 6|
+ | 1| 2| 3|null|
+ |null| 4| 5| 6|
Review Comment:
no, I just copy-paste the result from LLM.
And the new example result is wrong.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
