Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22655#discussion_r223214749
--- Diff: python/pyspark/sql/functions.py ---
@@ -2733,6 +2733,33 @@ def udf(f=None, returnType=StringType()):
| 8| JOHN DOE| 22|
+----------+--------------+------------+
"""
+
+ # The following table shows most of Python data and SQL type
conversions in normal UDFs that
+ # are not yet visible to the user. Some of behaviors are buggy and
might be changed in the near
+ # future. The table might have to be eventually documented externally.
+ # Please see SPARK-25666's PR to see the codes in order to generate
the table below.
+ #
+ #
+-----------------------------+--------------+----------+------+-------+------+----------+--------------------+-----------------------------+----------+----------------------+---------+--------------------+--------------+----------+--------------+-------------+-------------+
# noqa
+ # |SQL Type \ Python
Value(Type)|None(NoneType)|True(bool)|1(int)|1(long)|a(str)|a(unicode)|
1970-01-01(date)|1970-01-01 00:00:00(datetime)|1.0(float)|array('i',
[1])(array)|[1](list)| (1,)(tuple)|ABC(bytearray)|1(Decimal)|{'a':
1}(dict)|Row(a=1)(Row)|Row(a=1)(Row)| # noqa
--- End diff --
Any difference between last two `Row(a=1)(Row)`?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
