ueshin commented on a change in pull request #27165: [SPARK-28264][PYTHON][SQL]
Support type hints in pandas UDF and rename/move inconsistent pandas UDF types
URL: https://github.com/apache/spark/pull/27165#discussion_r369294506
##########
File path: python/pyspark/sql/pandas/group_ops.py
##########
@@ -74,6 +79,11 @@ def apply(self, udf):
assert isinstance(self, GroupedData)
+ warnings.warn(
+ "It is preferred to use 'applyInPandas' over this "
+ "API. This API will be deprecated in the future releases. See
SPARK-28264 for "
+ "more details.", UserWarning)
Review comment:
Seems trying to always show this warning even when using `applyInPandas`?
Maybe we should move the following logic into `applyInPandas` and call it
from here after showing the warning?
```py
>>> df.groupby("id").applyInPandas(normalize, schema="id long, v
double").show()
/.../pyspark/sql/pandas/group_ops.py:85: UserWarning: It is preferred to use
'applyInPandas' over this API. This API will be deprecated in the future
releases. See SPARK-28264 for more details.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
