itholic commented on PR #43214:
URL: https://github.com/apache/spark/pull/43214#issuecomment-1746610365
This functionality works fine in manual testing with Python interpreter:
```python
>>> spark # check if the current session is Spark Connect session.
<pyspark.sql.connect.session.SparkSession object at 0x105b3fbe0>
>>> import pyspark.pandas as ps
>>> import numpy as np
>>> psdf = ps.DataFrame({"A": [1, 2, 3]})
>>> np_name = "arccosh"
>>> np_func = getattr(np, np_name)
>>> np_func(psdf)
A
0 0.000000
1 1.316958
2 1.762747
```
But failed in UT:
```cmd
spark % python/run-tests --testnames
'pyspark.pandas.tests.connect.test_parity_numpy_compat
NumPyCompatParityTests.test_np_spark_compat_frame'
...
======================================================================
FAIL [3.103s]: test_np_spark_compat_frame
(pyspark.pandas.tests.connect.test_parity_numpy_compat.NumPyCompatParityTests)
----------------------------------------------------------------------
...
pyspark.errors.exceptions.base.PySparkTypeError: [NOT_COLUMN_OR_STR]
Argument `col` should be a Column or str, got Column.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
...
AssertionError: Test in 'arccosh' function was failed.
----------------------------------------------------------------------
Ran 1 test in 5.956s
```
Let me test how it works on GitHub Actions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
