Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/20487
  
    Ah, nope. If PyArrow version is lower then we claim, for example, 0.7.0, 
seems tests go failed:
    
    ```
    ======================================================================
    ERROR: test_vectorized_udf_wrong_return_type 
(pyspark.sql.tests.ScalarPandasUDF)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/.../spark/python/pyspark/sql/tests.py", line 4019, in 
test_vectorized_udf_wrong_return_type
        f = pandas_udf(lambda x: x * 1.0, MapType(LongType(), LongType()))
      File "/.../spark/python/pyspark/sql/functions.py", line 2309, in 
pandas_udf
        return _create_udf(f=f, returnType=return_type, evalType=eval_type)
      File "/.../spark/python/pyspark/sql/udf.py", line 47, in _create_udf
        require_minimum_pyarrow_version()
      File "/.../spark/python/pyspark/sql/utils.py", line 132, in 
require_minimum_pyarrow_version
        "however, your version was %s." % pyarrow.__version__)
    ImportError: pyarrow >= 0.8.0 must be installed on calling Python process; 
however, your version was 0.7.0.
    
    ----------------------------------------------------------------------
    Ran 33 tests in 8.098s
    
    FAILED (errors=33)
    ```
    
    Will clarify it in PR description.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to