Github user HyukjinKwon commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20487#discussion_r165714499
  
    --- Diff: python/pyspark/sql/dataframe.py ---
    @@ -1923,6 +1923,9 @@ def toPandas(self):
             0    2  Alice
             1    5    Bob
             """
    +        from pyspark.sql.utils import require_minimum_pandas_version
    --- End diff --
    
    `toPandas` seems already failed when it includes types `TimestampType`:
    
    ```
    >>> import datetime
    >>> spark.createDataFrame([[datetime.datetime.now()]]).toPandas()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/.../spark/python/pyspark/sql/dataframe.py", line 1978, in toPandas
        _check_series_convert_timestamps_local_tz(pdf[field.name], timezone)
      File "/.../spark/python/pyspark/sql/types.py", line 1775, in 
_check_series_convert_timestamps_local_tz
        return _check_series_convert_timestamps_localize(s, None, timezone)
      File "/.../spark/python/pyspark/sql/types.py", line 1750, in 
_check_series_convert_timestamps_localize
        require_minimum_pandas_version()
      File "/.../spark/python/pyspark/sql/utils.py", line 128, in 
require_minimum_pandas_version
        "your version was %s." % (minimum_pandas_version, pandas.__version__))
    ImportError: Pandas >= 0.19.2 must be installed; however, your version was 
0.16.0.
    ```
    
    Since we set the supported version, I think we should better explicitly 
require the version. Let me know if anyone thinks differently ..


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to