itholic commented on pull request #34509:
URL: https://github.com/apache/spark/pull/34509#issuecomment-972567563
Not a big deal, but could you add a simple example to PR description what
issue is resolved with before/after ??
e.g.
Before:
```python
>>> spark.createDataFrame(pd.DataFrame({"A": ['a', 'b', 'c']},
dtype="string"))
.../spark/python/pyspark/sql/pandas/conversion.py:402: UserWarning:
createDataFrame attempted Arrow optimization because
'spark.sql.execution.arrow.pyspark.enabled' is set to true; however, failed by
the reason below:
Cannot specify a mask or a size when passing an object that is converted
with the __arrow_array__ protocol.
Attempting non-optimization as
'spark.sql.execution.arrow.pyspark.fallback.enabled' is set to true.
warnings.warn(msg)
DataFrame[A: string]
```
After:
```python
>>> spark.createDataFrame(pd.DataFrame({"A": ['a', 'b', 'c']},
dtype="string"))
DataFrame[A: string]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
