Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/20678#discussion_r172751054
--- Diff: docs/sql-programming-guide.md ---
@@ -1689,6 +1689,10 @@ using the call `toPandas()` and when creating a
Spark DataFrame from a Pandas Da
`createDataFrame(pandas_df)`. To use Arrow when executing these calls,
users need to first set
the Spark configuration 'spark.sql.execution.arrow.enabled' to 'true'.
This is disabled by default.
+In addition, optimizations enabled by 'spark.sql.execution.arrow.enabled'
could fallback automatically
+to non-optimized implementations if an error occurs before the actual
computation within Spark.
--- End diff --
very minor nit: `non-optimized implementations` --> `non-Arrow optimization
implementation`
this matches the description in the paragraph below
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
