Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20830#discussion_r174857693
--- Diff: python/pyspark/sql/tests.py ---
@@ -3551,12 +3544,12 @@ def test_null_conversion(self):
self.assertTrue(all([c == 1 for c in null_counts]))
def _toPandas_arrow_toggle(self, df):
- self.spark.conf.set("spark.sql.execution.arrow.enabled", "false")
- try:
+ with self.sql_conf({"spark.sql.execution.arrow.enabled": False}):
pdf = df.toPandas()
- finally:
- self.spark.conf.set("spark.sql.execution.arrow.enabled",
"true")
- pdf_arrow = df.toPandas()
+
+ with self.sql_conf({"spark.sql.execution.arrow.enabled": True}):
--- End diff --
We can omit this when we use the default value or set the value in setup
method, but I'm okay if we want to show the value explicitly.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
