Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/18754#discussion_r158422348
--- Diff: python/pyspark/sql/types.py ---
@@ -1617,7 +1617,7 @@ def to_arrow_type(dt):
elif type(dt) == DoubleType:
arrow_type =
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/18754#discussion_r139872489
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowWriter.scala
---
@@ -224,6 +226,25 @@ private[arrow] class DoubleWriter(val
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18754#discussion_r139835312
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowWriter.scala
---
@@ -224,6 +226,25 @@ private[arrow] class
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18754#discussion_r139777828
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowWriter.scala
---
@@ -224,6 +226,25 @@ private[arrow] class
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18754#discussion_r139762422
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowWriter.scala
---
@@ -224,6 +226,25 @@ private[arrow] class
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/18754#discussion_r139612110
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowWriter.scala
---
@@ -224,6 +226,25 @@ private[arrow] class DoubleWriter(val
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18754#discussion_r139562172
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowWriter.scala
---
@@ -224,6 +226,25 @@ private[arrow] class
GitHub user ueshin opened a pull request:
https://github.com/apache/spark/pull/18754
[WIP][SPARK-21552][SQL] Add DecimalType support to ArrowWriter.
## What changes were proposed in this pull request?
Decimal type is not yet supported in `ArrowWriter`.
This is adding
