Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/18754#discussion_r158422348
--- Diff: python/pyspark/sql/types.py ---
@@ -1617,7 +1617,7 @@ def to_arrow_type(dt):
elif type(dt) == DoubleType:
arrow_type = pa.float64()
elif type(dt) == DecimalType:
- arrow_type = pa.decimal(dt.precision, dt.scale)
+ arrow_type = pa.decimal128(dt.precision, dt.scale)
--- End diff --
@wesm @BryanCutler Is this a right way to define decimal type for Arrow?
I also wonder if there is a limit for precision and scale?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
