This is an automated email from the ASF dual-hosted git repository.
yangjie01 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 08e741b92b8f [SPARK-48604][SQL] Replace deprecated `new
ArrowType.Decimal(precision, scale)` method call
08e741b92b8f is described below
commit 08e741b92b8fc9e43c838d0849317916218414ce
Author: Wei Guo <[email protected]>
AuthorDate: Thu Jun 13 18:11:30 2024 +0800
[SPARK-48604][SQL] Replace deprecated `new ArrowType.Decimal(precision,
scale)` method call
### What changes were proposed in this pull request?
This pr replaces deprecated classes and methods of `arrow-vector` called in
Spark:
- `Decimal(int precision, int scale)` -> `Decimal(
JsonProperty("precision") int precision,
JsonProperty("scale") int scale,
JsonProperty("bitWidth") int bitWidth
)`
All `arrow-vector` related Spark classes, I made a double check, only in
`ArrowUtils` there is a deprecated method call.
### Why are the changes needed?
Clean up deprecated API usage.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Passed GA.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #46961 from wayneguow/deprecated_arrow.
Authored-by: Wei Guo <[email protected]>
Signed-off-by: yangjie01 <[email protected]>
---
sql/api/src/main/scala/org/apache/spark/sql/util/ArrowUtils.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/sql/api/src/main/scala/org/apache/spark/sql/util/ArrowUtils.scala
b/sql/api/src/main/scala/org/apache/spark/sql/util/ArrowUtils.scala
index d9bd3b0e612b..6852fe09ef96 100644
--- a/sql/api/src/main/scala/org/apache/spark/sql/util/ArrowUtils.scala
+++ b/sql/api/src/main/scala/org/apache/spark/sql/util/ArrowUtils.scala
@@ -51,7 +51,7 @@ private[sql] object ArrowUtils {
case BinaryType if !largeVarTypes => ArrowType.Binary.INSTANCE
case _: StringType if largeVarTypes => ArrowType.LargeUtf8.INSTANCE
case BinaryType if largeVarTypes => ArrowType.LargeBinary.INSTANCE
- case DecimalType.Fixed(precision, scale) => new
ArrowType.Decimal(precision, scale)
+ case DecimalType.Fixed(precision, scale) => new
ArrowType.Decimal(precision, scale, 8 * 16)
case DateType => new ArrowType.Date(DateUnit.DAY)
case TimestampType if timeZoneId == null =>
throw SparkException.internalError("Missing timezoneId where it is
mandatory.")
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
