This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 2daa66ec7856 [SPARK-45638][SQL][AVRO] Improve test coverage of decimal
conversion
2daa66ec7856 is described below
commit 2daa66ec785616eae2010f0ad80e569bdecdb2ea
Author: Yanan Xin <[email protected]>
AuthorDate: Tue Oct 24 12:48:01 2023 +0800
[SPARK-45638][SQL][AVRO] Improve test coverage of decimal conversion
### What changes were proposed in this pull request?
-Improve test coverage of decimal conversion
### Why are the changes needed?
Test improvement
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Unit test
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #43491 from yananxin-db/avro_decimal_type.
Authored-by: Yanan Xin <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
---
.../src/test/scala/org/apache/spark/sql/avro/AvroSuite.scala | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git
a/connector/avro/src/test/scala/org/apache/spark/sql/avro/AvroSuite.scala
b/connector/avro/src/test/scala/org/apache/spark/sql/avro/AvroSuite.scala
index b22bd74e87d5..a08452c88308 100644
--- a/connector/avro/src/test/scala/org/apache/spark/sql/avro/AvroSuite.scala
+++ b/connector/avro/src/test/scala/org/apache/spark/sql/avro/AvroSuite.scala
@@ -815,6 +815,16 @@ abstract class AvroSuite
}
}
+ test("SPARK-45638: Avro should read decimal values with the file schema,
same scale") {
+ // write schema has precision and scale as 3
+ // read schema has precision as 4 and scale as 3
+ withTempPath { path =>
+ sql("SELECT 0.314 a").write.format("avro").save(path.toString)
+ val data = spark.read.schema("a DECIMAL(4,
3)").format("avro").load(path.toString).collect()
+ assert(data.map(_(0)).contains(new java.math.BigDecimal("0.314")))
+ }
+ }
+
test("SPARK-43380: Fix Avro data type conversion" +
" of decimal type to avoid producing incorrect results") {
withTempPath { path =>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
