ajantha-bhat commented on code in PR #8246:
URL: https://github.com/apache/iceberg/pull/8246#discussion_r1285376558
##########
spark/v3.4/spark/src/test/java/org/apache/iceberg/spark/data/TestParquetAvroReader.java:
##########
@@ -84,7 +84,8 @@ public class TestParquetAvroReader {
optional(22, "jumpy", Types.DoubleType.get()),
required(23, "koala", Types.TimeType.get()),
required(24, "couch rope", Types.IntegerType.get())))),
- optional(2, "slide", Types.StringType.get()));
+ optional(2, "slide", Types.StringType.get()),
+ optional(25, "foo", Types.DecimalType.of(7, 5)));
Review Comment:
Even though I fixed the actual issue,
the test case passes the read part and fails for decimal type for Equals()
method.
Line 220:
`Assert.assertEquals("Record " + recordNum + " should match expected",
expected, actual);`
Looks like Avro genericData doesn't have Decimal type. Was the data suppose
to be Fixed instead of Decimal?
```
Unknown datum type java.math.BigDecimal: 0.00040
org.apache.avro.AvroRuntimeException: Unknown datum type
java.math.BigDecimal: 0.00040
at
org.apache.avro.generic.GenericData.getSchemaName(GenericData.java:933)
at
org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:892)
at org.apache.avro.generic.GenericData.compare(GenericData.java:1178)
at org.apache.avro.generic.GenericData.compare(GenericData.java:1154)
at
org.apache.avro.generic.GenericData$Record.equals(GenericData.java:287)
at org.junit.Assert.isEquals(Assert.java:133)
at org.junit.Assert.equalsRegardingNull(Assert.java:129)
at org.junit.Assert.assertEquals(Assert.java:112)
at
org.apache.iceberg.spark.data.TestParquetAvroReader.testCorrectness(TestParquetAvroReader.java:220)
```
Need to dig deeper on this.
But @Fokko, do you know about this? (tagged you since you are involved in
all 3 of the project - avro, parquet, iceberg)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
