dongjoon-hyun commented on a change in pull request #31319:
URL: https://github.com/apache/spark/pull/31319#discussion_r564801959
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetRowConverter.scala
##########
@@ -242,6 +242,19 @@ private[parquet] class ParquetRowConverter(
}
}
+ /**
+ * Get a precision and a scale to interpret parquet decimal values.
+ * 1. If there is a decimal metadata, we read decimal values with the given
precision and scale.
+ * 2. If there is no metadata, we read decimal values with scale `0` because
it's plain integers
+ * when it is written into INT32/INT64/BINARY/FIXED_LEN_BYTE_ARRAY types.
Review comment:
This is the new logic, @cloud-fan .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
