dongjoon-hyun edited a comment on pull request #31284:
URL: https://github.com/apache/spark/pull/31284#issuecomment-768858103
I confirmed that this bug still exists in the vectorized reader while MR
reader can read this correctly like the test Parquet file in this PR.
```scala
scala> sql("set spark.sql.parquet.enableVectorizedReader=false")
res4: org.apache.spark.sql.DataFrame = [key: string, value: string]
scala>
spark.read.parquet("/tmp/decimal32-written-as-64-bit.snappy.parquet").show
21/01/27 23:27:24 WARN Types$BasePrimitiveBuilder: Decimal with 9 digits is
stored in an INT64, but fits in an INT32. See
https://github.com/apache/parquet-format/blob/master/LogicalTypes.md.
21/01/27 23:27:24 WARN Types$BasePrimitiveBuilder: Decimal with 9 digits is
stored in an INT64, but fits in an INT32. See
https://github.com/apache/parquet-format/blob/master/LogicalTypes.md.
21/01/27 23:27:24 WARN Types$BasePrimitiveBuilder: Decimal with 9 digits is
stored in an INT64, but fits in an INT32. See
https://github.com/apache/parquet-format/blob/master/LogicalTypes.md.
+-----------+
| _c0|
+-----------+
| 79205949.2|
| 98684298.7|
| 54024799.8|
| null|
| 35799107.8|
| 49413105.9|
| 9253639.6|
| 42684715.7|
|-99999999.9|
| 20448609.4|
+-----------+
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
