cloud-fan commented on a change in pull request #31921:
URL: https://github.com/apache/spark/pull/31921#discussion_r598473997
##########
File path:
sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedColumnReader.java
##########
@@ -565,6 +565,10 @@ private void readIntBatch(int rowId, int num,
WritableColumnVector column) throw
canReadAsIntDecimal(column.dataType())) {
defColumn.readIntegers(
num, column, rowId, maxDefLevel, (VectorizedValuesReader)
dataColumn);
+ } else if (column.dataType() == DataTypes.LongType) {
+ // We use LongType to handle UINT32
+ defColumn.readIntegersAsUnsigned(
Review comment:
can we follow
https://github.com/apache/spark/commit/38fbe560fd08168e90c575f7707368ddf758c3a9
and check if dictionary encoding also needs update?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
