lirui-apache commented on a change in pull request #11327: [FLINK-16450][hive]
Integrate parquet columnar row reader to hive
URL: https://github.com/apache/flink/pull/11327#discussion_r388874283
##########
File path:
flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/connectors/hive/read/HiveTableInputFormat.java
##########
@@ -113,16 +114,74 @@ public void
configure(org.apache.flink.configuration.Configuration parameters) {
@Override
public void open(HiveTableInputSplit split) throws IOException {
- if (!useMapRedReader &&
useOrcVectorizedRead(split.getHiveTablePartition())) {
+ HiveTablePartition partition = split.getHiveTablePartition();
+ if (!useMapRedReader && useOrcVectorizedRead(partition)) {
this.reader = new HiveVectorizedOrcSplitReader(
hiveVersion, jobConf, fieldNames,
fieldTypes, selectedFields, split);
+ } else if (!useMapRedReader &&
useParquetVectorizedRead(partition)) {
+ this.reader = new HiveVectorizedParquetSplitReader(
+ hiveVersion, jobConf, fieldNames,
fieldTypes, selectedFields, split);
} else {
this.reader = new HiveMapredSplitReader(jobConf,
partitionKeys, fieldTypes, selectedFields, split,
HiveShimLoader.loadHiveShim(hiveVersion));
}
currentReadCount = 0L;
}
+ private boolean isVectorizationSupport(LogicalType t) {
Review comment:
I think this method actually returns whether a type is **not** supported?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
