lokeshj1703 commented on code in PR #12888:
URL: https://github.com/apache/hudi/pull/12888#discussion_r1976369100
##########
hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieInputFormatUtils.java:
##########
@@ -537,9 +538,13 @@ public static String getTableBasePath(InputSplit split,
JobConf jobConf) throws
/**
* `schema.on.read` and skip merge not implemented
*/
- public static boolean shouldUseFilegroupReader(final JobConf jobConf, final
InputSplit split) {
- return
jobConf.getBoolean(HoodieReaderConfig.FILE_GROUP_READER_ENABLED.key(),
HoodieReaderConfig.FILE_GROUP_READER_ENABLED.defaultValue())
- &&
!jobConf.getBoolean(HoodieCommonConfig.SCHEMA_EVOLUTION_ENABLE.key(),
HoodieCommonConfig.SCHEMA_EVOLUTION_ENABLE.defaultValue())
- && !(split instanceof BootstrapBaseFileSplit);
+ public static boolean shouldUseFilegroupReader(final JobConf jobConf, final
InputSplit split) throws IOException {
+ if (!(split instanceof FileSplit)) {
Review Comment:
https://github.com/apache/hudi/blob/d2a617f7b6c78da7614cce8db1bac7ea4e5bf7b4/hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieParquetInputFormat.java#L120-L124
The file group reader is only created when input split is of type FileSplit.
Therefore added this constraint.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
