satishkotha commented on a change in pull request #1817:
URL: https://github.com/apache/hudi/pull/1817#discussion_r465908401
##########
File path:
hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/realtime/HoodieParquetRealtimeInputFormat.java
##########
@@ -165,11 +261,15 @@ private static void
cleanProjectionColumnIds(Configuration conf) {
LOG.info("Creating record reader with readCols :" +
jobConf.get(ColumnProjectionUtils.READ_COLUMN_NAMES_CONF_STR)
+ ", Ids :" +
jobConf.get(ColumnProjectionUtils.READ_COLUMN_IDS_CONF_STR));
// sanity check
- ValidationUtils.checkArgument(split instanceof HoodieRealtimeFileSplit,
+ ValidationUtils.checkArgument(split instanceof HoodieRealtimeFileSplit ||
split instanceof HoodieMORIncrementalFileSplit,
Review comment:
High level question, is it possible to make baseFile optional in
HoodieRealtimeFileSplit instead of creating new class
HoodieMORIncrementalFileSplit? We may also have to make changes in RecordReader
classes if baseFile is not present.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
