xiarixiaoyao commented on pull request #3203:
URL: https://github.com/apache/hudi/pull/3203#issuecomment-940777833
@danny0405 @nsivabalan @leesf
update the code. addressed all comments。
1. recover HUDI-1969, and remove RealTimeMergedRecordReader.java which is no
needed.
2. add more docs on
HoodieParquetRealtimeInputFormat.listStatusForIncrementalMode and split this
function.
3. keep PathWithLogFileStatus unSplitable. just like spark
MergeOnReadIncrementalRelation. now both hbase index and flink write table will
be ok.
last things: now we have no way to distinguish between the newly added data
and the updated data in a log file. we should make the split unSplitable(just
like the logical MergeOnReadIncrementalRelation)
if we can find a way to distinguish added data and updated data, we can
remove this limited.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
