rangareddy commented on issue #14537: URL: https://github.com/apache/hudi/issues/14537#issuecomment-5032430103
This issue was reviewed as part of the JIRA-migrated backlog triage. Findings: the core capability requested here has landed in code. PR #3033 (HUDI-1969, Hudi 0.9.0) added realtime-view reading of log-only file groups, and on current master HoodieMergeOnReadTableInputFormat#createFileStatusUnchecked creates a RealtimeFileStatus from the latest log file when a file slice has no base file, with TestHoodieRealtimeRecordReader#testLogOnlyReader covering a log-only slice read; since the Feb-2022 refactor (PR #4667/#4743) listing goes through the Hudi file-system view rather than Hive's default path filter, so dot-prefixed log files are no longer dropped, and Hudi 1.0's file-group-reader-based Hive record reader also handles log-only slices. The _ro view intentionally excludes log-only file groups (read-optimized semantics; HoodieCopyOnWriteTableInputFormat#checkIfValidFileSlice, PR #6254). However, JIRA HUDI-2762 remains Reopened, a maintainer reported it still reproducible in the Kafka Connect flow in Apr 2022, and no PR references this ticket, so the original Kafka Connect scenario has not been re-validated end to end. Keeping this open pending validation of the Kafka Connect + Hive sync path on a recent release (0.14.x/1.x); if someone can confirm _rt returns rows for insert-only log file groups there, this can be closed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
