danny0405 commented on code in PR #12385:
URL: https://github.com/apache/hudi/pull/12385#discussion_r1864743337
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadIncrementalRelation.scala:
##########
@@ -209,7 +209,14 @@ trait HoodieIncrementalRelationTrait extends
HoodieBaseRelation {
protected lazy val includedCommits: immutable.Seq[HoodieInstant] =
queryContext.getInstants.asScala.toList
- protected lazy val commitsMetadata =
includedCommits.map(getCommitMetadata(_, super.timeline)).asJava
+ protected lazy val commitsMetadata = includedCommits.map(
+ i => {
Review Comment:
Let's check why we need to resolve the commit metadata for an archived
instant, even in incremental query scenarios, the reader should fallback to
snapshot query with specific filters instead of resolving the data files from
commit metadata (because the data files are very probably been cleaned.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
