voonhous commented on code in PR #19458:
URL: https://github.com/apache/hudi/pull/19458#discussion_r3701557389
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieBaseRelation.scala:
##########
@@ -81,7 +80,6 @@ case class HoodieTableState(tablePath: String,
recordKeyField: String,
orderingFields: List[String],
usesVirtualKeys: Boolean,
Review Comment:
Verified: after this PR `latestCommitTimestamp` is the only field with any
reader (2 sites in `HoodieMergeOnReadRDDV2.compute`). The other seven are
pre-existing rot -- the reads died in #17457 / #13544 / #12981, and
`usesVirtualKeys` was never read at all.
Keeping them out of this PR deliberately; filed #19465 to collapse
`HoodieTableState` into a plain `latestCommitTimestamp` parameter on
`HoodieMergeOnReadRDDV2` (a 1-field case class is not worth keeping).
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieHadoopFsRelationFactory.scala:
##########
@@ -335,7 +335,7 @@ class
HoodieMergeOnReadIncrementalHadoopFsRelationFactoryV2(override val sqlCont
isBootstrap:
Boolean,
rangeType:
RangeType = RangeType.OPEN_CLOSED)
extends HoodieMergeOnReadIncrementalHadoopFsRelationFactory(sqlContext,
metaClient, options, schemaSpec, isBootstrap,
- MergeOnReadIncrementalRelationV2(sqlContext, options, metaClient,
schemaSpec, None, rangeType))
+ MergeOnReadIncrementalRelationV2(sqlContext, options, metaClient,
schemaSpec, rangeType))
Review Comment:
Good catch. The description now enumerates all three producer classes:
- metadata-table reads + the schema-on-read branch: the only ones that reach
a `LogicalRelation`
- incremental factories (MOR and COW, non-metadata): build
`MergeOnReadIncrementalRelationV1/V2` internally, consumed only via
`listFileSplits` / `getRequiredFilters`
- streaming sources with the file group reader disabled: consumed via
`buildScan`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
