hudi-agent commented on code in PR #18865:
URL: https://github.com/apache/hudi/pull/18865#discussion_r3431036853
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestRecordLevelIndex.scala:
##########
@@ -167,11 +169,19 @@ class TestRecordLevelIndex extends
RecordLevelIndexTestBase with SparkDatasetMix
.mode(SaveMode.Overwrite)
.save(basePath)
assertEquals(10, spark.read.format("hudi").load(basePath).count())
+ if (rliInitDeferred) {
+ // With defer enabled, the first commit should NOT have initialized the
RLI partition.
+ metaClient = HoodieTableMetaClient.reload(metaClient)
+
assertFalse(metaClient.getTableConfig.getMetadataPartitions.contains(MetadataPartitionType.RECORD_INDEX.getPartitionPath),
+ "RLI partition should not be initialized after the first commit when
defer is enabled")
+ }
val props =
TypedProperties.fromMap(JavaConverters.mapAsJavaMapConverter(options).asJava)
val writeConfig = HoodieWriteConfig.newBuilder()
.withProps(props)
.withPath(basePath)
.build()
+ // Constructing the metadata writer here will initialize RLI (lazily, on
this second metadata-writer entry)
Review Comment:
🤖 nit: "second metadata-writer entry" is a bit opaque — could you rephrase
to something like "second call to `metadataWriter()`" or "second invocation" so
it's immediately clear what "entry" refers to?
<sub><i>- AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestRecordLevelIndex.scala:
##########
@@ -301,6 +311,108 @@ class TestRecordLevelIndex extends
RecordLevelIndexTestBase with SparkDatasetMix
}
}
+ @ParameterizedTest
+ @ValueSource(booleans = Array(true, false))
+ def testPartitionedRecordLevelIndexDefer(streamingWriteEnabled: Boolean):
Unit = {
Review Comment:
🤖 nit: could you use `HoodieTimeline.DELTA_COMMIT_ACTION` and
`HoodieTimeline.COMPACTION_ACTION` here (and on line 325) instead of the bare
string literals? Ties the assertions to the canonical constants and avoids
silent failures if the name ever changes.
<sub><i>- AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
