wombatu-kun commented on code in PR #19444:
URL: https://github.com/apache/hudi/pull/19444#discussion_r3701636507
##########
hudi-client/hudi-java-client/src/test/java/org/apache/hudi/client/TestHoodieJavaWriteClientInsert.java:
##########
@@ -140,6 +144,61 @@ public void
testWriteClientAndTableServiceClientWithTimelineServer(
writeClient.close();
}
+ /**
+ * HUDI-5011: exercises a Java-engine write config against both marker types
with the embedded timeline
+ * server. {@code HoodieWriteConfig.Builder} defaults {@link
MarkerType#DIRECT} for
+ * {@link EngineType#JAVA}, so the timeline-server-based path is only
reached when
+ * {@code hoodie.write.markers.type} is set explicitly. Note the default
applies to the config's engine
+ * type, not to the client: {@code HoodieJavaWriteClient} never inspects it,
so a config left on the
+ * builder's SPARK default would resolve to TIMELINE_SERVER_BASED on its own.
+ *
+ * <p>Backup for the remote file system view is disabled so that a
timeline-server failure fails the test
+ * rather than silently falling back to a local view, matching
+ * {@code HoodieJavaClientTestHarness#getConfigBuilder}.
+ */
+ @ParameterizedTest
+ @EnumSource(MarkerType.class)
+ public void testInsertWithEmbeddedTimelineServerAndMarkerType(MarkerType
markerType) throws Exception {
+ HoodieWriteConfig config = makeHoodieClientConfigBuilder(basePath)
+ .withMarkersType(markerType.name())
+ .withFileSystemViewConfig(FileSystemViewStorageConfig.newBuilder()
+ .withEnableBackupForRemoteFileSystemView(false).build())
+ .build();
+
+ HoodieJavaWriteClient writeClient = getHoodieWriteClient(config);
+ assertTrue(writeClient.getTimelineServer().isPresent(),
+ "The embedded timeline server should be running for marker type " +
markerType);
+
+ List<HoodieRecord> records = new ArrayList<>();
+ records.add(createSimpleRecord("1", "2021-09-11T16:16:41.415Z", 1));
+ records.add(createSimpleRecord("2", "2021-09-11T16:16:41.415Z", 2));
+
+ String commitTime = makeNewCommitTime(1, "%09d");
+ WriteClientTestUtils.startCommitWithTime(writeClient, commitTime);
+ List<WriteStatus> statuses = writeClient.insert(records, commitTime);
+
+ // Inspect the markers before commit removes them. Only the
timeline-server path writes MARKERS.type,
+ // so this is what would notice a silent fallback to DirectWriteMarkers.
+ metaClient = HoodieTableMetaClient.reload(metaClient);
+ StoragePath markerDir = new
StoragePath(metaClient.getMarkerFolderPath(commitTime));
+ boolean markerTypeFileExists =
MarkerUtils.doesMarkerTypeFileExist(metaClient.getStorage(), markerDir);
+ if (markerType == MarkerType.TIMELINE_SERVER_BASED) {
+ assertTrue(markerTypeFileExists,
+ "Timeline-server-based markers should have written " +
MarkerUtils.MARKER_TYPE_FILENAME);
+ } else {
+ assertFalse(markerTypeFileExists,
Review Comment:
The DIRECT arm asserts only that MARKERS.type is absent, and nothing writes
that file when the marker type is DIRECT, so this arm also passes if the insert
produced no markers at all. Assert the instant's marker directory is non-empty
here too, so it fails on a direct-marker path that writes nothing - follow-up,
not a blocker.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
