nsivabalan commented on code in PR #6266:
URL: https://github.com/apache/hudi/pull/6266#discussion_r939116536
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/MarkerBasedRollbackUtils.java:
##########
@@ -54,25 +64,35 @@ public static List<String> getAllMarkerPaths(HoodieTable
table, HoodieEngineCont
String instant, int
parallelism) throws IOException {
String markerDir = table.getMetaClient().getMarkerFolderPath(instant);
FileSystem fileSystem = table.getMetaClient().getFs();
- Option<MarkerType> markerTypeOption =
MarkerUtils.readMarkerType(fileSystem, markerDir);
+ Option<MarkerType> markerTypeOption = readMarkerType(fileSystem,
markerDir);
- // If there is no marker type file "MARKERS.type", we assume "DIRECT"
markers are used
+ // If there is no marker type file "MARKERS.type", first assume "DIRECT"
markers are used.
+ // If not, then fallback to "TIMELINE_SERVER_BASED" markers.
Review Comment:
there is one flow where we could end up in this situation, but I could not
piece end to end fully together.
Lets say, rollback was triggered using timeline based markers. Towards the
end, when everything is complete, we go ahead and delete the marker directory.
During this, lets say marker type file was deleted, but before deleting all
markers, process crashed. We might end up in this state. But from looking at
the code, rollback is actually complete by now (i.e. original commit files
would have been deleted, and infact rollback commit would also be completed by
now. So, a failure during deletion of marker directory should not have any
impact. rollback will never be re-triggered for this case.
Let me know if you guys can think of a scenario where this could happen.
would really love to find the root cause.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
