nsivabalan commented on code in PR #12746:
URL: https://github.com/apache/hudi/pull/12746#discussion_r1936593279
##########
hudi-hadoop-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFormatWriter.java:
##########
@@ -102,8 +102,11 @@ private FSDataOutputStream getOutputStream() throws
IOException {
boolean created = false;
while (!created) {
try {
+ if (storage.exists(logFile.getPath())) {
Review Comment:
illustration
C1:
basefile1 with commit time 001.
C2:
lf1 (.d4c17444-7d2c-4608-8ad0-fd20efedf1c8-0_001.log.1_1-0-1)
C3:
supposed to add lf2 with name
(.d4c17444-7d2c-4608-8ad0-fd20efedf1c8-0_001.log.2_1-0-1) // check for version
no.
, but when we reach L105 above, log file name refers to the latest known log
file. and so it refers to
.d4c17444-7d2c-4608-8ad0-fd20efedf1c8-0_001.log.1_1-0-1. We go ahead and try
creating the file (which creates the marker),
but then log file creation fails.
and we rollover and then end up creating marker and actual data file for
.d4c17444-7d2c-4608-8ad0-fd20efedf1c8-0_001.log.2_1-0-1
When we reach marker reconciliation step for this dc 003, we deduce that log
file w/ version 2 is spurious and delete it. and commit 003.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
