Beutlin opened a new issue, #18098:
URL: https://github.com/apache/iceberg/issues/18098

   ### Apache Iceberg version
   1.11.0 (iceberg-flink-runtime-2.0)
   
   ### Query engine
   Flink
   
   ### Please describe the bug
   
   After a **stateless restart** (no checkpoint/savepoint restored, 
`isRestored() == false`) of a job using the modern `IcebergSink` (not legacy 
`FlinkSink`), the job keeps consuming/processing records and checkpoints 
succeed, but **no new Iceberg snapshots are ever committed again**. 
Data/manifest files keep being written to storage, but snapshot history stays 
frozen. No errors or warnings — looks like a broken deserializer, except files 
are physically written, just never committed.
   
   ### Root cause
   
   During commit, `SinkUtil.getMaxCommittedCheckpointId(table, jobId, 
operatorId, branch)` walks the snapshot chain backwards for a snapshot with a 
matching `(job-id, operator-id)`, and returns its 
`flink.max-committed-checkpoint-id`. Since `job-id`/`operator-id` stay stable 
across a stateless restart, it finds a snapshot from the **previous run** and 
returns its old, high checkpoint id (e.g. `13304`). The new run's checkpoint 
counter restarts at `1`, so every new commit's checkpoint id is lower than the 
"already committed" value and gets **silently discarded** — indefinitely, until 
the counter organically exceeds the old value.
   
   The legacy `FlinkFilesCommitter`/`FlinkSink` guards this exact lookup with 
`isRestored()` and skips it on a stateless start. `IcebergSink`'s commit path 
appears to run the lookup unconditionally.
   
   ### How to reproduce
   
   1. Run a job with `IcebergSink.forRowData()...uidSuffix(...)`, checkpointing 
enabled, long enough to accumulate several commits.
   2. Restart the job **without** restoring state (fresh stateless start, same 
job-id/operator-id — e.g. same deployment restarted from scratch).
   3. Consumer lag stays 0, checkpoints complete, but no new snapshots appear.
   
   ### Environment note
   
   Observed on Amazon Managed Service for Apache Flink (MSF), which triggered 
the stateless restart. Root cause is in Iceberg's own connector code 
(`org.apache.iceberg.flink.sink`), independent of runtime — should reproduce on 
any Flink cluster.
   
   ### Workaround
   
   Reverting to the deprecated `FlinkSink` API (`uidPrefix` instead of 
`uidSuffix`) avoids the issue, since its `getMaxCommittedCheckpointId` call is 
guarded by `isRestored()`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to