wzhero1 commented on PR #7027:
URL: https://github.com/apache/paimon/pull/7027#issuecomment-4108413248
Based on actual implementation:
PR Comment work items:
1. Empty plan logging: Adjusted log message to "No snapshots to expire,
skipping Flink job submission." (still returns early, not submitting an empty
job)
2. Sink operator rename: Renamed sink operator from "SnapshotExpire" to
"SnapshotExpireCommit" to distinguish from worker phase
3. Inline executeTask: Removed single-use method, inlined task execution
directly into flatMap
4. Progress logging: Added per-task progress log ("Processing expire task
{}/{}, {}"), start/end logs with elapsed time
5. Failure recovery test: Added testExpireRecoveryAfterPartialFailure with
failure injection via volatile int failAfterTasks
6. Test class rename: ExpireSnapshotsActionITCase →
ParallelExpireSnapshotsActionITCase
7. Disabled test comments: Added @Disabled annotations with explanations
for tests not suitable for parallel mode
8. Polymorphic refactoring: Replaced TaskType enum +
ExpireSnapshotsExecutor switch dispatch with 4 polymorphic SnapshotExpireTask
subclasses, deleted ExpireSnapshotsExecutor
9. Logic migration: Moved planner/plan/task/report classes from
paimon-core to paimon-flink package
Additional changes (not from comments):
- ExpireContext simplification: Removed delegate methods, callers access
underlying objects directly (context.snapshotDeletion().xxx()). Made
taggedSnapshots and snapshotCache
final constructor parameters, only skippingSet retains a setter (mutated
during manifest deletion)
- snapshotCache in parallel mode: Propagated planner's pre-collected
snapshot cache to both workers and sink via serialization, avoiding redundant
snapshot file reads
- Trigger condition: Changed to forceStartFlinkJob || parallelism > 1 —
either condition enters Flink parallel pipeline
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
