SEZ9 commented on issue #12972: URL: https://github.com/apache/seatunnel/issues/12972#issuecomment-6393660062
Thanks for the detailed report and the clear reproduction steps. This does look like a real gap: since Hazelcast does not delete expired entries from the MapStore and `FileMapStore` does not persist the expiration time, finished jobs reloaded after a full master restart come back without a TTL and never expire. Since you are willing to submit a PR, please go ahead. A couple of directions worth considering in the PR: persisting the finish/expire time alongside each finished job entry so already-expired entries can be skipped on reload and the remaining TTL re-applied, and/or a periodic cleanup on the master that removes entries older than `history-job-expire-minutes` from the `engine_finishedJob*` maps and the MapStore. One question to help scope the fix: have you observed this with the other MapStore backends as well, or only with the local file backend? <!-- streview-comment:2026 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
