wwli05 opened a new issue, #7116: URL: https://github.com/apache/hudi/issues/7116
**Describe the problem you faced** we are using flink compaction async. sometimes, the same file group compact task will be distributed to different task manager even though the compaction plan distribute the compact task using keyby filegroupid(hudi-4397), this will lead to the exception: The file being written is in an invalid state. Probably caused by an error thrown previously. Current state: COLUMN **To Reproduce** Steps to reproduce the behavior: 1. job manager is killed and restart 2.all task managers register again 3.say task manager1 will be assigned different file group id compact task, which maybe distribute to task manager2 before task manager re-register i noticed that, when task manager re-registered, the compact task will continue to run which started before this re-register happened **Environment Description** * Hudi version : 0.11.0 + hudi-4397 * flink version : 1.14.4 * Running on Docker? (yes/no) : yes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
