blackcheckren commented on issue #7216: URL: https://github.com/apache/hudi/issues/7216#issuecomment-1514061766
I encountered the same problem, Spark Structrued streaming program would appear about every 4~10 days, which caused me to monitor the program running every day, and once it happened, I had to abort the task and delete the driver pod because the program was running on k8s. Because this program is incremental data write data lake, only this task is running on the table, therefore, I did not configure concurrency control. Check the official hudi documentation to configure concurrency control when multiple tasks are written to the same table. Have you solved this problem by configuring occ? @wqwl611 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
