Re: [I] [SUPPORT] After enable speculation execution of spark compaction job, some broken parquet files might be generated [hudi]

Wed, 23 Oct 2024 23:58:56 -0700 


zhangyue19921010 commented on issue #9615:
URL: https://github.com/apache/hudi/issues/9615#issuecomment-2434370942

   > @KnightChess We solve the problem by the following update. <img 
alt="image" width="2000" 
src="https://private-user-images.githubusercontent.com/1525333/265580015-05483dc8-b6dd-45d0-a93c-6d6e341f8767.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mjk3NDc2NTIsIm5iZiI6MTcyOTc0NzM1MiwicGF0aCI6Ii8xNTI1MzMzLzI2NTU4MDAxNS0wNTQ4M2RjOC1iNmRkLTQ1ZDAtYTkzYy02ZDZlMzQxZjg3NjcucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MTAyNCUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDEwMjRUMDUyMjMyWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YjY0MDEzZTBlMDE3OWQwMjEzYTYzMjMxMzI5ZGUwNGEyNTZkNDZhZjQ2YTRhZWEzZTk0NDdkMzJkMzc0NTI3MSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.P2kEs03O7wv55fp2ESxaQi5_EqRP1BUo39g4ma-XRl0";>
   > 
   > It could work, but not perfect. Could you share your solution?
   
   As a temp solution, this logic need also take file status modify time into 
account, we only pick up the fileslice with smaller modify time.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to