Re: [PR] [CELEBORN-1856]Support stage-rerun when read partition by chunkOffsets when enable optimize skew partition read [celeborn]

Thu, 20 Mar 2025 00:10:48 -0700 


Z1Wu commented on PR #3118:
URL: https://github.com/apache/celeborn/pull/3118#issuecomment-2739415076

   I also do some simple tests in a mini-cluster, and tests work as expected. 
Looking forward to this feature being merged into master.
   
   - skew optimization occurs 
   
![image](https://github.com/user-attachments/assets/ef7b2475-541a-4f68-800b-03f76871dd23)
   
   
   - delete shuffle data manually to trigger fetch fail 
   
![image](https://github.com/user-attachments/assets/da77dfbe-82bf-42ca-80d2-893f0a068763)
   
   - rerun all task in shuffle map stages 
   
![image](https://github.com/user-attachments/assets/dc3cdb0d-e948-4878-ad85-e536f2a089d1)
   
   - ** rerun all tasks in failed stage ** ( work as expecteed)
   
![image](https://github.com/user-attachments/assets/9e646e03-df0e-4969-b011-cc67a922f8cf)
   
   - not sort happens during skew-handling 
   
![image](https://github.com/user-attachments/assets/2565faee-8337-4551-bccb-5d737b014293)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to