[GitHub] [spark] jackylee-ch commented on pull request #37601: [SPARK-40150][SQL] Merging file partition dynamically

Mon, 22 Aug 2022 05:43:37 -0700 


jackylee-ch commented on PR #37601:
URL: https://github.com/apache/spark/pull/37601#issuecomment-1222306953

   Here is the performance benefits of this pr. After this PR, there will be 
proper file partitions and less speculation tasks.
   Before this PR:
   <img width="1409" alt="6b41df429d8112ec24c886e1064b73c1" 
src="https://user-images.githubusercontent.com/10897625/185923110-593c0af8-31a3-4f92-8f08-ed6923805c6a.png";>
   After this PR:
   <img width="1409" alt="93ce9aa8cbe6066640d84853e1d584a1" 
src="https://user-images.githubusercontent.com/10897625/185923064-8e429b29-1f32-4359-a263-3903271cd444.png";>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to