[GitHub] [kylin] zzcclp commented on pull request #1507: KYLIN-4841 Spark RDD cache is invalid when building with spark engine

Mon, 14 Dec 2020 00:41:08 -0800 


zzcclp commented on pull request #1507:
URL: https://github.com/apache/kylin/pull/1507#issuecomment-744278051


   ## Test Result
   
   ### Before this patch
   
![image](https://user-images.githubusercontent.com/9430290/102057877-99368380-3e29-11eb-9c4d-f416a853b72c.png)
   
   As shown as above, every 'flatMapToPair' operation executes 'shuffle read'.
   
   
   ### After this patch
   
![image](https://user-images.githubusercontent.com/9430290/102057882-9cca0a80-3e29-11eb-9a25-4955192daaf4.png)
   
   After this patch, the time of every 'flatMapToPair' operation took is 
smaller than before, and it directly reads cached data and doesn't do 'shuffle 
read'.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to