Re: [PR] [CELEBORN-1838] Interrupt spark task should not report fetch failure [celeborn]

[via GitHub]

FMX commented on PR #3068:
URL: https://github.com/apache/celeborn/pull/3068#issuecomment-2597357344

   Thanks for your remind, there will be manual tests for this.
   
   > it is impossible to add UT for speculation.
   > 
   > 
https://github.com/apache/spark/blob/d5da49d56d7dec5f8a96c5252384d865f7efd4d9/core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala#L236-L244
   > 
   > <img alt="image" width="867" 
src="https://private-user-images.githubusercontent.com/6757692/389046303-f93bd14f-0f34-4c81-a8db-13be511405d9.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzcwODQ2MzYsIm5iZiI6MTczNzA4NDMzNiwicGF0aCI6Ii82NzU3NjkyLzM4OTA0NjMwMy1mOTNiZDE0Zi0wZjM0LTRjODEtYThkYi0xM2JlNTExNDA1ZDkucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI1MDExNyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNTAxMTdUMDMyNTM2WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9OTAwNTE1YTU5MjE0OWFiOWY4N2RhN2E5YTA3ZWNlNjA4MWE5Mzk5Y2U2OTAyZDg3Yjk0NTFjNWE2MjI1ODViNCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.xpEYhEDmAd8cPdoJBMPjodObHMyvGv45nRleyeFgcaA";>
   > For local master, it would not start the speculationScheduler.
   > 
   > 
https://github.com/apache/spark/blob/d5da49d56d7dec5f8a96c5252384d865f7efd4d9/core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala#L322-L346
   > 
   > <img alt="image" width="1010" 
src="https://private-user-images.githubusercontent.com/6757692/389127451-477729a4-2fc1-47e9-b128-522c6e2ceb48.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzcwODQ2MzYsIm5iZiI6MTczNzA4NDMzNiwicGF0aCI6Ii82NzU3NjkyLzM4OTEyNzQ1MS00Nzc3MjlhNC0yZmMxLTQ3ZTktYjEyOC01MjJjNmUyY2ViNDgucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI1MDExNyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNTAxMTdUMDMyNTM2WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YzgxMTlkMDQ3NGU4ZjFhNjU2MThhZjgwN2EzMzUwNTlkNjk1NmE1NGY3YjFhZDRmNmFkYTFhMTRkOGNiOTc1ZiZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.tR0TAxRmtyQrhuTe7ZgnClkgDKgQ0NmwKNV115j62jI";>
   > and it is also not allowed to launch speculative task on the same host.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org