baohe-zhang edited a comment on pull request #31945: URL: https://github.com/apache/spark/pull/31945#issuecomment-807693044
I think we can create a new ticket to work on reducing the chance of the stat file not found, such as applying the methods proposed by @dhruve. And I think the goal of this PR is to make the behavior consistent (either returning all-0 metrics or omitting the missing stat files, but not returning the sum of a subset of existent stat files) when the stat file not found exception occurs. What's your opinions? @attilapiros @dongjoon-hyun -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
