HyukjinKwon edited a comment on pull request #33644: URL: https://github.com/apache/spark/pull/33644#issuecomment-894889792
I am okay with this in general. We have some places like this (launching separate a job to compute on executor) e.g., listing files or single partition when operating a global window in Spark SQL. Not sure if the configuration is the best way though. Let me add some more people who might be interested in here: @jiangxb1987 @Ngone51 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
