HyukjinKwon edited a comment on issue #25545: [SPARK-28843][PYTHON] Set OMP_NUM_THREADS to executor cores for python URL: https://github.com/apache/spark/pull/25545#issuecomment-524679889 It is possible that one task is running and less cores are used, say, 1 process and 8 or 16 threads (this happens time to time when, for instance, people want to read CSV with multi-line mode, and apply Python UDF or RDD API).
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
