HyukjinKwon commented on pull request #28661: URL: https://github.com/apache/spark/pull/28661#issuecomment-636460089
Let me summarize a bit here. I think there's a bit of confusions here. The change here is pretty conservative: - In Spark 2.4.x and Spark 3.0, we already hid the JVM stracktrace by default in Python 2, for common exceptions thrown from JVM, for example `AnalysisException`. - This PR fixes it to match in Python 3 as well. - Additionally, I added Python UDFs to show the exceptions from Python workers without useless and same JVM stacktraces _if_ some exceptions are thrown from Python workers There's no radical change but this is already what we have done in the codebase. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
