cloud-fan commented on issue #22149: [SPARK-25158][SQL]Executor accidentally exit because ScriptTransformationWriterThread throws TaskKilledException. URL: https://github.com/apache/spark/pull/22149#issuecomment-456086679 the fix LGTM, but can we provide a better test? IIUC the test can be: 1. create a UDF which would throw exception when producing data 2. use `spark.range(...).select(myUDF(...))` as input plan and do script transform 3. assert that the query fails, but the executor is still alive.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
