Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20710#discussion_r171993101
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/WriteToDataSourceV2.scala
---
@@ -198,7 +201,7 @@ object DataWritingSparkTask extends Logging {
})(catchBlock = {
// If there is an error, abort this writer
logError(s"Writer for partition ${context.partitionId()} is
aborting.")
- dataWriter.abort()
+ if (dataWriter != null) dataWriter.abort()
logError(s"Writer for partition ${context.partitionId()} aborted.")
--- End diff --
nit: add comment that the exception will be rethrown.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
