This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new 6a7aa0d [SPARK-30563][SQL] Disable using commit coordinator with
NoopDataSource
6a7aa0d is described below
commit 6a7aa0d1666aa57213ef16cc858dbf105f0810f4
Author: Peter Toth <[email protected]>
AuthorDate: Fri Mar 6 10:30:59 2020 +0900
[SPARK-30563][SQL] Disable using commit coordinator with NoopDataSource
### What changes were proposed in this pull request?
This PR disables using commit coordinator with `NoopDataSource`.
### Why are the changes needed?
No need for a coordinator in benchmarks.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Existing UTs.
Closes #27791 from peter-toth/SPARK-30563-disalbe-commit-coordinator.
Authored-by: Peter Toth <[email protected]>
Signed-off-by: HyukjinKwon <[email protected]>
(cherry picked from commit 72b52a3cdf2b8661c0f82db645e805ff6b085dfc)
Signed-off-by: HyukjinKwon <[email protected]>
---
.../org/apache/spark/sql/execution/datasources/noop/NoopDataSource.scala | 1 +
1 file changed, 1 insertion(+)
diff --git
a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/noop/NoopDataSource.scala
b/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/noop/NoopDataSource.scala
index 4fad0a2..851cc51 100644
---
a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/noop/NoopDataSource.scala
+++
b/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/noop/NoopDataSource.scala
@@ -61,6 +61,7 @@ private[noop] object NoopWriteBuilder extends WriteBuilder
with SupportsTruncate
private[noop] object NoopBatchWrite extends BatchWrite {
override def createBatchWriterFactory(info: PhysicalWriteInfo):
DataWriterFactory =
NoopWriterFactory
+ override def useCommitCoordinator(): Boolean = false
override def commit(messages: Array[WriterCommitMessage]): Unit = {}
override def abort(messages: Array[WriterCommitMessage]): Unit = {}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
