This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.3 by this push:
new 4c09616f72d [SPARK-38085][SQL][FOLLOWUP] Do not fail too early for
DeleteFromTable
4c09616f72d is described below
commit 4c09616f72d563f6c731654c8d69a8b36e40983a
Author: Wenchen Fan <[email protected]>
AuthorDate: Mon May 2 11:28:36 2022 +0900
[SPARK-38085][SQL][FOLLOWUP] Do not fail too early for DeleteFromTable
### What changes were proposed in this pull request?
`DeleteFromTable` has been in Spark for a long time and there are existing
Spark extensions to compile `DeleteFromTable` to physical plans. However, the
new analyzer rule `RewriteDeleteFromTable` fails very early if the v2 table
does not support delete. This breaks certain Spark extensions which can still
execute `DeleteFromTable` for certain v2 tables.
This PR simply removes the error throwing in `RewriteDeleteFromTable`. It's
a safe change because:
1. the new delete-related rules only match v2 table with
`SupportsRowLevelOperations`, so won't be affected by this change
2. the planner rule will fail eventually if the v2 table doesn't support
deletion. Spark eagerly executes commands so Spark users can still see this
error immediately.
### Why are the changes needed?
To not break existing Spark extesions.
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
existing tests
Closes #36402 from cloud-fan/follow.
Lead-authored-by: Wenchen Fan <[email protected]>
Co-authored-by: Wenchen Fan <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit 5630f700768432396a948376f5b46b00d4186e1b)
Signed-off-by: Hyukjin Kwon <[email protected]>
---
.../apache/spark/sql/catalyst/analysis/RewriteDeleteFromTable.scala | 4 ----
1 file changed, 4 deletions(-)
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/RewriteDeleteFromTable.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/RewriteDeleteFromTable.scala
index 85af999902e..d473254a08f 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/RewriteDeleteFromTable.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/RewriteDeleteFromTable.scala
@@ -23,7 +23,6 @@ import
org.apache.spark.sql.catalyst.plans.logical.{DeleteFromTable, Filter, Log
import org.apache.spark.sql.connector.catalog.{SupportsDelete,
SupportsRowLevelOperations, TruncatableTable}
import org.apache.spark.sql.connector.write.RowLevelOperation.Command.DELETE
import org.apache.spark.sql.connector.write.RowLevelOperationTable
-import org.apache.spark.sql.errors.QueryCompilationErrors
import org.apache.spark.sql.execution.datasources.v2.DataSourceV2Relation
import org.apache.spark.sql.util.CaseInsensitiveStringMap
@@ -52,9 +51,6 @@ object RewriteDeleteFromTable extends RewriteRowLevelCommand {
// don't rewrite as the table supports deletes only with filters
d
- case DataSourceV2Relation(t, _, _, _, _) =>
- throw QueryCompilationErrors.tableDoesNotSupportDeletesError(t)
-
case _ =>
d
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
