rdblue commented on a change in pull request #30562:
URL: https://github.com/apache/spark/pull/30562#discussion_r533717460
##########
File path:
sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/SupportsDelete.java
##########
@@ -28,6 +28,25 @@
*/
@Evolving
public interface SupportsDelete {
+
+ /**
+ * Checks whether it is possible to delete data from a data source table
that matches filter
+ * expressions.
+ * <p>
+ * Rows should be deleted from the data source iff all of the filter
expressions match.
+ * That is, the expressions must be interpreted as a set of filters that are
ANDed together.
+ * <p>
+ * Spark will call this method to check if the delete is possible without
significant effort.
+ * Otherwise, Spark will try to rewrite the delete operation and produce
row-level changes
+ * if the data source table supports deleting individual records.
+ *
+ * @param filters filter expressions, used to select rows to delete when all
expressions match
+ * @return true if the delete operation can be performed
+ */
+ default boolean canDeleteWhere(Filter[] filters) {
+ return true;
Review comment:
Unfortunately, this would change the assumptions for existing
implementations. Right now, if this interface is implemented, Spark will call
`deleteWhere` for the delete. Returning `false` would cause Spark to skip it
where the return of this method is used.
The original idea was to try to delete using `deleteWhere`, and if that
fails to run a more expensive delete. But when we started implementing the more
expensive delete, we needed to know during job planning, not job execution,
whether the metadata-only delete can be done. This method solves that problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
