This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new c045a425bf0c [SPARK-46043][SQL][FOLLOWUP] Remove the catalog and
identifier check in DataSourceV2Relation
c045a425bf0c is described below
commit c045a425bf0c472f164e3ef75a8a2c68d72d61d3
Author: allisonwang-db <[email protected]>
AuthorDate: Thu Dec 14 10:39:01 2023 -0800
[SPARK-46043][SQL][FOLLOWUP] Remove the catalog and identifier check in
DataSourceV2Relation
### What changes were proposed in this pull request?
https://github.com/apache/spark/pull/43949 added a check in the `name`
method of `DataSourceV2Relation`, which can be overly strict. This PR removes
the check and revert to use `table.name()` when either catalog or identifier is
empty.
### Why are the changes needed?
To reduce the chance of having breaking changes.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Existing unit tests.
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #44348 from allisonwang-db/spark-46043-followup.
Authored-by: allisonwang-db <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
---
.../spark/sql/execution/datasources/v2/DataSourceV2Relation.scala | 8 +-------
1 file changed, 1 insertion(+), 7 deletions(-)
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala
index 8dae9904bc82..9c7d776edc65 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala
@@ -66,13 +66,7 @@ case class DataSourceV2Relation(
import org.apache.spark.sql.connector.catalog.CatalogV2Implicits._
(catalog, identifier) match {
case (Some(cat), Some(ident)) =>
s"${quoteIfNeeded(cat.name())}.${ident.quoted}"
- case (None, None) => table.name()
- case _ =>
- throw SparkException.internalError(
- "Invalid catalog and identifier pair. Both 'catalog' and
'identifier' must be " +
- s"specified or leave as None. Current input - " +
- s"catalog: '${catalog.map(_.name()).getOrElse(None)}', " +
- s"identifier: ${identifier.map(_.quoted).getOrElse(None)}.")
+ case _ => table.name()
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
