This is an automated email from the ASF dual-hosted git repository.
cloud-fan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 90d1fbecbdbd [SPARK-54710][SQL] Show Partitions As Json
90d1fbecbdbd is described below
commit 90d1fbecbdbdfe671eee6de6a96c67a65c064d51
Author: ashutosh-jindal <[email protected]>
AuthorDate: Fri Jun 19 17:14:48 2026 -0700
[SPARK-54710][SQL] Show Partitions As Json
### What changes were proposed in this pull request?
Support SHOW PARTITIONS ... [AS JSON] to display table metadata in JSON
format.
SQL Ref Spec:
```
SHOW PARTITIONS table_name [ PARTITION clause ] [ AS JSON ]
```
Output:
json_metadata: String
### Why are the changes needed?
The Spark SQL command `SHOW PARTITIONS` displays partitions list meant more
for human consumption. With the introduction of the `AS JSON` option, the
command now returns table metadata as a JSON string suitable for machine
consumption.
### Does this PR introduce _any_ user-facing change?
Yes, this provides a new option to display `SHOW PARTITIONS` metadata in
JSON format. See below for the JSON output schema:
```json
{
"partitions": [{"col": "val", ...}, ...]
}
```
### How was this patch tested?
Added tests in `v1/ShowPartitionsSuite.scala`,
`v2/ShowPartitionsSuite.scala` and `ShowPartitionsParserSuite.scala`.
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #55441 from ashutosh-jindal/partitions-as-json.
Lead-authored-by: ashutosh-jindal
<[email protected]>
Co-authored-by: Ashutosh Jindal <[email protected]>
Co-authored-by: Wenchen Fan <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
---
docs/sql-ref-syntax-aux-show-partitions.md | 59 ++++++++++-
.../spark/sql/catalyst/parser/SqlBaseParser.g4 | 2 +-
.../spark/sql/catalyst/parser/AstBuilder.scala | 2 +-
.../spark/sql/errors/QueryCompilationErrors.scala | 4 +
.../spark/sql/execution/SparkSqlParser.scala | 23 ++++
.../command/ShowPartitionsJsonCommand.scala | 93 ++++++++++++++++
.../spark/sql/execution/command/tables.scala | 49 ++++-----
.../execution/command/v1/ShowPartitionsSuite.scala | 117 +++++++++++++++++++++
.../execution/command/v2/ShowPartitionsSuite.scala | 12 +++
9 files changed, 332 insertions(+), 29 deletions(-)
diff --git a/docs/sql-ref-syntax-aux-show-partitions.md
b/docs/sql-ref-syntax-aux-show-partitions.md
index 0b2ed3507e29..35dccf1f3608 100644
--- a/docs/sql-ref-syntax-aux-show-partitions.md
+++ b/docs/sql-ref-syntax-aux-show-partitions.md
@@ -28,7 +28,7 @@ partition spec.
### Syntax
```sql
-SHOW PARTITIONS table_identifier [ partition_spec ]
+SHOW PARTITIONS table_identifier [ partition_spec ] [ AS JSON ]
```
### Parameters
@@ -46,6 +46,31 @@ SHOW PARTITIONS table_identifier [ partition_spec ]
**Syntax:** `PARTITION ( partition_col_name = partition_col_val [ , ... ]
)`
+* **AS JSON**
+
+ An optional parameter to return the partition list as a single-row JSON
document
+ instead of the default tabular format. Only supported for V1 (session
catalog / Hive
+ metastore) tables.
+
+ **Syntax:** `[ AS JSON ]`
+
+ **Output schema:** A single column named `json_metadata` of type `STRING
NOT NULL`.
+
+ **Schema:**
+
+ Below is the full JSON schema.
+ In actual output, the JSON is not pretty-printed (see Examples).
+
+ ```json
+ {
+ "partitions": [{"col": "val", ...}, ...]
+ }
+ ```
+
+ | Field | Type | Description |
+ |---|---|---|
+ | `partitions` | array of objects | Each element is a JSON object mapping
partition column names to their string values. The array is empty when no
partitions match the optional `partition_spec`. Elements appear in the order
returned by the underlying catalog (lexicographic for in-memory and Hive
catalogs); no order is guaranteed across catalogs.|
+
### Examples
```sql
@@ -100,6 +125,38 @@ SHOW PARTITIONS customer PARTITION (city = 'San Jose');
+----------------------+
|state=CA/city=San Jose|
+----------------------+
+
+-- List all partitions as a JSON
+SHOW PARTITIONS customer AS JSON;
++--------------------------------------------------------------------------------------------------------------+
+|json_metadata
|
++--------------------------------------------------------------------------------------------------------------+
+|{"partitions":[{"state":"AZ","city":"Peoria"},{"state":"CA","city":"Fremont"},{"state":"CA","city":"San
Jose"}]}|
++--------------------------------------------------------------------------------------------------------------+
+
+-- Filter with a partial partition spec and return results as JSON
+SHOW PARTITIONS customer PARTITION (state = 'CA') AS JSON;
++-----------------------------------------------------------------------------------+
+|json_metadata
|
++-----------------------------------------------------------------------------------+
+|{"partitions":[{"state":"CA","city":"Fremont"},{"state":"CA","city":"San
Jose"}]} |
++-----------------------------------------------------------------------------------+
+
+-- Filter with a full partition spec as JSON
+SHOW PARTITIONS customer PARTITION (state = 'CA', city = 'Fremont') AS JSON;
++--------------------------------------------------+
+|json_metadata |
++--------------------------------------------------+
+|{"partitions":[{"state":"CA","city":"Fremont"}]} |
++--------------------------------------------------+
+
+-- When no partitions match the spec, AS JSON returns an empty array
+SHOW PARTITIONS customer PARTITION (state = 'TX') AS JSON;
++------------------+
+|json_metadata |
++------------------+
+|{"partitions":[]} |
++------------------+
```
### Related Statements
diff --git
a/sql/api/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBaseParser.g4
b/sql/api/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBaseParser.g4
index bb44041f67dd..cce03c169aac 100644
---
a/sql/api/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBaseParser.g4
+++
b/sql/api/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBaseParser.g4
@@ -373,7 +373,7 @@ statement
((FROM | IN) ns=multipartIdentifier)?
#showColumns
| SHOW VIEWS ((FROM | IN) identifierReference)?
(LIKE? pattern=stringLit)?
#showViews
- | SHOW PARTITIONS identifierReference partitionSpec?
#showPartitions
+ | SHOW PARTITIONS identifierReference partitionSpec? (AS JSON)?
#showPartitions
| SHOW functionScope=simpleIdentifier? FUNCTIONS ((FROM | IN)
ns=identifierReference)?
(LIKE? (legacy=multipartIdentifier | pattern=stringLit))?
#showFunctions
| SHOW PROCEDURES ((FROM | IN) identifierReference)?
#showProcedures
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
index 23bdeaa16584..4dc63760e6a2 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
@@ -4430,7 +4430,7 @@ class AstBuilder extends DataTypeAstBuilder
/**
* Create an [[UnresolvedTable]] from an identifier reference.
*/
- private def createUnresolvedTable(
+ protected def createUnresolvedTable(
ctx: IdentifierReferenceContext,
commandName: String,
suggestAlternative: Boolean = false): LogicalPlan = withOrigin(ctx) {
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala
index 7f76f48cc7ef..239aaf3e29e8 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala
@@ -1746,6 +1746,10 @@ private[sql] object QueryCompilationErrors extends
QueryErrorsBase with Compilat
notSupportedInJDBCCatalog("CREATE TABLE ... USING ...")
}
+ def showPartitionsAsJsonNotSupportedForV2TablesError(): Throwable = {
+ notSupportedForV2TablesError("SHOW PARTITIONS AS JSON")
+ }
+
def cannotCreateJDBCTableUsingLocationError(): Throwable = {
notSupportedInJDBCCatalog("CREATE TABLE ... LOCATION ...")
}
diff --git
a/sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala
b/sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala
index bce51f7c23fa..8ff2161a2965 100644
---
a/sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala
+++
b/sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala
@@ -1480,6 +1480,29 @@ class SparkSqlAstBuilder extends AstBuilder {
}
}
+ /**
+ * Overrides `SHOW PARTITIONS` parsing to intercept the `AS JSON` variant.
+ *
+ * When `AS JSON` is absent, parsing is delegated to the superclass
+ * ([[AstBuilder#visitShowPartitions]]), which produces a [[ShowPartitions]]
logical plan.
+ *
+ * When `AS JSON` is present, this method produces a
[[ShowPartitionsJsonCommand]] directly -
+ * a runnable command that returns partition metadata as a single-row JSON
document.
+ *
+ * The syntax of using this command in SQL is:
+ * {{{
+ * SHOW PARTITIONS multi_part_name [partition_spec] [AS JSON];
+ * }}}
+ */
+ override def visitShowPartitions(ctx: ShowPartitionsContext): LogicalPlan =
withOrigin(ctx) {
+ if (ctx.JSON == null) return super.visitShowPartitions(ctx)
+ val relation = createUnresolvedTable(ctx.identifierReference, "SHOW
PARTITIONS AS JSON")
+ val partitionKeys = Option(ctx.partitionSpec).map { specCtx =>
+ UnresolvedPartitionSpec(visitNonOptionalPartitionSpec(specCtx), None)
+ }
+ ShowPartitionsJsonCommand(relation, partitionKeys.map(_.spec))
+ }
+
override def visitShowProcedures(ctx: ShowProceduresContext): LogicalPlan =
withOrigin(ctx) {
val ns = if (ctx.identifierReference != null) {
withIdentClause(ctx.identifierReference, UnresolvedNamespace(_))
diff --git
a/sql/core/src/main/scala/org/apache/spark/sql/execution/command/ShowPartitionsJsonCommand.scala
b/sql/core/src/main/scala/org/apache/spark/sql/execution/command/ShowPartitionsJsonCommand.scala
new file mode 100644
index 000000000000..c74a7d8af034
--- /dev/null
+++
b/sql/core/src/main/scala/org/apache/spark/sql/execution/command/ShowPartitionsJsonCommand.scala
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.spark.sql.execution.command
+
+import org.json4s._
+import org.json4s.JsonAST.JObject
+import org.json4s.jackson.JsonMethods.{compact, render}
+
+import org.apache.spark.sql.{Row, SparkSession}
+import org.apache.spark.sql.catalyst.analysis.ResolvedTable
+import org.apache.spark.sql.catalyst.catalog.CatalogTypes.TablePartitionSpec
+import org.apache.spark.sql.catalyst.expressions.{Attribute,
AttributeReference}
+import org.apache.spark.sql.catalyst.plans.logical.LogicalPlan
+import org.apache.spark.sql.connector.catalog.V1Table
+import org.apache.spark.sql.errors.QueryCompilationErrors
+import org.apache.spark.sql.types.{MetadataBuilder, StringType}
+
+/**
+ * A command for users to list the partition names of a table as a JSON
document. If a partition
+ * spec is specified, only partitions matching the spec are returned.
Otherwise all partitions are
+ * returned.
+ *
+ * The output is a single row with a `json_metadata` column containing a JSON
object of the form:
+ * `{"partitions": [{"col": "val", ...}, ...]}`, where each element of the
array is a JSON object
+ * mapping partition column names to their string values.
+ *
+ * This command is the AS JSON variant of [[ShowPartitions]] and is produced
by the parser
+ * when `SHOW PARTITIONS ... AS JSON` is issued. Only V1 (session catalog /
Hive metastore) tables
+ * are supported.
+ *
+ * The syntax of using this command in SQL is:
+ * {{{
+ * SHOW PARTITIONS multi_part_name [partition_spec] AS JSON;
+ * }}}
+ */
+case class ShowPartitionsJsonCommand(
+ child: LogicalPlan,
+ spec: Option[TablePartitionSpec],
+ override val output: Seq[Attribute] = Seq(
+ AttributeReference(
+ "json_metadata",
+ StringType,
+ nullable = false,
+ new MetadataBuilder().putString("comment", "Partition list of the
table").build())()))
+ extends UnaryRunnableCommand {
+
+ /**
+ * Converts a partition name string (e.g. `"year=2015/month=1"`) into a
[[JObject]] where each
+ * path segment becomes a key-value pair (e.g.
`{"year":"2015","month":"1"}`).
+ */
+ private def partitionNameToJson(partName: String): JObject = {
+ JObject(partName.split("/").map { segment =>
+ val eqIdx = segment.indexOf('=')
+ if (eqIdx >= 0) {
+ segment.substring(0, eqIdx) -> JString(segment.substring(eqIdx + 1))
+ } else {
+ segment -> JString("")
+ }
+ }.toList)
+ }
+
+ override def run(sparkSession: SparkSession): Seq[Row] = {
+ child match {
+ case ResolvedTable(_, _, t: V1Table, _) =>
+ val partNames = ShowPartitionsHelper.listV1PartitionNames(
+ sparkSession, t.catalogTable, spec, "SHOW PARTITIONS AS JSON")
+ Seq(Row(compact(render(JObject("partitions" ->
+ JArray(partNames.map(partitionNameToJson).toList))))))
+
+ // Non-V1 tables are currently not supported.
+ case _ =>
+ throw
QueryCompilationErrors.showPartitionsAsJsonNotSupportedForV2TablesError()
+ }
+ }
+
+ override protected def withNewChildInternal(newChild: LogicalPlan):
LogicalPlan = {
+ copy(child = newChild)
+ }
+}
diff --git
a/sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala
b/sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala
index ca534706635a..e0a4922efe9e 100644
---
a/sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala
+++
b/sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala
@@ -1062,37 +1062,34 @@ case class ShowPartitionsCommand(
spec: Option[TablePartitionSpec]) extends LeafRunnableCommand {
override def run(sparkSession: SparkSession): Seq[Row] = {
- val catalog = sparkSession.sessionState.catalog
- val table = catalog.getTableMetadata(tableName)
- val tableIdentWithDB = table.identifier.quotedString
-
- /**
- * Validate and throws an [[AnalysisException]] exception under the
following conditions:
- * 1. If the table is not partitioned.
- * 2. If it is a datasource table.
- */
+ val table = sparkSession.sessionState.catalog.getTableMetadata(tableName)
+ ShowPartitionsHelper.listV1PartitionNames(sparkSession, table,
spec).map(Row(_))
+ }
+}
+object ShowPartitionsHelper {
+ /**
+ * Used by [[ShowPartitionsCommand]] and [[ShowPartitionsJsonCommand]] to
+ * extract partition names of V1 tables.
+ */
+ def listV1PartitionNames(
+ sparkSession: SparkSession,
+ table: CatalogTable,
+ spec: Option[TablePartitionSpec],
+ commandName: String = "SHOW PARTITIONS"): Seq[String] = {
+ val tableIdentWithDB = table.identifier.quotedString
if (table.partitionColumnNames.isEmpty) {
throw
QueryCompilationErrors.showPartitionNotAllowedOnTableNotPartitionedError(
tableIdentWithDB)
}
-
- DDLUtils.verifyPartitionProviderIsHive(sparkSession, table, "SHOW
PARTITIONS")
-
- /**
- * Normalizes the partition spec w.r.t the partition columns and case
sensitivity settings,
- * and validates the spec by making sure all the referenced columns are
- * defined as partitioning columns in table definition. An
AnalysisException exception is
- * thrown if the partitioning spec is invalid.
- */
- val normalizedSpec = spec.map(partitionSpec =>
PartitioningUtils.normalizePartitionSpec(
- partitionSpec,
- table.partitionSchema,
- table.identifier.quotedString,
- sparkSession.sessionState.conf.resolver))
-
- val partNames = catalog.listPartitionNames(tableName, normalizedSpec)
- partNames.map(Row(_))
+ DDLUtils.verifyPartitionProviderIsHive(sparkSession, table, commandName)
+ val normalizedSpec = spec.map(partitionSpec =>
+ PartitioningUtils.normalizePartitionSpec(
+ partitionSpec,
+ table.partitionSchema,
+ tableIdentWithDB,
+ sparkSession.sessionState.conf.resolver))
+ sparkSession.sessionState.catalog.listPartitionNames(table.identifier,
normalizedSpec)
}
}
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/ShowPartitionsSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/ShowPartitionsSuite.scala
index 0f64fa49f486..8789fb20fc06 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/ShowPartitionsSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/ShowPartitionsSuite.scala
@@ -20,6 +20,7 @@ package org.apache.spark.sql.execution.command.v1
import org.apache.spark.sql.{AnalysisException, Row, SaveMode}
import org.apache.spark.sql.catalyst.analysis.UnresolvedAttribute
import org.apache.spark.sql.catalyst.util.quoteIdentifier
+import
org.apache.spark.sql.connector.catalog.CatalogManager.SESSION_CATALOG_NAME
import org.apache.spark.sql.execution.command
/**
@@ -105,6 +106,73 @@ trait ShowPartitionsSuiteBase extends
command.ShowPartitionsSuiteBase {
Row("p1=__HIVE_DEFAULT_PARTITION__"))
}
}
+
+ test("show partitions as JSON") {
+ withNamespaceAndTable("ns", "dateTable") { t =>
+ createDateTable(t)
+ val df = spark.sql(s"SHOW PARTITIONS $t AS JSON")
+ assert(df.schema.length == 1)
+ assert(df.schema.head.name == "json_metadata")
+ checkAnswer(
+ df,
+ Row(
+
"""{"partitions":[{"year":"2015","month":"1"},{"year":"2015","month":"2"},""" +
+ """{"year":"2016","month":"2"},{"year":"2016","month":"3"}]}"""))
+ }
+ }
+
+ test("show partitions with spec as JSON") {
+ withNamespaceAndTable("ns", "dateTable") { t =>
+ createDateTable(t)
+ checkAnswer(
+ spark.sql(s"SHOW PARTITIONS $t PARTITION(year=2015) AS JSON"),
+
Row("""{"partitions":[{"year":"2015","month":"1"},{"year":"2015","month":"2"}]}"""))
+ checkAnswer(
+ spark.sql(s"SHOW PARTITIONS $t PARTITION(year=2015, month=1) AS JSON"),
+ Row("""{"partitions":[{"year":"2015","month":"1"}]}"""))
+ checkAnswer(
+ spark.sql(s"SHOW PARTITIONS $t PARTITION(month=2) AS JSON"),
+
Row("""{"partitions":[{"year":"2015","month":"2"},{"year":"2016","month":"2"}]}"""))
+ }
+ }
+
+ test("show partitions as JSON with no partitions matching the spec") {
+ withNamespaceAndTable("ns", "dateTable") { t =>
+ createDateTable(t)
+ checkAnswer(
+ spark.sql(s"SHOW PARTITIONS $t PARTITION(year=9999) AS JSON"),
+ Row("""{"partitions":[]}"""))
+ }
+ }
+
+ test("null as a partition value AS JSON") {
+ val t = "part_table"
+ withTable(t) {
+ sql(s"CREATE TABLE $t (col1 INT, p1 STRING) $defaultUsing PARTITIONED BY
(p1)")
+ sql(s"INSERT INTO TABLE $t PARTITION (p1 = null) SELECT 0")
+ checkAnswer(
+ sql(s"SHOW PARTITIONS $t AS JSON"),
+ Row("""{"partitions":[{"p1":"__HIVE_DEFAULT_PARTITION__"}]}"""))
+ checkAnswer(
+ sql(s"SHOW PARTITIONS $t PARTITION (p1 = null) AS JSON"),
+ Row("""{"partitions":[{"p1":"__HIVE_DEFAULT_PARTITION__"}]}"""))
+ }
+ }
+
+ test("non-partitioning columns AS JSON") {
+ withNamespaceAndTable("ns", "dateTable") { t =>
+ createDateTable(t)
+ checkError(
+ exception = intercept[AnalysisException] {
+ sql(s"SHOW PARTITIONS $t PARTITION(abcd=2015, xyz=1) AS JSON")
+ },
+ condition = "PARTITIONS_NOT_FOUND",
+ parameters = Map(
+ "partitionList" -> "`abcd`",
+ "tableName" -> s"`$SESSION_CATALOG_NAME`.`ns`.`datetable`")
+ )
+ }
+ }
}
/**
@@ -168,6 +236,20 @@ class ShowPartitionsSuite extends ShowPartitionsSuiteBase
with CommandSuiteBase
}
}
+ test("show partitions of non-partitioned table AS JSON") {
+ withNamespaceAndTable("ns", "not_partitioned_table") { t =>
+ sql(s"CREATE TABLE $t (col1 int) $defaultUsing")
+ val tableName =
+
UnresolvedAttribute.parseAttributeName(t).map(quoteIdentifier).mkString(".")
+ checkError(
+ exception = intercept[AnalysisException] {
+ sql(s"SHOW PARTITIONS $t AS JSON")
+ },
+ condition = "INVALID_PARTITION_OPERATION.PARTITION_SCHEMA_IS_EMPTY",
+ parameters = Map("name" -> tableName))
+ }
+ }
+
test("SPARK-33904: null and empty string as partition values") {
withNamespaceAndTable("ns", "tbl") { t =>
createNullPartTable(t, "parquet")
@@ -177,4 +259,39 @@ class ShowPartitionsSuite extends ShowPartitionsSuiteBase
with CommandSuiteBase
checkAnswer(spark.table(t), Row(0, null) :: Row(1, null) :: Nil)
}
}
+
+ test("null and empty string as partition values AS JSON") {
+ withNamespaceAndTable("ns", "tbl") { t =>
+ createNullPartTable(t, "parquet")
+ checkAnswer(
+ sql(s"SHOW PARTITIONS $t AS JSON"),
+ Row("""{"partitions":[{"part":"__HIVE_DEFAULT_PARTITION__"}]}"""))
+ }
+ }
+
+ test("partition value containing '/' AS JSON") {
+ withTable("slash_part") {
+ sql(s"CREATE TABLE slash_part (col1 INT, p STRING) $defaultUsing
PARTITIONED BY (p)")
+ sql("INSERT INTO TABLE slash_part PARTITION (p = 'a/b') SELECT 0")
+ checkAnswer(
+ sql("SHOW PARTITIONS slash_part AS JSON"),
+ Row("""{"partitions":[{"p":"a%2Fb"}]}"""))
+ // Filtering by the escaped spec also works and returns the same encoded
value.
+ checkAnswer(
+ sql("SHOW PARTITIONS slash_part PARTITION (p = 'a/b') AS JSON"),
+ Row("""{"partitions":[{"p":"a%2Fb"}]}"""))
+ }
+ }
+
+ test("partition value containing '=' AS JSON") {
+ // '=' is encoded as '%3D'; splitting on the first '=' in each segment is
still
+ // correct because a column name never contains an unencoded '='.
+ withTable("eq_part") {
+ sql(s"CREATE TABLE eq_part (col1 INT, p STRING) $defaultUsing
PARTITIONED BY (p)")
+ sql("INSERT INTO TABLE eq_part PARTITION (p = 'a=b') SELECT 0")
+ checkAnswer(
+ sql("SHOW PARTITIONS eq_part AS JSON"),
+ Row("""{"partitions":[{"p":"a%3Db"}]}"""))
+ }
+ }
}
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/command/v2/ShowPartitionsSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/command/v2/ShowPartitionsSuite.scala
index 1fb1c4889060..403ca9fa8edc 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/command/v2/ShowPartitionsSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/command/v2/ShowPartitionsSuite.scala
@@ -77,4 +77,16 @@ class ShowPartitionsSuite extends
command.ShowPartitionsSuiteBase with CommandSu
checkAnswer(spark.table(t), Row(0, "") :: Row(1, null) :: Nil)
}
}
+
+ test("SHOW PARTITIONS AS JSON is not supported for V2 tables") {
+ withNamespaceAndTable("ns", "dateTable") { t =>
+ createDateTable(t)
+ checkError(
+ exception = intercept[AnalysisException] {
+ sql(s"SHOW PARTITIONS $t AS JSON")
+ },
+ condition = "NOT_SUPPORTED_COMMAND_FOR_V2_TABLE",
+ parameters = Map("cmd" -> "SHOW PARTITIONS AS JSON"))
+ }
+ }
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
