[
https://issues.apache.org/jira/browse/SPARK-55645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18060878#comment-18060878
]
David Tagatac commented on SPARK-55645:
---------------------------------------
Implemented in
[https://github.com/tagatac/spark/commit/e874c1a29d36d0c82b9298616e0aa05cfc2f8f4e|https://github.com/tagatac/spark/commit/e874c1a29d36d0c82b9298616e0aa05cfc2f8f4e.]
I am testing the `DESCRIBE EXTENDED` functionality before opening a PR.
> Include Hive Metastore SerDeInfo field NAME in Spark representation
> -------------------------------------------------------------------
>
> Key: SPARK-55645
> URL: https://issues.apache.org/jira/browse/SPARK-55645
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.5.6
> Reporter: David Tagatac
> Priority: Minor
>
> The [Hive Metastore
> API|https://github.com/apache/hive/blob/5160d3af392248255f68e41e1e0557eae4d95273/metastore/if/hive_metastore.thrift#L260]
> SerDeInfo struct includes a `name` field. However, this field is not
> included in the [data structure used to represent the API
> response|https://github.com/apache/spark/blob/ac4e7947e4efa4ca2484d9393b14322f4e636df5/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala#L143-L149]
>
> {code:java}
> org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat{code}
>
> As a result, extension developers do not have a good way to use this field
> without storing it somewhere unintuitive like
> `CatalogStorageFormat.properties` with some arbitrary key like "serde.name".
> Including this field as a corresponding field in the
> `org.apache.spark.sql.catalyst.catalog` data model will allow for a better
> representation of Hive Metastore tables in Spark.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
