[
https://issues.apache.org/jira/browse/HUDI-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17911694#comment-17911694
]
Mansi Patel commented on HUDI-8853:
-----------------------------------
Reproduction steps:
{code:java}
spark-shell --jars /usr/lib/hudi/hudi-spark-bundle.jar \
--conf "spark.serializer=org.apache.spark.serializer.KryoSerializer" \
--conf
"spark.sql.catalog.spark_catalog=org.apache.spark.sql.hudi.catalog.HoodieCatalog"
\
--conf
"spark.sql.extensions=org.apache.spark.sql.hudi.HoodieSparkSessionExtension"
{code}
> Spark sql ALTER TABLE queries are failing on EMR
> ------------------------------------------------
>
> Key: HUDI-8853
> URL: https://issues.apache.org/jira/browse/HUDI-8853
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Mansi Patel
> Priority: Major
>
> Some of the spark sql DDL queries are failing on EMR. Failed queries are
> listed here
> 1. ALTER TABLE DROP COLUMN
> 2. ALTER TABLE REPLACE COLUMN
> 3. ALTER TABLE RENAME COLUMN
> {code:java}
> scala> spark.sql("ALTER TABLE mansipp_hudi_fgac_table DROP COLUMN
> creation_date"); org.apache.spark.sql.AnalysisException:
> [UNSUPPORTED_FEATURE.TABLE_OPERATION] The feature is not supported: Table
> `spark_catalog`.`default`.`mansipp_hudi_fgac_table` does not support DROP
> COLUMN. Please check the current catalog and namespace to make sure the
> qualified table name is expected, and also check the catalog implementation
> which is configured by "spark.sql.catalog". at
> org.apache.spark.sql.errors.QueryCompilationErrors$.unsupportedTableOperationError(QueryCompilationErrors.scala:847)
> at
> org.apache.spark.sql.errors.QueryCompilationErrors$.unsupportedTableOperationError(QueryCompilationErrors.scala:837)
> at
> org.apache.spark.sql.catalyst.analysis.ResolveSessionCatalog$$anonfun$apply$1.applyOrElse(ResolveSessionCatalog.scala:110)
> {code}
> {code:java}
> scala> spark.sql("ALTER TABLE mansipp_hudi_fgac_table REPLACE COLUMNS (id
> int, name varchar(10), city string)");
> org.apache.spark.sql.AnalysisException: [UNSUPPORTED_FEATURE.TABLE_OPERATION]
> The feature is not supported: Table
> `spark_catalog`.`default`.`mansipp_hudi_fgac_table` does not support REPLACE
> COLUMNS. Please check the current catalog and namespace to make sure the
> qualified table name is expected, and also check the catalog implementation
> which is configured by "spark.sql.catalog".
> at
> org.apache.spark.sql.errors.QueryCompilationErrors$.unsupportedTableOperationError(QueryCompilationErrors.scala:847)
> at
> org.apache.spark.sql.errors.QueryCompilationErrors$.unsupportedTableOperationError(QueryCompilationErrors.scala:837)
> at
> org.apache.spark.sql.catalyst.analysis.ResolveSessionCatalog$$anonfun$apply$1.applyOrElse(ResolveSessionCatalog.scala:66)
> at
> org.apache.spark.sql.catalyst.analysis.ResolveSessionCatalog$$anonfun$apply$1.applyOrElse(ResolveSessionCatalog.scala:52)
> {code}
> {code:java}
> scala> spark.sql("ALTER TABLE mansipp_hudi_fgac_table RENAME COLUMN
> creation_date TO creation_date_renamed"); 25/01/09 00:38:42 WARN HiveConf:
> HiveConf of name hive.server2.thrift.url does not exist SLF4J: Failed to load
> class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation
> (NOP) logger implementation SLF4J: See
> http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
> org.apache.spark.sql.AnalysisException: [UNSUPPORTED_FEATURE.TABLE_OPERATION]
> The feature is not supported: Table
> `spark_catalog`.`default`.`mansipp_hudi_fgac_table` does not support RENAME
> COLUMN. Please check the current catalog and namespace to make sure the
> qualified table name is expected, and also check the catalog implementation
> which is configured by "spark.sql.catalog". at
> org.apache.spark.sql.errors.QueryCompilationErrors$.unsupportedTableOperationError(QueryCompilationErrors.scala:847)
> at org.apache.spark.sql.errors.QueryCompila
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
