[
https://issues.apache.org/jira/browse/HUDI-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mansi Patel updated HUDI-8853:
------------------------------
Description:
Some of the spark sql DDL queries are failing on EMR. Failed queries are listed
here
1. ALTER TABLE DROP COLUMN
2. ALTER TABLE REPLACE COLUMN
3. ALTER TABLE RENAME COLUMN
{code:java}
scala> spark.sql("ALTER TABLE mansipp_hudi_fgac_table DROP COLUMN
creation_date"); org.apache.spark.sql.AnalysisException:
[UNSUPPORTED_FEATURE.TABLE_OPERATION] The feature is not supported: Table
`spark_catalog`.`default`.`mansipp_hudi_fgac_table` does not support DROP
COLUMN. Please check the current catalog and namespace to make sure the
qualified table name is expected, and also check the catalog implementation
which is configured by "spark.sql.catalog". at
org.apache.spark.sql.errors.QueryCompilationErrors$.unsupportedTableOperationError(QueryCompilationErrors.scala:847)
at
org.apache.spark.sql.errors.QueryCompilationErrors$.unsupportedTableOperationError(QueryCompilationErrors.scala:837)
at
org.apache.spark.sql.catalyst.analysis.ResolveSessionCatalog$$anonfun$apply$1.applyOrElse(ResolveSessionCatalog.scala:110)
{code}
{code:java}
scala> spark.sql("ALTER TABLE mansipp_hudi_fgac_table REPLACE COLUMNS (id int,
name varchar(10), city string)");
org.apache.spark.sql.AnalysisException: [UNSUPPORTED_FEATURE.TABLE_OPERATION]
The feature is not supported: Table
`spark_catalog`.`default`.`mansipp_hudi_fgac_table` does not support REPLACE
COLUMNS. Please check the current catalog and namespace to make sure the
qualified table name is expected, and also check the catalog implementation
which is configured by "spark.sql.catalog".
at
org.apache.spark.sql.errors.QueryCompilationErrors$.unsupportedTableOperationError(QueryCompilationErrors.scala:847)
at
org.apache.spark.sql.errors.QueryCompilationErrors$.unsupportedTableOperationError(QueryCompilationErrors.scala:837)
at
org.apache.spark.sql.catalyst.analysis.ResolveSessionCatalog$$anonfun$apply$1.applyOrElse(ResolveSessionCatalog.scala:66)
at
org.apache.spark.sql.catalyst.analysis.ResolveSessionCatalog$$anonfun$apply$1.applyOrElse(ResolveSessionCatalog.scala:52)
{code}
{code:java}
scala> spark.sql("ALTER TABLE mansipp_hudi_fgac_table RENAME COLUMN
creation_date TO creation_date_renamed"); 25/01/09 00:38:42 WARN HiveConf:
HiveConf of name hive.server2.thrift.url does not exist SLF4J: Failed to load
class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation
(NOP) logger implementation SLF4J: See
http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
org.apache.spark.sql.AnalysisException: [UNSUPPORTED_FEATURE.TABLE_OPERATION]
The feature is not supported: Table
`spark_catalog`.`default`.`mansipp_hudi_fgac_table` does not support RENAME
COLUMN. Please check the current catalog and namespace to make sure the
qualified table name is expected, and also check the catalog implementation
which is configured by "spark.sql.catalog". at
org.apache.spark.sql.errors.QueryCompilationErrors$.unsupportedTableOperationError(QueryCompilationErrors.scala:847)
at org.apache.spark.sql.errors.QueryCompila
{code}
was:
Some of the spark sql DDL queries are failing on EMR. Failed queries are listed
here
1. ALTER TABLE DROP COLUMN
2. ALTER TABLE REPLACE COLUMN
3. ALTER TABLE RENAME COLUMN
{code:java}
scala> spark.sql("ALTER TABLE mansipp_hudi_fgac_table DROP COLUMN
creation_date"); org.apache.spark.sql.AnalysisException:
[UNSUPPORTED_FEATURE.TABLE_OPERATION] The feature is not supported: Table
`spark_catalog`.`default`.`mansipp_hudi_fgac_table` does not support DROP
COLUMN. Please check the current catalog and namespace to make sure the
qualified table name is expected, and also check the catalog implementation
which is configured by "spark.sql.catalog". at
org.apache.spark.sql.errors.QueryCompilationErrors$.unsupportedTableOperationError(QueryCompilationErrors.scala:847)
at
org.apache.spark.sql.errors.QueryCompilationErrors$.unsupportedTableOperationError(QueryCompilationErrors.scala:837)
at
org.apache.spark.sql.catalyst.analysis.ResolveSessionCatalog$$anonfun$apply$1.applyOrElse(ResolveSessionCatalog.scala:110)
{code}
> Spark sql ALTER TABLE queries are failing on EMR
> ------------------------------------------------
>
> Key: HUDI-8853
> URL: https://issues.apache.org/jira/browse/HUDI-8853
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Mansi Patel
> Priority: Major
>
> Some of the spark sql DDL queries are failing on EMR. Failed queries are
> listed here
> 1. ALTER TABLE DROP COLUMN
> 2. ALTER TABLE REPLACE COLUMN
> 3. ALTER TABLE RENAME COLUMN
> {code:java}
> scala> spark.sql("ALTER TABLE mansipp_hudi_fgac_table DROP COLUMN
> creation_date"); org.apache.spark.sql.AnalysisException:
> [UNSUPPORTED_FEATURE.TABLE_OPERATION] The feature is not supported: Table
> `spark_catalog`.`default`.`mansipp_hudi_fgac_table` does not support DROP
> COLUMN. Please check the current catalog and namespace to make sure the
> qualified table name is expected, and also check the catalog implementation
> which is configured by "spark.sql.catalog". at
> org.apache.spark.sql.errors.QueryCompilationErrors$.unsupportedTableOperationError(QueryCompilationErrors.scala:847)
> at
> org.apache.spark.sql.errors.QueryCompilationErrors$.unsupportedTableOperationError(QueryCompilationErrors.scala:837)
> at
> org.apache.spark.sql.catalyst.analysis.ResolveSessionCatalog$$anonfun$apply$1.applyOrElse(ResolveSessionCatalog.scala:110)
> {code}
> {code:java}
> scala> spark.sql("ALTER TABLE mansipp_hudi_fgac_table REPLACE COLUMNS (id
> int, name varchar(10), city string)");
> org.apache.spark.sql.AnalysisException: [UNSUPPORTED_FEATURE.TABLE_OPERATION]
> The feature is not supported: Table
> `spark_catalog`.`default`.`mansipp_hudi_fgac_table` does not support REPLACE
> COLUMNS. Please check the current catalog and namespace to make sure the
> qualified table name is expected, and also check the catalog implementation
> which is configured by "spark.sql.catalog".
> at
> org.apache.spark.sql.errors.QueryCompilationErrors$.unsupportedTableOperationError(QueryCompilationErrors.scala:847)
> at
> org.apache.spark.sql.errors.QueryCompilationErrors$.unsupportedTableOperationError(QueryCompilationErrors.scala:837)
> at
> org.apache.spark.sql.catalyst.analysis.ResolveSessionCatalog$$anonfun$apply$1.applyOrElse(ResolveSessionCatalog.scala:66)
> at
> org.apache.spark.sql.catalyst.analysis.ResolveSessionCatalog$$anonfun$apply$1.applyOrElse(ResolveSessionCatalog.scala:52)
> {code}
> {code:java}
> scala> spark.sql("ALTER TABLE mansipp_hudi_fgac_table RENAME COLUMN
> creation_date TO creation_date_renamed"); 25/01/09 00:38:42 WARN HiveConf:
> HiveConf of name hive.server2.thrift.url does not exist SLF4J: Failed to load
> class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation
> (NOP) logger implementation SLF4J: See
> http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
> org.apache.spark.sql.AnalysisException: [UNSUPPORTED_FEATURE.TABLE_OPERATION]
> The feature is not supported: Table
> `spark_catalog`.`default`.`mansipp_hudi_fgac_table` does not support RENAME
> COLUMN. Please check the current catalog and namespace to make sure the
> qualified table name is expected, and also check the catalog implementation
> which is configured by "spark.sql.catalog". at
> org.apache.spark.sql.errors.QueryCompilationErrors$.unsupportedTableOperationError(QueryCompilationErrors.scala:847)
> at org.apache.spark.sql.errors.QueryCompila
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
