Aireed opened a new issue, #2755:
URL: https://github.com/apache/amoro/issues/2755
### What happened?
use desc formatted table in spark-sql cli, we got a exception.
`
spark-sql> desc formatted hive_by_spark;
24/04/16 17:23:42 WARN HiveConf: HiveConf of name
hive.exec.stagingdir.for.destination.not.exists does not exist
24/04/16 17:23:42 WARN HiveConf: HiveConf of name
hive.metastore.zookeeper.product.namespace does not exist
24/04/16 17:23:42 WARN HiveConf: HiveConf of name
hive.internal.table.location.specification.enabled does not exist
24/04/16 17:23:42 WARN HiveConf: HiveConf of name
hive.support.sql11.reserved.keywords does not exist
24/04/16 17:23:43 WARN HiveConf: HiveConf of name
hive.exec.stagingdir.for.destination.not.exists does not exist
24/04/16 17:23:43 WARN HiveConf: HiveConf of name
hive.metastore.zookeeper.product.namespace does not exist
24/04/16 17:23:43 WARN HiveConf: HiveConf of name
hive.internal.table.location.specification.enabled does not exist
24/04/16 17:23:43 WARN HiveConf: HiveConf of name
hive.support.sql11.reserved.keywords does not exist
24/04/16 17:23:44 ERROR SparkSQLDriver: Failed in [desc formatted
hive_by_spark]
java.lang.IllegalArgumentException: Multiple entries with same key:
base.write.format=parquet and base.write.format=parquet
at
com.netease.arctic.shade.org.apache.iceberg.relocated.com.google.common.collect.ImmutableMap.conflictException(ImmutableMap.java:378)
at
com.netease.arctic.shade.org.apache.iceberg.relocated.com.google.common.collect.ImmutableMap.checkNoConflict(ImmutableMap.java:372)
at
com.netease.arctic.shade.org.apache.iceberg.relocated.com.google.common.collect.RegularImmutableMap.checkNoConflictInKeyBucket(RegularImmutableMap.java:246)
at
com.netease.arctic.shade.org.apache.iceberg.relocated.com.google.common.collect.RegularImmutableMap.fromEntryArrayCheckingBucketOverflow(RegularImmutableMap.java:133)
at
com.netease.arctic.shade.org.apache.iceberg.relocated.com.google.common.collect.RegularImmutableMap.fromEntryArray(RegularImmutableMap.java:95)
at
com.netease.arctic.shade.org.apache.iceberg.relocated.com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:572)
at
com.netease.arctic.shade.org.apache.iceberg.relocated.com.google.common.collect.ImmutableMap$Builder.buildOrThrow(ImmutableMap.java:600)
at
com.netease.arctic.shade.org.apache.iceberg.relocated.com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:587)
at
com.netease.arctic.spark.table.ArcticSparkTable.properties(ArcticSparkTable.java:149)
at
com.netease.arctic.spark.sql.execution.DescribeKeyedTableExec.$anonfun$addTableDetails$1(DescribeKeyedTableExec.scala:113)
at scala.collection.immutable.List.foreach(List.scala:431)
at
com.netease.arctic.spark.sql.execution.DescribeKeyedTableExec.addTableDetails(DescribeKeyedTableExec.scala:112)
at
com.netease.arctic.spark.sql.execution.DescribeKeyedTableExec.run(DescribeKeyedTableExec.scala:88)
at
org.apache.spark.sql.execution.datasources.v2.V2CommandExec.result$lzycompute(V2CommandExec.scala:43)
at
org.apache.spark.sql.execution.datasources.v2.V2CommandExec.result(V2CommandExec.scala:43)
at
org.apache.spark.sql.execution.datasources.v2.V2CommandExec.executeCollect(V2CommandExec.scala:49)
at
org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.$anonfun$applyOrElse$1(QueryExecution.scala:98)
at
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$6(SQLExecution.scala:109)
at
org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:169)
at
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:95)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
at
org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64)
`
### Affects Versions
master
### What engines are you seeing the problem on?
Spark
### How to reproduce
_No response_
### Relevant log output
_No response_
### Anything else
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's Code of Conduct
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
