danhuawang opened a new issue, #11049:
URL: https://github.com/apache/gravitino/issues/11049
### Version
main branch
### Describe what's wrong
Create view for remote IRC encounter ServiceFailureException: Server error:
NotFoundException: Metadata object catalog_2.gt_spark_ddl_view_56782 or owner
molly is not found.
Then recreate the view pop SCHEMA_ALREADY_EXISTS.
```
spark-sql (gt_spark_ddl_view)> CREATE DATABASE
remote_irc_a.gt_spark_ddl_view_56782;
26/05/12 14:45:48 ERROR SparkSQLDriver: Failed in [CREATE DATABASE
remote_irc_a.gt_spark_ddl_view_56782]
org.apache.iceberg.exceptions.ServiceFailureException: Server error:
NotFoundException: Metadata object catalog_2.gt_spark_ddl_view_56782 or owner
molly is not found
at
org.apache.iceberg.rest.ErrorHandlers$DefaultErrorHandler.accept(ErrorHandlers.java:243)
at
org.apache.iceberg.rest.ErrorHandlers$NamespaceErrorHandler.accept(ErrorHandlers.java:192)
at
org.apache.iceberg.rest.ErrorHandlers$NamespaceErrorHandler.accept(ErrorHandlers.java:173)
at org.apache.iceberg.rest.HTTPClient.throwFailure(HTTPClient.java:240)
at org.apache.iceberg.rest.HTTPClient.execute(HTTPClient.java:336)
at org.apache.iceberg.rest.HTTPClient.execute(HTTPClient.java:297)
at org.apache.iceberg.rest.BaseHTTPClient.post(BaseHTTPClient.java:100)
at
org.apache.iceberg.rest.RESTSessionCatalog.createNamespace(RESTSessionCatalog.java:550)
at
org.apache.iceberg.catalog.BaseSessionCatalog$AsCatalog.createNamespace(BaseSessionCatalog.java:128)
at
org.apache.iceberg.rest.RESTCatalog.createNamespace(RESTCatalog.java:227)
at
org.apache.iceberg.spark.SparkCatalog.createNamespace(SparkCatalog.java:483)
at
org.apache.spark.sql.execution.datasources.v2.CreateNamespaceExec.run(CreateNamespaceExec.scala:47)
at
org.apache.spark.sql.execution.datasources.v2.V2CommandExec.result$lzycompute(V2CommandExec.scala:43)
at
org.apache.spark.sql.execution.datasources.v2.V2CommandExec.result(V2CommandExec.scala:43)
at
org.apache.spark.sql.execution.datasources.v2.V2CommandExec.executeCollect(V2CommandExec.scala:49)
at
org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.$anonfun$applyOrElse$1(QueryExecution.scala:107)
at
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$6(SQLExecution.scala:125)
at
org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:201)
at
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:108)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:900)
at
org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:66)
at
org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:107)
at
org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:98)
at
org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:461)
at
org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(origin.scala:76)
at
org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:461)
at
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDownWithPruning(LogicalPlan.scala:32)
at
org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning(AnalysisHelper.scala:267)
at
org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning$(AnalysisHelper.scala:263)
at
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:32)
at
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:32)
at
org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:437)
at
org.apache.spark.sql.execution.QueryExecution.eagerlyExecuteCommands(QueryExecution.scala:98)
at
org.apache.spark.sql.execution.QueryExecution.commandExecuted$lzycompute(QueryExecution.scala:85)
at
org.apache.spark.sql.execution.QueryExecution.commandExecuted(QueryExecution.scala:83)
at org.apache.spark.sql.Dataset.<init>(Dataset.scala:220)
at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:100)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:900)
at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:97)
at
org.apache.spark.sql.SparkSession.$anonfun$sql$4(SparkSession.scala:691)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:900)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:682)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:713)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:744)
at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:651)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLDriver.run(SparkSQLDriver.scala:68)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.processCmd(SparkSQLCLIDriver.scala:501)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.$anonfun$processLine$1(SparkSQLCLIDriver.scala:619)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.$anonfun$processLine$1$adapted(SparkSQLCLIDriver.scala:613)
at scala.collection.Iterator.foreach(Iterator.scala:943)
at scala.collection.Iterator.foreach$(Iterator.scala:943)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
at scala.collection.IterableLike.foreach(IterableLike.scala:74)
at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.processLine(SparkSQLCLIDriver.scala:613)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:310)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1029)
at
org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:194)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:217)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:91)
at
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1120)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1129)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Server error: NotFoundException: Metadata object
catalog_2.gt_spark_ddl_view_56782 or owner molly is not found
org.apache.iceberg.exceptions.ServiceFailureException: Server error:
NotFoundException: Metadata object catalog_2.gt_spark_ddl_view_56782 or owner
molly is not found
at
org.apache.iceberg.rest.ErrorHandlers$DefaultErrorHandler.accept(ErrorHandlers.java:243)
at
org.apache.iceberg.rest.ErrorHandlers$NamespaceErrorHandler.accept(ErrorHandlers.java:192)
at
org.apache.iceberg.rest.ErrorHandlers$NamespaceErrorHandler.accept(ErrorHandlers.java:173)
at org.apache.iceberg.rest.HTTPClient.throwFailure(HTTPClient.java:240)
at org.apache.iceberg.rest.HTTPClient.execute(HTTPClient.java:336)
at org.apache.iceberg.rest.HTTPClient.execute(HTTPClient.java:297)
at org.apache.iceberg.rest.BaseHTTPClient.post(BaseHTTPClient.java:100)
at
org.apache.iceberg.rest.RESTSessionCatalog.createNamespace(RESTSessionCatalog.java:550)
at
org.apache.iceberg.catalog.BaseSessionCatalog$AsCatalog.createNamespace(BaseSessionCatalog.java:128)
at
org.apache.iceberg.rest.RESTCatalog.createNamespace(RESTCatalog.java:227)
at
org.apache.iceberg.spark.SparkCatalog.createNamespace(SparkCatalog.java:483)
at
org.apache.spark.sql.execution.datasources.v2.CreateNamespaceExec.run(CreateNamespaceExec.scala:47)
at
org.apache.spark.sql.execution.datasources.v2.V2CommandExec.result$lzycompute(V2CommandExec.scala:43)
at
org.apache.spark.sql.execution.datasources.v2.V2CommandExec.result(V2CommandExec.scala:43)
at
org.apache.spark.sql.execution.datasources.v2.V2CommandExec.executeCollect(V2CommandExec.scala:49)
at
org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.$anonfun$applyOrElse$1(QueryExecution.scala:107)
at
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$6(SQLExecution.scala:125)
at
org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:201)
at
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:108)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:900)
at
org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:66)
at
org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:107)
at
org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:98)
at
org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:461)
at
org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(origin.scala:76)
at
org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:461)
at
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDownWithPruning(LogicalPlan.scala:32)
at
org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning(AnalysisHelper.scala:267)
at
org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning$(AnalysisHelper.scala:263)
at
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:32)
at
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:32)
at
org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:437)
at
org.apache.spark.sql.execution.QueryExecution.eagerlyExecuteCommands(QueryExecution.scala:98)
at
org.apache.spark.sql.execution.QueryExecution.commandExecuted$lzycompute(QueryExecution.scala:85)
at
org.apache.spark.sql.execution.QueryExecution.commandExecuted(QueryExecution.scala:83)
at org.apache.spark.sql.Dataset.<init>(Dataset.scala:220)
at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:100)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:900)
at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:97)
at
org.apache.spark.sql.SparkSession.$anonfun$sql$4(SparkSession.scala:691)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:900)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:682)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:713)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:744)
at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:651)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLDriver.run(SparkSQLDriver.scala:68)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.processCmd(SparkSQLCLIDriver.scala:501)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.$anonfun$processLine$1(SparkSQLCLIDriver.scala:619)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.$anonfun$processLine$1$adapted(SparkSQLCLIDriver.scala:613)
at scala.collection.Iterator.foreach(Iterator.scala:943)
at scala.collection.Iterator.foreach$(Iterator.scala:943)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
at scala.collection.IterableLike.foreach(IterableLike.scala:74)
at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.processLine(SparkSQLCLIDriver.scala:613)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:310)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1029)
at
org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:194)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:217)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:91)
at
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1120)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1129)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
```
```
spark-sql (gt_spark_ddl_view)> CREATE DATABASE
remote_irc_a.gt_spark_ddl_view_56782;
[SCHEMA_ALREADY_EXISTS] Cannot create schema `gt_spark_ddl_view_56782`
because it already exists.
```
### Error message and/or stacktrace
see above
### How to reproduce
1. Add user molly to the remote metalake
2. Create role1 and grant it to user molly, role1's definition as following
```
{
"code": 0,
"role": {
"name": "role_1",
"audit": {
"creator": "admin",
"createTime": "2026-04-29T10:55:49.203369087Z",
"lastModifier": "admin",
"lastModifiedTime": "2026-05-06T11:22:57.146619903Z"
},
"properties": {
"k1": "v1"
},
"securableObjects": [
{
"type": "catalog",
"privileges": [
{
"name": "create_schema",
"condition": "allow"
}
],
"fullName": "catalog_1"
},
{
"type": "metalake",
"privileges": [
{
"name": "use_catalog",
"condition": "allow"
},
{
"name": "use_schema",
"condition": "allow"
}
],
"fullName": "metalake_b"
},
{
"type": "schema",
"privileges": [
{
"name": "select_table",
"condition": "allow"
},
{
"name": "modify_table",
"condition": "allow"
},
{
"name": "create_table",
"condition": "deny"
}
],
"fullName": "catalog_1.product_s3"
}
]
}
}
```
3. Spark to create database with the user molly
```
spark.sql.extensions
org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions
spark.sql.catalog.gravitino_irc_b org.apache.iceberg.spark.SparkCatalog
spark.sql.catalog.gravitino_irc_b.type rest
spark.sql.catalog.gravitino_irc_b.uri http://34.23.68.122:9001/iceberg/
spark.sql.catalog.gravitino_irc_b.header.X-Iceberg-Access-Delegation
vended-credentials
spark.sql.catalog.gravitino_irc_b.rest.auth.type
com.dremio.iceberg.authmgr.oauth2.OAuth2Manager
spark.sql.catalog.gravitino_irc_b.rest.auth.oauth2.issuer-url
http://******:8080/realms/myrealm
spark.sql.catalog.gravitino_irc_b.rest.auth.oauth2.client-id postman-client
spark.sql.catalog.gravitino_irc_b.rest.auth.oauth2.client-secret XXXXXXX
spark.sql.catalog.gravitino_irc_b.rest.auth.oauth2.grant-type password
spark.sql.catalog.gravitino_irc_b.rest.auth.oauth2.resource-owner.username
molly
spark.sql.catalog.gravitino_irc_b.rest.auth.oauth2.resource-owner.password
******
spark.sql.catalog.gravitino_irc_b.rest.auth.oauth2.scope openid
spark.sql.catalog.gravitino_irc_b.rest.auth.oauth2.token-refresh.enabled true
spark.sql.catalog.gravitino_irc_b.rest.auth.oauth2.token-refresh.idle-timeout
PT2M
spark.sql.catalog.remote_irc_a org.apache.iceberg.spark.SparkCatalog
spark.sql.catalog.remote_irc_a.type rest
spark.sql.catalog.remote_irc_a.uri http://34.75.126.12:9001/iceberg/
spark.sql.catalog.remote_irc_a.warehouse catalog_2
spark.sql.catalog.remote_irc_a.prefix catalog_2
spark.sql.catalog.remote_irc_a.header.X-Iceberg-Access-Delegation
vended-credentials
spark.sql.catalog.remote_irc_a.rest.auth.type
com.dremio.iceberg.authmgr.oauth2.OAuth2Manager
spark.sql.catalog.remote_irc_a.rest.auth.oauth2.issuer-url
http://******:8080/realms/myrealm
spark.sql.catalog.remote_irc_a.rest.auth.oauth2.client-id postman-client
spark.sql.catalog.remote_irc_a.rest.auth.oauth2.client-secret XXXXXXX
spark.sql.catalog.remote_irc_a.rest.auth.oauth2.grant-type password
spark.sql.catalog.remote_irc_a.rest.auth.oauth2.resource-owner.username molly
spark.sql.catalog.remote_irc_a.rest.auth.oauth2.resource-owner.password
******
spark.sql.catalog.remote_irc_a.rest.auth.oauth2.scope openid
spark.sql.catalog.remote_irc_a.rest.auth.oauth2.token-refresh.enabled true
spark.sql.catalog.remote_irc_a.rest.auth.oauth2.token-refresh.idle-timeout
PT2M
spark.locality.wait.node 0
```
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
