azexcy opened a new issue, #17704:
URL: https://github.com/apache/shardingsphere/issues/17704
## Bug Report
Sometimes, add resource will failed.
### Which version of ShardingSphere did you use?
master 9886434bd2a1800d9323e9e01c954f661601401c
### Which project did you use? ShardingSphere-JDBC or ShardingSphere-Proxy?
ShardingSphere-Proxy
### Expected behavior
`ADD RESOURCE ` dist sql will success
### Actual behavior
Sometimes, the stack exception message is as follows, the
`docker-java-stream--350783097` prefix can be ignored(because it was running at
Docker)
```
Error: 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy - [ERROR] 2022-05-16 05:00:38.836 [Connection-3-ThreadExecutor]
o.a.s.p.b.t.d.r.r.AddResourceBackendHandler - Add resource failed
[INFO ] 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy -
org.apache.shardingsphere.infra.config.exception.ShardingSphereConfigurationException:
Can not load table: HikariDataSource HikariDataSource (HikariPool-4) has been
closed.
[INFO ] 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy - at
org.apache.shardingsphere.singletable.datanode.SingleTableDataNodeLoader.loadSchemaTableNames(SingleTableDataNodeLoader.java:109)
[INFO ] 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy - at
org.apache.shardingsphere.singletable.datanode.SingleTableDataNodeLoader.load(SingleTableDataNodeLoader.java:78)
[INFO ] 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy - at
org.apache.shardingsphere.singletable.datanode.SingleTableDataNodeLoader.load(SingleTableDataNodeLoader.java:62)
[INFO ] 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy - at
org.apache.shardingsphere.singletable.rule.SingleTableRule.<init>(SingleTableRule.java:70)
[INFO ] 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy - at
org.apache.shardingsphere.singletable.rule.builder.SingleTableRuleBuilder.build(SingleTableRuleBuilder.java:40)
[INFO ] 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy - at
org.apache.shardingsphere.singletable.rule.builder.SingleTableRuleBuilder.build(SingleTableRuleBuilder.java:35)
[INFO ] 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy - at
org.apache.shardingsphere.infra.rule.builder.schema.SchemaRulesBuilder.buildRules(SchemaRulesBuilder.java:56)
[INFO ] 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy - at
org.apache.shardingsphere.mode.metadata.MetaDataContextsBuilder.getDatabaseRules(MetaDataContextsBuilder.java:106)
[INFO ] 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy - at
org.apache.shardingsphere.mode.metadata.MetaDataContextsBuilder.addDatabase(MetaDataContextsBuilder.java:85)
[INFO ] 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy - at
org.apache.shardingsphere.mode.manager.ContextManager.buildChangedMetaDataContextWithAddedDataSource(ContextManager.java:534)
[INFO ] 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy - at
org.apache.shardingsphere.mode.manager.ContextManager.refreshMetaDataContext(ContextManager.java:507)
[INFO ] 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy - at
org.apache.shardingsphere.mode.manager.ContextManager.addResource(ContextManager.java:281)
[INFO ] 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy - at
org.apache.shardingsphere.proxy.backend.text.distsql.rdl.resource.AddResourceBackendHandler.execute(AddResourceBackendHandler.java:63)
[INFO ] 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy - at
org.apache.shardingsphere.proxy.backend.text.distsql.rdl.resource.AddResourceBackendHandler.execute(AddResourceBackendHandler.java:44)
[INFO ] 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy - at
org.apache.shardingsphere.proxy.backend.text.DatabaseRequiredBackendHandler.execute(DatabaseRequiredBackendHandler.java:51)
[INFO ] 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy - at
org.apache.shardingsphere.proxy.frontend.postgresql.command.query.simple.PostgreSQLComQueryExecutor.execute(PostgreSQLComQueryExecutor.java:76)
[INFO ] 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy - at
org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.executeCommand(CommandExecutorTask.java:107)
[INFO ] 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy - at
org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.run(CommandExecutorTask.java:77)
[INFO ] 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy - at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[INFO ] 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy - at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[INFO ] 2022-05-16 05:00:39.430 [docker-java-stream--350783097]
:Scaling-Proxy - at java.lang.Thread.run(Thread.java:748)
```
### Reason analyze (If you can)
I'm not sure, but i execute `ADD RESOURCE` dist sql in the code, just like
```
protected void addResource(final Connection connection, final String
username, final String password) throws SQLException {
connection.createStatement().execute("ADD RESOURCE ds_0
(URL='jdbc:postgresql://127.0.0.1:5432/ds_0',USER=postgres,PASSWORD=root))");
connection.createStatement().execute("ADD RESOURCE ds_1
(URL='jdbc:postgresql://127.0.0.1:5432/ds_1',USER=postgres,PASSWORD=root))");
connection.createStatement().execute("ADD RESOURCE ds_2
(URL='jdbc:postgresql://127.0.0.1:5432/ds_2',USER=postgres,PASSWORD=root))");
connection.createStatement().execute("ADD RESOURCE ds_3
(URL='jdbc:postgresql://127.0.0.1:5432/ds_3',USER=postgres,PASSWORD=root))");
connection.createStatement().execute("ADD RESOURCE ds_4
(URL='jdbc:postgresql://127.0.0.1:5432/ds_4',USER=postgres,PASSWORD=root))");
}
```
So, between add resource operations, very short intervals;
And I test the code in `psql`
```
DROP RESOURCE ds_0 IGNORE SINGLE TABLES;
DROP RESOURCE ds_1 IGNORE SINGLE TABLES;
DROP RESOURCE ds_2 IGNORE SINGLE TABLES;
DROP RESOURCE ds_3 IGNORE SINGLE TABLES;
DROP RESOURCE ds_4 IGNORE SINGLE TABLES;
ADD RESOURCE ds_0
(URL="jdbc:postgresql://127.0.0.1:5432/ds_0",USER=postgres,PASSWORD=root);
ADD RESOURCE ds_1
(URL="jdbc:postgresql://127.0.0.1:5432/ds_1",USER=postgres,PASSWORD=root);
ADD RESOURCE ds_2
(URL="jdbc:postgresql://127.0.0.1:5432/ds_2",USER=postgres,PASSWORD=root);
ADD RESOURCE ds_3
(URL="jdbc:postgresql://127.0.0.1:5432/ds_3",USER=postgres,PASSWORD=root);
ADD RESOURCE ds_4
(URL="jdbc:postgresql://127.0.0.1:5432/ds_4",USER=postgres,PASSWORD=root);
```
if repeat execute the follows code in a very short period of time, some
exception will be thrown
```
[ERROR] 2022-05-16 17:00:21.109 [Connection-3-ThreadExecutor]
o.a.s.p.b.t.d.r.r.AddResourceBackendHandler - Add resource failed
java.lang.IllegalArgumentException: HikariDataSource HikariDataSource
(HikariPool-68) has been closed.
at
org.apache.shardingsphere.infra.database.type.DatabaseTypeEngine.getDatabaseType(DatabaseTypeEngine.java:101)
at
org.apache.shardingsphere.infra.database.type.DatabaseTypeEngine.getDatabaseType(DatabaseTypeEngine.java:90)
at
org.apache.shardingsphere.infra.metadata.ShardingSphereMetaData.createResource(ShardingSphereMetaData.java:75)
at
org.apache.shardingsphere.infra.metadata.ShardingSphereMetaData.create(ShardingSphereMetaData.java:69)
at
org.apache.shardingsphere.mode.metadata.MetaDataContextsBuilder.getMetaDataMap(MetaDataContextsBuilder.java:130)
at
org.apache.shardingsphere.mode.metadata.MetaDataContextsBuilder.build(MetaDataContextsBuilder.java:117)
at
org.apache.shardingsphere.mode.manager.ContextManager.buildChangedMetaDataContextWithAddedDataSource(ContextManager.java:537)
at
org.apache.shardingsphere.mode.manager.ContextManager.refreshMetaDataContext(ContextManager.java:507)
at
org.apache.shardingsphere.mode.manager.ContextManager.addResource(ContextManager.java:281)
at
org.apache.shardingsphere.proxy.backend.text.distsql.rdl.resource.AddResourceBackendHandler.execute(AddResourceBackendHandler.java:63)
at
org.apache.shardingsphere.proxy.backend.text.distsql.rdl.resource.AddResourceBackendHandler.execute(AddResourceBackendHandler.java:44)
at
org.apache.shardingsphere.proxy.backend.text.DatabaseRequiredBackendHandler.execute(DatabaseRequiredBackendHandler.java:51)
at
org.apache.shardingsphere.proxy.frontend.postgresql.command.query.simple.PostgreSQLComQueryExecutor.execute(PostgreSQLComQueryExecutor.java:76)
at
org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.executeCommand(CommandExecutorTask.java:107)
at
org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.run(CommandExecutorTask.java:77)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.sql.SQLException: HikariDataSource HikariDataSource
(HikariPool-68) has been closed.
at
com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:96)
at
org.apache.shardingsphere.infra.database.type.DatabaseTypeEngine.getDatabaseType(DatabaseTypeEngine.java:98)
```
another exception is
```
[ERROR] 2022-05-16 17:00:22.048 [Connection-3-ThreadExecutor]
o.a.s.p.f.c.CommandExecutorTask - Exception occur:
org.apache.shardingsphere.infra.distsql.exception.resource.RequiredResourceMissedException:
Resources [ds_0] do not exist in database sharding_db.
at
org.apache.shardingsphere.proxy.backend.text.distsql.rdl.resource.DropResourceBackendHandler.lambda$checkResourceNameExisted$1(DropResourceBackendHandler.java:71)
at
org.apache.shardingsphere.infra.distsql.exception.DistSQLException.predictionThrow(DistSQLException.java:43)
at
org.apache.shardingsphere.proxy.backend.text.distsql.rdl.resource.DropResourceBackendHandler.checkResourceNameExisted(DropResourceBackendHandler.java:71)
at
org.apache.shardingsphere.proxy.backend.text.distsql.rdl.resource.DropResourceBackendHandler.check(DropResourceBackendHandler.java:63)
at
org.apache.shardingsphere.proxy.backend.text.distsql.rdl.resource.DropResourceBackendHandler.execute(DropResourceBackendHandler.java:55)
at
org.apache.shardingsphere.proxy.backend.text.distsql.rdl.resource.DropResourceBackendHandler.execute(DropResourceBackendHandler.java:46)
at
org.apache.shardingsphere.proxy.backend.text.DatabaseRequiredBackendHandler.execute(DatabaseRequiredBackendHandler.java:51)
at
org.apache.shardingsphere.proxy.frontend.postgresql.command.query.simple.PostgreSQLComQueryExecutor.execute(PostgreSQLComQueryExecutor.java:76)
at
org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.executeCommand(CommandExecutorTask.java:107)
at
org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.run(CommandExecutorTask.java:77)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
```
So, maybe the problem is ContextManager refresh metadata asynchronous? Is it
concurrently insecure?
### Steps to reproduce the behavior, such as: SQL to execute, sharding rule
configuration, when exception occur etc.
PostgreSQL: 12
### Example codes for reproduce this issue (such as a github link).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail:
[email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
