Beyondeclipse commented on a change in pull request #11345:
URL: https://github.com/apache/shardingsphere/pull/11345#discussion_r673020337
##########
File path:
shardingsphere-infra/shardingsphere-infra-common/src/main/java/org/apache/shardingsphere/infra/metadata/schema/builder/SchemaBuilder.java
##########
@@ -130,15 +137,30 @@ private static void appendRemainTables(final
SchemaBuilderMaterials materials, f
return Optional.empty();
}
- private static void appendDialectRemainTables(final
DialectTableMetaDataLoader dialectLoader, final SchemaBuilderMaterials
materials, final Map<String, TableMetaData> tables) throws SQLException {
+ private static void appendDialectRemainTables(final
DialectTableMetaDataLoader dialectLoader,
+ final SchemaBuilderMaterials materials, final Map<String,
TableMetaData> tables, final Collection<String> ruleLogicTables) throws
SQLException {
+
+ Map<String, String> actualTable2LogicTableMap =
getActualTable2LogicTableMap(materials, ruleLogicTables);
+
Collection<Future<Map<String, TableMetaData>>> futures = new
LinkedList<>();
- Collection<String> existedTables =
getExistedTables(materials.getRules(), tables);
for (DataSource each : materials.getDataSourceMap().values()) {
- futures.add(EXECUTOR_SERVICE.submit(() -> dialectLoader.load(each,
existedTables)));
+ futures.add(EXECUTOR_SERVICE.submit(() -> dialectLoader.load(each,
Collections.emptyList())));
Review comment:
Because 'dialectLoader.load' is more efficient and I want it to load all
the tables. After that , I can simply change the sharding-tables to
logic-tables by 'actualTable2LogicTableMap' without connecting db.
##########
File path:
shardingsphere-infra/shardingsphere-infra-common/src/main/java/org/apache/shardingsphere/infra/metadata/schema/builder/SchemaBuilder.java
##########
@@ -84,43 +89,45 @@
private static Map<String, TableMetaData>
buildActualTableMetaDataMap(final SchemaBuilderMaterials materials) throws
SQLException {
Map<String, TableMetaData> result = new
HashMap<>(materials.getRules().size(), 1);
- appendRemainTables(materials, result);
- for (ShardingSphereRule rule : materials.getRules()) {
- if (rule instanceof TableContainedRule) {
- for (String table : ((TableContainedRule) rule).getTables()) {
- if (!result.containsKey(table)) {
- TableMetaDataBuilder.load(table,
materials).map(optional -> result.put(table, optional));
+ Collection<String> ruleLogicTables = materials.getRules().stream()
+ .filter(rule -> rule instanceof TableContainedRule)
+ .flatMap(rule -> ((TableContainedRule)
rule).getTables().stream())
+ .collect(Collectors.toSet());
+ appendRemainTables(materials, result, ruleLogicTables);
+
+ // Append more tables in 'TableContainedRule' by multi-thread
+ Collection<String> toAppendLoadTables = ruleLogicTables.stream()
+ .filter(table -> !result.containsKey(table))
+ .collect(Collectors.toList());
+ if (CollectionUtils.isNotEmpty(toAppendLoadTables)) {
+ Collection<Future<Optional<TableMetaData>>> futures = new
ArrayList<>(toAppendLoadTables.size());
+ for (String table : toAppendLoadTables) {
+ futures.add(EXECUTOR_SERVICE.submit(() ->
TableMetaDataBuilder.load(table, materials)));
+ }
+ for (Future<Optional<TableMetaData>> each : futures) {
+ try {
+ each.get().map(optional -> result.put(optional.getName(),
optional));
+ } catch (final InterruptedException | ExecutionException ex) {
+ if (ex.getCause() instanceof SQLException) {
+ throw (SQLException) ex.getCause();
}
+ throw new ShardingSphereException(ex);
}
}
}
return result;
}
- private static void appendRemainTables(final SchemaBuilderMaterials
materials, final Map<String, TableMetaData> tables) throws SQLException {
+ private static void appendRemainTables(final SchemaBuilderMaterials
materials, final Map<String, TableMetaData> tables,
+ final Collection<String> ruleLogicTables) throws SQLException {
Optional<DialectTableMetaDataLoader> dialectLoader =
findDialectTableMetaDataLoader(materials);
if (dialectLoader.isPresent()) {
- appendDialectRemainTables(dialectLoader.get(), materials, tables);
+ appendDialectRemainTables(dialectLoader.get(), materials, tables,
ruleLogicTables);
return;
}
appendDefaultRemainTables(materials, tables);
}
- private static Map<String, TableMetaData> buildLogicTableMetaDataMap(final
SchemaBuilderMaterials materials, final Map<String, TableMetaData> tables)
throws SQLException {
Review comment:
I just moved this method to the bottom, since it was in the methods
belong to 'buildActualTableMetaDataMap'.
##########
File path:
shardingsphere-infra/shardingsphere-infra-common/src/main/java/org/apache/shardingsphere/infra/metadata/schema/builder/SchemaBuilder.java
##########
@@ -84,43 +89,45 @@
private static Map<String, TableMetaData>
buildActualTableMetaDataMap(final SchemaBuilderMaterials materials) throws
SQLException {
Map<String, TableMetaData> result = new
HashMap<>(materials.getRules().size(), 1);
- appendRemainTables(materials, result);
- for (ShardingSphereRule rule : materials.getRules()) {
- if (rule instanceof TableContainedRule) {
- for (String table : ((TableContainedRule) rule).getTables()) {
- if (!result.containsKey(table)) {
- TableMetaDataBuilder.load(table,
materials).map(optional -> result.put(table, optional));
+ Collection<String> ruleLogicTables = materials.getRules().stream()
+ .filter(rule -> rule instanceof TableContainedRule)
+ .flatMap(rule -> ((TableContainedRule)
rule).getTables().stream())
+ .collect(Collectors.toSet());
+ appendRemainTables(materials, result, ruleLogicTables);
+
+ // Append more tables in 'TableContainedRule' by multi-thread
Review comment:
OK
##########
File path:
shardingsphere-infra/shardingsphere-infra-common/src/main/java/org/apache/shardingsphere/infra/metadata/schema/builder/SchemaBuilder.java
##########
@@ -84,43 +89,45 @@
private static Map<String, TableMetaData>
buildActualTableMetaDataMap(final SchemaBuilderMaterials materials) throws
SQLException {
Map<String, TableMetaData> result = new
HashMap<>(materials.getRules().size(), 1);
- appendRemainTables(materials, result);
- for (ShardingSphereRule rule : materials.getRules()) {
- if (rule instanceof TableContainedRule) {
- for (String table : ((TableContainedRule) rule).getTables()) {
- if (!result.containsKey(table)) {
- TableMetaDataBuilder.load(table,
materials).map(optional -> result.put(table, optional));
+ Collection<String> ruleLogicTables = materials.getRules().stream()
+ .filter(rule -> rule instanceof TableContainedRule)
+ .flatMap(rule -> ((TableContainedRule)
rule).getTables().stream())
+ .collect(Collectors.toSet());
+ appendRemainTables(materials, result, ruleLogicTables);
Review comment:
I will extract these code to method 'appendRemainRuleLogicTables', and
ruleLogicTables is useful for exclude loaded tables.
##########
File path:
shardingsphere-infra/shardingsphere-infra-common/src/main/java/org/apache/shardingsphere/infra/metadata/schema/builder/SchemaBuilder.java
##########
@@ -84,43 +89,45 @@
private static Map<String, TableMetaData>
buildActualTableMetaDataMap(final SchemaBuilderMaterials materials) throws
SQLException {
Map<String, TableMetaData> result = new
HashMap<>(materials.getRules().size(), 1);
- appendRemainTables(materials, result);
- for (ShardingSphereRule rule : materials.getRules()) {
- if (rule instanceof TableContainedRule) {
- for (String table : ((TableContainedRule) rule).getTables()) {
- if (!result.containsKey(table)) {
- TableMetaDataBuilder.load(table,
materials).map(optional -> result.put(table, optional));
+ Collection<String> ruleLogicTables = materials.getRules().stream()
+ .filter(rule -> rule instanceof TableContainedRule)
+ .flatMap(rule -> ((TableContainedRule)
rule).getTables().stream())
+ .collect(Collectors.toSet());
+ appendRemainTables(materials, result, ruleLogicTables);
+
+ // Append more tables in 'TableContainedRule' by multi-thread
+ Collection<String> toAppendLoadTables = ruleLogicTables.stream()
+ .filter(table -> !result.containsKey(table))
+ .collect(Collectors.toList());
+ if (CollectionUtils.isNotEmpty(toAppendLoadTables)) {
+ Collection<Future<Optional<TableMetaData>>> futures = new
ArrayList<>(toAppendLoadTables.size());
Review comment:
OK.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
