morrySnow commented on code in PR #31060:
URL: https://github.com/apache/doris/pull/31060#discussion_r1499087668
##########
docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-ASYNC-MATERIALIZED-VIEW.md:
##########
@@ -160,7 +160,7 @@ KEY(k1,k2)
```
##### partition
-There are two types of partitioning methods for materialized views. If no
partitioning is specified, there will be a default single partition. If a
partitioning field is specified, the system will automatically deduce the
source base table of that field and synchronize all partitions of the base
table (currently supporting `OlapTable` and `hive`). (Limitation: the current
base table can only have one partitioning field.)
+There are two types of partitioning methods for materialized views. If no
partitioning is specified, there will be a default single partition. If a
partitioning field is specified, the system will automatically deduce the
source base table of that field and synchronize all partitions of the base
table (currently supporting `OlapTable` and `hive`). (Limitation: If the base
table is an `OlapTable`,it can only have one partition field)
Review Comment:
```suggestion
There are two types of partitioning methods for materialized views. If no
partitioning is specified, there will be a default single partition. If a
partitioning field is specified, the system will automatically deduce the
source base table of that field and synchronize all partitions of the base
table (currently supporting `OlapTable` and `hive`). (Limitation: If the base
table is an `OlapTable`, it can only have one partition field)
```
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/ListPartitionItem.java:
##########
@@ -80,6 +82,17 @@ public PartitionKeyDesc toPartitionKeyDesc() {
return PartitionKeyDesc.createIn(inValues);
}
+ @Override
+ public PartitionKeyDesc toPartitionKeyDesc(int pos) {
+ List<List<PartitionValue>> inValues =
partitionKeys.stream().map(PartitionInfo::toPartitionValue)
+ .collect(Collectors.toList());
+ Set<List<PartitionValue>> res = Sets.newHashSet();
+ for (List<PartitionValue> list : inValues) {
+ res.add(Lists.newArrayList(list.get(pos)));
Review Comment:
should check size here to avoid out of bound exception
##########
fe/fe-core/src/main/java/org/apache/doris/mtmv/MTMVPartitionUtil.java:
##########
@@ -113,26 +112,49 @@ public static void alignMvPartition(MTMV mtmv,
MTMVRelatedTableIf relatedTable)
*
* @param relatedTable
* @param tableProperties
+ * @param relatedCol
* @return
* @throws AnalysisException
*/
public static List<AllPartitionDesc>
getPartitionDescsByRelatedTable(MTMVRelatedTableIf relatedTable,
- Map<String, String> tableProperties) throws AnalysisException {
+ Map<String, String> tableProperties, String relatedCol) throws
AnalysisException {
HashMap<String, String> partitionProperties = Maps.newHashMap();
List<AllPartitionDesc> res = Lists.newArrayList();
- Map<Long, PartitionItem> relatedTableItems =
relatedTable.getPartitionItems();
- for (Entry<Long, PartitionItem> entry : relatedTableItems.entrySet()) {
- PartitionKeyDesc oldPartitionKeyDesc =
entry.getValue().toPartitionKeyDesc();
+ Set<PartitionKeyDesc> relatedPartitionDescs =
getRelatedPartitionDescs(relatedTable, relatedCol);
+ for (PartitionKeyDesc partitionKeyDesc : relatedPartitionDescs) {
SinglePartitionDesc singlePartitionDesc = new
SinglePartitionDesc(true,
- generatePartitionName(oldPartitionKeyDesc),
- oldPartitionKeyDesc, partitionProperties);
+ generatePartitionName(partitionKeyDesc),
+ partitionKeyDesc, partitionProperties);
// mtmv can only has one partition col
singlePartitionDesc.analyze(1, tableProperties);
res.add(singlePartitionDesc);
}
return res;
}
+ private static Set<PartitionKeyDesc>
getRelatedPartitionDescs(MTMVRelatedTableIf relatedTable, String relatedCol)
+ throws AnalysisException {
+ int pos = getPos(relatedTable, relatedCol);
+ Set<PartitionKeyDesc> res = Sets.newHashSet();
+ for (Entry<Long, PartitionItem> entry :
relatedTable.getPartitionItems().entrySet()) {
+ PartitionKeyDesc partitionKeyDesc =
entry.getValue().toPartitionKeyDesc(pos);
+ res.add(partitionKeyDesc);
+ }
+ return res;
+ }
+
+ private static int getPos(MTMVRelatedTableIf relatedTable, String
relatedCol) throws AnalysisException {
+ List<Column> partitionColumns = relatedTable.getPartitionColumns();
+ for (int i = 0; i < partitionColumns.size(); i++) {
+ if
(partitionColumns.get(i).getName().equalsIgnoreCase(relatedCol)) {
+ return i;
+ }
+ }
+ throw new AnalysisException(
+ String.format("getRelatedColPos error, relatedCol: %s,
partitionColumns: %s", relatedCol,
+ partitionColumns));
+ }
Review Comment:
same code in two place?
##########
fe/fe-core/src/main/java/org/apache/doris/mtmv/MTMVPartitionUtil.java:
##########
@@ -54,27 +57,25 @@ public class MTMVPartitionUtil {
*
* @param mtmv
* @param partitionId
+ * @param relatedPartitionIds
* @param tables
* @param excludedTriggerTables
* @return
* @throws AnalysisException
*/
- public static boolean isMTMVPartitionSync(MTMV mtmv, Long partitionId,
Set<BaseTableInfo> tables,
+ public static boolean isMTMVPartitionSync(MTMV mtmv, Long partitionId,
Set<Long> relatedPartitionIds,
+ Set<BaseTableInfo> tables,
Set<String> excludedTriggerTables) throws AnalysisException {
boolean isSyncWithPartition = true;
if (mtmv.getMvPartitionInfo().getPartitionType() ==
MTMVPartitionType.FOLLOW_BASE_TABLE) {
MTMVRelatedTableIf relatedTable =
mtmv.getMvPartitionInfo().getRelatedTable();
// if follow base table, not need compare with related table, only
should compare with related partition
excludedTriggerTables.add(relatedTable.getName());
- PartitionItem item =
mtmv.getPartitionInfo().getItemOrAnalysisException(partitionId);
- Map<Long, PartitionItem> relatedPartitionItems =
relatedTable.getPartitionItems();
- long relatedPartitionId = getExistPartitionId(item,
- relatedPartitionItems);
- if (relatedPartitionId == -1L) {
+ if (CollectionUtils.isEmpty(relatedPartitionIds)) {
LOG.warn("can not found related partition: " + partitionId);
Review Comment:
should we print mtmv name and table name for easy debug?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
