[
https://issues.apache.org/jira/browse/KYLIN-5387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17688766#comment-17688766
]
ASF GitHub Bot commented on KYLIN-5387:
---------------------------------------
liukun4515 commented on code in PR #2072:
URL: https://github.com/apache/kylin/pull/2072#discussion_r1106538769
##########
src/spark-project/engine-spark/src/main/scala/org/apache/kylin/engine/spark/job/stage/build/CostBasedPlanner.scala:
##########
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kylin.engine.spark.job.stage.build
+
+import org.apache.kylin.engine.spark.job.SegmentJob
+import org.apache.kylin.engine.spark.job.stage.BuildParam
+import org.apache.kylin.engine.spark.model.SegmentFlatTableDesc
+import org.apache.kylin.engine.spark.smarter.IndexDependencyParser
+import org.apache.kylin.metadata.cube.cuboid.AdaptiveSpanningTree
+import
org.apache.kylin.metadata.cube.cuboid.AdaptiveSpanningTree.AdaptiveTreeBuilder
+import org.apache.kylin.metadata.cube.model.NDataSegment
+
+class CostBasedPlanner(jobContext: SegmentJob, dataSegment: NDataSegment,
buildParam: BuildParam)
+ extends FlatTableAndDictBase(jobContext, dataSegment, buildParam) {
+ override def execute(): Unit = {
+ val (cost, sourceCount) = generateCostTable()
+ getRecommendedLayoutAndUpdateMetadata(cost, sourceCount)
+
+ val result = jobContext.updateIndexPlanIfNeed()
+ if (result) {
+ // update span tree and table desc with the new build job layouts
Review Comment:
partition and no-partition have different logic and different construct for
the `AdaptiveSpanningTree` and `SegmentFlatTableDesc`, they are hard to be
combined together.
On the original codebase, the code for partition and no-partition are
separated, such as `PartitionMaterializedFactTableView` and
`MaterializedFactTableView`.
> Migrate cube planner phase 1 to kylin5
> --------------------------------------
>
> Key: KYLIN-5387
> URL: https://issues.apache.org/jira/browse/KYLIN-5387
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine
> Affects Versions: 5.0-alpha
> Reporter: Kun Liu
> Assignee: Kun Liu
> Priority: Major
>
> kylin3.1 support the cube planner to recommend cuboid when building the first
> segment.
>
> We need to migrate the cost based algorithm to the kylin5, and leverage the
> cube planner algorithm with the index in kylin5
>
> designe doc:
> https://docs.google.com/document/d/1wUNd8U1u-w8T-qQUReplPhDFkETj3AtFr_Shex501ls/edit?usp=sharing
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
