[
https://issues.apache.org/jira/browse/KYLIN-6061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17930200#comment-17930200
]
Guoliang Sun commented on KYLIN-6061:
-------------------------------------
h3. Dev Design
Enable Gluten for Build and Asynchronous Query
- Add parameters to determine whether to enable Gluten:
- `kylin.engine.gluten.enabled`: Controls whether to use Gluten for building.
Default is `false` (Gluten is not used for building).
- `kylin.unique-async-query.gluten.enabled`: Controls whether to use Gluten
for asynchronous queries. Default is `false` (Gluten is not used for
asynchronous queries).
When Not Using Gluten for Build/Asynchronous Query:
- Exclude Gluten-related parameters when generating `spark-submit`.
- In `exchangeSparkConf`, exclude Gluten-related parameters when retrieving
Spark build parameters.
When Using Gluten for Build/Asynchronous Query:
- Retain Gluten-related build parameters.
Mismatch Between Build Task Sub-steps and Actual Execution Sub-steps:
- Add `BuildStepExec` to execute the index build task and include the
`getStageId` method to calculate the `stageId`.
- Default increment for `stageId` is `+1`.
> Support building index files using Gluten
> -----------------------------------------
>
> Key: KYLIN-6061
> URL: https://issues.apache.org/jira/browse/KYLIN-6061
> Project: Kylin
> Issue Type: Improvement
> Affects Versions: 5.0.0
> Reporter: Guoliang Sun
> Assignee: Guoliang Sun
> Priority: Major
> Fix For: 5.0.2
>
>
> h3. Enable Gluten for Build and Asynchronous Query
> - Add the following parameters:
> - `kylin.engine.gluten.enabled`: Controls whether to use Gluten for
> building. Default is `false` (Gluten is not used for building).
> - `kylin.unique-async-query.gluten.enabled`: Controls whether to use Gluten
> for asynchronous queries. Default is `false` (Gluten is not used for
> asynchronous queries).
> Mismatch Between Build Task Sub-steps and Actual Execution Sub-steps
> - The flat table statistics update is partially unaccounted for in the time
> statistics.
> - According to the build logs, the most time-consuming parts are global
> dictionary generation and flat table creation, which do not align with the
> page display.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
