[
https://issues.apache.org/jira/browse/KYLIN-5595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoxiang Yu resolved KYLIN-5595.
---------------------------------
Resolution: Fixed
> [kylin 5.0] Launch Job Node not initialize spark session issue
> --------------------------------------------------------------
>
> Key: KYLIN-5595
> URL: https://issues.apache.org/jira/browse/KYLIN-5595
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine
> Affects Versions: 5.0-beta
> Reporter: Li Can
> Assignee: Li Can
> Priority: Major
> Fix For: 5.0-beta
>
> Attachments: image (87).png, image (88).png
>
>
> Saving model will execute 'checkFlatTableSql' method at job node, and it
> will not skip the step by default. When execute 'checkFlatTableSql' method,
> it will initialize spark session if the job node just started, the process of
> getting spark session costs too much time.
> The pic 87 shows that get spark session costs more than 63s, and the
> execution of checking sql costs more than 2s, it is not friendly for saving
> model first time after node launched, and it is also unreasonable.
> So I suggest that the job node's process of spark session initialization
> should be consistent with the query node, it means that the spark session
> should be initialized as the node just started. And the spark session is a
> singleton model, just need once initialization, as the pic 88 display.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
