Shaofeng SHI commented on KYLIN-3140:

Okay I see; you want to control the concurrent new jobs, but don't want to it 
be impacted by the merge jobs. The auto-merge job can not be discarded easily 
as once the old is discarded, a new will be sumbmitted immediately.

> Auto merge jobs should not block user build jobs
> ------------------------------------------------
>                 Key: KYLIN-3140
>                 URL: https://issues.apache.org/jira/browse/KYLIN-3140
>             Project: Kylin
>          Issue Type: Improvement
>          Components: Job Engine
>            Reporter: Wang, Gang
>            Assignee: Shaofeng SHI
>            Priority: Major
> Although in the latest version, Kylin support concurrent jobs. If the 
> concurrency is set to 1, there is some possibility that cube build job will 
> have dead lock. Say, when there is some issue which causes merge job failed, 
> even when you discard the job, another job will be launched and failed again 
> due to auto merge policy. And this failed merge job blocks user to build 
> incremental segment.
> Even if the concurrency is set to larger than 1, the auto merge jobs occupy 
> some concurrency quota. 
> While, from user perspective, they don't care much about the auto merge jobs, 
> and the auto merge jobs should not block the building/refresh jobs they 
> submit manually.
> A better way may be separating the auto merge jobs from the job queue, 
> parameter max-building-segments only limit jobs submitted by users.

This message was sent by Atlassian JIRA

Reply via email to