liyang created KYLIN-2136:
-----------------------------
Summary: Enhance cubing algorithm selection
Key: KYLIN-2136
URL: https://issues.apache.org/jira/browse/KYLIN-2136
Project: Kylin
Issue Type: Improvement
Reporter: liyang
Assignee: liyang
To be more protective to select in-mem cubing.
There were reports that in-mem cubing was selected but run slowly (even
timeout) in a few small PoC clusters. Apart from the limited resource in PoC,
another important cause is the algorithm selection didn't concern the scale of
the job. When mappers are many but mapper slots are few, it will take many
rounds to finish all the mappers, and the total time of in-mem cubing becomes
slow.
The scale of the job should be taken into consideration when auto selecting
cubing algorithm. Namely a mapper limit configuration is introduced, to stop
in-mem cubing when the number of mappers goes beyond the threshold.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
