hailin.huang created KYLIN-4200:
-----------------------------------
Summary: SparkCubingBylayer is not rubust using CubeStatsReader to
estimateLayerPartitionNum
Key: KYLIN-4200
URL: https://issues.apache.org/jira/browse/KYLIN-4200
Project: Kylin
Issue Type: Bug
Components: Job Engine
Affects Versions: v2.6.4
Reporter: hailin.huang
In prod env, I found some scene as follow:
if user has measure of Bitmap, the Spark tasks often block at job 0,1 which
compute layer 0,1.
after analyze spark log, I found that, spark use CubeStatsReader to
estimateLayerPartition Num, If layer 0 has cuboid 255, and its size is 10M,
according to the default configuration pararm ,the partition will be 1, however
it's quick small for this amout of data.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
