[
https://issues.apache.org/jira/browse/KYLIN-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038183#comment-17038183
]
ASF GitHub Bot commented on KYLIN-4185:
---------------------------------------
nichunen commented on pull request #1005: KYLIN-4185: optimize CuboidSizeMap by
using historical segments
URL: https://github.com/apache/kylin/pull/1005
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> CubeStatsReader estimate wrong cube size
> ----------------------------------------
>
> Key: KYLIN-4185
> URL: https://issues.apache.org/jira/browse/KYLIN-4185
> Project: Kylin
> Issue Type: Improvement
> Reporter: ZhouKang
> Assignee: ZhouKang
> Priority: Major
>
> CubeStatsReader estimate wrong cube size, which cause a lot of problems.
> when the estimated size is much larger than the real size, the spark
> application's executor number is small, and cube build step will take a long
> time. sometime the step will failed due to the large dataset.
> When the estimated size is much smaller than the real size. the cuboid file
> in HDFS is small, and there are much of cuboid file.
>
> In our production environment, both the two situation happened.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
