[
https://issues.apache.org/jira/browse/KYLIN-1500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15219455#comment-15219455
]
Zhong Yanghong commented on KYLIN-1500:
---------------------------------------
Thanks Mahong for this confirmation. Previously I'm afraid disjoint gaps are
filled parallelly, for each gap is assigned a thread. However, from log, I saw
they are filled sequentially. I don't know where this is implemented. Could you
give a clue?
I'll divide long gap into several pieces for automation later.
> streaming_fillgap cause out of memory
> -------------------------------------
>
> Key: KYLIN-1500
> URL: https://issues.apache.org/jira/browse/KYLIN-1500
> Project: Kylin
> Issue Type: Improvement
> Components: streaming
> Affects Versions: v1.4.0
> Reporter: qianqiaoneng
>
> streaming_fillgap.sh will try to fill gap when there is some error of the
> streaming job. But no matter how long the gap it is, how big the missing data
> it is, kylin tried to fill gap in one thread, which sometimes cause out of
> memory error. Need streaming_fillgap.sh to be more intelligent based on gap
> time range and data volume.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
