[
https://issues.apache.org/jira/browse/KYLIN-2436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hongbin ma updated KYLIN-2436:
------------------------------
Description:
Kylin's aggregation operator can spill intermediate results to disk when its
estimated memory usage exceeds some threshold (kylin.query.coprocessor.mem.gb
to be specific). While it's a useful feature in general to prevent RegionServer
from OOM, there are times when aborting this kind of memory-hungry query
immediately is a more suitable choice to users.
To accommodate this requirement, I suggest adding a new configuration named
-*kylin.storage.hbase.coprocessor-spill-enabled*-
+*kylin.storage.partition.aggr-spill-enabled*+. The default value would be
true, which will keep the same behavior as before. If changed to false, query
that uses more aggregation memory than threshold will fail immediately.
was:
Kylin's aggregation operator can spill intermediate results to disk when its
estimated memory usage exceeds some threshold (kylin.query.coprocessor.mem.gb
to be specific). While it's a useful feature in general to prevent RegionServer
from OOM, there are times when aborting this kind of memory-hungry query
immediately is a more suitable choice to users.
To accommodate this requirement, I suggest adding a new configuration named
*kylin.storage.hbase.coprocessor-spill-enabled*. The default value would be
true, which will keep the same behavior as before. If changed to false, query
that uses more aggregation memory than threshold will fail immediately.
> add a configuration knob to disable spilling of aggregation cache
> -----------------------------------------------------------------
>
> Key: KYLIN-2436
> URL: https://issues.apache.org/jira/browse/KYLIN-2436
> Project: Kylin
> Issue Type: Improvement
> Components: Storage - HBase
> Affects Versions: v1.6.0
> Reporter: Dayue Gao
> Assignee: Dayue Gao
> Fix For: v2.0.0
>
>
> Kylin's aggregation operator can spill intermediate results to disk when its
> estimated memory usage exceeds some threshold (kylin.query.coprocessor.mem.gb
> to be specific). While it's a useful feature in general to prevent
> RegionServer from OOM, there are times when aborting this kind of
> memory-hungry query immediately is a more suitable choice to users.
> To accommodate this requirement, I suggest adding a new configuration named
> -*kylin.storage.hbase.coprocessor-spill-enabled*-
> +*kylin.storage.partition.aggr-spill-enabled*+. The default value would be
> true, which will keep the same behavior as before. If changed to false, query
> that uses more aggregation memory than threshold will fail immediately.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
