[jira] [Commented] (KYLIN-5267) force limit to protect the service when SQL has no limit
[ https://issues.apache.org/jira/browse/KYLIN-5267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17612011#comment-17612011 ] liyang commented on KYLIN-5267: --- If what you need is something beyond the above two configs, feel free to propose and PR. > force limit to protect the service when SQL has no limit > > > Key: KYLIN-5267 > URL: https://issues.apache.org/jira/browse/KYLIN-5267 > Project: Kylin > Issue Type: Improvement > Components: Query Engine >Affects Versions: v4.0.1 >Reporter: Liu Zhao >Priority: Minor > > Kylin4 queries use Spark, and the result data will be sent to the driver > (that is, query service). If the user's SQL returns a large data set, it may > cause the query oom to be unavailable. To solve this problem, can we provide > a means for users to enforce limit when SQL has no limit to protect services. > I am here https://issues.apache.org/jira/browse/KYLIN-2649 You can see the > idea above, but can reduce the limit on the SQL structure？ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5267) force limit to protect the service when SQL has no limit
[ https://issues.apache.org/jira/browse/KYLIN-5267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17612009#comment-17612009 ] liyang commented on KYLIN-5267: --- As a result of KYLIN-2649, a config parameter called "kylin.query.force-limit" is already added. Does that help your use case? Also "kylin.query.scan-threshold" is another config parameter that is often used to limit big queries going too wild. > force limit to protect the service when SQL has no limit > > > Key: KYLIN-5267 > URL: https://issues.apache.org/jira/browse/KYLIN-5267 > Project: Kylin > Issue Type: Improvement > Components: Query Engine >Affects Versions: v4.0.1 >Reporter: Liu Zhao >Priority: Minor > > Kylin4 queries use Spark, and the result data will be sent to the driver > (that is, query service). If the user's SQL returns a large data set, it may > cause the query oom to be unavailable. To solve this problem, can we provide > a means for users to enforce limit when SQL has no limit to protect services. > I am here https://issues.apache.org/jira/browse/KYLIN-2649 You can see the > idea above, but can reduce the limit on the SQL structure？ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5267) force limit to protect the service when SQL has no limit
[ https://issues.apache.org/jira/browse/KYLIN-5267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605865#comment-17605865 ] zhaoliu commented on KYLIN-5267: 如果有需要，我可以修复此问题 > force limit to protect the service when SQL has no limit > > > Key: KYLIN-5267 > URL: https://issues.apache.org/jira/browse/KYLIN-5267 > Project: Kylin > Issue Type: Improvement > Components: Query Engine >Affects Versions: v4.0.1 >Reporter: zhaoliu >Priority: Minor > > Kylin4查询使用spark，结果数据会被发送到driver（即query上），如果用户的sql会返回极大的数据集，那么就可能导致query > oom从而服务不可用。针对这个问题，我们能否提供一种手段给用户，让其可以对没有limit的语句强制limit，以保护服务。我在 > https://issues.apache.org/jira/browse/KYLIN-2649 上看到这个思路，但是否可以对sql结构再降低些限制呢 -- This message was sent by Atlassian Jira (v8.20.10#820010)
