[
https://issues.apache.org/jira/browse/IMPALA-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17230330#comment-17230330
]
Fucun Chu commented on IMPALA-10317:
------------------------------------
[~amansinha] In our scenario, this parameter is configured in the resource pool
queue to prevent analysts from using very bad join conditions in interactive
queries.
Use an external system to collect and monitor time-consuming tasks, notify or
cancel them, EXEC_TIME_LIMIT_S is used in the workflow scheduling system.
There is a similar system variable
[MAX_JOIN_SIZE|https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_max_join_size]
in MySQL。
> Add query option that limits join #rows at runtime
> --------------------------------------------------
>
> Key: IMPALA-10317
> URL: https://issues.apache.org/jira/browse/IMPALA-10317
> Project: IMPALA
> Issue Type: New Feature
> Components: Backend
> Reporter: Fucun Chu
> Assignee: Fucun Chu
> Priority: Major
> Attachments: query82_summary.png
>
>
> Reject queries that rows produced too bigger by join operator when executing
> the query.
> This is a mechanism to protect the cluster from potentially harmful queries.
> When the cardinality of the table is very large and the join conditions are
> very bad, the number of rows produced by the join will be very large,
> sometimes tens of billions, which affects the cluster status and other
> running queries.
> In our environment, the NUM_JOIN_ROWS_PRODUCED_LIMIT query option is added to
> limit the number of rows produced by a single join operator.
> Implementation refers to
> [IMPALA-6034|https://issues.apache.org/jira/browse/IMPALA-6034] and summary
> (see the figure below), check the join operator #rows size
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
