[
https://issues.apache.org/jira/browse/IMPALA-13566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17936934#comment-17936934
]
ASF subversion and git services commented on IMPALA-13566:
----------------------------------------------------------
Commit d0127c2401fb5acccb33d3be87dabb79c976edc7 in impala's branch
refs/heads/master from Xuebin Su
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=d0127c240 ]
IMPALA-13566: Expose query cancellation status to UDFs
Previously, the evaluation of a UDF was not interruptible. When the
impalad was evaluating a UDF on a batch, the client had to wait
for the end of processing the batch before it could get the results or
cancel the query. Such a period could be long if the UDF takes a long
time to run, e.g. the sleep() UDF.
This patch tries to mitigate this issue by exposing the query
cancellation status in RuntimeState to the UDF so that the UDF can check
the status by itself and return early. This can significantly reduce the
waiting time when cancelling a long-running query.
As an example, this patch makes the sleep() UDF interruptible by
checking the query cancellation status inside the UDF every 200ms,
controlled by variable SLEEP_UNINTERRUPTIBLE_INTERVAL_MS. This means
that we only need to wait about 200ms to cancel the query.
Testing:
- Added a new test case in tests/query_test/test_cancellation.py to
ensure that we can interrupt sleep().
- Added a new connection class called MinimalHS2Connection in
tests/common/impala_connection.py to support manipulating one
operation from multiple connections for the new test case.
Change-Id: I9430167f7e46bbdf66153abb4645541cd8cf0142
Reviewed-on: http://gerrit.cloudera.org:8080/22280
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Csaba Ringhofer <[email protected]>
> Expose query cancellation status to UDF interface
> -------------------------------------------------
>
> Key: IMPALA-13566
> URL: https://issues.apache.org/jira/browse/IMPALA-13566
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Riza Suminto
> Assignee: Xuebin Su
> Priority: Major
>
> Some Impala UDF implementation can be complex and time consuming. If UDF
> evaluation happen in uninterruptible call path like in Scan Predicate
> evaluation, UDF may block query fragment cancellation for long time. We
> should consider exposing query cancellation status to UDF interface. UDF
> implementation can periodically check for this status and fail fast upon
> finding cancellation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
