Xuebin Su has uploaded a new patch set (#18). ( http://gerrit.cloudera.org:8080/22280 )
Change subject: IMPALA-13566: Expose query cancellation status to UDFs ...................................................................... IMPALA-13566: Expose query cancellation status to UDFs Previously, the evaluation of a UDF was not interruptible. When the impalad was evaluating a UDF on a batch, the client had to wait for the end of processing the batch before it could get the results or cancel the query. Such a period could be long if the UDF takes a long time to run, e.g. the sleep() UDF. This patch tries to mitigate this issue by exposing the query cancellation status in RuntimeState to the UDF so that the UDF can check the status by itself and return early. This can significantly reduce the waiting time when cancelling a long-running query. As an example, this patch makes the sleep() UDF interruptible by checking the query cancellation status inside the UDF every 200ms, controlled by variable SLEEP_UNINTERRUPTIBLE_INTERVAL_MS. This means that we only need to wait about 200ms to cancel the query. Testing: - Added a new test case in tests/query_test/test_cancellation.py to ensure that we can interrupt sleep(). - Added a new connection class called MinimalHS2Connection in tests/common/impala_connection.py to support manipulating one operation from multiple connections for the new test case. Change-Id: I9430167f7e46bbdf66153abb4645541cd8cf0142 --- M be/src/exprs/utility-functions-ir.cc M be/src/udf/udf.cc M be/src/udf/udf.h M tests/common/impala_connection.py M tests/query_test/test_cancellation.py 5 files changed, 185 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/80/22280/18 -- To view, visit http://gerrit.cloudera.org:8080/22280 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9430167f7e46bbdf66153abb4645541cd8cf0142 Gerrit-Change-Number: 22280 Gerrit-PatchSet: 18 Gerrit-Owner: Xuebin Su <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Surya Hebbar <[email protected]> Gerrit-Reviewer: Xuebin Su <[email protected]>
