[
https://issues.apache.org/jira/browse/IMPALA-7031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16550510#comment-16550510
]
Adriano commented on IMPALA-7031:
---------------------------------
Hi [~tarmstrong], sorry to answer with delay.
I made some tests on Impala 2.10 (C5.13, but most probably this behaviour is
only in the next versions):
* from the Impala WebUi we can only cancel the query in flight, in this case:
!Screen Shot 2018-07-20 at 10.19.42.png!
* the coordinator will cancel the query as requested:
{code:java}
I0720 01:03:52.288815 28385 status.cc:122] Cancelled from Impala's debug web
interface
@ 0x83e789 impala::Status::Status()
@ 0xabce21 impala::ImpalaHttpHandler::CancelQueryHandler()
@ 0xbdb7da impala::Webserver::RenderUrlWithTemplate()
@ 0xbdcd27 impala::Webserver::BeginRequestCallback()
@ 0xbe9a70 (unknown)
@ 0xbec1ed (unknown)
@ 0xbec87d (unknown)
@ 0x7f301df39aa1 start_thread
@ 0x7f301dc86c4d clone
I0720 01:03:52.288822 28385 impala-server.cc:989] UnregisterQuery():
query_id=944b1add28b19313:eb712db100000000
I0720 01:03:52.288844 28385 impala-server.cc:1075] Cancel():
query_id=944b1add28b19313:eb712db100000000
I0720 01:03:52.288862 28385 coordinator.cc:910] Cancel()
query_id=944b1add28b19313:eb712db100000000
I0720 01:03:52.288868 28385 coordinator-backend-state.cc:367] sending
CancelQueryFInstances rpc for query_id=944b1add28b19313:eb712db100000000
backend=host-10-17-102-61.coe.cloudera.com:22000
I0720 01:03:52.288873 28385 client-cache.cc:46]
GetClient(host-10-17-102-61.coe.cloudera.com:22000)
I0720 01:03:52.288877 28385 client-cache.cc:56] GetClient(): returning cached
client for host-10-17-102-61.coe.cloudera.com:22000
I0720 01:03:52.288935 28314 rpc-trace.cc:190] RPC call:
ImpalaInternalService.CancelQueryFInstances(from 10.17.102.61:57249)
I0720 01:03:52.288945 28314 impala-internal-service.cc:63]
CancelQueryFInstances(): query_id=944b1add28b19313:eb712db100000000
{code}
* On the client side the query fail with the error below:
{code:java}
java.sql.SQLException: [Simba][ImpalaJDBCDriver](500051) ERROR processing
query/statement. Error Code: 0, SQL state: TStatus(statusCode:ERROR_STATUS,
sqlState:HY000, errorMessage:Invalid query handle:
944b1add28b19313:eb712db100000000), Query: SELECT COUNT(*) as `expr_0` FROM
`db`.`table`.
at com.cloudera.hivecommon.api.HS2Client.executeStatementInternal(Unknown
Source)
at com.cloudera.hivecommon.api.HS2Client.executeStatement(Unknown Source)
at
com.cloudera.hivecommon.dataengine.HiveJDBCNativeQueryExecutor.executeQuery(Unknown
Source)
at
com.cloudera.hivecommon.dataengine.HiveJDBCDSIExtQueryExecutor.execute(Unknown
Source)
at com.cloudera.jdbc.common.SStatement.executeNoParams(Unknown Source)
at com.cloudera.jdbc.common.SStatement.executeQuery(Unknown Source)
at JdbcTest.runTest(JdbcTest.java:43)
Caused by: com.cloudera.support.exceptions.GeneralException:
[Simba][ImpalaJDBCDriver](500051) ERROR processing query/statement. Error Code:
0, SQL state: TStatus(statusCode:ERROR_STATUS, sqlState:HY000,
errorMessage:Invalid query handle: 944b1add28b19313:eb712db100000000), Query:
SELECT COUNT(*) as `expr_0` FROM `db`.`table`.
... 7 more{code}
* Because the coordinator provide this answer if the query is closed/cancelled
{code:java}
I0720 01:03:52.301568 28384 status.cc:55] ReportExecStatus(): Received report
for unknown query ID (probably closed or cancelled):
944b1add28b19313:eb712db100000000
@ 0x83d66a impala::Status::Status()
@ 0xa90176 impala::ImpalaServer::ReportExecStatus()
@ 0xd4b60e
impala::ImpalaInternalServiceProcessor::process_ReportExecStatus()
@ 0xd47859 impala::ImpalaInternalServiceProcessor::dispatchCall()
@ 0x80e4ac apache::thrift::TDispatchProcessor::process()
@ 0x9d9edf
apache::thrift::server::TAcceptQueueServer::Task::run()
@ 0x9d44f9 impala::ThriftThread::RunRunnable()
@ 0x9d52d2
boost::detail::function::void_function_obj_invoker0<>::invoke()
@ 0xbd3fa2 impala::Thread::SuperviseThread()
@ 0xbd4704 boost::detail::thread_data<>::run()
@ 0xe608ea (unknown)
@ 0x7f301df39aa1 start_thread
@ 0x7f301dc86c4d clone
{code}
>From the client/user prospective the error "Invalid query handle" is not
>indicative and they think that something is broken (ignoring that someone else
>cancelled the query using the Administrative WebUi). If in the future we can
>provide some more info on the reason why the query failed will be good.
PS: I was able to replicate also the "waiting to be closed" state but there we
have no errors on the client side (the effect is that the query will stay in
waiting to be closed on the coordinator.
This happen when a multi-threads client submit a query with the thread 'A' and
another thread 'B' cancel 'A' (manual intervention by the user on the client
that cancel the operation). But in this scenario the client have no errors (the
side effect is that the query will remain in "waiting to be closed" on the
impala coordinator (holding resources). I reproduced this scenario with Tableau
Desktop, when you open this application it start immediately to submit queries
and fetch data and typically the user cancel often this first operation (so
they can work on something different) and the effect is a number of
queries(depending by the number of the clients) in "waiting to be closed" on
the coordinator. I found that there was a driver that triggered automatically
an '{color:#000000}ImpalaAutoCloseOperation{color}' (in this case the driver
will close always the query cancelled from the client and this 'waiting to be
closed' will not happen anymore, iirc this will be implemented from the next
drivers for impala after July 2018).
Hope that it can be of help.
> Detailed failure reason for clients when a query is cancelled from the WebUi
> ----------------------------------------------------------------------------
>
> Key: IMPALA-7031
> URL: https://issues.apache.org/jira/browse/IMPALA-7031
> Project: IMPALA
> Issue Type: New Feature
> Reporter: Adriano
> Priority: Major
> Attachments: Screen Shot 2018-07-20 at 10.19.42.png
>
>
> In big clusters with many jdbc/odbc users, in order to save resources are
> often implemented scripts that automatically cancel queries (e.g. long
> running queries) (the scripts typically are using the Impala Webui).
> Typical Scenario:
> # A jdbc/odbc client submit a query
> # The Coordinator start the query execution
> # The query is cancelled from the Coordinator WebUi
> # The jdbc/odbc client ask to the Coordinator the query status
> (GetOperationStatus)
> # The Coordinator answer "unknown query ID" (as the query was cancelled)
> # For the client perspective the query failed for "unknown query ID"
> Currently, if a running query is cancelled from the impalad WebUI, the client
> will just receive an 'unknown query ID' error on the next
> fetch/getOperationStatus attempt. It would be good to be able to explicitly
> call out this case.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
