[
https://issues.apache.org/jira/browse/IMPALA-12878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824584#comment-17824584
]
ASF subversion and git services commented on IMPALA-12878:
----------------------------------------------------------
Commit 54642b2549c38625c868f09815b61170e568cdb9 in impala's branch
refs/heads/master from Michael Smith
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=54642b254 ]
IMPALA-12878: Report invalid query if finalized
Fixes a case introduced by IMPALA-12493 where a query is closed again in
the middle of unregistering - which appears to be possible with
cancel_query_and_validate_state - returns "Query not yet running"
instead of "Invalid or unknown query handle".
IMPALA-12493 moved checks for whether a query is inflight before an
atomic CompareAndSwap to mark a QueryDriver as finalized. As part of
that change, a query that was Finalized and removed from
inflight_queries - but not yet removed from query_driver_map_ - could
report "Query not yet running". Updates the check for that error to also
verify the query has not been finalized; if it has, we let the next
conditional handle that case and return "Invalid or unknown query
handle".
Also ensures cancelled queries are closed in test_web_pages tests so
assertions that num_in_flight_queries == 0 at the start of tests are
valid when running the whole suite serially. This doesn't come up in
automated tests because the cases that care about that assertion are
marked execute_serially, while the cases that failed to close a query
were not.
Testing: adds a test to test_cancellation that ensures we get the
expected "Invalid or unknown query handle" when close() is delayed in
calling QueryDriver::Finalize.
Change-Id: I3bf910f499147a09352f9dcb755037b0d8616dfd
Reviewed-on: http://gerrit.cloudera.org:8080/21110
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> TestResultSpoolingCancellation.test_cancellation failed in UBSAN build
> ----------------------------------------------------------------------
>
> Key: IMPALA-12878
> URL: https://issues.apache.org/jira/browse/IMPALA-12878
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 4.4.0
> Reporter: Riza Suminto
> Assignee: Michael Smith
> Priority: Major
> Labels: broken-build
> Fix For: Impala 4.4.0
>
>
> TestResultSpoolingCancellation.test_cancellation failed in UBSAN build. The
> following are the error messages.
> {code:java}
> Error Message
> query_test/test_result_spooling.py:356: in test_cancellation
> vector.get_value('cancel_delay')) util/cancel_util.py:99: in
> cancel_query_and_validate_state assert 'Invalid or unknown query handle'
> in str(close_error) E assert 'Invalid or unknown query handle' in
> "ImpalaBeeswaxException:\n INNER EXCEPTION: <class
> 'beeswaxd.ttypes.BeeswaxException'>\n MESSAGE: Query not yet running\n" E
> + where "ImpalaBeeswaxException:\n INNER EXCEPTION: <class
> 'beeswaxd.ttypes.BeeswaxException'>\n MESSAGE: Query not yet running\n" =
> str(ImpalaBeeswaxException())
> Stacktrace
> query_test/test_result_spooling.py:356: in test_cancellation
> vector.get_value('cancel_delay'))
> util/cancel_util.py:99: in cancel_query_and_validate_state
> assert 'Invalid or unknown query handle' in str(close_error)
> E assert 'Invalid or unknown query handle' in "ImpalaBeeswaxException:\n
> INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'>\n MESSAGE: Query
> not yet running\n"
> E + where "ImpalaBeeswaxException:\n INNER EXCEPTION: <class
> 'beeswaxd.ttypes.BeeswaxException'>\n MESSAGE: Query not yet running\n" =
> str(ImpalaBeeswaxException())
> Standard Error
> SET
> client_identifier=query_test/test_result_spooling.py::TestResultSpoolingCancellation::()::test_cancellation[query:selectl_returnflagfromtpch_parquet.lineitem|protocol:beeswax|exec_option:{'test_replan':1;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disa;
> -- executing against localhost:21000
> use tpch_parquet;
> -- 2024-03-05 23:36:03,429 INFO MainThread: Started query
> 8241403abf8b7dbb:4db1deee00000000
> SET
> client_identifier=query_test/test_result_spooling.py::TestResultSpoolingCancellation::()::test_cancellation[query:selectl_returnflagfromtpch_parquet.lineitem|protocol:beeswax|exec_option:{'test_replan':1;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disa;
> SET spool_query_results=true;
> SET test_replan=1;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET exec_single_node_rows_threshold=0;
> -- executing async: localhost:21000
> select l_returnflag from tpch_parquet.lineitem;
> -- 2024-03-05 23:36:03,437 INFO MainThread: Started query
> 4a44182cfdd8b9ff:657a80a800000000
> SET
> client_identifier=query_test/test_result_spooling.py::TestResultSpoolingCancellation::()::test_cancellation[query:selectl_returnflagfromtpch_parquet.lineitem|protocol:beeswax|exec_option:{'test_replan':1;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disa;
> -- connecting to: localhost:21000
> -- 2024-03-05 23:36:03,437 INFO Thread-24: Could not connect to ('::1',
> 21000, 0, 0)
> Traceback (most recent call last):
> File
> "/data/jenkins/workspace/impala-cdw-master-staging-core-ubsan/repos/Impala/infra/python/env-gcc10.4.0/lib/python2.7/site-packages/thrift/transport/TSocket.py",
> line 137, in open
> handle.connect(sockaddr)
> File
> "/data/jenkins/workspace/impala-cdw-master-staging-core-ubsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/python-2.7.16/lib/python2.7/socket.py",
> line 228, in meth
> return getattr(self._sock,name)(*args)
> error: [Errno 111] Connection refused
> -- fetching results from: <tests.common.impala_connection.OperationHandle
> object at 0x7fd2b172a410>
> -- getting state for operation:
> <tests.common.impala_connection.OperationHandle object at 0x7fd2b172a410>
> -- canceling operation: <tests.common.impala_connection.OperationHandle
> object at 0x7fd2b172a410>
> -- closing query for operation handle:
> <tests.common.impala_connection.OperationHandle object at 0x7fd2b172a410>
> -- getting runtime profile operation:
> <tests.common.impala_connection.OperationHandle object at 0x7fd2b172a410>
> {code}
> Error message "Query not yet running" is recently introduced in IMPALA-12493.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
