Joe McDonnell created IMPALA-13294:
--------------------------------------
Summary: Add option to use long polling for
get_state/GetOperationStatus
Key: IMPALA-13294
URL: https://issues.apache.org/jira/browse/IMPALA-13294
Project: IMPALA
Issue Type: Task
Components: Backend, Clients
Affects Versions: Impala 4.5.0
Reporter: Joe McDonnell
Assignee: Joe McDonnell
Currently, Impala does an execute call, then the client polls waiting for the
operation to finish (or error out). The client sleeps between polls, and this
sleep time can be a substantial percentage of a short query's execution time.
Long polling allows status RPCs like HS2's GetOperationStatus and Beeswax's
get_state to wait for query completion on the server side for a configurable
amount of time. If the query completes during that time, the client can be
notified immediately. If the query does not complete, then the wait times out
and the client does a new status RPC.
Clients can adjust to this functionality by keeping track of how much time was
spent in the status RPC, then only doing a client side sleep if the RPC was
shorter than the desired sleep time. This allows a client to maintain its old
behavior with old Impalas that don't have long polling while avoiding
unnecessary sleeps when using long polling.
Hive has used long polling for a long time (see HIVE-5217).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
