Janaki Lahorani created IMPALA-8082:
---------------------------------------
Summary: Save intermediate state and data if applicable
Key: IMPALA-8082
URL: https://issues.apache.org/jira/browse/IMPALA-8082
Project: IMPALA
Issue Type: Improvement
Reporter: Janaki Lahorani
When a query is stalled, it will be beneficial to flush the state and if needed
data as well, to disk (temporary space) so that the query can be suspended and
resources can be freed. The query can resume execution at a later point when
it becomes un-stalled. The amount of space that can be used should probably be
configurable. There should be life cycle management to clean up this space and
abort stalled queries. In reality, this space will be quite big. If it is
getting filled up then there is a problem that needs to be analyzed and
addressed - may be in code, or in terms of management and logistics at
deployment. Consequently necessary tools, logging and diagnostics should be
built in tandem.
When a query crashes, it could potentially affect many queries that are running
in that process. It looks like end user is required to manually restart all
these queries. If there is an infrastructure that saved stages, then the
non-crashed queries could be restarted from a saved point and these could be
used to finish running the queries without requiring a user intervention.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
