Eugene Koifman created HIVE-16361:
-------------------------------------
Summary: Automatically kill runaway processes
Key: HIVE-16361
URL: https://issues.apache.org/jira/browse/HIVE-16361
Project: Hive
Issue Type: Improvement
Components: Transactions
Affects Versions: 1.0.0
Reporter: Eugene Koifman
Priority: Critical
HIVE-13249 added an enforceable limit on how many transactions can be opened
concurrently where the system starts to reject new work to prevent the system
getting to a point where it cannot manage the load.
Another condition to guard against is a runaway process (which would usually be
some app (e.g. Storm) using Streaming Ingest API) that create a very large
number of transactions very quickly all of which immediately get aborted due to
some misconfiguration. This can cause large amount of metatdata to accumulate
in the ACID system slowing everything down and causing instability.
Now that we have TXNS.TXN_AGENT_INFO information we could probably use that
refuse work from a client even before we open any txns if it passes some
"runaway client" heuristic.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
