[
https://issues.apache.org/jira/browse/HIVE-13249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196611#comment-15196611
]
Eugene Koifman commented on HIVE-13249:
---------------------------------------
AcidHouseKeeper uses a ScheduledExecutorService. It run multiple tasks on
separate schedules. I don't think there was ever a proposal to run any
threads on client side.
The idea Wei and I discussed was to run a single thread per metastore JVM to
count number of txns periodically and check the computed value in
TxnHandler.openTxnx() each time it's called.
I think this is conceptually the same as your idea.
It's easy enough to have HouseKeeper run multiple tasks, but it complicates
testing since it makes it harder to just run one iteration of a particular
task. We'd need to do some refactoring in HouseKeepers to make sure this is
possible - then they can be combined into a single HouseKeeper that runs
multiple periodic tasks.
Wei, I said earlier that putting this computation in AcidHouseKeeper was a bad
idea but I was wrong. Since there is a single AcidHouseKeeper per JVM, the
task that it runs can easily just set a static variable on TxnHandler with
results of the computation which openTxns() can read.
As far as testing, look at TestTxnHandler for example, there are multiple
examples openTxns() calls. In fact each call can open many txnxs at once.
TestTxnCommands.testTimeOutReaper() has an example on how to run the
HouseKeeper in UT, but like I said, you'd need to refactor it a bit if you want
to run multiple tasks in it.
> Hard upper bound on number of open transactions
> -----------------------------------------------
>
> Key: HIVE-13249
> URL: https://issues.apache.org/jira/browse/HIVE-13249
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
> Affects Versions: 2.0.0
> Reporter: Wei Zheng
> Assignee: Wei Zheng
> Attachments: HIVE-13249.1.patch
>
>
> We need to have a safeguard by adding an upper bound for open transactions to
> avoid huge number of open-transaction requests, usually due to improper
> configuration of clients such as Storm.
> Once that limit is reached, clients will start failing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
