[
https://issues.apache.org/jira/browse/HIVE-13249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15246639#comment-15246639
]
Eugene Koifman commented on HIVE-13249:
---------------------------------------
[~wzheng]
1. I think HIVE_MAX_OPEN_TXNS should higher by default. 50K? 100K? (10 nodes
streaming into table with 100 partitions and txn batches of 10)
2. HIVE_COUNT_OPEN_TXNS_INTERVAL This should probably be 1s.
3. TxnHandler.countOpenTxns()
- Seems like this should be at least a WARN _LOG.debug("Transaction database
not properly configured, " + ..._
- _numOpenTxns = new AtomicLong(rs.getLong(1));_ why does this need AtomicLong?
Seems like volatile would be sufficient
- Seems like _LOG.debug("Failed to update number of open transactions");_
should be at least at info
- this method should probably have retry logic like all public methods (e.g.
_addDynamicPartitions()_ )
4. TxnHandler
- shouldn't _maxOpenTxns_ and _tooManyOpenTxns_ be static just like
_numOpenTxns_?
5. TxnHandler.openTxns()
- Perhaps the exception message should read something like "Maximum allowed
number of open transactions has been reached. See hive.max.open.txns."
- This should probably be logged at WARN before throwing (include current
number of open txns)
6. AcidOpenTxnCountService
- Why is _getStartDelayMs()_ using HIVE_COUNT_OPEN_TXNS_INTERVAL. Could
probably just hardcode the value
7. TestTxnCommadns2. testOpenTxnsCounter
- please see runHouseKeeperService() - it has logic to prevent it from spinning
forever. In fact, you can just use this method and pass in the instance of the
the new service.
- it would make sense if this test also committed or aborted some txns to make
sure following openTxns() succeeds
> Hard upper bound on number of open transactions
> -----------------------------------------------
>
> Key: HIVE-13249
> URL: https://issues.apache.org/jira/browse/HIVE-13249
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
> Affects Versions: 2.0.0
> Reporter: Wei Zheng
> Assignee: Wei Zheng
> Attachments: HIVE-13249.1.patch, HIVE-13249.2.patch
>
>
> We need to have a safeguard by adding an upper bound for open transactions to
> avoid huge number of open-transaction requests, usually due to improper
> configuration of clients such as Storm.
> Once that limit is reached, clients will start failing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
