[jira] [Updated] (HIVE-14778) document threading model of Streaming API
[ https://issues.apache.org/jira/browse/HIVE-14778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-14778: -- Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) pushed to master https://github.com/apache/hive/commit/20304c0705c4ad861b5915dacceaa6d6bdfe91fc Thanks Alan for the review > document threading model of Streaming API > - > > Key: HIVE-14778 > URL: https://issues.apache.org/jira/browse/HIVE-14778 > Project: Hive > Issue Type: Bug > Components: HCatalog, Transactions >Affects Versions: 0.14.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Fix For: 2.2.0 > > Attachments: HIVE-14778.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > The model is not obvious and needs to be documented properly. > A StreamingConnection internally maintains 2 MetaStoreClient objects (each > has 1 Thrift client for actual RPC). Let's call them "primary" and > "heartbeat". Each TransactionBatch created from a given StreamingConnection, > gets a reference to both of these MetaStoreClients. > So the model is that there is at most 1 outstanding (not closed) > TransactionBatch for any given StreamingConnection and for any given > TransactionBatch there can be at most 2 threads accessing it concurrently. 1 > thread calling TransactionBatch.heartbeat() (and nothing else) and the other > calling all other methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14778) document threading model of Streaming API
[ https://issues.apache.org/jira/browse/HIVE-14778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-14778: -- Attachment: HIVE-14778.patch > document threading model of Streaming API > - > > Key: HIVE-14778 > URL: https://issues.apache.org/jira/browse/HIVE-14778 > Project: Hive > Issue Type: Bug > Components: HCatalog, Transactions >Affects Versions: 0.14.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-14778.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > The model is not obvious and needs to be documented properly. > A StreamingConnection internally maintains 2 MetaStoreClient objects (each > has 1 Thrift client for actual RPC). Let's call them "primary" and > "heartbeat". Each TransactionBatch created from a given StreamingConnection, > gets a reference to both of these MetaStoreClients. > So the model is that there is at most 1 outstanding (not closed) > TransactionBatch for any given StreamingConnection and for any given > TransactionBatch there can be at most 2 threads accessing it concurrently. 1 > thread calling TransactionBatch.heartbeat() (and nothing else) and the other > calling all other methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14778) document threading model of Streaming API
[ https://issues.apache.org/jira/browse/HIVE-14778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-14778: -- Status: Patch Available (was: Open) > document threading model of Streaming API > - > > Key: HIVE-14778 > URL: https://issues.apache.org/jira/browse/HIVE-14778 > Project: Hive > Issue Type: Bug > Components: HCatalog, Transactions >Affects Versions: 0.14.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-14778.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > The model is not obvious and needs to be documented properly. > A StreamingConnection internally maintains 2 MetaStoreClient objects (each > has 1 Thrift client for actual RPC). Let's call them "primary" and > "heartbeat". Each TransactionBatch created from a given StreamingConnection, > gets a reference to both of these MetaStoreClients. > So the model is that there is at most 1 outstanding (not closed) > TransactionBatch for any given StreamingConnection and for any given > TransactionBatch there can be at most 2 threads accessing it concurrently. 1 > thread calling TransactionBatch.heartbeat() (and nothing else) and the other > calling all other methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
