[
https://issues.apache.org/jira/browse/PHOENIX-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16077271#comment-16077271
]
James Taylor commented on PHOENIX-3997:
---------------------------------------
Ah, yes - thanks for the reminder, [~sergey.soldatov]. See also this comment on
the lock member variable:
{code}
/**
* This lock used for synchronizing the state of
* {@link UngroupedAggregateRegionObserver#scansReferenceCount},
* {@link UngroupedAggregateRegionObserver#isRegionClosing} variables used
to avoid possible
* dead lock situation in case below steps:
* 1. We get read lock when we start writing local indexes, deletes etc..
* 2. when memstore reach threshold, flushes happen. Since they use read
(shared) lock they
* happen without any problem until someone tries to obtain write lock.
* 3. at one moment we decide to split/bulkload/close and try to acquire
write lock.
* 4. Since that moment all attempts to get read lock will be blocked. I.e.
no more
* flushes will happen. But we continue to fill memstore with local index
batches and
* finally we get RTBE.
*
* The solution to this is to not allow or delay operations acquire the
write lock.
* 1) In case of split we just throw IOException so split won't happen but
it will not cause any harm.
* 2) In case of bulkload failing it by throwing the exception.
* 3) In case of region close by balancer/move wait before closing the
reason and fail the query which
* does write after reading.
*
* See PHOENIX-3111 for more info.
*/
{code}
> UngroupedAggregateRegionObserver.commitBatchWithHTable() should not check the
> memstore size and wait for flush.
> ----------------------------------------------------------------------------------------------------------------
>
> Key: PHOENIX-3997
> URL: https://issues.apache.org/jira/browse/PHOENIX-3997
> Project: Phoenix
> Issue Type: Bug
> Reporter: Thomas D'Silva
> Assignee: Geoffrey Jacoby
> Fix For: 4.12.0
>
> Attachments: PHOENIX-3997.patch
>
>
> [~ankit.singhal]
> In UngroupedAggregateRegionObserver.commitBatchWithHTable() do we need to
> check the memstore size and wait for the flush. We are using a htable to
> write the mutations.
> {code}
> // When memstore size reaches blockingMemstoreSize we are waiting 3 seconds
> for the
> // flush happen which decrease the memstore size and then writes
> allowed on the region.
> for (int i = 0; region.getMemstoreSize().get() > blockingMemstoreSize
> && i < 30; i++) {
> try {
> checkForRegionClosing();
> Thread.sleep(100);
> } catch (InterruptedException e) {
> Thread.currentThread().interrupt();
> throw new IOException(e);
> }
> }
> logger.debug("Committing batch of " + mutations.size() + " mutations
> for " + table);
> try {
> table.batch(mutations);
> } catch (InterruptedException e) {
> throw new RuntimeException(e);
> }
> {code}
> FYI [~jamestaylor] [~apurtell]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
