[
https://issues.apache.org/jira/browse/PHOENIX-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16077286#comment-16077286
]
Geoffrey Jacoby commented on PHOENIX-3997:
------------------------------------------
[~jamestaylor] [~sergey.soldatov] - there are two issues being discussed here,
and it's important not to conflate them. There's a copy/paste of the identical
flush waiting code in two different places:
1. commitBatchWithHTable() - This was introduced as part of PHOENIX-3271, and
is used for committing UPSERT SELECTs that select from one table and commit to
a _different_ table. In this case, the flush waiting code does not seem to
serve a purpose, because it will not create additional flushes on the Region
the coprocessor is executing on.
2. commitBatch(). This was introduced as part of PHOENIX-3111 to prevent a
split deadlock issue. (PHOENIX-3827 then fixed another one.) Something like it
is still necessary given the way ReentrantReadWriteLocks are still a little bit
fair even in NonFair mode (they won't starve a write lock at the top of the
queue, which makes deadlocks possible.)
*This patch only deals with case 1.*
[~apurtell] has also suggested a refactoring of Case 2's logic to instead use
the built-in logic HRegions have for waiting for flushes and compactions, which
seems to me a good idea, but something to do in a separate JIRA because the
behavior between the current code and the HRegion method isn't quite the same
(no explicit timeout in Region/HRegion, for example, and no check for the
region closing, which would probably have to be maintained in some form in the
coprocessor) and that would need more testing.
> UngroupedAggregateRegionObserver.commitBatchWithHTable() should not check the
> memstore size and wait for flush.
> ----------------------------------------------------------------------------------------------------------------
>
> Key: PHOENIX-3997
> URL: https://issues.apache.org/jira/browse/PHOENIX-3997
> Project: Phoenix
> Issue Type: Bug
> Reporter: Thomas D'Silva
> Assignee: Geoffrey Jacoby
> Fix For: 4.12.0
>
> Attachments: PHOENIX-3997.patch
>
>
> [~ankit.singhal]
> In UngroupedAggregateRegionObserver.commitBatchWithHTable() do we need to
> check the memstore size and wait for the flush. We are using a htable to
> write the mutations.
> {code}
> // When memstore size reaches blockingMemstoreSize we are waiting 3 seconds
> for the
> // flush happen which decrease the memstore size and then writes
> allowed on the region.
> for (int i = 0; region.getMemstoreSize().get() > blockingMemstoreSize
> && i < 30; i++) {
> try {
> checkForRegionClosing();
> Thread.sleep(100);
> } catch (InterruptedException e) {
> Thread.currentThread().interrupt();
> throw new IOException(e);
> }
> }
> logger.debug("Committing batch of " + mutations.size() + " mutations
> for " + table);
> try {
> table.batch(mutations);
> } catch (InterruptedException e) {
> throw new RuntimeException(e);
> }
> {code}
> FYI [~jamestaylor] [~apurtell]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
