[
https://issues.apache.org/jira/browse/PHOENIX-938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14056785#comment-14056785
]
Jesse Yates commented on PHOENIX-938:
-------------------------------------
Started looking at this today.... at first, 0.98.3 is pretty simple. Then it
starts to get complicated as 0.98.4 completely changed how rpc scheduling is
implemented (HBASE-11355). I don't know if we have the bandwidth to continually
monitor all the possible changes to the scheduler code to support this.
Further, as we look to real transactions, this implementation becomes somewhat
moot; maybe we just leave the code as-is?
The nitty-gritty of the details is that 0.98.4 introduced the idea of an
RpcExecutor (which is a great improvement over the current munging), but that
isn't in 0.98.3, so we would either need to port that class to phoenix (losing
any updates from the HBase community), but that's kinda already what I was
doing with this patch, so maybe that's alright for now.
Now, we could have a whole reflection framework to support the different HBase
versions we are running (which becomes a testing pain, but doable) and then
pick the most optimal one (0.98.4+ just uses RpcExecutor as-is, 0.98.3 uses the
copied code, <=0.98.2 ignores). Or we can copy the changed implementations back
and just use the same thing everywhere, but we loose out on changes... There
really isn't a clean solution here :-/
Really, this stems from the RpcScheduler code being a private interface in
HBase but wanting to leverage it outside HBase.
thoughts [~jamestaylor]?
> Use higher priority queue for index updates to prevent deadlock
> ---------------------------------------------------------------
>
> Key: PHOENIX-938
> URL: https://issues.apache.org/jira/browse/PHOENIX-938
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.0.0, 4.1
> Reporter: James Taylor
> Assignee: Jesse Yates
> Fix For: 5.0.0, 4.1
>
> Attachments: phoenix-938-4.0-v0.patch, phoenix-938-master-v0.patch,
> phoenix-938-master-v1.patch
>
>
> With our current global secondary indexing solution, a batched Put of table
> data causes a RS to do a batch Put to other RSs. This has the potential to
> lead to a deadlock if all RS are overloaded and unable to process the pending
> batched Put. To prevent this, we should use a higher priority queue to submit
> these Puts so that they're always processed before other Puts. This will
> prevent the potential for a deadlock under high load. Note that this will
> likely require some HBase 0.98 code changes and would not be feasible to
> implement for HBase 0.94.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
