[
https://issues.apache.org/jira/browse/HBASE-22457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16846258#comment-16846258
]
Lars Hofhansl commented on HBASE-22457:
---------------------------------------
> No, but this is no more or less fast then the close then open we do for
> 'alter' processing. It would be implemented the same way, ideally.
We came to that conclusion as well in a discussion in the office. Just alter
some minor thing on the Table/ColumnDescriptor so that all regions are
closed/reopened resulting "Wouldn't it be nice if we had a tool that could do
that without forcing us to change something." :)
> Scanner wrapping is a key thing. Without it I don't think Phoenix works.
Oh totally agree. Though perhaps it is structurally somehow possible to ensure
that passed scanner is either wrapped or closed. I can't think of anything,
though.
> Harden the HBase HFile reader reference counting
> ------------------------------------------------
>
> Key: HBASE-22457
> URL: https://issues.apache.org/jira/browse/HBASE-22457
> Project: HBase
> Issue Type: Brainstorming
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Priority: Major
> Attachments: 22457-random-1.5.txt
>
>
> The problem that any coprocessor hook that replaces a passed scanner without
> closing it can cause an incorrect reference count.
> This was bad and wrong before of course, but now it has pretty bad
> consequences, since an incorrect reference could will prevent HFiles from
> being archived indefinitely.
> All hooks that are passed a scanner and return a scanner are suspect, since
> the returned scanner may or may not close the passed scanner:
> * preCompact
> * preCompactScannerOpen
> * preFlush
> * preFlushScannerOpen
> * preScannerOpen
> * preStoreScannerOpen
> * preStoreFileReaderOpen...? (not sure about this one, it could mess with the
> reader)
> I sampled the Phoenix and also Tephra code, and found a few instances where
> this is happening.
> And for those I filed issued: TEPHRA-300, PHOENIX-5291
> (We're not using Tephra)
> The Phoenix ones should be rare. In our case we are seeing readers with
> refCount > 1000.
> Perhaps there are other issues, a path where not all exceptions are caught
> and scanner is left open that way perhaps. (Generally I am not a fan of
> reference counting in complex systems - it's too easy to miss something. But
> that's a different discussion. :) ).
> Let's brainstorm some way in which we can harden this.
> [~ram_krish], [~anoop.hbase], [~apurtell]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
