[
https://issues.apache.org/jira/browse/HBASE-23349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026433#comment-17026433
]
ramkrishna.s.vasudevan commented on HBASE-23349:
------------------------------------------------
I will list out my points here
-> Generally the scanners created at the Region level due to incoming user
requests have a lease mechanism. It is basically to release the resources. So
any long running scan there will be a point of end for the scan or may be even
if the user
is not responsive or not closing the scan we have the lease mechanism. In any
such case the scanners gets closed and the resources get released.
-> The case where it cannot happen is when CPs create their own scanners. Then
there are chances that if the CP scanner fails or does not release the
resources we may hold up the underlying resources and even recover lease will
not work
-> One way to solve this is to have a lease mechanism for CP scans also so that
we don end up in scans being alive for a longer time.
-> Coming to the benefit of the ref count based mechanism, it solves the sync
block issues which was happening for every next call. But not only that we have
two other benefits
For the first benefit, pls refer to this user mailing list
http://apache-hbase.679495.n3.nabble.com/Extremely-long-flush-times-td4104190.html
http://mail-archives.apache.org/mod_mbox/hbase-dev/201208.mbox/%3c6548f17059905b48b2a6f28ce3692baa0ce29...@oaexch4server.oa.oclc.org%3E
(see the later part of it).
We are helping the readers to carry on without the impact of flushes and the
reverse way where flushes are not blocked due to readers. Here the scans are
heavier where either you have filters applied, more deletes/versions to skip
through.
In such cases having a non sync way of readers always helps.
The other benefit is that, the current readers need not reset itself, load
the new files(after compaction which may not be in cache), reseek to the last
fetched row and then again proceed with the scan. Obviously it means that
the next scan that comes will have to anyway read from the filesystem and
then load to the cache but atleast the ongoing scans are not impacted.
-> Finally I would like to mention that Phoenix like cases where there could be
a query that reads large amount of data and has filtering applied along with
heavy writes, it may be obvious that we may face the issues
as the users have faced in the mailing list.
I am fine if every body agrees to revert the patch and put back the sync way of
readers (or any other better soln). Just saying because it should be giving a
view that am in favour of the exisiting behaviour and against any changes to it.
> Low refCount preventing archival of compacted away files
> --------------------------------------------------------
>
> Key: HBASE-23349
> URL: https://issues.apache.org/jira/browse/HBASE-23349
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 3.0.0, 2.3.0, 1.6.0
> Reporter: Viraj Jasani
> Assignee: Viraj Jasani
> Priority: Major
> Fix For: 3.0.0, 2.3.0, 1.6.0
>
>
> We have observed that refCount on compacted away store files as low as 1 is
> prevent archival.
> {code:java}
> regionserver.HStore - Can't archive compacted file
> hdfs://{{root-dir}}/hbase/data/default/t1/12a9e1112e0371955b3db8d3ebb2d298/cf1/73b72f5ddfce4a34a9e01afe7b83c1f9
> because of either isCompactedAway=true or file has reference,
> isReferencedInReads=true, refCount=1, skipping for now.
> {code}
> We should come up with core code (run as part of discharger thread)
> gracefully resolve reader lock issue by resetting ongoing scanners to start
> pointing to new store files instead of compacted away store files.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
