[
https://issues.apache.org/jira/browse/HBASE-23349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990068#comment-16990068
]
Viraj Jasani edited comment on HBASE-23349 at 12/6/19 6:38 PM:
---------------------------------------------------------------
Thanks [~ram_krish]
What you are suggesting is notify scanner to reset by discharger thread itself
(scanner reset used to happen only before HBASE-13082 right?) Also, as of now
notifyChangedReadersObservers() is called only during store flush, so if we
start using it by discharger thread, the thread should try to reset heap for
all the scanners and not for specific one right? Since the thread might not
have context of a specific scanner. And if so, we can directly reset refCount
to 0 for all compacted away store files for a given store? Because once we
reset heap and lastTop, may be we don't need to worry about refCount?
To sum up, by default, if archival of store files can't be done due to refCount
> 0 till 2 min of discharger thread run, the thread should immediately run
notifyChangedReadersObservers() which should reset heap for all existing
scanners. Please let me know if my understanding is not correct.
In a way this could take care of open scanners gracefully.
was (Author: vjasani):
Thanks [~ram_krish]
What you are suggesting is notify scanner to reset by discharger thread itself
(scanner reset used to happen only before HBASE-13082 right?) Also, as of now
notifyChangedReadersObservers() is called only during store flush, so if we
start using it by discharger thread, the thread should try to reset heap for
all the scanners and not for specific one right? Since the thread might not
have context of a specific scanner. And if so, we can directly reset refCount
to 0 for all compacted away store files for a given store? Because once we
reset heap and lastTop, may be we don't need to worry about refCount?
To sum up, by default, if archival of store files can't be done due to refCount
> 0 till 2 min of discharger thread run, the thread should immediately run
notifyChangedReadersObservers() which should reset heap for all existing
scanners. Please let me know if my understanding is not correct.
> Reader lock on compacted store files preventing archival of compacted files
> ---------------------------------------------------------------------------
>
> Key: HBASE-23349
> URL: https://issues.apache.org/jira/browse/HBASE-23349
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 3.0.0, 2.3.0, 1.6.0
> Reporter: Viraj Jasani
> Assignee: Viraj Jasani
> Priority: Major
> Fix For: 3.0.0, 2.3.0, 1.6.0
>
> Attachments: HBASE-23349.master.000.patch,
> HBASE-23349.master.001.patch, HBASE-23349.master.002.patch
>
>
> refCounts on compacted away store files as low as 1 can also prevent archival.
> {code:java}
> regionserver.HStore - Can't archive compacted file
> hdfs://{{root-dir}}/hbase/data/default/t1/12a9e1112e0371955b3db8d3ebb2d298/cf1/73b72f5ddfce4a34a9e01afe7b83c1f9
> because of either isCompactedAway=true or file has reference,
> isReferencedInReads=true, refCount=1, skipping for now.
> {code}
> We should come up with core code blocking reader lock if client or
> coprocessor has held the lock for significantly high amount of
> time(configurable - mostly same as discharger thread interval) or gracefully
> resolve reader lock issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
