[
https://issues.apache.org/jira/browse/HBASE-21551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell closed HBASE-21551.
---------------------------------------
> Memory leak when use scan with STREAM at server side
> ----------------------------------------------------
>
> Key: HBASE-21551
> URL: https://issues.apache.org/jira/browse/HBASE-21551
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: Zheng Hu
> Assignee: Zheng Hu
> Priority: Blocker
> Fix For: 3.0.0-alpha-1, 2.2.0, 2.1.2, 2.0.4
>
> Attachments: HBASE-21551.v1.patch, HBASE-21551.v2.patch,
> HBASE-21551.v3.patch, heap-dump.jpg
>
>
> We open the RegionServerScanner with STREAM as following:
> {code}
> RegionScannerImpl#initializeScanners
> |---> HStore#getScanner
> |----------> StoreScanner()
> |------->
> StoreFileScanner#getScannersForStoreFiles
> |------>
> HStoreFile#getStreamScanner #1
> {code}
> In #1, we put the StoreFileReader into a concurrent hash map streamReaders,
> but not remove the StreamReader from streamReaders until closing the store
> file.
> So if we scan with stream with so many times, the streamReaders hash map
> will be exploded. we can see the heap dump in the attached heap-dump.jpg.
> I found this bug, because when i benchmark the scan performance by using YCSB
> in a cluster (heap size of RS is 50g), the Rs was easy to occur a long time
> full gc ( ~ 110 sec)....
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
