[
https://issues.apache.org/jira/browse/HBASE-18221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051781#comment-16051781
]
ramkrishna.s.vasudevan commented on HBASE-18221:
------------------------------------------------
I verified with a test case. Ya the problem is not directly by
CompacitonHFileDischarger but it is due to the fact that a compaction happens
when we try to switch to pread and recreate scanners.
But [~Apache9] - even if we try to add the compacted files also - still there
is a gap between the time the compacted files gets updated in
StorefileManager#addCompactionResults() and the time we get the compactedFile
list.
Am saying this because I even tried that but still there is gap so the best
way is to protect the trySwitchToPread under HStore's reentrant lock - then we
are fine. And I don't find anything wrong there because every scanner is
created under HStore's lock only.
What is your opinion? Thanks.
> Switch from pread to stream should happen under HStore's reentrant lock
> -----------------------------------------------------------------------
>
> Key: HBASE-18221
> URL: https://issues.apache.org/jira/browse/HBASE-18221
> Project: HBase
> Issue Type: Sub-task
> Components: Scanners
> Affects Versions: 2.0.0, 3.0.0, 2.0.0-alpha-1
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0, 3.0.0, 2.0.0-alpha-2
>
>
> Found this while debugging HBASE-18186. When we try to reopen the scanners on
> the storefiles while trying to switch over from pread to stream, we do not
> use the HStore's reentrant lock to get the current Storefiles from the
> StoreFileManager. All the scan APIs are guarded under that and we must do it
> here also other wise the CompactedHfileDischarger may cause race issues with
> the HStore's datastructures like here
> {code}
> 2017-06-14 18:16:17,223 WARN
> [RpcServer.default.FPBQ.Fifo.handler=23,queue=1,port=16020]
> regionserver.StoreScanner: failed to switch to stream read
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.getScannersForStoreFiles(StoreFileScanner.java:133)
> at
> org.apache.hadoop.hbase.regionserver.HStore.getScanners(HStore.java:1221)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.trySwitchToStreamRead(StoreScanner.java:997)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.shipped(StoreScanner.java:1134)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.shipped(KeyValueHeap.java:445)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.shipped(HRegion.java:6459)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices$RegionScannerShippedCallBack.run(RSRpcServices.java:339)
> at
> org.apache.hadoop.hbase.ipc.ServerCall.setResponse(ServerCall.java:252)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:166)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:278)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:258)
> {code}
> I have a working patch fixing this problem. Will do some more testing and try
> to upload the patch after I write a test case for this.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
