[
https://issues.apache.org/jira/browse/HBASE-22309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack updated HBASE-22309:
----------------------------------
Issue Type: Improvement (was: Bug)
Priority: Critical (was: Major)
Summary: Replace Shipper Interface with Netty's ReferenceCounted; add
ExtendCell#retain/ExtendCell#release (was: Replace the Shipper interface by
using ExtendCell#retain or ExtendCell#release)
> Replace Shipper Interface with Netty's ReferenceCounted; add
> ExtendCell#retain/ExtendCell#release
> -------------------------------------------------------------------------------------------------
>
> Key: HBASE-22309
> URL: https://issues.apache.org/jira/browse/HBASE-22309
> Project: HBase
> Issue Type: Improvement
> Reporter: Zheng Hu
> Assignee: Zheng Hu
> Priority: Critical
>
> We've some discussion about the Shipper interface.
> {code}
> /**
> * This interface denotes a scanner as one which can ship cells. Scan
> operation do many RPC requests
> * to server and fetch N rows/RPC. These are then shipped to client. At the
> end of every such batch
> * {@link #shipped()} will get called.
> */
> @InterfaceAudience.Private
> public interface Shipper {
> /**
> * Called after a batch of rows scanned and set to be returned to client.
> Any in between cleanup
> * can be done here.
> */
> void shipped() throws IOException;
> }
> {code}
> it seems not an elegant ways...for example:
> 1. we want to keep the previous cell in the scanner, we must deep clone
> the kv before an shipping, otherwise the ship will free all the ByteBuffers,
> the prevCell will point to an unknown area.
> 2. when switch from PREAD to STREAM in a long scan, we also have accomplish
> this in shipped() method. if not , once we close the PREAD scanner, the
> un-shipped cell will also point to an unknown memory area, because the
> scanner closing free all ByteBuffes.
> ....
> If we change to use refCnt to manage the RPC memory release or retain, we
> can just call prevCell.retain.. then it memory won't be free unless the
> prevCell reach the end of life and call prevCell#release. I mean we can
> replace all the shipper logic by using cell#release and cell#retain.
> One concern is about the API, actually the ExtendCell is an pure server side
> type, we can make the ExtendCell extend the Netty's ReferenceCounted
> interface , and provide an retain() and release() methods. we won't maitain
> an refCnt in ExtendCell, the refCnt is still in HFileBlock. Once we encoded
> an ExtendCell to CellScanner, we can release the extendCell, and it will
> release its blocks backend... so in theory, will have no performance loss...
> Anyway, that would be a big change, so maybe need to create another new
> feature branch to address this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
