Zheng Hu created HBASE-22309:
--------------------------------
Summary: Replace the Shipper interface by using ExtendCell#retain
or ExtendCell#release
Key: HBASE-22309
URL: https://issues.apache.org/jira/browse/HBASE-22309
Project: HBase
Issue Type: Bug
Reporter: Zheng Hu
Assignee: Zheng Hu
We've some discussion about the Shipper interface.
{code}
/**
* This interface denotes a scanner as one which can ship cells. Scan operation
do many RPC requests
* to server and fetch N rows/RPC. These are then shipped to client. At the end
of every such batch
* {@link #shipped()} will get called.
*/
@InterfaceAudience.Private
public interface Shipper {
/**
* Called after a batch of rows scanned and set to be returned to client. Any
in between cleanup
* can be done here.
*/
void shipped() throws IOException;
}
{code}
it seems not an elegant ways...for example:
1. we want to keep the previous cell in the scanner, we must deep clone the
kv before an shipping, otherwise the ship will free all the ByteBuffers, the
prevCell will point to an unknown area.
2. when switch from PREAD to STREAM in a long scan, we also have accomplish
this in shipped() method. if not , once we close the PREAD scanner, the
un-shipped cell will also point to an unknown memory area, because the scanner
closing free all ByteBuffes.
....
If we change to use refCnt to manage the RPC memory release or retain, we can
just call prevCell.retain.. then it memory won't be free unless the prevCell
reach the end of life and call prevCell#release. I mean we can replace all the
shipper logic by using cell#release and cell#retain.
One concern is about the API, actually the ExtendCell is an pure server side
type, we can make the ExtendCell extend the Netty's ReferenceCounted interface
, and provide an retain() and release() methods. we won't maitain an refCnt
in ExtendCell, the refCnt is still in HFileBlock. Once we encoded an
ExtendCell to CellScanner, we can release the extendCell, and it will release
its blocks backend... so in theory, will have no performance loss...
Anyway, that would be a big change, so maybe need to create another new
feature branch to address this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
