[
https://issues.apache.org/jira/browse/HBASE-21917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16775318#comment-16775318
]
Anoop Sam John commented on HBASE-21917:
----------------------------------------
{code}
public void verifyChunkedSums(ByteBuffer data, ByteBuffer checksums,
String fileName, long basePos)
throws ChecksumException {
if (type.size == 0) return;
if (data.hasArray() && checksums.hasArray()) {
verifyChunkedSums(
data.array(), data.arrayOffset() + data.position(), data.remaining(),
checksums.array(), checksums.arrayOffset() + checksums.position(),
fileName, basePos); // -------> private method, no access
return;
}
if (NativeCrc32.isAvailable()) {
NativeCrc32.verifyChunkedSums(bytesPerChecksum, type.id, checksums, data,
fileName, basePos); -------> NativeCrc32 is not public, no access.
return;
}
{code}
Based on this code in Hadoop, if we can pass ByteBuffer to this Hadoop API and
that BB is DBB , we will not go into 1st if block.l But if native CRC is
available (if so, we wanted to make use of it), Hadoop can call it using the BB
that we pass. Native API calls means it would work with DBB with out any copy.
(?) So if we used ByteBuff#asSubByteBuffer API, and mostly it will not incur
any copy (as explained above) we can make use of native API with out any extra
on heap copy and GC over head no?
> Make the HFileBlock#validateChecksum can accept ByteBuff as an input.
> ---------------------------------------------------------------------
>
> Key: HBASE-21917
> URL: https://issues.apache.org/jira/browse/HBASE-21917
> Project: HBase
> Issue Type: Sub-task
> Reporter: Zheng Hu
> Assignee: Zheng Hu
> Priority: Major
> Attachments: HBASE-21917.v1.patch, HBASE-21917.v2.patch,
> HBASE-21917.v3.patch, HBASE-21917.v4.patch
>
>
> I've tried to make a patch for HBASE-21879, most of work seems to be fine,
> but the trouble is:
> HFileBlock#validateChecksum can only accept ByteBuffer as its input, while
> after the HBASE-21916, we will use an ourself-defined ByteBuff (which can be
> SingleByteBuff or MultiByteBuff).
> Now, need to create our own ByteBuff checksum validation method, should not
> be so hard but an separate issue will be more clearer.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
