[
https://issues.apache.org/jira/browse/HBASE-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12849549#action_12849549
]
Kay Kay commented on HBASE-2368:
--------------------------------
| could you explain to me how this might be different than having the
TableOutputFormat assume/use a write buffer?
This is complementary to the same, except with autoFlush set to true /
writeBuffer set to false. We have a Mapper job that was generating 1:many
puts at the Mapper level. So instead of writing individual puts to the stream,
bulk add the puts and write to the stream once.
| (though not to TOF).
Specifically this patch addresses the TableRecordWriter (of TOF) and as you had
pointed , this is moot when we are using HTable directly , but complementary at
the Mapper / Combiner level , when we can consolidate puts and write them in
bulk.
> BulkPut - Writable class compatible with TableRecordWriter for bulk puts
> agnostic of region server mapping at Mapper/Combiner level
> ------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-2368
> URL: https://issues.apache.org/jira/browse/HBASE-2368
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: client
> Reporter: Kay Kay
> Fix For: 0.21.0
>
> Attachments: HBASE-2368.patch
>
>
> TableRecordWriter currently accepts only a put/delete as writables. Some
> mapper processes might want to consolidate the 'put's and insert them in
> bulk. Useful in combiners / mappers - to send across a bunch of puts from one
> stage to another , while maintaining a very similar region-server-mapping
> agnostic api at respective levels.
> New type - BulkPut ( Writable ) introduced that is just a consolidation of
> Puts. Eventually , the TableRecordWriter bulk inserts the puts together into
> the hbase eco-system.
> Patch made against trunk only. But since , it does not break any backward
> compatibility - it can be an useful addition to the branch as well.
> Let me know your comments on the same.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
