[
https://issues.apache.org/jira/browse/ACCUMULO-551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268804#comment-13268804
]
Keith Turner commented on ACCUMULO-551:
---------------------------------------
Its a work in progress, but I have some working code for this available on
github.
https://github.com/keith-turner/accumulo/tree/ACCUMULO-551
> Experiment with multi-node batch writer
> ---------------------------------------
>
> Key: ACCUMULO-551
> URL: https://issues.apache.org/jira/browse/ACCUMULO-551
> Project: Accumulo
> Issue Type: Task
> Reporter: Keith Turner
> Assignee: Keith Turner
> Fix For: 1.5.0
>
>
> Accumulo has a batch writer that batches mutations by tablet server for
> writes. This works well until there are alot of tablet servers being written
> to at which point only a small amount of data is being sent to each tablet
> server. Would it be better for the client to batch writes for multiple
> tablet servers and send them to one server which writes directly to the
> tablet servers?
> One possible way to do this is to :
>
> * batch mutations by rack on the client
> * send all of those mutations to one random tablet server on the rack
> * have the random tablet server write to the other servers on the rack
> This cuts down on the number of direct connections the client has to make.
> Could have the following benefits.
> * Tablet servers can keep connections open to other tablet servers.
> * A write pipeline
> Would be interesting to run some test and see how well this works.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
