Todd Lipcon has submitted this change and it was merged.

Change subject: KUDU-456 Implement AUTO_FLUSH_BACKGROUND flush mode

KUDU-456 Implement AUTO_FLUSH_BACKGROUND flush mode

Implemented AUTO_FLUSH_BACKGROUND for the Kudu C++ client library.
the KuduSession::Apply() method blocks if total amount of data
for pending operations reaches the buffer size limit.  The limit
on the buffer size can be set by the
KuduSession::SetMutationBufferSpace() method.

The background flush logic checks whether at least one of the
following two conditions is satisfied to determine whether it's time
to flush the accumulated write operations:
  * The over-the-watermark criterion: check whether the total size of
    the freshly submitted (i.e. not-yet-scheduled-for-flush) write
    operations is over the threshold.  The threshold can be set as
    the percentage of the total buffer size using the
    KuduSession::SetMutationBufferFlushWatermark() method.
  * The maximum wait time criterion: check whether the current batch
    of operations has been accumulating for more than the maximum
    wait time.  The maximum wait time can be specified in milliseconds
    using the KuduSession::SetMutationBufferFlushInterval() method.
    A KuduSession object uses RPC messenger's thread pool to monitor
    batches' maximum wait time.

Added functionality to control the maximum number of batchers
per session.  If number of batchers is at the limit already,
KuduSession::Apply() blocks until it's possible to add a new batcher
to accommodate the incoming operation.

Modified behavior of the KuduSession::Flush(): now it waits until all
batchers are flushed before returning.

This change also addresses the following JIRA issue:
  KUDU-1376 KuduSession::SetMutationBufferSpace is not defined

Change-Id: I34905c30b3aad96f53cf7a1822b1cde6d25f33a8
Reviewed-by: Adar Dembo <>
Tested-by: Kudu Jenkins
M python/kudu/tests/
M src/kudu/client/
M src/kudu/client/batcher.h
M src/kudu/client/client-internal.h
M src/kudu/client/
M src/kudu/client/
M src/kudu/client/client.h
M src/kudu/client/
M src/kudu/client/error_collector.h
M src/kudu/client/
M src/kudu/client/session-internal.h
M src/kudu/client/
M src/kudu/client/write_op.h
13 files changed, 1,509 insertions(+), 251 deletions(-)

  Adar Dembo: Looks good to me, approved
  Kudu Jenkins: Verified

To view, visit
To unsubscribe, visit

Gerrit-MessageType: merged
Gerrit-Change-Id: I34905c30b3aad96f53cf7a1822b1cde6d25f33a8
Gerrit-PatchSet: 29
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <>
Gerrit-Reviewer: Adar Dembo <>
Gerrit-Reviewer: Alexey Serbin <>
Gerrit-Reviewer: Dan Burkert <>
Gerrit-Reviewer: Dinesh Bhat <>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <>

Reply via email to