[
https://issues.apache.org/jira/browse/HBASE-19486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16299017#comment-16299017
]
Niels Basjes edited comment on HBASE-19486 at 12/20/17 8:27 PM:
----------------------------------------------------------------
[~jinghe] I used similar naming from what I know from Kafka which has a setting
which effectively does something very similar:
Look for {{linger.ms}} in this page: https://kafka.apache.org/documentation/
I personally think that having a similar naming compared to Kafka for this
makes it easier for "Streaming people" to find.
If you guys really prefer different naming then that's fine and I'll rename
everything to make it consistent with the rest of HBase.
was (Author: nielsbasjes):
@jinghe I used similar naming from what I know from Kafka which has a setting
which effectively does something very similar:
Look for {{linger.ms}} in this page: https://kafka.apache.org/documentation/
I personally think that having a similar naming compared to Kafka for this
makes it easier for "Streaming people" to find.
If you guys really prefer different naming then that's fine and I'll rename
everything to make it consistent with the rest of HBase.
> Automatically flush BufferedMutator after a timeout
> ----------------------------------------------------
>
> Key: HBASE-19486
> URL: https://issues.apache.org/jira/browse/HBASE-19486
> Project: HBase
> Issue Type: Improvement
> Components: Client
> Reporter: Niels Basjes
> Assignee: Niels Basjes
> Attachments: HBASE-19486-20171212-2117.patch,
> HBASE-19486-20171218-1229.patch, HBASE-19486-20171218-1300.patch,
> HBASE-19486-20171219-0933.patch, HBASE-19486-20171219-1026.patch,
> HBASE-19486-20171219-1122-trigger-qa-run.patch,
> HBASE-19486-20171220-1612-trigger-qa-run.patch
>
>
> I'm working on several projects where we are doing stream / event type
> processing instead of batch type processing. We mostly use Apache Flink and
> Apache Beam for these projects.
> When we ingest a continuous stream of events and feed that into HBase via a
> BufferedMutator this all works fine. The buffer fills up at a predictable
> rate and we can make sure it flushes several times per second into HBase by
> tuning the buffer size.
> We also have situations where the event rate is unpredictable. Some times
> because the source is in reality a batch job that puts records into Kafka,
> sometimes because it is the "predictable in production" application in our
> testing environment (where only the dev triggers a handful of events).
> For these kinds of use cases we need a way to 'force' the BufferedMutator to
> automatically flush any records in the buffer even if the buffer is not full.
> I'll put up a pull request with a proposed implementation for review against
> the master (i.e. 3.0.0).
> When approved I would like to backport this to the 1.x and 2.x versions of
> the client in the same (as close as possible) way.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
