[
https://issues.apache.org/jira/browse/FLINK-33857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-33857:
-----------------------------------
Labels: pull-request-available (was: )
> Expose BulkProcessor.flush via RequestIndexer to allow custom flush logic
> -------------------------------------------------------------------------
>
> Key: FLINK-33857
> URL: https://issues.apache.org/jira/browse/FLINK-33857
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / ElasticSearch
> Affects Versions: elasticsearch-3.0.1
> Reporter: Peter Schulz
> Priority: Major
> Labels: pull-request-available
>
> There's a bug in elasticsearch that may lead to underestimated bulk request
> sizes. We are hit by this bug and therefore, I would like to propose a simple
> improvement to work around it:
> {{ElasticsearchSinkWriter}} uses {{RequestIndexer}} as a facade for ES'
> {{BulkProcessor}}. As of now, only the {{add(…)}} methods are delegated.
> Under the hood, the {{BulkProcessor}} takes care of flushing bulk requests to
> the server when any of the configurable limits is hit (there's another bug:
> flushing will happen _after_ the limit has exceeded).
> (+) *Proposal:* Expose {{BulkProcessor.flush()}} via
> {{RequestIndexer.flush()}}. This way we can easily implement logic to decide
> when to flush inside the emitter.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
