nicoloboschi opened a new pull request, #15244:
URL: https://github.com/apache/pulsar/pull/15244
### Motivation
In the ElasticSearch sink connector the `bulkFlushIntervalInMs` is currently
disabled by default.
This option is used to control the maximum time that the messages are
retained in memory before being flushed to the ES server (along with
`bulkActions` and `bulkSizeInMb`) when bulk mode is enabled.
`bulkActions` and `bulkSizeInMb` are based on the traffic and if
`bulkFlushIntervalInMs` is disabled, the messages can be blocked in the sink
forever. This is a problem because then the function subscriptions are stuck
(messages are not acked since correctly sent to elastic).
`bulkFlushIntervalInMs` is a time-based setting and it will avoid this kind
of scenario by design. However this could increase the overhead with
low-traffic. With high traffic there should not be visible changes.
### Modifications
Enable `bulkFlushIntervalInMs` by default with 10 seconds as value. 10
seconds is just a reasonable value, nothing special behind it
- [x] `doc`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
