chenzihao5 opened a new pull request, #172: URL: https://github.com/apache/flink-connector-elasticsearch/pull/172
## Purpose of the change https://issues.apache.org/jira/browse/FLINK-41457 In low-volume scenarios, users often require written data to be searchable immediately. However, the Elasticsearch sink always sends bulk requests with refresh=false, so the data only becomes visible after the next periodic index refresh. This PR exposes the refresh parameter of the Bulk API in the sink. ## Brief change log - Add BulkRefreshPolicy (NONE / IMMEDIATE / WAIT_UNTIL), mapping to refresh=false / true / wait_for. - ES 6/7: add setBulkRefreshPolicy to ElasticsearchSinkBuilderBase and apply it to every bulk request. - ES 8: add setBulkRefreshPolicy to Elasticsearch8AsyncSinkBuilder and apply it in Elasticsearch8AsyncWriter. - Table / SQL: add option 'sink.bulk-flush.refresh-policy' for elasticsearch-6, elasticsearch-7 and elasticsearch-8. ## Verifying this change - Extended ElasticsearchSinkBuilderBaseTest, ElasticsearchDynamicTableFactoryBaseTest and Elasticsearch8AsyncSinkBuilderTest to check that the policy is propagated from the builder / table options to the sink, and that invalid values are rejected. ## Impact and documentation - Public API: adds the @PublicEvolving enum BulkRefreshPolicy, new builder methods and a new table option. No existing API is changed. - Behavior: the default is NONE, so existing jobs are not affected. - Documentation: updated the DataStream and Table Elasticsearch docs (en and zh) to describe the option. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
