chenzihao5 opened a new pull request, #172:
URL: https://github.com/apache/flink-connector-elasticsearch/pull/172

   ## Purpose of the change
   
   https://issues.apache.org/jira/browse/FLINK-41457
   
   In low-volume scenarios, users often require written data to be searchable 
immediately. However, the Elasticsearch sink always sends bulk requests with 
refresh=false, so the data only becomes visible after the next periodic index 
refresh. This PR exposes the refresh parameter of the Bulk API in the sink.
   
   ## Brief change log
   
   - Add BulkRefreshPolicy (NONE / IMMEDIATE / WAIT_UNTIL), mapping to 
refresh=false / true / wait_for.
   - ES 6/7: add setBulkRefreshPolicy to ElasticsearchSinkBuilderBase and apply 
it to every bulk request.
   - ES 8: add setBulkRefreshPolicy to Elasticsearch8AsyncSinkBuilder and apply 
it in Elasticsearch8AsyncWriter.
   - Table / SQL: add option 'sink.bulk-flush.refresh-policy' for 
elasticsearch-6, elasticsearch-7 and elasticsearch-8.
   
   ## Verifying this change
   
   - Extended ElasticsearchSinkBuilderBaseTest, 
ElasticsearchDynamicTableFactoryBaseTest and Elasticsearch8AsyncSinkBuilderTest 
to check that the policy is propagated from the builder / table options to the 
sink, and that invalid values are rejected.
   
   ## Impact and documentation
   
   - Public API: adds the @PublicEvolving enum BulkRefreshPolicy, new builder 
methods and a new table option. No existing API is changed.
   - Behavior: the default is NONE, so existing jobs are not affected.
   - Documentation: updated the DataStream and Table Elasticsearch docs (en and 
zh) to describe the option.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to