Abacn commented on code in PR #30824:
URL: https://github.com/apache/beam/pull/30824#discussion_r1571119296
##########
sdks/java/io/elasticsearch/src/main/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIO.java:
##########
@@ -834,13 +859,27 @@ public Read withMetadata() {
*
href="https://www.elastic.co/guide/en/elasticsearch/reference/7.17/search-request-scroll.html";>scroll
* API</a> Default is "5m". Change this only if you get "No search context
found" errors.
*
+ * @deprecated use {@link Read#withIteratorKeepalive} instead
Review Comment:
I see when setUsePITSearch is not enabled, iteratorKeepalive has the same
effect as scrollKeepalive before:
https://github.com/apache/beam/pull/30824/files#diff-e119923661d41dfc1911da8f6e29cc0e2e9a0a50129718c1c20c7d1aa32119bfR1269
it still sets `scroll` parameter taken the scrollKeepalive value.
My understanding is that `scrollKeepalive` isn't really deprecated. It is
now the config `iteratorKeepalive` goes to different param depending on whether
PIT is used or not.
Can we keep the old naming `withScrollKeepalive` and document that this
parameter now goes to set iterator alive time when PIT is enabled? So no Beam
API change or deprecation needed
##########
sdks/java/io/elasticsearch-tests/elasticsearch-tests-common/src/test/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIOTestCommon.java:
##########
@@ -228,24 +228,42 @@ void testRead() throws Exception {
ElasticsearchIO.read()
.withConnectionConfiguration(connectionConfiguration)
// set to default value, useful just to test parameter passing.
- .withScrollKeepalive("5m")
+ .withIteratorKeepalive("5m")
// set to default value, useful just to test parameter passing.
.withBatchSize(100L));
PAssert.thatSingleton(output.apply("Count",
Count.globally())).isEqualTo(numDocs);
pipeline.run();
}
+ void testReadPIT() throws Exception {
Review Comment:
Can we add a comment that PIT is available since elasticsearch version 8
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
