[
https://issues.apache.org/jira/browse/BEAM-12577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394589#comment-17394589
]
Jonathan Gallon commented on BEAM-12577:
----------------------------------------
Hi [~egalpin],
Thank you for your reply.
My use case is simple: I have a streaming pipeline that process documents and
store some results into Elasticsearch.
Generally, everything works fine: my pipeline waits, receives an event,
processes it, and calls ElasticsearchIO.
My number of nodes is fixed.
But sometimes, something happens between my pipeline and Elasticsearch (a short
network outage for example), the pipeline lost the connection with
Elasticsearch and in this case, the pipeline is unusable and must be deleted
and redeploy.
It seems there is no mechanism to recreate the RestClient if this one lost the
connection, or no keepalive enable.
> In stream mode, ElasticsearchIO transform becomes unavailable
> -------------------------------------------------------------
>
> Key: BEAM-12577
> URL: https://issues.apache.org/jira/browse/BEAM-12577
> Project: Beam
> Issue Type: Bug
> Components: io-java-elasticsearch
> Affects Versions: 2.29.0
> Environment: Dataflow GCP
> Reporter: Jonathan Gallon
> Priority: P2
> Labels: GCP
>
> If you use the ElasticSearchIO transform for a workflow in stream mode, and
> if something happens temporarily between the RestClient and the server, any
> future calls will result in a
> {quote}java.io.IOException: Connection reset by peer
> {quote}
> As there is not reconnection system ou keepalive enable, the client seems not
> able to reconnect and the workflow cannot work anymore.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
