[
https://issues.apache.org/jira/browse/FLINK-3857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272197#comment-15272197
]
ASF GitHub Bot commented on FLINK-3857:
---------------------------------------
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/1962#discussion_r62173042
--- Diff:
flink-streaming-connectors/flink-connector-elasticsearch2/src/main/java/org/apache/flink/streaming/connectors/elasticsearch2/ElasticsearchSink.java
---
@@ -177,7 +180,72 @@ public void open(Configuration configuration) {
if (LOG.isInfoEnabled()) {
LOG.info("Created Elasticsearch TransportClient {}",
client);
}
+ }
+
+ @Override
+ public void invoke(final T element) {
+ ParameterTool params = ParameterTool.fromMap(userConfig);
+
+ if (params.has(CONFIG_NO_OF_CONN_RETRIES) &&
params.getInt(CONFIG_NO_OF_CONN_RETRIES) > 0) {
+ final Timer timer = new Timer(true);
+ TimerTask task = new TimerTask() {
+ @Override
+ public void run() {
+ // verify that we actually are
connected to a cluster
+ ImmutableList<DiscoveryNode> nodes =
ImmutableList.copyOf(((TransportClient) client).connectedNodes());
+ if (nodes.isEmpty()) {
+ if (LOG.isInfoEnabled()) {
+ LOG.info("Connection
Lost..Trying to reconnect to Elasticsearch nodes...");
+ }
+ open(new Configuration());
+ } else {
+ timer.cancel();
+
intializeAndCallElasticSearchSinkFunction(element);
+ }
+ }
+ };
+
+ timer.scheduleAtFixedRate(task, 0, 3000);
+
+ try {
+ Thread.sleep(3000 *
params.getInt(CONFIG_NO_OF_CONN_RETRIES));
+ } catch (InterruptedException e) {
+ throw new RuntimeException(e.getMessage());
+ }
+ timer.cancel();
+ // verify that we actually are connected to a cluster
+ ImmutableList<DiscoveryNode> nodes =
ImmutableList.copyOf(((TransportClient) client).connectedNodes());
+ if (nodes.isEmpty()) {
+ throw new RuntimeException("Client is not
connected to any Elasticsearch nodes!");
+ }
+ } else {
+ intializeAndCallElasticSearchSinkFunction(element);
+ }
+ }
+
+ @Override
+ public void close() {
+ if (bulkProcessor != null) {
+ bulkProcessor.close();
+ bulkProcessor = null;
+ }
+
+ if (client != null) {
+ client.close();
+ }
+
+ if (hasFailure.get()) {
+ Throwable cause = failureThrowable.get();
+ if (cause != null) {
+ throw new RuntimeException("An error occured in
ElasticsearchSink.", cause);
+ } else {
+ throw new RuntimeException("An error occured in
ElasticsearchSink.");
+ }
+ }
+ }
+
+ private void intializeAndCallElasticSearchSinkFunction(T element) {
--- End diff --
Where is the method initializing the ES sink?
> Add reconnect attempt to Elasticsearch host
> -------------------------------------------
>
> Key: FLINK-3857
> URL: https://issues.apache.org/jira/browse/FLINK-3857
> Project: Flink
> Issue Type: Improvement
> Components: Streaming Connectors
> Affects Versions: 1.1.0, 1.0.2
> Reporter: Fabian Hueske
> Assignee: Subhobrata Dey
>
> Currently, the connection to the Elasticsearch host is opened in
> {{ElasticsearchSink.open()}}. In case the connection is lost (maybe due to a
> changed DNS entry), the sink fails.
> I propose to catch the Exception for lost connections in the {{invoke()}}
> method and try to re-open the connection for a configurable number of times
> with a certain delay.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
