[
https://issues.apache.org/jira/browse/BEAM-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16074385#comment-16074385
]
Etienne Chauchot commented on BEAM-2541:
----------------------------------------
[~rxminus], thanks for using Beam! And thanks for opening this ticket. It is a
fair point. What we could do is either postpone this check or remove this check
with the new upcoming ES IO that support both v2 and v5 of Elasticsearch.
> ElasticsearchIO breaks for non locally accessible DB
> ----------------------------------------------------
>
> Key: BEAM-2541
> URL: https://issues.apache.org/jira/browse/BEAM-2541
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-extensions
> Reporter: Rik Nauta
> Assignee: Etienne Chauchot
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> We have a ES (2.4) database running in production that is not accessible from
> dev-machines because of VPS firewall rules. However, we do create the Beam
> graph locally on our machines and submit it to Google Dataflow for execution.
> However because of this check (line 213)
> {code:java}
> private static void checkVersion(ConnectionConfiguration
> connectionConfiguration)
> throws IOException {
> RestClient restClient = connectionConfiguration.createClient();
> Response response = restClient.performRequest("GET", "", new
> BasicHeader("", ""));
> JsonNode jsonNode = parseResponse(response);
> String version = jsonNode.path("version").path("number").asText();
> boolean version2x = version.startsWith("2.");
> restClient.close();
> checkArgument(
> version2x,
> "ConnectionConfiguration.create(addresses, index, type): "
> + "the Elasticsearch version to connect to is different of 2.x.
> "
> + "This version of the ElasticsearchIO is only compatible with
> Elasticsearch v2.x");
> }
> {code}
> the creation of the graph fails because it can't complete the restClient
> request from our dev machines. This check should probably not happen this
> early since there is no guarantee that the database is reachable when
> building the graph
> {code:java}
> exception in thread "main" java.net.ConnectException
> at
> org.apache.http.nio.pool.RouteSpecificPool.timeout(RouteSpecificPool.java:168)
> at
> org.apache.http.nio.pool.AbstractNIOConnPool.requestTimeout(AbstractNIOConnPool.java:561)
> at
> org.apache.http.nio.pool.AbstractNIOConnPool$InternalSessionRequestCallback.timeout(AbstractNIOConnPool.java:822)
> at
> org.apache.http.impl.nio.reactor.SessionRequestImpl.timeout(SessionRequestImpl.java:183)
> at
> org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processTimeouts(DefaultConnectingIOReactor.java:210)
> at
> org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:155)
> at
> org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:348)
> at
> org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:192)
> at
> org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
