David Yan created BEAM-3764:
-------------------------------
Summary: Add retry logic in ExampleUtils that handles network
flakes.
Key: BEAM-3764
URL: https://issues.apache.org/jira/browse/BEAM-3764
Project: Beam
Issue Type: Improvement
Components: examples-java
Reporter: David Yan
Assignee: Reuven Lax
This exception is thrown when there is a network flake when running the traffic
examples. Should we retry instead of throwing the exception and failing the job?
{{Exception in thread "main" java.io.IOException: Error getting access token
for service account: }}
{{ at
com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:319)}}
{{ at
com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:149)}}
{{ at
com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:135)}}
{{ at
com.google.auth.http.HttpCredentialsAdapter.initialize(HttpCredentialsAdapter.java:96)}}
{{ at
com.google.cloud.hadoop.util.ChainingHttpRequestInitializer.initialize(ChainingHttpRequestInitializer.java:52)}}
{{ at
com.google.api.client.http.HttpRequestFactory.buildRequest(HttpRequestFactory.java:93)}}
{{ at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.buildHttpRequest(AbstractGoogleClientRequest.java:300)}}
{{ at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)}}
{{ at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)}}
{{ at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)}}
{{ at
org.apache.beam.examples.common.ExampleUtils.executeNullIfNotFound(ExampleUtils.java:397)}}
{{ at
org.apache.beam.examples.common.ExampleUtils.setupPubsubTopic(ExampleUtils.java:283)}}
{{ at
org.apache.beam.examples.common.ExampleUtils.setupPubsub(ExampleUtils.java:131)}}
{{ at
org.apache.beam.examples.common.ExampleUtils.setup(ExampleUtils.java:105)}}
{{ at
org.apache.beam.examples.complete.TrafficMaxLaneFlow.main(TrafficMaxLaneFlow.java:334)}}
{{Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed connection
during handshake}}
{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:994)}}
{{ at
sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1379)}}
{{ at
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1407)}}
{{ at
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1391)}}
{{ at
sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)}}
{{ at
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)}}
{{ at
sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1316)}}
{{ at
sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1291)}}
{{ at
sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:250)}}
{{ at
com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:77)}}
{{ at
com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981)}}
{{ at
com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:317)}}
{{ ... 14 more}}
{{Caused by: java.io.EOFException: SSL peer shut down incorrectly}}
{{ at sun.security.ssl.InputRecord.read(InputRecord.java:505)}}
{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
{{ ... 25 more}}
Add retry code somewhere here?
https://github.com/apache/beam/blob/29859eb54d05b96a9db477e7bb04537510273bd2/examples/java/src/main/java/org/apache/beam/examples/common/ExampleUtils.java#L398
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
