[jira] [Created] (BEAM-4378) Update javadoc for set/getRegion in DataflowPipelineOptions.
David Yan created BEAM-4378: --- Summary: Update javadoc for set/getRegion in DataflowPipelineOptions. Key: BEAM-4378 URL: https://issues.apache.org/jira/browse/BEAM-4378 Project: Beam Issue Type: Improvement Components: runner-dataflow Reporter: David Yan Assignee: Thomas Groh Dataflow now supports the region flag. We should update the javadoc here: https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/options/DataflowPipelineOptions.java#L107 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-3764) Add retry logic in ExampleUtils that handles network flakes.
David Yan created BEAM-3764:
---
Summary: Add retry logic in ExampleUtils that handles network
flakes.
Key: BEAM-3764
URL: https://issues.apache.org/jira/browse/BEAM-3764
Project: Beam
Issue Type: Improvement
Components: examples-java
Reporter: David Yan
Assignee: Reuven Lax
This exception is thrown when there is a network flake when running the traffic
examples. Should we retry instead of throwing the exception and failing the job?
{{Exception in thread "main" java.io.IOException: Error getting access token
for service account: }}
{{ at
com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:319)}}
{{ at
com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:149)}}
{{ at
com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:135)}}
{{ at
com.google.auth.http.HttpCredentialsAdapter.initialize(HttpCredentialsAdapter.java:96)}}
{{ at
com.google.cloud.hadoop.util.ChainingHttpRequestInitializer.initialize(ChainingHttpRequestInitializer.java:52)}}
{{ at
com.google.api.client.http.HttpRequestFactory.buildRequest(HttpRequestFactory.java:93)}}
{{ at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.buildHttpRequest(AbstractGoogleClientRequest.java:300)}}
{{ at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)}}
{{ at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)}}
{{ at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)}}
{{ at
org.apache.beam.examples.common.ExampleUtils.executeNullIfNotFound(ExampleUtils.java:397)}}
{{ at
org.apache.beam.examples.common.ExampleUtils.setupPubsubTopic(ExampleUtils.java:283)}}
{{ at
org.apache.beam.examples.common.ExampleUtils.setupPubsub(ExampleUtils.java:131)}}
{{ at
org.apache.beam.examples.common.ExampleUtils.setup(ExampleUtils.java:105)}}
{{ at
org.apache.beam.examples.complete.TrafficMaxLaneFlow.main(TrafficMaxLaneFlow.java:334)}}
{{Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed connection
during handshake}}
{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:994)}}
{{ at
sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1379)}}
{{ at
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1407)}}
{{ at
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1391)}}
{{ at
sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)}}
{{ at
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)}}
{{ at
sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1316)}}
{{ at
sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1291)}}
{{ at
sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:250)}}
{{ at
com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:77)}}
{{ at
com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981)}}
{{ at
com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:317)}}
{{ ... 14 more}}
{{Caused by: java.io.EOFException: SSL peer shut down incorrectly}}
{{ at sun.security.ssl.InputRecord.read(InputRecord.java:505)}}
{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
{{ ... 25 more}}
Add retry code somewhere here?
https://github.com/apache/beam/blob/29859eb54d05b96a9db477e7bb04537510273bd2/examples/java/src/main/java/org/apache/beam/examples/common/ExampleUtils.java#L398
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (BEAM-3613) SpannerIO: Typo in "witHost"
[ https://issues.apache.org/jira/browse/BEAM-3613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Yan resolved BEAM-3613. - Resolution: Fixed Fix Version/s: 2.4.0 > SpannerIO: Typo in "witHost" > > > Key: BEAM-3613 > URL: https://issues.apache.org/jira/browse/BEAM-3613 > Project: Beam > Issue Type: Bug > Components: sdk-java-gcp >Reporter: David Yan >Assignee: Chamikara Jayalath >Priority: Minor > Fix For: 2.4.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > There is a typo in SpannerIO.java. > "witHost" should be "withHost". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-3613) SpannerIO: Typo in "witHost"
[ https://issues.apache.org/jira/browse/BEAM-3613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351178#comment-16351178 ] David Yan commented on BEAM-3613: - [https://github.com/apache/beam/pull/4592] open. > SpannerIO: Typo in "witHost" > > > Key: BEAM-3613 > URL: https://issues.apache.org/jira/browse/BEAM-3613 > Project: Beam > Issue Type: Bug > Components: sdk-java-gcp >Reporter: David Yan >Assignee: Chamikara Jayalath >Priority: Minor > > There is a typo in SpannerIO.java. > "witHost" should be "withHost". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-3613) SpannerIO: Typo in "witHost"
David Yan created BEAM-3613: --- Summary: SpannerIO: Typo in "witHost" Key: BEAM-3613 URL: https://issues.apache.org/jira/browse/BEAM-3613 Project: Beam Issue Type: Bug Components: sdk-java-gcp Reporter: David Yan Assignee: Chamikara Jayalath There is a typo in SpannerIO.java. "witHost" should be "withHost". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
