steven zhang created BEAM-12500:
-----------------------------------
Summary: Dataflow SocketException (SSLException) error while
trying to send message from Cloud Pub/Sub to BigQuery
Key: BEAM-12500
URL: https://issues.apache.org/jira/browse/BEAM-12500
Project: Beam
Issue Type: Bug
Components: sdk-java-core, sdk-java-harness
Affects Versions: 2.29.0, 2.28.0, 2.27.0, 2.26.0, 2.25.0
Environment: GCP services used (such as BigQuery): Pub/Sub
-->Dataflow-->BigQuery
Reporter: steven zhang
*Issue*: I found an issue of seeing SSLException when using beam version
(>2.25) and could not upgrade with advanced version of beam in the dataflow job
(streaming). Only versions below 2.25 is possible to work. Could not be sure if
this issue is connected to [1]
[1]https://bugs.openjdk.java.net/browse/JDK-8241372
*Log*: {{Error message from worker: java.lang.RuntimeException:
java.net.SocketException: Connection reset
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll(BigQueryServicesImpl.java:932)
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll(BigQueryServicesImpl.java:972)
org.apache.beam.sdk.io.gcp.bigquery.StreamingWriteFn.flushRows(StreamingWriteFn.java:195)
org.apache.beam.sdk.io.gcp.bigquery.StreamingWriteFn.finishBundle(StreamingWriteFn.java:149)
Caused by: java.net.SocketException: Connection reset
java.net.SocketInputStream.read(SocketInputStream.java:210)
java.net.SocketInputStream.read(SocketInputStream.java:141)
sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
sun.security.ssl.InputRecord.read(InputRecord.java:503)
sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983)
sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:940)
sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
java.io.BufferedInputStream.read(BufferedInputStream.java:345)
sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:552)
sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:609)
sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:696)
java.io.FilterInputStream.read(FilterInputStream.java:133)
sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:3393)
com.google.api.client.http.javanet.NetHttpResponse$SizeValidatingInputStream.read(NetHttpResponse.java:164)
java.io.FilterInputStream.read(FilterInputStream.java:133)
java.io.FilterInputStream.read(FilterInputStream.java:107)
com.google.common.io.ByteStreams.exhaust(ByteStreams.java:273)
com.google.api.client.http.ConsumingInputStream.close(ConsumingInputStream.java:40)
java.util.zip.InflaterInputStream.close(InflaterInputStream.java:227)
java.util.zip.GZIPInputStream.close(GZIPInputStream.java:136)
com.fasterxml.jackson.core.json.UTF8StreamJsonParser._closeInput(UTF8StreamJsonParser.java:254)
com.fasterxml.jackson.core.base.ParserBase.close(ParserBase.java:369)
com.google.api.client.json.jackson2.JacksonParser.close(JacksonParser.java:47)
com.google.api.client.json.JsonParser.parse(JsonParser.java:363)
com.google.api.client.json.JsonParser.parse(JsonParser.java:335)
com.google.api.client.json.JsonObjectParser.parseAndClose(JsonObjectParser.java:79)
com.google.api.client.json.JsonObjectParser.parseAndClose(JsonObjectParser.java:73)
com.google.api.client.http.HttpResponse.parseAs(HttpResponse.java:451)
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.lambda$insertAll$1(BigQueryServicesImpl.java:857)
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$BoundedExecutorService$SemaphoreCallable.call(BigQueryServicesImpl.java:1304)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
