Brian Hulette created BEAM-11159:
------------------------------------
Summary: Consider using official GCP client in TestPubsub
Key: BEAM-11159
URL: https://issues.apache.org/jira/browse/BEAM-11159
Project: Beam
Issue Type: Improvement
Components: dsl-sql, sdk-java-core
Reporter: Brian Hulette
Assignee: Brian Hulette
TestPubsub currently uses Beam's custom PubSub client, which occasionally leads
to flaky failures due to requests that timed out or could be retried, e.g.:
{code}
1)
failNotFound_matchingDCSchema_sendsMessages(com.google.cloud.dataflow.sqllauncher.PubsubWriteIT)
io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: Deadline expired before
operation could complete.
at
io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:240)
at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:221)
at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:140)
at
com.google.pubsub.v1.SubscriberGrpc$SubscriberBlockingStub.createSubscription(SubscriberGrpc.java:1674)
at
org.apache.beam.sdk.io.gcp.pubsub.PubsubGrpcClient.createSubscription(PubsubGrpcClient.java:352)
at
org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.createRandomSubscription(PubsubClient.java:429)
at
org.apache.beam.sdk.io.gcp.pubsub.TestPubsub.initializePubsub(TestPubsub.java:123)
at
org.apache.beam.sdk.io.gcp.pubsub.TestPubsub.access$200(TestPubsub.java:57)
at
org.apache.beam.sdk.io.gcp.pubsub.TestPubsub$1.evaluate(TestPubsub.java:102)
at
org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:266)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:305)
{code}
We should look into using the GCP client instead which should have better tuned
timeouts and retry policies by default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
