Ah, SET project = apache-beam-testing; gives the following exception:
io.grpc.StatusRuntimeException: PERMISSION_DENIED: User not authorized
to perform this action.
at
io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:222)
at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:203)
at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:132)
at
com.google.pubsub.v1.PublisherGrpc$PublisherBlockingStub.createTopic(PublisherGrpc.java:666)
at
org.apache.beam.sdk.io.gcp.pubsub.PubsubGrpcClient.createTopic(PubsubGrpcClient.java:302)
at
org.apache.beam.sdk.io.gcp.pubsub.TestPubsub.initializePubsub(TestPubsub.java:102)
at
org.apache.beam.sdk.io.gcp.pubsub.TestPubsub.access$200(TestPubsub.java:42)
at
org.apache.beam.sdk.io.gcp.pubsub.TestPubsub$1.evaluate(TestPubsub.java:85)
which should not happen because jenkins should already have credentials to
access GCP.
-Rui
On Thu, Jul 26, 2018 at 3:30 PM Andrew Pilloud <[email protected]> wrote:
> Beam SQL CLI does not accept beamTestPipelineOptions. Also, gradle is
> invoking your test not the Beam SQL CLI. You'll need to set the options in
> your integration test by executing 'SET project = ...' in the Beam SQL
> connection you've launched for test.
>
> Andrew
>
> On Thu, Jul 26, 2018 at 3:06 PM Rui Wang <[email protected]> wrote:
>
>> The code path of reading pubsub through BeamSQL goes through this line of
>> code:
>>
>> PubsubIO.Read<PubsubMessage> read =
>> PubsubIO.readMessagesWithAttributes().fromTopic(getTopic());
>>
>> -Rui
>>
>> On Thu, Jul 26, 2018 at 2:58 PM Rui Wang <[email protected]> wrote:
>>
>>> Hi Community,
>>>
>>> I am facing a runtime exception when I try to read from pubsub by Beam
>>> SQL in JUnit tests (PR: https://github.com/apache/beam/pull/6006). The
>>> exception is "Cannot create subscription because pipeline option 'project'
>>> not specified". Based on existing JUnit tests which also read from
>>> pubsub by Beam SQL, I added the following code to my build.gradle and
>>> it seems didn't work. Is there someone who could know what's wrong in my
>>> .gradle file?
>>>
>>>
>>> task endToEndTest(type: Test) {
>>> group = "Verification"
>>> def gcpProject = project.findProperty('gcpProject') ?:
>>> 'apache-beam-testing'
>>> def gcsTempRoot = project.findProperty('gcsTempRoot') ?:
>>> 'gs://temp-storage-for-end-to-end-tests/'
>>>
>>> // Disable Gradle cache (it should not be used because the IT's won't
>>> run).
>>> outputs.upToDateWhen { false }
>>>
>>> def pipelineOptions = [
>>> "--project=${gcpProject}",
>>> "--tempLocation=${gcsTempRoot}",
>>> "--blockOnRun=false"]
>>>
>>> systemProperty "beamTestPipelineOptions",
>>> JsonOutput.toJson(pipelineOptions)
>>>
>>> include '**/BeamSqlLineIT.class'
>>> classpath =
>>> project(":beam-sdks-java-extensions-sql-jdbc").sourceSets.test.runtimeClasspath
>>> testClassesDirs =
>>> files(project(":beam-sdks-java-extensions-sql-jdbc").sourceSets.test.output.classesDirs)
>>> useJUnit { }
>>> }
>>>
>>> task postCommit {
>>> group = "Verification"
>>> description = "Various integration tests"
>>> dependsOn endToEndTest
>>> }
>>>
>>>
>>> -Rui
>>>
>>
