Martin,
If you ran "gsutil ls" against the bucket from the instance using the
service account, is it able to list the bucket? Can you describe a bit
more about how the service account's access to the bucket was configured?
Is it added to one of the project roles, explicitly granted on the bucket,
or something else?
Thanks,
James
On Fri, May 26, 2017 at 2:04 AM, Martin Eden <martineden...@gmail.com>
wrote:
> Hi all,
>
> I have a Google Compute instance with a service account configured.
>
> I have installed Nifi 1.2.0 on it.
>
> In the Nifi UI I am describing a flow.
>
> The first processor is a ListGCSBucket processor. It is pointing to a
> bucket for which the service account is added as owner and I am not setting
> a Prefix.
>
> For the GCPCredentialsControllerService I have enabled, one by one all the
> possible properties
> Use Application Default Credentials
> Use Compute Engine Credentials
> Service Account JSON File
> Service Account JSON
>
> to no avail.
>
> In any configuration I get the below.
>
> Any suggestions on how to make this work would be much appreciated,
> M
>
> 2017-05-26 06:59:59,882 ERROR [Timer-Driven Process Thread-3]
> o.a.n.p.gcp.storage.ListGCSBucket ListGCSBucket[id=xxx-xxx-xxx-xxx]
> ListGCSBucket[id=xxx-xxx-xxx-xxx] failed to process session due to
> com.google.cloud.storage.StorageException: Not Found: {}
>
> com.google.cloud.storage.StorageException: Not Found
>
> at
> com.google.cloud.storage.spi.DefaultStorageRpc.translate(
> DefaultStorageRpc.java:202)
>
> at
> com.google.cloud.storage.spi.DefaultStorageRpc.list(
> DefaultStorageRpc.java:294)
>
> at com.google.cloud.storage.StorageImpl$8.call(
> StorageImpl.java:297)
>
> at com.google.cloud.storage.StorageImpl$8.call(
> StorageImpl.java:294)
>
> at com.google.cloud.RetryHelper.doRetry(RetryHelper.java:179)
>
> at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.
> java:244)
>
> at
> com.google.cloud.storage.StorageImpl.listBlobs(StorageImpl.java:293)
>
> at com.google.cloud.storage.StorageImpl.list(StorageImpl.java:260)
>
> at
> org.apache.nifi.processors.gcp.storage.ListGCSBucket.
> onTrigger(ListGCSBucket.java:262)
>
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(
> AbstractProcessor.java:27)
>
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(
> StandardProcessorNode.java:1118)
>
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(
> ContinuallyRunProcessorTask.java:144)
>
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(
> ContinuallyRunProcessorTask.java:47)
>
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(
> TimerDrivenSchedulingAgent.java:132)
>
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>
> at java.util.concurrent.FutureTask.runAndReset(
> FutureTask.java:308)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$
> ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(
> ScheduledThreadPoolExecutor.java:294)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
>
> at java.lang.Thread.run(Thread.java:748)
>
> Caused by:
> com.google.api.client.googleapis.json.GoogleJsonResponseException: 404 Not
> Found
>
> {
>
> "code" : 404,
>
> "errors" : [ {
>
> "domain" : "global",
>
> "message" : "Not Found",
>
> "reason" : "notFound"
>
> } ],
>
> "message" : "Not Found"
>
> }
>
> at
> com.google.api.client.googleapis.json.GoogleJsonResponseException.from(
> GoogleJsonResponseException.java:146)
>
> at
> com.google.api.client.googleapis.services.json.
> AbstractGoogleJsonClientRequest.newExceptionOnError(
> AbstractGoogleJsonClientRequest.java:113)
>
> at
> com.google.api.client.googleapis.services.json.
> AbstractGoogleJsonClientRequest.newExceptionOnError(
> AbstractGoogleJsonClientRequest.java:40)
>
> at
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.
> interceptResponse(AbstractGoogleClientRequest.java:321)
>
> at
> com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1049)
>
> at
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest.
> executeUnparsed(AbstractGoogleClientRequest.java:419)
>
> at
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest.
> executeUnparsed(AbstractGoogleClientRequest.java:352)
>
> at
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest.
> execute(AbstractGoogleClientRequest.java:469)
>
> at
> com.google.cloud.storage.spi.DefaultStorageRpc.list(
> DefaultStorageRpc.java:277)
>
> ... 19 common frames omitted
>
