devinbost opened a new issue #8435:
URL: https://github.com/apache/pulsar/issues/8435
I have a heartbeat topic that receives messages on the topic
`persistent://public/default/canary`, and I configured retention on this topic
to test tiered storage in GCP. When I manually trigger offloading from the
topic, I get the error `java.util.concurrent.CompletionException:
org.jclouds.rest.ResourceNotFoundException: Not Found`, as shown below:
`bin/pulsar-admin topics offload --size-threshold 10M
persistent://public/default/canary`
> Offload triggered for persistent://public/default/canary for messages
before 873151:0:-1
`bin/pulsar-admin topics offload-status -w
persistent://public/default/canary`
> Error in offload
> null
>
> Reason: Error offloading:
org.apache.bookkeeper.mledger.ManagedLedgerException:
java.util.concurrent.CompletionException:
org.jclouds.rest.ResourceNotFoundException: Not Found
On the broker and bookie containers, I have the GCP JSON key file located at
`conf/auth/tiered-storage.key`
In my broker.conf file, I have the GCS attributes configured as follows:
```
gcsManagedLedgerOffloadRegion=us-west-3
gcsManagedLedgerOffloadBucket=gs://my-bucket-name
gcsManagedLedgerOffloadMaxBlockSizeInBytes=67108864
gcsManagedLedgerOffloadReadBufferSizeInBytes=1048576
gcsManagedLedgerOffloadServiceAccountKeyFile=/pulsar/conf/auth/tiered-storage.key
```
The tiered-storage.key file looks as follows:
```
{
"type": "service_account",
"project_id": "my-project-id",
"private_key_id": "c8myid12345",
"private_key": "-----BEGIN PRIVATE KEY-----\nABCDEFG...XYZ=\n-----END
PRIVATE KEY-----\n",
"client_email": "[email protected]",
"client_id": "123456",
"auth_uri": "https://accounts.google.com/o/oauth2/auth";,
"token_uri": "https://oauth2.googleapis.com/token";,
"auth_provider_x509_cert_url":
"https://www.googleapis.com/oauth2/v1/certs";,
"client_x509_cert_url":
"https://www.googleapis.com/robot/v1/metadata/x509/example-cert-url%40identity-example.iam.gserviceaccount.com";
}
```
It seems that the key file path is incorrect. However, when I tried altering
the key file to contain invalid JSON, I got an error message that the JSON was
invalid when I triggered a manual offload of the topic, which implies that
Pulsar was able to access the key file.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
