GitHub user y00273676 created a discussion: Within EKS to offload a topic to S3
got AuthorizationException
**Describe the bug**
As deploying Pulsar inside AWS EKS (Kubernetes hosted by AWS), while trying to
use the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to assume the AWS
credential in order to offload ledgers into S3 Bucket, I've encountered
exceptions like:
```
Error in offload
null
Reason: Error offloading: org.apache.bookkeeper.mledger.ManagedLedgerException:
java.util.concurrent.CompletionException:
org.jclouds.rest.AuthorizationException: The AWS Access Key Id you provided
does not exist in our records.
command terminated with exit code 1
```
Login the eks pod
```
root@pulsar-broker-3:/pulsar# echo $AWS_SECRET_ACCESS_KEY
****************URQQ
root@pulsar-broker-3:/pulsar# echo $AWS_SECRET_ACCESS_KEY
****************I+Gz
root@pulsar-broker-3:/pulsar# aws configure list
Name Value Type Location
---- ----- ---- --------
profile <not set> None None
access_key ****************URQQ env
secret_key ****************I+Gz env
region cn-northwest-1 env ['AWS_REGION',
'AWS_DEFAULT_REGION']
```
So the AWS_SECRET_ACCESS_KEY and AWS_SECRET_ACCESS_KEY is valid
**To Reproduce**
Steps to reproduce the behavior:
precondition： the aws s3 bucket was created
1. Deploy Pulsar Broker within a EKS pod, if that is too costy, you can inject
environement variables and broker.conf like this:
```
broker.conf:
managedLedgerOffloadDriver: "aws-s3"
s3ManagedLedgerOffloadBucket: "xxxxxxx"
s3ManagedLedgerOffloadRegion: "cn-northwest-1"
env:
- name: "AWS_ACCESS_KEY_ID"
value: "****************URQQ "
- name: "AWS_SECRET_ACCESS_KEY"
value: "****************I+Gz"
```
2. Set the. offload threshold manually
```
kubectl -n pulsar exec -ti toolset-0 -c pulsar-toolset -- ./bin/pulsar-admin
--auth-params "token:xxxxxxxxxx" --auth-plugin
"org.apache.pulsar.client.impl.auth.AuthenticationToken" --admin-url
http://pulsar-broker.pulsar:8080 topics offload --size-threshold 1K yourTopic
```
3. Send pulsar msg to you topic
4. Get offload status
```
kubectl -n pulsar exec -ti toolset-0 -c pulsar-toolset -- ./bin/pulsar-admin
--auth-params "token:xxxxxxxx" --auth-plugin
"org.apache.pulsar.client.impl.auth.AuthenticationToken" --admin-url
http://pulsar-broker.pulsar:8080 topics offload-status -w yourTopic
```
GitHub link: https://github.com/apache/pulsar/discussions/19791
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
