btsken opened a new issue #6837:
URL: https://github.com/apache/pulsar/issues/6837
**Describe the bug**
I'm using apachepulsar/pulsar-all:2.5.1 image. When I query topic and enable
offload in conf/presto/catalog/pulsar.properties, I'll get below error.
```
2020-04-24T07:40:52.124Z INFO
Query-20200424_074049_00000_7eak2-146
org.apache.bookkeeper.mledger.offload.OffloaderUtils Found offloader
OffloaderDefinition(name=filesystem, description=fileSystem based offloader
implementation,
offloaderFactoryClass=org.apache.bookkeeper.mledger.offload.filesystem.FileSystemLedgerOffloaderFactory)
from /pulsar/offloaders/tiered-storage-file-system-2.5.1.nar
2020-04-24T07:40:52.381Z INFO
Query-20200424_074049_00000_7eak2-146
org.apache.bookkeeper.mledger.offload.OffloaderUtils Found and loaded 2
offloaders
2020-04-24T07:40:52.400Z ERROR
Query-20200424_074049_00000_7eak2-146
org.apache.pulsar.sql.presto.PulsarConnectorCache Failed to create
offloader:
2020-04-24T07:40:52.400Z ERROR
Query-20200424_074049_00000_7eak2-146
org.apache.pulsar.sql.presto.PulsarSplitManager Failed to get splits
java.lang.RuntimeException: java.lang.RuntimeException: Either
s3ManagedLedgerOffloadRegion or s3ManagedLedgerOffloadServiceEndpoint must be
set if s3 offload enabled
at
org.apache.pulsar.sql.presto.PulsarConnectorCache.initManagedLedgerOffloader(PulsarConnectorCache.java:178)
at
org.apache.pulsar.sql.presto.PulsarConnectorCache.<init>(PulsarConnectorCache.java:86)
at
org.apache.pulsar.sql.presto.PulsarConnectorCache.getConnectorCache(PulsarConnectorCache.java:92)
at
org.apache.pulsar.sql.presto.PulsarSplitManager.getSplitsNonPartitionedTopic(PulsarSplitManager.java:235)
at
org.apache.pulsar.sql.presto.PulsarSplitManager.getSplits(PulsarSplitManager.java:133)
at
com.facebook.presto.split.SplitManager.getSplits(SplitManager.java:64)
at
com.facebook.presto.sql.planner.DistributedExecutionPlanner$Visitor.visitTableScan(DistributedExecutionPlanner.java:146)
at
com.facebook.presto.sql.planner.DistributedExecutionPlanner$Visitor.visitTableScan(DistributedExecutionPlanner.java:122)
at
com.facebook.presto.sql.planner.plan.TableScanNode.accept(TableScanNode.java:136)
at
com.facebook.presto.sql.planner.DistributedExecutionPlanner.doPlan(DistributedExecutionPlanner.java:108)
at
com.facebook.presto.sql.planner.DistributedExecutionPlanner.doPlan(DistributedExecutionPlanner.java:113)
at
com.facebook.presto.sql.planner.DistributedExecutionPlanner.plan(DistributedExecutionPlanner.java:85)
at
com.facebook.presto.execution.SqlQueryExecution.planDistribution(SqlQueryExecution.java:385)
at
com.facebook.presto.execution.SqlQueryExecution.start(SqlQueryExecution.java:287)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Either s3ManagedLedgerOffloadRegion
or s3ManagedLedgerOffloadServiceEndpoint must be set if s3 offload enabled
at
org.apache.pulsar.sql.presto.PulsarConnectorCache.initManagedLedgerOffloader(PulsarConnectorCache.java:171)
... 16 more
```
but in sql-worker logs, it print
```
2020-04-27T00:51:04.328Z INFO main Bootstrap
pulsar.offloader-properties {}
{s3ManagedLedgerOffloadBucket=pulsar-topic-offload,
s3ManagedLedgerOffloadRegion=Default,
s3ManagedLedgerOffloadServiceEndpoint=http://12.34.56.78:9999}
```
**To Reproduce**
Steps to reproduce the behavior:
1. docker run -it -p 6650:6650 -p 8080:8080 -p 8081:8081 -p 8443:8443 -p
6651:6651 apachepulsar/pulsar-all:2.5.1
2. enter container
3. ./bin/pulsar standalone
4. ./bin/pulsar-client consume public/default/eat -s ken -n 0
5. vim conf/presto/catalog/pulsar.properties
```
pulsar.offloaders-directory = /pulsar/offloaders
pulsar.managed-ledger-offload-max-threads = 2
pulsar.offloader-properties = \
{"s3ManagedLedgerOffloadBucket": "offload-bucket", \
"s3ManagedLedgerOffloadRegion": "us-west-2", \
"s3ManagedLedgerOffloadServiceEndpoint": "http://s3.amazonaws.com"}
```
6. export AWS_ACCESS_KEY_ID=XXXXXXXX
export AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXX
./bin/pulsar sql-worker run
7. ./bin/pulsar sql
8. select * from pulsar."public/default".eat;
**Expected behavior**
get data in "public/default".eat
**Desktop (please complete the following information):**
- OS: ubuntu
ps 2.4.2 can work!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
