enriched commented on issue #7675:
URL: https://github.com/apache/pulsar/issues/7675#issuecomment-694520538
@Shenfeng1011 is this fixed for you now? I am trying to run the presto
connector with offloading enabled and appear to be getting the same exception:
```
2020-09-17T19:49:39.093Z ERROR remote-task-callback-11
io.airlift.concurrent.BoundedExecutor Task failed
java.lang.IllegalArgumentException: io.prestosql.server.TaskUpdateRequest
could not be converted to JSON
at io.airlift.json.JsonCodec.toJsonBytes(JsonCodec.java:214)
at
io.prestosql.server.remotetask.HttpRemoteTask.sendUpdate(HttpRemoteTask.java:513)
at
io.airlift.concurrent.BoundedExecutor.drainQueue(BoundedExecutor.java:78)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No
serializer found for class
org.apache.pulsar.common.policies.data.OffloadPolicies and no properties
discovered to create BeanSerializer (to avoid exception, disable
SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain:
io.prestosql.server.TaskUpdateRequest["sources"]->com.google.common.collect.SingletonImmutableList[0]->io.prestosql.execution.TaskSource["splits"]->com.google.common.collect.RegularImmutableSet[0]->io.prestosql.execution.ScheduledSplit["split"]->io.prestosql.metadata.Split["connectorSplit"]->org.apache.pulsar.sql.presto.PulsarSplit["offloadPolicies"])
at
com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
at
com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
at
com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:404)
at
com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.failForEmpty(UnknownSerializer.java:71)
at
com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.serialize(UnknownSerializer.java:33)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:722)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSerializerBase.java:607)
at
io.prestosql.metadata.AbstractTypedJacksonModule$InternalTypeSerializer.serialize(AbstractTypedJacksonModule.java:115)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:722)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:166)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:722)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:166)
at
com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:145)
at
com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107)
at
com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:722)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:166)
at
com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
at
com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
at
com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:722)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:166)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:400)
at
com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1429)
at
com.fasterxml.jackson.databind.ObjectWriter._configAndWriteValue(ObjectWriter.java:1135)
at
com.fasterxml.jackson.databind.ObjectWriter.writeValueAsBytes(ObjectWriter.java:1029)
at io.airlift.json.JsonCodec.toJsonBytes(JsonCodec.java:211)
... 5 more
```
I do have a slightly different setup though. I am installing presto using
[prestosql v332
tarball](https://repo1.maven.org/maven2/io/prestosql/presto-server/332/presto-server-332.tar.gz)
and then installing the [pulsar-presto-connector
tarball](https://maxviewpublic.blob.core.windows.net/pulsar/pulsar-presto-connector.tar.gz)
that gets built in `/pulsar-sql/presto-plusar-plugin`.
My presto config for the pulsar connector looks like:
```properties
connector.name=pulsar
pulsar.broker-service-url=http://my-pulsar-broker:8080
pulsar.zookeeper-uri=my-pulsar-zookeeper:2181
pulsar.max-entry-read-batch-size=100
pulsar.target-num-splits=2
pulsar.max-split-message-queue-size=10000
pulsar.max-split-entry-queue-size=1000
pulsar.managed-ledger-offload-driver=s3
pulsar.offloaders-directory=/pulsar/offloaders
pulsar.managed-ledger-offload-max-threads=2
pulsar.offloader-properties = \
{"s3ManagedLedgerOffloadBucket": "pulsar-offload", \
"s3ManagedLedgerOffloadServiceEndpoint": "http://my-minio:9000";, \
"s3ManagedLedgerOffloadRegion": "us-east-1"}
pulsar.namespace-delimiter-rewrite-enable=false
pulsar.rewrite-namespace-delimiter=/
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
