Shenfeng1011 commented on issue #7675: URL: https://github.com/apache/pulsar/issues/7675#issuecomment-694572650
> @Shenfeng1011 is this fixed for you now? I am trying to run the presto connector with offloading enabled and appear to be getting the same exception: > > ``` > 2020-09-17T19:49:39.093Z ERROR remote-task-callback-11 io.airlift.concurrent.BoundedExecutor Task failed > java.lang.IllegalArgumentException: io.prestosql.server.TaskUpdateRequest could not be converted to JSON > at io.airlift.json.JsonCodec.toJsonBytes(JsonCodec.java:214) > at io.prestosql.server.remotetask.HttpRemoteTask.sendUpdate(HttpRemoteTask.java:513) > at io.airlift.concurrent.BoundedExecutor.drainQueue(BoundedExecutor.java:78) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class org.apache.pulsar.common.policies.data.OffloadPolicies and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: io.prestosql.server.TaskUpdateRequest["sources"]->com.google.common.collect.SingletonImmutableList[0]->io.prestosql.execution.TaskSource["splits"]->com.google.common.collect.RegularImmutableSet[0]->io.prestosql.execution.ScheduledSplit["split"]->io.prestosql.metadata.Split["connectorSplit"]->org.apache.pulsar.sql.presto.PulsarSplit["offloadPolicies"]) > at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77) > at com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191) > at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:404) > at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.failForEmpty(UnknownSerializer.java:71) > at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.serialize(UnknownSerializer.java:33) > at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) > at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:722) > at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSerializerBase.java:607) > at io.prestosql.metadata.AbstractTypedJacksonModule$InternalTypeSerializer.serialize(AbstractTypedJacksonModule.java:115) > at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) > at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:722) > at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:166) > at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) > at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:722) > at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:166) > at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:145) > at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107) > at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25) > at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) > at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:722) > at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:166) > at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119) > at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79) > at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18) > at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) > at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:722) > at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:166) > at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) > at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:400) > at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1429) > at com.fasterxml.jackson.databind.ObjectWriter._configAndWriteValue(ObjectWriter.java:1135) > at com.fasterxml.jackson.databind.ObjectWriter.writeValueAsBytes(ObjectWriter.java:1029) > at io.airlift.json.JsonCodec.toJsonBytes(JsonCodec.java:211) > ... 5 more > ``` > > I do have a slightly different setup though. I am installing presto using [prestosql v332 tarball](https://repo1.maven.org/maven2/io/prestosql/presto-server/332/presto-server-332.tar.gz) and then installing the [pulsar-presto-connector tarball](https://maxviewpublic.blob.core.windows.net/pulsar/pulsar-presto-connector.tar.gz) that gets built in `/pulsar-sql/presto-plusar-plugin`. > > My presto config for the pulsar connector looks like: > > ```ini > connector.name=pulsar > pulsar.broker-service-url=http://my-pulsar-broker:8080 > pulsar.zookeeper-uri=my-pulsar-zookeeper:2181 > pulsar.max-entry-read-batch-size=100 > pulsar.target-num-splits=2 > pulsar.max-split-message-queue-size=10000 > pulsar.max-split-entry-queue-size=1000 > pulsar.managed-ledger-offload-driver=s3 > pulsar.offloaders-directory=/pulsar/offloaders > pulsar.managed-ledger-offload-max-threads=2 > pulsar.offloader-properties = \ > {"s3ManagedLedgerOffloadBucket": "pulsar-offload", \ > "s3ManagedLedgerOffloadServiceEndpoint": "http://my-minio:9000";, \ > "s3ManagedLedgerOffloadRegion": "us-east-1"} > pulsar.namespace-delimiter-rewrite-enable=false > pulsar.rewrite-namespace-delimiter=/ > ``` this issue is resolved in 2.6.1 but i have not verified .you can have a try , ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
