[
https://issues.apache.org/jira/browse/AVRO-3154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426011#comment-17426011
]
Rasmus Helbig Hansen edited comment on AVRO-3154 at 10/8/21, 7:07 AM:
----------------------------------------------------------------------
I've noticed same, but after reading the internal logic, it is clear that the
custom coders feature does *not* support logical types. The support for enums
also appears slightly reduced when writer schema != reader schema.
However, I don't worry much about it, because the fast-reader feature, which is
enabled by default (and prevents custom decode from being called, unless
disabled), is often _faster_ than the custom decode equivalent. Custom decode
is walking the schema during decode, and is doing other tidbits during every
message decode, which is making it relatively slow. Custom encode on the other
hand can be a fair bit faster than the default.
I did make a patch for myself, in order to have support for logical types, but
it is definitely not housebroken. Without some refactoring, to get rid of the
dependency to the ResolvingDecoder, the custom decode isn't really fast either.
There is potential for having much faster decode though.
was (Author: rasmusatnordea):
I've noticed same, but after reading the internal logic, it is clear that the
custom coders feature does *not* support logical types. The support for enums
also appears slightly reduced when writer schema != reader schema.
However, I don't worry much about it, because the fast-reader feature, which is
enabled by default (and prevents custom decode from being called, unless
disabled), is often _faster_ than the custom decode equivalent. Custom decode
is walking the schema during decode, and is during other tidbits during every
message decode, which is making it relatively slow. Custom encode on the other
hand can be a fair bit faster than the default.
I did make a patch for myself, in order to have support for logical types, but
it is definitely not housebroken. Without some refactoring, to get rid of the
dependency to the ResolvingDecoder, the custom decode isn't really fast either.
There is potential for having much faster decode though.
> Missing customEncode method for Java class generated from schema with logical
> type timestamp-millis
> ---------------------------------------------------------------------------------------------------
>
> Key: AVRO-3154
> URL: https://issues.apache.org/jira/browse/AVRO-3154
> Project: Apache Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.10.2
> Environment: Java version:
> OpenJDK Runtime Environment AdoptOpenJDK-16.0.1+9 (build 16.0.1+9)
> OpenJDK 64-Bit Server VM AdoptOpenJDK-16.0.1+9 (build 16.0.1+9, mixed mode,
> sharing)
> MacOs default console
> Reporter: Jakub Hałun
> Priority: Major
> Attachments: TableChangeAvro.java
>
>
> I'm trying to compile avro schema to java class with the command:
> {{java -jar avro-tools-1.10.2.jar compile schema }}
> Prepared schema is as follows:
> ^{^
> ^"type": "record",^
> ^"name": "TableChangeAvro",^
> ^"fields": [^
> ^{^
> ^"name": "ownerProjectId",^
> ^"type": "string"^
> ^},^
> ^{^
> ^"name": "dataset",^
> ^"type": "string"^
> ^},^
> ^{^
> ^"name": "table",^
> ^"type": "string"^
> ^},^
> ^{^
> ^"name": "timestamp",^
> ^"type" : {^
> ^"type" : "long",^
> ^"logicalType" : "timestamp-millis"^
> ^}^
> ^}^
> ^]^
> ^}^
> The schema compiles successfully to Java class (attached), but the class has
> missing methods:
> public void customEncode(Encoder out) throws IOException
> public void customDecode(ResolvingDecoder in) throws IOException
>
> Because of that, I'm unable to use it for Google Pub/Sub publication as
> intended. The problem occurs for the schema only when the type with
> timestamp-millis logical type is included.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
