[
https://issues.apache.org/jira/browse/AVRO-2493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17124337#comment-17124337
]
Dan Lipofsky commented on AVRO-2493:
------------------------------------
[~pavel.likin] [~trav017] is it possible to use custom conversions with IDL?
Decimal seems to work OK but not {{FixedSizeString}}.
In particular with the {{FixedSizeString}} example, which needs {{minLength}}
and {{maxLength}}, what would be the syntax to pass these in from IDL?
also I tried this
{noformat}
@namespace("org.apache.avro.codegentest.testdata")
protocol CustConv {
record LogicalTypesWithCustomConversion {
decimal(9,2) nonNullCustomField;
union { null, decimal(9,2) } nullableCustomField;
@logicalType("fixed-size-string")
string nonNullFixedSizeString;
}
}
{noformat}
Which causes {{nonNullFixedSizeString}} to be {{String}} instead of
{{FixedSizeString}} in the DTO.
(if I use {{bytes}} instead of {{string}} the DTO has {{ByteBuffer}} which is
still no good)
After modifying {{TestCustomConversion}} to compile, I see running it generates
{noformat}
[main] WARN org.apache.avro.LogicalTypes - Ignoring invalid logical type for
name: fixed-size-string
{noformat}
and not surprisingly doesn't appear to call the {{validate}} method in
{{FixedSizeStringConversion}}.
The {{testNullValues}} method also isn't happy and throws
{noformat}
Field nullableCustomField type:UNION pos:1 not set and has no default value
Path in schema: --> nullableCustomField
at
org.apache.avro.generic.GenericData.getDefaultValue(GenericData.java:1176)
at
org.apache.avro.data.RecordBuilderBase.defaultValue(RecordBuilderBase.java:138)
at
org.apache.avro.codegentest.testdata.LogicalTypesWithCustomConversion$Builder.build(LogicalTypesWithCustomConversion.java:403)
at
org.apache.avro.codegentest.TestCustomConversion.testNullValues(TestCustomConversion.java:39)
{noformat}
This testing was done using the code from {{codegen-test}}, stripped down to
just the custom conversion stuff and minimally modified.
> Unable to register Logical Type for custom Conversion class
> -----------------------------------------------------------
>
> Key: AVRO-2493
> URL: https://issues.apache.org/jira/browse/AVRO-2493
> Project: Apache Avro
> Issue Type: Bug
> Components: java, logical types
> Affects Versions: 1.9.0
> Reporter: Travis Yocum
> Priority: Major
> Fix For: 1.10.0
>
>
> I have created a custom conversion class for Java's (1.8)
> java.time.OffsetDateTime.
> {code:java}
> public class OffsetDateTimeConversion extends Conversion<OffsetDateTime> {
> private static final DateTimeFormatter DATE_TIME_FORMATTER =
> DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ss.nnnnnnZZZZZ");
> @Override
> public Class<OffsetDateTime> getConvertedType() {
> return OffsetDateTime.class;
> }
> @Override
> public String getLogicalTypeName() {
> return "offset-date-time";
> }
> @Override
> public OffsetDateTime fromCharSequence(CharSequence value, Schema schema,
> LogicalType type) {
> return OffsetDateTime.parse(value, DATE_TIME_FORMATTER);
> }
> @Override
> public CharSequence toCharSequence(OffsetDateTime value, Schema schema,
> LogicalType type) {
> return value.format(DATE_TIME_FORMATTER);
> }
> }
> {code}
> And my simple schema to test (including a field that uses the built-in
> "time-millis" converter in Avro 1.9 for testing purposes):
> TimeTest.avsc
>
> {code:java}
> {
> "type": "record",
> "name": "TimeTest",
> "namespace": "time.test",
> "fields": [
> {
> "name": "createTime",
> "type": {
> "type": "int",
> "logicalType": "time-millis"
> }
> },
> {
> "name": "createDateTime",
> "type": {
> "type": "string",
> "logicalType": "offset-date-time"
> }
> }
> ]
> }
> {code}
>
> I've also created a LogicalType for my conversion field that I need to
> register:
>
> {code:java}
> public class OffsetDateTimeLogicalType extends LogicalType {
> public static final String LOGICAL_DATE_TIME_NAME = "offset-date-time";
> public OffsetDateTimeLogicalType() {
> super(LOGICAL_DATE_TIME_NAME);
> }
> @Override
> public void validate(Schema schema) {
> super.validate(schema);
> if (schema.getType() != Schema.Type.STRING) {
> throw new IllegalArgumentException("Logical type
> 'offset-date-time' must be of type string");
> }
> }
> }{code}
>
> I've been debugging the avro-maven-plugin and have been able to pinpoint
> where my issue is occurring while parsing the schema:
>
> {code:java}
> // parse logical type if present
> result.logicalType = LogicalTypes.fromSchemaIgnoreInvalid(result);
> {code}
>
> The schema's "logicalType" property is being set to null because it's not a
> registered logical type.
> The code I need to execute is:
>
> {code:java}
> LogicalTypes.register(getLogicalTypeName(), schema -> new
> OffsetDateTimeLogicalType());
> {code}
> Without modifying the plugin code, I see no way to execute this register so
> that the Schema.parse method (which is called before the conversion class is
> executed) will recognize this LogicalType.
>
> The plugin has a config property <enableDecimalLogicalType> to allow for
> registering the Decimal logical types but nothing to enable other logical
> types.
> Am I missing something or is this a real issue?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
