[
https://issues.apache.org/jira/browse/AVRO-797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Douglas Kaminsky updated AVRO-797:
----------------------------------
Description:
In my application, I generate an Avro protocol programmatically based on a
model. While my model permits me to include things of type java.util.Object or
java.lang.Enum, Avro only supports these notions via the concept of unions.
Therefore, to support this functionality, I iterate over my list of schemas and
create a union of every type (or every enum type) in the protocol (and the null
schema) to create my "any type" schema. This would be sufficient except that
this new schema doesn't exist when I create the schemas in my model that would
use it, so I need to put in a bogus placeholder schema, and this creates the
added complexity of needing to programatically reconstruct my schemas, which is
risky and problematic at best.
What I propose to address this is a new type of schema that acts as a layer of
indirection and is not resolved during protocol generation (so no schema
reconstruction is necessary).
Use Case #1: Translate generic "object" from model into Avro Schema
Field field = new Field("payload", Schema.createAny(), null, null); // default
- any named type
List<Field> fields = new ArrayList<Field>();
fields.add(field);
Schema record = Schema.createRecord("Foo", null, "com.acme", false);
record.setFields(fields);
Use Case #2: Translate generic "enum" from model into Avro Schema
Field field = new Field("payload", Schema.createAny(Type.ENUM), null, null); //
specifies a placeholder only for enums
List<Field> fields = new ArrayList<Field>();
fields.add(field);
Schema record = Schema.createRecord("Foo", null, "com.acme", false);
record.setFields(fields);
The consumer of the protocol should treat an "any" type schema as the union of
every named type in the protocol, constrained by the specified Schema type.
If there is sufficient interest, I would be willing to provide a Java patch for
this functionality.
was:
In my application, I generate an Avro protocol programmatically based on a
model. While my model permits me to include things of type java.util.Object or
java.lang.Enum, Avro only supports these notions via the notion of unions.
Therefore, to support this functionality, I iterate over my list of schemas and
create a union of every type (or every enum type) in the protocol (and the null
schema) to create my "any type" schema. This would be sufficient except that
this new schema doesn't exist when I create the schemas in my model that would
use it, so I need to put in a bogus placeholder schema, and this creates the
added complexity of needing to programatically reconstruct my schemas, which is
risky and problematic at best.
What I propose to address this is a new type of schema that acts as a layer of
indirection and is not resolved during protocol generation (so no schema
reconstruction is necessary).
Use Case #1: Translate generic "object" from model into Avro Schema
Field field = new Field("payload", Schema.createAny(), null, null); // default
- any named type
List<Field> fields = new ArrayList<Field>();
fields.add(field);
Schema record = Schema.createRecord("Foo", null, "com.acme", false);
record.setFields(fields);
Use Case #2: Translate generic "enum" from model into Avro Schema
Field field = new Field("payload", Schema.createAny(Type.ENUM), null, null); //
specifies a placeholder only for enums
List<Field> fields = new ArrayList<Field>();
fields.add(field);
Schema record = Schema.createRecord("Foo", null, "com.acme", false);
record.setFields(fields);
The consumer of the protocol should treat an "any" type schema as the union of
every named type in the protocol, constrained by the specified Schema type.
If there is sufficient interest, I would be willing to provide a Java patch for
this functionality.
> Built-in support for "any" types
> --------------------------------
>
> Key: AVRO-797
> URL: https://issues.apache.org/jira/browse/AVRO-797
> Project: Avro
> Issue Type: New Feature
> Components: java, spec
> Affects Versions: 1.5.0
> Reporter: Douglas Kaminsky
>
> In my application, I generate an Avro protocol programmatically based on a
> model. While my model permits me to include things of type java.util.Object
> or java.lang.Enum, Avro only supports these notions via the concept of unions.
> Therefore, to support this functionality, I iterate over my list of schemas
> and create a union of every type (or every enum type) in the protocol (and
> the null schema) to create my "any type" schema. This would be sufficient
> except that this new schema doesn't exist when I create the schemas in my
> model that would use it, so I need to put in a bogus placeholder schema, and
> this creates the added complexity of needing to programatically reconstruct
> my schemas, which is risky and problematic at best.
> What I propose to address this is a new type of schema that acts as a layer
> of indirection and is not resolved during protocol generation (so no schema
> reconstruction is necessary).
> Use Case #1: Translate generic "object" from model into Avro Schema
> Field field = new Field("payload", Schema.createAny(), null, null); //
> default - any named type
> List<Field> fields = new ArrayList<Field>();
> fields.add(field);
> Schema record = Schema.createRecord("Foo", null, "com.acme", false);
> record.setFields(fields);
> Use Case #2: Translate generic "enum" from model into Avro Schema
> Field field = new Field("payload", Schema.createAny(Type.ENUM), null, null);
> // specifies a placeholder only for enums
> List<Field> fields = new ArrayList<Field>();
> fields.add(field);
> Schema record = Schema.createRecord("Foo", null, "com.acme", false);
> record.setFields(fields);
> The consumer of the protocol should treat an "any" type schema as the union
> of every named type in the protocol, constrained by the specified Schema type.
> If there is sufficient interest, I would be willing to provide a Java patch
> for this functionality.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
