[
https://issues.apache.org/jira/browse/CRUNCH-547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Wills updated CRUNCH-547:
------------------------------
Attachment: CRUNCH-547b.patch
Added a test that will fail w/o the fix, will commit this rev soon.
[~aeckstein] I think at this point the assumption of nullable fields inside of
a Tuple (Pair, Tuple3, etc., including the value-side of the implicit Pair<K,
V> inside of a PTable) is so baked in that we couldn't back it out w/o breaking
a bunch of pipelines. Billion dollar mistake and all that.
> AvroType operators can create nested unions
> -------------------------------------------
>
> Key: CRUNCH-547
> URL: https://issues.apache.org/jira/browse/CRUNCH-547
> Project: Crunch
> Issue Type: Bug
> Components: IO
> Reporter: Adric Eckstein
> Attachments: CRUNCH-547.patch, CRUNCH-547b.patch
>
>
> When using avro reflection to create PTypes, the type might represent a union
> type, as specified by the @Union annotation. Derived types from this, such
> as tableOf() and collections() will create a null union from this (already
> union) type and create a nested union which throws an AvroRuntimeException.
> @org.apache.avro.reflect.Union({ Foo1.class, Foo2.class })
> public static interface Foo {
> public String getFoo();
> }
> public static class Foo1 implements Foo {
> private final Double value = 1.0;
> @Override
> public String getFoo() {
> return value.toString();
> }
> }
> public static class Foo2 implements Foo {
> private final Integer value = 2;
> @Override
> public String getFoo() {
> return value.toString();
> }
> }
> Schema schema = ReflectData.get().getSchema(Foo.class);
> AvroType<Foo> ptype = Avros.reflects(Foo.class, schema);
> Avros.tableOf(Avros.strings(), ptype);
> Avros.collections(ptype);
> Exception in thread "main" org.apache.avro.AvroRuntimeException: Nested
> union: [["double","string"],"null"]
> ...
> Seems like the methods Avros.allowNulls() and AvroTableType.nullable() both
> are calling Schema.createUnion(unionSchema, nullSchema), and could add an
> extra case statement to use the already existing union schema.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
