[
https://issues.apache.org/jira/browse/BEAM-6607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761983#comment-16761983
]
Kenneth Knowles commented on BEAM-6607:
---------------------------------------
Interesting catch. The code looks like it does what you expect:
https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/Schema.java#L505
But in fact this API is super confusing and I'd call it a usability bug. We
should just delete that static method and let the user call {{withNullable}} on
the element type. But if you delete that method and adjust your code, it looks
like there's still a bug in the coder.
[~reuvenlax] is a good reviewer for your PR
> SchemaCoder cannot encode row with null value in array
> ------------------------------------------------------
>
> Key: BEAM-6607
> URL: https://issues.apache.org/jira/browse/BEAM-6607
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
> Reporter: Mike Pedersen
> Assignee: Kenneth Knowles
> Priority: Major
>
> This example fails with a "cannot encode null Integer" exception:
> {code:java}
> import com.google.common.io.ByteStreams;
> import org.apache.beam.sdk.schemas.Schema;
> import org.apache.beam.sdk.schemas.SchemaCoder;
> import org.apache.beam.sdk.values.Row;
> import java.io.IOException;
> import java.util.Collections;
> public class Main {
> public static void main(String[] args) throws IOException {
> Schema schema = Schema.builder()
> .addField("a", Schema.FieldType.array(Schema.FieldType.INT32,
> true))
> .build();
> Row row =
> Row.withSchema(schema).addValue(Collections.singletonList(null)).build();
> SchemaCoder.of(schema).encode(row, ByteStreams.nullOutputStream());
> }
> }{code}
> Note that null in the array should be OK, as the nullable parameter to
> Schema.FieldType.array is true.
> An easy way of solving this could be to wrap inner coders with a
> NullableCoder, but a better way would probably to have something like a
> NullableIterableCoder that uses a bitset similarly to how the SchemaCoder
> encodes nullable fields.
> I'll probably take a stab at fixing this and making a pull request.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
