[
https://issues.apache.org/jira/browse/BEAM-6607?focusedWorklogId=195534&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-195534
]
ASF GitHub Bot logged work on BEAM-6607:
----------------------------------------
Author: ASF GitHub Bot
Created on: 07/Feb/19 08:38
Start Date: 07/Feb/19 08:38
Worklog Time Spent: 10m
Work Description: Noctune commented on pull request #7770: [BEAM-6607]
Fix bug where RowCoder fails if array/map has null values
URL: https://github.com/apache/beam/pull/7770
This wraps inner coders in a NullableCoder if the inner type of an array or
value type of a map is nullable.
I fixed this slightly differently than was suggested in the JIRA issue.
Instead of adding it for both `mapCoder` and `listCoder`, I wrap the coder in
`getCoder`. This makes more sense to me as nullability is (now) a property of
the FieldType itself. I also deprecated the nullable array and map methods that
@kennknowles considered a usability bug.
I also moved the RowCoder tests. They were in a weird location that did not
seem like it was on purpose.
@reuvenlax was suggested as reviewer.
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/)
| --- | --- | --- | --- | --- | ---
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
| --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
</br> [](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/)
| --- | --- | ---
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 195534)
Time Spent: 10m
Remaining Estimate: 0h
> SchemaCoder cannot encode row with null value in array
> ------------------------------------------------------
>
> Key: BEAM-6607
> URL: https://issues.apache.org/jira/browse/BEAM-6607
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
> Reporter: Mike Pedersen
> Assignee: Mike Pedersen
> Priority: Major
> Labels: triaged
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This example fails with a "cannot encode null Integer" exception:
> {code:java}
> import com.google.common.io.ByteStreams;
> import org.apache.beam.sdk.schemas.Schema;
> import org.apache.beam.sdk.schemas.SchemaCoder;
> import org.apache.beam.sdk.values.Row;
> import java.io.IOException;
> import java.util.Collections;
> public class Main {
> public static void main(String[] args) throws IOException {
> Schema schema = Schema.builder()
> .addField("a", Schema.FieldType.array(Schema.FieldType.INT32,
> true))
> .build();
> Row row =
> Row.withSchema(schema).addValue(Collections.singletonList(null)).build();
> SchemaCoder.of(schema).encode(row, ByteStreams.nullOutputStream());
> }
> }{code}
> Note that null in the array should be OK, as the nullable parameter to
> Schema.FieldType.array is true.
> An easy way of solving this could be to wrap inner coders with a
> NullableCoder, but a better way would probably to have something like a
> NullableIterableCoder that uses a bitset similarly to how the SchemaCoder
> encodes nullable fields.
> I'll probably take a stab at fixing this and making a pull request.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
