[
https://issues.apache.org/jira/browse/BEAM-13294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Pilloud updated BEAM-13294:
----------------------------------
Resolution: Fixed
Status: Resolved (was: Open)
> innerBroadcastJoin fails when batch column Nullable, streaming column not
> -------------------------------------------------------------------------
>
> Key: BEAM-13294
> URL: https://issues.apache.org/jira/browse/BEAM-13294
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql, dsl-sql-zetasql, sdk-java-core
> Affects Versions: 2.21.0, 2.22.0, 2.23.0, 2.24.0, 2.25.0, 2.26.0, 2.27.0,
> 2.28.0, 2.29.0, 2.30.0, 2.31.0, 2.32.0, 2.33.0, 2.34.0, 2.35.0
> Reporter: Andrew Pilloud
> Assignee: Andrew Pilloud
> Priority: P1
> Fix For: 2.36.0
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> The join always fail when given a batch column that is Nullable and a
> streaming column that is Not Nullable. This was discovered in
> BeamSideInputJoinRel but is likely an issue in the Beam core join library.
> Trivial reproduction with a small test diff:
> ./gradlew :runners:direct-java:needsRunnerTests --tests
> org.apache.beam.sdk.schemas.transforms.JoinTest.testInnerJoinDifferentKeys
> {code}
> ---
> a/sdks/java/core/src/test/java/org/apache/beam/sdk/schemas/transforms/JoinTest.java
> +++
> b/sdks/java/core/src/test/java/org/apache/beam/sdk/schemas/transforms/JoinTest.java
> @@ -51,7 +51,7 @@ public class JoinTest {
> .build();
> private static final Schema CG_SCHEMA_2 =
> Schema.builder()
> - .addStringField("user2")
> + .addNullableField("user2", Schema.FieldType.STRING)
> .addInt32Field("count2")
> .addStringField("country2")
> .build();
> {code}
> At a lower level:
> ./gradlew :runners:direct-java:needsRunnerTests --tests
> org.apache.beam.sdk.schemas.transforms.CoGroupTest.testCoGroupByDifferentFields
> {code}
> ---
> a/sdks/java/core/src/test/java/org/apache/beam/sdk/schemas/transforms/CoGroupTest.java
> +++
> b/sdks/java/core/src/test/java/org/apache/beam/sdk/schemas/transforms/CoGroupTest.java
> @@ -237,7 +237,7 @@ public class CoGroupTest {
>
> private static final Schema CG_SCHEMA_2 =
> Schema.builder()
> - .addStringField("user2")
> + .addNullableField("user2", Schema.FieldType.STRING)
> .addInt32Field("count2")
> .addStringField("country2")
> .build();
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
