[
https://issues.apache.org/jira/browse/BEAM-9992?focusedWorklogId=457560&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-457560
]
ASF GitHub Bot logged work on BEAM-9992:
----------------------------------------
Author: ASF GitHub Bot
Created on: 11/Jul/20 12:23
Start Date: 11/Jul/20 12:23
Worklog Time Spent: 10m
Work Description: darshanj edited a comment on pull request #11967:
URL: https://github.com/apache/beam/pull/11967#issuecomment-657055611
@amaliujia
Thanks for pointing out.
I tried to debug and see coder is throwing the exception.
Any idea/suggestion how do we fix this/make it work? Test below it passes.
But combination of byte string and null fails.
```
@Test
public void testUnionAllBytes() {
String sql = "SELECT DISTINCT val.BYTES "
+"from (select b\"1\" BYTES union all "
// +"select cast(NULL as bytes) union all "
+"select b\"-1\" union all "
+"select b\"1\" ) val";
// +"select cast(NULL as bytes)) val";
ZetaSQLQueryPlanner zetaSQLQueryPlanner = new
ZetaSQLQueryPlanner(config);
BeamRelNode beamRelNode = zetaSQLQueryPlanner.convertToBeamRel(sql);
PCollection<Row> stream = BeamSqlRelUtils.toPCollection(pipeline,
beamRelNode);
final Schema schema =
Schema.builder().addNullableField("field1",FieldType.BYTES).build();
byte[] nullValue = null;
PAssert.that(stream)
.containsInAnyOrder(
// Row.withSchema(schema).addValues(nullValue).build(),
Row.withSchema(schema).addValues("-1".getBytes(StandardCharsets.UTF_8)).build(),
Row.withSchema(schema).addValues("1".getBytes(StandardCharsets.UTF_8)).build());
pipeline.run().waitUntilFinish(Duration.standardMinutes(PIPELINE_EXECUTION_WAITTIME_MINUTES));
}
```
Was this scenerio working with old code, as i don't see such a test in
BeamUnionRelTest?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 457560)
Time Spent: 3h (was: 2h 50m)
> Migrate BeamSQL's SET operators to Sets transforms
> --------------------------------------------------
>
> Key: BEAM-9992
> URL: https://issues.apache.org/jira/browse/BEAM-9992
> Project: Beam
> Issue Type: New Feature
> Components: dsl-sql, dsl-sql-zetasql
> Reporter: Darshan Jani
> Priority: P2
> Time Spent: 3h
> Remaining Estimate: 0h
>
> As par of [BEAM-9946|https://issues.apache.org/jira/browse/BEAM-9946] we have
> new Sets transforms for intersect,union and except.
> This jira is to use them to remove existing Set operators in BeamSQL code.
> Tasks:
> # Remove:
> [BeamSetOperatorRelBase.java|https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamSetOperatorRelBase.java]
> # use SetFns transforms from
> ##
> [BeamIntersectRel.java|https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamIntersectRel.java]
> ##
> [BeamMinusRel.java|https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamMinusRel.java]
> ##
> [BeamMinusRel.java|https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamMinusRel.java]
> ##
> [BeamUnionRel.java|https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamUnionRel.java]
> #
> Remove:[BeamSetOperatorsTransforms.java|https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/transform/BeamSetOperatorsTransforms.java]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
