[
https://issues.apache.org/jira/browse/BEAM-6566?focusedWorklogId=193463&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-193463
]
ASF GitHub Bot logged work on BEAM-6566:
----------------------------------------
Author: ASF GitHub Bot
Created on: 01/Feb/19 17:32
Start Date: 01/Feb/19 17:32
Worklog Time Spent: 10m
Work Description: nickpan47 commented on issue #7700: [BEAM-6566] Add
IWYU plugin, activate for Beam SQL, fix errors
URL: https://github.com/apache/beam/pull/7700#issuecomment-459802274
@kennknowles liked the idea to add end-to-end tests. I noticed that there
are some user example project such as word-count-example that are generated as
part of the build. It would be nice to add some SQL example projects their as
well, and make sure the release validation process actually builds and runs
those example applications. I can create a JIRA to add BeamSQL example project,
if it is not there yet.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 193463)
Time Spent: 50m (was: 40m)
> SqlTransform does not work for beam version above 2.6.0
> -------------------------------------------------------
>
> Key: BEAM-6566
> URL: https://issues.apache.org/jira/browse/BEAM-6566
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql
> Affects Versions: 2.7.0, 2.8.0, 2.9.0
> Reporter: Xuefeng Zhang
> Assignee: Kenneth Knowles
> Priority: Critical
> Fix For: 2.10.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> *Issue*:
> Beam versions above 2.6.0 do not work for SqlTransform. By looking at the
> code, those versions use PCollection.getSchema, this function never works
> even for 2.6.0
> *Details:*
> Beam 2.6.0, class BeamPcollectionTable which is used by SqlTransform:
> public BeamPCollectionTable(PCollection<Row> upstream) \{ super(((RowCoder)
> upstream.getCoder()).getSchema()); this.upstream = upstream; }
> But for Beam 2.7.0 and 2.8.0, it is changed to :
> https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/schema/BeamPCollectionTable.java
> So that got below errors after upgrading beam version from 2.6.0 to 2.9.0
> java.lang.IllegalStateException: Cannot call getSchema when there is no schema
> at org.apache.beam.sdk.values.PCollection.getSchema(PCollection.java:328)
> at
> org.apache.beam.sdk.extensions.sql.impl.schema.BeamPCollectionTable.<init>(BeamPCollectionTable.java:34)
> at
> org.apache.beam.sdk.extensions.sql.SqlTransform.toTableMap(SqlTransform.java:111)
> at
> org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:91)
> at
> org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:76)
> at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
> at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
> at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
> *Codes:*
> Schema schema = Schema.builder()
> .addStringField("weightMarketValue")
> .addStringField("ticker")
> .addStringField("ratingLongTermFitchRaw")
> .build();
> Row row = Row.withSchema(schema)
> .addValues("weightMarketValue 1", "ticker 1", "ratingLongTermFitchRaw 1")
> .build();
> Version 1:
>
> {color:#FF0000}PCollection<Row> input = p.apply(Create.of(row){color}
> {color:#FF0000} .withSchema(schema, SerializableFunctions.identity(),
> SerializableFunctions.identity()){color}
> {color:#FF0000} .withCoder(RowCoder.of(schema)));{color}
> PCollection<Row> output = input.apply(SqlTransform.query("select * from
> PCOLLECTION"));
> Version 2:
> {color:#FF0000}PCollection<Row> input = p.apply(Create.of(row){color}
> {color:#FF0000}.withRowSchema(schema){color}
> {color:#FF0000}.withCoder(RowCoder.of(schema)));{color}
> PCollection<Row> output = input.apply(SqlTransform.query("select * from
> PCOLLECTION"));
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
