[jira] [Created] (BEAM-5733) Pushdown filter to table scan
Rui Wang created BEAM-5733: -- Summary: Pushdown filter to table scan Key: BEAM-5733 URL: https://issues.apache.org/jira/browse/BEAM-5733 Project: Beam Issue Type: Sub-task Components: dsl-sql Reporter: Rui Wang Assignee: Rui Wang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5376) Row interface doesn't support nullability on all fields.
[
https://issues.apache.org/jira/browse/BEAM-5376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang closed BEAM-5376.
--
Resolution: Fixed
Fix Version/s: Not applicable
> Row interface doesn't support nullability on all fields.
>
>
> Key: BEAM-5376
> URL: https://issues.apache.org/jira/browse/BEAM-5376
> Project: Beam
> Issue Type: Improvement
> Components: dsl-sql
>Reporter: Andrew Pilloud
>Assignee: Andrew Pilloud
>Priority: Major
> Fix For: Not applicable
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> For example:
> {code:java}
> public boolean getBoolean(int idx);{code}
> Should be:
> {code:java}
> public Boolean getBoolean(int idx);{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (BEAM-5504) Pubsub Write by BeamSQL
[ https://issues.apache.org/jira/browse/BEAM-5504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang reassigned BEAM-5504: -- Assignee: (was: Rui Wang) > Pubsub Write by BeamSQL > --- > > Key: BEAM-5504 > URL: https://issues.apache.org/jira/browse/BEAM-5504 > Project: Beam > Issue Type: New Feature > Components: dsl-sql >Reporter: Rui Wang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5202) register UDF/UDAF with ServiceLoader
[ https://issues.apache.org/jira/browse/BEAM-5202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang closed BEAM-5202. -- Resolution: Fixed Fix Version/s: Not applicable > register UDF/UDAF with ServiceLoader > > > Key: BEAM-5202 > URL: https://issues.apache.org/jira/browse/BEAM-5202 > Project: Beam > Issue Type: Improvement > Components: dsl-sql >Reporter: Xu Mingmin >Assignee: Xu Mingmin >Priority: Major > Fix For: Not applicable > > Time Spent: 3h 10m > Remaining Estimate: 0h > > create a ServiceLoader for UDF/UDAF register, if the autoLoad option is set > to TRUE. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5696) add table name to row type when creating BeamCalciteTable
Rui Wang created BEAM-5696: -- Summary: add table name to row type when creating BeamCalciteTable Key: BEAM-5696 URL: https://issues.apache.org/jira/browse/BEAM-5696 Project: Beam Issue Type: Improvement Components: dsl-sql Reporter: Rui Wang Assignee: Rui Wang see: [BeamCalciteTable.java#L56|https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamCalciteTable.java#L56] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5690) Issue with GroupByKey in BeamSql using SparkRunner
[
https://issues.apache.org/jira/browse/BEAM-5690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16644016#comment-16644016
]
Rui Wang commented on BEAM-5690:
Cannot find the thread based on the link above ^^
[~kenn]
Can you re-paste or give the title of the thread?
> Issue with GroupByKey in BeamSql using SparkRunner
> --
>
> Key: BEAM-5690
> URL: https://issues.apache.org/jira/browse/BEAM-5690
> Project: Beam
> Issue Type: Task
> Components: runner-spark
>Reporter: Kenneth Knowles
>Priority: Major
>
> Reported on user@
> {quote}We are trying to setup a pipeline with using BeamSql and the trigger
> used is default (AfterWatermark crosses the window).
> Below is the pipeline:
>
>KafkaSource (KafkaIO)
>---> Windowing (FixedWindow 1min)
>---> BeamSql
>---> KafkaSink (KafkaIO)
>
> We are using Spark Runner for this.
> The BeamSql query is:
> {code}select Col3, count(*) as count_col1 from PCOLLECTION GROUP BY Col3{code}
> We are grouping by Col3 which is a string. It can hold values string[0-9].
>
> The records are getting emitted out at 1 min to kafka sink, but the output
> record in kafka is not as expected.
> Below is the output observed: (WST and WET are indicators for window start
> time and window end time)
> {code}
> {"count_col1":1,"Col3":"string5","WST":"2018-10-09 09-55-00
> +","WET":"2018-10-09 09-56-00 +"}
> {"count_col1":3,"Col3":"string7","WST":"2018-10-09 09-55-00
> +","WET":"2018-10-09 09-56-00 +"}
> {"count_col1":2,"Col3":"string8","WST":"2018-10-09 09-55-00
> +","WET":"2018-10-09 09-56-00 +"}
> {"count_col1":1,"Col3":"string2","WST":"2018-10-09 09-55-00
> +","WET":"2018-10-09 09-56-00 +"}
> {"count_col1":1,"Col3":"string6","WST":"2018-10-09 09-55-00
> +","WET":"2018-10-09 09-56-00 +"}
> {"count_col1":0,"Col3":"string6","WST":"2018-10-09 09-55-00
> +","WET":"2018-10-09 09-56-00 +"}
> {"count_col1":0,"Col3":"string6","WST":"2018-10-09 09-55-00
> +","WET":"2018-10-09 09-56-00 +"}
> {"count_col1":0,"Col3":"string6","WST":"2018-10-09 09-55-00
> +","WET":"2018-10-09 09-56-00 +"}
> {"count_col1":0,"Col3":"string6","WST":"2018-10-09 09-55-00
> +","WET":"2018-10-09 09-56-00 +"}
> {"count_col1":0,"Col3":"string6","WST":"2018-10-09 09-55-00
> +","WET":"2018-10-09 09-56-00 +"}
> {"count_col1":0,"Col3":"string6","WST":"2018-10-09 09-55-00
> +","WET":"2018-10-09 09-56-00 0}
> {code}
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (BEAM-5644) make Planner configurable
[ https://issues.apache.org/jira/browse/BEAM-5644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5644: --- Description: We can make planner configurable here: [BeamQueryPlanner.java#L145|https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamQueryPlanner.java#L145] By doing so, we can have different planner implementation to support different SQL dialect. was: We can make planner switchable here: [BeamQueryPlanner.java#L145|https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamQueryPlanner.java#L145] By doing so, we can have different planner implementation to support different SQL dialect. > make Planner configurable > -- > > Key: BEAM-5644 > URL: https://issues.apache.org/jira/browse/BEAM-5644 > Project: Beam > Issue Type: New Feature > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > We can make planner configurable here: > [BeamQueryPlanner.java#L145|https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamQueryPlanner.java#L145] > > By doing so, we can have different planner implementation to support > different SQL dialect. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5644) make Planner configurable
[ https://issues.apache.org/jira/browse/BEAM-5644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5644: --- Summary: make Planner configurable (was: make Planner switchable ) > make Planner configurable > -- > > Key: BEAM-5644 > URL: https://issues.apache.org/jira/browse/BEAM-5644 > Project: Beam > Issue Type: New Feature > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > We can make planner switchable here: > [BeamQueryPlanner.java#L145|https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamQueryPlanner.java#L145] > > By doing so, we can have different planner implementation to support > different SQL dialect. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5641) Move mock classes to sql src/main
[ https://issues.apache.org/jira/browse/BEAM-5641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang closed BEAM-5641. -- Resolution: Fixed Fix Version/s: Not applicable > Move mock classes to sql src/main > - > > Key: BEAM-5641 > URL: https://issues.apache.org/jira/browse/BEAM-5641 > Project: Beam > Issue Type: Improvement > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Fix For: Not applicable > > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5596) Build BeamSQL as a Library
[ https://issues.apache.org/jira/browse/BEAM-5596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5596: --- Summary: Build BeamSQL as a Library (was: BeamSQL as a Library) > Build BeamSQL as a Library > -- > > Key: BEAM-5596 > URL: https://issues.apache.org/jira/browse/BEAM-5596 > Project: Beam > Issue Type: New Feature > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Fix For: Not applicable > > > Build BeamSQL as a jar which does not include any extra dependencies. > Therefore we can use this jar to use BeamSQL as a lib. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5644) make Planner switchable
[ https://issues.apache.org/jira/browse/BEAM-5644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5644: --- Description: We can make planner switchable here: [BeamQueryPlanner.java#L145|https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamQueryPlanner.java#L145] By doing so, we can have different planner implementation to support different SQL dialect. was: We can make planner switchable here: [BeamQueryPlanner.java#L145|[https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamQueryPlanner.java#L145].] By doing so, we can have different planner implementation to support different SQL dialect. > make Planner switchable > > > Key: BEAM-5644 > URL: https://issues.apache.org/jira/browse/BEAM-5644 > Project: Beam > Issue Type: Improvement > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > > We can make planner switchable here: > [BeamQueryPlanner.java#L145|https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamQueryPlanner.java#L145] > > By doing so, we can have different planner implementation to support > different SQL dialect. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5644) make Planner switchable
[ https://issues.apache.org/jira/browse/BEAM-5644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5644: --- Issue Type: New Feature (was: Improvement) > make Planner switchable > > > Key: BEAM-5644 > URL: https://issues.apache.org/jira/browse/BEAM-5644 > Project: Beam > Issue Type: New Feature > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > > We can make planner switchable here: > [BeamQueryPlanner.java#L145|https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamQueryPlanner.java#L145] > > By doing so, we can have different planner implementation to support > different SQL dialect. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5644) make Planner switchable
Rui Wang created BEAM-5644: -- Summary: make Planner switchable Key: BEAM-5644 URL: https://issues.apache.org/jira/browse/BEAM-5644 Project: Beam Issue Type: Improvement Components: dsl-sql Reporter: Rui Wang Assignee: Rui Wang We can make planner switchable here: [BeamQueryPlanner.java#L145|[https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamQueryPlanner.java#L145].] By doing so, we can have different planner implementation to support different SQL dialect. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5641) Move mock classes to sql src/main
Rui Wang created BEAM-5641: -- Summary: Move mock classes to sql src/main Key: BEAM-5641 URL: https://issues.apache.org/jira/browse/BEAM-5641 Project: Beam Issue Type: Improvement Components: dsl-sql Reporter: Rui Wang Assignee: Rui Wang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (BEAM-4625) Support ALL and SOME kind of comparison operator
[ https://issues.apache.org/jira/browse/BEAM-4625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang reassigned BEAM-4625: -- Assignee: (was: Xu Mingmin) > Support ALL and SOME kind of comparison operator > > > Key: BEAM-4625 > URL: https://issues.apache.org/jira/browse/BEAM-4625 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5610) Make BeamCalciteTable public
[ https://issues.apache.org/jira/browse/BEAM-5610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16636228#comment-16636228 ] Rui Wang commented on BEAM-5610: Instead, we might can make BeamCalciteTable as an inner class of BeamCalciteSchema. > Make BeamCalciteTable public > > > Key: BEAM-5610 > URL: https://issues.apache.org/jira/browse/BEAM-5610 > Project: Beam > Issue Type: Improvement > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Fix For: Not applicable > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5596) BeamSQL as a Library
[ https://issues.apache.org/jira/browse/BEAM-5596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang closed BEAM-5596. -- Resolution: Not A Problem Fix Version/s: Not applicable > BeamSQL as a Library > > > Key: BEAM-5596 > URL: https://issues.apache.org/jira/browse/BEAM-5596 > Project: Beam > Issue Type: New Feature > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Fix For: Not applicable > > > Build BeamSQL as a jar which does not include any extra dependencies. > Therefore we can use this jar to use BeamSQL as a lib. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5596) BeamSQL as a Library
[ https://issues.apache.org/jira/browse/BEAM-5596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16636226#comment-16636226 ] Rui Wang commented on BEAM-5596: `./gradlew jar` indeed builds a unshaded jar. > BeamSQL as a Library > > > Key: BEAM-5596 > URL: https://issues.apache.org/jira/browse/BEAM-5596 > Project: Beam > Issue Type: New Feature > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Fix For: Not applicable > > > Build BeamSQL as a jar which does not include any extra dependencies. > Therefore we can use this jar to use BeamSQL as a lib. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5610) Make BeamCalciteTable public
[ https://issues.apache.org/jira/browse/BEAM-5610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang closed BEAM-5610. -- Resolution: Won't Fix Fix Version/s: Not applicable > Make BeamCalciteTable public > > > Key: BEAM-5610 > URL: https://issues.apache.org/jira/browse/BEAM-5610 > Project: Beam > Issue Type: Improvement > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Fix For: Not applicable > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5610) Make BeamCalciteTable public
Rui Wang created BEAM-5610: -- Summary: Make BeamCalciteTable public Key: BEAM-5610 URL: https://issues.apache.org/jira/browse/BEAM-5610 Project: Beam Issue Type: Improvement Components: dsl-sql Reporter: Rui Wang Assignee: Rui Wang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5530) Migrate to java.time lib instead of joda-time
[ https://issues.apache.org/jira/browse/BEAM-5530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16634661#comment-16634661 ] Rui Wang commented on BEAM-5530: cc [~kenn], [~kedin], [~apilloud] We might need to pay attention on this JIRA. > Migrate to java.time lib instead of joda-time > - > > Key: BEAM-5530 > URL: https://issues.apache.org/jira/browse/BEAM-5530 > Project: Beam > Issue Type: Improvement > Components: dependencies >Reporter: Alexey Romanenko >Priority: Major > Fix For: 3.0.0 > > > Joda-time has been used till moving to Java 8. For now, these two time > libraries are used together. It will make sense finally to move everywhere to > only one lib - *java.time* - as a standard Java time library (see mail list > discussion: > [https://lists.apache.org/thread.html/b10f6f9daed44f5fa65e315a44b68b2f57c3e80225f5d549b84918af@%3Cdev.beam.apache.org%3E]). > > Since this migration will introduce breaking API changes, then we should > address it to 3.0 release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5596) BeamSQL as a Library
[ https://issues.apache.org/jira/browse/BEAM-5596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16634629#comment-16634629 ] Rui Wang commented on BEAM-5596: `./gradlew jar` can build a unshaded jar, which might be the jar that this Jira needs. > BeamSQL as a Library > > > Key: BEAM-5596 > URL: https://issues.apache.org/jira/browse/BEAM-5596 > Project: Beam > Issue Type: New Feature > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > > Build BeamSQL as a jar which does not include any extra dependencies. > Therefore we can use this jar to use BeamSQL as a lib. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5596) BeamSQL as a Library
[ https://issues.apache.org/jira/browse/BEAM-5596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5596: --- Description: Build BeamSQL as a jar which does not include any extra dependencies. Therefore we can use this jar to use BeamSQL as a lib. > BeamSQL as a Library > > > Key: BEAM-5596 > URL: https://issues.apache.org/jira/browse/BEAM-5596 > Project: Beam > Issue Type: New Feature > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > > Build BeamSQL as a jar which does not include any extra dependencies. > Therefore we can use this jar to use BeamSQL as a lib. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5596) BeamSQL as a Library
Rui Wang created BEAM-5596: -- Summary: BeamSQL as a Library Key: BEAM-5596 URL: https://issues.apache.org/jira/browse/BEAM-5596 Project: Beam Issue Type: New Feature Components: dsl-sql Reporter: Rui Wang Assignee: Rui Wang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5506) Update Beam documentation
[ https://issues.apache.org/jira/browse/BEAM-5506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang closed BEAM-5506. -- Resolution: Fixed Fix Version/s: Not applicable > Update Beam documentation > - > > Key: BEAM-5506 > URL: https://issues.apache.org/jira/browse/BEAM-5506 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Fix For: Not applicable > > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5506) Update Beam documentation
Rui Wang created BEAM-5506: -- Summary: Update Beam documentation Key: BEAM-5506 URL: https://issues.apache.org/jira/browse/BEAM-5506 Project: Beam Issue Type: Sub-task Components: dsl-sql Reporter: Rui Wang Assignee: Rui Wang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5505) Disable flattening on Row in Apache Calcite
Rui Wang created BEAM-5505: -- Summary: Disable flattening on Row in Apache Calcite Key: BEAM-5505 URL: https://issues.apache.org/jira/browse/BEAM-5505 Project: Beam Issue Type: Sub-task Components: dsl-sql Reporter: Rui Wang Assignee: Rui Wang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5505) Disable Row flattening in Apache Calcite
[ https://issues.apache.org/jira/browse/BEAM-5505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5505: --- Summary: Disable Row flattening in Apache Calcite (was: Disable flattening on Row in Apache Calcite) > Disable Row flattening in Apache Calcite > > > Key: BEAM-5505 > URL: https://issues.apache.org/jira/browse/BEAM-5505 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5504) Pubsub Write by BeamSQL
Rui Wang created BEAM-5504: -- Summary: Pubsub Write by BeamSQL Key: BEAM-5504 URL: https://issues.apache.org/jira/browse/BEAM-5504 Project: Beam Issue Type: New Feature Components: dsl-sql Reporter: Rui Wang Assignee: Rui Wang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5112) Investigate if Calcite can generate functions that we need
[ https://issues.apache.org/jira/browse/BEAM-5112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5112: --- Summary: Investigate if Calcite can generate functions that we need (was: Investigate if Calcite can generate functions we need) > Investigate if Calcite can generate functions that we need > -- > > Key: BEAM-5112 > URL: https://issues.apache.org/jira/browse/BEAM-5112 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Assignee: Andrew Pilloud >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5151) Add EXTERNAL to CREATE TABLE statement
[ https://issues.apache.org/jira/browse/BEAM-5151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang closed BEAM-5151. -- Resolution: Fixed Fix Version/s: Not applicable > Add EXTERNAL to CREATE TABLE statement > -- > > Key: BEAM-5151 > URL: https://issues.apache.org/jira/browse/BEAM-5151 > Project: Beam > Issue Type: Improvement > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Fix For: Not applicable > > Time Spent: 4h > Remaining Estimate: 0h > > BeamSQL allows [CREATE > TABLE|https://beam.apache.org/documentation/dsls/sql/create-table/] > statements to register virtual tables from external storage systems (e.g. > BigQuery). > > BeamSQL is not a storage system, so any table registered by "CREATE TABLE" > statement is essentially equivalent to be registered by "CREATE EXTERNAL > TABLE", which requires the user to provide a LOCATION and BeamSQL will > register the table outside of current execution environment based on LOCATION. > > So I propose to add EXTERNAL keyword to "CREATE TABLE" in BeamSQL to help > users understand they are registering tables, and BeamSQL does not create non > existing tables by running CREATE TABLE (at least on some storage systems, if > not all). > > We can make the EXTERNAL keyword either required or optional. > > If we make the EXTERNAL keyword required: > > Pros: > a. We can get rid of the registering table semantic on CREATE TABLE. > b, We keep the room that we could add CREATE TABLE back in the future if we > want CREATE TABLE to create, rather than not only register tables in BeamSQL. > > Cons: > 1. CREATE TABLE syntax will not be supported so existing BeamSQL pipelines > which has CREATE TABLE require changes. > 2. It's required to type tedious EXTERNAL keyword every time, especially in > SQL Shell. > > If we make the EXTERNAL keyword optional, we will have reversed pros and cons > above. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5258) Investigate if we can disable Row type flattening in Calcite
[ https://issues.apache.org/jira/browse/BEAM-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang closed BEAM-5258. -- Resolution: Fixed Fix Version/s: Not applicable > Investigate if we can disable Row type flattening in Calcite > > > Key: BEAM-5258 > URL: https://issues.apache.org/jira/browse/BEAM-5258 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Fix For: Not applicable > > > Either disable the flattening in PlannerImpl or Flattener could be a good > start. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5258) Investigate if we can disable Row type flattening in Calcite
[ https://issues.apache.org/jira/browse/BEAM-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604918#comment-16604918 ] Rui Wang commented on BEAM-5258: The feedback from Calcite community saying that disabling flatterer makes sense. It might not be a minor change though. > Investigate if we can disable Row type flattening in Calcite > > > Key: BEAM-5258 > URL: https://issues.apache.org/jira/browse/BEAM-5258 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > > Either disable the flattening in PlannerImpl or Flattener could be a good > start. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (BEAM-5259) Test nested select query with nested Row
[
https://issues.apache.org/jira/browse/BEAM-5259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang reassigned BEAM-5259:
--
Assignee: (was: Rui Wang)
> Test nested select query with nested Row
>
>
> Key: BEAM-5259
> URL: https://issues.apache.org/jira/browse/BEAM-5259
> Project: Beam
> Issue Type: Sub-task
> Components: dsl-sql
>Reporter: Rui Wang
>Priority: Major
>
> add tests to something like
>
>
> {code:java}
> SELECT i.row.string from (
> select row from table where id = 1
> ) as i;
> {code}
> , if it does not work, have a solution to fix it.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (BEAM-4710) Read complex types of data by SQL
[ https://issues.apache.org/jira/browse/BEAM-4710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang reassigned BEAM-4710: -- Assignee: (was: Rui Wang) > Read complex types of data by SQL > - > > Key: BEAM-4710 > URL: https://issues.apache.org/jira/browse/BEAM-4710 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Priority: Major > > Support reading complex types of data by SQL. Typical complex types include > nested ROW, nested ARRAY, etc. > > Complex types might be different for different data sources. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (BEAM-5189) select Inner row of nested row does not work
[ https://issues.apache.org/jira/browse/BEAM-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang reassigned BEAM-5189: -- Assignee: (was: Rui Wang) > select Inner row of nested row does not work > > > Key: BEAM-5189 > URL: https://issues.apache.org/jira/browse/BEAM-5189 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Priority: Major > > We need to support SELECT row.row...row. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5274) Handle NoSuchElementException When select from an empty table and insert into another table
[ https://issues.apache.org/jira/browse/BEAM-5274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang closed BEAM-5274. -- Resolution: Fixed Fix Version/s: Not applicable > Handle NoSuchElementException When select from an empty table and insert into > another table > --- > > Key: BEAM-5274 > URL: https://issues.apache.org/jira/browse/BEAM-5274 > Project: Beam > Issue Type: Improvement > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Fix For: Not applicable > > Time Spent: 2h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5282) Investigate if we can insert into a table with Row type
[ https://issues.apache.org/jira/browse/BEAM-5282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603614#comment-16603614 ] Rui Wang commented on BEAM-5282: We cannot insert into a table with Row type. > Investigate if we can insert into a table with Row type > > > Key: BEAM-5282 > URL: https://issues.apache.org/jira/browse/BEAM-5282 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Fix For: Not applicable > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (BEAM-5121) Investigate flattening issue of nested Row
[ https://issues.apache.org/jira/browse/BEAM-5121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584484#comment-16584484 ] Rui Wang edited comment on BEAM-5121 at 9/4/18 9:23 PM: It turns out that SELECT row.row does not work. was (Author: amaliujia): It turns out that SELECT row.row.row_member (or even row.row) does not work. > Investigate flattening issue of nested Row > -- > > Key: BEAM-5121 > URL: https://issues.apache.org/jira/browse/BEAM-5121 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Fix For: Not applicable > > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (BEAM-5258) Investigate if we can disable Row type flattening in Calcite
[ https://issues.apache.org/jira/browse/BEAM-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang reassigned BEAM-5258: -- Assignee: Rui Wang > Investigate if we can disable Row type flattening in Calcite > > > Key: BEAM-5258 > URL: https://issues.apache.org/jira/browse/BEAM-5258 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > > Either disable the flattening in PlannerImpl or Flattener could be a good > start. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5258) Investigate if we can disable Row type flattening in Calcite
[ https://issues.apache.org/jira/browse/BEAM-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16599367#comment-16599367 ] Rui Wang commented on BEAM-5258: We should kick off this one by discussing it with Calcite community. > Investigate if we can disable Row type flattening in Calcite > > > Key: BEAM-5258 > URL: https://issues.apache.org/jira/browse/BEAM-5258 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Priority: Major > > Either disable the flattening in PlannerImpl or Flattener could be a good > start. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5282) Investigate if we can insert into a table with Row type
[ https://issues.apache.org/jira/browse/BEAM-5282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang closed BEAM-5282. -- Resolution: Fixed Fix Version/s: Not applicable > Investigate if we can insert into a table with Row type > > > Key: BEAM-5282 > URL: https://issues.apache.org/jira/browse/BEAM-5282 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Fix For: Not applicable > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5282) Investigate if we can insert into a table with Row type
Rui Wang created BEAM-5282: -- Summary: Investigate if we can insert into a table with Row type Key: BEAM-5282 URL: https://issues.apache.org/jira/browse/BEAM-5282 Project: Beam Issue Type: Sub-task Components: dsl-sql Reporter: Rui Wang Assignee: Rui Wang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5260) Investigate if nested row constructor works
[
https://issues.apache.org/jira/browse/BEAM-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang closed BEAM-5260.
--
Resolution: Fixed
Fix Version/s: Not applicable
> Investigate if nested row constructor works
>
>
> Key: BEAM-5260
> URL: https://issues.apache.org/jira/browse/BEAM-5260
> Project: Beam
> Issue Type: Sub-task
> Components: dsl-sql
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
> Fix For: Not applicable
>
>
> Check if
>
> {code:java}
> SELECT ROW(row.string, ROW(row.row.int, row.row.long)){code}
> {{works.}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (BEAM-5260) Investigate if nested row constructor works
[
https://issues.apache.org/jira/browse/BEAM-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16599044#comment-16599044
]
Rui Wang commented on BEAM-5260:
The nested row constructor works but the nested queries do not work.
> Investigate if nested row constructor works
>
>
> Key: BEAM-5260
> URL: https://issues.apache.org/jira/browse/BEAM-5260
> Project: Beam
> Issue Type: Sub-task
> Components: dsl-sql
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>
> Check if
>
> {code:java}
> SELECT ROW(row.string, ROW(row.row.int, row.row.long)){code}
> {{works.}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (BEAM-5210) VARBINARY is not supported in BeamSqlPrimitive
[
https://issues.apache.org/jira/browse/BEAM-5210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang closed BEAM-5210.
--
Resolution: Fixed
Fix Version/s: Not applicable
> VARBINARY is not supported in BeamSqlPrimitive
> --
>
> Key: BEAM-5210
> URL: https://issues.apache.org/jira/browse/BEAM-5210
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql
>Reporter: Xu Mingmin
>Assignee: Xu Mingmin
>Priority: Major
> Fix For: Not applicable
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> See error as below
> {code}
> Caused by: java.lang.UnsupportedOperationException: Unsupported Beam SQL type
> in expression: VARBINARY
> at
> org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.BeamSqlPrimitive.accept(BeamSqlPrimitive.java:158)
> at
> org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.BeamSqlPrimitive.(BeamSqlPrimitive.java:44)
> at
> org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.BeamSqlPrimitive.of(BeamSqlPrimitive.java:56)
> at
> org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.BeamSqlInputRefExpression.evaluate(BeamSqlInputRefExpression.java:42)
> at
> org.apache.beam.sdk.extensions.sql.impl.interpreter.BeamSqlExpressionEnvironments$ListEnvironment.getLocalRef(BeamSqlExpressionEnvironments.java:92)
> at
> org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.BeamSqlLocalRefExpression.evaluate(BeamSqlLocalRefExpression.java:44)
> at
> org.apache.beam.sdk.extensions.sql.impl.interpreter.BeamSqlFnExecutor.lambda$execute$0(BeamSqlFnExecutor.java:557)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.beam.sdk.extensions.sql.impl.interpreter.BeamSqlFnExecutor.execute(BeamSqlFnExecutor.java:558)
> at
> org.apache.beam.sdk.extensions.sql.impl.rel.BeamCalcRel$CalcFn.processElement(BeamCalcRel.java:116)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (BEAM-5258) Investigate if we can disable Row type flattening in Calcite
[ https://issues.apache.org/jira/browse/BEAM-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang reassigned BEAM-5258: -- Assignee: (was: Rui Wang) > Investigate if we can disable Row type flattening in Calcite > > > Key: BEAM-5258 > URL: https://issues.apache.org/jira/browse/BEAM-5258 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Priority: Major > > Either disable the flattening in PlannerImpl or Flattener could be a good > start. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5274) Handle NoSuchElementException When select from an empty table and insert into another table
Rui Wang created BEAM-5274: -- Summary: Handle NoSuchElementException When select from an empty table and insert into another table Key: BEAM-5274 URL: https://issues.apache.org/jira/browse/BEAM-5274 Project: Beam Issue Type: Improvement Components: dsl-sql Reporter: Rui Wang Assignee: Rui Wang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5271) Support INSERT OVERWRITE Statement
Rui Wang created BEAM-5271: -- Summary: Support INSERT OVERWRITE Statement Key: BEAM-5271 URL: https://issues.apache.org/jira/browse/BEAM-5271 Project: Beam Issue Type: Improvement Components: dsl-sql Reporter: Rui Wang We can support INSERT OVERWRITE to insert to a table but overwrite existing table contents. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (BEAM-5260) Investigate if nested row constructor works
[
https://issues.apache.org/jira/browse/BEAM-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596898#comment-16596898
]
Rui Wang edited comment on BEAM-5260 at 8/29/18 10:12 PM:
--
Furthermore, we expect this works:
{code:java}
SELECT ROW(row_col.$0, ROW(row_col.$1.$0, row_col.$1.$1) row_col.$2) FROM
(SELECT ROW(1, ROW(2, 3), 4) as row_col){code}
was (Author: amaliujia):
Furthermore, we expect
{code:java}
SELECT ROW(row_col.$0, ROW(row_col.$1.$0, row_col.$1.$1) row_col.$2) FROM
(SELECT ROW(1, ROW(2, 3), 4) as row_col){code}
> Investigate if nested row constructor works
>
>
> Key: BEAM-5260
> URL: https://issues.apache.org/jira/browse/BEAM-5260
> Project: Beam
> Issue Type: Sub-task
> Components: dsl-sql
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>
> Check if
>
> {code:java}
> SELECT ROW(row.string, ROW(row.row.int, row.row.long)){code}
> {{works.}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (BEAM-5260) Investigate if nested row constructor works
[
https://issues.apache.org/jira/browse/BEAM-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596898#comment-16596898
]
Rui Wang commented on BEAM-5260:
Furthermore, we expect
{code:java}
SELECT ROW(row_col.$0, ROW(row_col.$1.$0, row_col.$1.$1) row_col.$2) FROM
(SELECT ROW(1, ROW(2, 3), 4) as row_col){code}
> Investigate if nested row constructor works
>
>
> Key: BEAM-5260
> URL: https://issues.apache.org/jira/browse/BEAM-5260
> Project: Beam
> Issue Type: Sub-task
> Components: dsl-sql
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>
> Check if
>
> {code:java}
> SELECT ROW(row.string, ROW(row.row.int, row.row.long)){code}
> {{works.}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (BEAM-5258) Investigate if we can disable Row type flattening in Calcite
[ https://issues.apache.org/jira/browse/BEAM-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5258: --- Summary: Investigate if we can disable Row type flattening in Calcite (was: Investigate if we can disable Row type flattening) > Investigate if we can disable Row type flattening in Calcite > > > Key: BEAM-5258 > URL: https://issues.apache.org/jira/browse/BEAM-5258 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > > Either disable the flattening in PlannerImpl or Flattener could be a good > start. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5260) Investigate if nested row constructor works
[
https://issues.apache.org/jira/browse/BEAM-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang updated BEAM-5260:
---
Description:
Check if
{code:java}
SELECT ROW(row.string, ROW(row.row.int, row.row.long)){code}
{{works.}}
was:
Check if
`SELECT ROW(row.string, ROW(row.row.int, row.row.long))` works.
> Investigate if nested row constructor works
>
>
> Key: BEAM-5260
> URL: https://issues.apache.org/jira/browse/BEAM-5260
> Project: Beam
> Issue Type: Sub-task
> Components: dsl-sql
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>
> Check if
>
> {code:java}
> SELECT ROW(row.string, ROW(row.row.int, row.row.long)){code}
> {{works.}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (BEAM-5259) Test nested select query with nested Row
[
https://issues.apache.org/jira/browse/BEAM-5259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang updated BEAM-5259:
---
Description:
add tests to something like
{code:java}
SELECT i.row.string from (
select row from table where id = 1
) as i;
{code}
, if it does not work, have a solution to fix it.
was:
Test something like
SELECT i.row.string from (
select row from table where id = 1
) as i;
> Test nested select query with nested Row
>
>
> Key: BEAM-5259
> URL: https://issues.apache.org/jira/browse/BEAM-5259
> Project: Beam
> Issue Type: Sub-task
> Components: dsl-sql
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>
> add tests to something like
>
>
> {code:java}
> SELECT i.row.string from (
> select row from table where id = 1
> ) as i;
> {code}
> , if it does not work, have a solution to fix it.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (BEAM-5260) Investigate if nested row constructor works
[ https://issues.apache.org/jira/browse/BEAM-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5260: --- Description: Check if `SELECT ROW(row.string, ROW(row.row.int, row.row.long))` works. was: Check if ```SELECT ROW(row.string, ROW(row.row.int, row.row.long))``` works. > Investigate if nested row constructor works > > > Key: BEAM-5260 > URL: https://issues.apache.org/jira/browse/BEAM-5260 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > > Check if > > `SELECT ROW(row.string, ROW(row.row.int, row.row.long))` works. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5260) Investigate if nested row constructor works
[ https://issues.apache.org/jira/browse/BEAM-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5260: --- Description: Check if ```SELECT ROW(row.string, ROW(row.row.int, row.row.long))``` works. was: Check if SELECT ROW(row.string, ROW(row.row.int, row.row.long)) works. > Investigate if nested row constructor works > > > Key: BEAM-5260 > URL: https://issues.apache.org/jira/browse/BEAM-5260 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > > Check if > > ```SELECT ROW(row.string, ROW(row.row.int, row.row.long))``` works. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5260) Investigate if nested row constructor works
Rui Wang created BEAM-5260: -- Summary: Investigate if nested row constructor works Key: BEAM-5260 URL: https://issues.apache.org/jira/browse/BEAM-5260 Project: Beam Issue Type: Sub-task Components: dsl-sql Reporter: Rui Wang Assignee: Rui Wang Check if SELECT ROW(row.string, ROW(row.row.int, row.row.long)) works. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5259) Test nested select query with nested Row
Rui Wang created BEAM-5259: -- Summary: Test nested select query with nested Row Key: BEAM-5259 URL: https://issues.apache.org/jira/browse/BEAM-5259 Project: Beam Issue Type: Sub-task Components: dsl-sql Reporter: Rui Wang Assignee: Rui Wang Test something like SELECT i.row.string from ( select row from table where id = 1 ) as i; -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5258) Investigate if we can disable Row type flattening
Rui Wang created BEAM-5258: -- Summary: Investigate if we can disable Row type flattening Key: BEAM-5258 URL: https://issues.apache.org/jira/browse/BEAM-5258 Project: Beam Issue Type: Sub-task Components: dsl-sql Reporter: Rui Wang Assignee: Rui Wang Either disable the flattening in PlannerImpl or Flattener could be a good start. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5252) Fix the failing complex type test due to misusing reversed keyword of Calcite
[ https://issues.apache.org/jira/browse/BEAM-5252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang closed BEAM-5252. -- Resolution: Fixed Fix Version/s: Not applicable > Fix the failing complex type test due to misusing reversed keyword of Calcite > - > > Key: BEAM-5252 > URL: https://issues.apache.org/jira/browse/BEAM-5252 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Critical > Fix For: Not applicable > > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5252) Fix the failing complex type test due to misusing reversed keyword of Calcite
Rui Wang created BEAM-5252: -- Summary: Fix the failing complex type test due to misusing reversed keyword of Calcite Key: BEAM-5252 URL: https://issues.apache.org/jira/browse/BEAM-5252 Project: Beam Issue Type: Sub-task Components: dsl-sql Reporter: Rui Wang Assignee: Rui Wang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5121) Investigate flattening issue of nested Row
[ https://issues.apache.org/jira/browse/BEAM-5121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang closed BEAM-5121. -- Resolution: Fixed Fix Version/s: Not applicable > Investigate flattening issue of nested Row > -- > > Key: BEAM-5121 > URL: https://issues.apache.org/jira/browse/BEAM-5121 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Fix For: Not applicable > > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5251) Throw meaningful exception when have Calcite reversed words in query
[ https://issues.apache.org/jira/browse/BEAM-5251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5251: --- Issue Type: Improvement (was: Bug) > Throw meaningful exception when have Calcite reversed words in query > > > Key: BEAM-5251 > URL: https://issues.apache.org/jira/browse/BEAM-5251 > Project: Beam > Issue Type: Improvement > Components: dsl-sql >Reporter: Rui Wang >Priority: Major > > Calcite throws some hard-to-understand error message when wrongly use Calcite > reversed keywords. It is better to throw something useful when having this > case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5251) Throw meaningful exception when have Calcite reversed words in query
Rui Wang created BEAM-5251: -- Summary: Throw meaningful exception when have Calcite reversed words in query Key: BEAM-5251 URL: https://issues.apache.org/jira/browse/BEAM-5251 Project: Beam Issue Type: Bug Components: dsl-sql Reporter: Rui Wang Calcite throws some hard-to-understand error message when wrongly use Calcite reversed keywords. It is better to throw something useful when having this case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5189) select Inner row of nested row does not work
[ https://issues.apache.org/jira/browse/BEAM-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5189: --- Description: We need to support SELECT row.row...row. (was: Right now, the field access to nested row does not work for any access to sub row's field. We need to support SELECT row.row...row.field.) > select Inner row of nested row does not work > > > Key: BEAM-5189 > URL: https://issues.apache.org/jira/browse/BEAM-5189 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > > We need to support SELECT row.row...row. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5189) select Inner row of nested row does not work
[ https://issues.apache.org/jira/browse/BEAM-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5189: --- Summary: select Inner row of nested row does not work (was: Field access of nested Row) > select Inner row of nested row does not work > > > Key: BEAM-5189 > URL: https://issues.apache.org/jira/browse/BEAM-5189 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > > Right now, the field access to nested row does not work for any access to sub > row's field. > > We need to support SELECT row.row...row.field. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5189) Field access of nested Row
[ https://issues.apache.org/jira/browse/BEAM-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5189: --- Description: Right now, the field access to nested row does not work for any access to sub row's field. We need to support SELECT row.row...row.field. was: Right now, the field access to nested row does not work for any access to sub row's field. We need to support it. > Field access of nested Row > -- > > Key: BEAM-5189 > URL: https://issues.apache.org/jira/browse/BEAM-5189 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > > Right now, the field access to nested row does not work for any access to sub > row's field. > > We need to support SELECT row.row...row.field. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5189) Field access of nested Row
Rui Wang created BEAM-5189: -- Summary: Field access of nested Row Key: BEAM-5189 URL: https://issues.apache.org/jira/browse/BEAM-5189 Project: Beam Issue Type: Sub-task Components: dsl-sql Reporter: Rui Wang Assignee: Rui Wang Right now, the field access to nested row does not work for any access to sub row's field. We need to support it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5157) Improve user friendliness of using SET
[ https://issues.apache.org/jira/browse/BEAM-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586594#comment-16586594 ] Rui Wang commented on BEAM-5157: One possible way is to retrieve all possible options and return it to users. > Improve user friendliness of using SET > -- > > Key: BEAM-5157 > URL: https://issues.apache.org/jira/browse/BEAM-5157 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Priority: Major > > Right now, SET only supports a few defined property in Beam. However, when > users have some exceptions of missing properties, users would need to set > some properties. At this moment, users might not have clues on the property > names. So we should have a way to list a bunch of properties that user need > to set. > > One example is that user should set tempLocation when using BigQuery. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5121) Investigate flattening issue of nested Row
[ https://issues.apache.org/jira/browse/BEAM-5121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584484#comment-16584484 ] Rui Wang commented on BEAM-5121: It turns out that SELECT row.row.row_member (or even row.row) does not work. > Investigate flattening issue of nested Row > -- > > Key: BEAM-5121 > URL: https://issues.apache.org/jira/browse/BEAM-5121 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5141) Improve error message when SET unregistered options
[ https://issues.apache.org/jira/browse/BEAM-5141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang closed BEAM-5141. -- Resolution: Fixed Fix Version/s: Not applicable > Improve error message when SET unregistered options > > > Key: BEAM-5141 > URL: https://issues.apache.org/jira/browse/BEAM-5141 > Project: Beam > Issue Type: Improvement > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Fix For: Not applicable > > Time Spent: 5h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5140) Add EXTERNAL to CREATE TABLE statement
[ https://issues.apache.org/jira/browse/BEAM-5140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang closed BEAM-5140. -- Resolution: Duplicate Fix Version/s: Not applicable > Add EXTERNAL to CREATE TABLE statement > -- > > Key: BEAM-5140 > URL: https://issues.apache.org/jira/browse/BEAM-5140 > Project: Beam > Issue Type: Improvement > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Fix For: Not applicable > > > add EXTERNAL keyword to CREATE TABLE and make EXTERNAL optional. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5101) Improve Floor and Ceil tests at DSL level
[ https://issues.apache.org/jira/browse/BEAM-5101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang closed BEAM-5101. -- Resolution: Fixed Fix Version/s: Not applicable > Improve Floor and Ceil tests at DSL level > - > > Key: BEAM-5101 > URL: https://issues.apache.org/jira/browse/BEAM-5101 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Fix For: Not applicable > > Time Spent: 2h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5159) Too many retries when failed to write to BigQuery table
[ https://issues.apache.org/jira/browse/BEAM-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5159: --- Summary: Too many retries when failed to write to BigQuery table (was: Too many retries when cannot write to BigQuery table ) > Too many retries when failed to write to BigQuery table > - > > Key: BEAM-5159 > URL: https://issues.apache.org/jira/browse/BEAM-5159 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Priority: Major > > BeamSQL retries multiple times when inserting illegal values to BigQuery. It > might worth optimizing a bit to terminate the query sooner in Shell when > having exceptions that cannot be solved by retries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5157) Improve user friendliness of using SET
[ https://issues.apache.org/jira/browse/BEAM-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5157: --- Description: Right now, SET only supports a few defined property in Beam. However, when users have some exceptions of missing properties, users would need to set some properties. At this moment, users might not have clues on the property names. So we should have a way to list a bunch of properties that user need to set. One example is that user should set tempLocation when using BigQuery. was: Right now, SET only supports a few defined property in Beam. However, when users have some exceptions of missing properties, users would need to set some properties. At this moment, users might not have clue on the property names. So we should have a way to list a bunch of properties that user need to set. One example is user should set tempLocation when using BigQuery. > Improve user friendliness of using SET > -- > > Key: BEAM-5157 > URL: https://issues.apache.org/jira/browse/BEAM-5157 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Priority: Major > > Right now, SET only supports a few defined property in Beam. However, when > users have some exceptions of missing properties, users would need to set > some properties. At this moment, users might not have clues on the property > names. So we should have a way to list a bunch of properties that user need > to set. > > One example is that user should set tempLocation when using BigQuery. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5159) Too many retries when cannot write to BigQuery table
[ https://issues.apache.org/jira/browse/BEAM-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5159: --- Description: BeamSQL retries multiple times when inserting illegal values to BigQuery. It might worth optimizing a bit to terminate the query sooner in Shell when having exceptions that cannot be solved by retries. (was: BeamSQL retries multiple times when inserting illegal values to BigQuery. It might worth optimizing a bit to terminiate the query sooner in Shell.) > Too many retries when cannot write to BigQuery table > -- > > Key: BEAM-5159 > URL: https://issues.apache.org/jira/browse/BEAM-5159 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Priority: Major > > BeamSQL retries multiple times when inserting illegal values to BigQuery. It > might worth optimizing a bit to terminate the query sooner in Shell when > having exceptions that cannot be solved by retries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5159) Too many retries when cannot write to BigQuery table
[ https://issues.apache.org/jira/browse/BEAM-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5159: --- Description: BeamSQL retries multiple times when inserting illegal values to BigQuery. It might worth optimizing a bit to terminiate the query sooner in Shell. (was: BeamSQL retries multiple times when inserting illegal values to BigQuery. It might worth optimizing a bit to finish the query sooner in Shell.) > Too many retries when cannot write to BigQuery table > -- > > Key: BEAM-5159 > URL: https://issues.apache.org/jira/browse/BEAM-5159 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Priority: Major > > BeamSQL retries multiple times when inserting illegal values to BigQuery. It > might worth optimizing a bit to terminiate the query sooner in Shell. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5158) BeamSQL and SQL Shell Improvement
[ https://issues.apache.org/jira/browse/BEAM-5158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5158: --- Summary: BeamSQL and SQL Shell Improvement (was: SQL Shell Improvement) > BeamSQL and SQL Shell Improvement > - > > Key: BEAM-5158 > URL: https://issues.apache.org/jira/browse/BEAM-5158 > Project: Beam > Issue Type: Improvement > Components: dsl-sql >Reporter: Rui Wang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5159) Too many/infinity retries when cannot write to BigQuery table
Rui Wang created BEAM-5159: -- Summary: Too many/infinity retries when cannot write to BigQuery table Key: BEAM-5159 URL: https://issues.apache.org/jira/browse/BEAM-5159 Project: Beam Issue Type: Sub-task Components: dsl-sql Reporter: Rui Wang BeamSQL retries multiple times when inserting illegal values to BigQuery. It might worth optimizing a bit to finish the query sooner in Shell. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5159) Too many retries when cannot write to BigQuery table
[ https://issues.apache.org/jira/browse/BEAM-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5159: --- Summary: Too many retries when cannot write to BigQuery table (was: Too many/infinity retries when cannot write to BigQuery table ) > Too many retries when cannot write to BigQuery table > -- > > Key: BEAM-5159 > URL: https://issues.apache.org/jira/browse/BEAM-5159 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Priority: Major > > BeamSQL retries multiple times when inserting illegal values to BigQuery. It > might worth optimizing a bit to finish the query sooner in Shell. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5157) Improve user friendliness of using SET
[ https://issues.apache.org/jira/browse/BEAM-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5157: --- Issue Type: Sub-task (was: Improvement) Parent: BEAM-5158 > Improve user friendliness of using SET > -- > > Key: BEAM-5157 > URL: https://issues.apache.org/jira/browse/BEAM-5157 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Priority: Major > > Right now, SET only supports a few defined property in Beam. However, when > users have some exceptions of missing properties, users would need to set > some properties. At this moment, users might not have clue on the property > names. So we should have a way to list a bunch of properties that user need > to set. > > One example is user should set tempLocation when using BigQuery. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5158) SQL Shell Improvement
Rui Wang created BEAM-5158: -- Summary: SQL Shell Improvement Key: BEAM-5158 URL: https://issues.apache.org/jira/browse/BEAM-5158 Project: Beam Issue Type: Improvement Components: dsl-sql Reporter: Rui Wang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5157) Improve user friendliness of using SET
Rui Wang created BEAM-5157: -- Summary: Improve user friendliness of using SET Key: BEAM-5157 URL: https://issues.apache.org/jira/browse/BEAM-5157 Project: Beam Issue Type: Improvement Components: dsl-sql Reporter: Rui Wang Right now, SET only supports a few defined property in Beam. However, when users have some exceptions of missing properties, users would need to set some properties. At this moment, users might not have clue on the property names. So we should have a way to list a bunch of properties that user need to set. One example is user should set tempLocation when using BigQuery. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (BEAM-4717) Test JSON to Beam types conversion
[ https://issues.apache.org/jira/browse/BEAM-4717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang reassigned BEAM-4717: -- Assignee: (was: Rui Wang) > Test JSON to Beam types conversion > -- > > Key: BEAM-4717 > URL: https://issues.apache.org/jira/browse/BEAM-4717 > Project: Beam > Issue Type: Improvement > Components: dsl-sql >Reporter: Rui Wang >Priority: Major > > Should improve PubSub test coverage by testing more data types. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (BEAM-4754) Type conversion testing
[ https://issues.apache.org/jira/browse/BEAM-4754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang reassigned BEAM-4754: -- Assignee: (was: Rui Wang) > Type conversion testing > > > Key: BEAM-4754 > URL: https://issues.apache.org/jira/browse/BEAM-4754 > Project: Beam > Issue Type: Improvement > Components: dsl-sql >Reporter: Rui Wang >Priority: Major > > We should have a type system testing to test all types of Beam SQL from > different types (Avro, JSON, etc.) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5142) Improve Pubsub e2e test
[ https://issues.apache.org/jira/browse/BEAM-5142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang closed BEAM-5142. -- Resolution: Fixed Fix Version/s: Not applicable > Improve Pubsub e2e test > --- > > Key: BEAM-5142 > URL: https://issues.apache.org/jira/browse/BEAM-5142 > Project: Beam > Issue Type: Improvement > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Fix For: Not applicable > > Time Spent: 2h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5151) Add EXTERNAL to CREATE TABLE statement
Rui Wang created BEAM-5151: -- Summary: Add EXTERNAL to CREATE TABLE statement Key: BEAM-5151 URL: https://issues.apache.org/jira/browse/BEAM-5151 Project: Beam Issue Type: Improvement Components: dsl-sql Reporter: Rui Wang Assignee: Rui Wang BeamSQL allows [CREATE TABLE|https://beam.apache.org/documentation/dsls/sql/create-table/] statements to register virtual tables from external storage systems (e.g. BigQuery). BeamSQL is not a storage system, so any table registered by "CREATE TABLE" statement is essentially equivalent to be registered by "CREATE EXTERNAL TABLE", which requires the user to provide a LOCATION and BeamSQL will register the table outside of current execution environment based on LOCATION. So I propose to add EXTERNAL keyword to "CREATE TABLE" in BeamSQL to help users understand they are registering tables, and BeamSQL does not create non existing tables by running CREATE TABLE (at least on some storage systems, if not all). We can make the EXTERNAL keyword either required or optional. If we make the EXTERNAL keyword required: Pros: a. We can get rid of the registering table semantic on CREATE TABLE. b, We keep the room that we could add CREATE TABLE back in the future if we want CREATE TABLE to create, rather than not only register tables in BeamSQL. Cons: 1. CREATE TABLE syntax will not be supported so existing BeamSQL pipelines which has CREATE TABLE require changes. 2. It's required to type tedious EXTERNAL keyword every time, especially in SQL Shell. If we make the EXTERNAL keyword optional, we will have reversed pros and cons above. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5141) Improve error message when SET unregistered options
[ https://issues.apache.org/jira/browse/BEAM-5141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5141: --- Summary: Improve error message when SET unregistered options (was: Improve error message when SET a undefined property ) > Improve error message when SET unregistered options > > > Key: BEAM-5141 > URL: https://issues.apache.org/jira/browse/BEAM-5141 > Project: Beam > Issue Type: Improvement > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5142) Improve Pubsub e2e test
Rui Wang created BEAM-5142: -- Summary: Improve Pubsub e2e test Key: BEAM-5142 URL: https://issues.apache.org/jira/browse/BEAM-5142 Project: Beam Issue Type: Improvement Components: dsl-sql Reporter: Rui Wang Assignee: Rui Wang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5140) Add EXTERNAL to CREATE TABLE statement
Rui Wang created BEAM-5140: -- Summary: Add EXTERNAL to CREATE TABLE statement Key: BEAM-5140 URL: https://issues.apache.org/jira/browse/BEAM-5140 Project: Beam Issue Type: Improvement Components: dsl-sql Reporter: Rui Wang Assignee: Rui Wang add EXTERNAL keyword to CREATE TABLE and make EXTERNAL optional. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5141) Improve error message when SET a undefined property
Rui Wang created BEAM-5141: -- Summary: Improve error message when SET a undefined property Key: BEAM-5141 URL: https://issues.apache.org/jira/browse/BEAM-5141 Project: Beam Issue Type: Improvement Components: dsl-sql Reporter: Rui Wang Assignee: Rui Wang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5131) Support ANY (sub query)
Rui Wang created BEAM-5131: -- Summary: Support ANY (sub query) Key: BEAM-5131 URL: https://issues.apache.org/jira/browse/BEAM-5131 Project: Beam Issue Type: Sub-task Components: dsl-sql Reporter: Rui Wang SELECT order_id FROM order WHERE item_id = ANY (SELECT item_id FROM special_item_table). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5130) Support ALL (sub query)
[ https://issues.apache.org/jira/browse/BEAM-5130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated BEAM-5130: --- Description: Example: SELECT order_id FROM order WHERE item_id = ALL (SELECT item_id FROM special_item_table). was: Example: SELECT order_id FROM order WHERE item_id = ANY (SELECT item_id FROM special_item_table). > Support ALL (sub query) > --- > > Key: BEAM-5130 > URL: https://issues.apache.org/jira/browse/BEAM-5130 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Priority: Major > > Example: > > SELECT order_id FROM order WHERE item_id = ALL (SELECT item_id FROM > special_item_table). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5130) Support ALL (sub query)
Rui Wang created BEAM-5130: -- Summary: Support ALL (sub query) Key: BEAM-5130 URL: https://issues.apache.org/jira/browse/BEAM-5130 Project: Beam Issue Type: Sub-task Components: dsl-sql Reporter: Rui Wang Example: SELECT order_id FROM order WHERE item_id = ANY (SELECT item_id FROM special_item_table). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5129) Support Rank() OVER
Rui Wang created BEAM-5129: -- Summary: Support Rank() OVER Key: BEAM-5129 URL: https://issues.apache.org/jira/browse/BEAM-5129 Project: Beam Issue Type: Sub-task Components: dsl-sql Reporter: Rui Wang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5103) Test aggregation functions at DSL levels
[ https://issues.apache.org/jira/browse/BEAM-5103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang closed BEAM-5103. -- Resolution: Fixed Fix Version/s: Not applicable > Test aggregation functions at DSL levels > > > Key: BEAM-5103 > URL: https://issues.apache.org/jira/browse/BEAM-5103 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Fix For: Not applicable > > Time Spent: 5h 40m > Remaining Estimate: 0h > > Typical aggregation functions include COUNT, SUM, MAX, MIN, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5106) Test conditional functions at DSL level
[ https://issues.apache.org/jira/browse/BEAM-5106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang closed BEAM-5106. -- Resolution: Fixed Fix Version/s: Not applicable > Test conditional functions at DSL level > --- > > Key: BEAM-5106 > URL: https://issues.apache.org/jira/browse/BEAM-5106 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Fix For: Not applicable > > Time Spent: 4h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (BEAM-5128) Fix Floor and Ceil functions
[ https://issues.apache.org/jira/browse/BEAM-5128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang reassigned BEAM-5128: -- Assignee: (was: Xu Mingmin) > Fix Floor and Ceil functions > > > Key: BEAM-5128 > URL: https://issues.apache.org/jira/browse/BEAM-5128 > Project: Beam > Issue Type: Sub-task > Components: dsl-sql >Reporter: Rui Wang >Priority: Major > > Fool and Ceil converts non decimal type to double type no matter what the > field type is. So the Row will fail the code once it detects the mismatch > between field type and value. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5128) Fix Floor and Ceil functions
Rui Wang created BEAM-5128: -- Summary: Fix Floor and Ceil functions Key: BEAM-5128 URL: https://issues.apache.org/jira/browse/BEAM-5128 Project: Beam Issue Type: Sub-task Components: dsl-sql Reporter: Rui Wang Assignee: Xu Mingmin Fool and Ceil converts non decimal type to double type no matter what the field type is. So the Row will fail the code once it detects the mismatch between field type and value. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-4807) Upgrade calcite to 1.17.0
[ https://issues.apache.org/jira/browse/BEAM-4807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang closed BEAM-4807. -- Resolution: Fixed Fix Version/s: Not applicable > Upgrade calcite to 1.17.0 > - > > Key: BEAM-4807 > URL: https://issues.apache.org/jira/browse/BEAM-4807 > Project: Beam > Issue Type: New Feature > Components: dsl-sql >Reporter: Andrew Pilloud >Assignee: Andrew Pilloud >Priority: Major > Fix For: Not applicable > > Time Spent: 40m > Remaining Estimate: 0h > > We should upgrade calcite. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
