[jira] [Commented] (CALCITE-1345) Streaming GROUP BY
[
https://issues.apache.org/jira/browse/CALCITE-1345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837212#comment-15837212
]
Atri Sharma commented on CALCITE-1345:
--
[~cloneman] I have been trying out a PoC for this, but got pushed due to
personal things. I would love to collaborate with you on this.
> Streaming GROUP BY
> --
>
> Key: CALCITE-1345
> URL: https://issues.apache.org/jira/browse/CALCITE-1345
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Atri Sharma
>
> Implement streaming {{GROUP BY}}, including parsing, validation, and
> execution using the Interpreter (probably a variation of AggregateNode),
> intermittently returning results when a monotonic column indicates that the
> next set of results are ready. Add test(s) to StreamingTest. Example query
> should include {{FLOOR(rowtime TO HOUR)}}; the {{TUMBLE}}, {{HOP}} and
> {{SESSION}} functions are out of scope of this JIRA case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CALCITE-1603) Support parsing tumble window in the group by clause
[ https://issues.apache.org/jira/browse/CALCITE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837191#comment-15837191 ] Julian Hyde commented on CALCITE-1603: -- You should definitely build on the work I did in https://github.com/julianhyde/calcite/commits/768-hop. > Support parsing tumble window in the group by clause > > > Key: CALCITE-1603 > URL: https://issues.apache.org/jira/browse/CALCITE-1603 > Project: Calcite > Issue Type: Improvement >Reporter: Haohui Mai >Assignee: Julian Hyde > > Recent changes in Flink (e.g., FLINK-4691) has added the support for tumbling > windows in streams. > This jira proposes to add supports for parsing the tumbling window in the > group by clause as suggested in http://calcite.apache.org/docs/stream.html, > so that it is possible to expose tumble window support at the SQL layer in > Flink. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CALCITE-1603) Support parsing tumble window in the group by clause
Haohui Mai created CALCITE-1603: --- Summary: Support parsing tumble window in the group by clause Key: CALCITE-1603 URL: https://issues.apache.org/jira/browse/CALCITE-1603 Project: Calcite Issue Type: Improvement Reporter: Haohui Mai Assignee: Julian Hyde Recent changes in Flink (e.g., FLINK-4691) has added the support for tumbling windows in streams. This jira proposes to add supports for parsing the tumbling window in the group by clause as suggested in http://calcite.apache.org/docs/stream.html, so that it is possible to expose tumble window support at the SQL layer in Flink. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CALCITE-1345) Streaming GROUP BY
[
https://issues.apache.org/jira/browse/CALCITE-1345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837056#comment-15837056
]
Haohui Mai commented on CALCITE-1345:
-
[~atris], just wondering are you actively working on this jira? If not I would
like to work on it. Many thanks.
> Streaming GROUP BY
> --
>
> Key: CALCITE-1345
> URL: https://issues.apache.org/jira/browse/CALCITE-1345
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Atri Sharma
>
> Implement streaming {{GROUP BY}}, including parsing, validation, and
> execution using the Interpreter (probably a variation of AggregateNode),
> intermittently returning results when a monotonic column indicates that the
> next set of results are ready. Add test(s) to StreamingTest. Example query
> should include {{FLOOR(rowtime TO HOUR)}}; the {{TUMBLE}}, {{HOP}} and
> {{SESSION}} functions are out of scope of this JIRA case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CALCITE-1602) Remove uses of deprecated APIs
[
https://issues.apache.org/jira/browse/CALCITE-1602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde updated CALCITE-1602:
-
Summary: Remove uses of deprecated APIs (was: Remove uses of deprecated
external APIs)
> Remove uses of deprecated APIs
> --
>
> Key: CALCITE-1602
> URL: https://issues.apache.org/jira/browse/CALCITE-1602
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>
> Calcite should not use deprecated methods and classes, especially in
> third-party libraries. I propose that we adopt a zero-tolerance policy,
> fixing deprecated calls as soon as they occur. As long as the third-party
> libraries use semantic versioning, this should give us prior warning before
> things disappear.
> You can find the calls to deprecated APIs by adding
> {{-Xlint:deprecation}} then running
> {{mvn clean package -DskipTests | grep 'has been deprecated' | grep -v 'in
> org.apache.calcite'}}. Currently there are 51 when you run in JDK9 and Guava
> 20.0. Here are the first few:
> {noformat}
> [WARNING]
> /home/jhyde/open1/calcite.2/linq4j/src/main/java/org/apache/calcite/linq4j/tree/Types.java:[254,38]
> getPackage(java.lang.String) in java.lang.Package has been deprecated
> [WARNING]
> /home/jhyde/open1/calcite.2/linq4j/src/test/java/org/apache/calcite/linq4j/test/Linq4jTest.java:[736,25]
> Integer(int) in java.lang.Integer has been deprecated
> [WARNING]
> /home/jhyde/open1/calcite.2/linq4j/src/test/java/org/apache/calcite/linq4j/test/Linq4jTest.java:[737,26]
> Integer(int) in java.lang.Integer has been deprecated
> [WARNING]
> /home/jhyde/open1/calcite.2/linq4j/src/test/java/org/apache/calcite/linq4j/test/Linq4jTest.java:[738,25]
> Integer(int) in java.lang.Integer has been deprecated
> [WARNING]
> /home/jhyde/open1/calcite.2/core/src/main/java/org/apache/calcite/rex/RexBuilder.java:[539,47]
> ROUND_HALF_DOWN in java.math.BigDecimal has been deprecated
> {noformat}
> We should fix these (and other deprecated APIs when we run on other JDK
> versions and other Guava versions), then add a nightly check that none are
> added back.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CALCITE-1602) Remove uses of deprecated external APIs
[
https://issues.apache.org/jira/browse/CALCITE-1602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837031#comment-15837031
]
Julian Hyde commented on CALCITE-1602:
--
I'm working on this. The fix will fix both Avatica and Calcite in one swoop.
Enforcement will be asynchronous: If people introduce warnings the build won't
fail, but my
[calcite-regress.sh|https://github.com/julianhyde/share/blob/master/tools/calcite-regress.sh]
script will, and you'll soon get an email from me. By the way, I run that
script every night on various combinations of JDK1.7, JDK1.8, JDK9, Guava
14.0.1, Guava 20.0 and Guava 21.0.
> Remove uses of deprecated external APIs
> ---
>
> Key: CALCITE-1602
> URL: https://issues.apache.org/jira/browse/CALCITE-1602
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>
> Calcite should not use deprecated methods and classes, especially in
> third-party libraries. I propose that we adopt a zero-tolerance policy,
> fixing deprecated calls as soon as they occur. As long as the third-party
> libraries use semantic versioning, this should give us prior warning before
> things disappear.
> You can find the calls to deprecated APIs by adding
> {{-Xlint:deprecation}} then running
> {{mvn clean package -DskipTests | grep 'has been deprecated' | grep -v 'in
> org.apache.calcite'}}. Currently there are 51 when you run in JDK9 and Guava
> 20.0. Here are the first few:
> {noformat}
> [WARNING]
> /home/jhyde/open1/calcite.2/linq4j/src/main/java/org/apache/calcite/linq4j/tree/Types.java:[254,38]
> getPackage(java.lang.String) in java.lang.Package has been deprecated
> [WARNING]
> /home/jhyde/open1/calcite.2/linq4j/src/test/java/org/apache/calcite/linq4j/test/Linq4jTest.java:[736,25]
> Integer(int) in java.lang.Integer has been deprecated
> [WARNING]
> /home/jhyde/open1/calcite.2/linq4j/src/test/java/org/apache/calcite/linq4j/test/Linq4jTest.java:[737,26]
> Integer(int) in java.lang.Integer has been deprecated
> [WARNING]
> /home/jhyde/open1/calcite.2/linq4j/src/test/java/org/apache/calcite/linq4j/test/Linq4jTest.java:[738,25]
> Integer(int) in java.lang.Integer has been deprecated
> [WARNING]
> /home/jhyde/open1/calcite.2/core/src/main/java/org/apache/calcite/rex/RexBuilder.java:[539,47]
> ROUND_HALF_DOWN in java.math.BigDecimal has been deprecated
> {noformat}
> We should fix these (and other deprecated APIs when we run on other JDK
> versions and other Guava versions), then add a nightly check that none are
> added back.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CALCITE-1602) Remove uses of deprecated external APIs
[
https://issues.apache.org/jira/browse/CALCITE-1602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836918#comment-15836918
]
Josh Elser commented on CALCITE-1602:
-
bq. I propose that we adopt a zero-tolerance policy, fixing deprecated calls as
soon as they occur.
+1
bq. We should fix these (and other deprecated APIs when we run on other JDK
versions and other Guava versions), then add a nightly check that none are
added back.
+1
> Remove uses of deprecated external APIs
> ---
>
> Key: CALCITE-1602
> URL: https://issues.apache.org/jira/browse/CALCITE-1602
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>
> Calcite should not use deprecated methods and classes, especially in
> third-party libraries. I propose that we adopt a zero-tolerance policy,
> fixing deprecated calls as soon as they occur. As long as the third-party
> libraries use semantic versioning, this should give us prior warning before
> things disappear.
> You can find the calls to deprecated APIs by adding
> {{-Xlint:deprecation}} then running
> {{mvn clean package -DskipTests | grep 'has been deprecated' | grep -v 'in
> org.apache.calcite'}}. Currently there are 51 when you run in JDK9 and Guava
> 20.0. Here are the first few:
> {noformat}
> [WARNING]
> /home/jhyde/open1/calcite.2/linq4j/src/main/java/org/apache/calcite/linq4j/tree/Types.java:[254,38]
> getPackage(java.lang.String) in java.lang.Package has been deprecated
> [WARNING]
> /home/jhyde/open1/calcite.2/linq4j/src/test/java/org/apache/calcite/linq4j/test/Linq4jTest.java:[736,25]
> Integer(int) in java.lang.Integer has been deprecated
> [WARNING]
> /home/jhyde/open1/calcite.2/linq4j/src/test/java/org/apache/calcite/linq4j/test/Linq4jTest.java:[737,26]
> Integer(int) in java.lang.Integer has been deprecated
> [WARNING]
> /home/jhyde/open1/calcite.2/linq4j/src/test/java/org/apache/calcite/linq4j/test/Linq4jTest.java:[738,25]
> Integer(int) in java.lang.Integer has been deprecated
> [WARNING]
> /home/jhyde/open1/calcite.2/core/src/main/java/org/apache/calcite/rex/RexBuilder.java:[539,47]
> ROUND_HALF_DOWN in java.math.BigDecimal has been deprecated
> {noformat}
> We should fix these (and other deprecated APIs when we run on other JDK
> versions and other Guava versions), then add a nightly check that none are
> added back.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CALCITE-1602) Remove uses of deprecated external APIs
Julian Hyde created CALCITE-1602:
Summary: Remove uses of deprecated external APIs
Key: CALCITE-1602
URL: https://issues.apache.org/jira/browse/CALCITE-1602
Project: Calcite
Issue Type: Bug
Reporter: Julian Hyde
Assignee: Julian Hyde
Calcite should not use deprecated methods and classes, especially in
third-party libraries. I propose that we adopt a zero-tolerance policy, fixing
deprecated calls as soon as they occur. As long as the third-party libraries
use semantic versioning, this should give us prior warning before things
disappear.
You can find the calls to deprecated APIs by adding
{{-Xlint:deprecation}} then running {{mvn
clean package -DskipTests | grep 'has been deprecated' | grep -v 'in
org.apache.calcite'}}. Currently there are 51 when you run in JDK9 and Guava
20.0. Here are the first few:
{noformat}
[WARNING]
/home/jhyde/open1/calcite.2/linq4j/src/main/java/org/apache/calcite/linq4j/tree/Types.java:[254,38]
getPackage(java.lang.String) in java.lang.Package has been deprecated
[WARNING]
/home/jhyde/open1/calcite.2/linq4j/src/test/java/org/apache/calcite/linq4j/test/Linq4jTest.java:[736,25]
Integer(int) in java.lang.Integer has been deprecated
[WARNING]
/home/jhyde/open1/calcite.2/linq4j/src/test/java/org/apache/calcite/linq4j/test/Linq4jTest.java:[737,26]
Integer(int) in java.lang.Integer has been deprecated
[WARNING]
/home/jhyde/open1/calcite.2/linq4j/src/test/java/org/apache/calcite/linq4j/test/Linq4jTest.java:[738,25]
Integer(int) in java.lang.Integer has been deprecated
[WARNING]
/home/jhyde/open1/calcite.2/core/src/main/java/org/apache/calcite/rex/RexBuilder.java:[539,47]
ROUND_HALF_DOWN in java.math.BigDecimal has been deprecated
{noformat}
We should fix these (and other deprecated APIs when we run on other JDK
versions and other Guava versions), then add a nightly check that none are
added back.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CALCITE-1500) Allow materialization-applied rels to run pre-processing programs
[
https://issues.apache.org/jira/browse/CALCITE-1500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836432#comment-15836432
]
Julian Hyde commented on CALCITE-1500:
--
I'm not sure we should be baking any session state into a Program. So, for
instance, we should not add {{List materializations,
List lattices}} parameters to the {{Programs.standard()}}. Maybe
we need to discuss changing the {{Program.run}} interface. In fact we need to
complete the discussion in CALCITE-1536 and figure out which state we believe
should belong to the program, the planner, the rel node tree, and which should
be passed into and out of the {{Program.run}} method.
Comments:
* I am concerned that making getRules() and ENUMERABLE_RULES public will make
it more difficult to refactor in future
* Please use "Returns ..." rather than "Return ..." in method javadoc
* How about renaming MaterializationOptUtil to RelOptMaterializations?
* The Pair of Pair data structure you are passing to Hook.PROGRAM is unwieldy.
Either define a struct with 3 fields, or create a Tuple3 class (analogous to
Pair).
> Allow materialization-applied rels to run pre-processing programs
> -
>
> Key: CALCITE-1500
> URL: https://issues.apache.org/jira/browse/CALCITE-1500
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.10.0
>Reporter: Maryann Xue
>Assignee: Maryann Xue
>
> VolcanoPlanner now takes the "originalRoot" as the input for
> materialized-view substitution, so the programs used in
> {{Prepare.optimize()}} will not be applied to these substituted rels. For
> example, a correlated subquery will be de-correlated but its equivalents with
> materialization substitutions will not be de-correlated. So it would be nice
> to have a way for the substituted rels to run specific programs too before
> starting volcano planning.
> The easiest solution might be using the new "root" for materialization
> substitution instead, but it would be based on the assumption that those
> "pre-processing" programs are simple ones like de-correlation and
> field-trimming. In order to allow a more general pre-processing program set,
> one that could have different optimization output for the original rel and
> for the materialization substituted rels, we'd better have an interface or
> some approach to run pre-processing programs for rels after materialization
> substitution.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CALCITE-1500) Allow materialization-applied rels to run pre-processing programs
[
https://issues.apache.org/jira/browse/CALCITE-1500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836432#comment-15836432
]
Julian Hyde edited comment on CALCITE-1500 at 1/24/17 6:33 PM:
---
I'm not sure we should be baking any session state into a Program. So, for
instance, we should not add {{List materializations,
List lattices}} parameters to the {{Programs.standard()}}. Maybe
we need to discuss changing the {{Program.run}} interface. In fact we need to
complete the discussion in CALCITE-1536 and figure out which state we believe
should belong to the program, the planner, the rel node tree, and which should
be passed into and out of the {{Program.run}} method.
Comments:
* I am concerned that making getRules() and ENUMERABLE_RULES public will make
it more difficult to refactor in future
* Please use "Returns ..." rather than "Return ..." in method javadoc
* How about renaming MaterializationOptUtil to RelOptMaterializations?
* The Pair of Pair data structure you are passing to Hook.PROGRAM is unwieldy.
Either define a struct with 3 fields, or create a Tuple3 class (analogous to
Pair).
* Please rename {{testSubquery}} to {{testSubQuery}}. I am trying to
standardize on sub-query being two words.
was (Author: julianhyde):
I'm not sure we should be baking any session state into a Program. So, for
instance, we should not add {{List materializations,
List lattices}} parameters to the {{Programs.standard()}}. Maybe
we need to discuss changing the {{Program.run}} interface. In fact we need to
complete the discussion in CALCITE-1536 and figure out which state we believe
should belong to the program, the planner, the rel node tree, and which should
be passed into and out of the {{Program.run}} method.
Comments:
* I am concerned that making getRules() and ENUMERABLE_RULES public will make
it more difficult to refactor in future
* Please use "Returns ..." rather than "Return ..." in method javadoc
* How about renaming MaterializationOptUtil to RelOptMaterializations?
* The Pair of Pair data structure you are passing to Hook.PROGRAM is unwieldy.
Either define a struct with 3 fields, or create a Tuple3 class (analogous to
Pair).
> Allow materialization-applied rels to run pre-processing programs
> -
>
> Key: CALCITE-1500
> URL: https://issues.apache.org/jira/browse/CALCITE-1500
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.10.0
>Reporter: Maryann Xue
>Assignee: Maryann Xue
>
> VolcanoPlanner now takes the "originalRoot" as the input for
> materialized-view substitution, so the programs used in
> {{Prepare.optimize()}} will not be applied to these substituted rels. For
> example, a correlated subquery will be de-correlated but its equivalents with
> materialization substitutions will not be de-correlated. So it would be nice
> to have a way for the substituted rels to run specific programs too before
> starting volcano planning.
> The easiest solution might be using the new "root" for materialization
> substitution instead, but it would be based on the assumption that those
> "pre-processing" programs are simple ones like de-correlation and
> field-trimming. In order to allow a more general pre-processing program set,
> one that could have different optimization output for the original rel and
> for the materialization substituted rels, we'd better have an interface or
> some approach to run pre-processing programs for rels after materialization
> substitution.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
