[jira] [Commented] (FLINK-34156) Move Flink Calcite rules from Scala to Java
[
https://issues.apache.org/jira/browse/FLINK-34156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825119#comment-17825119
]
Sergey Nuyanzin commented on FLINK-34156:
-
the more people the better for the community
may be however it would also make sense to double check with them what they are
going to do, especially if they are new to Flink community
> Move Flink Calcite rules from Scala to Java
> ---
>
> Key: FLINK-34156
> URL: https://issues.apache.org/jira/browse/FLINK-34156
> Project: Flink
> Issue Type: Technical Debt
> Components: Table SQL / Planner
>Reporter: Sergey Nuyanzin
>Assignee: Sergey Nuyanzin
>Priority: Major
> Fix For: 2.0.0
>
>
> This is an umbrella task for migration of Calcite rules from Scala to Java
> mentioned at [https://cwiki.apache.org/confluence/display/FLINK/2.0+Release]
> The reason is that since 1.28.0 ( CALCITE-4787 - Move core to use Immutables
> instead of ImmutableBeans ) Calcite started to use Immutables
> ([https://immutables.github.io/]) and since 1.29.0 removed ImmutableBeans (
> CALCITE-4839 - Remove remnants of ImmutableBeans post 1.28 release ). All
> rule configuration related api which is not Immutables based is marked as
> deprecated. Since Immutables implies code generation while java compilation
> it is seems impossible to use for rules in Scala code.
> We could follow steps from javadocs of {{org.apache.calcite.plan.RelRule}}
> written for migration from deprecated java api to Immutables.
> It would work for scala to java migration as well.
> Please keep in mind that there is +*no need*+ to migrate rules extending
> +ConverterRule+ since these rules do not have such problem.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-34156) Move Flink Calcite rules from Scala to Java
[
https://issues.apache.org/jira/browse/FLINK-34156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824595#comment-17824595
]
Jacky Lau commented on FLINK-34156:
---
[~Sergey Nuyanzin]
If the community is short on manpower, I can also get my colleagues involved in
submitting PRs and reviewing code.
> Move Flink Calcite rules from Scala to Java
> ---
>
> Key: FLINK-34156
> URL: https://issues.apache.org/jira/browse/FLINK-34156
> Project: Flink
> Issue Type: Technical Debt
> Components: Table SQL / Planner
>Reporter: Sergey Nuyanzin
>Assignee: Sergey Nuyanzin
>Priority: Major
> Fix For: 2.0.0
>
>
> This is an umbrella task for migration of Calcite rules from Scala to Java
> mentioned at [https://cwiki.apache.org/confluence/display/FLINK/2.0+Release]
> The reason is that since 1.28.0 ( CALCITE-4787 - Move core to use Immutables
> instead of ImmutableBeans ) Calcite started to use Immutables
> ([https://immutables.github.io/]) and since 1.29.0 removed ImmutableBeans (
> CALCITE-4839 - Remove remnants of ImmutableBeans post 1.28 release ). All
> rule configuration related api which is not Immutables based is marked as
> deprecated. Since Immutables implies code generation while java compilation
> it is seems impossible to use for rules in Scala code.
> We could follow steps from javadocs of {{org.apache.calcite.plan.RelRule}}
> written for migration from deprecated java api to Immutables.
> It would work for scala to java migration as well.
> Please keep in mind that there is +*no need*+ to migrate rules extending
> +ConverterRule+ since these rules do not have such problem.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-34156) Move Flink Calcite rules from Scala to Java
[
https://issues.apache.org/jira/browse/FLINK-34156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824594#comment-17824594
]
Jacky Lau commented on FLINK-34156:
---
[~Sergey Nuyanzin] I agree that these can be set to a low priority, but since
these PRs have already been submitted, I personally think there is no need to
close them. Besides, in the long run, to maintain uniformity in rule coding,
these will eventually need to be migrated, and the community is also trying to
avoid writing new code in Scala as much as possible. Moreover, Calcite itself
is written in Java. So, I believe there is no need to close the PRs for the
converter rules. What do you think?
> Move Flink Calcite rules from Scala to Java
> ---
>
> Key: FLINK-34156
> URL: https://issues.apache.org/jira/browse/FLINK-34156
> Project: Flink
> Issue Type: Technical Debt
> Components: Table SQL / Planner
>Reporter: Sergey Nuyanzin
>Assignee: Sergey Nuyanzin
>Priority: Major
> Fix For: 2.0.0
>
>
> This is an umbrella task for migration of Calcite rules from Scala to Java
> mentioned at [https://cwiki.apache.org/confluence/display/FLINK/2.0+Release]
> The reason is that since 1.28.0 ( CALCITE-4787 - Move core to use Immutables
> instead of ImmutableBeans ) Calcite started to use Immutables
> ([https://immutables.github.io/]) and since 1.29.0 removed ImmutableBeans (
> CALCITE-4839 - Remove remnants of ImmutableBeans post 1.28 release ). All
> rule configuration related api which is not Immutables based is marked as
> deprecated. Since Immutables implies code generation while java compilation
> it is seems impossible to use for rules in Scala code.
> We could follow steps from javadocs of {{org.apache.calcite.plan.RelRule}}
> written for migration from deprecated java api to Immutables.
> It would work for scala to java migration as well.
> Please keep in mind that there is +*no need*+ to migrate rules extending
> +ConverterRule+ since these rules do not have such problem.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-34156) Move Flink Calcite rules from Scala to Java
[
https://issues.apache.org/jira/browse/FLINK-34156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824390#comment-17824390
]
Sergey Nuyanzin commented on FLINK-34156:
-
I would suggest put it on hold with the lowest minor priority
We shouldn't priorities working on it when the real issue is not fixed and
these changes do not change the situation
We could come back to these converter rules once the main issue is fixed
> Move Flink Calcite rules from Scala to Java
> ---
>
> Key: FLINK-34156
> URL: https://issues.apache.org/jira/browse/FLINK-34156
> Project: Flink
> Issue Type: Technical Debt
> Components: Table SQL / Planner
>Reporter: Sergey Nuyanzin
>Assignee: Sergey Nuyanzin
>Priority: Major
> Fix For: 2.0.0
>
>
> This is an umbrella task for migration of Calcite rules from Scala to Java
> mentioned at [https://cwiki.apache.org/confluence/display/FLINK/2.0+Release]
> The reason is that since 1.28.0 ( CALCITE-4787 - Move core to use Immutables
> instead of ImmutableBeans ) Calcite started to use Immutables
> ([https://immutables.github.io/]) and since 1.29.0 removed ImmutableBeans (
> CALCITE-4839 - Remove remnants of ImmutableBeans post 1.28 release ). All
> rule configuration related api which is not Immutables based is marked as
> deprecated. Since Immutables implies code generation while java compilation
> it is seems impossible to use for rules in Scala code.
> We could follow steps from javadocs of {{org.apache.calcite.plan.RelRule}}
> written for migration from deprecated java api to Immutables.
> It would work for scala to java migration as well.
> Please keep in mind that there is +*no need*+ to migrate rules extending
> +ConverterRule+ since these rules do not have such problem.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-34156) Move Flink Calcite rules from Scala to Java
[
https://issues.apache.org/jira/browse/FLINK-34156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824380#comment-17824380
]
Jacky Lau commented on FLINK-34156:
---
[~Sergey Nuyanzin]
I've noticed that some Scala code is being converted to Java in other PRs.
Given that migrating these few doesn't pose any problems, maybe we should leave
the rest as is. Since the issues for these have already been created and the
PRs have been submitted, let's just continue with them. What do you think?
> Move Flink Calcite rules from Scala to Java
> ---
>
> Key: FLINK-34156
> URL: https://issues.apache.org/jira/browse/FLINK-34156
> Project: Flink
> Issue Type: Technical Debt
> Components: Table SQL / Planner
>Reporter: Sergey Nuyanzin
>Assignee: Sergey Nuyanzin
>Priority: Major
> Fix For: 2.0.0
>
>
> This is an umbrella task for migration of Calcite rules from Scala to Java
> mentioned at [https://cwiki.apache.org/confluence/display/FLINK/2.0+Release]
> The reason is that since 1.28.0 ( CALCITE-4787 - Move core to use Immutables
> instead of ImmutableBeans ) Calcite started to use Immutables
> ([https://immutables.github.io/]) and since 1.29.0 removed ImmutableBeans (
> CALCITE-4839 - Remove remnants of ImmutableBeans post 1.28 release ). All
> rule configuration related api which is not Immutables based is marked as
> deprecated. Since Immutables implies code generation while java compilation
> it is seems impossible to use for rules in Scala code.
> We could follow steps from javadocs of {{org.apache.calcite.plan.RelRule}}
> written for migration from deprecated java api to Immutables.
> It would work for scala to java migration as well.
> Please keep in mind that there is +*no need*+ to migrate rules extending
> +ConverterRule+ since these rules do not have such problem.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-34156) Move Flink Calcite rules from Scala to Java
[ https://issues.apache.org/jira/browse/FLINK-34156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824275#comment-17824275 ] Sergey Nuyanzin commented on FLINK-34156: - I do not see the real benefit from such migration Rules written both in Scala and Java are working fine together, so it was not an issue before, I don't think it is an issue now. The real reason of this migration is inability to use non deprecated code to configure these rules in Scala We could think about full migration to java in future, however i guess it should be discussed in ML as well > Move Flink Calcite rules from Scala to Java > --- > > Key: FLINK-34156 > URL: https://issues.apache.org/jira/browse/FLINK-34156 > Project: Flink > Issue Type: Technical Debt > Components: Table SQL / Planner >Reporter: Sergey Nuyanzin >Assignee: Sergey Nuyanzin >Priority: Major > Fix For: 2.0.0 > > > This is an umbrella task for migration of Calcite rules from Scala to Java > mentioned at https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > The reason is that since 1.28.0 ( CALCITE-4787 - Move core to use Immutables > instead of ImmutableBeans ) Calcite started to use Immutables > (https://immutables.github.io/) and since 1.29.0 removed ImmutableBeans ( > CALCITE-4839 - Remove remnants of ImmutableBeans post 1.28 release ). All > rule configuration related api which is not Immutables based is marked as > deprecated. Since Immutables implies code generation while java compilation > it is seems impossible to use for rules in Scala code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-34156) Move Flink Calcite rules from Scala to Java
[ https://issues.apache.org/jira/browse/FLINK-34156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824266#comment-17824266 ] Jacky Lau commented on FLINK-34156: --- hi [~Sergey Nuyanzin] Thanks for your summary. Although the current ConverterRule does not have the issue described, and the current Flink planner module has achieved Scala-free status, I still suggest migrate to Java to maintain consistency in the rules. What do you think? And i am familiar with planner model and calcite, I am very happy to participate in the review work. > Move Flink Calcite rules from Scala to Java > --- > > Key: FLINK-34156 > URL: https://issues.apache.org/jira/browse/FLINK-34156 > Project: Flink > Issue Type: Technical Debt > Components: Table SQL / Planner >Reporter: Sergey Nuyanzin >Assignee: Sergey Nuyanzin >Priority: Major > Fix For: 2.0.0 > > > This is an umbrella task for migration of Calcite rules from Scala to Java > mentioned at https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > The reason is that since 1.28.0 ( CALCITE-4787 - Move core to use Immutables > instead of ImmutableBeans ) Calcite started to use Immutables > (https://immutables.github.io/) and since 1.29.0 removed ImmutableBeans ( > CALCITE-4839 - Remove remnants of ImmutableBeans post 1.28 release ). All > rule configuration related api which is not Immutables based is marked as > deprecated. Since Immutables implies code generation while java compilation > it is seems impossible to use for rules in Scala code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-34156) Move Flink Calcite rules from Scala to Java
[ https://issues.apache.org/jira/browse/FLINK-34156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824223#comment-17824223 ] Yunhong Zheng commented on FLINK-34156: --- Thanks, [~Sergey Nuyanzin] . I will try to review these PRs. > Move Flink Calcite rules from Scala to Java > --- > > Key: FLINK-34156 > URL: https://issues.apache.org/jira/browse/FLINK-34156 > Project: Flink > Issue Type: Technical Debt > Components: Table SQL / Planner >Reporter: Sergey Nuyanzin >Assignee: Sergey Nuyanzin >Priority: Major > Fix For: 2.0.0 > > > This is an umbrella task for migration of Calcite rules from Scala to Java > mentioned at https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > The reason is that since 1.28.0 ( CALCITE-4787 - Move core to use Immutables > instead of ImmutableBeans ) Calcite started to use Immutables > (https://immutables.github.io/) and since 1.29.0 removed ImmutableBeans ( > CALCITE-4839 - Remove remnants of ImmutableBeans post 1.28 release ). All > rule configuration related api which is not Immutables based is marked as > deprecated. Since Immutables implies code generation while java compilation > it is seems impossible to use for rules in Scala code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-34156) Move Flink Calcite rules from Scala to Java
[
https://issues.apache.org/jira/browse/FLINK-34156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824162#comment-17824162
]
Sergey Nuyanzin commented on FLINK-34156:
-
Thanks everyone for volunteering
Yes I think we could have something like collaboration here
It would be great if everyone here will not only submit PRs but also help to
review it
About the procedure to migrate: I would suggest to follow the steps already
described at javadocs of {{org.apache.calcite.plan.RelRule}}
In fact the steps are written for migration from deprecated java api to
Immutables. It would work for scala to java migration as well.
Also it would make sense to mention that there is *NO* need to migrate rules
extending {{ConverterRule}} since such rules do not have the problem described
in issue description.
> Move Flink Calcite rules from Scala to Java
> ---
>
> Key: FLINK-34156
> URL: https://issues.apache.org/jira/browse/FLINK-34156
> Project: Flink
> Issue Type: Technical Debt
> Components: Table SQL / Planner
>Reporter: Sergey Nuyanzin
>Assignee: Sergey Nuyanzin
>Priority: Major
> Fix For: 2.0.0
>
>
> This is an umbrella task for migration of Calcite rules from Scala to Java
> mentioned at https://cwiki.apache.org/confluence/display/FLINK/2.0+Release
> The reason is that since 1.28.0 ( CALCITE-4787 - Move core to use Immutables
> instead of ImmutableBeans ) Calcite started to use Immutables
> (https://immutables.github.io/) and since 1.29.0 removed ImmutableBeans (
> CALCITE-4839 - Remove remnants of ImmutableBeans post 1.28 release ). All
> rule configuration related api which is not Immutables based is marked as
> deprecated. Since Immutables implies code generation while java compilation
> it is seems impossible to use for rules in Scala code.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-34156) Move Flink Calcite rules from Scala to Java
[ https://issues.apache.org/jira/browse/FLINK-34156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17818607#comment-17818607 ] Yunhong Zheng commented on FLINK-34156: --- Hi, [~Sergey Nuyanzin] . Since I have been continuously involved in the development related to table-planner and calcite, I am quite familiar with this area. Could I possibly join this work to help you to deal with some subtasks? Looking forward your reply, Thanks. > Move Flink Calcite rules from Scala to Java > --- > > Key: FLINK-34156 > URL: https://issues.apache.org/jira/browse/FLINK-34156 > Project: Flink > Issue Type: Technical Debt > Components: Table SQL / Planner >Reporter: Sergey Nuyanzin >Assignee: Sergey Nuyanzin >Priority: Major > Fix For: 2.0.0 > > > This is an umbrella task for migration of Calcite rules from Scala to Java > mentioned at https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > The reason is that since 1.28.0 ( CALCITE-4787 - Move core to use Immutables > instead of ImmutableBeans ) Calcite started to use Immutables > (https://immutables.github.io/) and since 1.29.0 removed ImmutableBeans ( > CALCITE-4839 - Remove remnants of ImmutableBeans post 1.28 release ). All > rule configuration related api which is not Immutables based is marked as > deprecated. Since Immutables implies code generation while java compilation > it is seems impossible to use for rules in Scala code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-34156) Move Flink Calcite rules from Scala to Java
[ https://issues.apache.org/jira/browse/FLINK-34156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17808635#comment-17808635 ] Jiabao Sun commented on FLINK-34156: Thanks [~Sergey Nuyanzin]. If there is a need for assistance, please feel free to ping me at any time. > Move Flink Calcite rules from Scala to Java > --- > > Key: FLINK-34156 > URL: https://issues.apache.org/jira/browse/FLINK-34156 > Project: Flink > Issue Type: Technical Debt > Components: Table SQL / Planner >Reporter: Sergey Nuyanzin >Assignee: Sergey Nuyanzin >Priority: Major > Fix For: 2.0.0 > > > This is an umbrella task for migration of Calcite rules from Scala to Java > mentioned at https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > The reason is that since 1.28.0 ( CALCITE-4787 - Move core to use Immutables > instead of ImmutableBeans ) Calcite started to use Immutables > (https://immutables.github.io/) and since 1.29.0 removed ImmutableBeans ( > CALCITE-4839 - Remove remnants of ImmutableBeans post 1.28 release ). All > rule configuration related api which is not Immutables based is marked as > deprecated. Since Immutables implies code generation while java compilation > it is seems impossible to use for rules in Scala code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-34156) Move Flink Calcite rules from Scala to Java
[ https://issues.apache.org/jira/browse/FLINK-34156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17808489#comment-17808489 ] Sergey Nuyanzin commented on FLINK-34156: - Thanks for volunteering currently this is only MVP activity meaning that the main part is aiming for later (2.0 ) > Move Flink Calcite rules from Scala to Java > --- > > Key: FLINK-34156 > URL: https://issues.apache.org/jira/browse/FLINK-34156 > Project: Flink > Issue Type: Technical Debt > Components: Table SQL / Planner >Reporter: Sergey Nuyanzin >Assignee: Sergey Nuyanzin >Priority: Major > Fix For: 2.0.0 > > > This is an umbrella task for migration of Calcite rules from Scala to Java > mentioned at https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > The reason is that since 1.28.0 ( CALCITE-4787 - Move core to use Immutables > instead of ImmutableBeans ) Calcite started to use Immutables > (https://immutables.github.io/) and since 1.29.0 removed ImmutableBeans ( > CALCITE-4839 - Remove remnants of ImmutableBeans post 1.28 release ). All > rule configuration related api which is not Immutables based is marked as > deprecated. Since Immutables implies code generation while java compilation > it is seems impossible to use for rules in Scala code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-34156) Move Flink Calcite rules from Scala to Java
[ https://issues.apache.org/jira/browse/FLINK-34156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17808430#comment-17808430 ] Jacky Lau commented on FLINK-34156: --- hi [~Sergey Nuyanzin] can I also help with this task? > Move Flink Calcite rules from Scala to Java > --- > > Key: FLINK-34156 > URL: https://issues.apache.org/jira/browse/FLINK-34156 > Project: Flink > Issue Type: Technical Debt > Components: Table SQL / Planner >Reporter: Sergey Nuyanzin >Assignee: Sergey Nuyanzin >Priority: Major > Fix For: 2.0.0 > > > This is an umbrella task for migration of Calcite rules from Scala to Java > mentioned at https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > The reason is that since 1.28.0 ( CALCITE-4787 - Move core to use Immutables > instead of ImmutableBeans ) Calcite started to use Immutables > (https://immutables.github.io/) and since 1.29.0 removed ImmutableBeans ( > CALCITE-4839 - Remove remnants of ImmutableBeans post 1.28 release ). All > rule configuration related api which is not Immutables based is marked as > deprecated. Since Immutables implies code generation while java compilation > it is seems impossible to use for rules in Scala code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-34156) Move Flink Calcite rules from Scala to Java
[ https://issues.apache.org/jira/browse/FLINK-34156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17808425#comment-17808425 ] Jiabao Sun commented on FLINK-34156: Hi [~Sergey Nuyanzin], can I help with this task? > Move Flink Calcite rules from Scala to Java > --- > > Key: FLINK-34156 > URL: https://issues.apache.org/jira/browse/FLINK-34156 > Project: Flink > Issue Type: Technical Debt > Components: Table SQL / Planner >Reporter: Sergey Nuyanzin >Assignee: Sergey Nuyanzin >Priority: Major > Fix For: 2.0.0 > > > This is an umbrella task for migration of Calcite rules from Scala to Java > mentioned at https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > The reason is that since 1.28.0 ( CALCITE-4787 - Move core to use Immutables > instead of ImmutableBeans ) Calcite started to use Immutables > (https://immutables.github.io/) and since 1.29.0 removed ImmutableBeans ( > CALCITE-4839 - Remove remnants of ImmutableBeans post 1.28 release ). All > rule configuration related api which is not Immutables based is marked as > deprecated. Since Immutables implies code generation while java compilation > it is seems impossible to use for rules in Scala code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
