[jira] [Commented] (FLINK-24024) Support session Window TVF
[ https://issues.apache.org/jira/browse/FLINK-24024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17810627#comment-17810627 ] Jane Chan commented on FLINK-24024: --- Fixed in master 4c1ae285fb672bbe65f894251ee9a7cfc49db6a6 > Support session Window TVF > --- > > Key: FLINK-24024 > URL: https://issues.apache.org/jira/browse/FLINK-24024 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Jing Zhang >Assignee: Sergey Nuyanzin >Priority: Major > Labels: pull-request-available, stale-assigned > > > # Fix calcite syntax CALCITE-4337 > # Introduce session window TVF in Flink > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-24024) Support session Window TVF
[ https://issues.apache.org/jira/browse/FLINK-24024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17804517#comment-17804517 ] Sergey Nuyanzin commented on FLINK-24024: - [~xuyangzhong] thanks a lot for working on this and really sorry for the delay I will try to have a look this week > Support session Window TVF > --- > > Key: FLINK-24024 > URL: https://issues.apache.org/jira/browse/FLINK-24024 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Jing Zhang >Assignee: Sergey Nuyanzin >Priority: Major > Labels: pull-request-available, stale-assigned > > > # Fix calcite syntax CALCITE-4337 > # Introduce session window TVF in Flink > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-24024) Support session Window TVF
[ https://issues.apache.org/jira/browse/FLINK-24024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17804078#comment-17804078 ] xuyang commented on FLINK-24024: Hi, [~Sergey Nuyanzin] Would you like to review this pr? The pr is ready and the CI has passed. > Support session Window TVF > --- > > Key: FLINK-24024 > URL: https://issues.apache.org/jira/browse/FLINK-24024 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Jing Zhang >Assignee: Sergey Nuyanzin >Priority: Major > Labels: pull-request-available, stale-assigned > > > # Fix calcite syntax CALCITE-4337 > # Introduce session window TVF in Flink > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-24024) Support session Window TVF
[ https://issues.apache.org/jira/browse/FLINK-24024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773517#comment-17773517 ] xuyang commented on FLINK-24024: Hi, [~Sergey Nuyanzin] [~twalthr] ，I did a basic poc on my branch here '[https://github.com/xuyangzhong/flink/tree/FLINK-24024'|https://github.com/xuyangzhong/flink/tree/FLINK-24024'.] . In this branch, at the grammar part and planner part the session window tvf will be supported. So I think the current calcite version 1.32.0 has met the requirements to support session window tvf. WDYT? > Support session Window TVF > --- > > Key: FLINK-24024 > URL: https://issues.apache.org/jira/browse/FLINK-24024 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Jing Zhang >Assignee: Sergey Nuyanzin >Priority: Major > Labels: pull-request-available, stale-assigned > > > # Fix calcite syntax CALCITE-4337 > # Introduce session window TVF in Flink > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-24024) Support session Window TVF
[ https://issues.apache.org/jira/browse/FLINK-24024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752063#comment-17752063 ] Timo Walther commented on FLINK-24024: -- Thanks for the background information [~Sergey Nuyanzin]. I also don't fully understand the code in SqlToRelConverter but I guess after some investigation we can figure out this part. If you identify some major blocker from the Calcite side, let me know. Would be great to come up with some early hacky prototype. We can then iterate to finally deprecate the old window syntax with a replacement. > Support session Window TVF > --- > > Key: FLINK-24024 > URL: https://issues.apache.org/jira/browse/FLINK-24024 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Jing Zhang >Assignee: Sergey Nuyanzin >Priority: Major > Labels: pull-request-available, stale-assigned > > > # Fix calcite syntax CALCITE-4337 > # Introduce session window TVF in Flink > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-24024) Support session Window TVF
[
https://issues.apache.org/jira/browse/FLINK-24024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751995#comment-17751995
]
Sergey Nuyanzin commented on FLINK-24024:
-
[~twalthr] sorry for the delay, probably I forgot to put some updates
{quote}
so I guess CALCITE-4337 is the missing piece to finally make SESSION TVF work
in Flink? Or is the shortcoming you are describing another Calcite issue?
{quote}
in fact this is not clear
the error above could be fixed by overriding of
{{org.apache.flink.table.planner.functions.sql.SqlSessionTableFunction#tableCharacteristic}}
for {{org.apache.flink.table.planner.functions.sql.SqlSessionTableFunction}}
1. if {{SqlToRelConverter}}'s config is without expanding (Flink's case) then
it doesn't substitute the query [1] for {{SET_SEMANTICS_TABLE}}. And a number
of Flink cases require turned off expansion ({{withExpand(false)}}) [2]. This
is currently not clear why do we need expansion... However without query
substitution it is unable to generate input for
{{ProjectWindowTableFunctionTransposeRule}}
2. If we disable requirement for expansion for {{SET_SEMANTICS_TABLE}} then we
move a bit further and fail with unabling of generation of execution plan
because of missing conversions {{LogicalSortExchange[convention: NONE ->
LOGICAL]}} or {{LogicalExchange[convention: NONE -> LOGICAL]}} in case of no
{{ORDER BY}} clause.
I'd like to to try add these conversions to see if dinally it works or not.
Currently I see that it (however it is a bit unfamiliar Flink's area for me,
so it will be try/fail/learn and time consuming)
[1]
https://github.com/apache/calcite/blob/calcite-1.32.0/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L1377
[2]
https://github.com/apache/flink/blob/fa61dd439ece9e2b9ef710ce2f7eed54c4d4403a/flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/delegation/PlannerContext.java#L306
> Support session Window TVF
> ---
>
> Key: FLINK-24024
> URL: https://issues.apache.org/jira/browse/FLINK-24024
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / API
>Reporter: Jing Zhang
>Assignee: Sergey Nuyanzin
>Priority: Major
> Labels: pull-request-available, stale-assigned
>
>
> # Fix calcite syntax CALCITE-4337
> # Introduce session window TVF in Flink
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-24024) Support session Window TVF
[ https://issues.apache.org/jira/browse/FLINK-24024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751733#comment-17751733 ] Timo Walther commented on FLINK-24024: -- [~Sergey Nuyanzin] so I guess CALCITE-4337 is the missing piece to finally make SESSION TVF work in Flink? Or is the shortcoming you are describing another Calcite issue? > Support session Window TVF > --- > > Key: FLINK-24024 > URL: https://issues.apache.org/jira/browse/FLINK-24024 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Jing Zhang >Assignee: Sergey Nuyanzin >Priority: Major > Labels: pull-request-available, stale-assigned > > > # Fix calcite syntax CALCITE-4337 > # Introduce session window TVF in Flink > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-24024) Support session Window TVF
[
https://issues.apache.org/jira/browse/FLINK-24024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17743845#comment-17743845
]
Sergey Nuyanzin commented on FLINK-24024:
-
Currently it seems that from one side SQL Standard's syntax like
{code:sql}
SESSION(data [PARTITION BY (keycols, ...)], DESCRIPTOR(timecol), gap)
{code}
is supported, however queries with {{PARTITION BY}}, {{ORDER BY}} do not pass
Calcite's validation and fail with
{quote}
Only tables with set semantics may be partitioned. Invalid PARTITION BY clause
in the 0-th operand of table function 'SESSION'
{quote}
queries without {{PARTITION BY}}, {{ORDER BY}} seems working
> Support session Window TVF
> ---
>
> Key: FLINK-24024
> URL: https://issues.apache.org/jira/browse/FLINK-24024
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / API
>Reporter: Jing Zhang
>Assignee: Sergey Nuyanzin
>Priority: Major
> Labels: pull-request-available, stale-assigned
>
>
> # Fix calcite syntax CALCITE-4337
> # Introduce session window TVF in Flink
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-24024) Support session Window TVF
[ https://issues.apache.org/jira/browse/FLINK-24024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741591#comment-17741591 ] Sergey Nuyanzin commented on FLINK-24024: - Thanks for the response [~jingzhang] Then I will try to continue here > Support session Window TVF > --- > > Key: FLINK-24024 > URL: https://issues.apache.org/jira/browse/FLINK-24024 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Jing Zhang >Assignee: Jing Zhang >Priority: Major > Labels: pull-request-available, stale-assigned > > > # Fix calcite syntax CALCITE-4337 > # Introduce session window TVF in Flink > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-24024) Support session Window TVF
[ https://issues.apache.org/jira/browse/FLINK-24024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736980#comment-17736980 ] Jing Zhang commented on FLINK-24024: [~twalthr] [~Sergey Nuyanzin] Terribly sorry for late response. I think CALCITE-4865 is enough for Flink to support session window. The work does not depend on other subtask of CALCITE-4864 because Flink uses self defined `SqlWindowTableFunction`. > Support session Window TVF > --- > > Key: FLINK-24024 > URL: https://issues.apache.org/jira/browse/FLINK-24024 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Jing Zhang >Assignee: Jing Zhang >Priority: Major > Labels: pull-request-available, stale-assigned > > > # Fix calcite syntax CALCITE-4337 > # Introduce session window TVF in Flink > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-24024) Support session Window TVF
[ https://issues.apache.org/jira/browse/FLINK-24024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732444#comment-17732444 ] Sergey Nuyanzin commented on FLINK-24024: - [~jingzhang] could you please let us know if you are going to continue with this? I'm asking since required Calcite 1.31 upgrade PR is in review/addressing feedback process, and I would also like dive deeper here to check whether Calcite 1.31 is enough here or not. Also would like to help with this task (if you're going to continue I could help with review). > Support session Window TVF > --- > > Key: FLINK-24024 > URL: https://issues.apache.org/jira/browse/FLINK-24024 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Jing Zhang >Assignee: Jing Zhang >Priority: Major > Labels: pull-request-available, stale-assigned > > > # Fix calcite syntax CALCITE-4337 > # Introduce session window TVF in Flink > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-24024) Support session Window TVF
[ https://issues.apache.org/jira/browse/FLINK-24024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682573#comment-17682573 ] Timo Walther commented on FLINK-24024: -- [~jingzhang] what is the status of CALCITE-4864? We are approaching Calcite 1.30 soon. I see CALCITE-4865 has been implemented in Calcite 1.31. Is this enough to support session windows or do we need more subtasks of CALCITE-4864? > Support session Window TVF > --- > > Key: FLINK-24024 > URL: https://issues.apache.org/jira/browse/FLINK-24024 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Jing Zhang >Assignee: Jing Zhang >Priority: Major > Labels: pull-request-available, stale-assigned > > > # Fix calcite syntax CALCITE-4337 > # Introduce session window TVF in Flink > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-24024) Support session Window TVF
[ https://issues.apache.org/jira/browse/FLINK-24024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447377#comment-17447377 ] Martijn Visser commented on FLINK-24024: [~qingru zhang] Thanks for confirming :) > Support session Window TVF > --- > > Key: FLINK-24024 > URL: https://issues.apache.org/jira/browse/FLINK-24024 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Jing Zhang >Assignee: Jing Zhang >Priority: Major > Labels: pull-request-available > > > # Fix calcite syntax CALCITE-4337 > # Introduce session window TVF in Flink > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (FLINK-24024) Support session Window TVF
[ https://issues.apache.org/jira/browse/FLINK-24024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447360#comment-17447360 ] Jing Zhang commented on FLINK-24024: [~MartijnVisser] The old implementation has been reverted. We would continue to work on this issue after CALCITE-4864 is done and we upgrade flink dependency on calcite. > Support session Window TVF > --- > > Key: FLINK-24024 > URL: https://issues.apache.org/jira/browse/FLINK-24024 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Jing Zhang >Assignee: Jing Zhang >Priority: Major > Labels: pull-request-available > > > # Fix calcite syntax CALCITE-4337 > # Introduce session window TVF in Flink > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (FLINK-24024) Support session Window TVF
[ https://issues.apache.org/jira/browse/FLINK-24024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17439089#comment-17439089 ] Martijn Visser commented on FLINK-24024: [~godfreyhe] Is the revert still needed or has it been completed? > Support session Window TVF > --- > > Key: FLINK-24024 > URL: https://issues.apache.org/jira/browse/FLINK-24024 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: JING ZHANG >Assignee: JING ZHANG >Priority: Major > Labels: pull-request-available > > > # Fix calcite syntax CALCITE-4337 > # Introduce session window TVF in Flink > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-24024) Support session Window TVF
[ https://issues.apache.org/jira/browse/FLINK-24024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409209#comment-17409209 ] godfrey he commented on FLINK-24024: yes, Only "34d51002796afd800fc99d47e3dd06acbd403531" ([FLINK-23544][table-planner] Window TVF Supports session window in plan) will be reverted. I'm preparing the revert commit and validating all tests, and will push it after success. > Support session Window TVF > --- > > Key: FLINK-24024 > URL: https://issues.apache.org/jira/browse/FLINK-24024 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Affects Versions: 1.15.0 >Reporter: JING ZHANG >Assignee: JING ZHANG >Priority: Major > Labels: pull-request-available > > > # Fix calcite syntax CALCITE-4337 > # Introduce session window TVF in Flink > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-24024) Support session Window TVF
[ https://issues.apache.org/jira/browse/FLINK-24024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408822#comment-17408822 ] Timo Walther commented on FLINK-24024: -- No worries, the SQL standard is large and even topics that look easy at the beginning turn out to be more detailed. It would be great if you can help on the Calcite side [~qingru zhang] so that we can offer SESSION windows in 1.15. How do we proceed with 1.14? Are you preparing a revert PR? I guess the runtime part can still stay in the code base but only 34d51002796afd800fc99d47e3dd06acbd403531 needs to be reverted? > Support session Window TVF > --- > > Key: FLINK-24024 > URL: https://issues.apache.org/jira/browse/FLINK-24024 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Affects Versions: 1.15.0 >Reporter: JING ZHANG >Assignee: JING ZHANG >Priority: Major > Labels: pull-request-available > > > # Fix calcite syntax CALCITE-4337 > # Introduce session window TVF in Flink > -- This message was sent by Atlassian Jira (v8.3.4#803005)
