Re: [DISCUSS] Some quesitons about the new introduced window table functions (TUMBLE, HOP and SESSION)

2020-10-25 Thread Danny Chan
Thanks for the sharing, after some research we found that many DB engines has poor support for nested row type, for example, assumes: my_type is a UDT with schema as (a: int, b: varchar(20)) table T has schema as (f0: my_type, f1: int, f2: varchar(20)) PostgreSQL: select f0.a, max(f1) from T

Re: [DISCUSS] Some quesitons about the new introduced window table functions (TUMBLE, HOP and SESSION)

2020-10-23 Thread Rui Wang
Yes. I had offline chats with Kenneth and Julian. I think the primary reason that window metadata is not as a STRUCT, is just because STRUCT might not be well supported by engines. Even in Calcite, ROW (STRUCT) might now be supported well. Meanwhile, it seems like there is no serious issue for

Re: [DISCUSS] Some quesitons about the new introduced window table functions (TUMBLE, HOP and SESSION)

2020-10-22 Thread Pengcheng Liu
Hi, Danny, I think there is already an explanation in [1], as the author noted: > The return value of Tumble is a relation that includes all columns of data as well as > additional event time columns wstart and wend. It was considered to place the original > row in a nested row and wstart and

Re: [DISCUSS] Some quesitons about the new introduced window table functions (TUMBLE, HOP and SESSION)

2020-10-22 Thread Danny Chan
Rui Wang, can you help to consult with the author of the window TVF syntax ? We want to hear more suggestions. cc @Rui Wang Rui Wang 于2020年10月22日周四 下午12:01写道: > This seems reasonable to merge window metadata as a column (more compact > format). > > Maybe Julian can comment whether there was

Re: [DISCUSS] Some quesitons about the new introduced window table functions (TUMBLE, HOP and SESSION)

2020-06-08 Thread Rui Wang
On Mon, Jun 8, 2020 at 12:04 AM Viliam Durina wrote: > > 2. rename the hop_size to emit_frequency. Because that's what this > argument tries to say: e.g. emit in every x time_unit. > > It's not about how often you emit. By "emit" I mean the moment when the > rows are sent out. For example, when

Re: [DISCUSS] Some quesitons about the new introduced window table functions (TUMBLE, HOP and SESSION)

2020-06-08 Thread Viliam Durina
> 2. rename the hop_size to emit_frequency. Because that's what this argument tries to say: e.g. emit in every x time_unit. It's not about how often you emit. By "emit" I mean the moment when the rows are sent out. For example, when working with event time and the input stalls, you might emit

Re: [DISCUSS] Some quesitons about the new introduced window table functions (TUMBLE, HOP and SESSION)

2020-06-06 Thread Rui Wang
Thanks Danny for your summary! For question one, I prefer the ordering of parameter in option two. However, for argument names, I think it might be better to 1. name the window size explicitly by "window_size". 2. rename the hop_size to emit_frequency. Because that's what this argument tries to

[DISCUSS] Some quesitons about the new introduced window table functions (TUMBLE, HOP and SESSION)

2020-06-04 Thread Danny Chan
Hi, all, in the last 2 releases, Rui Wang has contributed the window table functions to replace the legacy group window syntax, the idea comes from the SQL-2016 polymorphic table functions. But the current implementation also got some feedbacks(or confusion), this thread tries to have a