Re: [DISCUSS] Correct the terminology of "Time-windowed Join" to "Interval Join" in Table API & SQL

2019-12-22 Thread Jark Wu
I agree with Jingsong, we are discussing to align the "concepts", not align the "implementations". For the "concepts", the "Time-windowed Join" in SQL and "Interval Join" in DataStream are the same thing. Best, Jark On Mon, 23 Dec 2019 at 15:16, Jingsong Li wrote: > Hi Danny, > > >

[jira] [Created] (FLINK-15361) ParquetTableSource should pass predicate in projectFields

2019-12-22 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-15361: Summary: ParquetTableSource should pass predicate in projectFields Key: FLINK-15361 URL: https://issues.apache.org/jira/browse/FLINK-15361 Project: Flink

Re: [DISCUSS] Correct the terminology of "Time-windowed Join" to "Interval Join" in Table API & SQL

2019-12-22 Thread Jingsong Li
Hi Danny, > DatasStream interval join and Table/SQL Time-windowed Join are not equivalent In my opinion, there is no difference between table and DataStream except that outer join is not implemented in DataStream. KeyedStream has defined equivalent conditions. Other conditions can be completed

Re: Flink On K8s, build docker image very slowly, is there some way to make it faster?

2019-12-22 Thread vino yang
Hi Lake, Can you clearly count or identify which steps are taking a long time? Best, Vino LakeShen 于2019年12月23日周一 下午2:46写道: > Hi community , when I run the flink task on k8s , the first thing is that > to build the flink task jar to > Docker Image . I find that It would spend much time to

Re: Flink On K8s, build docker image very slowly, is there some way to make it faster?

2019-12-22 Thread Xintong Song
Hi Lake, Usually building a docker image should not take much time (typically less than 2 minutes). It is probably the network issue that causes the long time for image building. Of course we will need more information (e.g., logs) to confirm that, but according to our experience pulling the

Flink On K8s, build docker image very slowly, is there some way to make it faster?

2019-12-22 Thread LakeShen
Hi community , when I run the flink task on k8s , the first thing is that to build the flink task jar to Docker Image . I find that It would spend much time to build docker image. Is there some way to makr it faster. Thank your replay.

Re: [DISCUSS] Correct the terminology of "Time-windowed Join" to "Interval Join" in Table API & SQL

2019-12-22 Thread Danny Chan
Thanks Jark for bringing up this discussion, just look at the api definitions, it seems that Flink DatasStream interval join and Table/SQL Time-windowed Join are not equivalent for the join conditions: The Interval Join only supports event time columnbs comparison of the joined streams[1]; while

Re: [DISCUSS] Correct the terminology of "Time-windowed Join" to "Interval Join" in Table API & SQL

2019-12-22 Thread Jingsong Li
Thanks Jark for bringing this. +1 to use a unify name: "Interval Join" before 1.10 is release. I think maybe "Interval Join" was come from SQL world too in [1]. Another candidate is to use "Range Join", But considering DataStream, I am OK with "Interval". [1]

[DISCUSS] Correct the terminology of "Time-windowed Join" to "Interval Join" in Table API & SQL

2019-12-22 Thread Jark Wu
Hi everyone, Currently, in the Table API & SQL documentation[1], we call the joins with time conditions as "Time-windowed Join". However, the same feature is called "Interval Join" in DataStream[2]. We should align the terminology in Flink project. >From my point of view, "Interval Join" is more

Re: [DISCUSS] Some feedback after trying out the new FLIP-49 memory configurations

2019-12-22 Thread Xintong Song
> > +1 to not have more options for off-heap memory, that would get confusing > fast. We can keep the name "off-heap" for both task and framework, as long > as they mean the same thing: native plus direct, and fully counted into > MaxDirectMemory. I would suggest to update the config descriptions

Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL

2019-12-22 Thread Jark Wu
Hi Forward, Thanks for creating the FLIP. +1 to start a vote. @Hequn Cheng @Kurt Young , could you help to review the design doc too? Best, Jark On Mon, 23 Dec 2019 at 10:10, tison wrote: > modified: > >

Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL

2019-12-22 Thread tison
modified: https://lists.apache.org/x/thread.html/b3c0265cc2b660fe11ce550b84a831a7606de12908ff7ff0959a4794@%3Cdev.flink.apache.org%3E

Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL

2019-12-22 Thread tison
FYI the previous discussion is here[1]. Best, tison. [1]

Re: [DISCUSS] What parts of the Python API should we focus on next ?

2019-12-22 Thread jincheng sun
Hi Bowen, Your suggestions are very helpful for expanding the PyFlink ecology. I also mentioned above to integrate notebooks,Jupyter and Zeppelin are both very excellent notebooks. The process of integrating Jupyter and Zeppelin also requires the support of Jupyter and Zeppelin community

Re: [DISCUSS] FLIP-27: Refactor Source Interface

2019-12-22 Thread Becket Qin
Hi Steven, I think the current proposal is what you mentioned - a Kafka source that can be constructed in either BOUNDED or UNBOUNDED mode. And Flink can get the boundedness by invoking getBoundedness(). So one can create a Kafka source by doing something like the following: new

[ANNOUNCE] Weekly Community Update 2019/51

2019-12-22 Thread Hequn Cheng
Dear community, Happy to share this week's brief community digest with updates on Flink 1.10 and Flink 1.9.2, a proposal to integrate Flink Docker image publication into Flink release process, a discussion on new features of PyFlink and a couple of blog posts. Enjoy. Flink Development

[jira] [Created] (FLINK-15360) Yarn e2e test is broken with building docker image

2019-12-22 Thread Yang Wang (Jira)
Yang Wang created FLINK-15360: - Summary: Yarn e2e test is broken with building docker image Key: FLINK-15360 URL: https://issues.apache.org/jira/browse/FLINK-15360 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-15359) Remove unused YarnConfigOptions

2019-12-22 Thread Zili Chen (Jira)
Zili Chen created FLINK-15359: - Summary: Remove unused YarnConfigOptions Key: FLINK-15359 URL: https://issues.apache.org/jira/browse/FLINK-15359 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-15358) [configuration] the flink conf yaml file will skip the rest of value after a `#` comment sign

2019-12-22 Thread BlaBlabla (Jira)
BlaBlabla created FLINK-15358: - Summary: [configuration] the flink conf yaml file will skip the rest of value after a `#` comment sign Key: FLINK-15358 URL: https://issues.apache.org/jira/browse/FLINK-15358