Re: [DISCUSS] Remove registration of TableSource/TableSink in Table Env and ConnectTableDescriptor

Hi Kurt, I fully agree with the proposal. Yes it was an omission that we did not deprecate the TableEnvironment#fromTableSource in the previous version. I would vote to remove all those methods altogether. Best, Dawid On 05/02/2020 07:36, Kurt Young wrote: > Hi all, > > I'd like to bring up a

[jira] [Created] (FLINK-15912) Add Context to improve TableSourceFactory and TableSinkFactory

Jingsong Lee created FLINK-15912: Summary: Add Context to improve TableSourceFactory and TableSinkFactory Key: FLINK-15912 URL: https://issues.apache.org/jira/browse/FLINK-15912 Project: Flink

Re: [VOTE] Release Flink Python API(PyFlink) 1.9.2 to PyPI, release candidate #0

Hi, Thanks for driving this, Jincheng. +1 (non-binding) - Verified signatures and checksums. - `pip install apache-flink-1.9.2.tar.gz` successfully. - Start local pyflink shell via `pyflink-shell.sh local` and try the examples in the help message, run well and no exception. - Try a word count

Re: [DISCUSS] Support User-Defined Table Function in PyFlink

Hi Hequn, Thanks for your feedback. Good suggestion. I will avoid Scala code in the flink-table module. Best, Xingbo Hequn Cheng 于2020年2月4日周二 下午10:14写道： > Hi Xingbo, > > Thanks a lot for bringing up the discussion. Looks good from my side. > One suggestion beyond the document: it would be

Re: [DISCUSS] Support User-Defined Table Function in PyFlink

Hi Jincheng, Thanks for your feed back. The more details we can discussed in the JIRA and PR. :) Best, Xingbo jincheng sun 于2020年2月4日周二 下午9:09写道： > Thanks for bring up this discussion Xingbo! > > The the design is pretty nice for me! This feature is really need which > mentioned in FLIP-58.

[jira] [Created] (FLINK-15911) Flink does not work over NAT

Till Rohrmann created FLINK-15911: - Summary: Flink does not work over NAT Key: FLINK-15911 URL: https://issues.apache.org/jira/browse/FLINK-15911 Project: Flink Issue Type: Bug

Re: [VOTE] Improve TableFactory to add Context

+1 form my side. Thanks for driving this. Btw, could you also attach a JIRA issue with the changes described in it, so that users can find the issue through the mailing list in the future. Best, Jark On Wed, 5 Feb 2020 at 13:38, Kurt Young wrote: > +1 from my side. > > Best, > Kurt > > > On

[jira] [Created] (FLINK-15910) Add NOTICE files for Stateful Functions

Tzu-Li (Gordon) Tai created FLINK-15910: --- Summary: Add NOTICE files for Stateful Functions Key: FLINK-15910 URL: https://issues.apache.org/jira/browse/FLINK-15910 Project: Flink Issue

Re: [DISCUSS] Upload the Flink Python API 1.9.x to PyPI for user convenience.

Hi all, Thanks Dian and all of your feedback! I have bring up the VOTE thread for the first RC [1]. I would appreciate if you could check it. Best, Jincheng [1] https://lists.apache.org/thread.html/r4b43a94db6731de4d450cee828ac2effd5a47bb8a8374e31af79b08a%40%3Cdev.flink.apache.org%3E dianfu

[VOTE] Release Flink Python API(PyFlink) 1.9.2 to PyPI, release candidate #0

Hi everyone, Please review and vote on the release candidate #0 for the PyFlink version 1.9.2, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * the official

[DISCUSS] Remove registration of TableSource/TableSink in Table Env and ConnectTableDescriptor

Hi all, I'd like to bring up a discussion about removing registration of TableSource and TableSink in TableEnvironment as well as in ConnectTableDescriptor. The affected method would be: TableEnvironment::registerTableSource TableEnvironment::fromTableSource TableEnvironment::registerTableSink

[jira] [Created] (FLINK-15909) Add PyPI release process into the subsequent release of 1.9.x

sunjincheng created FLINK-15909: --- Summary: Add PyPI release process into the subsequent release of 1.9.x Key: FLINK-15909 URL: https://issues.apache.org/jira/browse/FLINK-15909 Project: Flink

[jira] [Created] (FLINK-15908) Add Description of support 'pip install' to 1.9.x documents

sunjincheng created FLINK-15908: --- Summary: Add Description of support 'pip install' to 1.9.x documents Key: FLINK-15908 URL: https://issues.apache.org/jira/browse/FLINK-15908 Project: Flink

Re: [VOTE] Release 1.10.0, release candidate #1

Hi Gary, Thanks for the reply. --> On Tue, Feb 4, 2020 at 5:20 AM Gary Yao wrote: > Hi Thomas, > > > 2) Was there a change in how job recovery reflects in the uptime metric? > > Didn't uptime previously reset to 0 on recovery (now it just keeps > > increasing)? > > The uptime is the

Re: [VOTE] Improve TableFactory to add Context

+1 from my side. Best, Kurt On Wed, Feb 5, 2020 at 10:59 AM Jingsong Li wrote: > Hi all, > > Interface updated. > Please re-vote. > > Best, > Jingsong Lee > > On Tue, Feb 4, 2020 at 1:28 PM Jingsong Li wrote: > > > Hi all, > > > > I would like to start the vote for the improve of > >

[jira] [Created] (FLINK-15907) expose private method in Configuration so that user can extend from it

Steven Zhen Wu created FLINK-15907: -- Summary: expose private method in Configuration so that user can extend from it Key: FLINK-15907 URL: https://issues.apache.org/jira/browse/FLINK-15907 Project:

[DISCUSS] Support scalar vectorized Python UDF in PyFlink

Hi all, Scalar Python UDF has already been supported in the coming release 1.10 (FLIP-58[1]). It operates one row at a time. It works in the way that the Java operator serializes one input row to bytes and sends them to the Python worker; the Python worker deserializes the input row and

[jira] [Created] (FLINK-15906) physical memory exceeded causing being killed by yarn

liupengcheng created FLINK-15906: Summary: physical memory exceeded causing being killed by yarn Key: FLINK-15906 URL: https://issues.apache.org/jira/browse/FLINK-15906 Project: Flink Issue

Re: [VOTE] Improve TableFactory to add Context

Hi all, Interface updated. Please re-vote. Best, Jingsong Lee On Tue, Feb 4, 2020 at 1:28 PM Jingsong Li wrote: > Hi all, > > I would like to start the vote for the improve of > TableFactory, which is discussed and > reached a consensus in the discussion thread[2]. > > The vote will be open

Re: [DISCUSS] Improve TableFactory

Hi Timo, G ood catch! I really love the idea 2, a full Flink config looks very good to me. Try to understand your first one, actually we don't have `TableIdentifier` class now. But TableFactory already indicate table. So I am OK. New Context should be: /** * Context of table source

Re: [DISCUSS] Upload the Flink Python API 1.9.x to PyPI for user convenience.

Hi Jincheng, Thanks for the proposal. I think this is a good idea. Usually Python users will firstly try to install a Python package from PyPI when they are trying it out. This would definitely benefit Python users. Regards, Dian > 在 2020年2月5日，上午8:56，jincheng sun 写道： > > I see, thanks for

Re: [DISCUSS] Upload the Flink Python API 1.9.x to PyPI for user convenience.

Thanks everyone for your positive feedback, I will continue to push forward the subsequent release process. Best, Jincheng jincheng sun 于2020年2月5日周三 上午8:56写道： > I see, thanks for the confirmation Aljoscha! > > Best, > Jincheng > > > > Aljoscha Krettek 于2020年2月4日周二 下午9:58写道： > >> Yes, as I

Re: [DISCUSS] Upload the Flink Python API 1.9.x to PyPI for user convenience.

I see, thanks for the confirmation Aljoscha! Best, Jincheng Aljoscha Krettek 于2020年2月4日周二 下午9:58写道： > Yes, as I said I think it's still good to release. > > On 04.02.20 13:49, jincheng sun wrote: > > Hi Aljoscha, > > > > I agree that the coming PyFlink package of 1.10 is enough for users who

[jira] [Created] (FLINK-15905) Fix Race Condition when releasing shared memory resource

Stephan Ewen created FLINK-15905: Summary: Fix Race Condition when releasing shared memory resource Key: FLINK-15905 URL: https://issues.apache.org/jira/browse/FLINK-15905 Project: Flink

Re: [DISCUSS] Include flink-ml-api and flink-ml-lib in opt

Around a year ago I started a discussion on reducing the amount of jars we ship with the distribution. While there was no definitive conclusion there was a shared sentiment

Re: [DISCUSS] Include flink-ml-api and flink-ml-lib in opt

I think there is no such rule that APIs go automatically into opt/ and "libraries" not. The contents of opt/ have mainly grown over time w/o following a strict rule. I think the decisive factor for what goes into Flink's binary distribution should be how core it is to Flink. Of course another

Re: [DISCUSS] Support User-Defined Table Function in PyFlink

Hi Xingbo, Thanks a lot for bringing up the discussion. Looks good from my side. One suggestion beyond the document: it would be nice to avoid Scala code in the flink-table module since we would like to get rid of Scala in the long-term[1][2]. Best, Hequn [1]

Re: [DISCUSS] Upload the Flink Python API 1.9.x to PyPI for user convenience.

Yes, as I said I think it's still good to release. On 04.02.20 13:49, jincheng sun wrote: Hi Aljoscha, I agree that the coming PyFlink package of 1.10 is enough for users who just want to try out the latest version of PyFlink. However, I think that there are also cases where the old versions

[jira] [Created] (FLINK-15904) Make Kafka Consumer work with activated "disableGenericTypes()"

Aljoscha Krettek created FLINK-15904: Summary: Make Kafka Consumer work with activated "disableGenericTypes()" Key: FLINK-15904 URL: https://issues.apache.org/jira/browse/FLINK-15904 Project:

Re: [VOTE] Release 1.10.0, release candidate #1

Hi Thomas, > 2) Was there a change in how job recovery reflects in the uptime metric? > Didn't uptime previously reset to 0 on recovery (now it just keeps > increasing)? The uptime is the difference between the current time and the time when the job transitioned to RUNNING state. By default we

[jira] [Created] (FLINK-15903) Make REST port configurable in end to end test scripts

Robert Metzger created FLINK-15903: -- Summary: Make REST port configurable in end to end test scripts Key: FLINK-15903 URL: https://issues.apache.org/jira/browse/FLINK-15903 Project: Flink

[jira] [Created] (FLINK-15902) Improve the Python API doc of the version they are added

Dian Fu created FLINK-15902: --- Summary: Improve the Python API doc of the version they are added Key: FLINK-15902 URL: https://issues.apache.org/jira/browse/FLINK-15902 Project: Flink Issue Type:

Re: [DISCUSS] Support User-Defined Table Function in PyFlink

Thanks for bring up this discussion Xingbo! The the design is pretty nice for me! This feature is really need which mentioned in FLIP-58. So, I think is better to create the JIRA and open the PR, then more detail can be reviewed. :) Best, Jincheng Xingbo Huang 于2020年2月3日周一 下午3:02写道： > Hi

Re: [DISCUSS] Improve TableFactory

Hi Jingsong, some last minute changes from my side: 1. rename `getTableIdentifier` to `getObjectIdentifier` to keep the API obvious. Otherwise people expect a `TableIdentifier` class being returned here. 2. rename `getTableConfig` to `getConfiguration()` in the future this will not only be

Re: [DISCUSS] Upload the Flink Python API 1.9.x to PyPI for user convenience.

Hi Aljoscha, I agree that the coming PyFlink package of 1.10 is enough for users who just want to try out the latest version of PyFlink. However, I think that there are also cases where the old versions are useful. Usually users tend to prepare their development environment according to the

Re: [DISCUSS] have separate Flink distributions with built-in Hive dependencies

Hi Stephan, As Jingsong stated, in our documentation the recommended way to add Hive deps is to use exactly what users have installed. It's just we ask users to manually add those jars, instead of automatically find them based on env variables. I prefer to keep it this way for a while, and see if

Re: [DISCUSS] Include flink-ml-api and flink-ml-lib in opt

Thanks for the suggestion, Till. I am curious about how do we usually decide when to put the jars into the opt folder? Technically speaking, it seems that `flink-ml-api` should be put into the opt directory because they are actually API instead of libraries, just like CEP and Table.

Re: [DISCUSS] have separate Flink distributions with built-in Hive dependencies

Hi all, For your information, we have document the dependencies detailed information [1]. I think it's a lot clearer than before, but it's worse than presto and spark (they avoid or have built-in hive dependency). I thought about Stephan's suggestion: - The hive/lib has 200+ jars, but we only

[jira] [Created] (FLINK-15901) Support partitioned generic table in HiveCatalog

Rui Li created FLINK-15901: -- Summary: Support partitioned generic table in HiveCatalog Key: FLINK-15901 URL: https://issues.apache.org/jira/browse/FLINK-15901 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-15900) JoinITCase#testRightJoinWithPk failed on Travis

Gary Yao created FLINK-15900: Summary: JoinITCase#testRightJoinWithPk failed on Travis Key: FLINK-15900 URL: https://issues.apache.org/jira/browse/FLINK-15900 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-15899) Stateful Functions root POM should use the ASF parent POM

Tzu-Li (Gordon) Tai created FLINK-15899: --- Summary: Stateful Functions root POM should use the ASF parent POM Key: FLINK-15899 URL: https://issues.apache.org/jira/browse/FLINK-15899 Project:

[jira] [Created] (FLINK-15897) Deserialize the Python UDF execution results when sending them out

Dian Fu created FLINK-15897: --- Summary: Deserialize the Python UDF execution results when sending them out Key: FLINK-15897 URL: https://issues.apache.org/jira/browse/FLINK-15897 Project: Flink

[jira] [Created] (FLINK-15898) Make Stateful Functions release-ready

Tzu-Li (Gordon) Tai created FLINK-15898: --- Summary: Make Stateful Functions release-ready Key: FLINK-15898 URL: https://issues.apache.org/jira/browse/FLINK-15898 Project: Flink Issue

Re: [DISCUSS] Upload the Flink Python API 1.9.x to PyPI for user convenience.

Hi, I think that's a good idea, but we will also soon have Flink 1.10 anyways. Best, Aljoscha On 04.02.20 07:25, Hequn Cheng wrote: Hi Jincheng, +1 for this proposal. From the perspective of users, I think it would nice to have PyFlink on PyPI which makes it much easier to install PyFlink.