Re: [DISCUSS] Switch to Azure Pipelines as the primary CI tool / switch off Travis

2020-03-25 Thread Dian Fu
Hi Robert, Thanks a lot for your great work! Overall I'm +1 to switch to Azure as the primary CI tool if it's stable enough as I think there is no need to run both the travis and Azure for one single PR. However, there are still some improvements need to do and it would be great if these

[jira] [Created] (FLINK-16759) HiveModuleTest failed to compile on release-1.10

2020-03-24 Thread Dian Fu (Jira)
Dian Fu created FLINK-16759: --- Summary: HiveModuleTest failed to compile on release-1.10 Key: FLINK-16759 URL: https://issues.apache.org/jira/browse/FLINK-16759 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-16747) Performance improvements for Python UDF

2020-03-24 Thread Dian Fu (Jira)
Dian Fu created FLINK-16747: --- Summary: Performance improvements for Python UDF Key: FLINK-16747 URL: https://issues.apache.org/jira/browse/FLINK-16747 Project: Flink Issue Type: Improvement

Re: [DISCUSS] Introduce TableFactory for StatefulSequenceSource

2020-03-22 Thread Dian Fu
Thanks Jingsong for bringing up this discussion. +1 to this proposal. I think Bowen's proposal makes much sense to me. This is also a painful problem for PyFlink users. Currently there is no built-in easy-to-use table source/sink and it requires users to write a lot of code to trying out

[jira] [Created] (FLINK-16650) Support LocalZonedTimestampType for Python UDF of blink planner

2020-03-18 Thread Dian Fu (Jira)
Dian Fu created FLINK-16650: --- Summary: Support LocalZonedTimestampType for Python UDF of blink planner Key: FLINK-16650 URL: https://issues.apache.org/jira/browse/FLINK-16650 Project: Flink Issue

Re: [VOTE] FLIP-112: Support User-Defined Metrics for Python UDF

2020-03-17 Thread Dian Fu
+1 (binding) On Tue, Mar 17, 2020 at 10:35 AM jincheng sun wrote: > +1 > > Best, > Jincheng > > > > Hequn Cheng 于2020年3月16日周一 上午10:01写道: > > > Hi everyone, > > > > I'd like to start the vote of FLIP-112[1] which is discussed and reached > > consensus in the discussion thread[2]. > > The vote

Re: [VOTE] FLIP-106: Support Python UDF in SQL Function DDL

2020-03-17 Thread Dian Fu
+1 (binding) On Tue, Mar 17, 2020 at 6:56 PM Hequn Cheng wrote: > +1 (binding) > > Best, > Hequn > > > On Mar 17, 2020, at 5:03 PM, Benchao Li wrote: > > > > +1 (non-binding) > > > > BTW it's in the same thread in my gmail too. > > > > > > > > Kurt Young 于2020年3月17日周二 上午11:47写道: > > > >>

[jira] [Created] (FLINK-16608) Support primitive data types for vectorized Python UDF

2020-03-15 Thread Dian Fu (Jira)
Dian Fu created FLINK-16608: --- Summary: Support primitive data types for vectorized Python UDF Key: FLINK-16608 URL: https://issues.apache.org/jira/browse/FLINK-16608 Project: Flink Issue Type

[jira] [Created] (FLINK-16607) Update flink-fn-execution.proto adding more schema information

2020-03-15 Thread Dian Fu (Jira)
Dian Fu created FLINK-16607: --- Summary: Update flink-fn-execution.proto adding more schema information Key: FLINK-16607 URL: https://issues.apache.org/jira/browse/FLINK-16607 Project: Flink Issue

[jira] [Created] (FLINK-16606) Throw exceptions for the data types which are not currently supported

2020-03-15 Thread Dian Fu (Jira)
Dian Fu created FLINK-16606: --- Summary: Throw exceptions for the data types which are not currently supported Key: FLINK-16606 URL: https://issues.apache.org/jira/browse/FLINK-16606 Project: Flink

Re: [DISCUSS] FLIP-112: Support User-Defined Metrics for Python UDF

2020-03-12 Thread Dian Fu
Hi Hequn, Thanks for driving this. +1 to this feature. Just one minor comment: It seems that we will add an API get_metric_group for the Python class FunctionContext, could you update the FLIP reflecting this? Thanks, Dian > 在 2020年3月10日,下午3:38,Wei Zhong 写道: > > Hi Hequn, > > Thanks for

[jira] [Created] (FLINK-16538) Restructure Python Table API documentation

2020-03-11 Thread Dian Fu (Jira)
Dian Fu created FLINK-16538: --- Summary: Restructure Python Table API documentation Key: FLINK-16538 URL: https://issues.apache.org/jira/browse/FLINK-16538 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-16486) Add documentation for vectorized Python UDF

2020-03-07 Thread Dian Fu (Jira)
Dian Fu created FLINK-16486: --- Summary: Add documentation for vectorized Python UDF Key: FLINK-16486 URL: https://issues.apache.org/jira/browse/FLINK-16486 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-16485) Support vectorized Python UDF in the batch mode of old planner

2020-03-07 Thread Dian Fu (Jira)
Dian Fu created FLINK-16485: --- Summary: Support vectorized Python UDF in the batch mode of old planner Key: FLINK-16485 URL: https://issues.apache.org/jira/browse/FLINK-16485 Project: Flink Issue

[jira] [Created] (FLINK-16484) Support all the data types in vectorized Python UDF

2020-03-07 Thread Dian Fu (Jira)
Dian Fu created FLINK-16484: --- Summary: Support all the data types in vectorized Python UDF Key: FLINK-16484 URL: https://issues.apache.org/jira/browse/FLINK-16484 Project: Flink Issue Type: Sub

[jira] [Created] (FLINK-16483) Add Python building blocks to make sure the basic functionality of vectorized Python UDF could work

2020-03-07 Thread Dian Fu (Jira)
Dian Fu created FLINK-16483: --- Summary: Add Python building blocks to make sure the basic functionality of vectorized Python UDF could work Key: FLINK-16483 URL: https://issues.apache.org/jira/browse/FLINK-16483

Re: [DISCUSS] Disable "Squash and merge" button for Flink repository on GitHub

2020-03-05 Thread Dian Fu
Hi Jark, Thanks for starting this discussion. Personally I also love the "squash and merge" button. It's very convenient. Regarding to the email address "noreply", it seems that there are two cases: - The email address in the original commit is already "noreply". In this case, this issue will

Re: Flink dev blog

2020-03-05 Thread Dian Fu
;>>>> Because, the dev blogs are very good learning materials. I >> believe >>>> many >>>>>>> users will be interested in these posts. It's just like >>>>>>> "Technology Deep Dive" talks in Flink Forward which attracts many >>&g

Re: Flink dev blog

2020-03-03 Thread Dian Fu
Big +1 on this idea. It will benefit both the developers and users a lot. Regarding to the place to hold these blogs, my preference is 3) as I notice that there are already a few high quality blogs on flink web-site[1] and I guess that may be a good place to start with. We just need to figure

[jira] [Created] (FLINK-16337) Add RelNodes and Rules for vectorized Python UDF execution

2020-02-28 Thread Dian Fu (Jira)
Dian Fu created FLINK-16337: --- Summary: Add RelNodes and Rules for vectorized Python UDF execution Key: FLINK-16337 URL: https://issues.apache.org/jira/browse/FLINK-16337 Project: Flink Issue Type

[jira] [Created] (FLINK-16271) Introduce ArrowScalarFunctionOperator for vectorized Python UDF execution

2020-02-24 Thread Dian Fu (Jira)
Dian Fu created FLINK-16271: --- Summary: Introduce ArrowScalarFunctionOperator for vectorized Python UDF execution Key: FLINK-16271 URL: https://issues.apache.org/jira/browse/FLINK-16271 Project: Flink

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

2020-02-20 Thread Dian Fu
Congrats Jingsong! > 在 2020年2月21日,上午11:39,Jark Wu 写道: > > Congratulations Jingsong! Well deserved. > > Best, > Jark > > On Fri, 21 Feb 2020 at 11:32, zoudan wrote: > >> Congratulations! Jingsong >> >> >> Best, >> Dan Zou >>

Re: [ANNOUNCE] Apache Flink-shaded 10.0 released

2020-02-19 Thread Dian Fu
Thanks Chesnay for the great work and everyone involved! Regards, Dian > 在 2020年2月20日,上午12:21,Zhijiang 写道: > > Thanks Chesnay for making the release efficiently and also thanks to all the > other participants! > > Best, > Zhijiang > > >

Re: [DISCUSS] Kicking off the 1.11 release cycle

2020-02-19 Thread Dian Fu
Thanks Stephan kicking off this discussion and Zhijiang volunteering as one of the release managers. +1 for the "feature freeze date" around end of April. There are still more than 2 months left, so I think it's a reasonable time. Thanks, Dian > 在 2020年2月19日,下午10:52,Aljoscha Krettek 写道: > >

[jira] [Created] (FLINK-16121) Introduce ArrowReader and ArrowWriter for Arrow format data read and write

2020-02-17 Thread Dian Fu (Jira)
Dian Fu created FLINK-16121: --- Summary: Introduce ArrowReader and ArrowWriter for Arrow format data read and write Key: FLINK-16121 URL: https://issues.apache.org/jira/browse/FLINK-16121 Project: Flink

[jira] [Created] (FLINK-16114) Support Scalar Vectorized Python UDF in PyFlink

2020-02-17 Thread Dian Fu (Jira)
Dian Fu created FLINK-16114: --- Summary: Support Scalar Vectorized Python UDF in PyFlink Key: FLINK-16114 URL: https://issues.apache.org/jira/browse/FLINK-16114 Project: Flink Issue Type: New

[RESULT][VOTE] FLIP-97: Support scalar vectorized Python UDF in PyFlink

2020-02-17 Thread Dian Fu
Hi all, Thanks you all for the discussion and votes. So far, we have - 3 binding +1 votes (Jincheng, Hequn, Dian) - 1 non-binding +1 votes (Jingsong) - No -1 votes The voting time has passed and there are enough +1 votes. Therefore, I'm happy to announce that FLIP-97[1] has been accepted.

Re: [VOTE] Support scalar vectorized Python UDF in PyFlink

2020-02-16 Thread Dian Fu
e >> >> On Thu, Feb 13, 2020 at 11:45 AM jincheng sun >> wrote: >> >>> +1 (binding) >>> >>> Best, >>> Jincheng >>> >>> >>> Dian Fu 于2020年2月12日周三 下午1:31写道: >>> >>>> Hi all, >>>&

[jira] [Created] (FLINK-16109) Move the Python scalar operators and table operators to separate package

2020-02-16 Thread Dian Fu (Jira)
Dian Fu created FLINK-16109: --- Summary: Move the Python scalar operators and table operators to separate package Key: FLINK-16109 URL: https://issues.apache.org/jira/browse/FLINK-16109 Project: Flink

Re: [ANNOUNCE] New Documentation Style Guide

2020-02-15 Thread Dian Fu
Thanks for the great work! This is very helpful to keep the documentation style consistent across the whole project. It's also very helpful for non-native English contributors like me. > 在 2020年2月15日,下午3:42,Jark Wu 写道: > > Great summary! Thanks for adding the translation specification in it.

[jira] [Created] (FLINK-16053) Remove redundant metrics in PyFlink

2020-02-13 Thread Dian Fu (Jira)
Dian Fu created FLINK-16053: --- Summary: Remove redundant metrics in PyFlink Key: FLINK-16053 URL: https://issues.apache.org/jira/browse/FLINK-16053 Project: Flink Issue Type: Improvement

Re: [VOTE] Support Python ML Pipeline API

2020-02-13 Thread Dian Fu
+1 (binding) Regards, Dian > 在 2020年2月14日,下午2:49,Yuan Mei 写道: > > +1 vote > > This is one of the most important things for Flink ML framework to be > widely adopted since most data scientists use python. > > Best > > Yuan > > On Fri, Feb 14, 2020 at 9:45 AM Hequn Cheng wrote: > >> Hi

Re: [VOTE] Release flink-shaded 10.0, release candidate #3

2020-02-13 Thread Dian Fu
+1 (non-binding) - Verified the signature and checksum - Checked the release note that all the tickets included in this release are there - Checked the website PR and it LGTM - Checked the notice file of the newly added module flink-shade-zookeeper-3 and it LGTM Regards, Dian > 在

Re: [ANNOUNCE] Apache Flink Python API(PyFlink) 1.9.2 released

2020-02-12 Thread Dian Fu
Thanks for the great work, Jincheng. Regards, Dian > 在 2020年2月13日,下午1:32,jincheng sun 写道: > > Hi everyone, > > The Apache Flink community is very happy to announce the release of Apache > Flink Python API(PyFlink) 1.9.2, which is the first release to PyPI for the > Apache Flink Python API

Re: [ANNOUNCE] Apache Flink 1.10.0 released

2020-02-12 Thread Dian Fu
Thanks Gary & Yu and everyone involved, Great work! Regards, Dian > 在 2020年2月12日,下午11:57,Haibo Sun 写道: > > Thanks Gary & Yu. Great work! > > Best, > Haibo > > At 2020-02-12 21:31:00, "Yu Li" wrote: > >The Apache Flink community is very happy to announce the release of Apache > >Flink

[VOTE] Support scalar vectorized Python UDF in PyFlink

2020-02-11 Thread Dian Fu
Hi all, I'd like to start the vote of FLIP-97[1] which is discussed and reached consensus in the discussion thread[2]. The vote will be open for at least 72 hours. Unless there is an objection, I will try to close it by Feb 17, 2020 08:00 UTC if we have received sufficient votes. Regards,

Re: [VOTE] FLIP-55: Introduction of a Table API Java Expression DSL

2020-02-11 Thread Dian Fu
t;>> >>> @Jark I think it makes sense to shorten them, I will do it I hope people >>> that already voted don't mind. >>> >>> @Dian That's a valid concern. I would not discard the '$' as a column >>> expression for java and scala. I think once we introd

Re: [DISCUSS] Support scalar vectorized Python UDF in PyFlink

2020-02-11 Thread Dian Fu
Hi all, Thanks you all participating this discussion and sharing your thoughts. It seems that we have reached consensus on the design now. I will start a VOTE thread if there are no other feedbacks. Thanks, Dian On Tue, Feb 11, 2020 at 10:23 AM Dian Fu wrote: > Hi Jingsong, > > You

Re: [VOTE] FLIP-55: Introduction of a Table API Java Expression DSL

2020-02-11 Thread Dian Fu
Hi Dawid, Thanks for driving this feature. The design looks very well for me overall. I have only one concern: $ is not allowed to be used in the identifier of Python and so we have to come out with another symbol when aligning this feature in the Python Table API. I noticed that there are

Re: [VOTE] Release Flink Python API(PyFlink) 1.9.2 to PyPI, release candidate #1

2020-02-10 Thread Dian Fu
+1 (non-binding) - Verified the signature and checksum - Pip installed the package successfully: pip install apache-flink-1.9.2.tar.gz - Run word count example successfully. Regards, Dian > 在 2020年2月11日,上午11:44,jincheng sun 写道: > > > +1 (binding) > > - Install the PyFlink by `pip install`

Re: [DISCUSS] Support scalar vectorized Python UDF in PyFlink

2020-02-10 Thread Dian Fu
lize that "batch.size" should be a larger value, so "bundle.size" > must also be increased. Now the default value of "bundle.size" is only 1000. > I think you can update design to provide meaningful default value for > "batch.size" and "bundle.si

[jira] [Created] (FLINK-15974) Support to use the Python UDF directly in the Python Table API

2020-02-10 Thread Dian Fu (Jira)
Dian Fu created FLINK-15974: --- Summary: Support to use the Python UDF directly in the Python Table API Key: FLINK-15974 URL: https://issues.apache.org/jira/browse/FLINK-15974 Project: Flink Issue

[jira] [Created] (FLINK-15973) Optimize the execution plan where it refers the Python UDF result field in the where clause

2020-02-10 Thread Dian Fu (Jira)
Dian Fu created FLINK-15973: --- Summary: Optimize the execution plan where it refers the Python UDF result field in the where clause Key: FLINK-15973 URL: https://issues.apache.org/jira/browse/FLINK-15973

[jira] [Created] (FLINK-15971) Adjust the default value of bundle size and bundle time

2020-02-10 Thread Dian Fu (Jira)
Dian Fu created FLINK-15971: --- Summary: Adjust the default value of bundle size and bundle time Key: FLINK-15971 URL: https://issues.apache.org/jira/browse/FLINK-15971 Project: Flink Issue Type

[jira] [Created] (FLINK-15970) Optimize the Python UDF execution to only serialize the value

2020-02-10 Thread Dian Fu (Jira)
Dian Fu created FLINK-15970: --- Summary: Optimize the Python UDF execution to only serialize the value Key: FLINK-15970 URL: https://issues.apache.org/jira/browse/FLINK-15970 Project: Flink Issue

Re: [DISCUSS] Support scalar vectorized Python UDF in PyFlink

2020-02-10 Thread Dian Fu
Hi Jincheng, Hequn & Jingsong, Thanks a lot for your suggestions. I have created FLIP-97[1] for this feature. > One little suggestion: maybe it would be nice if we can add some performance explanation in the document? (I just very curious:)) Thanks for the suggestion. I have updated the design

Re: [DISCUSS] Support Python ML Pipeline API

2020-02-09 Thread Dian Fu
Hi Hequn, Thanks for bringing up the discussion. +1 to this feature. The design LGTM. It's great that the Python ML users could use both the Java Pipeline Transformer/Estimator/Model classes and the Python Pipeline Transformer/Estimator/Model in the same job. Regards, Dian On Mon, Feb 10, 2020

[jira] [Created] (FLINK-15929) test_dependency failed on travis

2020-02-05 Thread Dian Fu (Jira)
Dian Fu created FLINK-15929: --- Summary: test_dependency failed on travis Key: FLINK-15929 URL: https://issues.apache.org/jira/browse/FLINK-15929 Project: Flink Issue Type: Test Components

[jira] [Created] (FLINK-15902) Improve the Python API doc of the version they are added

2020-02-04 Thread Dian Fu (Jira)
Dian Fu created FLINK-15902: --- Summary: Improve the Python API doc of the version they are added Key: FLINK-15902 URL: https://issues.apache.org/jira/browse/FLINK-15902 Project: Flink Issue Type

[jira] [Created] (FLINK-15897) Deserialize the Python UDF execution results when sending them out

2020-02-04 Thread Dian Fu (Jira)
Dian Fu created FLINK-15897: --- Summary: Deserialize the Python UDF execution results when sending them out Key: FLINK-15897 URL: https://issues.apache.org/jira/browse/FLINK-15897 Project: Flink

[jira] [Created] (FLINK-15875) Bump Beam to 2.19.0

2020-02-03 Thread Dian Fu (Jira)
Dian Fu created FLINK-15875: --- Summary: Bump Beam to 2.19.0 Key: FLINK-15875 URL: https://issues.apache.org/jira/browse/FLINK-15875 Project: Flink Issue Type: Improvement Components: API

Re: [ANNOUNCE] Yu Li became a Flink committer

2020-01-23 Thread Dian Fu
Congrats Yu! > 在 2020年1月23日,下午6:47,Hequn Cheng 写道: > > Congratulations Yu! > Thanks a lot for being the release manager of the big 1.10 release. You are > doing a very good job! > > Best, Hequn > > > On Thu, Jan 23, 2020 at 6:29 PM Jingsong Li wrote: > >> Congratulations Yu, well deserved!

Re: [ANNOUNCE] Dian Fu becomes a Flink committer

2020-01-17 Thread Dian Fu
rote: >> >>> Congratulations Dian, well deserved! >>> >>> On Thu, Jan 16, 2020 at 5:58 PM jincheng sun >>> wrote: >>> >>>> Hi everyone, >>>> >>>> I'm very happy to announce that Dian accepted the offer of th

[jira] [Created] (FLINK-15493) FlinkKafkaInternalProducerITCase.testProducerWhenCommitEmptyPartitionsToOutdatedTxnCoordinator failed on travis

2020-01-06 Thread Dian Fu (Jira)
Dian Fu created FLINK-15493: --- Summary: FlinkKafkaInternalProducerITCase.testProducerWhenCommitEmptyPartitionsToOutdatedTxnCoordinator failed on travis Key: FLINK-15493 URL: https://issues.apache.org/jira/browse/FLINK

Re: [DISCUSS] Set default planner for SQL Client to Blink planner in 1.10 release

2020-01-05 Thread Dian Fu
+1 to set blink planner as the default planner for SQL client considering that so many features added since 1.10 are only available in the blink planner. > 在 2020年1月6日,上午11:04,Rui Li 写道: > > +1. I think it improves user experience. > > On Mon, Jan 6, 2020 at 10:18 AM Zhenghua Gao

Re: [DISCUSS] FLIP-85: Delayed Job Graph Generation

2019-12-29 Thread Dian Fu
Hi all, Sorry to jump into this discussion. Thanks everyone for the discussion. I'm very interested in this topic although I'm not an expert in this part. So I'm glad to share my thoughts as following: 1) It's better to have a whole design for this feature As we know, there are two deployment

[jira] [Created] (FLINK-15414) KafkaITCase#prepare failed in travis

2019-12-26 Thread Dian Fu (Jira)
Dian Fu created FLINK-15414: --- Summary: KafkaITCase#prepare failed in travis Key: FLINK-15414 URL: https://issues.apache.org/jira/browse/FLINK-15414 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-15413) ScalarOperatorsTest failed in travis

2019-12-26 Thread Dian Fu (Jira)
Dian Fu created FLINK-15413: --- Summary: ScalarOperatorsTest failed in travis Key: FLINK-15413 URL: https://issues.apache.org/jira/browse/FLINK-15413 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-15412) LocalExecutorITCase#testParameterizedTypes failed in travis

2019-12-26 Thread Dian Fu (Jira)
Dian Fu created FLINK-15412: --- Summary: LocalExecutorITCase#testParameterizedTypes failed in travis Key: FLINK-15412 URL: https://issues.apache.org/jira/browse/FLINK-15412 Project: Flink Issue

[jira] [Created] (FLINK-15271) Add documentation about the Python environment requirements

2019-12-15 Thread Dian Fu (Jira)
Dian Fu created FLINK-15271: --- Summary: Add documentation about the Python environment requirements Key: FLINK-15271 URL: https://issues.apache.org/jira/browse/FLINK-15271 Project: Flink Issue

[jira] [Created] (FLINK-15270) Add documentation about how to specify third-party dependencies via API for Python UDFs

2019-12-15 Thread Dian Fu (Jira)
Dian Fu created FLINK-15270: --- Summary: Add documentation about how to specify third-party dependencies via API for Python UDFs Key: FLINK-15270 URL: https://issues.apache.org/jira/browse/FLINK-15270

Re: [ANNOUNCE] Zhu Zhu becomes a Flink committer

2019-12-15 Thread Dian Fu
Congrats Zhu Zhu! > 在 2019年12月15日,下午6:23,Zhu Zhu 写道: > > Thanks everyone for the warm welcome! > It's my honor and pleasure to improve Flink with all of you in the > community! > > Thanks, > Zhu Zhu > > Benchao Li 于2019年12月15日周日 下午3:54写道: > >> Congratulations!:) >> >> Hequn Cheng

Re: [ANNOUNCE] Apache Flink 1.8.3 released

2019-12-11 Thread Dian Fu
Thanks Hequn for being the release manager and everyone who contributed to this release. Regards, Dian > 在 2019年12月12日,下午2:24,Hequn Cheng 写道: > > Hi, > > The Apache Flink community is very happy to announce the release of Apache > Flink 1.8.3, which is the third bugfix release for the

Re: [ANNOUNCE] Feature freeze for Apache Flink 1.10.0 release

2019-12-10 Thread Dian Fu
Hi Yu & Gary, Thanks for your great work. Looking forward to the 1.10 release. Regards, Dian > 在 2019年12月11日,上午10:29,Danny Chan 写道: > > Hi Yu & Gary, > > Thanks for your nice job ~ > > Best, > Danny Chan > 在 2019年11月19日 +0800 PM11:44,dev@flink.apache.org,写道: >> >> Hi Yu & Gary, >> >>

Re: [VOTE] Release 1.8.3, release candidate #3

2019-12-10 Thread Dian Fu
+1 (non-binding) - verified signatures and checksums - build from source without tests with Scala 2.11 and Scala 2.12 - start standalone cluster and web ui is accessible - submit word count example of both batch and stream, there is no suspicious log output - run a couple of tests in IDE -

[RESULT] [VOTE] FLIP-88: PyFlink User-Defined Function Resource Management

2019-12-05 Thread Dian Fu
Hi everyone, Thanks for the discussion and votes. So far we have received 4 approving votes, 3 of which are binding and there is no -1 votes: * Jincheng (binding) * Hequn (binding) * Jark (binding) * Jingsong (non-binding) Therefore, I'm happy to announce that FLIP-88 has been accepted.

Re: [DISCUSS] PyFlink User-Defined Function Resource Management

2019-12-05 Thread Dian Fu
e/FLINK-15035 > > Best, > Jingsong Lee > > On Tue, Dec 3, 2019 at 6:07 PM Dian Fu wrote: > >> Hi Jingsong, >> >> Thanks for your valuable feedback. I have updated the "Example" section >> describing how to use these options in a Python Table AP

Re: [VOTE] Improve the Pyflink command line options (Adjustment to FLIP-78)

2019-12-04 Thread Dian Fu
+1 (non-binding) Regards, Dian > 在 2019年12月5日,上午11:11,Jark Wu 写道: > > +1 (binding) > > Best, > Jark > > On Thu, 5 Dec 2019 at 10:45, Wei Zhong wrote: > >> Hi all, >> >> According to our previous discussion in [1], I'd like to bring up a vote >> to apply the adjustment [2] to the

Re: [DISCUSS] Drop Kafka 0.8/0.9

2019-12-04 Thread Dian Fu
+1 for dropping them. Just FYI: there was a similar discussion few months ago [1]. [1] http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/DISCUSS-Drop-older-versions-of-Kafka-Connectors-0-9-0-10-for-Flink-1-10-td29916.html#a29997

Re: [DISCUSS] Improve the Pyflink command line options (Adjustment to FLIP-78)

2019-12-04 Thread Dian Fu
Thanks for bringing up this discussion Wei. +1 for this proposal! As these options are proposed in 1.10, it will be great if we can improve them in 1.10. Then it will not cause compatible issues. Thanks, Dian > 在 2019年12月5日,上午10:01,jincheng sun 写道: > > Hi all, > > Thanks for the quick

Re: [DISCUSS] Voting from apache.org addresses

2019-12-04 Thread Dian Fu
>> I don't want to be very strict, but I think it should be counted as >>> non-binding, if it comes from non apache address, yes. >>> >>> Anybody should be able to verify a vote. Moreover I think this the only >>> way to "encourage" all committers t

Re: [DISCUSS] Voting from apache.org addresses

2019-12-04 Thread Dian Fu
ange anything for non-committers/non-PMC. However if you >>> are a committer and vote from non apache address your vote will be >>> non-binding, because we cannot verify you are indeed a committer (we might >>> don't know your other address). >>> >&g

Re: [DISCUSS] Voting from apache.org addresses

2019-12-03 Thread Dian Fu
Thanks Dawid for start this discussion. I have the same feeling with Xuefu and Jingsong. Besides that, according to the bylaws, for some kinds of votes, only the votes from active PMC members are binding, such as product release. So an email address doesn't help here. Even if a vote is from a

Re: [VOTE] Setup a secur...@flink.apache.org mailing list

2019-12-03 Thread Dian Fu
gt; project website. I think many people are not aware of the > secur...@apache.org address. If they failed to find information in the > Flink site, they will simply assume there is no special procedure for > security problems. > > Thanks, > > Jiangjie (Becket) Qin > >

Re: [DISCUSS] Expose or setup a secur...@flink.apache.org mailing list for security report and discussion

2019-12-03 Thread Dian Fu
-apache-org-mailing-list-tt35205.html [2] https://github.com/apache/flink-web/pull/287 On Thu, Nov 21, 2019 at 3:58 PM Dian Fu wrote: > Hi all, > > There are no new feedbacks and it seems that we have received enough > feedback about setup a secur...@flink.apache.org mailing list[1] fo

[jira] [Created] (FLINK-15046) Add guideline on how to report security issues

2019-12-03 Thread Dian Fu (Jira)
Dian Fu created FLINK-15046: --- Summary: Add guideline on how to report security issues Key: FLINK-15046 URL: https://issues.apache.org/jira/browse/FLINK-15046 Project: Flink Issue Type: Improvement

Re: [DISCUSS] Drop RequiredParameters and OptionType

2019-12-03 Thread Dian Fu
+1 to remove them. It seems that we should also drop the class Option as it's currently only used in RequiredParameters. > 在 2019年12月3日,下午8:34,Robert Metzger 写道: > > +1 on removing it. > > On Tue, Dec 3, 2019 at 12:31 PM Stephan Ewen > wrote: > I just stumbled across

Re: [DISCUSS] PyFlink User-Defined Function Resource Management

2019-12-03 Thread Dian Fu
doc: >> The memory used by the Java operator is currently accounted as the task > on-heap memory. We can revisit this if we find it's a problem in the future. > I agree that we can ignore it now, But we can add some content to the > document to remind the user, What do you think? >

Re: [VOTE] Setup a secur...@flink.apache.org mailing list

2019-12-03 Thread Dian Fu
evidence that such a mailing list is necessary. Thanks, Dian > 在 2019年12月3日,下午4:53,Dian Fu 写道: > > Actually I have tried to find out the reason why so many apache projects > choose to set up a project specific security mailing list in case that the > general secur...@apache.o

Re: [VOTE] Setup a secur...@flink.apache.org mailing list

2019-12-03 Thread Dian Fu
t;> Thanks for driving this, Dian. >> >> +1 from me, for the reasons I mentioned in the discussion thread. >> >> On Tue, Nov 26, 2019 at 12:08 PM Dian Fu wrote: >> >>> NOTE: Only PMC votes is binding. >>> >>> Thanks for sharing your t

Re: [VOTE] FLIP-88: PyFlink User-Defined Function Resource Management

2019-12-02 Thread Dian Fu
g, and set resources to OneInputTransformation? >> - Are all buffer.memory off-heap memory? I took a look >> to AbstractPythonScalarFunctionOperator, there is a forwardedInputQueue, is >> this one a heap queue? So we need heap memory too? >> >> Hope to get your rep

Re: [DISCUSS] PyFlink User-Defined Function Resource Management

2019-12-02 Thread Dian Fu
wardedInputQueue is a buffer used in the Java operator and its memory is accounted as the on-heap memory. Regards, Dian > > Hope to get your reply. > > > Best, > > Jingsong Lee > > On Tue, Nov 26, 2019 at 12:17 PM Dian Fu wrote: > >> Thanks for your

[VOTE] FLIP-88: PyFlink User-Defined Function Resource Management

2019-12-01 Thread Dian Fu
Hi all, I'd like to start the vote of FLIP-88 [1] since that we have reached an agreement on the design in the discussion thread [2]. This vote will be open for at least 72 hours. Unless there is an objection, I will try to close it by Dec 5, 2019 08:00 UTC if we have received sufficient

Re: [VOTE] Release 1.8.3, release candidate #1

2019-11-29 Thread Dian Fu
ssed the > JIRA problems and updated the website PR. > It would be great if you can take another look. > > Best, > Hequn > > On Fri, Nov 29, 2019 at 2:21 PM Dian Fu wrote: > >> Hi Hequn, >> >> Thanks a lot for the great work! >> >> I found the

Re: [VOTE] Release 1.8.3, release candidate #1

2019-11-28 Thread Dian Fu
Hi Hequn, Thanks a lot for the great work! I found the following minor issues: 1. The announcement PR for website has one minor issue. (Have left comments on the PR) 2. The following JIRAs are included 1.8.3, but the fix version are not updated and so not reflected in the release note:

Re: [DISCUSS] PyFlink User-Defined Function Resource Management

2019-11-25 Thread Dian Fu
ho works >> on >>> this part. >>> >>> Thank you~ >>> >>> Xintong Song >>> >>> >>> [1] >>> >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-53%3A+Fine+Grained+Operator+Resource+Management >>>

Re: [VOTE] Setup a secur...@flink.apache.org mailing list

2019-11-25 Thread Dian Fu
I am not sure if it falls into any of >>> the categories we have listed in our bylaws. Are committers votes >>> binding or just PMCs'? (Personally I think it should be PMCs') Is this a >>> binding vote or just an informational vote? >>> >>> Best, >&g

[jira] [Created] (FLINK-14944) Unstable test FlinkFnExecutionSyncTests.test_flink_fn_execution_pb2_synced

2019-11-25 Thread Dian Fu (Jira)
Dian Fu created FLINK-14944: --- Summary: Unstable test FlinkFnExecutionSyncTests.test_flink_fn_execution_pb2_synced Key: FLINK-14944 URL: https://issues.apache.org/jira/browse/FLINK-14944 Project: Flink

Re: [ANNOUNCE] Apache Flink-shaded 9.0 released

2019-11-24 Thread Dian Fu
Thanks Chesnay for the great work and thanks to everyone who has contributed to this release. Regards, Dian > 在 2019年11月25日,上午10:22,Zhu Zhu 写道: > > Thanks a lot to Chesnay for the great work to release Flink-shaded 9.0! > And thanks for the efforts to make this release possible to all the >

Re: [DISCUSS] Remove old WebUI

2019-11-24 Thread Dian Fu
+1 to drop the old UI. > 在 2019年11月25日,上午10:59,Zhenghua Gao 写道: > > +1 to drop the old one. > > *Best Regards,* > *Zhenghua Gao* > > > On Thu, Nov 21, 2019 at 8:05 PM Chesnay Schepler wrote: > >> Hello everyone, >> >> Flink 1.9 shipped with a new UI, with the old one being kept around as

[jira] [Created] (FLINK-14891) PythonScalarFunctionOperator should be chained with upstream operators by default

2019-11-21 Thread Dian Fu (Jira)
Dian Fu created FLINK-14891: --- Summary: PythonScalarFunctionOperator should be chained with upstream operators by default Key: FLINK-14891 URL: https://issues.apache.org/jira/browse/FLINK-14891 Project

[jira] [Created] (FLINK-14888) Move the Python SqlDialect to module table

2019-11-21 Thread Dian Fu (Jira)
Dian Fu created FLINK-14888: --- Summary: Move the Python SqlDialect to module table Key: FLINK-14888 URL: https://issues.apache.org/jira/browse/FLINK-14888 Project: Flink Issue Type: Task

Re: [VOTE] Release flink-shaded 9.0, release candidate #1

2019-11-21 Thread Dian Fu
+1 (non-binding) - verified the signature and checksum - checked the maven central artifices Regards, Dian On Wed, Nov 20, 2019 at 8:47 PM tison wrote: > +1 (non-binding) > > Best, > tison. > > > Aljoscha Krettek 于2019年11月20日周三 下午6:58写道: > > > +1 (binding) > > > > Best, > > Aljoscha > > >

[VOTE] Setup a secur...@flink.apache.org mailing list

2019-11-21 Thread Dian Fu
Hi all, According to our previous discussion in [1], I'd like to bring up a vote to set up a secur...@flink.apache.org mailing list. The vote will be open for at least 72 hours (excluding weekend). I'll try to close it by 2019-11-26 18:00 UTC, unless there is an objection or not enough votes.

Re: [DISCUSS] Expose or setup a secur...@flink.apache.org mailing list for security report and discussion

2019-11-20 Thread Dian Fu
org> or secur...@apache.org. So I'd like to start the vote for setup a secur...@flink.apache.org mailing list to make the final decision. Thanks, Dian > 在 2019年11月19日,下午6:06,Dian Fu 写道: > > Hi all, > > Thanks for sharing your thoughts. Appreciated! Let me try to summarize the >

[jira] [Created] (FLINK-14866) A few documentation links are broken

2019-11-19 Thread Dian Fu (Jira)
Dian Fu created FLINK-14866: --- Summary: A few documentation links are broken Key: FLINK-14866 URL: https://issues.apache.org/jira/browse/FLINK-14866 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-14865) PyFlinkBlinkStreamUserDefinedFunctionTests#test_udf_in_join_condition_2 failed in travis

2019-11-19 Thread Dian Fu (Jira)
Dian Fu created FLINK-14865: --- Summary: PyFlinkBlinkStreamUserDefinedFunctionTests#test_udf_in_join_condition_2 failed in travis Key: FLINK-14865 URL: https://issues.apache.org/jira/browse/FLINK-14865

Re: [VOTE] FLIP-83: Flink End-to-end Performance Testing Framework

2019-11-19 Thread Dian Fu
+1 (non-binding) Regards, Dian > 在 2019年11月19日,下午6:31,Piotr Nowojski 写道: > > +1 (non-binding) > > Piotrek > >> On 19 Nov 2019, at 04:20, Yang Wang wrote: >> >> +1 (non-binding) >> >> It is great to have a new end-to-end test framework, even it is only for >> performance tests now. >> >>

Re: [DISCUSS] Expose or setup a secur...@flink.apache.org mailing list for security report and discussion

2019-11-19 Thread Dian Fu
However, AFAIK, Flink doesn't have an official process to > > > >> report vulnerabilities. > > > >> It would be nice to have one to protect Flink users and response > > > >> security > > > >> problems quickly. > > &

Re: [DISCUSS] Release flink-shaded 9.0

2019-11-19 Thread Dian Fu
teps up I'll kick it off later today myself; this > would mean a release on Friday. > > On 19/11/2019 09:17, Dian Fu wrote: > > Hi Chesnay, > > > > Thanks a lot for kicking off this release. +1 to release flink-shaded > 9.0. > > > > I'm willing t

<    1   2   3   4   5   6   7   8   >