Re: [DISCUSS] Release Flink 1.5.1

Hi Chesnay, I made FLINK-9554 as blocker of 1.5.1, as scala shell still doesn't work in yarn mode in the latest master branch. (I believe it is the same in branch-1.5) Zhijiang(wangzhijiang999) 于2018年7月3日周二 下午12:04写道： > Hi Chesnay, > > Agree

Re: [VOTE] Release 1.5.1, release candidate #2

+1. Build form source successfully. Tested scala-shell in local and yarn mode, works well. Till Rohrmann 于2018年7月10日周二 下午3:37写道： > +1 (binding) > > - Verified that no new dependencies were added for which the LICENSE and > NOTICE files need to be adapted. > - Build 1.5.1 from the source

Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard

+1 for the proposal of Yadong. I'd like to add one more comment: We should provide more development guidance when developing the next dashboard via angular 7. As Till mentioned before, one big concern is lack of frontend guys in flink community, I believe more development guidance will attract

Re: [DISCUSS] Task speculative execution for Flink batch

+1 for the speculative execution for Flink batch, Speculative execution is used in lots of batch execution engine like mr, tez and spark. This would be a great improvement for Flink in batch scenario. Jin Sun 于2018年11月7日周三 上午8:38写道： > I think this is target for batch at the very beginning, the

Why UDF can not be non-static inner class ?

can not be non-static inner class ? -- Best Regards Jeff Zhang

Re: [DISCUSS] Towards a leaner flink-dist

be >> extended to cover flink-dist. For example, the yarn and mesos code could >> be spliced out into separate jars that could be added to lib manually. >> >> Let me know what you think. >> >> Regards, >> >> Chesnay >> >> -- Best Regards Jeff Zhang

Re: [DISCUSS] Flink client api enhancement for downstream project

taking a savepoint, cancelling the job). > > [1] https://issues.apache.org/jira/browse/FLINK-4272 > > Cheers, > Till > > On Tue, Dec 11, 2018 at 10:13 AM Jeff Zhang wrote: > > > Hi Folks, > > > > I am trying to integrate flink into apache zeppelin which i

Re: [DISCUSS] Releasing Flink 1.6.3

Problem with restoring FlinkKafkaProducer: FLINK-10353 > > > > LockableTypeSerializer duplication problem: FLINK-10816 > > > > > > > > Another issue where the fix isn't merged yet, but we should probably > > > > include is FLINK-11083 (problem with restoring table state, PR is > > > > available). > > > > > > > > What do you think? > > > > > > > > Cheers, > > > > Gordon > > > > > > > > [1] > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Creating-last-bug-fix-release-for-1-5-branch-td25794.html > > > > > > > > > > -- Best Regards Jeff Zhang

Re: [DISCUSS] Flink client api enhancement for downstream project

> ExecutionEnvironment should give you access to the ClusterClient and to the > job (maybe in the form of the JobGraph or a job plan). > > Cheers, > Till > > On Thu, Dec 13, 2018 at 4:36 AM Jeff Zhang wrote: > > > Hi Till, > > Thanks for the feedback. You are right tha

Re: [ANNOUNCE] Apache Flink 1.5.6 released

; Please check out the release blog post for an overview of the >>> improvements >>> for this bugfix release: >>> https://flink.apache.org/news/2018/12/22/release-1.5.6.html >>> >>> The full release notes are available in Jira: >>> >>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12344315 >>> >>> We would like to thank all contributors of the Apache Flink community who >>> made this release possible! >>> >>> Regards, >>> Thomas >>> >> -- Best Regards Jeff Zhang

Re: [DISCUSS] Support Interactive Programming in Flink Table API

Hi Becket, Introducing CacheHandle seems too complicated. That means users have to maintain Handler properly. And since cache is just a hint for optimizer, why not just return Table itself for cache method. This hint info should be kept in Table I believe. So how about adding method cache and

[DISCUSS] Flink client api enhancement for downstream project

fer it for future release, 1,2,3,4,5 could be done at once I believe. Let me know your comments and feedback, thanks -- Best Regards Jeff Zhang

Re: [DISCUSS] Creating last bug fix release for 1.5 branch

-1.5 branch contains 45 unreleased fixes. Some of the fixes > address > > >> serializer duplication problems (FLINK-10839, FLINK-10693), fixing > > >> retractions (FLINK-10674) or prevent a deadlock in the > > >> SpillableSubpartition (FLINK-10491). I think it would be nice for our > users > > >> if we officially terminated the Flink 1.5.x support with a last 1.5.6 > > >> release. What do you think? > > >> > > >> Cheers, > > >> Till > > >> > > > -- Best Regards Jeff Zhang

Re: [DISCUSS] FLIP-38 Support python language in flink TableAPI

cument for FLIP-38 can be found here: > > > https://docs.google.com/document/d/1ybYt-0xWRMa1Yf5VsuqGRtOfJBz4p74ZmDxZYg3j_h8/edit?usp=sharing > > I am looking forward to your comments and feedback. > > Best, > Jincheng > -- Best Regards Jeff Zhang

Re: [DISCUSS] Python (and Non-JVM) Language Support in Flink

; > > [1]. > > > > > > Some > > > > > > >> discussion was also brought up there regarding non-jvm > language > > > > > support > > > > > > >> strategy in general. To avoid polluting the survey thread, we > > are > > > > > > starting > > > > > > >> this discussion thread and would like to move the discussions > > > here. > > > > > > >> > > > > > > >> In the interest of facilitating the discussion, we would like > to > > > > first > > > > > > >> share the following design doc which describes what we have > done > > > at > > > > > > Alibaba > > > > > > >> about Python API for Flink. It could serve as a good reference > > to > > > > the > > > > > > >> discussion. > > > > > > >> > > > > > > >> [DISCUSS] Flink Python API > > > > > > >> < > > > > > > >> > > > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1JNGWdLwbo_btq9RVrc1PjWJV3lYUgPvK0uEWDIfVNJI/edit?usp=drive_web > > > > > > >>> > > > > > > >> > > > > > > >> As of now, we've implemented and delivered Python UDF for SQL > > for > > > > the > > > > > > >> internal users at Alibaba. > > > > > > >> We are starting to implement Python API. > > > > > > >> > > > > > > >> To recap and continue the discussion from the survey thread, I > > > agree > > > > > > with > > > > > > >> @Stephan that we should figure out in which general direction > > > Python > > > > > > >> support should go. Stephan also list three options there: > > > > > > >> * Option (1): Language portability via Apache Beam > > > > > > >> * Option (2): Implement own Python API > > > > > > >> * Option (3): Implement own portability layer > > > > > > >> > > > > > > >> From my perspective, > > > > > > >> (1). Flink language APIs and Beam's languages support are not > > > > mutually > > > > > > >> exclusive. > > > > > > >> It is nice that Beam has Python/NodeJS/Go APIs, and support > > Flink > > > as > > > > > the > > > > > > >> runner. > > > > > > >> Flink's own Python(or NodeJS/Go) APIs will benefit Flink's > > > > ecosystem. > > > > > > >> > > > > > > >> (2). Python API / portability layer > > > > > > >> To support non-JVM languages in Flink, > > > > > > >> * at client side, Flink would provide language interfaces, > > which > > > > > will > > > > > > >> translate user's application to Flink StreamGraph. > > > > > > >> * at server side, Flink would execute user's UDF code at > runtime > > > > > > >> The non-JVM languages communicate with JVM via RPC(or > low-level > > > > > socket, > > > > > > >> embedded interpreter and so on). What the portability layer > can > > do > > > > > > maybe is > > > > > > >> abstracting the RPC layer. When the portability layer is > ready, > > > > still > > > > > > there > > > > > > >> are lots of stuff to do for a specified language. Say, Python, > > we > > > > may > > > > > > still > > > > > > >> have to write the interface classes by hand for the users > > because > > > > > > generated > > > > > > >> code without detailed documentation is unacceptable for users, > > or > > > > > handle > > > > > > >> the serialization issue of lambda/closure which is not a > > built-in > > > > > > feature > > > > > > >> in Python. Maybe, we can start with Python API, then extend > to > > > > other > > > > > > >> languages and abstract the logic in common as the portability > > > layer. > > > > > > >> > > > > > > >> --- > > > > > > >> References: > > > > > > >> [1] [SURVEY] Usage of flink-python and flink-streaming-python > > > > > > >> > > > > > > >> Regards, > > > > > > >> Xianda > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- Best Regards Jeff Zhang

Re: [DISCUSS] Move flink-python to opt

the Python batch API. I think this > will make the streaming/batch experience more consistent and would > result in a cleaner default classpath. > > Best, > > Ufuk > -- Best Regards Jeff Zhang

Re: [ANNOUNCE] Apache Flink-shaded 6.0 released

t; who made this release possible! > > Regards, > Chesnay > > -- Best Regards Jeff Zhang

Re: [DISCUSS] Adding a mid-term roadmap to the Flink website

;> breaking) >>>> - Add fine fault tolerance, scheduling, caching also to DataStream API >>>> >>>> *Streaming State Evolution* >>>> - Let all built-in serializers support stable evolution >>>> - First class support for other evolvable formats (Protobuf, Thrift) >>>> - Savepoint input/output format to modify / adjust savepoints >>>> >>>> *Simpler Event Time Handling* >>>> - Event Time Alignment in Sources >>>> - Simpler out-of-the box support in sources >>>> >>>> *Checkpointing* >>>> - Consistency of Side Effects: suspend / end with savepoint (FLIP-34) >>>> - Failed checkpoints explicitly aborted on TaskManagers (not only on >>>> coordinator) >>>> >>>> *Automatic scaling (adjusting parallelism)* >>>> - Reactive scaling >>>> - Active scaling policies >>>> >>>> *Kubernetes Integration* >>>> - Active Kubernetes Integration (Flink actively manages containers) >>>> >>>> *SQL Ecosystem* >>>> - Extended Metadata Stores / Catalog / Schema Registries support >>>> - DDL support >>>> - Integration with Hive Ecosystem >>>> >>>> *Simpler Handling of Dependencies* >>>> - Scala in the APIs, but not in the core (hide in separate class >>>> loader) >>>> - Hadoop-free by default >>>> >>>> -- Best Regards Jeff Zhang

Re: [DISCUSS] Flink client api enhancement for downstream project

_job_api.proto > [3] > > https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/PipelineResult.java > [4] > > https://lists.apache.org/thread.html/064c75c5d10f0806095b14f6d76942598917a14429c1acbddd151fe2@%3Cdev.flink.apache.org%3E > >

Re: [ANNOUNCE] New Flink PMC member Thomas Weise

participating in lots of discussions on our mailing > lists, working on topics that are of joint interest of Flink and Beam, and > giving talks on Flink at many events. > > Please join me in welcoming and congratulating Thomas! > > Best, > Fabian > -- Best Regards Jeff Zhang

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

ow them to > follow > >>>>> the > >>>>>> Apache way. > >>>>>> > >>>>>> Is it possible? > >>>>>> I believe it is, because a number of other Apache projects are > running > >>>>>> non-English user@ mailing lists. > >>>>>> Apache OpenOffice, Cocoon, OpenMeetings, CloudStack all have > >>>> non-English > >>>>>> lists: http://mail-archives.apache.org/mod_mbox/ > >>>>>> One thing I want to make very clear in this discussion is that all > >>>>> project > >>>>>> decisions, developer discussions, JIRA tickets etc. need to happen > in > >>>>>> English, as this is the primary language of the Apache Foundation > and > >>>> our > >>>>>> community. > >>>>>> We should also clarify this on the page listing the mailing lists. > >>>>>> > >>>>>> How? > >>>>>> If there is consensus in this discussion thread, I would request the > >>>> new > >>>>>> mailing list next Monday. > >>>>>> In case of discussions, I will start a vote on Monday or when the > >>>>>> discussions have stopped. > >>>>>> Then, we should put the new list on our website and start promoting > it > >>>>> (in > >>>>>> said DingTalk group and on social media). > >>>>>> > >>>>>> Let me know what you think about this idea :) > >>>>>> > >>>>>> Best, > >>>>>> Robert > >>>>>> > >>>>>> > >>>>>> PS: In case you are wondering what ZH stands for: > >>>>>> https://en.wiktionary.org/wiki/ZH > >>>>>> > >>>>> > >>>> > >> > >> -- > > Best, > > Kurt > > -- Best Regards Jeff Zhang

Re: [DISCUSS] Drop Elasticssearch 1 connector

n't seen any development for over a > > hear, yet still incurred maintenance overhead due to licensing and > testing. > > > > > > > -- Best Regards Jeff Zhang

[Discuss] Add JobListener (hook) in flink job lifecycle

obId, String savepointPath); } Let me know your comment and concern, thanks. -- Best Regards Jeff Zhang

Re: [DISCUSS] Flink client api enhancement for downstream project

s out a bit more. > > > On Tue, Feb 12, 2019 at 4:58 AM Jeff Zhang wrote: > > > Hi folks, > > > > Sorry for late response, It seems we reach consensus on this, I will > create > > FLIP for this with more detailed design > > > > > &

Re: What does flink session mean ?

l > > On Sun, Jun 2, 2019 at 3:20 PM Jeff Zhang wrote: > >> >> Hi Folks, >> >> >> When I read the flink client api code, the concept of session is a little >> vague and unclear to me. It looks like the session concept is only applied >> in batch

What does flink session mean ?

this ? Thanks. -- Best Regards Jeff Zhang

Re: [DISCUSS] Adopting a Code Style and Quality Guide

t; > > > > > > > This guide is not fix and final, we should discuss it and expand and > > adjust > > > it over time. The guide tries to strike a balance between too much > > contents > > > (don't force someone to read a book before contributing) and being > > > comprehensive enough. The guide looks long, but much of it are > component > > or > > > aspect specific parts, like how to deal with new dependencies, > > > configuration changes, etc. > > > > > > I an curious to hear whether you think such a guide is in principle a > > good > > > idea. > > > If yes, is the one here a good starting point? > > > > > > Best, > > > Stephan > > > > > > -- Best Regards Jeff Zhang

Re: [DISCUSS] Flink client api enhancement for downstream project

annot be exposed to users because it uses > quite some classes from the optimiser and runtime packages. > > * What happens when a failure/restart in the client happens? There need to > be a way of re-establishing the connection to the job, set up the listeners > again, etc. > >

Re: [DISCUSS] Deprecate previous Python APIs

n multiple different implementations and > confuse users that way. > Given that the existing Python APIs are a bit limited and not under active > development, I would suggest to deprecate them in favor of the new API. > > Best, > Stephan > > -- Best Regards Jeff Zhang

Re: Something wrong with travis?

flink dir /home/travis/flink_cache/x/flink does not exist. > > Exiting build." > > > > Anyone have a clue on what happened and how to fix this? > > > > Best, > > Kurt > > > > -- Best Regards Jeff Zhang

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

ncheng has been a committer since July 2017. He has been very active > on > > > Flink's Table API / SQL component, as well as helping with releases. > > > > > > Congratulations & Welcome Jincheng! > > > > > > Best, > > > Robert > > > > > > -- Best Regards Jeff Zhang

Re: [DISCUSS] solve unstable build capacity problem on TravisCI

>> > > > > > >> > > > > So my questions are: > >> > > > > > >> > > > > - Has anyone else experienced the same problem or have similar > >> > > > observation > >> > > > > on TravisCI? (I suspect it has things to do with time zone) > >> > > > > > >> > > > > - What pricing plan of TravisCI is Flink currently using? Is it > >> the > >> > > free > >> > > > > plan for open source projects? What are the guaranteed build > >> capacity > >> > > of > >> > > > > the current plan? > >> > > > > > >> > > > > - If the current pricing plan (either free or paid) can't > provide > >> > > stable > >> > > > > build capacity, can we upgrade to a higher priced plan with > larger > >> > and > >> > > > more > >> > > > > stable build capacity? > >> > > > > > >> > > > > BTW, another factor that contribute to the productivity problem > is > >> > that > >> > > > > our build is slow - we run full build for every PR and a > >> successful > >> > > full > >> > > > > build takes ~5h. We definitely have more options to solve it, > for > >> > > > instance, > >> > > > > modularize the build graphs and reuse artifacts from the > previous > >> > > build. > >> > > > > But I think that can be a big effort which is much harder to > >> > accomplish > >> > > > in > >> > > > > a short period of time and may deserve its own separate > >> discussion. > >> > > > > > >> > > > > [1] https://travis-ci.org/apache/flink/pull_requests > >> > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > > > -- Best Regards Jeff Zhang

Re: [ANNOUNCE] Feature freeze for Apache Flink 1.9.0 release

> for 1.9 will be cut following that date. > > >> > > >> We’re roughly one week away from this date, but please keep in mind > > that we > > >> still shouldn’t rush things. If you feel that there may be problems > with > > >> this schedule for the things you are working on, please let us know > > here. > > >> > > >> Cheers, > > >> Gordon > > >> > > >> [1] > > >> > > >> > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Features-for-Apache-Flink-1-9-0-td28701.html > > >> > > > > > -- Best Regards Jeff Zhang

Re: [DISCUSS] Flink client api enhancement for downstream project

he design document. >>> >>> We are also working on refactoring ClusterClient to allow flexible and >>> efficient job management in our real-time platform. >>> We would like to draft a document to share our ideas with you. >>> >>> I thin

Re: [DISCUSS] Update our Roadmap

jincheng(cc) is driving the python effort, I think he can help to prepare it. Fabian Hueske 于2019年4月29日周一 下午10:15写道： > Hi everyone, > > Since we had no more comments on this thread, I think we proceed to update > the roadmap. > > @Jeff Zhang I agree, we should add

Re: [Discuss] Add JobListener (hook) in flink job lifecycle

he case, the Flink job program is embedded into the Kylin's >> executable context. >> >> If we could have this listener, it would be easier to integrate with >> Kylin. >> >> Best, >> Vino >> >> Jeff Zhang 于2019年4月18日周四 下午1:30写道： >> &

Re: [ANNOUNCE] Rong Rong becomes a Flink committer

e Flink project. >> >> Rong has been contributing to Flink for many years, mainly working on SQL >> and Yarn security features. He's also frequently helping out on the >> user@f.a.o mailing lists. >> >> Congratulations Rong! >> >> Best, Fabian >> (on behalf of the Flink PMC) >> > -- Best Regards Jeff Zhang

Re: Introducing Flink's Plugin mechanism

f some other potential use cases pop > up in the future, more people will be aware. > > Piotr Nowojski > > > [1] https://issues.apache.org/jira/browse/FLINK-11952 < > https://issues.apache.org/jira/browse/FLINK-11952> -- Best Regards Jeff Zhang

Re: [Discuss] Add JobListener (hook) in flink job lifecycle

e listeners are expected to do anything on the job, should some > helper class to manipulate the jobs be passed to the listener method? > Otherwise users may not be able to easily take action. > > Thanks, > > Jiangjie (Becket) Qin > > > > > On Wed, Apr 24, 2019 at 2:43

Re: SQL CLI and JDBC

> > > > > > > > SELECT colA, COUNT(*) > > > > FROM tab > > > > GROUP BY colA; > > > > > > > > If tab is a continuously growing table, no row of the queries result > > > > will ever be final because a new row with any value of colA can be > > > > added at any point in time. > > > > JDBC does not support to retract or update result rows that were > > > > emitted before. > > > > > > > > Best, Fabian > > > > > > > > > > > > Am So., 7. Apr. 2019 um 11:31 Uhr schrieb Hanan Yehudai < > > > > hanan.yehu...@radcom.com>: > > > > > > > > > I didn’t see any docs on this - is there a JDBC Driver that allows > > > > > the same functionalities as the SQL CLI ? > > > > > If not , is it on the roadmap ? > > > > > > > > > > > > > > > > > > > > -- Best Regards Jeff Zhang

Re: [DISCUSS] Update our Roadmap

ed from the roadmap. > All other issues are still ongoing efforts. > > Are there any other efforts that we want to put on the roadmap? > > Best, Fabian > -- Best Regards Jeff Zhang

Re: [Discuss] Add JobListener (hook) in flink job lifecycle

e Configuration or some > other mechanism for example. That way it would not need to be exposed via > the ExecutionEnvironment at all. > > Cheers, > Till > > On Fri, Apr 19, 2019 at 11:12 AM Jeff Zhang wrote: > >> >>> The ExecutionEnvironment is usually used

Re: [DISCUSS] Vectorization Support in Flink

posal. >> >> >> >> [image: >> https://lh5.googleusercontent.com/hjXkXGImWOjaiB8zF0SKIMoItY6VCBm-BmJWWEXRo0ZPHdwLgKzCmIoNKef1YPCaAA7NXN6RvO-nwBBXBee52KeAtBjyIvh_NcAuChvW3BEtQuZGL5GPddqxL_iMV7HvEVCC6k-m] >> >> >> >> Special thanks to @Kurt Young’s team for all the kind help. >> >> Special thanks to @Piotr Nowojski for all the valuable feedback and help >> suggestions. >> >> >> Best, >> >> Liya Fan >> > -- Best Regards Jeff Zhang

Re: [DISCUSS] solve unstable build capacity problem on TravisCI

;>> merges the branch into master, and then runs it. (Fun fact: This is > why > >>>> a PR without merge conflicts is not being run on Travis.) > >>>> > >>>> And ultimately, everyone can already make use of this approach anyway. >

Re: [DISCUSS] Flink client api enhancement for downstream project

the document[1]. > > > > > > 3. Expose ClusterClient as public api or just expose api in > > > ExecutionEnvironment > > > and delegate them to ClusterClient? Further, in either way is it worth > to > > > introduce a JobClient which is an encapsulation of ClusterClient

Re: [DISCUSS] Best practice to run flink on kubernetes

aming/WindowJoin.jar > > > > [1]. > > https://ci.apache.org/projects/flink/flink-docs-release-1.8/ops/deployment/kubernetes.html > > [2].https://github.com/lyft/flinkk8soperator > > [3]. > > https://docs.google.com/document/d/1Zmhui_29VASPcBOEqyMWnF3L6WEWZ4kedrCqya0WaAk/edit# > -- Best Regards Jeff Zhang

Re: [Discuss] What should the "Data Source" be translated into Chinese

has a good suggestion for "数据汇" which > corresponds to source ("数据源"). I asked Committer Jark Wu, he is also fine > with it. I think "数据汇" is a good representation of flow charactiristics so > I would like to use it. > > > I want to hear more thoughts from the community whether we should > translate it and what it should be translated into. > > > Thanks, > WangHW > -- Best Regards Jeff Zhang

Re: How to handle Flink Job with 400MB+ Uberjar with 800+ containers ?

e(FencedAkkaRpcActor.java:40) >>> at >>> akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165) >>> at akka.actor.Actor$class.aroundReceive(Actor.scala:502) >>> at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95) >>> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526) >>> at akka.actor.ActorCell.invoke(ActorCell.scala:495) >>> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257) >>> at akka.dispatch.Mailbox.run(Mailbox.scala:224) >>> at akka.dispatch.Mailbox.exec(Mailbox.scala:234) >>> at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) >>> at >>> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) >>> at >>> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) >>> at >>> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) >>> >>> >>> >>> -- Best Regards Jeff Zhang

Re: [DISCUSS] Flink client api enhancement for downstream project

> > > > > > > >> > an existing cluster. In this way we are able to > consider > > > > > per-job > > > > > > > > > > issues > > > > > > > > > > >> > individually an

Re: [VOTE] FLIP-58: Flink Python User-Defined Function for Table API

72 hours. Unless there is an > > >>>>>> objection, I will try to close it by Sept 2, 2019 00:00 UTC if we > > have > > >>>>>> received sufficient votes. > > >>>>>> > > >>>>>> PS: This doesn't mean that we cannot further improve the design. > We > > >> can > > >>>>>> still discuss the implementation details case by case in the JIRA > as > > >> long > > >>>>>> as it doesn't affect the overall design. > > >>>>>> > > >>>>>> [1] > > >>>>>> > > >>>>> > > >> > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-58%3A+Flink+Python+User-Defined+Function+for+Table+API > > >>>>>> < > > >>>>>> > > >>>>> > > >> > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-58:+Flink+Python+User-Defined+Function+for+Table+API > > >>>>>>> > > >>>>>> [2] > > >>>>>> > > >>>>> > > >> > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Python-User-Defined-Function-for-Table-API-td31673.html > > >>>>>> < > > >>>>>> > > >>>>> > > >> > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Python-User-Defined-Function-for-Table-API-td31673.html > > >>>>>>> > > >>>>>> > > >>>>>> Thanks, > > >>>>>> Dian > > >>>>> > > >>> > > >> > > >> > > > > > -- Best Regards Jeff Zhang

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

Kloudas is joining the Flink >>> PMC. >>> >> Kostas is contributing to Flink for many years and puts lots of >>> effort in helping our users and growing the Flink community. >>> >> Please join me in congratulating Kostas! >>> > >>> > congratulation Kostas! >>> > >>> > regards. >>> >>> -- Best Regards Jeff Zhang

Re: [DISCUSS] Flink client api enhancement for downstream project

dea Tison! > >>>> > >>>> I will create the umbrella issue and post it here so that we are all > >>>> on the same page! > >>>> > >>>> Cheers, > >>>> Kostas > >>>> > >>>>

Re: [PROPOSAL] Merge NewClusterClient into ClusterClient

who anyway want to program > directly to internal classes/interfaces are considered to prepare to > make adaptations when bump version of Flink. > > Best, > tison. > > [1] https://issues.apache.org/jira/browse/FLINK-14096 > [2] > > https://github.com/apache/flink/commit/dc9e4494dddfed36432e6bbf6cd3231530bc2e01 > -- Best Regards Jeff Zhang

Re: [ANNOUNCE] Hequn becomes a Flink committer

r of the Flink project. > > Hequn has been contributing to Flink for many years, mainly working on > SQL/Table API features. He's also frequently helping out on the user > mailing lists and helping check/vote the release. > > Congratulations Hequn! > > Best, Jincheng > (on behalf of the Flink PMC) > > > -- Best Regards Jeff Zhang

Re: [VOTE] Publish the PyFlink into PyPI

hours. It is adopted by a simple > > > majority with a minimum of three positive votes. > > > > > > See discussion threads for more details [1]. > > > > > > Thanks, > > > Jincheng > > > > > > [1] > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Publish-the-PyFlink-into-PyPI-td30095.html > > > > > -- Best Regards Jeff Zhang

Re: [DISCUSS] Removing the flink-mapr-fs module

it? It would a) solve the dependency problem and > b) make our build a tiny smidgen more lightweight. > > Aljoscha > > [1] > https://lists.apache.org/thread.html/16c16db8f4b94dc47e638f059fd53be936d5da423376a8c1092eaad1@%3Cdev.flink.apache.org%3E -- Best Regards Jeff Zhang

Re: YARN parallelism vs Config

sm. > > Thanks in advance, > Best Regards, > Dom. > -- Best Regards Jeff Zhang

Re: [ANNOUNCE] Kete Young is now part of the Flink PMC

rg，写道： > > > > > > Congratulations Kurt, Well deserved. > > > -- Best Regards Jeff Zhang

Re: [ANNOUNCE] Jiangjie (Becket) Qin has been added as a committer to the Flink project

ark > > > > > > > > On Thu, 18 Jul 2019 at 15:56, Paul Lam > wrote: > > > > > > > > > Congrats Becket! > > > > > > > > > > Best, > > > > > Paul Lam > > > > > > > > > > > 在 2019年7月18日，15:41，Robert Metzger 写道： > > > > > > > > > > > > Hi all, > > > > > > > > > > > > I'm excited to announce that Jiangjie (Becket) Qin just became a > > > Flink > > > > > > committer! > > > > > > > > > > > > Congratulations Becket! > > > > > > > > > > > > Best, > > > > > > Robert (on behalf of the Flink PMC) > > > > > > > > > > > > > > > > > > > > -- Best Regards Jeff Zhang

Re: [DISCUSS] Publish the PyFlink into PyPI

gt; >>> > > > >>> As I know, PySpark takes Option 1) and Apache Beam takes > Option > > > 2). > > > >>> > > > >>> From the points of my view, I prefer option 2) as it's pretty > > > safer as > > > >>> it eliminate the risk of deleting old releases occasionally and at > > the > > > same > > > >>> time keeps the trace of who is operating. > > > >>> > > > >>> 2. How to handle Scala_2.11 and Scala_2.12 > > > >>> > > > >>> The PyFlink package bundles the jars in the package. As we know, > > there > > > are > > > >>> two versions of jars for each module: one for Scala 2.11 and the > > other > > > for > > > >>> Scala 2.12. So there will be two PyFlink packages theoretically. We > > > need to > > > >>> decide which one to publish to PyPI or both. If both packages will > be > > > >>> published to PyPI, we may need two projects, such as pyflink_211 > and > > > >>> pyflink_212 separately. Maybe more in the future such as > pyflink_213. > > > >>> > > > >>> (BTW, I think we should bring up a discussion for dorp > > Scala_2.11 > > > in > > > >>> Flink 1.10 release due to 2.13 is available in early June.) > > > >>> > > > >>> From the points of my view, for now, we can only release the > > > scala_2.11 > > > >>> version, due to scala_2.11 is our default version in Flink. > > > >>> > > > >>> 3. Legal problems of publishing to PyPI > > > >>> > > > >>> As @Chesnay Schepler pointed out in > > > FLINK-13011[3], > > > >>> publishing PyFlink to PyPI means that we will publish binaries to a > > > >>> distribution channel not owned by Apache. We need to figure out if > > > there > > > >>> are legal problems. From my point of view, there are no problems > as a > > > few > > > >>> Apache projects such as Spark, Beam, etc have already done it. > > Frankly > > > >>> speaking, I am not familiar with this problem, welcome any feedback > > on > > > this > > > >>> if somebody is more family with this. > > > >>> > > > >>> Great thanks to @ueqt for willing to dedicate PyPI's project name > > > `pyflink` > > > >>> to the Apache Flink community!!! > > > >>> Great thanks to @Dian for the offline effort!!! > > > >>> > > > >>> Best, > > > >>> Jincheng > > > >>> > > > >>> [1] > > > >>> > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-38%3A+Python+Table+API > > > >>> [2] > > > >>> > > > > > > https://ci.apache.org/projects/flink/flink-docs-master/flinkDev/building.html#build-pyflink > > > >>> [3] https://issues.apache.org/jira/browse/FLINK-13011 > > > >>> > > > > > > > > > > > > > -- Best Regards Jeff Zhang

Re: [DISCUSS] Flink client api enhancement for downstream project

gt; > the previous emails, would be very helpful (IMHO). > > > > [1] https://issues.apache.org/jira/browse/FLINK-10864 > > [2] https://issues.apache.org/jira/browse/FLINK-10862 > > [3] https://issues.apache.org/jira/browse/FLINK-10879 > > > > Best, >

Re: [ANNOUNCE] Zhijiang Wang has been added as a committer to the Flink project

Congratulations Zhijiang! Robert Metzger 于2019年7月22日周一 下午10:12写道： > Hey all, > > We've added another committer to the Flink project: Zhijiang Wang. > > Congratulations Zhijiang! > > Best, > Robert > (on behalf of the Flink PMC) > -- Best Regards Jeff Zhang

Re: [DISCUSS] Expose or setup a secur...@flink.apache.org mailing list for security report and discussion

the security mailing > > list. > > > > Looking forward to your feedback! > > > > Regards, > > Dian > > > > [1] https://flink.apache.org/community.html > > [2] https://commons.apache.org/mail-lists.html > > [3] https://hadoop.apache.org/mailing_lists.html > > [4] https://spark.apache.org/community.html > > [5] https://kafka.apache.org/project-security.html > > [6] https://hive.apache.org/mailing_lists.html > -- Best Regards Jeff Zhang

Re: [ANNOUNCE] Jark Wu is now part of the Flink PMC

He has been writing many blogs about Flink, > > also driving the translation work of Flink website and documentation. He > is > > very active in China community as he gives talks about Flink at many > events > > in China. > > > > Congratulations & Welcome Jark! > > > > Best, > > Jincheng (on behalf of the Flink PMC) > > -- Best Regards Jeff Zhang

Re: [DISCUSS] Migrate build infrastructure from Travis CI to Azure Pipelines

> >>>> - I'm working on setting up the full matrix of builds (different > > >>>> scala, > > >>>> hadoop etc. versions) for the nightlies > > >>>> > > >>>> *Next Steps:* > > >>>> - I propose to document the entire build system in the Flink Wiki > > >>>> - Once Azure can cover the same pull request tests as Travis, I > > >>>> would set > > >>>> it up to run in parallel (including Flinkbot posting links to > > >>>> Azure). I > > >>>> hope that this phase lasts for 1-2 weeks only, so that we do not > > >>>> have to > > >>>> maintain things concurrently. I will monitor the build stability > > >>>> closely, > > >>>> but would expect some support with debugging potential issues from > the > > >>>> contributors. > > >>>> - Once there are no problems with the new setup, we remove the > Travis > > >>>> setup. > > >>>> - Independently, I will work on triggering builds from master / > > >>>> release - > > >>>> branch pushes, as well as cron builds from the master branch ... > > >>>> all this > > >>>> will be described in the Wiki. > > >>>> > > >>>> > > >>>> *Timeline:*- Once I have the feeling that people are supportive of > the > > >>>> idea, I will start documenting in the Wiki. The first pull requests > > >>> should > > >>>> show up after a few more days. > > >>>> I will do a one month parental leave starting some time later in > > >>> December, > > >>>> which will probably delay things a bit. I hope to have everything > > >>> finished > > >>>> by end of January. > > >>>> > > >>>> I'm happy to hear your thoughts on this work. > > >>>> If nobody objects, I will start documenting the system and prepare > > >>>> everything for the migration. > > >>>> > > >>>> Best, > > >>>> Robert > > >>>> > > >>>> > > >>>> > > >>>> [1] > > >>>> > > >>>> > > >>> > > > https://lists.apache.org/thread.html/b90aa518fcabce94f8e1de4132f46120fae613db6e95a2705f1bd1ea@%3Cdev.flink.apache.org%3E > > >>> > > >>>> [2] https://issues.apache.org/jira/browse/INFRA-17030 > > >>>> [3] https://github.com/rmetzger/flink/tree/azure_playground > > >>>> [4] > > >>> > https://dev.azure.com/rmetzger/Flink/_build?definitionId=4&_a=summary > > > > > > > > > > > > > > -- Best Regards Jeff Zhang

Re: [DISCUSS] have separate Flink distributions with built-in Hive dependencies

though we've supported almost all Hive versions [3] now. >> >> I want to hear what the community think about this, and how to achieve it >> if we believe that's the way to go. >> >> Cheers, >> Bowen >> >> [1] https://flink.apache.org/downloads.html >> [2] >> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/hive/#dependencies >> [3] >> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/hive/#supported-hive-versions >> > -- Best Regards Jeff Zhang

Re: [ANNOUNCE] Zhu Zhu becomes a Flink committer

gt; > > > > > > >> > > > > > > > > > >> I'm very happy to announce that Zhu Zhu accepted the > offer > > > of > > > > > the > > > > > > > > Flink > > > > > > > > > PMC > > > > > > > > > >> to become a committer of the Flink project. > > > > > > > > > >> > > > > > > > > > >> Zhu Zhu has been an active community member for more > than > > a > > > > year > > > > > > > now. > > > > > > > > > Zhu > > > > > > > > > >> Zhu played an essential role in the scheduler > refactoring, > > > > > helped > > > > > > > > > >> implementing fine grained recovery, drives FLIP-53 and > > fixed > > > > > > various > > > > > > > > > bugs > > > > > > > > > >> in the scheduler and runtime. Zhu Zhu also helped the > > > > community > > > > > by > > > > > > > > > >> reporting issues, answering user mails and being active > on > > > the > > > > > dev > > > > > > > > > mailing > > > > > > > > > >> list. > > > > > > > > > >> > > > > > > > > > >> Congratulations Zhu Zhu! > > > > > > > > > >> > > > > > > > > > >> Best, Till > > > > > > > > > >> (on behalf of the Flink PMC) > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Xuefu Zhang > > > > > > > > > > > > > > "In Honey We Trust!" > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- Best Regards Jeff Zhang

Re: [ANNOUNCE] Apache Flink 1.8.3 released

e notes are available in Jira: > https://issues.apache.org/jira/projects/FLINK/versions/12346112 > > We would like to thank all contributors of the Apache Flink community who > made this release possible! > Great thanks to @Jincheng as a mentor during this release. > > Regards, > Hequn > > > -- Best Regards Jeff Zhang

Re: [VOTE] FLIP-81: Executor-related new ConfigOptions

> >> The FLIP can be found in [1] and the discussion thread in [2]. > > > >> > > > >> As per the bylaws, the vote will stay open until Friday 26-10 (3 > days) > > > >> or until it gets 3 votes. > > > >> > > > >> Thank you, > > > >> Kostas > > > >> > > > >> [1] > > > >> > > > > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=133631524 > > > >> [2] > > > >> > > > > > > https://lists.apache.org/thread.html/a4dd8e0c7b79350c59f5afefc1bc583dac99abcf94caaa8c22017974@%3Cdev.flink.apache.org%3E > > > >> > > > > > > > > > -- Best Regards Jeff Zhang

Re: [DISCUSS] How to prepare the Python environment of PyFlink release in the current Flink release process

k-mailing-list-archive.1008284.n3.nabble.com/VOTE-Publish-the-PyFlink-into-PyPI-tt31201.html > [2] > https://issues.apache.org/jira/browse/FLINK-13011?focusedCommentId=16947307=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16947307 -- Best Regards Jeff Zhang

Re: [VOTE] FLIP-59: Enable execution configuration from Configuration object

splay/FLINK/FLIP-59%3A+Enable+execution+configuration+from+Configuration+object > > > >. > > > >> > > > >> The discussion thread can be found here < > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-59-Enable-execution-configuration-from-Configuration-object-td32359.html > > > >. > > > >> > > > >> This vote will be open for at least 72 hours and requires consensus > to > > > be accepted. > > > >> > > > >> Best, > > > >> Dawid > > > > > > > -- Best Regards Jeff Zhang

Re: [DISCUSS] Remove old WebUI

> (exclusive to the new UI), so I wanted to check what people think about > > > dropping the old UI for 1.10. > > > > > > -- Best Regards Jeff Zhang

Re: [DISCUSS] Releasing Flink 1.9.1

e release process once > blocker issues has been merged. What do you think? > > If there is any other blocker issues need to be fixed in 1.9.1, please let > me know. > > Cheers, > Jark > -- Best Regards Jeff Zhang

Re: [VOTE] FLIP-74: Flink JobClient API

> > > > > [1] > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-74%3A+Flink+JobClient+API > > > [2] > > > > > > https://lists.apache.org/x/thread.html/b2e22a45aeb94a8d06b50c4de078f7b23d9ff08b8226918a1a903768@%3Cdev.flink.apache.org%3E > > > > > > -- Best Regards Jeff Zhang

Re: [DISCUSS] Drop Python 2 support for 1.10

; [3] > > https://spark.apache.org/news/plan-for-dropping-python-2-support.html > > > <https://spark.apache.org/news/plan-for-dropping-python-2-support.html > > > > > [4] > > > > > > https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E > > > < > > > > > > https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E > > > > > > > [5] > > > > > > https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5 > > > < > > > > > > https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5 > > > > > > > -- Best Regards Jeff Zhang

Re: [VOTE] FLIP-78: Flink Python UDF Environment and Dependency Management

iki.apache.org/confluence/display/FLINK/FLIP-78:+Flink+Python+UDF+Environment+and+Dependency+Management > > >>> > > >> [2] > > >> > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Python-UDF-Environment-and-Dependency-Management-td33514.html > > >> < > > >> > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Python-UDF-Environment-and-Dependency-Management-td33514.html > > >>> > > >> > > >> > > >> > > > > > -- Best Regards Jeff Zhang

Re: Flink dev blog

gt; > scope and even be linked from release notes. > > > > > > > > > > > > > > > > Examples from our team to give a more specific idea: > > > > > > > > * Deprecated checkpoint lock and mailbox model > > > > > > > > * Revised interface for two phase commit sinks and new JDBC > > sink > > > > > > > > * N-ary input operators > > > > > > > > * Unaligned checkpoints > > > > > > > > * Operator factories > > > > > > > > * Plugins > > > > > > > > > > > > > > > > These articles would be less formal than a FLIP (but could > link > > > to > > > > > them > > > > > > > to > > > > > > > > avoid redundancy) and focus more on how other developers are > > > > actually > > > > > > > > impacted by the changes. It can also be used to share > > experiences > > > > > > during > > > > > > > > the implementation. > > > > > > > > > > > > > > > > We would like to ask the other teams if they see a benefit of > > > such > > > > a > > > > > > blog > > > > > > > > and would like to contribute. Bonus points if you could > provide > > > > > > examples > > > > > > > of > > > > > > > > your topics. > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > > > > Arvid > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Benchao Li > > > School of Electronics Engineering and Computer Science, Peking > University > > > Tel:+86-15650713730 > > > Email: libenc...@gmail.com; libenc...@pku.edu.cn > > > > > > -- Best Regards Jeff Zhang

Re: [VOTE] Release 1.10.0, release candidate #2

> https://linuxize.com/post/how-to-list-files-in-linux-using-the-ls-command/ > > > Best, > > Jingsong Lee > > On Thu, Feb 6, 2020 at 3:09 PM Jeff Zhang wrote: > > > -1, I just found one critical issue > > https://issues.apache.org/jira/browse/FLINK-1593

Re: [VOTE] Release 1.10.0, release candidate #2

ure/ReleaseNote.jspa?projectId=12315522=12345845 > > > [2] https://dist.apache.org/repos/dist/dev/flink/flink-1.10.0-rc2/ > > > [3] https://dist.apache.org/repos/dist/release/flink/KEYS > > > [4] > > https://repository.apache.org/content/repositories/orgapacheflink-1332 > > > [5] https://github.com/apache/flink/releases/tag/release-1.10.0-rc2 > > > [6] https://github.com/apache/flink-web/pull/302 > > > [7] https://github.com/apache/flink-web/pull/301 > > > > > > -- Best Regards Jeff Zhang

Re: [Discussion] Job generation / submission hooks & Atlas integration

a bit weird to have 2 hooks instead of one. > > It would be much nicer if we could somehow go back from JobGraph to > StreamGraph or at least have an easy way to access source/sink UDFS. > > What do you think? > > Cheers, > Gyula > -- Best Regards Jeff Zhang

Re: [DISCUSS] Include flink-ml-api and flink-ml-lib in opt

ink-ml-api` and `flink-ml-lib` in opt. > >> > >> BTW: I think would be great if bring up a discussion for upload > >> > >> multiple > >> > >> Jars at the same time. as PyFlink JOB also can have the benefit > >> > >> if > >> > >> we > >> > >> do > >> > >> that improvement. > >> > >> Best, > >> Jincheng > >> > >> > >> Hequn Cheng 于2020年1月8日周三 > 上午11:50写道： > >> > >> > >> Hi everyone, > >> > >> FLIP-39[1] rebuilds Flink ML pipeline on top of TableAPI > >> > >> which > >> > >> moves > >> > >> Flink > >> > >> ML a step further. Base on it, users can develop their ML > >> > >> jobs > >> > >> and > >> > >> more > >> > >> and > >> > >> more machine learning platforms are providing ML services. > >> > >> However, the problem now is the jars of flink-ml-api and > >> > >> flink-ml-lib > >> > >> are > >> > >> only exist on maven repo. Whenever users want to submit ML > >> > >> jobs, > >> > >> they > >> > >> can > >> > >> only depend on the ml modules and package a fat jar. This > >> > >> would be > >> > >> inconvenient especially for the machine learning platforms on > >> > >> which > >> > >> nearly > >> > >> all jobs depend on Flink ML modules and have to package a fat > >> > >> jar. > >> > >> Given this, it would be better to include jars of > >> > >> flink-ml-api > >> > >> and > >> > >> flink-ml-lib in the `opt` folder, so that users can directly > >> > >> use > >> > >> the > >> > >> jars > >> > >> with the binary release. For example, users can move the jars > >> > >> into > >> > >> the > >> > >> `lib` folder or use -j to upload the jars. (Currently, -j > >> > >> only > >> > >> support > >> > >> upload one jar. Supporting multi jars for -j can be discussed > >> > >> in > >> > >> another > >> > >> discussion.) > >> > >> Putting the jars in the `opt` folder instead of the `lib` > >> > >> folder > >> > >> is > >> > >> because > >> > >> currently, the ml jars are still optional for the Flink > >> > >> project by > >> > >> default. > >> > >> What do you think? Welcome any feedback! > >> > >> Best, > >> > >> Hequn > >> > >> [1] > >> > >> > >> > >> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs > >> > >> > >> > -- Best Regards Jeff Zhang

Re: [DISCUSS] Upload the Flink Python API 1.9.x to PyPI for user convenience.

; forward to your feedback! >>> >>> Best, >>> Jincheng >>> >>> [1] >>> >>> https://lists.apache.org/thread.html/4a4d23c449f26b66bc58c71cc1a5c6079c79b5049c6c6744224c5f46%40%3Cdev.flink.apache.org%3E >>> [2] >>> >>> https://lists.apache.org/thread.html/8273a5e8834b788d8ae552a5e177b69e04e96c0446bb90979444deee%40%3Cprivate.flink.apache.org%3E >>> [3] >>> >>> https://lists.apache.org/thread.html/ra27644a4e111476b6041e8969def4322f47d5e0aae8da3ef30cd2926%40%3Cdev.flink.apache.org%3E >>> >> -- Best Regards Jeff Zhang

Re: [DISCUSS] Release flink-shaded 10.0

t; > > > intended were bundled in the flink-shaded-hadoop-2-uber jar, which > was > > > > flagged by security checks. > > > > > > > > Are there any other changes that people are interested in doing? > > > > > > > > > > > > Regards, > > > > > > > > Chesnay > > > > > > > > > > > > > > -- Best Regards Jeff Zhang

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

tulations Jingsong! Well deserved. > > > > > > Best, > > > Jark > > > > > > On Fri, 21 Feb 2020 at 11:32, zoudan wrote: > > > > > >> Congratulations! Jingsong > > >> > > >> > > >> Best, > > >> Dan Zou > > >> > > > > > -- Best Regards Jeff Zhang

Re: [ANNOUNCE] Apache Flink 1.10.0 released

> of Apache Flink 1.10.0, which is the latest major release. > >>>>>>>> > >>>>>>>> Apache Flink® is an open-source stream processing framework for > distributed, high-performing, always-available, and accurate data streaming > applications. > >>>>>>>> > >>>>>>>> The release is available for download at: > >>>>>>>> https://flink.apache.org/downloads.html > >>>>>>>> > >>>>>>>> Please check out the release blog post for an overview of the > improvements for this new major release: > >>>>>>>> https://flink.apache.org/news/2020/02/11/release-1.10.0.html > >>>>>>>> > >>>>>>>> The full release notes are available in Jira: > >>>>>>>> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12345845 > >>>>>>>> > >>>>>>>> We would like to thank all contributors of the Apache Flink > community who made this release possible! > >>>>>>>> > >>>>>>>> Cheers, > >>>>>>>> Gary & Yu > >> > >> > > > > > > -- > > Best, Jingsong Lee > -- Best Regards Jeff Zhang

Re: [ANNOUNCE] Dian Fu becomes a Flink committer

me congratulating Dian for becoming a Flink committer ! > > Best, > Jincheng(on behalf of the Flink PMC) > -- Best Regards Jeff Zhang

Re: [DISCUSS] Set default planner for SQL Client to Blink planner in 1.10 release

efault planner for the whole Table API & SQL is another >> topic >> >> and is out of scope of this discussion. >> >> >> >> What do you think? >> >> >> >> Best, >> >> Jark >> >> >> >> [1]: >> >> >> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table >> >> [2]: >> >> >> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n >> >> [3]: >> >> >> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication >> >> [4]: >> >> >> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html >> >> [5]: >> >> >> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100 >> > >> >> > > -- > Best, Jingsong Lee > -- Best Regards Jeff Zhang

Re: [DISCUSS] Drop Bucketing Sink

t; > > > [1] https://issues.apache.org/jira/browse/FLINK-16227 > > [2] https://issues.apache.org/jira/browse/FLINK-13396 > > [3] https://issues.apache.org/jira/browse/FLINK-9749 > -- Best Regards Jeff Zhang

Re: [DISCUSS] Releasing "fat" and "slim" Flink distributions

nk-shaded-netty-tcnative-dynamic-2.0.25.Final-9.0.jar > - flink-sql-client_2.12-1.10.0.jar > - flink-state-processor-api_2.12-1.10.0.jar > - flink-swift-fs-hadoop-1.10.0.jar > > Current Flink dist is 267M. If we removed everything from opt we would > go down to 126M. I would reccomend this, because the large majority of > the files in opt are probably unused. > > What do you think? > > Best, > Aljoscha > > -- Best Regards Jeff Zhang

Re: [VOTE] FLIP-120: Support conversion between PyFlink Table and Pandas DataFrame

ttp://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-120-Support-conversion-between-PyFlink-Table-and-Pandas-DataFrame-tt39611.html > > > < > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-120-Support-conversion-between-PyFlink-Table-and-Pandas-DataFrame-tt39611.html > > > > > > > -- Best Regards Jeff Zhang

Re: [ANNOUNCE] New Apache Flink PMC Member - Hequn Chen

nk and Flink ML Pipeline. > Besides, > > > Hequn is also very active in the community since the beginning. > > > > > > Congratulations, Hequn! Looking forward to your future contributions. > > > > > > Thanks, > > > > > > Jiangjie (Becket) Qin > > > (On behalf of the Apache Flink PMC) > > > > > -- Best Regards Jeff Zhang

Re: [DISCUSS] FLIP-120: Support conversion between PyFlink Table and Pandas DataFrame

feedback! > > Regards, > Dian > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-120%3A+Support+conversion+between+PyFlink+Table+and+Pandas+DataFrame -- Best Regards Jeff Zhang

Re: [ANNOUNCE] New Committers and PMC member

- Konstantin Knauf joined as a committer. You may know him, > for > > > > > >> example, > > > > > >>> from the weekly community updates. > > > > > >>> > > > > > >>> - Dawid Wysakowicz joined the PMC. Dawid is one of the main > > > > developers > > > > > >> on > > > > > >>> the Table API. > > > > > >>> > > > > > >>> - Zhijiang Wang joined the PMC. Zhijiang is a veteran of > Flink's > > > > > >> network > > > > > >>> / data shuffle system. > > > > > >>> > > > > > >>> A warm welcome to your new roles in the Flink project! > > > > > >>> > > > > > >>> Best, > > > > > >>> Stephan > > > > > >>> > > > > > >> > > > > > >> > > > > > >> -- > > > > > >> Best, Jingsong Lee > > > > > >> > > > > > > > > > > > > > > > > > > > > -- Best Regards Jeff Zhang

Re: [DISCUSS] FLIP-120: Support conversion between PyFlink Table and Pandas DataFrame

this design. Does that make sense to > you? > > Thanks, > Dian > > [1] https://issues.apache.org/jira/browse/FLINK-14807 < > https://issues.apache.org/jira/browse/FLINK-14807> > > 在 2020年4月1日，上午11:14，Jeff Zhang 写道： > > > > Thanks Dian for driving this, definitel

[ANNOUNCE] Flink on Zeppelin (Zeppelin 0.9 is released)

m/RBHa2lTIg5 <https://t.co/sUapN40tvI?amp=1> 4) Advanced usage https://link.medium.com/CAekyoXIg5 <https://t.co/MXolULmafZ?amp=1> Welcome to use flink on zeppelin and give feedback and comments. -- Best Regards Jeff Zhang

Re: [DISCUSS] Introduce The Batch/Stream ExecutionEnvironment in Yarn mode

e demo as shown in the picture .the parameter named 'env' > containes all the operators about job ,like sources,maps,etc.. > > Thank you for your attention. > > > > -- Best Regards Jeff Zhang

Re: [VOTE] Guarantee that @PublicEvolving classes are API and binary compatible across bug fix releases (x.y.u and x.y.v)

a vote about it. > >>>>>> > >>>>>> The proposal is that the Flink community starts to guarantee > >>>>>> that @PublicEvolving classes will be API and binary compatible > >> across > >>>> bug > >>>>>> fix releases of the same minor version. This means that a version > >> x.y.u > >>>>> is > >>>>>> API and binary compatible to x.y.v with u <= v wrt all > >> @PublicEvolving > >>>>>> classes. > >>>>>> > >>>>>> The voting options are the following: > >>>>>> > >>>>>> * +1, Provide the above described guarantees > >>>>>> * -1, Do not provide the above described guarantees (please provide > >>>>>> specific comments) > >>>>>> > >>>>>> The vote will be open for at least 72 hours. It is adopted by > >> majority > >>>>>> approval with at least 3 PMC affirmative votes. > >>>>>> > >>>>>> [1] > >>>>>> > >>>>>> > >>>>> > >>>> > >> > https://lists.apache.org/thread.html/rb0d0f887b291a490ed3773352c90ddf5f11e3d882dc501e3b8cf0ed0%40%3Cdev.flink.apache.org%3E > >>>>>> > >>>>>> Cheers, > >>>>>> Till > >>>>>> > >>>>> > >>>>> > >>>> > >> > > -- Best Regards Jeff Zhang

Re: [DISCUSS] Rework History Server into Global Dashboard

terId (yarn app id for instance) that we > can use to enhance CLI experience in multi cluster (lot of per-job > clusters) environments. Please let us know what you think! Gyula > -- Best Regards Jeff Zhang

Re: [DISCUSS] Drop Scala 2.11

t;>> have both dropped 2.11 support in recent versions. In fact, Flink's > >>>>>> universal Kafka connector is stuck on 2.4 because that is the last > >>>>> version > >>>>>> with scala 2.11 support. > >>>>>> > >>>>>> What are people's thoughts on dropping Scala 2.11? How many are > >>>>>> still > >>>>> using > >>>>>> it in production? > >>>>>> > >>>>>> Seth > >>>>>> > >>>>> > >>>>> > >>>> > >>>> -- > >>>> Matthieu Bonneviot > >>>> Senior RD Engineer, DataDome > >>>> M +33 7 68 29 79 34 <+33+7+68+29+79+34> > >>>> E matthieu.bonnev...@datadome.co > >>>> W www.datadome.co > >>>> < > >>>> > http://www.datadome.co?utm_source=WiseStamp_medium=email_term=_content=_campaign=signature > >>>> > >>>>> > >>>> > >>>> [image: facebook] > >>>> < > >>>> > https://www.facebook.com/datadome/?utm_source=WiseStamp_medium=email_term=_content=_campaign=signature > >>>> > >>>>> > >>>> [image: > >>>> linkedin] > >>>> < > >>>> > https://fr.linkedin.com/company/datadome?utm_source=WiseStamp_medium=email_term=_content=_campaign=signature > >>>> > >>>>> > >>>> [image: > >>>> twitter] > >>>> < > >>>> > https://twitter.com/data_dome?utm_source=WiseStamp_medium=email_term=_content=_campaign=signature > >>>> > >>>>> > >>>> > >>> > >> > > > > > > -- Best Regards Jeff Zhang

Re: [ANNOUNCE] New Flink Committer: David Anderson

helping Flink users > to > > > > solve > > > > > >their problems (2nd in the all-time ranking, 1st in the last 30 > > > days). A > > > > > >similar level of activity can be found on the user@ mailing list. > > > > > > > > > > > >Please join me in congratulating David for becoming a Flink > committer! > > > > > > > > > > > >Best, > > > > > >Robert > > > > > > > > > > > > > > > > > > -- > > > > Arvid Heise | Senior Java Developer > > > > <https://www.ververica.com/> > > > > Follow us @VervericaData > > > > -- > > > > Join Flink Forward <https://flink-forward.org/> - The Apache Flink > > Conference > > > > Stream Processing | Event Driven | Real Time > > > > -- > > > > Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany > > > > -- > > Ververica GmbH > > Registered at Amtsgericht Charlottenburg: HRB 158244 B > > Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji > > (Toni) Cheng > -- Best Regards Jeff Zhang

  1   2   >