FW: Why streaming over agg not supporting ranged upper boundary?

From: Renjie Liu Date: Tuesday, November 1, 2022 at 15:13 To: u...@flink.apache.org Subject: Why streaming over agg not supporting ranged upper boundary? Hi, flink community: Per the doc here https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/dev/table/sql/queries/over-agg

Re: [DISCUSS] Remove the slides under "Community & Project Info"

Hi, Stephan: Can we put project wiki in some place? I think it's a great place to find flip lists On Mon, Aug 27, 2018 at 9:56 AM vino yang wrote: > +1 > The reason is the same as Hequn, because we have given a link to > SlideShare under the "Flink Forward" section. > > Thanks, vino. > > Hequn

Re: Flink 1.7 Development Priorities

Hi: Can we include task manager isolation for jobs in 1.7.0? This is a feature that blocks us from deploying flink more widely in our compan? I've opened issues for that and want to help to contribute that, but lacks guidance and reviews from

Re: [Discussion] Move cluster creation functions from flink-cli.sh to other scripts

you mean to not support deploying Flink clusters via > `FLINK_HOME/bin/flink -m yarn-cluster`? I think starting a dedicated job > cluster for a given job that way is quite useful. > > Cheers, > Till > > On Thu, Aug 9, 2018 at 4:33 AM Renjie Liu wrote: > > > Hi: > >

Re: [DISCUSS] Rust add adapter for parquet

This thread is for arrow, I'm sorry for sending it to flink community by mistake. On Mon, Aug 20, 2018 at 3:57 PM Fabian Hueske wrote: > Hi Renjie, > > Did you intend to send this mail to dev@arrow.a.o instead of dev@flink.a.o > ? > > Best, Fabian > > 2018-08-20 4

Fwd: [DISCUSS] Rust add adapter for parquet

the developers of this project and the Arrow Rust implementation is quite large but still it may make sense to also start a discussion there. Uwe On Thu, Aug 16, 2018, at 9:14 AM, Renjie Liu wrote: > Hi, all: > > Now the rust component is approaching a stable state and rust reader for > parq

[Discussion] Move cluster creation functions from flink-cli.sh to other scripts

Hi: Flink 1.5.0 brings new deployment mode and we are able to provid a single cluster for our jobs which manages resources elastically. However, this also brings challenges for our cluster management since users can submit jobs or create yarn cluster in the same flink-cli.sh. Here is the

[jira] [Created] (FLINK-10101) Mesos web ui url is missing.

Renjie Liu created FLINK-10101: -- Summary: Mesos web ui url is missing. Key: FLINK-10101 URL: https://issues.apache.org/jira/browse/FLINK-10101 Project: Flink Issue Type: Bug

Re: [VOTE] Release 1.6.0, release candidate #1

This bug is also introduced in flink 1.5.0, and I'm working on a fix for it. Hope to include the fix in 1.6.0. https://issues.apache.org/jira/browse/FLINK-9936 On Thu, Jul 26, 2018 at 8:33 PM Piotr Nowojski wrote: > -1 > > There is one more bug. Flip6 in 1.5.0 seems like introduced regression

[jira] [Created] (FLINK-9936) Mesos resource manager unable to connect to master after failover

Renjie Liu created FLINK-9936: - Summary: Mesos resource manager unable to connect to master after failover Key: FLINK-9936 URL: https://issues.apache.org/jira/browse/FLINK-9936 Project: Flink

[jira] [Created] (FLINK-9727) Allow user to pass resource tag expression in command line

Renjie Liu created FLINK-9727: - Summary: Allow user to pass resource tag expression in command line Key: FLINK-9727 URL: https://issues.apache.org/jira/browse/FLINK-9727 Project: Flink Issue

[jira] [Created] (FLINK-9726) Enable job task manager isolation use tag support

Renjie Liu created FLINK-9726: - Summary: Enable job task manager isolation use tag support Key: FLINK-9726 URL: https://issues.apache.org/jira/browse/FLINK-9726 Project: Flink Issue Type: New

[jira] [Created] (FLINK-9725) Add tag support to standalone cluster manager.

Renjie Liu created FLINK-9725: - Summary: Add tag support to standalone cluster manager. Key: FLINK-9725 URL: https://issues.apache.org/jira/browse/FLINK-9725 Project: Flink Issue Type: New

[jira] [Created] (FLINK-9724) Add tag support to yarn resource manager

Renjie Liu created FLINK-9724: - Summary: Add tag support to yarn resource manager Key: FLINK-9724 URL: https://issues.apache.org/jira/browse/FLINK-9724 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-9723) Add tags support to mesos resource manager

Renjie Liu created FLINK-9723: - Summary: Add tags support to mesos resource manager Key: FLINK-9723 URL: https://issues.apache.org/jira/browse/FLINK-9723 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-9722) Add resource tag predicate into slot manager

Renjie Liu created FLINK-9722: - Summary: Add resource tag predicate into slot manager Key: FLINK-9722 URL: https://issues.apache.org/jira/browse/FLINK-9722 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-9721) Introduce ResourceTagExpression class

Renjie Liu created FLINK-9721: - Summary: Introduce ResourceTagExpression class Key: FLINK-9721 URL: https://issues.apache.org/jira/browse/FLINK-9721 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-9720) Introduce ResourceTag class for tag support in scheduling

Renjie Liu created FLINK-9720: - Summary: Introduce ResourceTag class for tag support in scheduling Key: FLINK-9720 URL: https://issues.apache.org/jira/browse/FLINK-9720 Project: Flink Issue Type

[jira] [Created] (FLINK-9719) Mesos should use hostname as default jobmanager.rpc.address

Renjie Liu created FLINK-9719: - Summary: Mesos should use hostname as default jobmanager.rpc.address Key: FLINK-9719 URL: https://issues.apache.org/jira/browse/FLINK-9719 Project: Flink Issue

[jira] [Created] (FLINK-9671) Add configuration to enable task manager isolation.

Renjie Liu created FLINK-9671: - Summary: Add configuration to enable task manager isolation. Key: FLINK-9671 URL: https://issues.apache.org/jira/browse/FLINK-9671 Project: Flink Issue Type: New

[jira] [Created] (FLINK-9670) Introduce slot manager factory

Renjie Liu created FLINK-9670: - Summary: Introduce slot manager factory Key: FLINK-9670 URL: https://issues.apache.org/jira/browse/FLINK-9670 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-9669) Introduce task manager assignment store

Renjie Liu created FLINK-9669: - Summary: Introduce task manager assignment store Key: FLINK-9669 URL: https://issues.apache.org/jira/browse/FLINK-9669 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-9663) Allow jobs to be scheduled to task managers with special tags

Renjie Liu created FLINK-9663: - Summary: Allow jobs to be scheduled to task managers with special tags Key: FLINK-9663 URL: https://issues.apache.org/jira/browse/FLINK-9663 Project: Flink Issue

[jira] [Created] (FLINK-9662) Task manager isolation for jobs

Renjie Liu created FLINK-9662: - Summary: Task manager isolation for jobs Key: FLINK-9662 URL: https://issues.apache.org/jira/browse/FLINK-9662 Project: Flink Issue Type: New Feature

Re: [DISCUSS] FLIP-6 Problems

ome point. > > It should be conceptually rather simple, the dispatcher creates per job a > new container launch context with the "JobManagerRunner" and starts that. > In practice, it is quite a bit of work still, with all the details of Yarn > to take care of. > > > &

Re: [DISCUSS] FLIP-6 Problems

; > Concerning the resource specification, the corresponding issue can be found > here [3]. > > [1] https://issues.apache.org/jira/browse/FLINK-8886 > [2] https://issues.apache.org/jira/browse/FLINK-9537 > [3] https://issues.apache.org/jira/browse/FLINK-5131 > > Cheers, > Til

Re: [DISCUSS] FLIP-6 Problems

> Best, > Stephan > > > > On Tue, Jun 5, 2018 at 4:20 PM, Renjie Liu > wrote: > > > Hi, Till: > > > > > >1. Does the community has any plan to add task manager isolation into > >the session mode? > >2. Is there any issues to tra

Re: [DISCUSS] FLIP-6 Problems

p to identify which TM is > running on which Mesos task? > > Cheers, > Till > > On Tue, Jun 5, 2018 at 5:13 AM Renjie Liu wrote: > > > -- Forwarded message - > > From: Renjie Liu > > Date: Tue, Jun 5, 2018 at 10:43 AM > > Subject: [DISCUSS]

Fwd: [DISCUSS] FLIP-6 Problems

-- Forwarded message - From: Renjie Liu Date: Tue, Jun 5, 2018 at 10:43 AM Subject: [DISCUSS] FLIP-6 Problems To: user Hi: We've deployed flink 1.5.0 and tested the new cluster manager, it's really great for flink to be elastic. However we've also found some problems

[jira] [Created] (FLINK-9117) Disable artifact download in mesos container environment.

Renjie Liu created FLINK-9117: - Summary: Disable artifact download in mesos container environment. Key: FLINK-9117 URL: https://issues.apache.org/jira/browse/FLINK-9117 Project: Flink Issue Type

Re: Flip 6 mesos support

ld upload your log to https://gist.github.com/ or > open a JIRA to which you attach the log. > > Cheers, > Till > > On Sun, Mar 25, 2018 at 9:04 AM, Renjie Liu <liurenjie2...@gmail.com> > wrote: > > > Hi: > > Could you please help to check whether there any mis

[jira] [Created] (FLINK-9096) flip 6 allocates different resources for different jobs

Renjie Liu created FLINK-9096: - Summary: flip 6 allocates different resources for different jobs Key: FLINK-9096 URL: https://issues.apache.org/jira/browse/FLINK-9096 Project: Flink Issue Type

[jira] [Created] (FLINK-9077) flip6 cluster framework doesn't work on mesos

Renjie Liu created FLINK-9077: - Summary: flip6 cluster framework doesn't work on mesos Key: FLINK-9077 URL: https://issues.apache.org/jira/browse/FLINK-9077 Project: Flink Issue Type: Bug

Re: Flip 6 mesos support

- Original message --------From: Renjie Liu < > liurenjie2...@gmail.com> Date: 3/23/18 1:38 AM (GMT-08:00) To: > dev@flink.apache.org Subject: Re: Flip 6 mesos support > Hi, Till:Attached is my log. > I'm also looking into this, could you please assign this bug to me? I'm

Re: Flip 6 mesos support

> should fix. > > Cheers, > Till > > On Fri, Mar 23, 2018 at 4:42 AM, Renjie Liu <liurenjie2...@gmail.com> > wrote: > > > Hi, Till: > > Has anybody succeeded to deploy flip 6 mode on mesos? > > > > I'm testing flip 6 using the master branch and

Re: Flip 6 mesos support

pecified by > `jobmanager.heap.mb`. > > Cheers, > Till > > On Wed, Mar 21, 2018 at 3:20 PM, Renjie Liu <liurenjie2...@gmail.com> > wrote: > > > Hi, Till: > > > > In fact, I want to ask the resources consume by job manager > > > > Till Rohrma

Re: Flip 6 mesos support

Wed, Mar 21, 2018 at 12:04 PM, Renjie Liu <liurenjie2...@gmail.com> > wrote: > > > So how many slots a job manager may consume? > > > > On Wed, Mar 21, 2018 at 6:50 PM Till Rohrmann <trohrm...@apache.org> > > wrote: > > > > > At the mom

Re: Flip 6 mesos support

Wed, Mar 21, 2018 at 11:33 AM, Renjie Liu <liurenjie2...@gmail.com> > wrote: > > > For example, we have 2 jobs. > > For job 1, I want to start job manger with 1 CPU and 100M memory. Job 1 > > need s10 slots, and I want to deploy these 10 slots in 2 task managers, > >

Re: Flip 6 mesos support

erent jobs exactly? > > Cheers, > Till > > On Wed, Mar 21, 2018 at 10:55 AM, Renjie Liu <liurenjie2...@gmail.com> > wrote: > > > Hi, Till: > > > > How to specify job manager and task manager resources for different jobs > in > > session mode? >

Re: Flip 6 mesos support

yi > > > > > > > > On Thu, Mar 15, 2018 at 5:44 AM, Till Rohrmann <trohrm...@apache.org > > > > > > wrote: > > > > > > > > > Hi Renjie, > > > > > > > > > > thanks for the pointer with the YARNSessionFI

Re: Flip 6 mesos support

> Best Regards, > Sihua Zhou > > > 发自网易邮箱大师 > > > On 03/15/2018 10:14，Renjie Liu<liurenjie2...@gmail.com> wrote： > Hi, Till: > In fact I'm asking how to deploy other components such as dispatcher, etc. > > Till Rohrmann <trohrm...@apache.org> 于 2018年

Re: Flip 6 mesos support

have to add `mode: old` to your > flink-conf.yaml. I'm really happy that you want to test it out :-) > > Cheers, > Till > > On Wed, Mar 14, 2018 at 3:03 PM, Renjie Liu <liurenjie2...@gmail.com> > wrote: > > > Hi Till: > > Is there any doc on deploying flink i

Re: Flip 6 mesos support

sosResourceManager for more details. As > dispatcher, we use the StandaloneDispatcher which is spawned by > the MesosSessionClusterEntrypoint. > > Cheers, > Till > > On Wed, Mar 14, 2018 at 9:32 AM, Renjie Liu <liurenjie2...@gmail.com> > wrote: > > > Hi all: >

Flip 6 mesos support

Hi all: I'm reading the source code and it seems that flip6 does not support mesos? According to the design, client send job graph to dispatcher and dispatcher spawn job mananger and resource manager for job execution. But I can't find dispatcher implementation for mesos. -- Liu, Renjie Software

[jira] [Created] (FLINK-8919) Add KeyedProcessFunctionWIthCleanupState

Renjie Liu created FLINK-8919: - Summary: Add KeyedProcessFunctionWIthCleanupState Key: FLINK-8919 URL: https://issues.apache.org/jira/browse/FLINK-8919 Project: Flink Issue Type: Improvement

Re: [DISCUSS] Add end-to-end tests using Docker Compose

+1 On Fri, 9 Mar 2018 at 5:47 PM Aljoscha Krettek wrote: > Hi All, > > Stephan pointed this out the other day to me, so here goes: as some of > you might now, there are end-to-end tests in flink-end-to-end tests that > run a proper Flink cluster (on the local machine) and

Re: StreamSQL queriable state

(besides the query to the state, of course). > > My2c. > > Best, > Stefano > > -Original Message- > From: Renjie Liu [mailto:liurenjie2...@gmail.com] > Sent: Monday, March 05, 2018 3:52 AM > To: dev@flink.apache.org > Subject: Re: StreamSQL queriabl

Re: StreamSQL queriable state

API in a follow-up PR. > > What do you think? > > Regards, > Timo > > > Am 3/5/18 um 3:52 AM schrieb Renjie Liu: > > Hi, Timo: > > I've read your QueryableStateTableSink implementation and that basically > > implementes what I want to do. I also want to ex

[jira] [Created] (FLINK-8858) SQL Client to submit long running query in file

Renjie Liu created FLINK-8858: - Summary: SQL Client to submit long running query in file Key: FLINK-8858 URL: https://issues.apache.org/jira/browse/FLINK-8858 Project: Flink Issue Type: New

Re: StreamSQL queriable state

operator implementations to get as a parameter the > StateDescriptor to be inquired. > > Looking forward for the design document, will be happy to give you > feedback. > > Best, > Stefano > > -Original Message- > From: Renjie Liu [mailto:liurenjie2...@gmail.com]

Re: StreamSQL queriable state

Great, thank you. I'll start by writing a design doc. On Fri, Mar 2, 2018 at 6:40 PM Timo Walther <twal...@apache.org> wrote: > I gave you contributor permissions in Jira. You should be able to assign > it to yourself now. > > Am 3/2/18 um 11:33 AM schrieb Renjie Liu: > >

Re: StreamSQL queriable state

; Regards, > Timo > > > Am 3/2/18 um 11:10 AM schrieb Renjie Liu: > > Hi, Timo, I've been planning on the same thing and would like to > contribute > > that. > > > > On Fri, Mar 2, 2018 at 6:05 PM Timo Walther <twal...@apache.org> wrote: > > >

Re: StreamSQL queriable state

Hi, Timo, I've been planning on the same thing and would like to contribute that. On Fri, Mar 2, 2018 at 6:05 PM Timo Walther wrote: > Hi Stefano, > > yes there are plan in this direction. Actually, I already worked on such > a QueryableStateTableSink [1] in the past but

Re: how to assign jira task to myself ?

+1, I want to implement some of the features and just don't know how to assign a task to me. On Sat, Dec 16, 2017 at 12:59 PM Lynch Lee wrote: > hi, team , > > could someone tell me how to assign a jira task to myself or assign me a > special permission ?? > > > Lynch

Re: [DISCUSS] Releasing Flink 1.4

Cool!!! On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann wrote: > Hi all, > > I want to revive the discussion about releasing Flink 1.4 [1] and the set > of features to include. > > The gist of the previous discussion was that we postponed the feature > freeze for 1.4 in

Re: FlinkML on slack

Hi, Stravros: Could you please all invite me to the FlinkML slack channel? On Sat, Jun 17, 2017 at 8:28 PM Andrew Psaltis wrote: > Thanks! > > On Sat, Jun 17, 2017 at 5:08 AM, Stavros Kontopoulos < > st.kontopou...@gmail.com> wrote: > > > Lokesh and Andrew just added

Re: [DISCUSS] Release 1.3.0 RC0 (Non voting, testing release candidate)

st one is related though. > > > > > > Thank you! > > > Gyula > > > > > > Till Rohrmann <trohrm...@apache.org> ezt írta (időpont: 2017. máj. > 11., > > Cs, > > > 9:14): > > > > > >> Hi Renjie, > > >&g

Re: [DISCUSS] Release 1.3.0 RC0 (Non voting, testing release candidate)

t; >> > > >> On 09.05.2017 10:17, Robert Metzger wrote: > > >>> Chesnay, I would propose to look into a CI solution for Windows. I'll > > try > > >>> out appveyor.com for my Flink fork and report back how well it works > > >> (let > >

Re: [DISCUSS] Release 1.3.0 RC0 (Non voting, testing release candidate)

ue, May 9, 2017 at 10:02 AM, Renjie Liu <liurenjie2...@gmail.com> > wrote: > >> Stephan: >> This means that we can use FLIP 6 for basic functionality now? Any >> document for setting up that? Will FLIP 6 be finished in release 1.3? >> >> On Tu

Re: [DISCUSS] Release 1.3.0 RC0 (Non voting, testing release candidate)

Stephan: This means that we can use FLIP 6 for basic functionality now? Any document for setting up that? Will FLIP 6 be finished in release 1.3? On Tue, May 9, 2017 at 3:43 PM Stephan Ewen <se...@apache.org> wrote: > @Renjie Liu: A good part of the FLIP-6 code is in there, bu

Re: [DISCUSS] Release 1.3.0 RC0 (Non voting, testing release candidate)

Hi, does this include the FLIP6? On Tue, May 9, 2017 at 2:29 AM Stephan Ewen wrote: > Did a quick test: Simply adding the > "org.apache.maven.plugins.shade.resource.ApacheNoticeResourceTransformer" > helps with NOTICE files, > but does not add the required BSD licence copies.

[jira] [Created] (FLINK-5445) flink kafka consumer should support rate control

Renjie Liu created FLINK-5445: - Summary: flink kafka consumer should support rate control Key: FLINK-5445 URL: https://issues.apache.org/jira/browse/FLINK-5445 Project: Flink Issue Type: Bug

Why checkpointing is synchronized with data processing?

Hi, flink dev team: I'm reading flink's source code and it seems that the process of saving checkpointing data to external storage system will block data processing, and I wonder why this has to be designed like this? Isn't is a potential risk that may increase latency of processing? Is the