@Guyla,
Thank you for your reply, the answer makes perfect sense. I have a follow up if
that’s ok.
IIUC this FLIP uses metrics that relate to backpressure at an operator level
(records in vs out, busy time etc…).
Could the FLIP also be used to auto-scale based on state-level metrics at an
Hong Liang Teoh created FLINK-29909:
---
Summary: Standardise connector package names
Key: FLINK-29909
URL: https://issues.apache.org/jira/browse/FLINK-29909
Project: Flink
Issue Type:
@JunRui:
There are 2 pieces that prevent scaling on minor load variations. Firstly
he algorithm / logic is intended to work on metrics averaged on a
configured time window (let's say last 5 minutes), this smoothens minor
variances and results in more stability. Secondly, in addition to the
Hello,
First of all thank you for tackling this theme, it is massive boon to Flink if
it gets in.
Following up on JunRui Lee’s question.
Have you considered making metrics collection getting triggered based on events
rather than periodic checks?
I.e if input source lag is increasing for the
Danny Cranmer created FLINK-29908:
-
Summary: Externalize and configure E2E tests
Key: FLINK-29908
URL: https://issues.apache.org/jira/browse/FLINK-29908
Project: Flink
Issue Type: Sub-task
Danny Cranmer created FLINK-29907:
-
Summary: Externalize AWS connectors from Flink core
Key: FLINK-29907
URL: https://issues.apache.org/jira/browse/FLINK-29907
Project: Flink
Issue Type:
Danny Cranmer created FLINK-29906:
-
Summary: Address tabs/spaces in checkstyle/spotless
Key: FLINK-29906
URL: https://issues.apache.org/jira/browse/FLINK-29906
Project: Flink
Issue Type:
Hi Max,
Thanks for writing this FLIP and initiating the discussion.
I just have a small question after reading the FLIP:
In the document, I didn't find the definition of when to trigger
autoScaling after some jobVertex reach the threshold. If I missed is,
please let me know.
IIUC, the proper
Danny Cranmer created FLINK-29905:
-
Summary: Migrate flink-connector-dynamodb-parent to
flink-connector-aws-parent
Key: FLINK-29905
URL: https://issues.apache.org/jira/browse/FLINK-29905
Project:
Hi, Michael
Please send email to commits-unsubscr...@flink.apache.org if you want to
unsubscribe the mail from comm...@flink.apache.org, you can refer[1] for more
details.
Best,
Leonard
[1]https://flink.apache.org/community.html#mailing-lists
> 2022年11月5日 下午10:04，Michael Stollery 写道：
>
>
unsubscribe
Best Regards,
Michael Stollery
On Sat, Nov 5, 2022 at 6:17 AM wrote:
> This is an automated email from the ASF dual-hosted git repository.
>
> leonard pushed a commit to branch master
> in repository https://gitbox.apache.org/repos/asf/flink.git
>
>
> The following commit(s) were
Hi Razin,
https://issues.apache.org/jira/browse/FLINK-29281 is the ticket to keep an
eye on regarding this topic.
Best regards,
Martijn
On Fri, Nov 4, 2022 at 11:11 PM Razin Bouzar
wrote:
> Hi all,
>
> Has there been any further work or discussions around this issue? The Netty
> security
Hey!
Thanks for the input!
The algorithm does not really differentiate between scaling up or down as
it’s concerned about finding the right parallelism to match the target
processing rate with just enough spare capacity.
Let me try to address your specific points:
1. The backlog growth rate
Hi Max,
Thanks a lot for the FLIP. It is an extremely attractive feature!
Just some follow up questions/thoughts after reading the FLIP:
In the doc, the discussion of the strategy of “scaling out” is thorough and
convincing to me but it seems that “scaling down” is less discussed. I have 2
Thanks for preparing the FLIP and kicking off the discussion, Max. Looking
forward to this. :-)
On Sat, Nov 5, 2022 at 9:27 AM Niels Basjes wrote:
> I'm really looking forward to seeing this in action.
>
> Niels
>
> On Fri, 4 Nov 2022, 19:37 Maximilian Michels, wrote:
>
>> Hi,
>>
>> I would
zhangjingcun created FLINK-29904:
Summary: Support “string1.notLike(string2)” built-in function in
Table API
Key: FLINK-29904
URL: https://issues.apache.org/jira/browse/FLINK-29904
Project: Flink
zhangjingcun created FLINK-29903:
Summary: Support “string1.notSimilar(string2)” built-in function
in Table API
Key: FLINK-29903
URL: https://issues.apache.org/jira/browse/FLINK-29903
Project: Flink
zhangjingcun created FLINK-29902:
Summary: Support “value1.notIn(TABLE)” built-in function in Table
API
Key: FLINK-29902
URL: https://issues.apache.org/jira/browse/FLINK-29902
Project: Flink
zhangjingcun created FLINK-29901:
Summary: Support “NOT IN” built-in function in Table API
Key: FLINK-29901
URL: https://issues.apache.org/jira/browse/FLINK-29901
Project: Flink
Issue Type:
Hong Liang Teoh created FLINK-29900:
---
Summary: Implement Table API for DynamoDB Sink
Key: FLINK-29900
URL: https://issues.apache.org/jira/browse/FLINK-29900
Project: Flink
Issue Type:
I'm really looking forward to seeing this in action.
Niels
On Fri, 4 Nov 2022, 19:37 Maximilian Michels, wrote:
> Hi,
>
> I would like to kick off the discussion on implementing autoscaling for
> Flink as part of the Flink Kubernetes operator. I've outlined an approach
> here which I find
Hi Jacky,
Apart from the continuous discussions, I think you can share current progress
or even make it public as one of the flink-packages [1]
[1] https://flink-packages.org/
Best
Yun Tang
From: Jacky Lau <281293...@qq.com.INVALID>
Sent: Friday, February 11,
Qingsheng Ren created FLINK-29899:
-
Summary: flink-runtime tests fail with exit code 137 returned from
Docker
Key: FLINK-29899
URL: https://issues.apache.org/jira/browse/FLINK-29899
Project: Flink
23 matches
Mail list logo
