The Apache Flink community is very happy to announce the release of Apache
Flink 1.14.6, which is the fifth bugfix release for the Apache Flink 1.14
series.
Apache Flink® is an open-source stream processing framework for
distributed, high-performing, always-available, and accurate data streaming
建议 dump 下 TM 内存看下具体内存使用情况
Best,
Lijie
lxk 于2022年9月28日周三 09:46写道：
> 最近Flink任务运行一段时间后就会自动停止。从JM和TM能看到的有效信息只有下面这段：
>
> 2022-09-24 07:18:16,303 INFO
> org.apache.flink.yarn.YarnTaskExecutorRunner [] - RECEIVED
> SIGNAL 15: SIGTERM. Shutting down as requested.
> 2022-09-24
其实可以和kafka的pull模型对比下，kafka消费是不断轮训pull。我的认知中flink应该不是吧？
flink应该仅仅是请求 result partition 的时候下游主动去上游请求？ 建立之后应该就是类似一条连接不断读取数据？
yanfei lei 于2022年9月22日周四 11:31写道：
>
> Hi,
> Flink社区有一篇关于Credit-based Flow Control的blog post
>
补充：flink1.15.2版本，standalone集群，基于zk的ha。
环境是公司自研容器环境。3个容器启JM+HistoryServer。剩下几百个容器都是TM。每个TM提供1个slot。
yidan zhao 于2022年9月27日周二 22:07写道：
>
> 此外，今天还做了个尝试，貌似和长时间没重启TM有关？重启后频率低很多会。
> 我预留的TM很多，比如500个TM，每个TM就提供1个slot，任务可能只用100个TM。
> 会不会剩下400的TM的连接，时间厂了就会出现某种问题？
>
> yidan zhao 于2022年9月27日周二 16:21写道：
> >
此外，今天还做了个尝试，貌似和长时间没重启TM有关？重启后频率低很多会。
我预留的TM很多，比如500个TM，每个TM就提供1个slot，任务可能只用100个TM。
会不会剩下400的TM的连接，时间厂了就会出现某种问题？
yidan zhao 于2022年9月27日周二 16:21写道：
>
> 打开了TM的debug日志后发现很多这种日志：
> Responding with error: class
> org.apache.flink.runtime.io.network.partition.PartitionNotFoundException
>
>
Hi Alfredo,
Did you consider implementing
org.apache.flink.streaming.api.checkpoint.CheckpointedFunction interface in
your broadcast function … the initializeState(…) function should give you
access to the state backend.
Kind regards
Thias
From: David Anderson
Sent: Tuesday, September 27,
yep that worked, thanks for the help
On Tue, Sep 27, 2022 at 3:05 AM yuxia wrote:
>
> You can change to "order by eventTIme". And it should work.
>
> You can sort on event time, but it must be time-ascending-order without
> 'limit'.
> If you still want to a descending order, I think you can try
I was thinking if I can use Flink to process large files and save result to
another file or database (jdbc).
So loading file into Flink table and then loop through rows inserting result to
some temp Flink table.
Then save result to jdbc table or file.
Sent: Thursday, September 22, 2022 at
Hi everyone,
I know the low level details of this are likely internal, but at a high
level we can say that operators usually have some state associated with
them. Particularly for error handling and job restarts, I imagine windows
must persist state, and operators in general probably persist
Logically it would make sense to be able to initialize BroadcastState in
the open method of a BroadcastProcessFunction, but in practice I don't
believe it can be done -- because the necessary Context isn't made
available.
Perhaps you could use the State Processor API to bootstrap some state into
打开了TM的debug日志后发现很多这种日志：
Responding with error: class
org.apache.flink.runtime.io.network.partition.PartitionNotFoundException
目前问题的直观表现是：提交任务后，一直报 LocalTransportException：
org.apache.flink.runtime.io.network.netty.exception.LocalTransportException:
Sending the partition request to
11 matches
Mail list logo
