Re: Re: PartitionNotFoundException

2023-04-09 Thread Weihua Hu
Hi, 可以提供下 JobManager 和相关 TaskManager 的日志吗? 一般来说 PartitionNotFoundException 只在作业启动建立链接的时候才会出现, 根据你的描述,应该是一个消费 Kafka 的流式任务,不太应该在运行一周后 出现 PartitionNotFoundException 可以检查下是否存在其他异常 Best, Weihua On Mon, Apr 10, 2023 at 9:51 AM zhan...@eastcom-sw.com < zhan...@eastcom-sw.com> wrote: >

Re: Quick question about flink document.

2023-04-09 Thread Hang Ruan
Hi, Dongwoo, I think there is no problem in this part. This part describes snapshotting Operator State, which is checkpointing. The checkpoint will store by the JobManager and use the checkpoint storage. Best, Hang Feng Jin 于2023年4月10日周一 00:32写道: > Hi Dongwoo > > > This can be quite

Re: Quick question about flink document.

2023-04-09 Thread Shammon FY
Hi Dongwoo I think there are two configurations about state, one is state backend and the other is snapshot storage. Flink will create a snapshot for each state when the stateful operator collects all checkpoint barriers. As @Feng mentioned above, users can config different state backend with

Re: Re: PartitionNotFoundException

2023-04-09 Thread Shammon FY
像上面提到的,流式作业可以设置taskmanager.network.tcp-connection.enable-reuse-across-jobs: false,一般作业影响不会有影响 Best, Shammon FY On Mon, Apr 10, 2023 at 9:27 AM zhan...@eastcom-sw.com < zhan...@eastcom-sw.com> wrote: > hi, 上周调整这两参数后,正常运行了近一个星期后 又重现了[PartitionNotFoundException]... > >

退订

2023-04-09 Thread 柒朵
退订

Re: Quick question about flink document.

2023-04-09 Thread Feng Jin
Hi Dongwoo This can be quite confusing. Before Flink 1.13, Flink's statebackend was actually a hybrid concept that included three types of statebackends: *MemoryStateBackend*, *FsStateBackend*, and *RocksDBStateBackend*. The default *MemoryStateBackend* uses heap as the backend, and the state

Quick question about flink document.

2023-04-09 Thread Dongwoo Kim
Hi community, I’m new to flink and trying to learn about the concepts of flink to prepare migrating heron application to flink. I have a quick question about this flink document.