CDC 分表同步快照顺序

2022-03-03 Thread
Hi, all 我们在使用Flink CDC同步多张表,然后合并slink到一张es表中。但表之间有数据流转关系,比如有table_1, table_2, table2, 一条数据A之前table_1,但后续可能更新到table_2。 想请教下,如果使用正则表达式匹配同步分表,是否可以保证数据有序无误呢? 也就是全部分表同时快照,且等待所有分表快照同步完后才开始处理binlog。 从文档[1]中看到对于单表这种模式是可以保证的,但不确定多表且有数据流转是否也一样。 [1]

回复: Flink failure rate restart not work as expect

2022-03-01 Thread
only. If has any best practice about pipelined-region failover restart or document about region that would be helpfull. Again, thx for your time to reply, that help us a lot. 发件人: 刘 家锹 发送时间: 2022年3月1日 23:06 收件人: Matthias Pohl ; user ; David Morávek 主题: Re

回复: Flink FailureRateRestartStrategy策略异常

2022-03-01 Thread
发件人: 胡伟华 发送时间: 2022年3月1日 21:07 收件人: user-zh@flink.apache.org 主题: Re: Flink FailureRateRestartStrategy策略异常 1. 可以根据作业的 region 数量、单 TaskManager Slot 数量综合来考虑,我个人建议要容忍 1~2 台 TM 故障而不直接退出 2. 预期也是会超过重试次数退出的,可以从日志看看没有退出的原因 > 2022年3月1日 下午3:08,刘 家锹 写道: > > 是的,我们用的pipeline

Re: Flink failure rate restart not work as expect

2022-03-01 Thread
kef> 发件人: 刘 家锹 发送时间: Tuesday, March 1, 2022 10:23:50 PM 收件人: Matthias Pohl ; user ; David Morávek 主题: Re: Flink failure rate restart not work as expect We didn't find any obvious configuration issues in our cluster. As far as I know, It works fine in most cases;

Re: Flink failure rate restart not work as expect

2022-03-01 Thread
successfully. As you said, yarn logs look it may have some problems, we try digging into it to see if we can find any hints. 获取 Outlook for iOS<https://aka.ms/o0ukef> 发件人: Matthias Pohl 发送时间: Tuesday, March 1, 2022 9:50:36 PM 收件人: 刘 家锹 ; user ; David Morávek

Flink FailureRateRestartStrategy策略异常

2022-02-28 Thread
你好,伙伴们 我们最近碰到一个关于FailureRateRestartStrategy策略的问题,有点困惑。情况是这样子的: Flink版本:0.10.1 部署方式: on Yarn FailureRateRestartStrategy配置:failuresIntervalMS=6,backoffTimeMS=15000,maxFailuresPerInterval=4

Flink failure rate restart not work as expect

2022-02-28 Thread
Hi, all We encounter some problem with FailureRateRestartStrategy, which confuse us and don't know how to solove it. Here's the situation: Flink version: 1.10.1 Development env: on Yarn FailureRateRestartStrategy: failuresIntervalMS=6,backoffTimeMS=15000,maxFailuresPerInterval=4 One of our