??????????????????checkpoint????

Robert.Zhang Tue, 25 Aug 2020 20:43:24 -0700

Hi Congxian,


????????????????????????????iteration source??barrier??????????????????
??barrier??????????????????????????????????operator??????barrier??????checkpoint??????????????
??????????????????????????????????????????????????????????????????????????operator????????
??????????????????????????barrier??checkpoint??????????????







????????????????????????????????????????????????????????checkpoint????????









---????????---
??????: "Congxian Qiu"<[email protected]&gt;
????????: 2020??8??25??(????) ????5:33
??????: "user-zh"<[email protected]&gt;;
????: Re: ????????????checkpoint????


Hi
&nbsp;&nbsp; ???? checkpoint ?????????????????????????????????????????? source 
???????????????????????????????????????? snapshot
?? source???? CPU ??????????????????????????????????????????????????????source 
?????? JM ?? rpc ????????
snapshot???????????????????????????????????? barrier ????????????
Best,
Congxian


Robert.Zhang <[email protected]&gt; ??2020??8??25?????? ????12:58??????

&gt; ????????????????????checkpoint ????????????web?????? iteration 
source??checkpoint??????????????
&gt; ??????????????iterative
&gt; 
stream??checkpoint??????????????????????loop??????????????????????????checkpoint????????????????????????
&gt; 
??????????chandylamport????????????????operator??barrier????????????????????
&gt; ????????????????????barrier????????????????????????????????????
&gt;
&gt; ---????????---
&gt; ??????: "Congxian Qiu"<[email protected]&amp;gt;
&gt; ????????: 2020??8??24??(????) ????8:21
&gt; ??????: "user-zh"<[email protected]&amp;gt;;
&gt; ????: Re: ????????????checkpoint????
&gt;
&gt;
&gt; Hi
&gt; &amp;nbsp;&amp;nbsp; ?????? ??Exceeded checkpoint tolerable failure 
threshold?? ????????
&gt; checkpoint
&gt; ?????????????????????????????????????????????? checkpoint 
??????????????????[1] ??????????????
&gt; &amp;nbsp;&amp;nbsp; ?????????????????????? unalign 
checkpoint??????????????????????????????????????
&gt;
&gt; [1] https://zhuanlan.zhihu.com/p/87131964
&gt; Best,
&gt; Congxian
&gt;
&gt;
&gt; Robert.Zhang <[email protected]&amp;gt; ??2020??8??21?????? ????6:31??????
&gt;
&gt; &amp;gt; Hello all,
&gt; &amp;gt; ????????????????????iterative stream job
&gt; &amp;gt; 
????checkpoint??????????????????????????????????????checkpoint????????????
&gt; &amp;gt; ????state 
????????????k??????????????????????org.apache.flink.util.FlinkRuntimeException:
&gt; &amp;gt; Exceeded checkpoint tolerable failure threshold.??????
&gt; &amp;gt;
&gt; &amp;gt;
&gt; &amp;gt; ??????????
&gt; &amp;gt; env.enableCheckpointing(10000, CheckpointingMode.EXACTLY_ONCE, 
true);
&gt; &amp;gt; CheckpointConfig checkpointConfig = env.getCheckpointConfig();
&gt; &amp;gt; checkpointConfig.setCheckpointTimeout(600000);
&gt; &amp;gt; checkpointConfig.setMinPauseBetweenCheckpoints(60000);
&gt; &amp;gt; checkpointConfig.setMaxConcurrentCheckpoints(4);
&gt; &amp;gt;
&gt; &amp;gt;
&gt; 
checkpointConfig.enableExternalizedCheckpoints(CheckpointConfig.ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION);
&gt; &amp;gt; checkpointConfig.setPreferCheckpointForRecovery(true);
&gt; &amp;gt; checkpointConfig.setTolerableCheckpointFailureNumber(2);
&gt; &amp;gt; checkpointConfig.enableUnalignedCheckpoints();
&gt; &amp;gt;
&gt; &amp;gt;
&gt; &amp;gt; ??????????????????????????????????????????????????????????????

??????????????????checkpoint????

回复