?????? flink1.10.1/1.11.1 ????sql ????group ?? ???????? ?????? ????????????

Sun, 02 Aug 2020 23:19:05 -0700 

    
????????????????????????????????????????????1.11.0????????hdfs????checkpoint??checkpoint????3??????
?????????? ????day ?? id ????groupby 
????????????????????????????7????????????????????????????????????????????watermark??????????????
tConfig.setIdleStateRetentionTime(Time.minutes(1440), Time.minutes(1440+10))




------------------ ???????? ------------------
??????:                                                                         
                                               "user-zh"                        
                                                            
<384939...@qq.com&gt;;
????????:&nbsp;2020??8??3??(??????) ????1:50
??????:&nbsp;"user-zh"<user-zh@flink.apache.org&gt;;

????:&nbsp;Re: flink1.10.1/1.11.1 ????sql ????group ?? ???????? ?????? 
????????????



hi????????
??????????????????????????????????
1????????????????3000????????????????????????????????????????????????
2??checkpoint??interval????????5??
3??????????????????????????????
4??????????????1??????on-yarn????

????????????????????
<http://apache-flink.147419.n8.nabble.com/file/t793/6.png&gt; 

18??????????????
<http://apache-flink.147419.n8.nabble.com/file/t793/9.png&gt; 

checkpoints??????
<http://apache-flink.147419.n8.nabble.com/file/t793/conf.png&gt; 

hdfs??????????
<http://apache-flink.147419.n8.nabble.com/file/t793/hdfs.png&gt; 

??????????????????
<http://apache-flink.147419.n8.nabble.com/file/t793/checkpoinsts1.png&gt; 


Congxian Qiu wrote
&gt; Hi&nbsp;&nbsp; ??????
&gt;&nbsp;&nbsp;&nbsp;&nbsp; ???????????????? checkpoint ???????????????????? 
checkpoint size ??????????????????????????????????????
&gt; checkpoint ???????????? hdfs ?? ls ???? checkpoint ????????????
&gt;&nbsp;&nbsp;&nbsp;&nbsp; 
??????????????????????????????????????????????????????state 
??????????????????????????
&gt; 
&gt; Best,
&gt; Congxian
&gt; 
&gt; 
&gt; ?????? <

&gt; 384939718@

&gt;&gt; ??2020??7??30?????? ????10:43??????
&gt; 
&gt;&gt; ??????
&gt;&gt;
&gt;&gt; flink1.11.1??????????????????????????backend????????????20??????????
&gt;&gt; ??????????????????????????????????????????????????????????????
&gt;&gt; StateBackend backend =new
&gt;&gt;
&gt;&gt; 
RocksDBStateBackend("hdfs:///checkpoints-data/"+yamlReader.getValueByKey("jobName").toString()+"/",false);
&gt;&gt; StateBackend backend =new
&gt;&gt;
&gt;&gt; 
FsStateBackend("hdfs:///checkpoints-data/"+yamlReader.getValueByKey("jobName").toString()+"/",false);
&gt;&gt;
&gt;&gt;
&gt;&gt; ????????????????RocksDBStateBackend??????????????????????????????
&gt;&gt; RocksDBStateBackend??
&gt;&gt; 
&amp;lt;http://apache-flink.147419.n8.nabble.com/file/t793/444.png&amp;gt;
&gt;&gt; FsStateBackend??
&gt;&gt; 
&amp;lt;http://apache-flink.147419.n8.nabble.com/file/t793/555.png&amp;gt;
&gt;&gt;
&gt;&gt;
&gt;&gt;
&gt;&gt;
&gt;&gt; --
&gt;&gt; Sent from: http://apache-flink.147419.n8.nabble.com/
&gt;&gt;





--
Sent from: http://apache-flink.147419.n8.nabble.com/

回复