?????? flink 1.11 interval join??????rocksdb????????????

Tue, 22 Dec 2020 03:00:41 -0800 

??????????slot????rocksdb 
block_cache_usage??????????????????????????usage????????????????block_cache_capacity??????????????????????????????????????????



????????????????mem table size??????????


------------------ ???????? ------------------
??????:                                                                         
                                               "user-zh"                        
                                                            
<[email protected]&gt;;
????????:&nbsp;2020??12??19??(??????) ????12:50
??????:&nbsp;"user-zh"<[email protected]&gt;;

????:&nbsp;Re: flink 1.11 interval join??????rocksdb????????????



????????????&nbsp; promethus??metrics,&nbsp;&nbsp; 
rocksdb_block_cache_usage????????????????
????????????????rocksdb ???????? ????????????????????

????????&nbsp; ?? ????????&nbsp; 3?? ????????




867127831 <[email protected]&gt; ??2020??12??18?????? ????3:15??????

&gt; Hi,
&gt;
&gt;
&gt; ????flink 1.11 on 
k8s????????????????join??sql??????rocksdb????backend??flink
&gt; 
managed????????????flink??????state.backend.rocksdb.memory.managed=true????????????k8s??pod????????????????????????????????????
&gt;
&gt;
&gt; flink sql:
&gt;
&gt;
&gt; insert into console_sink
&gt; select t1.*, t2.*
&gt; from t1 left join t2
&gt; on t1.unique_id = t2.unique_id
&gt; and t1.event_time BETWEEN t2.event_time - INTERVAL '1' HOUR AND
&gt; t2.event_time + INTERVAL '1' HOUR
&gt;
&gt;
&gt;
&gt; ??????????
&gt; state.backend=rocksdb;
&gt; state.backend.incremental=false;
&gt; state.backend.rocksdb.memory.managed=true
&gt; state.idle.retention.mintime='10 min';
&gt; state.idle.retention.maxtime='20 min';
&gt; checkpoint.time.interval='15 min';
&gt; source.idle-timeout='60000 ms';
&gt;
&gt; taskmanager.memory.flink.size =55 gb
&gt; taskmanager.memory.managed.fraction=0.85
&gt;
&gt;
&gt;
&gt;
&gt;
&gt;
&gt; ??????????
&gt; 1. checkpoint??size??????200G??????????state????????????????
&gt; 2. k8s 
pod????????????????????????????????????????????????pod????????????????pod??????????????pod????????
&gt; 3. ????????promethus??metrics,
&gt; 
????rocksdb_block_cache_usage????????????????????????rocksdb_block_cache_capacity????????????rocksdb_block_cache_usage????????????????flink
&gt; managed????????????????
&gt;
&gt;
&gt; 
????????????????flink??????rocksdb??????????????????????rocksdb_block_cache_usage????????????????????????????

回复