?????? Flink ???? RocksDB CPU ????

Tue, 01 Sep 2020 22:57:57 -0700 

Hi,???? Congxian ??????????


1?? Flink version 1.10 
2?? subtask ??????????????????get ?????????????????????????? subtask 
??????????????
?????????? subtask 
??????????????????????????????????????????????????????????????
3?? sst ????????????????????????????60M ?? sst ???????????? butask ?? 
blockcache ?????? 1g ??????


??????
4?????????????????????????????? NewIndexIterator 
???????????????????????????????????? NewIndexIterator ???????????????????????? 
NewIndexIterator ????????
NewIndexIterator ?????????????????????????????????? index ?? Cache 
??????????????????????????

??????????????????????????????????https://drive.google.com/file/d/1DG4eTbaaozG_4ZxPQc_svmoyEXCdKZqb/view?usp=sharing
5?? ???????? block cache ?????????????????????? 5 
???????????????????????????????????? 30 ????????????
??????????????????????
state.backend.rocksdb.memory.managed: true
state.backend.rocksdb.memory.write-buffer-ratio: 0.4
state.backend.rocksdb.memory.high-prio-pool-ratio: 0.5
taskmanager.memory.jvm-overhead.fraction: 0.05
state.backend.incremental: true
state.backend.rocksdb.block.cache-size: 3 gb
taskmanager.memory.managed.fraction: 0.5



6?? ???? RocksDB LOG ?????????? DEBUG?????? LOG 
??????https://drive.google.com/file/d/1JpHMjwTyW0ej-GtPbg6NLigohG7o3U4Y/view?usp=sharing


Thanks
fanrui


------------------ ???????? ------------------
??????:                                                                         
                                               "user-zh"                        
                                                            
<[email protected]&gt;;
????????:&nbsp;2020??9??2??(??????) ????1:38
??????:&nbsp;"user-zh"<[email protected]&gt;;

????:&nbsp;Re: Flink ???? RocksDB CPU ????



Hi
&nbsp;&nbsp;&nbsp; ????????????RocksDB#get ????????????????????contains&nbsp; 
?????? RocksDB ?? get ????
&nbsp;&nbsp;&nbsp; 1. ???????????????????? Flink??
&nbsp;&nbsp;&nbsp; 2. ???? subtask ???????????????????????????????????????? 
RocksDB ?? get ????????????????????????
&nbsp;&nbsp;&nbsp; 3. ?????????????????????? snappy ???????????????? IO 
???????????????????? load ????????????????????????????????
subtask ??????????????
Best,
Congxian


fanrui <[email protected]&gt; ??2020??9??1?????? ????9:14??????

&gt; ??????????
&gt; Flink ?????????? 1024?????????????????????????? subtask ?????????????????? 
subtask ??????
&gt; ?????? subtask ???????????????????????????????????????????????? 
CPU???????? MapState ?? contains ????????????????
&gt; CPU??
&gt;
&gt;
&gt;
&gt; --
&gt; Sent from: http://apache-flink.147419.n8.nabble.com/

回复