??????Flink????DAG??????????????????????

[??????]

????????????????????????


????????kafka????????????????????????????????????????????????





------------------ ???????? ------------------
??????: ???? <lx_la...@163.com&gt;
????????: 2019??11??9?? 20:56
??????: user-zh <user-zh@flink.apache.org&gt;
????: ??????Flink????DAG??????????????????????



????????????????????case????????????????????????????????????????????
????1??????????source-&gt;????????-&gt;????????????tmp??
????2??????????tmp-&gt;??????????????a-&gt;??????
????3??????????tmp-&gt;????????????b-&gt;??????


??????????????????????????DAG????????????
1.val source = ds from KafkaConnector;
2.val broadcast = ruleMap.broacast(); 
3.val tmp = source.connect(broadcast).process();&nbsp; 
4.val a = tmp.map().addSink();&nbsp; 
5.val b = tmp.flatmap().keyBy( 8 fields ).window( 5min ).aggregate( sum 
).addSink(); 


??????????3????tmp????????????????????????????4??sink????????????????????????5??sink??????????????4??5????sink
 to 
mysql????????????????????????????5????????????job??????5??tps??????????200W/s??????case??????????????????????????????????????????source????????????????????????????????
??????????????????????????????????
??????????????????????????????5????????????????????????4??
??????????????????????????
1??????5????????????????????????flatmap??1????????????500??????????tps??:500*200W/s??keyBy??8????????aggr??sum??
2??????5??????????????job??????????1??????????????????????1??????????????????3??????????3????????????????????4??
3??????5??????????????sink????????mysql??mysql??????IO????????
??????????3????????????????????????????????5??sink??????kafka??????mq??????????mq??????mysql??


?????????? flink??????????????????????????????????????????????