Hi ~,
1. ????????????Operator??uid/uidhash?????????????WebUI??????????????????URL?????????? 4. ??????????????????????????????????Kafka->HDFS????????????????HDFS??????????????????????????????????????????????????????????????min/max/distinct????????????????????state??????????checkpoint??????????????????GroupAggregator??state??????????????????????????????????????????????????????????????????????????????????Kafka????????????????HBase????Lookup????????????????????????SQL?????????? ?????????? ------------------ ???????? ------------------ ??????: "Jark Wu"<[email protected]>; ????????: 2019??10??17??(??????) ????11:05 ??????: "user-zh"<[email protected]>; ????: Re: Flink 1.9 SQL/TableAPI ????uid??State ???????? Hi?? 1. ?????? table ???????? table ???????????? Table API ?? SQL?? SQL ???????????? 3. ???????? HiveTableInputFormat ????????reachedEnd() ???????? false 4. ?????????????????????????????? ???????????????????????????? global min/max/distinct value?????????????????????????????????????????? value ???? Best, Jark > ?? 2019??10??17????10:43?????????? <[email protected]> ?????? > > Hi ~, > > > ?????????? > > > 1. ??????table api ??????uid/uidhash????sql???????? > ??????????????????: > 1. > env.addSource(kafkaSource).name('xxx').uid('myUid').map(...).toTable(...); > 2. tableEnv.register(table, ...); > 3. tableEnv.sqlQuery(mySQL); > > ????????Source??uid??????????????WebUI??????????????????????????????name?????????? > > > 2. ????????????????State Processor API?? > ??????????????????????????????uidhash???? > > > 3. ????????????????Hive Source > ??finish??????????????HiveCatalog??????????????????????reachedEnd????????????Hack, > ????????HiveTableInputFormat???? > > > 4. ??????????????????????????????stream sql, > ????????????????????????????????????????????????????????????????"????, > ????Only emit global min/max/distinct > value??????????retract???????????????????????????????????????????????? > > > ?????????? > > > ------------------ ???????? ------------------ > ??????: "Jark Wu"<[email protected]>; > ????????: 2019??10??16??(??????) ????4:04 > ??????: "user-zh"<[email protected]>; > > ????: Re: Flink 1.9 SQL/TableAPI ????uid??State ???????? > > > > Hi, > > > 1. table ?????? uid/uidhash ???????? > 2. ?????? Savepoint API ???? State Processor API ???????????????????????? > uid?? > 3. ???????????????????? hive source ??finish???????????????????? > checkpoint???? > > ???????????????????????????????????? static data???? > ???????????????????????????? streaming job ???????? long run > ???????????????????????????????? > > Best, > Jark > > > > On Wed, 16 Oct 2019 at 10:24, ???????? <[email protected]> wrote: > >> Hi ~, >> >> >> ??????Flink 1.9 >> SQL????????????????????????????????????Join??TopN??Distinct????????????????????????Operator??State???????????????????????????????????? >> 1. ????SQL??Table API??????????uid????uidhash??????????Kafka >> DataStreamSource??????uid????uidhash???????? >> 2. ????????Graph??????????SQL Job ??????GroupAggregator >> Operator????State??????????WebUI????????uidhash????SavePoint >> API??????????uid??????uidhash???????????????????????? >> 3. ??????????????????????????Union/Join????????Hive???? =?? ????????Finished >> Task >> ????????Job??????checkpoint?????????????????????????????????? >> >> >> ??????????
