So you would suggest flume over a custom akka-source from bahir? Jörn Franke <[email protected]> schrieb am So., 23. Apr. 2017 um 18:59 Uhr:
> I would use flume to import these sources to HDFS and then use flink or > Hadoop or whatever to process them. While it is possible to do it in flink, > you do not want that your processing fails because the web service is not > available etc. > Via flume which is suitable for this kind of tasks it is more controlled > and reliable. > > On 23. Apr 2017, at 18:02, Georg Heiler <[email protected]> wrote: > > New to flink I would like to do a small project to get a better feeling > for flink. I am thinking of getting some stats from several REST api (i.e. > Bitcoin course values from different exchanges) and comparing prices over > different exchanges in real time. > > Are there already some REST api sources for flink as a sample to get > started implementing a custom REST source? > > I was thinking about using https://github.com/timmolter/XChange to > connect to several exchanges. E.g. to make a single api call by hand would > look similar to > > val currencyPair = new CurrencyPair(Currency.XMR, Currency.BTC) > CertHelper.trustAllCerts() > val poloniex = > ExchangeFactory.INSTANCE.createExchange(classOf[PoloniexExchange].getName) > val dataService = poloniex.getMarketDataService > > generic(dataService) > raw(dataService.asInstanceOf[PoloniexMarketDataServiceRaw]) > System.out.println(dataService.getTicker(currencyPair)) > > How would be a proper way to make this available as a flink source? I have > seen > https://github.com/apache/flink/blob/master/flink-contrib/flink-connector-wikiedits/src/main/java/org/apache/flink/streaming/connectors/wikiedits/WikipediaEditsSource.java > but > new to flink am a bit unsure how to proceed. > > Regards, > Georg > >
