Hi William No iterations isn’t the solution as you can (will) end up in a deadlock. We concluded that storing the results from external lookup in Kafka and use these data as input to the cache was the only way
Med venlig hilsen / Best regards Lasse Nedergaard > Den 5. feb. 2019 kl. 10.22 skrev William Saar <will...@saar.se>: > > Thanks! Looks like iterations is indeed the way to go for now then... > > > > ----- Original Message ----- > From: "Lasse Nedergaard" <lassenederga...@gmail.com> > To:"Fabian Hueske" <fhue...@gmail.com> > Cc:"William Saar" <will...@saar.se>, "user" <user@flink.apache.org> > Sent:Mon, 4 Feb 2019 20:20:30 +0100 > Subject:Re: How to add caching to async function? > > > Hi William > > We have created a solution that do it. Please take a look at my presentation > from Flink forward. > https://www.slideshare.net/mobile/FlinkForward/flink-forward-berlin-2018-lasse-nedergaard-our-successful-journey-with-flink > Hopefully you can get inspired. > > Med venlig hilsen / Best regards > Lasse Nedergaard > > > Den 4. feb. 2019 kl. 19.04 skrev Fabian Hueske <fhue...@gmail.com>: > > Hi William, > > Does the cache need to be fault tolerant? > If not you could use a regular in-memory map as cache (+some LRU cleaning). > Or do you expect the cache to group too large for the memory? > > Best, Fabian > > >> Am Mo., 4. Feb. 2019 um 18:00 Uhr schrieb William Saar <will...@saar.se>: >> Hi, >> I am trying to implement an async function that looks up a value in a cache >> or, if the value doesn't exist in the cache, queries a web service, but I'm >> having trouble creating the cache. I've tried to create a RichAsyncFunction >> and add a map state as cache, but I'm getting: State is not supported in >> rich async functions. >> >> What is the best practice for doing this? I guess I could have a previous >> step with state and send the responses from the rich function back as an >> iteration, but I would guess that's the wrong approach... >> >> Thanks, >> William >> >>
