Re: Iterating over keys in state backend

Hi Kostas, In my use case I’m keeping track of the state of URLs during a web crawl. This represents both current state (“URL X should be crawled at time Y, and has an estimated value of Z), and is the source of URLs to be fed into the crawl infrastructure - it’s a floor wax and a dessert

Re: Iterating over keys in state backend

Hi Ken, So you have a queue where elements are sorted by timestamp and score, and when the time (event time I suppose) passes that of the timestamp of an element, you want to fetch the element and: if the score is too low you archive it if the score is OK you emit it. If I get it right,

Re: Iterating over keys in state backend

Hi Kostas, Thanks for responding. Details in-line below. > On Apr 27, 2017, at 1:19am, Kostas Kloudas > wrote: > > Hi Ken, > > Unfortunately, iterating over all keys is not currently supported. > > Do you have your own custom operator (because you mention “from

Re: Iterating over keys in state backend

Hi Ken, Unfortunately, iterating over all keys is not currently supported. Do you have your own custom operator (because you mention “from within the operator…”) or you have a process function (because you mention the “onTimer” method)? Also, could you describe your use case a bit more? You

Iterating over keys in state backend

Is there a way to iterate over all of the key/value entries in the state backend, from within the operator that’s making use of the same? E.g. I’ve got a ReducingState, and on a timed interval (inside of the onTimer method) I need to iterate over all KV state and emit the N “best” entries.