If you run multiple instances of your app you may not be able to access the state store you are trying to access from the instance you are trying from, i.e., it may be on another instance. If streams is in the RUNNING state, this would seem to be the issue.
On Wed, 7 Mar 2018 at 15:56 detharon <detha...@gmail.com> wrote: > I'm afraid that's not what I'm looking for, as I'm just trying to retrieve > the local data, from inside my application (but from outside the stream > topology), and in some cases it becomes impossible. That is, the stream > changes its state from "rebalancing" to "running", but the store is remains > inaccessible. > > I don't want to have access to state stores located on other instances. > > In pseudocode: > streams.start(); <- here's where the stream starts > loopOverLocalData(streams) <- periodical, asynchronous calls to > streams.store, which, if the problem occurs, always result in exceptions > being thrown. I expected to retrieve the local data this way. > > On 7 March 2018 at 16:20, Damian Guy <damian....@gmail.com> wrote: > > > If you have multiple streams instances then the store might only be > > available on one of the instances. Using `KafkaStreams.store(..)` will > only > > locate stores that are currently accessible by that instance. If you need > > to be able to locate stores on other instances, then you should probably > > have a read of: > > https://kafka.apache.org/10/documentation/streams/ > > developer-guide/interactive-queries.html#querying-remote- > > state-stores-for-the-entire-app > > > > There is also a convenient 3rd party lib that can help you with this: > > https://github.com/lightbend/kafka-streams-query > > > > On Wed, 7 Mar 2018 at 14:07 detharon <detha...@gmail.com> wrote: > > > > > Hello, I'm experiencing issues accessing the state stores outside the > > Kafka > > > stream. My application queries the state store every n seconds using > the > > > .all() method to retrieve all key value pairs. I know that the state > > store > > > might not be available, so I guard against the > InvalidStateStoreException > > > and in case it's caught, I simply invoke the .store method on my > stream, > > in > > > order to get a new store. The problem is, that for some reason the > store > > > never becomes available. > > > > > > Some facts: > > > - During that time stream processing works correctly, and it > successfully > > > puts and gets data to and from the store. > > > - Stream is in "running" state. I've started logging that because under > > > normal circumstances this exception is being thrown when the stream is > in > > > "rebalancing" phase, but after a while it's gone, as expected, so I can > > > distinguish between these two cases. > > > - It only happens when I run my app in multiple instances. > > > > > > I've set the log level to debug, but I can't see anything suspicious in > > the > > > logs, but maybe there's something particular I should pay attention to? > > > I access the store from inside an Akka actor, to which I pass the > > reference > > > to KafkaStreams object, if that matters. > > > > > > I ran out of ideas what might have caused that behavior, so any help > will > > > be greatly appreciated. > > > > > >
