Hi,

Let me explain my use case in brief:

Kakfa spout A picks up messages from Kakfa topic topic-A and creates
Queries via Esper in Storm bolt B.
This is done only once as soon as topology is deployed
Another Kafka spout C picks up realtime messages from Kafka topic -C which
will be processed by Esper engine in same bolt B.

The spout data from A and B are both partitioned by account numbers so that
the Esper engine in different worker processes gets same account numbers.

Now the problem:
In case of worker threads dying due to some issue or the supervisor node
getting kicked out of the cluster, we are observing that the bolt instance
may get assigned to new process/worker.
But the prepare method of the bolt is initializing Esper query
configuration. so every time the Esper query engine in that worker process
is initialized .Hence it loses the queries setup by one time messages from
Kafka spout A.

Any suggestions, how do we handle this?

-Manoj

Reply via email to