I also have an interest in such a use case. We pull database data and transform it into a star schema. Occasionally, we need to do a db:migrate on the source database, which requires building a new topology with updated tables, columns, etc. It would be nice to roll that in over the existing topology without interruption.
The plan now is to deactivate, kill, an existing topology, and then activate a new updated topology. On Fri, Aug 26, 2016 at 2:20 AM, Abhishek Agarwal <[email protected]> wrote: > Here is an interesting use case - To upgrade a topology without any > downtime. Let's say, the topology has only Kafka as a source and two > versions of it are running (different topology names of course) in parallel > and sharing the kafka input load. > > In old kafka spout, rolling upgrade is not possible, partition assignment is > derived from the number of tasks in the topology. > > In new kafka spout, partition assignment is done externally by Kafka server. > If I deploy two different topologies with same kafka consumer group id, is > it fair to assume that load will be automatically distributed across > topologies? Are there any corner cases to consider? > > -- > Regards, > Abhishek Agarwal > -- 是故勝兵先勝而後求戰，敗兵先戰而後求勝。
