I haven't tried it myself, but I guess you would just need to sync the sub-tree for the topology that you are migrating from /transactional/<id> on the old cluster to the new cluster. /transactional is the default value of transactional.zookeeper.root unless you have modified.
Similar is the case for KafkaSpout (non-trident) where the data is stored in /<root>/<spout-id>/ On Tue, Aug 11, 2015 at 5:50 PM Binh Nguyen Van <[email protected]> wrote: > Hi, > > I am working on a project that is using Storm Trident for streaming > processing and I am > using opaque transaction trident Kafka spout to read data from Kafka, > process it and > save result to database. > > The system is running for a while and we have more than one topologies > running. Now > we decided to slowly move our applications to new data center, we will > have new Kafka, > ZooKeeper, Storm and database cluster on new data center while still > remain the old > Kafka, ZooKeeper, Storm and database cluster for non-migrated topologies > to use. > > I wonder if any of you have done this before and could you please share > your experience? > How do you migrate transactional state that Storm keeps in ZK? We can copy > snapshot > files and recover from there when we migrate the first topology but we > can't do it when > we migrate other topologies since at that time the two ZK cluster are > diverse. > > Thanks > -Binh >
