Hi Zack, I'm also interested in hearing more about how folks generally handle the deployment process. In particular, versioning the topology in such a way that makes continuous integration possible without manual deployment steps.
This project (https://github.com/ptgoetz/storm-maven-plugin) seems like a step forward to integrating with Jenkins nicely, but it appears abandoned or perhaps replaced by something else? I'm not against using the Maven exec plugin to build something, but, I'd hate to do that when someone else has already built a solution. As far as handling the repo itself, I think it depends on how many spouts and bolts you're planning on re-using across different topologies. And, unless the upload time becomes burdensome, it doesn't seem that terrible to just maintain a single large super jar. If the jar size is an issue, you could break your project into Maven sub-modules within a single repo and have custom shaded builders for particular topologies to reduce the jar size. Regards, Tom On Mon, Jan 4, 2016 at 3:50 PM, Zack Allen <[email protected]> wrote: > Hi all, > > I would like to know the best practices in terms of the submission and > management of storm topologies. > > Do you manage 1 repo per topology, or have 1 repo with many topologies in > it? For example, I have a topology that counts words, a topology that sees > if an input is a prime number, and a topology that adds exclamation points > to a piece of text. > > What would be the best option in terms of updating the prime number > topology? I know you can remove one from the UI, but do you generally ship > a .jar with all of the topologies or do you have one strictly for that > piece of processing? I am leaning towards 1 repo for 1 topology to make > deployments more intuitive (push a change via github, assemble the .jar, > push to storm topology) but I would like to hear what everyone here does. > > Thanks! >
