topology.classpath is a topology level config, so Mike's potential alternative is to migrate that down from Ambari into the actual topologies we launch (assuming it works as expected).
Basically we're trading ease of use in Ambari vs. additional setup for each topology that's run. I'm in favor of pushing the config down to each topology (again, assuming it works), despite the additional topology work, because it seems more likely that users are going to see the issues from not restarting Storm and Metron immediately after install/start than they are from missing configs at the Flux level. The topology config just becomes a bullet point in instructions on creating a new topology, rather than a surprising and nonintuitive issue resulting from install. Once Metron installs and starts through Ambari, I would expect it to work immediately. On Tue, Apr 25, 2017 at 5:14 PM, Casey Stella <[email protected]> wrote: > I don't know that I'm offended by either approach, but the way we have it > now isn't offending me overly, honestly. > > Casey > > On Tue, Apr 25, 2017 at 5:11 PM, David Lyle <[email protected]> wrote: > > > I think you have one more restart than is strictly required but I get > your > > point. IIRC, when I did the original implementation there was a bit of a > > wrinkle getting the right configs on the right classpath so I punted in > > favor of topology.classpath. > > > > If you've got a more efficient way, I'm all for it. > > > > -D... > > > > > > On Tue, Apr 25, 2017 at 4:55 PM, Michael Miklavcic < > > [email protected]> wrote: > > > > > We currently define topology.classpath at the global level for Storm. > The > > > reason for this feature was to enable setting the classpath to include > > > things like hbase-site and core-site without modifying the topology jar > > > files on the cluster. That made things simpler, but this also has the > > > consequence that Metron's installation process via Ambari looks > something > > > like this: > > > > > > 1. Install Metron > > > 2. Start Metron - part of install process > > > 3. Restart Storm - bc of the new property that has been added that > > > needs to be distributed > > > 4. Restart Metron - the topologies won't get the new classpath > > otherwise > > > > > > We're in effect restarting Metron a couple times, which can take some > > time. > > > > > > A possible alternative here is to set the classpath on the individual > > > topologies. By doing so, we remove the need to restart Storm and Metron > > > during the install. The down side is that this requires us to duplicate > > > this setting for every flux file or topology, including newly-added > > > topologies. What do folks think about which would be better? > > > > > > Best, > > > Mike > > > > > >
