You can use the --artifacts or --jars flags to storm jar to include jars when you submit topologies. See https://storm.apache.org/releases/1.1.1/Command-line-client.html under the "jar" command. As I understand it this will upload the dependency jars to the supervisors for you.
2017-09-17 18:02 GMT+02:00 Roee Shenberg <[email protected]>: > Hi, > > We have a codebase that uses an external JAR dependency, and it seems > storm's "either bundle everything with the topology, or hard-code it on the > supervisor" attitude isn't good enough. > > We have two requirements that seem to be conflicting: > 1. Running multiple topologies with different versions of a dependency > 2. Using Java SecurityManager to enforce a policy > > These requirements conflict because requirement #1 implies we should use > an uberjar, and requirement #2 depends on our code being separated into > different JARs: the standard API provides us with a ProtectionDomain when > checking permissions, which has the JAR containing the given class as the > identifier for the code. > > (note: the java security permissions algorithm coalesces stack frames > belonging to the same ProtectionDomain so we can't actually see calls to > tainted classes when doing permissions checks when all classes are in the > same JAR) > > The two options I see are: > 1. externally provision our supervisors with all versions of the > dependency - this is a pain because part of storm's convenience is that it > deals with code provisioning for us. > 2. Use one-jar as the classloader (http://one-jar.sourceforge.net/) > > Am I missing something? Is there a better way to do this? > > Thanks, > Roee >
