Storm Topology deployment using pre deployed jar

[Phil Standen]

<http://stackoverflow.com/questions/43413549/storm-topology-deployment-using-pre-deployed-jar#>

<http://stackoverflow.com/questions/43413549/storm-topology-deployment-using-pre-deployed-jar#><http://stackoverflow.com/questions/43413549/storm-topology-deployment-using-pre-deployed-jar>http://stackoverflow.com/questions/43413549/storm-topology-deployment-using-pre-deployed-jar

[https://cdn.sstatic.net/Sites/stackoverflow/img/apple-touch-i...@2.png?v=73d79a89bded]<http://stackoverflow.com/questions/43413549/storm-topology-deployment-using-pre-deployed-jar>

Storm Topology deployment using pre deployed 
jar<http://stackoverflow.com/questions/43413549/storm-topology-deployment-using-pre-deployed-jar>
stackoverflow.com
We currently have a jar that contains 6 topologies. To deploy these topologies 
we currently do 6 separate calls using /bin/storm jar $LOCAL_JAR 
$TOPOLOGY_CLASS $TOPOLOGY_NAME $PS_ENV $ZK_QUORUM ...


We currently have a jar that contains 6 topologies. To deploy these topologies 
we currently do 6 separate calls using

/bin/storm jar $LOCAL_JAR  $TOPOLOGY_CLASS $TOPOLOGY_NAME $PS_ENV $ZK_QUORUM  
-c nimbus.host=$NIMBUS_HOST $STORM_CONFIG_ARGS


Looking at the log output, each time the topology is submitted the jar is also 
uploaded to nimbus i.e there are 6 lines like this

9937 [main] INFO  o.a.s.StormSubmitter - Successfully uploaded topology jar to 
assigned location:...


I want to avoid the multiple uploading of the jar. I have tried uploading the 
jar via scp and placing it in at "uploadedJarLocation" on the nimbus node ( I 
do this once). Then changing my deployment code to use the following for each 
of the topologies.

nimbusClient = NimbusClient.getConfiguredClient(storm_conf);
client = nimbusClient.getClient();
...
client.submitTopology(topologyName, uploadedJarLocation, jsonConf, 
topology.buildTopology());


This has sped things up and seems to work fine but I want to ask

Is this a safe approach, can I safely reference the uploadedJarLocation I 
pre-uploaded to nimbus via scp?

Are there any alternative methods to avoid the multiple jar upload?

I know about the StormSubmitter.submitJar as an alternative but have found this 
to be slow.