You can set the Zookeeper configuration through the daemon conf map when you use a LocalCluster.Builder, which should let you set many of the options you might normally put in storm.yaml https://github.com/apache/storm/blob/master/storm-server/src/main/java/org/apache/storm/LocalCluster.java#L173.
I don't believe there is a way to set the data dir for Zookeeper right now. As you noted, it is set to use the system's temporary directory https://github.com/apache/storm/blob/master/storm-server/src/main/java/org/apache/storm/testing/InProcessZookeeper.java#L39. Consider raising an issue if you think we should add a function to set this (could e.g. be a new function on the LocalCluster.Builder). A PR would obviously be appreciated if you're up for it :) I'm not sure what you mean by the LocalCluster(host, port) constructor being undocumented. What kind of documentation do you feel is missing? 2017-08-27 19:10 GMT+02:00 Alexandre Vermeerbergen <[email protected] >: > Hello, > > I would like to start topologies in LocalCluster mode for testing purposes > using my own settings for the Zookeeper server started by the LocalCluster > mode. > > For example, I would like Zookeeper to write in > e:\tmp\exclam\stormlocaldir and use port 123456. > > Storm documentation isn't quite verbose about how to set LocalCluster mode > options, unless I missed the appropriate page (in which please, please let > me know at which URL I can find how to pass options to LocalCluster mode). > > I tried to modify the main() method of ExclamationTopology classic Storm > sampe to fix Zookeeer properties, like this: > > public static void main(String[] args) throws Exception { > TopologyBuilder builder = new TopologyBuilder(); > > builder.setSpout("word", new TestWordSpout(), 10); > builder.setBolt("exclaim1", new ExclamationBolt(), > 3).shuffleGrouping("word"); > builder.setBolt("exclaim2", new ExclamationBolt(), > 2).shuffleGrouping("exclaim1"); > > Config conf = new Config(); > conf.setDebug(true); > > conf.put(Config.STORM_LOCAL_DIR, "E:\\tmp\\exclam\\ > stormlocaldir\\"); > conf.put(Config.STORM_ZOOKEEPER_ROOT, "E:\\tmp\\exclam\\ > stormlocaldir\\"); > conf.put(Config.STORM_ZOOKEEPER_PORT,12346); > LocalCluster cluster = new LocalCluster(); > cluster.submitTopology("test", conf, builder.createTopology()); > Utils.sleep(120000); > cluster.killTopology("test"); > cluster.shutdown(); > } > } > > But to my despair, all data is written into subdirectories > C:\Users\ave\AppData\Local\Temp\<some UUID> instead of to > E:\tmp\exclam\stormlocaldir\ subdirectories. > > Also, during the program's execution, Zookeeper listens on port 2000 > instead of listening on port 123456 (I checked this: it's not just a wrong > trace). > > I was able to partially by-pass my problem with data directory by setting > Java's system property which gives the temporary directory path, but this > is a very ugly by-pass: > > public static void main(String[] args) throws Exception { > TopologyBuilder builder = new TopologyBuilder(); > > builder.setSpout("word", new TestWordSpout(), 10); > builder.setBolt("exclaim1", new ExclamationBolt(), > 3).shuffleGrouping("word"); > builder.setBolt("exclaim2", new ExclamationBolt(), > 2).shuffleGrouping("exclaim1"); > > Config conf = new Config(); > conf.setDebug(true); > > System.setProperty("java.io.tmpdir", "E:\\tmp\\exclam\\ > stormlocaldir\\"); > conf.put(Config.STORM_ZOOKEEPER_PORT,12346); > LocalCluster cluster = new LocalCluster(); > cluster.submitTopology("test", conf, builder.createTopology()); > Utils.sleep(120000); > cluster.killTopology("test"); > cluster.shutdown(); > } > > But I found no by-pass to set Zookeeper's port. > > I am aware that another by-pass could be to use my own Zookeeper server > and then use the un-documented LocalCluster(host,port) constructor to start > a LocalCluster connect to this Zookeeper server, but frankly... isn't there > a simpler way to setup properties for the Zookeeper server started by > default constructor of LocalCluster ? > > Best regards, > Alexandre Vermeerbergen > > >
