Hello,
I just started working with Apache storm and your website has been a great a
help.In my current task I am trying to read N number of sub-directory given by
the user. Then N number of spout and bolt will do the action and the stream
will be save into a different bolt.
I have attached the topology structure and a simple java code. The
'something.java' file contains details on the error..
Model:
Folder1 -> SPOUT_1 -> BOLT_1 }Folder2 -> SPOUT_2 -> BOLT_2 }.
} --> AGR_BOLT -> OUTPUT_BOLT.
}FolderN -> SPOUT_N -> BOLT_N
}
Example code:public static void main(String[] args) throws Exception {
TopologyBuilder builder = new TopologyBuilder(); //Get the number of sub-folder
in given directory and create that many spout. for (int i = 0; i <
directoryInarr.length; i++){ builder.setSpout("spout"+Integer.toString(i),
new readFile(directoryInarr[i]), 1);
builder.setBolt(boltName+(Integer.toString(i)), new doProcess(),
3).fieldsGrouping("spout"+Integer.toString(i), new Fields("segment-id")); }
//Now I need to aggregate the streams from bolt0, bolt1, ..... boltN to a
different bolt. //so the usual way should be BoltDeclarer agrBolt = null;
for (int i = 0; i < directoryInarr.length; i++) { //trying for different #n
directory. this cau agrBolt = builder.setBolt("mux-segment", new
aggreaterProcess(framerate), 1).fieldsGrouping(boltName+Integer.toString(i),
"stream_"+Integer.toString(i), new Fields("mux_id")); }//This for loop
method does not work because and the error report says "id mux-segment" already
declared!!! //So I thought doing like this agrBolt =
builder.setBolt("mux-segment", new aggreaterProcess(framerate),
1).fieldsGrouping(boltName+Integer.toString(0), "stream_"+Integer.toString(0),
new Fields("mux_id")) .fieldsGrouping(boltName+Integer.toString(1),
"stream_"+Integer.toString(1), new Fields("mux_id")) ..... and so on... ; //But
the problem with this is that I dont know how many subfolder will be in the
directory...! // I saw one comment from you which says STORM-0.10.0-BETA1
dealt with the "BOLT OR SPOUT ALREADY DECLARED" ERROR... HOWEVER 10.0.BETA1
THROWS ERROR ON MY WINDOWS //MACHINE.
}
Could you please help me with this problem??
Regards,Raisul Bhuiyan
