Re: How are multiple spouts and fields grouping planned out?

2016-04-25 Thread Navin Ipe
Aha! That's exactly what I needed to know. I feel the official documentation should also explain the concept the way you explained it. Without the explanation you gave, it is very difficult to understand anything. Thank you very very much Matthias! :-) On Mon, Apr 25, 2016 at 4:46 PM, Matthias J.

Re: How are multiple spouts and fields grouping planned out?

2016-04-25 Thread Matthias J. Sax
That is correct. The constructor is only called once to create a single instance that is handed to the TopologyBuilder. Internally, this instance is serialized and deserialized multiple times to create the executors. (Instantiating an object via deserialization does not call the constructor.) So S

Re: How are multiple spouts and fields grouping planned out?

2016-04-25 Thread Navin Ipe
Hi Matthias, Thanks for the image and explanation. I understood the parallelism and tasks, but even now when I run builder.setSpout(partialSpoutName, new MongoSpout(),5).setNumTasks(100); There's a line in MongoSpout's constructor (System.out.println("MongoSpout created");) which is getting printe

Re: How are multiple spouts and fields grouping planned out?

2016-04-25 Thread Matthias J. Sax
Hi Navin, If you really want a "forward connection pattern", ie, all data of a single spout goes to a single bolt, your idea with a loop should work. Of course, as you do actually deploy distinct graphs, ie, the single parts of the topology do not exchange data, you could also just deploy many top

Re: How are multiple spouts and fields grouping planned out?

2016-04-25 Thread Navin Ipe
Thank you Matthias for your time and patient explanation. I'm now clear about the Fields grouping (an answer on Stackoverflow had confused me ). The first question still stands, where I'm unable to understand when multi

Re: How are multiple spouts and fields grouping planned out?

2016-04-24 Thread Matthias J. Sax
Hi Navin, I could not follow your email completely. Let me clarify a couple of things to get started. If you still have question, just ask again. A) A IRichBolt interface defines a bolt, and not a tuple. Thus, > class SomeTuple extends IRichBolt { > private Integer someID; > public Integer

How are multiple spouts and fields grouping planned out?

2016-04-24 Thread Navin Ipe
To parallelize some code, I considered having this topology. The single [Spout] or [Bolt] represent multiple Spouts or Bolts. *[Spout]--emit--->[Bolt A]--emit--->[Bolt B]* If any of the bolts in Bolt A emit a Tuple of value 1, and it gets processed by a certain bolt in Bolt B, then it is imperati