Hi -
In a separate thread I mentioned that my bolts stopped working when numWorkers
> 1.
After lots of painful testing, I have set up a simple case based on a modified
TestWordSpout and ExclamationBolt, and notice that when numworkers = 3, I get
the following behavior:
(Note that my spout is the same as TestWordSpout, except that it emits to
fields “rsid” and “word”, where rsid is random selection of “rsid1”, "rsid2”,
“rsid3”)
builder.setSpout("word", new TestRsidWordSpout(), 10);
builder.setBolt("exclaim1", new ExclamationBolt(), 3).fieldsGrouping("word",
new Fields("rsid"));
-> I get expected bolt execution, where each rsid is processed in a separate
worker, based on logging via ExclamationBolt
Now if I set the spout parallelism down to 1, things go off the rails:
builder.setSpout("word", new TestRsidWordSpout(), 1);
builder.setBolt("exclaim1", new ExclamationBolt(), 3).fieldsGrouping("word",
new Fields("rsid"));
-> I do NOT get expected bolt execution, ONLY those tuples routed to the same
worker where the spout executes are getting processed; the other tuples are
lost, based on logs (no evidence of executing bolts in this case)
I need to add some tracing to verify that ALL tuples are getting processed in
the first case (with spout parallelism =10), but at least all workers are able
to process tuples in that case.
If you have any ideas what is happening here, please let me know.
Thanks!
Tyson
