Thanks for your quick reply nathan. So I'm doing some debugging of my topology, and I've removed all the logic from my MultiPut function, replacing it with a single System.out.println() Then i am monitoring my logs to check when this gets printed out. It looks like every single one of my hosts (workers) hits this. Does this then indicate that I am processing many many partitions that each hit this multiPut and prints out? Thanks.
On Tue, Jun 3, 2014 at 3:29 PM, Nathan Marz <[email protected]> wrote: > When possible it will do as much aggregation Storm-side so as to minimize > amount it needs to interact with database. So if you do a persistent global > count, for example, it will compute the count for the batch (in parallel), > and then the task that finishes the global count will do a single > get/update/put to the database. > > > On Tue, Jun 3, 2014 at 3:08 PM, Raphael Hsieh <[email protected]> > wrote: > >> How does PersistentAggregate distribute the database calls across all the >> worked nodes ? >> Does it do the global aggregation then choose a single host to do a >> multiget/multiput to the external db ? >> >> Thanks >> -- >> Raphael Hsieh >> >> >> >> > > > > -- > Twitter: @nathanmarz > http://nathanmarz.com > -- Raphael Hsieh
