I'm all for getting things "right" but would this change affect getting 0.2.4 out the door? I think we should focus on finishing this release up.
Chris On Wed, Nov 12, 2014 at 9:23 AM, Aaron McCurry <[email protected]> wrote: > No it would an accumulator not a combiner/reducer. The command would have > to create an accumulator that would have to be able to accumulate results > as well as another accumulator instance. We could the accumulate other > instances optional, if implemented the accumulator could run on the shard > servers. If not implemented, all the accumulation would have to occur on > the controller. > > Aaron > > On Wed, Nov 12, 2014 at 9:18 AM, Garrett Barton <[email protected]> > wrote: > >> Would that be something similar to the Combiner paradigm in MR? >> >> On Wed, Nov 12, 2014 at 9:03 AM, Aaron McCurry <[email protected]> wrote: >> >> > On Tue, Nov 11, 2014 at 3:56 PM, Tim Williams <[email protected]> >> > wrote: >> > >> > > Thinking more about the current model, I'm a little concerned the >> > > current abstraction leaks an internal optimization. From the >> > > perspective of a command implementor the server combine is just an >> > > internal optimization? IOW, from a command perspective, it seems that >> > > we should be concerned about tables and shards and let >> > > 'server'-related things be plumbing underneath? >> > > >> > > We obviously want to well-define how exactly combine will be called >> > > (ie. for a given shard result, only ever once). I dunno, it smells a >> > > little funny right now, but then again, I know this method-smithing >> > > can be exhausting, just trying to get a feel for what others think? >> > > Anyone think it's worth going another round on this? >> > > >> > >> > I hear you on the abstraction/method sigs being a little weird for shard >> > server combines vs controller combines. I agree that I would like to let >> > the commands only have to deal with shard and table results. However I >> do >> > not want to remove an optimization path just for the sake of a cleaner >> api. >> > >> > I suppose we could change a bunch of the logic (again :-) ) and make the >> > combiner logic be an accumulator that can be serialized across shard and >> > controller servers. I haven't really thought this through but that may >> > allow us to make a single logic path for handling results. Thoughts? >> > >> > Aaron >> > >> > >> > > >> > > Thanks, >> > > --tim >> > > >> > >>
