On Tue, Jun 4, 2013 at 8:32 AM, Matt Franklin <[email protected]>wrote:
> On Mon, Jun 3, 2013 at 10:52 PM, Jason Letourneau > <[email protected]>wrote: > > > The current vision is that filters will be implemented agnostic to the > > overall processing architecture - there may be subscribers using lucene > dsl > > as part of the initial streams implemetation- but the interface won't > > dictate how a subscriber filters it's activities - > > > Maybe I am misunderstanding your statement. In my mind, we really need > inbound and outbound data pipelines. I don't think a simple outbound > filter can solve this easily. I don't see how the system can do > de-duplication, supersession, aggregation, etc during the outbound phase. > We will need to do a lot of processing before we hit an intermediate > persistence layer that can then be used by subscriber filters and query > endpoints. > > The pipeline components themselves should be pluggable and we just need a > series of workflow events that they can hook and do work against the > incoming data. > > Am I off base? > The rollup [1] reference is useful. There is an implied set use cases given the context for "views that support roll-up". Do we have a set of use cases documented (or that can be referenced) that would help drive where in the architecture the plug ins are needed? > > > > I don't know that we've > > figured out whether the subscriber delegate tells the aggregate its > filter > > via an interface or whether aggregator tells each subscriber about every > > activity and the subscriber filters - either way - you can implement > > filters however you want - provides they adhere to the common filter > > interface (which to my recollection is very simplistic) > > > > On Monday, June 3, 2013, Lavender, Beth A wrote: > > > > > Many of our current systems that will feed the integrated activity > stream > > > are noisy. For example, if I update a page 4 times in 5 minutes it > > > generates an activity for each one. I want to be able to set rule for > > > discard the last n activities if they have the same actor, verb, and > > object > > > in x time frame. > > > > > > This assumes a sub processor that detects the pattern and takes an > action > > > described in a rule. Where do rules and sub processors fit in this > > > architecture? Is anyone doing this in their existing systems? > > > > > > > > >
