Monica -- thanks for writing this! What you described with your use-case is basically the “Integrating filtering/processing services into publishers” model I described in my blog post – a new feed on the publisher for each filter applied to the original feed. In cases when the original feed being subscribed to is very frequently updated and subscribers aren’t interested in the whole feed but only small (and possibly overlapping) parts of it – this makes sense since a small number of duplicate notifications is easier to handle than a large number of unique notifications + their subsequent filtering. However, if the original feed isn’t frequently updated or the number of overlapping subscriptions is very large – this may be inefficient since a single update of the original feed could cause a large number of duplicate notifications. In that case, it may be more efficient to have a single feed publish all updates to the hub which subsequently filters them out. Overall, I think both models will co-exist since there will be a need for both.
Talking about filtering languages makes me think of Yahoo Pipes and wonder if something equivalent could be done with webhooks/PSHB. And I totally understand why you’d want to specify filters in the URL (either with parameters or with remote service URLs) -- the feed then equally supports both old polling-based and new push-based applications. This can also be achieved with stateful mapping of URLs on the publisher side, from URLs containing filter definitions to new opaque URLs, e.g. http://www.publisher.com/feed/include-foo-exclude-bar-language-eng/ -----> http://www.publisher.com/feed/1a2b3c4d. The subscriber would first obtain the new URL by invoking a publisher service, and the use the new URL for receiving updates from the feed, either by polling or subscribing to a hub. Not sure if my mediarss tag injection case is covered by activitystrea.ms, but will check it out. Ivan On Mon, Jan 11, 2010 at 22:08, Monica Keller <[email protected]> wrote: > Just wanted to point out what we are doing for the filtering case. We have a > lot of publishers and a lot of content so we use acitivtystrea.ms and then > we allow subscribers to specify filters using the activitystrea.ms concepts > (works on atom or rss) > > Filters include properties of the actor: location, gender, age, type of > user, culture > Properties of the activity: Verb, object type and Source > We also provide text matching and sampling rates > > Details here: > http://wiki.developer.myspace.com/index.php?title=Stream_Subscription_Query_Spec > > I believe there was interest in the activitystrea.ms community to define a > filtering/query language... as a separate spec of course > > My question here is whether you guys think that can be done via the topic > url. The nice thing about doing it via the topic url is that the > query/filtering spec also becomes useful when requesting a feed via GET > > Let me know this does not cover the "join" case as Ivan did with mediarss. > cliqset has a whatever feed to activitystrea.ms feed converter that could > maybe accomplish this normalization... > > On Mon, Jan 11, 2010 at 12:55 PM, Julien Genestoux > <[email protected]> wrote: >> >> I agree with Jeff that symetry doesn't necessarily mean that the >> publication (hub -> subscriber) and the ping (publisher -> hub) should be >> considered identical. As a matter of fact, the hub shouldn't trust the ping >> so it has to poll the feed anyway. Yet, there is nothing that prevents a hub >> from considering a publication like a ping (that's what superfeedr does >> actually). >> >> >> >> -- >> Julien Genestoux, >> >> http://twitter.com/julien51 >> http://superfeedr.com >> >> +1 (415) 830 6574 >> +33 (0)9 70 44 76 29 >> >> >> On Mon, Jan 11, 2010 at 9:41 PM, Jeff Lindsay <[email protected]> wrote: >>> >>> Yes. Although that symmetry might be functionally or high level in >>> practice than meaning the exact same interface/protocol. In my mind, it >>> means both use HTTP POST. ;) >>> >>> On Mon, Jan 11, 2010 at 12:36 PM, Alexis Richardson >>> <[email protected]> wrote: >>>> >>>> Symmetric: means that the model for a subscriber being updated by a >>>> publisher is the same as the model for a publisher updating a >>>> subscriber. >>>> >>>> >>>> On Mon, Jan 11, 2010 at 8:31 PM, Ivan Žužak <[email protected]> wrote: >>>> > Not sure what "symmetric" implies exactly, but if it means that >>>> > publisher, subscriber and hub define roles, not components, and that a >>>> > component may implement multiple roles -- then that's what I have in >>>> > mind also. A component may thus, for example, both subscribe and be >>>> > subscribed to. So, yeah, this sounds like an option to be specified. >>>> > >>>> > Cheers, >>>> > Ivan >>>> > >>>> > On Mon, Jan 11, 2010 at 20:45, Alexis Richardson >>>> > <[email protected]> wrote: >>>> >> Ivan >>>> >> >>>> >> Thanks! I am cc'ing Mike. >>>> >> >>>> >> I reckon that our contention is that being BOTH a (publishing) hub >>>> >> AND >>>> >> a subscriber requires treating the protocol as symmetric. >>>> >> >>>> >> This may require specifying, ideally as an option for PSHB. >>>> >> >>>> >> alexis >>>> >> >>>> >> >>>> >> >>>> >> On Mon, Jan 11, 2010 at 7:42 PM, Ivan Žužak <[email protected]> wrote: >>>> >>> Thanks Alexis! I responded to Mike on the blog. In short -- chaining >>>> >>> of hubs would not require changing the protocol, just the types of >>>> >>> components which implement parts of the protocol. Instead of having >>>> >>> just pure publishers, subscribers and hubs, there would be >>>> >>> components >>>> >>> that implement multiple roles (e.g. a hub that supports chaining >>>> >>> would >>>> >>> be both a hub and a subscriber). As Jeff said - this can all be >>>> >>> broken >>>> >>> down to webhooks. >>>> >>> >>>> >>> Regular PSHB subscription would still work as before. >>>> >>> Publishing/filtering would just be an extension which a hub MAY >>>> >>> support. Of course, this requires some kind of fallback negotiation >>>> >>> for cases when a component doesn't support an extension requested by >>>> >>> another component. >>>> >>> >>>> >>> Ivan >>>> >>> >>>> >>> On Mon, Jan 11, 2010 at 19:21, Alexis Richardson >>>> >>> <[email protected]> wrote: >>>> >>>> Ivan, all, >>>> >>>> >>>> >>>> Mike Bridgen has elaborated on this in the comments to the post. >>>> >>>> >>>> >>>> I am copying his comments here: >>>> >>>> >>>> >>>> --- >>>> >>>> >>>> >>>> pubsubhubbub (0.1, anyway) doesn’t chain together in the way you’ve >>>> >>>> illustrated, because it’s not symmetrical — hubs don’t get >>>> >>>> subscribed >>>> >>>> to other hubs (or indeed, subscribe themselves). While you wouldn’t >>>> >>>> have to change the protocol, you would have to change the idea of >>>> >>>> what >>>> >>>> a hub is. But I guess you are setting out to do that anyway. >>>> >>>> >>>> >>>> For processing I can subscribe the remote processing service to the >>>> >>>> hub, and subscribe myself to the remote processor. Taking into >>>> >>>> account >>>> >>>> the verification, it would probably go >>>> >>>> 1. Me -> Remote: Please give me a token for this hub to post to you >>>> >>>> 2. Me -> Remote: Please subscribe me to you >>>> >>>> 3. Me -> Hub: Please subscribe Remote using this token >>>> >>>> This requires me and the remote processing service to understand >>>> >>>> some >>>> >>>> generalised bits of PSHB, but nothing extra of the hub (I don’t >>>> >>>> think). >>>> >>>> >>>> >>>> --- >>>> >>>> >>>> >>>> Cheers, >>>> >>>> >>>> >>>> alexis >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Mon, Jan 11, 2010 at 5:21 PM, Alexis Richardson >>>> >>>> <[email protected]> wrote: >>>> >>>>> Ivan >>>> >>>>> >>>> >>>>> Possibly related to what Jeff says: how do you think hub-hub >>>> >>>>> chaining works? >>>> >>>>> >>>> >>>>> Separately does PSHB subscription still work in your model? >>>> >>>>> >>>> >>>>> Great article btw. >>>> >>>>> >>>> >>>>> alexis >>>> >>>>> >>>> >>>>> >>>> >>>>> On Mon, Jan 11, 2010 at 5:14 PM, Jeff Lindsay <[email protected]> >>>> >>>>> wrote: >>>> >>>>>> You should look into the greater webhooks ecosystem (slowly being >>>> >>>>>> called the >>>> >>>>>> Evented Web). It's all about the things your talking about here. >>>> >>>>>> http://webhooks.org >>>> >>>>>> Of particular interest might be Scriptlets (currently undergoing >>>> >>>>>> a major >>>> >>>>>> upgrade) and DrEval. >>>> >>>>>> -jeff >>>> >>>>>> >>>> >>>>>> On Mon, Jan 11, 2010 at 5:20 AM, Ivan Žužak <[email protected]> >>>> >>>>>> wrote: >>>> >>>>>>> >>>> >>>>>>> Hi all, >>>> >>>>>>> >>>> >>>>>>> Just wanted to point to my new blog post - http://bit.ly/5PMXGq. >>>> >>>>>>> In >>>> >>>>>>> short, it's about extending PSHB to support not only real-time >>>> >>>>>>> delivery of feeds but also their filtering and processing via >>>> >>>>>>> 3rd >>>> >>>>>>> party services. As I write in the post, I've discussed some of >>>> >>>>>>> these >>>> >>>>>>> ideas a few months back with Julien (over email) and Brett (over >>>> >>>>>>> FriendFeed) but never got around to starting a broader >>>> >>>>>>> discussion with >>>> >>>>>>> concrete ideas. >>>> >>>>>>> >>>> >>>>>>> Feedback is welcome and if it's mostly positive I think that >>>> >>>>>>> would be >>>> >>>>>>> a good signal to start defining an extension to the protocol >>>> >>>>>>> which >>>> >>>>>>> supports this. >>>> >>>>>>> >>>> >>>>>>> Thanks, >>>> >>>>>>> Ivan >>>> >>>>>> >>>> >>>>>> >>>> >>>>>> >>>> >>>>>> -- >>>> >>>>>> Jeff Lindsay >>>> >>>>>> http://webhooks.org -- Make the web more programmable >>>> >>>>>> http://shdh.org -- A party for hackers and thinkers >>>> >>>>>> http://tigdb.com -- Discover indie games >>>> >>>>>> http://progrium.com -- More interesting things >>>> >>>>>> >>>> >>>>> >>>> >>>> >>>> >>> >>>> >> >>>> > >>> >>> >>> >>> -- >>> Jeff Lindsay >>> http://webhooks.org -- Make the web more programmable >>> http://shdh.org -- A party for hackers and thinkers >>> http://tigdb.com -- Discover indie games >>> http://progrium.com -- More interesting things >> > >
